Skip to content

A/B validation of off-by-default shape-measurement options #813

Description

@cailmdaley

Several ngmix-module options ship off by default, pending real-data evidence — the design contract is that production defaults flip here, on data, never in the implementing PR. Each sub-issue is one A/B question with a named metric and a named decider.

Current options awaiting a verdict:

The decisive statistics (B-modes, leakage) need real area, which is why this lives in the First run on data (Nibi) milestone: the A/B runs are consumers of the P3 footprint (#801, #808) — pipeline run twice (or more) with config variants only, matched catalogues compared through the sp_validation machinery. Complementary evidence from image simulations (calibrated m/c truth) rides UNIONS-WL/MultiBand_ImSim#1.

Methodology (area needed, patch choice, matched-object vs matched-area comparison) needs group design — to discuss on a tomography call before the runs are scheduled.

— Claude, on behalf of Cail

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions