Record the pipeline's scientific decisions in astra.yaml

[cailmdaley]

Stacked on #852 (base is feat/snakemake-orchestration, since the record anchors into workflow/config/cfis/*). Documentation only — no code, no config, no behaviour change.

What

astra.yaml at the repo root records the scientific decisions embedded in ShapePipe's code and committed configs: 50 decisions across eight sub-analyses (masking, detection, preparation, star selection + PSF, shape measurement, PSF diagnostics, survey geometry, catalogue assembly), plus four cross-cutting ones. Each carries its rationale, the alternatives that were considered and why they were rejected, and an anchor back to the code or config that implements it. universes/committed.yaml pins the option this branch selects for every decision.

The format is ASTRA (uvx astra-tools@0.2.17 guide), applied at codebase level rather than to a single analysis. uvx astra-tools@0.2.17 validate passes on both files; all 146 anchors resolve.

Conventions live in the file's own header. In short: anchors are path::symbol or path#SECTION.KEY and never cite line numbers; [HARDCODED] marks a scientific value with no config exposure; [LINT] marks a place where the record and the code — or the code and itself — disagree; [PENDING #NNN] marks state not yet on develop.

Why it is worth reading

Writing it was not a documentation exercise. It surfaced:

• Nine lints — the code disagreeing with itself or its docs. Two are fixed by fix(psf): deterministic star train/validation split; align tile STAR_THRESH with the 80% convention #873 (the unseeded rand_split in setools, and the 22-vs-20 STAR_THRESH mismatch between PSF validation and science interpolation). The other seven stand, e.g. final_cat.param requesting IMAFLAGS_ISO the merged catalogue never receives; the documented-but-unimplemented TILE_LIST overlap flagging; additive rather than bitwise mask-plane combination.
• Nine [HARDCODED] values — scientific numbers with no config exposure, including PSFEx's compiled-in SAMPLE_AUTOSELECT defaults, which are a second star selection no config records.
• Ten drifts since publication — where Guinot+22 / Farrens+22 describe something the code no longer does (DETECT_MINAREA 10→5, DEBLEND_MINCONT 0.001→0.0005 on tiles, tile background AUTO→MANUAL 0, in-line spread-model classification disabled and deferred downstream, HSM moment initialisation→WCS centroids, ...). 16 decisions carry verbatim quotes from those papers as prior insights, machine-verified against the PDFs.

The standing instruction

CLAUDE.md gains a section stating that a scientific change is not finished until the record is — amended in the same PR. The membership test: would a different defensible choice change which objects enter the shear catalogue, or the numbers attached to them? Detection thresholds, masking geometry, star cuts, PSF degree, ngmix priors, flag semantics, completeness floors are in; manifest sentinels, chunk sizes, allocation strategy, directory layout are out.

Open for discussion

• Whether the record belongs at the repo root or under a subdirectory.
• The anchor grammar is prose today (a sentence in each rationale). A structured anchor field would be an upstream astra-spec change.
• Two sub-analyses (psf_diagnostics, survey_geometry) are thin: the committed Snakemake chain runs neither.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y2muA2sRojbxRNxU2SKQeP