cat_config: drop duplicate SP_v1.4.6 / SP_v1.3.6 keys, repair the survivor, guard against recurrence - #320
Merged
Merged
Conversation
…urvivor `cosmo_val/cat_config.yaml` carried two top-level `SP_v1.4.6:` keys and two `SP_v1.3.6:` keys. PyYAML keeps the last, so the second block of each pair was what every consumer saw — and both were stale. They arrived in merge c22f075, which resolved a conflict by keeping both sides (and renamed the dead `SSP_v1.4.6_msel` key to `SP_v1.4.6` in the process). Delete the shadowing blocks and repair the surviving `SP_v1.4.6` to the post-aa774b65d convention: * `cov_th` stays at A = 2894.03 deg^2 / n_e = 5.0935 (the nside-4096 footprint mask every other live entry uses), not the 2405 / 6.128 pair the shadowing block held. * `shear.redshift_distr` -> `shear.redshift_path`: the code only ever reads `redshift_path`, so the v1.4.6 n(z) was under a key nothing reads while the winning block pointed at the v1.0-era `dndz_SP_A.txt`. * `shear.mask` (read nowhere) -> entry-level `mask:` pointing at `mask_map_footprint_nside_4096.fits`, matching SP_v1.4.5 / SP_v1.4.6.3 and the mask `A` was measured from. * `shear.R: 1.0` — the delivered e1/e2 columns already have the response applied; the shadowing block's `R: 0.92` double-counted it and inflated xi_pm by 1/R^2 = 18% against a covariance that never sees R. `SP_v1.3.6`'s surviving block already carries the nside-4096 footprint mask; its cov_th still holds the old 2405 / 6.128 pair and needs regenerating separately.
`yaml.safe_load` silently keeps the last of a repeated key, so a merge that resolves a conflict by keeping both sides leaves the shadowed block invisible to every consumer *and* to every test. That is how two `SP_v1.4.6` and two `SP_v1.3.6` entries survived on develop. Add a `yaml.SafeLoader` subclass whose `construct_mapping` raises on a repeated key, and assert `cosmo_val/cat_config.yaml` parses clean with it. Fails on the pre-fix config with `duplicate key 'SP_v1.3.6' at line 522 (first seen at line 148)`.
…e-4096 footprint aa774b6 repointed SP_v1.3.6's mask to the 2894 deg² footprint but left its cov_th holding v1.4.6's old numbers. Recomputed from v1.3.6's own catalogue with sp_validation.survey; the same code path reproduces SP_v1.4.6.3's committed A / n_e / sigma_e bit-for-bit. n_psf left as it was.
cailmdaley
force-pushed
the
fix/cat-config-duplicate-keys
branch
from
September 5, 2026 14:08
6cf2638 to
ed0a0f6
Compare
Collaborator
Author
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cosmo_val/cat_config.yamlhad two top-levelSP_v1.4.6:keys and twoSP_v1.3.6:keys; PyYAML keeps the last, so the second block of each pair was the live one. The secondSP_v1.4.6was a long-dead one-off variant (SSP_v1.4.6_msel) that mergec22f07568renamed and kept alongside the real entry; it was never meant to beSP_v1.4.6.This PR:
SP_v1.4.6:shear.R: 1.0,cov_thA = 2894.03 deg² / n_e = 5.0935,shear.redshift_distr→shear.redshift_path(the key the code reads),shear.mask→ entry-levelmask: …/mask_map_footprint_nside_4096.fits, as inSP_v1.4.5andSP_v1.4.6.3;SP_v1.3.6'scov_th(A, n_e, sigma_e) from its own catalogue on the nside-4096 footprint — it had been left with v1.4.6's numbers; the same code path reproducesSP_v1.4.6.3's committed values bit-for-bit;cat_config.yamlparses clean under it (fails on the pre-fix file).Not touched:
SP_v1.4.6_glass_mock, which still carries the old A / n_e pair (see #319).— Claude on behalf of Cail