ngmix: record silent metacal failures in mcal_flags; fail loudly at 100%#854
Draft
cailmdaley wants to merge 1 commit into
Draft
ngmix: record silent metacal failures in mcal_flags; fail loudly at 100%#854cailmdaley wants to merge 1 commit into
cailmdaley wants to merge 1 commit into
Conversation
mcal_flags (NGMIX_MCAL_FLAGS) faithfully recorded failures ngmix *reported*, but defaulted an absent metacal-type result or missing 'flags' key to 0 -- success -- so a fitter failing silently (sentinel outputs, flags never set) produced a fully-populated catalogue with an all-zero flag column. Close the hole three ways: * absent result / absent 'flags' now contributes FLAG_NO_RESULT (2^30) to mcal_flags and to mcal_types_fail instead of counting as success; * a nominally clean fit (mcal_flags == 0) whose noshear shear is non-finite is flagged FLAG_NO_RESULT -- that contradiction is always a silent failure; * end of loop: RuntimeError when objects were processed but none fitted, and a w_log.error line when 100% of fitted objects carry nonzero mcal_flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019dA9sas4vvNBcNKRmi973L
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #853
Closes the "absent flag = success" hole found in the A/B campaign (#813) when a container's ngmix lacked the azgauss code path and every fit failed invisibly:
FLAG_NO_RESULT = 2**30: an absent metacal result (or missing'flags'key) now sets a dedicated, greppable bit instead of defaulting to 0 in bothget_mcal_flagsandmcal_types_fail.flags == 0) whose noshear shear is non-finite is flagged — the one contradiction that is always wrong, regardless of which sentinel a failing path emits.RuntimeErrorif 0 of N objects fitted (abort rather than write an empty catalogue), and a loudw_log.errorif 100% of fitted objects carry nonzero flags.Open reviewer decision: the
RuntimeErroron 0-fitted crashes the tile job instead of producing an empty catalogue. If you'd rather keep the pipeline flowing, it demotes to a second error-log line with no other change — say the word.Logic smoke-tested inside the production sims container (absent type → flagged; all-clean → 0; reported failures OR through; missing flags key → flagged).
— Claude (Fable), on behalf of Cail.
🤖 Generated with Claude Code
https://claude.ai/code/session_019dA9sas4vvNBcNKRmi973L