an association scan that mostly refuses to say anything - #50
Merged
Conversation
correlates day-level series against each other on device. the whole thing is built around not inventing findings: - moving-block permutation for the null. days are autocorrelated and a plain shuffle destroys exactly the structure that caused the spurious correlation, so it calls it significant. measured on pure noise at phi 0.6, day-wise shuffling published something in 69% of histories, 14-day blocks in 11%. at phi 0.8 it's 98% vs 12%. that's the whole reason for the block length. - benjamini-hochberg over the entire grid, and nothing is screened on effect size before the correction runs. - lag is derived per pair from when each variable actually happened, so today's strain lands on tonight's sleep and nothing runs backwards. same-day pairs are simultaneous, not directional, so they're refused. - weekday is removed from both series before testing. "late meals hurt my recovery" is usually saturday. - readiness is never explained by hrv/rhr/resp/temp. it IS them. a variable declares what it's made of and those pairs are refused. - sleep duration, longest sleep and efficiency collapse to one finding instead of three. - under 12 weeks it says how many nights it needs. if the grid is wider than the history could ever answer, it says that too rather than returning an empty list and looking like nothing's happening. - imported days are erased, not blended. ~15 ms for 6 variables over 90 days, 80 ms for 14 over a year. the journal path (journalCorrelations / journalNumericCorrelations) is untouched, this doesn't duplicate it.
review on #48, and it's right that the comments and the reason strings are callable surface rather than decoration. several of them described an earlier version of this. - blockLen doc said 7, ships 14. the table on associationMinPairedDays is where the 14 came from, so point at it. - associationLag can't return null and never could, so backwards_in_time was dead and simultaneous/not_an_input/misaligned_series were documented but never emitted. dropped the nullable return and the dead guard, and wrote out the reason list as exactly what comes out. the same-night refusal is real, it just lives in the +1 alignment rather than in a reason string — said so where the +1 happens. - two variables with the same key shared one grid slot and the loser's label could get published against the winner's series. same for two rows on one calendar day. abstain, like a misaligned series does. - coverage was checked after the count, which needs a 168-day window before it can bind, so every rarely-logged field came back need_pairs. "log this more often" and "wait 74 nights" are different instructions and only one of them is true for a field logged one day in twelve. rate first. - short_history used a literal 84 next to a constant that is 84. - tests for no_contrast, the variable-level constant refusal, and redundant_with in the json, which is the shape the edge will actually read.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughChangesAssociation scan
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant scanAssociations
participant StatisticalHelpers
participant AssociationScan
Caller->>scanAssociations: provide dates and daily variables
scanAssociations->>StatisticalHelpers: adjust, rank, lag, and test paired series
StatisticalHelpers-->>scanAssociations: correlations, p-values, and contrasts
scanAssociations->>AssociationScan: apply FDR and redundancy rules
AssociationScan-->>Caller: return findings, tested pairs, and refusals
Possibly related PRs
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-raise of #48 against main — it was merged into its stacked base branch by mistake, so main never got it. Same commits, already reviewed and resolved there.
Engine + tests only, no UI. The number that decided the design: on 200 synthetic histories of pure AR(1) noise, a day-wise shuffle publishes at least one false finding in 69% of histories at φ=0.6 and 98% at φ=0.8; 14-day moving blocks hold it at ~12% across the range. Both rows are in the suite as a live contrast.
Also: BH over the whole grid, lag declared per variable rather than scanned, weekday-median adjustment with the raw rho kept for disclosure, tautologies refused structurally, redundancy clustered at |rho| >= 0.7, and a discreteness guard that says so when no test could have published at any effect size.
Power is low and that is the honest answer, not a tuning failure — at 84 days only ~72 questions can reach significance at all, so the UI has to treat "nothing to say" as the normal state.
Summary by CodeRabbit