Repository navigation
feat: Mark override-affected evaluations in analytics events - #528
Draft
kinyoklion wants to merge 3 commits into
Draft
kinyoklion wants to merge 3 commits into
kinyoklion wants to merge 3 commits into
Conversation
kinyoklion
force-pushed
the
rlamb/overrides-python-override-store
branch
from
September 28, 2026 20:29
3fcdc80 to
19f88eb
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
September 28, 2026 20:29
042fce2 to
0e55650
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-override-store
branch
from
September 30, 2026 20:31
19f88eb to
7e7b3f4
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
September 30, 2026 20:31
0e55650 to
c79f244
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-override-store
branch
from
October 1, 2026 23:43
7e7b3f4 to
aa28264
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 1, 2026 23:43
c79f244 to
352d360
Compare
Carries the override-affected marking on the evaluation event input, so the event processor keys on the marking alone and never reads the evaluation reason. An evaluation marked as override-affected produces no individual feature event and no debug event, even when the flag requests them, and it is counted in summary events like any other evaluation. The marking is part of the summary counter key, so override-affected and ordinary evaluations of the same flag, variation, and version accumulate into separate counters, and a counter that aggregates marked evaluations carries the overrideAffected marker, present only when true, like the unknown marker. The evaluator passes each prerequisite record's own marking to its event, so a marked prerequisite record produces no individual event while an unaffected prerequisite inside a marked evaluation is recorded as usual. The client passes the top-level marking to the evaluation event, keeps the marking on the reason when a migration evaluation replaces it with a wrong-type error, and presents an override-affected flag in the all-flags state with trackEvents and trackReason false and no debugEventsUntilDate, so a consumer bootstrapped from the state sends no individual events for it. The async client mirrors these changes. The OVERRIDE specification vector runner now also checks the marking each evaluation contributes to its summary counter.
…ons after the loop closes When an evaluation raises and the flag definition came from the override layer, the EXCEPTION reason carries the override indicator, the default event record is marked so no individual event is produced, and the all-flags state presents the flag with tracking off. The async data system drops a flag change notification that arrives after the event loop closed instead of raising on the reload thread.
kinyoklion
force-pushed
the
rlamb/overrides-python-override-store
branch
from
October 3, 2026 01:18
aa28264 to
1c1bd5c
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 3, 2026 01:18
352d360 to
9f4108c
Compare
Member
Author
|
bugbot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9f4108c. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the fourth step of the flag overrides port described by the OVERRIDE specification. It is based on the override store branch because the phases are stacked; retarget to
feat/overridesonce that branch merges.EventInputEvaluationgainsoverride_affected, set by the client for the top-level evaluation and by the evaluator for each prerequisite record from that record's own marking. The event processor keys on this scalar alone and never reads the evaluation reason: an override-affected evaluation produces no individual feature event and no debug event, even when the flag requests them, while its index event and its summary counting are unchanged. An unaffected prerequisite record inside a marked evaluation is still recorded as usual.The summary counter key becomes
(variation, version, override_affected), so override-affected and ordinary evaluations of the same flag, variation, and version accumulate into separate counters instead of collapsing together. A counter that aggregates marked evaluations carriesoverrideAffected: truein the summary event; the property is present only when true, like theunknownmarker. Both the sync and async event processors share this dispatch and output code.In the all-flags state, an override-affected flag keeps its value, version, and marked reason but is presented with
trackEventsandtrackReasonoff and nodebugEventsUntilDate, so a consumer bootstrapped from the state sends no individual events for it. When a migration evaluation of an overridden flag replaces the reason with a wrong-type error, the new reason keeps the marking. The async client mirrors these changes.The OVERRIDE specification vector runner now also asserts the
summaryOverrideAffectedfield of each vector against the marking the client hands to the event processor.Tests cover the summarizer key split, the marker in the summary output, suppression of feature and debug events through both real event processors (including a marked prerequisite record and a mixed set of evaluations of one flag), the client-side marking of top-level and prerequisite records through a prerequisite tree, the all-flags tracking fields, the wrong-type reason, and an end-to-end payload check that an override-affected flag appears only in the summary.
SDK-3250
An evaluation that fails with an unexpected exception keeps the marking when the flag came from the override layer, so it produces no individual event and the all-flags state presents it with tracking off.
Note
Overview
Adds
override_affectedon evaluation analytics records so flag-override evaluations are handled differently from normal traffic in sync and async clients.Event pipeline: Override-affected evaluations still update summary counters and index events, but no individual
featureordebugevents—even when the flag has tracking or debug enabled. Summary counters are keyed by(variation, version, override_affected)so marked and unmarked evaluations do not merge; marked counters emitoverrideAffected: truein the summary payload.Client wiring: Top-level eval events pass
result.override_affectedfrom the evaluator; prerequisite eval events pass each prereq’s own marking. WRONG_TYPE (migration) and EXCEPTION paths preserveoverrideAffectedon the reason and set the event flag when the definition came from the override layer.all_flags_stateclearstrackEvents,trackReason, anddebugEventsUntilDatefor override-affected flags so bootstrapped consumers do not emit per-flag events.Async FDv2: Flag-change notifications after client close no longer raise when the event loop is closed (notification is dropped with a debug log).
Tests: New coverage for summarizer/processor behavior, OVERRIDE spec vectors (
summaryOverrideAffected), and client scenarios (prereq trees, failures, end-to-end HTTP payloads).Reviewed by Cursor Bugbot for commit 9f4108c. Bugbot is set up for automated code reviews on this repo. Configure here.