Skip to content

feat: Mark override-affected evaluations in analytics events - #528

Draft
kinyoklion wants to merge 3 commits into
rlamb/overrides-python-override-storefrom
rlamb/overrides-python-events
Draft

kinyoklion wants to merge 3 commits into
rlamb/overrides-python-override-storefrom
rlamb/overrides-python-events

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

This is the fourth step of the flag overrides port described by the OVERRIDE specification. It is based on the override store branch because the phases are stacked; retarget to feat/overrides once that branch merges.

EventInputEvaluation gains override_affected, set by the client for the top-level evaluation and by the evaluator for each prerequisite record from that record's own marking. The event processor keys on this scalar alone and never reads the evaluation reason: an override-affected evaluation produces no individual feature event and no debug event, even when the flag requests them, while its index event and its summary counting are unchanged. An unaffected prerequisite record inside a marked evaluation is still recorded as usual.

The summary counter key becomes (variation, version, override_affected), so override-affected and ordinary evaluations of the same flag, variation, and version accumulate into separate counters instead of collapsing together. A counter that aggregates marked evaluations carries overrideAffected: true in the summary event; the property is present only when true, like the unknown marker. Both the sync and async event processors share this dispatch and output code.

In the all-flags state, an override-affected flag keeps its value, version, and marked reason but is presented with trackEvents and trackReason off and no debugEventsUntilDate, so a consumer bootstrapped from the state sends no individual events for it. When a migration evaluation of an overridden flag replaces the reason with a wrong-type error, the new reason keeps the marking. The async client mirrors these changes.

The OVERRIDE specification vector runner now also asserts the summaryOverrideAffected field of each vector against the marking the client hands to the event processor.

Tests cover the summarizer key split, the marker in the summary output, suppression of feature and debug events through both real event processors (including a marked prerequisite record and a mixed set of evaluations of one flag), the client-side marking of top-level and prerequisite records through a prerequisite tree, the all-flags tracking fields, the wrong-type reason, and an end-to-end payload check that an override-affected flag appears only in the summary.

SDK-3250

An evaluation that fails with an unexpected exception keeps the marking when the flag came from the override layer, so it produces no individual event and the all-flags state presents it with tracking off.


Note

Overview
Adds override_affected on evaluation analytics records so flag-override evaluations are handled differently from normal traffic in sync and async clients.

Event pipeline: Override-affected evaluations still update summary counters and index events, but no individual feature or debug events—even when the flag has tracking or debug enabled. Summary counters are keyed by (variation, version, override_affected) so marked and unmarked evaluations do not merge; marked counters emit overrideAffected: true in the summary payload.

Client wiring: Top-level eval events pass result.override_affected from the evaluator; prerequisite eval events pass each prereq’s own marking. WRONG_TYPE (migration) and EXCEPTION paths preserve overrideAffected on the reason and set the event flag when the definition came from the override layer. all_flags_state clears trackEvents, trackReason, and debugEventsUntilDate for override-affected flags so bootstrapped consumers do not emit per-flag events.

Async FDv2: Flag-change notifications after client close no longer raise when the event loop is closed (notification is dropped with a debug log).

Tests: New coverage for summarizer/processor behavior, OVERRIDE spec vectors (summaryOverrideAffected), and client scenarios (prereq trees, failures, end-to-end HTTP payloads).

Reviewed by Cursor Bugbot for commit 9f4108c. Bugbot is set up for automated code reviews on this repo. Configure here.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-override-store branch from 3fcdc80 to 19f88eb Compare September 28, 2026 20:29
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-events branch from 042fce2 to 0e55650 Compare September 28, 2026 20:29
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-override-store branch from 19f88eb to 7e7b3f4 Compare September 30, 2026 20:31
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-events branch from 0e55650 to c79f244 Compare September 30, 2026 20:31
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-override-store branch from 7e7b3f4 to aa28264 Compare October 1, 2026 23:43
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-events branch from c79f244 to 352d360 Compare October 1, 2026 23:43
Carries the override-affected marking on the evaluation event input, so the
event processor keys on the marking alone and never reads the evaluation
reason. An evaluation marked as override-affected produces no individual
feature event and no debug event, even when the flag requests them, and it is
counted in summary events like any other evaluation. The marking is part of
the summary counter key, so override-affected and ordinary evaluations of the
same flag, variation, and version accumulate into separate counters, and a
counter that aggregates marked evaluations carries the overrideAffected
marker, present only when true, like the unknown marker.

The evaluator passes each prerequisite record's own marking to its event, so
a marked prerequisite record produces no individual event while an unaffected
prerequisite inside a marked evaluation is recorded as usual. The client
passes the top-level marking to the evaluation event, keeps the marking on the
reason when a migration evaluation replaces it with a wrong-type error, and
presents an override-affected flag in the all-flags state with trackEvents
and trackReason false and no debugEventsUntilDate, so a consumer bootstrapped
from the state sends no individual events for it. The async client mirrors
these changes.

The OVERRIDE specification vector runner now also checks the marking each
evaluation contributes to its summary counter.
…ons after the loop closes

When an evaluation raises and the flag definition came from the override layer, the EXCEPTION reason carries the override indicator, the default event record is marked so no individual event is produced, and the all-flags state presents the flag with tracking off. The async data system drops a flag change notification that arrives after the event loop closed instead of raising on the reload thread.
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-override-store branch from aa28264 to 1c1bd5c Compare October 3, 2026 01:18
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-python-events branch from 352d360 to 9f4108c Compare October 3, 2026 01:18
@kinyoklion

Copy link
Copy Markdown
Member Author

bugbot review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 9f4108c. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant