Skip to content

fix(cli): refresh identity persistence classification - #3069

Open
jrusso1020 wants to merge 1 commit into
mainfrom
fix/identity-persistence-refresh
Open

fix(cli): refresh identity persistence classification#3069
jrusso1020 wants to merge 1 commit into
mainfrom
fix/identity-persistence-refresh

Conversation

@jrusso1020

Copy link
Copy Markdown
Collaborator

Summary

  • key the in-process persistence classification to the active anonymous ID
  • preserve same-ID stickiness while reclassifying replacement IDs after config deletion or storage recovery
  • add regression coverage for durable-to-new-ID and process-only-to-recovered-ID transitions

This is a corrective follow-up to #3065. It does not change event names, distinct_id, render events, PostHog queries, or dashboard semantics.

Verification

  • focused telemetry and identity tests: 124 passed
  • full CLI suite: 2,437 passed in the sandbox; all 37 localhost-binding tests passed separately with loopback permission
  • workspace build
  • CLI typecheck
  • oxfmt, oxlint, diff check, tracked-artifact check, and fallow audit

@miga-heygen miga-heygen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: identity persistence reclassification

Clean corrective follow-up to #3065. The original design locked the classification to the process lifetime; this PR keys it to the anonymous ID, so a long-running process that replaces its ID (config deletion, storage recovery) correctly reclassifies the replacement.

The invariant change, and why it's safe

Before (process-level stickiness):

if (identityPersistence !== undefined) return;

Once classified, never changes. This is too aggressive — if a readConfigFresh() mints a new ID (e.g. config deleted between calls), the NEW id inherits the OLD id's classification.

After (per-ID stickiness):

if (identityClassification?.anonymousId === anonymousId) return;

Same ID → guard holds (anti-self-promotion intact). Different ID → reclassification proceeds.

Traced all four classification call sites through readConfig():

Path ID from disk? Classification Reclassifies on new ID?
No config file → mintAndCacheConfig() N/A unknown or process_only
Existing file, needs seed backfill, id on disk yes durable
Existing file, needs seed backfill, id minted no by write outcome
Existing file, no backfill needed either durable or process_only
Existing file corrupt → catch recovery N/A by write outcome

Every site passes config.anonymousId as the key. The guard correctly skips when the ID matches (stickiness) and proceeds when it differs (reclassification).

Anti-self-promotion still works

The critical scenario from #3065's review:

  1. No config → mint ID "A" → classify as unknown
  2. readConfigFresh() → cache cleared → file exists (we wrote it) → reads ID "A" → classifyIdentity("A", "durable") → guard: "A" === "A"rejected → stays unknown

The original test (line 751) still covers this. The new expect(getIdentityWriteOutcome()).toBe("ok") assertion at line 38 tightens the existing test.

New test coverage

"reclassifies a new id after a durable install's config is deleted" (line 41):

  1. Loads existing config → durable
  2. Deletes config file, calls readConfigFresh()
  3. New ID minted → correctly classified as unknown with ok write outcome
  4. Asserts replacement.anonymousId !== first.anonymousId

"reclassifies a new id after process-only storage recovers" (line 57):

  1. Writes fail → process_only/failed
  2. Writes recover, calls readConfigFresh()
  3. New ID minted → correctly classified as unknown with ok write outcome
  4. Asserts replacement.anonymousId !== first.anonymousId

Both tests verify the scenario that #3065's process-level stickiness got wrong. Good regression coverage.

Getter safety

The getters add a second ID-match check:

return identityClassification?.anonymousId === config.anonymousId
  ? identityClassification.persistence
  : "unknown";

This is belt-and-suspenders — if the config's ID somehow diverges from the classification's ID (shouldn't happen in normal flow, but could via direct writeConfig mutation), the getter returns the safe default rather than a stale classification. Correct defensive design.

process_only description update

The description expanded from "the write failed" to "not persisted by the identity-establishing path (the write failed or no write occurred)." This correctly covers the no-backfill-needed path at line 748 where a minted replacement ID has no write opportunity — it's genuinely process-only.

CI note

Two infra-level timeouts (Detect changes 52m, Analyze javascript-typescript 52m) — runner resource issues, not code-related. Core checks passing: CLI smoke, Producer tests (unit + integration), Typecheck, Lint, Format, Windows tests/render. A few still pending (Build, Test, SDK).

No blocking concerns. Clean correction that preserves the stickiness invariant while fixing the reclassification gap.

— Miga

@vanceingalls vanceingalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R1 review — LGTM, corrective follow-up cleanly narrows classifier keying and covers both replacement paths

Head reviewed: ecf8d62

Requester claims — verdict per claim:

  1. Classification-keying to active anonymous ID: CORRECT
  2. Reclassifies replacement ID after config deletion or storage recovery: CORRECT
  3. Same-ID stickiness intact: CORRECT
  4. No change to event names / distinct_id / render events / dashboard semantics: CORRECT
  5. Follow-up to #3065: CORRECT

Lens 1 — classification-keying (CORRECT)

packages/cli/src/telemetry/config.ts:579-586classifyIdentity(anonymousId, persistence, writeOutcome) early-returns iff identityClassification?.anonymousId === anonymousId. Different id → overwrites the whole record. The tuple {anonymousId, persistence, writeOutcome} moves as one unit, so a stale outcome can never survive an id transition.

All four writer call-sites updated to pass config.anonymousId: packages/cli/src/telemetry/config.ts:354-358 (mint-and-cache), 732, 734-738, 748, 760-764 (corrupt-recovery). No orphaned two-arg call left in the tree.

Lens 2 — same-ID stickiness (CORRECT)

Trace on the ephemeral-HOME fresh-mint case (the primary case that motivated unknown in #3065):

  1. mintAndCacheConfig classifies X as unknown/ok. cachedConfig = {X}.
  2. readConfigFresh() clears the cache; readConfig re-parses the just-written file; idFromDisk=true, bucketSeed present → hits the packages/cli/src/telemetry/config.ts:748 no-write branch → classifyIdentity(X, "durable").
  3. Early-return fires (previous anonymousId === X). Classification stays unknown.

The new assertion at packages/cli/src/telemetry/config.test.ts:757 (expect(getIdentityWriteOutcome()).toBe("ok") added to the existing sticky test) pins the outcome half of the tuple too — not just the persistence label — closing the gap where a hypothetical bug could preserve persistence and drop writeOutcome.

Lens 3 — no dashboard-semantics change (CORRECT)

packages/cli/src/telemetry/client.ts:113,116-117 — the only client-side changes are comment refinements on identity_persistence and config_write_outcome. No field rename, no new field, no call-site change on posthog.capture, no distinct_id touch, no render-event delta. Grepped the diff for capture|identify|distinct_id|render_ — zero hits outside comments. The three-valued domain of identity_persistence (durable | process_only | unknown) is unchanged; dashboards computing rates over that domain will see the SAME set of possible values. The data distribution shifts on churn workloads (correctly — replacement ids after deletion now report unknown instead of stale durable), which is the fix's intended effect.

Lens 4 — adversarial internal-boundary audit at the fix's execution graph

Applied feedback_adversarial_audit_fix_internal_boundaries — walked every remaining silent-misclassification path inside the fix's own boundary:

  • Reader-writer symmetry (getIdentityPersistence/getIdentityWriteOutcome at config.ts:594-611): both call readConfig() FIRST, then check identityClassification?.anonymousId === config.anonymousId. If readConfig() returned the cached path (no re-classify), classification already matches the cached id. If it re-entered the parse/mint path, classification is set/overwritten to the current id in that same call. The check is defensive — an actual observable divergence would require a mutation of cachedConfig outside readConfig/writeConfigWithResult, which grep confirms doesn't happen (cachedConfig = only at config.ts:359,742,750,778,818).
  • N > 2 replacements in one process: Each transition overwrites identityClassification wholesale; there is no per-id memory to leak. All later reads see only the latest tuple. Fine.
  • Corrupt-config recovery branch (config.ts:752-766): now passes config.anonymousId into classifyIdentity, so the recovery's minted id gets its own classification. Pre-fix bug (stale first-verdict wins) is closed here too.
  • Concurrency: All FS ops are sync; Node single-threaded; no race window inside the classify/read cycle.
  • First-run vs replacement: first-run mints X, classifyIdentity(X, "unknown", "ok") — distinguishable from any later reclassify because identityClassification starts undefined. install_predecessor_found=false from mintConfig() (unchanged) distinguishes genuine-first vs recovered-machine cases at the event layer.
  • Backward compat: old rolling processes on the pre-#3069 build continue to emit their sticky-per-process verdict. That's the pre-fix behavior for those processes — dashboards already accept it. New builds emit the corrected verdict. No mixed-population contract break.

Lens 5 — PR envelope

  • Single commit by james.russo@heygen.com (jrusso1020). No Co-Authored-By: trailers, no Generated with [Claude Code] footer. Envelope clean.
  • PR body uses ## Summary + ## Verification — same shape as parent PR #3065 which merged fine; hyperframes' What/Why/How/Test-plan template is not enforced by CI. Not blocking.
  • CI at review time: Build, Render on windows-latest, Semantic PR title, Tests on windows-latest, Typecheck all SUCCESS. Test, Test: runtime contract, regression still open. Approve here is code-review-only; merge should await the 3 open required gates.

Lens 6 — Standards checklist (mechanical)

Grepped the diff for \bas +[A-Z], \w+!\., \w+!\[, angle-bracket casts, and .message on untyped catch. Results:

  • Bare as T on changed lines: 0 residual (the existing JSON.parse(raw) as Partial<HyperframesConfig> at config.ts:711 is unchanged and outside the fix's touched lines).
  • Non-null ! assertions on changed lines: 0.
  • Angle-bracket casts: 0.
  • catch (e) { e.message } without narrowing: 0. Recovery branch uses bare catch { ... } — no error-value access.

Test coverage — does each new test fail at pre-patch HEAD?

  • packages/cli/src/telemetry/config.test.ts:762-777 (durable → deleted config → new id): pre-patch classifyIdentity early-returned on identityPersistence !== undefined, so the second classification would be ignored and the assertion expect(getIdentityPersistence()).toBe("unknown") would see stale "durable". FAILS pre-patch, passes post-patch.
  • packages/cli/src/telemetry/config.test.ts:779-799 (process_only → storage recovers → new id): symmetric — pre-patch would return stuck "process_only". FAILS pre-patch, passes post-patch.

Both tests genuinely exercise the bug being fixed, not just tautologies.


Findings summary: 0 blockers, 0 non-blocker fixes. All five requester claims verified as CORRECT.

Review by Via

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants