feat: add flag code-reference and extinction tools for cleanup sweeps - #83
feat: add flag code-reference and extinction tools for cleanup sweeps#83alohaninja wants to merge 4 commits into
Conversation
Flag-cleanup sweeps need to answer two questions: does this flag still
have code references, and when did its last reference go away. Neither
was answerable with the tools we exposed.
get-code-references maps to GET /api/v2/code-refs/repositories, which
accepts a flagKey parameter but does not honor it. Verified against
catamorphic: the response is identical with and without flagKey (4,569
bytes either way), and adding withReferencesForDefaultBranch=true returns
an unfiltered dump of every branch in every repository (11,103,275 bytes)
with no per-flag reference data at all. Agents reaching for it got a
context-exhausting payload and no answer.
Enable two operations already present in the spec but not exposed:
- check-flag-code-references -> getStatistics
(GET /api/v2/code-refs/statistics/{projectKey})
Flag-scoped and authoritative. Returns repositories with hunkCount,
fileCount, and latestCommitTime; an empty flags object means zero
references. 103 bytes when empty, versus 11MB from the repositories
endpoint.
- get-flag-extinctions -> getExtinctions
(GET /api/v2/code-refs/extinctions)
Returns the commit that removed a flag's last reference: revision,
full commit message, repository, and extinction timestamp. Replaces
git archaeology such as git log -S across source repositories.
Also correct two descriptions that were actively misleading:
- get-code-references no longer claims per-flag filtering, and warns
about the withReferencesForDefaultBranch payload size.
- get-flag-status-across-environments previously implied lastRequested
tracks code usage. It does not. Client-side allFlags() and useFlags()
emit an evaluation event per flag in the payload on js-client-sdk v3.x
and earlier, so a client-side-available flag keeps logging evaluations
with zero code references. The description now explains how to tell
real variation() traffic from a blanket allFlags() sweep, and points at
code references and extinctions for code truth. Background:
launchdarkly-labs/flag-delivery-labs ssingh/stuff/ghost-evaluations.md
Fix a latent grouping bug surfaced by enabling extinctions.
suggestions.yaml assigns x-speakeasy-group 'codereferences' (lowercase r)
to GET /api/v2/code-refs/extinctions, while every other Code references
operation uses 'codeReferences'. It was dormant while that operation was
ungenerated; once enabled, Speakeasy emitted two SDK groups for one tag
and split the class into Codereferences1/CodeReferences2, exposing both
sdk.codereferences and sdk.codeReferences. Corrected in spec-fixes.yaml
rather than suggestions.yaml, since spec-fixes applies last and survives
regeneration of the machine-generated suggestions overlay. The SDK keeps
a single CodeReferences class and a single sdk.codeReferences accessor.
Verified end to end over MCP stdio against catamorphic: 22 tools exposed,
both new tools return the payloads above, lint and build pass.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
bugbot run |
Bugbot flagged that regeneration dropped `initHooks(this)` and its import from SDKHooks, leaving src/hooks/registration.ts wired to nothing. The finding is correct. Root cause was local, not a generator change. genVersion is unchanged at 2.845.15 and the pinned speakeasyVersion 1.736.1 was used, so this is not template drift. `initHooks` has been present since the initial commit and the two prior regeneration commits (fc879ed, 64bcda2) preserved it. A local `speakeasy run` has no prior generation snapshot for the "merging custom edits" step to reconcile against, so the call was dropped. CI regeneration would not have produced this. Restored src/hooks/hooks.ts to main, so this PR no longer touches it. initHooks is currently an empty no-op, but registration.ts remains the documented extension point and must stay wired. Also normalized the version. Running generation three times locally inflated the bump to 0.6.5, skipping 0.6.3 and 0.6.4, which were never published. Regenerated with --set-version 0.6.3 so all version strings are written consistently by the generator rather than hand-edited. Verified after the change: lint and build pass, 22 tools still exposed, src/sdk/codereferences.ts still a single CodeReferences class with three methods, and both new tools still return the expected payloads over MCP stdio against catamorphic.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit bffd330. Configure here.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Three description defects, all in text an agent acts on literally. 1. check-flag-code-references overstated an empty result. The spec scopes this endpoint to flags "that have code references in the default branch", so an empty flags map means no references on the default branch of repositories onboarded to code-ref scanning, not proof of dead code. A flag referenced only on a release or maintenance branch, or in a repo never onboarded, also returns empty. Since no generated zod field carries .describe(), the tool description is the only text reaching the agent, so nothing else corrected it. The description now states the scope and directs the caller to confirm with get-flag-extinctions before treating a flag as dead. 2. get-flag-extinctions promised a repository name the tool cannot return. Extinction has exactly revision, message, time, flagKey, and projKey. Repository is the key of the enclosing items map, and generated zod objects .strip(), so any unmodelled field would be discarded before the agent sees it. Corrected, and added the two documented constraints that were missing: only the default branch is queried unless branchName is passed, and from and to must be supplied together. 3. get-code-references claimed the API does not honor flagKey. That was wrong, and the error was mine: the original measurement only tested flagKey without withReferencesForDefaultBranch, then generalized. Measured against catamorphic, flagKey does filter when withReferencesForDefaultBranch=true, returning 12 real references with path and hunks for a live flag versus none for a dead or nonexistent one. That combination also requires projKey, which returns a 500 when omitted. It is still the expensive path, since the reference data arrives alongside an ~11MB dump of every branch in every repository, but it is the only way to get file paths and hunk contents, which statistics cannot provide. The description now reflects the measured behavior instead of forbidding a working capability, which also stops it contradicting the generated SDK and docs text. Also flagged: both endpoints accept every filter as optional, so omitting flagKey silently inverts the scope to project-wide or account-wide. Left optional, since that matches the upstream contract and project-wide sweeps are legitimate, but both descriptions now warn about the scope change and the resulting payload size. Extinction.message is an unbounded commit body, so an unfiltered call can run to megabytes. Trimmed get-flag-status-across-environments. The identified-versus- anonymous context-kind heuristic was removed: allFlags() sprays using whatever context the SDK holds, so a logged-in customer app on js-client-sdk v3.x produces identified-context spray that the heuristic would misclassify as real variation() traffic. It generalized from two LaunchDarkly-internal apps and could drive a wrong archive decision in a customer account, and the response carries no context breakdown for an agent to act on anyway. Replaced with an explicit statement that this response cannot attribute evaluation sources. The mechanism stays documented in flag-delivery-labs ssingh/stuff/ghost-evaluations.md. Restored codeSamplesNamespace and codeSamplesRevisionDigest in .speakeasy/workflow.lock, dropped when the local code-samples step failed on an org mismatch. Nothing runs at PR time to restore them, so leaving them stripped would pin the registry overlay pre-PR and drop the TypeScript sample from the API reference for both new operations. workflow.lock now matches main exactly. Fixed the codereferences group typo at source in suggestions.yaml. The spec-fixes.yaml patch stays as the durable guard, since suggestions.yaml is machine-generated. Version normalized to a single 0.6.3 bump. Verified: lint and build pass, 22 tools, single CodeReferences class with three methods, all four descriptions byte-exact in the generated tools, and both new tools still return the expected payloads over MCP stdio.
|
Addressed in Blockers — both fixed
Phantom repository name. Confirmed: Your "suggestion" about the untested combination was actually a third blockerYou were right to push on this, and my claim was wrong. Measured:
The live response contains 12 real references with Worse than "steers agents off a working capability": Accepted with changes
Item 7 — trimmed, though your stated reason wasn't the strongest oneUnactionability is real, but the sharper problem is that the heuristic is unsound. Declined
Deferred
Note on manifest sizeDescription text went 5,523 → 6,464 chars. The two new tools grew because of the correctness caveats above; |
|
bugbot run |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…gbot)
Bugbot caught that the description told agents to set
withReferencesForDefaultBranch=true while the request schema types it as
z.string().optional(), matching the OpenAPI "if set to any value"
semantics. Verified through the MCP server against catamorphic:
boolean true -> MCP error -32602, zod invalid_type
(expected string, received boolean); the request never
reaches the API
string "true" -> passes validation and hits the API
So the documented invocation would have failed on an agent's first
attempt. The description now states the parameter is typed as a string
and to pass "true" rather than a JSON boolean.
Audited the other parameters referenced in the new descriptions for the
same trap: from and to on extinctions are z.number().int(), which matches
how the description presents them as Unix epoch milliseconds, and every
remaining parameter is a string key or name. withBranches is also a
string but is not referenced in any description.
Version normalized back to a single 0.6.3 bump.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e76a39d. Configure here.
1 participant
Why
Flag-cleanup sweeps need two answers: does this flag still have code references, and when did its last reference go away. Neither was answerable with the tools we expose, which sent agents down a misleading path.
get-code-referencesmaps toGET /api/v2/code-refs/repositories. That endpoint accepts aflagKeyparameter but does not honor it. Verified against catamorphic:projKey+flagKeyflagKeyhas no effect)+ withReferencesForDefaultBranch=true+ withBranches=falsecode-refs/statistics/{projectKey}?flagKey=…The generated client is fine, it serializes all four params. The API ignores them. So an agent asking "does this flag have code refs" got an 11MB unfiltered branch dump (25,011 branch objects across 10 repos, zero
hunks/flagKey/path fields) and no answer.What
Enable two operations already in the spec but never exposed as tools.
check-flag-code-references→getStatistics(GET /api/v2/code-refs/statistics/{projectKey})Flag-scoped and authoritative. Empty
flagsobject means zero references.{"flags":{"enable-google-oauth-sign-up":[{"name":"gonfalon","hunkCount":15,"fileCount":12,"latestCommitTime":1786630239000}]}}get-flag-extinctions→getExtinctions(GET /api/v2/code-refs/extinctions)Returns the commit that removed a flag's last reference.
{"revision":"bd9358bfd350a80881abbf224d16c396dc98d150", "message":"chore(login): remove the enable-oauth-multi-account-redirect flag (#68816)", "time":1786492311000,"flagKey":"enable-oauth-multi-account-redirect"}This replaces git archaeology. Investigating one flag took
git log -Splus deploy-channel history plus four rounds ofgh api compareto establish a removal commit and timestamp that this endpoint returns directly.Description corrections
get-code-referencesno longer claims per-flag filtering, and warns about thewithReferencesForDefaultBranchpayload size.get-flag-status-across-environmentspreviously impliedlastRequestedtracks code usage. It does not. Client-sideallFlags()/useFlags()emit an evaluation event per flag in the payload onjs-client-sdkv3.x and earlier, so a client-side-available flag keeps logging evaluations with zero code references. The description now explains how to separate realvariation()traffic (identified contexts, drains within days of a deploy) from a blanketallFlags()sweep (anonymous floor, persists until archive), and points at code references and extinctions for code truth. Background:launchdarkly-labs/flag-delivery-labs→ssingh/stuff/ghost-evaluations.md, whose own guidance is "judge cleanup on code references + intent, not raw eval counts."Latent bug fixed
Enabling extinctions surfaced a typo in the machine-generated
schemas/suggestions.yaml:Dormant while that operation was ungenerated. Once enabled, Speakeasy saw two groups for one tag, deleted
src/sdk/codereferences.ts, and emittedCodereferences1+CodeReferences2with bothsdk.codereferencesandsdk.codeReferencesaccessors. That would have been a silent breaking change to the SDK surface.Corrected in
schemas/spec-fixes.yamlrather thansuggestions.yaml, because spec-fixes applies last inworkflow.yamland therefore survives regeneration of the suggestions overlay. Result: a singleCodeReferencesclass withlistExtinctions,listRepositories,getStatistics, and onesdk.codeReferencesaccessor.Testing
Driven end to end over MCP stdio against catamorphic with a locally built server:
check-flag-code-referenceson a removed flag → 103 B,{"flags":{}}check-flag-code-referenceson a live flag → 435 B withhunkCount/fileCountget-flag-extinctions→ correct removal commit and2026-08-11T23:51:51Z, matching the merge time independently derived from git historynpm run lintandnpm run buildpasssrc/sdk/confirmed back to a singleCodeReferencesclassNotes for reviewers
schemas/mcp-enable-tools.yamlandschemas/spec-fixes.yaml. Everything else is Speakeasy output, committed to match the convention in fc879ed / d2d430e.0.6.2→0.6.3by Speakeasy's automatic versioning.launchdarkly-k7tvs the registry location) and is non-blocking; CI should handle it.check-flag-code-referencesis task-shaped to steer agents away fromget-code-references; happy to align differently.Note
Overview
Adds two MCP tools for flag-cleanup sweeps:
check-flag-code-references(per-flag reference counts on scanned default branches) andget-flag-extinctions(commits that removed a flag’s last reference). Both are wired through Speakeasy-generated SDK, funcs, models, and MCP registration; package version bumps to 0.6.3.Hand-edited
schemas/mcp-enable-tools.yamlenables those operations and rewrites agent guidance forget-code-references(flagKey alone does nothing; huge payloads withwithReferencesForDefaultBranch) andget-flag-status-across-environments(evaluation/lastRequestedis not proof of code usage).schemas/spec-fixes.yamlforcesx-speakeasy-group: codeReferenceson extinctions so generation does not split into duplicate SDK classes.Remaining diff is generated docs, lockfiles, and
schemas/output.json/suggestions.yamlalignment.Reviewed by Cursor Bugbot for commit e76a39d. Bugbot is set up for automated code reviews on this repo. Configure here.