Skip to content

The history page shows a code.bisect_candidates call - #669

Open
caseylee-TC wants to merge 3 commits into
ClickHouse:mainfrom
caseylee-TC:docs/bisect-candidates-example
Open

caseylee-TC wants to merge 3 commits into
ClickHouse:mainfrom
caseylee-TC:docs/bisect-candidates-example

Conversation

@caseylee-TC

@caseylee-TC caseylee-TC commented Nov 14, 2026 •

Copy link
Copy Markdown

Closes: #633

Changelog category (leave one):

  • Documentation (changelog entry is not required)

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

Not required for a documentation change.

Why

The "Bisect candidates" section of the history page listed the inputs of code.bisect_candidates but, unlike code.pr_check, showed no call. This adds one example with a version window and symbols that prints each candidate's PR number and score. It says the op needs merged-PR history and that the score ranks candidates without proving a cause.

Scope and risk

One docs page, about 24 added lines. No code, settings or generated files change. The example sits after the numbered steps so those steps stay together.

Compatibility, rollout and rollback

  • Wire: none.
  • Operator steps: none.
  • Rollback: reverting removes the added text and leaves nothing behind.

Tests

  • make lint and make typecheck pass.
  • make format-check, the two pytest files (240 passed) and docs_reference --check pass, as listed in the issue.
  • The field names (symbols, bad_version, good_version, candidates, pr_number, score) match docs/reference/operations/code.md. The input and output names are also confirmed in CodeBisectCandidatesInput and CodeBisectCandidatesOutput.
  • mkdocs build --strict passes and the rendered page looks right (screenshot attached).
  • Not run: the example against a live server with PR history loaded. UploadClient.send follows the dotted style of the code.pr_check example; another page uses ::, so I'm happy to change it.
  • I checked it against server/code/app/bisect_scan.py and with a
    throwaway test: with bad_version / good_version and no tracker, the plan clears the time
    window, and the PR-history read covers the last LOOM_CODE_BISECT_HISTORY_DAYS (90 by default)
Screenshot 2026-11-16 153515

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated (an agent wrote most of the diff): Claude (Anthropic) drafted the example and its two paragraphs from the repository's docs and code. I checked every field name against the reference page and the models, ran the checks above, and viewed the rendered page. I also adjusted the first paragraph to say the op ranks PRs by the files of a crashing function.
  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • make format-check, make lint, make typecheck and make test-unit pass; the integration suite passes if server behaviour changed. (docs-only change. make format-check, make lint and make typecheck pass, as do the doc-link and public-copy tests and the docs reference check. I stopped make test-unit locally before it finished; CI runs the full unit suite.)
  • Generated files are regenerated in this PR, not by hand: the SDK op tables, docs/reference/configuration.md and the docs/reference/ tables (python scripts/codegen/docs_reference.py), the reach manifest, schemas/schema.sql (make schema-dump) and the Codex plugin copies (python3 scripts/codegen/agent_plugin_sync.py). (not applicable: docs only)
  • A new migration is additive and numbered after the latest on main; no merged migration is edited. (not applicable: docs only)
  • A behaviour change to any file a plugin ships (hooks/, lib/, bin/, skills/, .mcp.json, the Codex plugin-defaults.json) bumps both plugin manifests' version, and no other open PR uses that version. (not applicable: docs only)
  • The frontend builds (npm --prefix frontend run build) and its tests pass, if frontend/ changed. (not applicable: docs only)
  • Docs under docs/ describe the system as it now is, and this description names any operator steps and wire changes.
  • Repository text has no dates, people, host names, internal namespaces, ticket or decision numbers; the history goes in this description (tests/unit/test_public_copy.py).
  • Commits are signed off (git commit -s), as CONTRIBUTING.md asks.

The section listed the inputs but, unlike code.pr_check, never showed a call. Add one example with a version window and symbols, and say that it needs PR history and that the score ranks candidates without proving a cause.

Signed-off-by: Casey Lee <caseylee.techcomm@gmail.com>
@zlareb1

zlareb1 commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

@caseylee-TC The example's first sentence does not hold in the setup it describes. With merged-PR history and no tracker, bad_version / good_version do not bound the scan:

  • _plan in server/code/app/bisect_scan.py resolves the two versions into versions and then sets before / after to None, because a version window replaces a time window.
  • The versions only filter the tracker scan (merged_prs_touching). Without a tracker the op falls back to _code_history_rows, which reads loom.code_pr_history over plan.before or now back to code_bisect_history_days (90 by default). The 26.2 → 26.3 window plays no part.

So the snippet, run as described, ranks the PRs of the last 90 days that touched the function's files, not the PRs between the two releases. Either of these would fix it:

  1. Say that the version window needs a tracker namespace bound to the repository whose PRs carry release labels, and that with PR history alone the op reads the last code_bisect_history_days.
  2. Keep the PR-history setup, drop bad_version / good_version from the example and describe the window as the last code_bisect_history_days.

The rest of the section (the score ranks candidates without proving a cause) is right.

With merged-PR history and no tracker, bad_version and good_version do not bound the scan; the op reads the last LOOM_CODE_BISECT_HISTORY_DAYS. Say so, and say the version window needs a linked tracker.

Signed-off-by: Casey Lee <caseylee.techcomm@gmail.com>
@caseylee-TC

Copy link
Copy Markdown
Author

Thanks @zlareb1, that's a good catch. I checked it against server/code/app/bisect_scan.py and with a
throwaway test: with bad_version / good_version and no tracker, the plan clears the time
window, and the PR-history read covers the last LOOM_CODE_BISECT_HISTORY_DAYS (90 by default).

I went with your option 1. In 4dcafc4 the paragraph above the example now says the version
window needs a tracker namespace linked to the repository whose PRs carry release labels, and
that with PR history alone the op reads the last LOOM_CODE_BISECT_HISTORY_DAYS. The example
and the rest of the section are unchanged. I'll update the original PR description with the new screenshot and the throwaway test.

I haven't run the example against a live server, and I haven't verified the release-label
condition myself; I took it from your description. Please say if the wording needs to change.

@zlareb1

zlareb1 commented Nov 16, 2026

Copy link
Copy Markdown
Member

Thanks, the fix is right except for one phrase, and that one came from my comment: "PRs carry release labels" is not accurate. The tracker scan does not read GitHub labels. It matches each PR record's minor_version field (server/tracker/store/source/code_history.py, minor_version IN {versions}), the record field listed in the tracker record contract. Suggested sentence:

The version window needs a tracker namespace linked to the repository whose PR records carry minor_version (the record contract).

With that change it can merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code history and PRs: the code.bisect_candidates section has no call example

2 participants