Skip to content

Token path: convert HTML once, retire the always-on bug ledger row, stage-scoped runbook reads, docs-ready fast path - #37

Merged
MendixMau merged 10 commits into
masterfrom
claude/ab-token-path
Sep 12, 2026
Merged

Token path: convert HTML once, retire the always-on bug ledger row, stage-scoped runbook reads, docs-ready fast path#37
MendixMau merged 10 commits into
masterfrom
claude/ab-token-path

Conversation

@MendixMau

@MendixMau MendixMau commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Direct lane (bin/, skills/, tests/, process/)

What changed and why (one paragraph):

A Markdown+HTML requirements project was slow and token-hungry through stages 1–4. Measured causes: agents read raw HTML (3–10× the tokens of its text); projects scaffolded before 2026-09-08 still list the 47,500-word bug-logs/mxcli-bugs.md as always-on in CLAUDE.local.md; the whole 11,700-word runbook is read every session; and a text-native corpus owes the ledger one disposition per file, sidecar PNG/CSS/JS included. Five commits, each independently droppable:

  1. bin/html-to-md.sh + bin/lib/html-to-md.py (stdlib only) — converts every HTML page under the project's source root to Markdown under analysis/knowledge-base/text/, keeps headings/lists/tables, drops script/style/nav/footer chrome, writes inline base64 images out as files so they can be vision-read, prefixes each .md with a section list (the read's denominator), and writes documents-index.md naming every file under the source root (pages, markdown, images, chrome). Never touches the originals, never touches the toolkit's own HTML surfaces (analysis/, architecture/, design/, the guide, the dashboard) — the source-root detection deliberately drops source-sufficiency's legacy analysis/ fallback so it cannot convert the project's own outputs. Idempotent; --force reconverts.
  2. bin/sync-project.sh retires the stale ledger row in a project's CLAUDE.local.md to the bin/bug-lookup.sh row and says so. (CLAUDE.md's integration block is written by bootstrap-project.md, not init-project.sh, so sync reports the replacement there instead of editing.)
  3. bin/gate-check.sh prints Read for this gate: skills/conversion-runbook.md §"<stage>" (lines A–B) + §1b — line ranges derived from the runbook's real headings, never hardcoded — and the runbook's top says a session reads its stage's section, not the file.
  4. Docs-ready fast path (runbook → Entry Modes → "Requirements-driven, docs-ready corpus", explicit opt-in): Stage 0 = inventory + scope interview; Stage 1 = html-to-md.sh once, --waive 1 with the stated reason (verified: --adopt 1 waives the wrong stages), ONE ledger glob mark against the documents index (mark now strips the source-root prefix so 'sources/**' matches inventory rels — verified 200/200 EXTRACTED); Stage 2 = thin BRD transform; Stages 3–4 unchanged and not skippable. css|js|map|woff|… get a chrome route in source-formats.tsv; bin/html-to-md.sh is routed like bug-lookup.sh.
  5. The experiment record — tests/ab/ (answer-key corpus generator + scorer, self-tested) and process/token-path-ab-2026-09-09.md.

Field evidence (process/token-path-ab-2026-09-09.md):

  • Wave 1, Sonnet, stages P–2, synthetic 200-file saved-webpage corpus with an answer key: master $7.94 / 19 min / recall 54%; branch $5.45 / 12.8 min / recall 81% (standard path) and $5.42 / 12.1 min / recall 94% (docs-ready). Rules with thresholds 47% → 87% → 100%; both screenshot-only requirements found on the branch (1/2 on master); ledger PASS in all arms — master needed 159 waivers of 200 files, the branch one glob mark.

  • Wave 2, Haiku, stages P–4, three corpora (synthetic docs, a single-file HTML app, a React/Express/SQL codebase): every branch run reached Stage 4 with real artifacts in 5–8 min for ≤ $1.03. One master run produced CONFIRMED decision rows with no artifacts; its replication did not reproduce it (one-off, not a defect — noted in the report, follow-up filed for the never-advanced PROJECT.md stage header).

  • On a fresh project here: html-to-md.sh 891 KB → 12,325 words (9.0×); source-ledger.sh mark 'sources/**' → 200 rows EXTRACTED, ledger PASS.

  • No client data anywhere in the diff — corpus is fictional and lives only on throwaway ab/arm-* branches; the generator regenerates it byte-identically from the committed key

  • Changed instruments: bash 3.2 portable, stdlib Python, both layouts (source root detection mirrors source-sufficiency.sh); fixtures test-html-to-md.sh 67/0, test-bug12-sync.sh 51/0 (+T11), test-bug03-gates.sh 13/0 (+T10), test-source-ledger.sh 42/0 (+T12), test-stage-p.sh 14/0

  • Golden input: the html-to-md fixture page is synthetic and says so in its CAPTURE.md — the repo's rule prefers a captured export; the first real saved-webpage corpus through this should replace it

  • CHANGELOG.md line per change, in the same commits

  • check-scripts 73/73 + 17/17 · render-routing --check in sync (baseline 74,147 / 80,000) · portability clean · leak guard clean · PR discipline clean

Deliberately not in this PR: the PROJECT.md stage-header fix and the small-project proportionality tier (the report's conclusion 5) — both follow-ups.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

Generated by Claude Code

claude added 10 commits September 9, 2026 10:15
@claude
html-to-md: convert a saved-webpage corpus to Markdown once, with a d…
b087dff 
…ocuments index

bin/html-to-md.sh <project-root> [--source <dir>] [--out <dir>] [--force] and
bin/lib/html-to-md.py (stdlib html.parser only). Every .html/.htm/.xhtml under the
source root becomes analysis/knowledge-base/text/<same path>.md: headings, lists,
tables, links, pre/code, bold/italic kept; script/style/noscript/svg/comments and
nav/sidebar/footer/cookie/breadcrumb chrome dropped, never <main>/<article> or
anything holding an h1-h3. Inline base64 images are decoded to <page>_images/NN.ext;
sidecar images are referenced by their project-relative path. Each .md carries a
header with source, raw size, words, images and the section list with md line
numbers. documents-index.md lists EVERY file under the root (sidecar png/css/js
included) with a role, so one ledger glob mark can point at it.

Field context: a requirements-driven Markdown+HTML project, where reading raw HTML
cost 3-10x the tokens of its text on every session. Fixture ratio on the synthetic
page is 2.2x (a small page; the gain scales with wrapper markup and inline images).

Fixture: tests/wave2/test-html-to-md.sh (67 assertions) over a synthetic
saved-webpage export with a CAPTURE.md explaining why synthetic is acceptable here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
sync-project: retire the always-on bug-logs/mxcli-bugs.md row to bin/…
6d7a926 
…bug-lookup.sh, and say so

Master replaced the 47,500-word ledger's baseline routing row with bin/bug-lookup.sh
on 2026-09-08; projects scaffolded before that still read the whole ledger every
session (measured cause two on a Markdown+HTML requirements project). Section 2d
already re-rendered a MARKED block silently — now the retirement is announced. A
hand-written UNMARKED table got 2d's whole-table refusal and kept the row; new
section 2e rewrites that one row in place, keeping the row's own path prefix and
every neighbouring row byte-identical (routing_row helper in skill-routing.sh).
CLAUDE.md is not edited: its Integration block is bootstrap-project.md's LLM merge,
not init-project.sh's — reported with the exact replacement row instead.

Fixture: tests/wave2/test-bug12-sync.sh T11 (marked, unmarked, --dry-run,
idempotence, CLAUDE.md report); 51/51.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
gate-check: name the runbook lines a stage needs, instead of the whol…
bb06bb8 
…e file

A stage-specific run now prints, near the top of its output:
  Read for this gate: skills/conversion-runbook.md §"Stage N — …" (lines A–B)
                      + §1b Live Checklist (lines C–D) — not the whole file
The spans are derived by grepping the runbook's actual headings on every run
(never hard-coded; keyed on the " — " every stage heading carries so that
"### Stage 0 runs in every entry mode" does not shadow "### Stage 0 — Triage");
build-ready reads Stage 5's section. The runbook's top carries the matching
"How to read this file: not whole" paragraph.

Measured cause three of a slow Markdown+HTML requirements project: the ~11,700
word runbook was read whole every session because CLAUDE.local.md says so.

Fixtures: test-bug03-gates.sh (13/13, new T10), test-source-ledger.sh (36/36),
test-stage-p.sh (14/14).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
runbook: the docs-ready fast path for a text-native requirements corpus
9a48400 
skills/conversion-runbook.md → Entry Modes gains "Requirements-driven, docs-ready
corpus": Stage 0 = source-sufficiency init + CAC-1 + sign-off (triage rows N/A,
text-native corpus); Stage 1 = bin/html-to-md.sh once, the KB is the converted
text + documents-index.md, ONE ledger glob mark pointing at the index, images
still read by vision with the index's count as the denominator; Stage 2 = one
use case per section with a sourceRef, rules lifted as stated, validation once;
Stages 3-4 unchanged and not skippable.

Run against the real gates before it was written; two things had to change:
- --adopt 1 is the wrong instrument (it waives Stage 0, which ran, and leaves
  Stage 1 demanding an extraction report), so the path records --waive 1 with
  the reason. Stage 2 still runs the ledger (index hidden → exit 1, verified).
- source-ledger.sh mark strips the source-root segment from a target, so
  'sources/**' matches the corpus instead of being refused.
Verified: a glob mark whose artifact names every basename → EXTRACTED for every
row including css/js/png, and image rows owe no --media.

Also: source-formats.tsv route `chrome` for css|map|woff|woff2|ttf|otf|eot|ico
(.js stays on the code route on purpose); bin/html-to-md.sh routed baseline
(ba, stages 0-1) and the surfaces re-rendered (74,147 of 80,000 words).

Fixture: tests/wave2/test-source-ledger.sh T12 (42/42).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
Record the token-path A/B: fixture generator, scorer, answer keys, an…
2f4dd7b 
…d the report

Wave 1 (Sonnet, P-2) and wave 2 (Haiku, P-4, three corpora) with the
per-stage mandated reading load, the non-reproducing hollow run, and the
proportionality finding. The synthetic corpus itself stays on the throwaway
ab/arm-* branches; the generator regenerates it byte-identically.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
Merge remote-tracking branch 'origin/master' into claude/ab-token-path
a7def7a 
# Conflicts:
#	CHANGELOG.md
@claude
test-bug12-sync: print the table and sync output when the retire-row …
a2c5581 
…assertions fail

CI fails 'table disturbed' and 'second run rewrote CLAUDE.local.md' on a
head that passes locally as root, as nobody, shallow, and through run-all.sh.
Show the file so the runner's difference is visible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
test-bug12-sync: bad() prints its detail argument (it was dropped), p…
95e5ff6 
…refixed so run-all shows it

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
sync-project: no backslashes in the awk -v regex (gawk rewrites them)…
10f0b66 
…; check-portability flags the pattern

Under gawk, the GitHub runner's awk, -v re="^\\|..." became "^|..." — an
alternation matching every line — and the retire-ledger-row rewrite replaced
the whole CLAUDE.local.md with copies of one row. mawk keeps the backslash,
which is why it passed everywhere locally. Bracket expressions carry none.
Verified: test-bug12-sync 51/0 under both mawk and gawk.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
check-portability: the awk -v backslash rule ignores $(...) substitut…
f887fe7 
…ions

bin/lib/skill-routing.sh passes -v ok="$(... | tr '\\n' ' ')" — the backslash
is the shell's, the value carries none. Strip substitutions before looking.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau
MendixMau merged commit 9337dec into master Sep 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MendixMau @claude