Skip to content

fix: address cold review findings in job-assessment, patterns and top-level docs - #29

Merged
darthrootbeer merged 4 commits into
mainfrom
fix/cold-review-job-assessment-and-top
Oct 2, 2026
Merged

darthrootbeer merged 4 commits into
mainfrom
fix/cold-review-job-assessment-and-top

Conversation

@darthrootbeer

Copy link
Copy Markdown
Owner

Context

An independent cold review of the repo found stale numbers, an edit to saved model runs that introduced a wrong figure, a prompt that named its own test answers, and several overstated claims. This PR fixes everything in the review outside the docs pipeline and the readability skill, which another PR covers. Ticket: JBSR-099.

Changes

  • Prompts 03, 04 and 06 were re-run on Sonnet and Haiku against the fixtures as they ship, and the saved records now hold real model answers with no edited figures. The wrong $78,000 target is gone, and the same slip in two older records is corrected.
  • Prompt 05 no longer names the kinds of gap to look for. A second, held-out profile with six gaps the prompt does not mention was added, with tests, and both profiles were run. Sonnet found 3 of 3 and 6 of 6. Haiku was PARTIAL on both (it found about half of the first and 5 of 6 on the second). The published Haiku results in the READMEs now say so.
  • GRADES.md and the job-assessment README now say who graded (a Claude model, Opus first, Sonnet 5.5 for the re-runs), that each prompt ran once per model, and that the earlier runs predate the pay change. The Verified block has real numbers from a fresh clone (542 passed, 2 skipped). The 'How it was built' section has a timeline paragraph.
  • Top README: names the two real skills, adds a short tour, keeps the docs pipeline as item 1, and says where the prompt-block rule applies. ROADMAP has a Done section and no longer has the linking note.
  • Memory-limit claims in the typed-memory pattern now say what was observed and in which Claude Code version, and say topic files may be read on demand. The three patterns say their prompt checks have no saved records.
  • Small fixes: .gitignore, the CI step no longer accepts a run with no tests, minimum versions in requirements, a broken example path in the hook README, and file links in two saved records wrapped as code.

Review notes: no git history was touched. The docs pipeline files and the readability skill are untouched.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XV3Ggh86cf2xwUyz28XaN6

  • Docs updated

@darthrootbeer
darthrootbeer merged commit 6647fa4 into main Oct 2, 2026
6 checks passed
@darthrootbeer
darthrootbeer deleted the fix/cold-review-job-assessment-and-top branch October 2, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant