Repository navigation
V66: the glass box, one row replayed through the model - #166
Merged
Merged
Conversation
Beside the Shapley explanation (what weighed), the what-if panel can now replay its row through the inspected model: the questions a decision tree asked and the training rows that share its leaf, the count of a forest's trees, the five rows of the file a k-NN compared it to, the prediction a boosting builds round after round with its error on the validation rows. The replay walks the model itself. Measured first on 2,560 test rows of the six demos: its last step equals predict on every row for the tree, the k-NN and the boosting; the forest missed three rows of titanic until its tie rule was copied. Questions are written in the units of the file. The measure also showed that the boosting's validation error is lowest well before its last round on 17 runs of 30. It is shown and sets nothing; early stopping is proposed as F9. New how-to page in both languages, limits, reference, README, PLAN row V66, version 1.66.0. F8 is complete. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AyBsUet8WybTG986ubroar
dapiced
marked this pull request as ready for review
November 13, 2026 18:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The third and last part of F8. For a row typed in the what-if panel the lab gave a prediction and a Shapley explanation: what weighed. A new Replay it step by step button tells what happened, for four families:
Other families say why there is nothing to replay.
train/glass-box.ts(pure):replayTree,replayForest,replayNeighbours,replayBoosting,stagedValidation,describeSplit, andreplayRow, the worker entry.gbdt.tsexportsbinValueand its node type;trainKnngains aneighboursmember off the hot path;TrainArtifacts.fitIndicesnames the rows a capped family was fitted on. No training figure changes.replay, responsereplay-result(run-derived, cleared with each new what-if row, saved nowhere).components/insights/GlassBoxView.tsx, rendered insideWhatIfPanel.Measured before building
One rule: the replay is the model, so its last step must equal
predict. Six demos, five seeds, 2,560 test rows:That last finding is shown in the panel, on validation rows and never on the test set, and sets nothing. Early stopping is proposed as F9 in PLAN.md, to be measured on its own.
Not in this wave
Replaying how a model was trained; a network, a linear model, a blend, or a forest tree by tree; stopping the boosting early; a re-imported model; drawing a whole tree.
Docs
New how-to page « replay a prediction » in both languages, limits, ML reference, link from « read a learning curve », README, PLAN row V66 (F8 removed as complete, F9 added), changelog, version 1.66.0.
Checks
xlsxtypes unreachable in my sandbox), format: clean.spreadsheet.test.ts(thexlsxtarball cannot be fetched in my sandbox; CI is the reference). 43 new tests, among them the invariant on every test row for each family and task.lab.spec.ts:31, the Excel upload, same cause). Newe2e/glass-box.spec.tsreplays a row through each family and checks the last step against the what-if prediction.Generated by Claude Code