Repository navigation
V71: what stood in for a missing cell is counted - #180
Merged
Merged
Conversation
A missing or unreadable number has always become the training median, and a missing category the most frequent one, with no count anywhere. On titanic deck is 77 % empty, under the 95 % that sets a column aside: it trained, three quarters of it a value nobody recorded, and neither the run, the report nor the model card said so. The encoder now counts what it fills, per column, empty cells apart from unreadable ones. The count travels with the run's summary and is said under the leaderboard, in the report and in the model card; a scored file says the same for its own rows. Nothing about the filling changes and no figure of any run moves. Measured first: keeping deck, dropping it or recoding it as known or unknown gives 0.813, 0.818 and 0.817 of mean accuracy over ten seeds, inside the noise. So the mark past one half is a reading rule, named as one, not a threshold drawn from scores. Docs in both languages, README, PLAN row V71, version 1.71.0. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AyBsUet8WybTG986ubroar
dapiced
marked this pull request as ready for review
November 19, 2026 23:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
G3. A missing or unreadable number has always become the training median, and a missing category the most frequent one; a missing text is left with no word. It was done with no count anywhere.
On titanic
deckis 77 % empty, under the 95 % that sets a column aside: it trained, three quarters of it a value nobody recorded, and neither the run, the report nor the model card said so.The encoder now counts what it fills, per column, empty cells apart from unreadable ones.
train/pipeline.ts:countFilled, over the rows of a run or of a scored file.train/filled.ts: the sentence, andreadFilled, a validator, since a summary can arrive through a share link.TrainSummary.filled: the count travels with the saved run and the share link.FilledNotice), in the HTML report, in the model card (whose limits name the columns filled past one half), and for a scored file inBatchScorePanel.Nothing about the filling changes and no figure of any run moves: every existing test passes untouched.
Measured first
On the six demos: titanic fills
deckon 688 rows of 891 (77 %),ageon 177 (20 %),embarkedandembark_townon 2; mpg fillshorsepoweron 6 of 398; the four others fill nothing.What a mostly filled column is worth, titanic, ten seeds, eight families:
deckHalf a point at most, inside the noise. The plan promised a « measured alert threshold »: it does not exist, since no score separates a harmless share from a harmful one. The mark kept is a reading rule and is called one: past one half, most of what the models read in the column is its fill value, and its importance is that of a mostly constant column.
Defects of this wave's own making
Two of the new tests expected the wrong thing and were right to fail:
n/ais a marker of a missing value, so it counts as empty and not as unreadable (the documentation now says so); and on a tie the most frequent category is the first in alphabetical order.Not in this wave
Another way to fill a cell (model-based imputation was refused twice); a « was missing » column, which the measure did not justify; setting aside a column because it is mostly filled; the Data Studio's recipe (G10).
Docs
ML reference (new section), limits, « score a batch », in both languages; README; PLAN row V71; changelog; version 1.71.0. Seven proposals remain in PLAN.md (G4 to G10).
Checks
xlsxtypes unreachable in my sandbox), format: clean.spreadsheet.test.ts(thexlsxtarball cannot be fetched in my sandbox; CI is the reference). 11 new tests.lab.spec.ts:31, the Excel upload, same cause). Newe2e/filled.spec.tson titanic.mainafter V70 merged (one commit, no conflict); the full suites ran before the rebase, the touched tests and the typecheck again after it.Generated by Claude Code