A python3 that exists and is not an interpreter said the engine was broken - #51
A python3 that exists and is not an interpreter said the engine was broken#51chongjiazhen wants to merge 1 commit into
Conversation
…roken The guards in tests/run.sh probe with `command -v python3`, which was written for an absent interpreter. On Windows the name on PATH is usually the Microsoft Store App Execution Alias: a zero-byte reparse point that `command -v` finds, that exits 49, and that prints an advert for the Store instead of running anything. Present, and not an interpreter, so the guard never fired and all 39 call sites ran against it. On a stock Windows 10 box (MinGW-w64 GCC 15.2, Python 3.13 installed and working under its own name) that reported 7 passed, 7 failed, 16 skipped. The failures read "range server did not start", "convert.py resume" and "serve suite": the engine called broken when what is wrong is a shim on PATH, which is what line 10 says this suite must never do. The 16 skips were the same cause one step earlier, since make_test_container.py is python3 too, so the synthetic container never built and every check needing one skipped for want of a container. Probe by running it. Windows installs the versionless `python`, and `py` besides, so a working interpreter is normally there under another name; shim the first one that answers into PATH ahead of the synthetic-container build, which reaches the call sites here and the subprocesses (check_budget.sh, the serve suite) without editing either. Same box now runs 44 passed, 0 failed, 13 skipped. The trace-simulator guard moves to PY_MISS with the rest. It is the one that would still have produced a false failure after this fix: with a real container it takes the `command -v` branch, the simulator returns nothing, and the suite reports "trace simulator disagrees with the engine" against an interpreter that never ran. With no working interpreter under any name, PY_MISS carries the reason and the checks SKIP loudly rather than failing: verified by shimming all three names to exit 49, which gives 7 passed, 0 failed, 24 skipped. Assisted by AI.
8ccad61 to
5131175
Compare
|
Context I should have put in the description: this repo has had two thorough It is the same kind as #36's gap 1, though, which you accepted in exactly Worth knowing for whoever merges: #48 also touches |
Follow-up to the suite fix in sqliteai#51, same finding one layer out: on Windows `python3` on PATH is usually the Microsoft Store App Execution Alias, a zero-byte reparse point that exists, exits 49, and prints an advert instead of running anything. run.sh can shim PATH for its own call sites, but that does not reach a recipe make runs itself, nor either tools script when run directly. Eleven call sites: three recipes, four in fetch_weights.sh, four in pipeline.sh. Makefile: serve-check, fuzz and fuzz-asan get $(PY). fetch_weights.sh and pipeline.sh get the same answer as $PY; pipeline.sh exports it so the fetch_weights.sh it drives inherits rather than resolving per stage. $(PY) is recursive rather than `:=` on purpose. Immediate assignment runs the probe on every make invocation, `make clean` and a no-op build included, and where python3 is the alias the first spawn is the alias itself: measured 196 ms per invocation on this box, which nearly triples a no-op make. Lazy, the probe runs only when a Python recipe expands it, and a no-op make is back to 0.664 s against 0.678 s on main. The fallback sits inside the shell for the same reason, since an ifeq on $(PY) would force the expansion at parse time. pipeline.sh is the one that misdiagnoses rather than merely failing: its first two probes end `2>/dev/null || echo 0`, so an interpreter that is not one yields 0 and the script dies "download (no index at $SRC)", naming a missing index that is present and readable. Nothing skips here. If no candidate answers, PY stays python3 and the recipe or script fails loudly at first use, which is right for a build target and for a 1.4 TB download: unlike the suite there is nothing to skip, and a run that cannot read its own index must stop. Verified on Windows 10, MinGW-w64 GCC 15.2. PY resolves to `python`; `make serve-check` reports OK (skipped=3) and `make fuzz FUZZ_RUNS=20` reports 20 cases, 0 crashed, 0 hung, both exit 0; with all three names shimmed to exit 49, serve-check fails loudly with Error 49 rather than skipping. The two tools scripts are not run end to end here, since that needs the full source weights: they carry bash -n plus a three-state check of the resolver. Assisted by AI.
tests/run.shguards its python3-backed checks withcommand -v python3, whichwas written for an absent interpreter. On Windows the name on PATH is usually
the Microsoft Store App Execution Alias: a zero-byte reparse point
(
AppInstallerPythonRedirector.exe) thatcommand -vfinds happily, that exits49, and that prints an advert for the Store instead of running anything.
Present, and not an interpreter. So the guard never fires and all 39 call
sites run against it.
This is #42's finding arriving through the same door wearing the opposite
disguise, and
run_uvalready carries the mental model one tool over: a uv thatis present and cannot work does not fail, it hangs.
What it looks like on a stock Windows dev box
Windows 10, MinGW-w64 GCC 15.2, Python 3.13 installed and working under its own
name,
make CC=gcc check:The failures read
range server did not start,convert.py resume,convert.py chat.json,serve suite. That is the engine called broken whenwhat is wrong is a shim on PATH, which is the one thing line 10 says this suite
must never do.
The 16 skips are the same cause one step earlier:
make_test_container.pyispython3 too, so the synthetic container never built,
SYNTHETICstayed 0, andevery check needing a container skipped for want of one. That is why this is not
a 7-check fix.
The trace-simulator guard is the one worth calling out: it takes the
command -vbranch with a real container present, the simulator then returnsnothing, and the suite reports
trace simulator disagrees with the engineagainst an interpreter that never ran. It moves to
PY_MISSwith the rest.The change
Probe by running it rather than by looking it up. Windows installs the
versionless
python, andpybesides, so a working interpreter is normallysitting right there under another name: shim the first one that answers into
PATH. Placed ahead of the synthetic-container build, so that build gets it too,
and it reaches the subprocesses (
check_budget.sh, the serve suite) withoutediting either or touching the 39 call sites.
If nothing answers under any name,
PY_MISScarries the reason and the checksSKIP loudly, which is what the
command -vguard always meant to do.Same box, after:
Verification
Three states, all on the box described above:
python3on PATH (manual shim, as a baseline)The third row is the one that matters for the guard: the eight checks skip with
no working python3 (the name on PATH is not an interpreter)rather thanfailing.
The first two rows agreeing is the point of the second: the patch reaches the
same state the environment fix would, so nothing is being papered over.
Not in this diff
Makefile,tools/fetch_weights.shandtools/pipeline.shcarry 13 more barepython3call sites with the same exposure. They are release and fetch toolingrather than the suite, and PATH set inside
run.shdoes not reach a recipe thatmakeruns itself, so they want a separate decision about whether theresolution belongs in the Makefile. Happy to do that as a follow-up if you want
it, in whichever shape you prefer.
Also worth knowing for anyone cloning on Windows: with git's default
core.autocrlf=truethe whole worktree smudges to CRLF, and a patch then diffsas a whole-file rewrite. Set
core.autocrlf=falsebefore cloning. Same familyas #47, one layer out, and not something a code change here can fix.
Assisted by AI.