[CuTe DSL] Raise a clear error when jit source is stale instead of silently misbehaving, issue #3395 - #3396
Open
yunweili3 wants to merge 1 commit into
Open
[CuTe DSL] Raise a clear error when jit source is stale instead of silently misbehaving, issue #3395#3396yunweili3 wants to merge 1 commit into
yunweili3 wants to merge 1 commit into
Conversation
…lently misbehaving
AST preprocessing is lazy (it runs on the first call) and re-reads the function
source from disk via inspect.getsourcelines at the in-memory code object's
co_firstlineno. If the source file was modified after the module was imported --
e.g. the package was upgraded in place under a long-lived process -- the
extracted slice is misaligned and belongs to a different statement or function.
Today that fails in confusing, context-dependent ways:
- the decorator check silently returns [] and the function runs unpreprocessed,
so its first dynamic `if` raises PHASE_DYNAMIC_TO_STATIC_BOOL ("Unable to
convert dynamic Boolean value to bool" in older wording);
- for top-level functions, exec resolves the name back to the original jit
wrapper and code replacement dies with "requires a code object with 0 free
vars, not 3";
- worst case, a slice landing on a different same-named decorated function is
transformed and silently swapped into this function's __code__ with no error.
Guard transform_function after parsing: if the extracted slice does not define
the expected function (wrong node type or name), or a same-named def declares a
different signature (parameters cross-checked against co_varnames/argcounts),
raise a DSLRuntimeError that names the mismatch and points at the stale-source
cause with a restart/reload suggestion. Lambdas are exempt (no retrievable def
block; already skipped by the decorator check). Also extend the parse-failure
suggestion to mention in-place source modification alongside REPL mode.
Verified with GPU-independent repros on CPython 3.10/3.12 against dsl 4.6.1 and
4.6.0.dev0, and for no-behavior-change on consistent sources against
flash-attention's SM100 forward kernels (bf16/fp8 compile and numerics
identical). Found while debugging Dao-AILab/flash-attention#2716.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem of #3395
@cute.jitAST preprocessing is lazy: it runs at the first call/compile of each function, and at that point it re-reads the function source from disk viainspect.getsourcelines, slicing the current file at the in-memory code object'sco_firstlineno(base_dsl/ast_preprocessor.py::transform_function).If the source file was modified after the module was imported — typically a package upgraded in place while a long-lived process (serving worker, orchestrator, notebook kernel) still holds the old modules — the extracted slice is misaligned, and there is no consistency check. What happens next depends on what text sits at the stale line number:
check_decoratorreturns False,transform_functionsilently returns[],run_preprocessorreturnsNone, and_preprocess_and_replace_codesilently skips the code replacement. The kernel then runs as plain Python with no control-flow staging, and its first dynamicifraises the confusingerror[PHASE_DYNAMIC_TO_STATIC_BOOL]: Cannot use a Runtime value (Staged value) value where a Python value (Meta value) boolean is required("Unable to convert dynamicBooleanvalue to bool" in older wording). Class methods hit this path.ValueError: f() requires a code object with 0 free vars, not 3.__call__methods of sibling classes in one file), the preprocessor transforms that function,execbinds it under the bare name, and the original function's__code__is replaced with the wrong function's body — no error at all.None of these point at the actual cause. Found while debugging Dao-AILab/flash-attention#2716, where attribution took ~20 experiment rounds across two engineers. Repros are GPU-independent and hit
nvidia-cutlass-dsl4.6.1 and 4.6.0.dev0 alike, on CPython 3.10 and 3.12.Fix
Guard
transform_functionright after parsing the extracted slice:If the slice is not a
FunctionDefnamedfunc_name, raise aDSLRuntimeErrornaming the mismatch and the likely cause, with a restart/reload suggestion:<lambda>is exempt (no retrievabledefblock; already skipped by the decorator check).If the name matches, cross-check the declared parameter list against the code object (
co_varnames/co_argcount/co_kwonlyargcount/ vararg / kwarg flags) — a stale offset can land on a same-named method of another class with a different signature.Extend the parse-failure suggestion (previously REPL-only) to also mention in-place source modification.
No behavior change for consistent sources; single-file change, +58/−1.
Validation
flash_fwd_sm100.pyin place → first trace): previouslyUnable to convert dynamic Boolean value to bool at compile time, now the clear error quoted above.Known residual (for discussion)
A same-name and same-signature swap (mode 3 with twin methods) is not detectable from the slice alone. We prototyped recompiling the slice and comparing code objects, but CPython emits different bytecode for identical source depending on compile context (class body vs wrapped block — the method-call specialization bits in
LOAD_GLOBAL/LOAD_ATTRopargs differ), so bytecode equality is not a sound invariant; it false-positived on real kernels and was dropped. The complete fix would capture the source (or a hash of it) eagerly at decoration time — happy to explore that in a follow-up if maintainers prefer.f = cute.jit(g)(non-@usage) silently skips preprocessing today and is unchanged by this PR.