[CuTeDSL] Let cute.compile opt into the compile cache - #3402
Open
aryanputta wants to merge 1 commit into
Open
Conversation
`cute.compile` could never use the content-addressed compile cache, so every process re-ran the full MLIR build even when a byte-identical artifact was already on disk. Two independent gates caused this, not one: 1. `CompileCallable._compile` set `kwargs["no_cache"] = True` unconditionally, overriding anything the caller passed. 2. `BaseDSL._setup_common` re-derived it: `if not no_cache and compile_only: no_cache = True`. Since `_compile` also always sets `compile_only=True`, removing only gate 1 would leave the cache disabled. This keeps the existing default and adds an opt-in rather than changing what compile-only does implicitly. `_compile` now uses `setdefault`, and `_setup_common` distinguishes "caller passed nothing" from an explicit value, applying the compile-only rule only in the former case. A caller that wants caching can now ask for it with `cute.compile(fn, ..., no_cache=False)`. Default behavior is unchanged: `cute.compile` with no `no_cache` argument still bypasses the cache and still prints the existing warning. No in-tree caller passes `no_cache` explicitly, so nothing else changes. The `keep_ptx` / `keep_cubin` / `keep_sass` rule above is deliberately left as an unconditional override, since dumping artifacts should always bypass the cache. Addresses NVIDIA#3398. Signed-off-by: Aryan Putta <aryansputta@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses #3398:
cute.compilenever engages the CuTeDSL content-addressed compile cache.As noted in the issue thread, there are two independent gates, not one, so the single-line change described in the issue would not have fixed it:
base_dsl/compiler.py,CompileCallable._compile, setkwargs["no_cache"] = Trueunconditionally, overriding anything the caller passed.base_dsl/dsl.py,BaseDSL._setup_common, re-derived it independently:Because
_compilealso always setscompile_only=True, removing gate 1 alone leaves everycute.compilein that branch with the cache still disabled.Approach
I deliberately did not change what compile-only does by default, because whether compile-only results are safe to cache is your call, not mine. My open question on the issue still stands:
compile_onlyappears to only bypass execution (dsl.py:2333-2334returnsjit_functioninstead of callingrun_compiled_program), andget_module_hashkeys on module bytecode plus envars plus compile options, which suggests caching would be sound. But compile-only also relaxes argument handling (dsl.py:1286-1298, type-only placeholders with no execution args) while the cached function carriesdynamic_args/dynamic_kwargsalongside the IR, so I could not rule out that the gate is a deliberate guard against those two build modes aliasing on one hash.So this PR adds an opt-in instead, which is correct under either answer:
_compileuseskwargs.setdefault("no_cache", True), so it no longer clobbers a caller-supplied value._setup_commondistinguishes "caller passed nothing" from an explicit value and applies the compile-only rule only in the former case.A caller who wants caching now writes
cute.compile(fn, ..., no_cache=False).If you confirm the gate is merely conservative, dropping the compile-only implication entirely becomes a two-line follow-up on top of this, and I am happy to do it.
Compatibility
Default behavior is unchanged.
cute.compilewith nono_cacheargument still bypasses the cache and still prints the same warning. No in-tree caller passesno_cacheexplicitly (checked acrosspython/CuTeDSLandexamples/python/CuTeDSL), so nothing in the repo changes behavior. Thekeep_ptx/keep_cubin/keep_sassrule immediately above is intentionally left as an unconditional override, since dumping artifacts should always bypass the cache.Testing
I do not have an NVIDIA GPU, so I have not exercised this at runtime, and I could not find a non-GPU unit-test harness for the compile/cache plumbing under
test/python/CuTeDSLto add coverage to. Everything above is from reading the call paths onmain. The change is confined to argument defaulting, and the untouched default path is byte-identical, but a runtime confirmation thatno_cache=Falseproduces a second-process cache hit would be worth having before merge. Happy to add a test if you can point me at the right harness.This change was developed with an AI coding assistant.