Skip to content

[CuTeDSL] Let cute.compile opt into the compile cache - #3402

Open
aryanputta wants to merge 1 commit into
NVIDIA:mainfrom
aryanputta:fix/cute-compile-cache-optin
Open

[CuTeDSL] Let cute.compile opt into the compile cache#3402
aryanputta wants to merge 1 commit into
NVIDIA:mainfrom
aryanputta:fix/cute-compile-cache-optin

Conversation

@aryanputta

Copy link
Copy Markdown

Summary

Addresses #3398: cute.compile never engages the CuTeDSL content-addressed compile cache.

As noted in the issue thread, there are two independent gates, not one, so the single-line change described in the issue would not have fixed it:

  1. base_dsl/compiler.py, CompileCallable._compile, set kwargs["no_cache"] = True unconditionally, overriding anything the caller passed.
  2. base_dsl/dsl.py, BaseDSL._setup_common, re-derived it independently:
if not no_cache and compile_only:
    no_cache = True
    self.print_warning("Cache is disabled as user wants to compile only.")

Because _compile also always sets compile_only=True, removing gate 1 alone leaves every cute.compile in that branch with the cache still disabled.

Approach

I deliberately did not change what compile-only does by default, because whether compile-only results are safe to cache is your call, not mine. My open question on the issue still stands: compile_only appears to only bypass execution (dsl.py:2333-2334 returns jit_function instead of calling run_compiled_program), and get_module_hash keys on module bytecode plus envars plus compile options, which suggests caching would be sound. But compile-only also relaxes argument handling (dsl.py:1286-1298, type-only placeholders with no execution args) while the cached function carries dynamic_args / dynamic_kwargs alongside the IR, so I could not rule out that the gate is a deliberate guard against those two build modes aliasing on one hash.

So this PR adds an opt-in instead, which is correct under either answer:

  • _compile uses kwargs.setdefault("no_cache", True), so it no longer clobbers a caller-supplied value.
  • _setup_common distinguishes "caller passed nothing" from an explicit value and applies the compile-only rule only in the former case.

A caller who wants caching now writes cute.compile(fn, ..., no_cache=False).

If you confirm the gate is merely conservative, dropping the compile-only implication entirely becomes a two-line follow-up on top of this, and I am happy to do it.

Compatibility

Default behavior is unchanged. cute.compile with no no_cache argument still bypasses the cache and still prints the same warning. No in-tree caller passes no_cache explicitly (checked across python/CuTeDSL and examples/python/CuTeDSL), so nothing in the repo changes behavior. The keep_ptx / keep_cubin / keep_sass rule immediately above is intentionally left as an unconditional override, since dumping artifacts should always bypass the cache.

Testing

I do not have an NVIDIA GPU, so I have not exercised this at runtime, and I could not find a non-GPU unit-test harness for the compile/cache plumbing under test/python/CuTeDSL to add coverage to. Everything above is from reading the call paths on main. The change is confined to argument defaulting, and the untouched default path is byte-identical, but a runtime confirmation that no_cache=False produces a second-process cache hit would be worth having before merge. Happy to add a test if you can point me at the right harness.

This change was developed with an AI coding assistant.

`cute.compile` could never use the content-addressed compile cache, so every
process re-ran the full MLIR build even when a byte-identical artifact was
already on disk.

Two independent gates caused this, not one:

1. `CompileCallable._compile` set `kwargs["no_cache"] = True` unconditionally,
   overriding anything the caller passed.
2. `BaseDSL._setup_common` re-derived it: `if not no_cache and compile_only:
   no_cache = True`. Since `_compile` also always sets `compile_only=True`,
   removing only gate 1 would leave the cache disabled.

This keeps the existing default and adds an opt-in rather than changing what
compile-only does implicitly. `_compile` now uses `setdefault`, and
`_setup_common` distinguishes "caller passed nothing" from an explicit value,
applying the compile-only rule only in the former case. A caller that wants
caching can now ask for it with `cute.compile(fn, ..., no_cache=False)`.

Default behavior is unchanged: `cute.compile` with no `no_cache` argument still
bypasses the cache and still prints the existing warning. No in-tree caller
passes `no_cache` explicitly, so nothing else changes. The `keep_ptx` /
`keep_cubin` / `keep_sass` rule above is deliberately left as an unconditional
override, since dumping artifacts should always bypass the cache.

Addresses NVIDIA#3398.

Signed-off-by: Aryan Putta <aryansputta@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant