Cache load_inline Torch headers in a Modal Volume

[msaroufim]

Reuse Torch C++ precompiled headers across Modal submissions. A CPU warmup writes a Volume; GPU runners mount it read-only and disable Modal API writes. The deployment workflow warms the cache before replacing runners.

Submissions are expected to use load_inline(), which uses the cache automatically; leave use_pch=False. Keys include installed headers, compiler, host flags, and include environment. Misses compile normally. CUDA templates and submission objects are not cached.

Measured on T4, CUDA 13.3, PyTorch 2.12, with fresh containers and normal Ninja parallelism:

Build	Direct GCC	Cached
CUTLASS GEMM, default CUDA headers	48.08s	45.61s
CUTLASS GEMM, no_implicit_headers=True	19.95s	12.58s
Minimal CUDA vector add	18.85s	10.00s

The GEMM instantiates two SM75 tile configurations with FP16 inputs and FP32 accumulation/output. PCH saved 37% of build time with minimal CUDA headers, but only 5% with default headers (overlapping timing ranges). NVCC remains the bottleneck. All 160 GEMM checks passed; maximum absolute error was 8.53e-6.

Unwarmed host flags miss the cache: a load_inline() case measured 18.08s → 20.30s. Separate-container variability means this difference does not isolate wrapper overhead. CXX=/usr/bin/g++ bypasses the wrapper.

Validation: the implementation passed all four GitHub checks, 41 local tests, 76 additional correctness checks (including supplemental compiler diagnostics), and all eight warmup profiles. The latest change only updates documentation; Ruff and diff checks pass. Separate untimed GCC traces confirm PCH consumption.

Usage and reproduction · Per-container results and Modal runs

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/libkernelbot
utils.py
Project Total

This report was generated by python-coverage-comment-action