Experiment: persistent evaluator for inline CUDA and Triton

[msaroufim]

KernelBot starts fresh Python processes for submission import and evaluation even inside a warm Modal function. This draft adds a reproducible experiment comparing that path with one persistent Python/CUDA worker behind run_program. It supports the bundled inline-CUDA and Triton examples and preserves the existing correctness and timing function bodies.

Inline CUDA saved 19–28% of total evaluation time with cold compilation caches. Four-submission batch results on T4:

Run	Fresh	Persistent	Less wall time
Two counterbalanced rounds, first allocation	173.8 s	125.4 s	27.8%
Complete repeat, second allocation	151.1 s	122.2 s	19.1%

In the complete repeat, median compile/import time was 31.74 → 29.85 seconds per submission, while the benchmark request dropped from 5.89 → 0.41 seconds. Compilation dominates the remaining time.

Kernel-score equivalence remains unresolved: reported kernel medians were 12–15% higher in persistent mode in the complete repeat. That ordered run does not isolate process state from clock/thermal drift or measurement variability. This needs investigation before ranked use.

Validation:

• All 24 timed submissions passed across the two experiments.
• Correct addition → incorrect subtraction → restored addition produced pass → fail → pass in the same process using the same requested extension name. Every request produced a native library in a private cold cache.
• Repository-wide Ruff and three CPU regression tests pass. The result audit checks timing statistics, compilation, native libraries, and numerical rejection of the wrong kernel.
• A local connection failure interrupted the first run's final reload check. All first-run timing totals were recovered and are explicitly marked as limited-detail evidence. The complete repeat finished successfully; batch checkpoints now preserve completed measurements.

The production runner is unchanged. This is a trusted-example, single-GPU benchmark prototype without arbitrary-code isolation or production timeout/CUDA-fault recovery. Both experiment sandboxes are stopped.

Measured report and evidence · Reproduction and request flow

Draft for discussion and iteration. Do not merge.

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/libkernelbot
utils.py
Project Total

This report was generated by python-coverage-comment-action