Skip to content

feat(compile-opt): CastFoldingPass folds casts that change no value (#1494) - #1496

Merged
michalharakal merged 1 commit into
developfrom
feature/1494-cast-folding-pass
Nov 10, 2026
Merged

michalharakal merged 1 commit into
developfrom
feature/1494-cast-folding-pass

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Closes #1494. Resolves Open Question 2 of SKEEP-008.

Problem

Two adjacent mixed-precision boundaries each cast back to the network dtype, so the tensor between them is widened only to be narrowed again. A BF16 dense followed by an FP16 stage exported:

%v13 = stablehlo.convert %v12 : (tensor<1x3xbf16>) -> tensor<1x3xf32>
%v14 = stablehlo.convert %v13 : (tensor<1x3xf32>) -> tensor<1x3xf16>

Change

CastFoldingPass (skainet-compile-opt) applies two value-preserving rewrites:

  • A cast after an exact widening (FP16 or BF16 to FP32, any of them to FP64) reads from the original tensor. bf16 -> f32 -> f16 becomes bf16 -> f16. The widening is removed once nothing else reads it; if it has another consumer it stays and only the second cast is rewired.
  • A cast to the dtype its input already has is removed. The first rewrite leaves this behind for f16 -> f32 -> f16, and DtypeForwardPropagationPass(targetFloatDtype = …) can produce it.

Never folded: the narrowing round trip f32 -> bf16 -> f32 (the rounding the layer asked for), narrow-to-narrow casts, and integer casts (Int32 to FP32 is not exact).

The pass has no options, as assessed in the issue: the exactness rule follows from the dtypes, and per-target variation already has TargetOptimizers.registerDagPasses. It runs right after DTypeConstraintResolutionPass in the default, aggressive and LLM pipelines of GraphOptimizationPipeline.

Beyond the issue

ComputeGraphExecutor could not run convert nodes, so "bit-identical before and after" could not be tested by execution. It now dispatches convert / cast / to, taking the target from the node's output dtype.

Tests

  • CastFoldingPassTest (12): each rule, the adjacent-boundary chain, a widening with a second consumer, cast aliases and dtype spellings, diagnostics, an untouched graph returned as the same instance.
  • MixedPrecisionExportTest (+2): the traced network exports three stablehlo.convert instead of four; the folded and unfolded cast chain executed on the CPU backend give identical raw bits.

Review notes

Test plan

  • ./gradlew apiCheck checkUnmarkedStubs
  • jvmTest for skainet-compile-opt, skainet-compile-hlo, skainet-compile-dag
  • linuxX64 compile of skainet-compile-opt and skainet-compile-dag
  • Native and wasm test targets (CI; not run locally)

🤖 Generated with Claude Code

…1494)

Two adjacent mixed-precision boundaries each cast back to the network
dtype, so the tensor between them is widened only to be narrowed again:
bf16 -> f32 -> f16. The pass applies two value-preserving rewrites:

- a cast that follows an exact widening (FP16 or BF16 to FP32, any of
  them to FP64) reads from the original tensor; the widening is removed
  once nothing else reads it;
- a cast to the dtype its input already has is removed, which the first
  rewrite leaves behind for f16 -> f32 -> f16 and which
  DtypeForwardPropagationPass can produce.

A narrowing round trip (f32 -> bf16 -> f32) is never folded: it is the
rounding the layer asked for. Narrow-to-narrow and integer casts are not
treated as exact. The pass has no options and runs after dtype constraint
resolution in the default, aggressive and LLM pipelines.

ComputeGraphExecutor dispatches convert / cast / to, taking the target
from the node's output dtype, so a graph with casts can be executed. The
new test runs the folded and the unfolded cast chain on the CPU backend
and compares raw bits.

CastFoldingPassTest covers each rule, a widening with a second consumer,
cast aliases and dtype spellings. MixedPrecisionExportTest pins three
stablehlo.convert instead of four for a BF16 dense followed by an FP16
stage. SKEEP-008 Open Question 2 is resolved.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1496 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

@michalharakal
michalharakal merged commit 851f94f into develop Nov 10, 2026
19 checks passed
@michalharakal michalharakal mentioned this pull request Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Fold cast pairs between adjacent mixed-precision boundaries (SKEEP-008 follow-up)

1 participant