Repository navigation
feat(compile-opt): CastFoldingPass folds casts that change no value (#1494) - #1496
Merged
Merged
Conversation
…1494) Two adjacent mixed-precision boundaries each cast back to the network dtype, so the tensor between them is widened only to be narrowed again: bf16 -> f32 -> f16. The pass applies two value-preserving rewrites: - a cast that follows an exact widening (FP16 or BF16 to FP32, any of them to FP64) reads from the original tensor; the widening is removed once nothing else reads it; - a cast to the dtype its input already has is removed, which the first rewrite leaves behind for f16 -> f32 -> f16 and which DtypeForwardPropagationPass can produce. A narrowing round trip (f32 -> bf16 -> f32) is never folded: it is the rounding the layer asked for. Narrow-to-narrow and integer casts are not treated as exact. The pass has no options and runs after dtype constraint resolution in the default, aggressive and LLM pipelines. ComputeGraphExecutor dispatches convert / cast / to, taking the target from the node's output dtype, so a graph with casts can be executed. The new test runs the folded and the unfolded cast chain on the CPU backend and compares raw bits. CastFoldingPassTest covers each rule, a widening with a second consumer, cast aliases and dtype spellings. MixedPrecisionExportTest pins three stablehlo.convert instead of four for a BF16 dense followed by an FP16 stage. SKEEP-008 Open Question 2 is resolved. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1494. Resolves Open Question 2 of SKEEP-008.
Problem
Two adjacent mixed-precision boundaries each cast back to the network dtype, so the tensor between them is widened only to be narrowed again. A BF16
densefollowed by an FP16stageexported:Change
CastFoldingPass(skainet-compile-opt) applies two value-preserving rewrites:bf16 -> f32 -> f16becomesbf16 -> f16. The widening is removed once nothing else reads it; if it has another consumer it stays and only the second cast is rewired.f16 -> f32 -> f16, andDtypeForwardPropagationPass(targetFloatDtype = …)can produce it.Never folded: the narrowing round trip
f32 -> bf16 -> f32(the rounding the layer asked for), narrow-to-narrow casts, and integer casts (Int32 to FP32 is not exact).The pass has no options, as assessed in the issue: the exactness rule follows from the dtypes, and per-target variation already has
TargetOptimizers.registerDagPasses. It runs right afterDTypeConstraintResolutionPassin the default, aggressive and LLM pipelines ofGraphOptimizationPipeline.Beyond the issue
ComputeGraphExecutorcould not runconvertnodes, so "bit-identical before and after" could not be tested by execution. It now dispatchesconvert/cast/to, taking the target from the node's output dtype.Tests
CastFoldingPassTest(12): each rule, the adjacent-boundary chain, a widening with a second consumer, cast aliases and dtype spellings, diagnostics, an untouched graph returned as the same instance.MixedPrecisionExportTest(+2): the traced network exports threestablehlo.convertinstead of four; the folded and unfolded cast chain executed on the CPU backend give identical raw bits.Review notes
CastFoldingPassadded;skainet-compile-opt.apiregenerated.explanation/mixed-precision.adoc, SKEEP-008 Open Questions (2 resolved, 4 links [Feature]: Decide whether a PrecisionBoundary carries DTypePolicy metadata into the compiled graph (SKEEP-008 follow-up) #1495), CHANGELOG.Test plan
./gradlew apiCheck checkUnmarkedStubsjvmTestforskainet-compile-opt,skainet-compile-hlo,skainet-compile-dagskainet-compile-optandskainet-compile-dag🤖 Generated with Claude Code