FA4 NVVM ICE on Thor (SM110): CUTE_DSL_ARCH set after cutlass-dsl import + hd256 2CTA dispatch

[DXICM]

Summary

FA4 on Jetson AGX Thor (SM110) hits an NVVM ICE during kernel compilation. Root cause is an env-var ordering bug in fa4_backend.py, amplified by the hd256 2CTA dispatch extension from #164 (7fd75d20). This affects every model that uses FA4 on Thor, not just GR00T.

Root Cause

fa4_backend.py imports cutlass-dsl before setting CUTE_DSL_ARCH=sm_101a. cutlass-dsl caches the device architecture at import time (sm_110a on Thor), so the later env-var assignment has no effect — kernels are then compiled for the wrong arch.

Combined with #164 (7fd75d20), which extended the hd256 2CTA dispatch path to SM110 without validation, this triggers an NVVM ICE at compile time on Thor.

Repro Conditions

• Jetson AGX Thor, SM110 (sm_110a)
• Any FA4-backed attention path with hd256 dispatch
• No external CUTE_DSL_ARCH override set before process start

Fix (in fix(fa4) commit 817cbf7, included in #177)

1. Set CUTE_DSL_ARCH before any cutlass-dsl import.
2. Restrict the hd256 2CTA dispatch to SM100 only.
3. Restore the _fa4_trimmed lazy loader.

Requires nvidia-cutlass-dsl >= 4.5.

Verification

After the fix, FA4 compiles and runs on Thor without the NVVM ICE and without needing an externally-set CUTE_DSL_ARCH (verified end-to-end with GR00T N1.6 on Thor, see #177).

Suggestion for upstream

Independently of #177, two small upstream changes would prevent this for everyone:

• Move the CUTE_DSL_ARCH assignment ahead of the cutlass-dsl import in fa4_backend.py.
• Gate the hd256 2CTA dispatch on SM100 (or validate it on SM110) in the path introduced by pi05 thor update: NVFP4 encoder/decoder tier + FA4 attention #164.