Add SmolVLA ONNX conversion Learning Path

content/learning-paths/embedded-and-microcontrollers/smolvla-onnx-conversion/_index.md

@@ -0,0 +1,65 @@
+---
+title: Convert SmolVLA to ONNX for Arm CPUs
+
+draft: true
+cascade:
+    draft: true
+
+description: Export SmolVLA from PyTorch to ONNX, create a packed INT4 weight-only model with TorchAO quantization, and run both models with ONNX Runtime on an Arm CPU.
+minutes_to_complete: 180
+who_is_this_for: Machine learning developers who want to deploy a vision-language-action policy with ONNX Runtime on an Arm CPU.
+learning_objectives:
+  - Export SmolVLA from PyTorch as an ONNX model.
+  - Run and validate the FP32 ONNX model with ONNX Runtime on an Arm CPU.
+  - Quantize eligible linear weights to INT4 and store them in a packed ONNX model.
+  - Run the FP32 and INT4 models with identical inputs and compare their action outputs and ONNX Runtime latency.
+prerequisites:
+  - An aarch64 Linux system, such as the Radxa Orion O6.
+  - Enough free storage for the model data, checkpoint weights, Python environment, and generated ONNX files.
+  - Git and Python 3.12.
+  - Familiarity with Python, PyTorch, and Linux command-line tools.
+author: ""
+generate_summary_faq: true
+rerun_summary: false
+rerun_faqs: false
+skilllevels: Advanced
+subjects: ML
+armips:
+  - Cortex-A
+operatingsystems:
+  - Linux
+tools_software_languages:
+  - Python
+  - PyTorch
+  - TorchAO
+  - ONNX
+  - ONNX Runtime
+  - LeRobot
+further_reading:
+  - resource:
+      title: PyTorch ONNX exporter documentation
+      link: https://docs.pytorch.org/docs/stable/onnx.html
+      type: documentation
+  - resource:
+      title: TorchAO documentation
+      link: https://docs.pytorch.org/ao/stable/
+      type: documentation
+  - resource:
+      title: ONNX Runtime 4-bit quantization
+      link: https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html#quantize-to-int4uint4
+      type: documentation
+  - resource:
+      title: ONNX Runtime Execution Providers
+      link: https://onnxruntime.ai/docs/execution-providers/
+      type: documentation
+  - resource:
+      title: KleidiAI optimized micro-kernels for Arm CPUs
+      link: https://github.com/ARM-software/kleidiai
+      type: GitHub Repository
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1
+layout: "learningpathall"
+learning_path_main_page: "yes"
+---

content/learning-paths/embedded-and-microcontrollers/smolvla-onnx-conversion/_next-steps.md

@@ -0,0 +1,8 @@
+---
+# ================================================================================
+#       FIXED, DO NOT MODIFY THIS FILE
+# ================================================================================
+weight: 21                  # The weight controls the order of the pages. _index.md always has weight 1.
+title: "Next Steps"         # Always the same, html page title.
+layout: "learningpathall"   # All files under learning paths have this same wrapper for Hugo processing.
+---

content/learning-paths/embedded-and-microcontrollers/smolvla-onnx-conversion/convert-and-validate-onnx.md

@@ -0,0 +1,88 @@
+---
+title: Export and validate the SmolVLA ONNX model
+description: Export SmolVLA from PyTorch as an FP32 ONNX model and validate its action output on an Arm CPU.
+weight: 3
+layout: learningpathall
+---
+
+## Understand SmolVLA
+
+SmolVLA is a compact vision-language-action model. It combines camera images,
+a task instruction, and the robot state to generate a sequence of robot
+actions.
+
+![SmolVLA combines camera images, a task instruction, and robot state in a vision-language model that conditions an action expert to generate an action sequence.#center](smolvla.png "SmolVLA architecture")
+
+Image source: [SmolVLA paper](https://arxiv.org/pdf/2506.01844).
+
+## Export SmolVLA to ONNX
+
+The checkpoint includes the SmolVLA policy and the LeRobot processors used
+before and after inference. The exporter writes the policy to an ONNX graph;
+the processors remain outside it.
+
+Run the exporter:
+
+```bash
+work/venv/bin/python scripts/export_onnx.py \
+  --checkpoint work/artifacts/smolvla_libero \
+  --output work/onnx/fp32/model.onnx \
+  --reference-dir work/onnx/fp32/reference
+```
+
+The exporter creates a fixed-shape model and a deterministic reference batch.
+The batch includes an explicit flow-matching noise tensor, so the PyTorch and
+ONNX Runtime paths receive the same inputs.
+
+The expected output ends with:
+
+```output
+PASS: ONNX Runtime matches PyTorch within atol=0.001 and rtol=0.001
+```
+
+The FP32 graph contains the exported policy computation. Large weights can be
+stored in external data files beside `model.onnx`, so keep the complete
+`work/onnx/fp32` directory together.
+
+## Review the model interface
+
+The graph accepts six preprocessed inputs:
+
+| Input | Type | Shape | Description |
+| --- | --- | --- | --- |
+| `camera1` | `float32` | `[1, 3, 512, 512]` | Primary RGB image scaled to `[0, 1]` |
+| `camera2` | `float32` | `[1, 3, 512, 512]` | Wrist RGB image scaled to `[0, 1]` |
+| `lang_tokens` | `int64` | `[1, 48]` | Tokenized task instruction |
+| `lang_attention_mask` | `int64` | `[1, 48]` | Valid language-token mask |
+| `state` | `float32` | `[1, 8]` | Normalized robot state |
+| `noise` | `float32` | `[1, 50, 32]` | Flow-matching noise |
+
+The model returns an action chunk with shape `[1, 50, 7]`: 50 steps with seven
+control values each.
+
+The ONNX boundary deliberately starts after preprocessing and ends before
+postprocessing:
+
+```text
+observation -> LeRobot preprocessor -> ONNX model -> LeRobot postprocessor -> robot action
+```
+
+Use the processors from the same checkpoint. They preserve the tokenization,
+normalization, and action conversion expected by the policy.
+
+## Inspect the validation report
+
+View the report written by the exporter:
+
+```bash
+work/venv/bin/python -m json.tool work/onnx/fp32/validation.json
+```
+
+Confirm that the report lists `CPUExecutionProvider`, reports an output shape
+of `[1, 50, 7]`, and passes validation.
+
+## What you've accomplished and what's next
+
+You have exported SmolVLA to FP32 ONNX and validated its action output
+with ONNX Runtime on an Arm CPU. Next, you will quantize the eligible linear
+weights to packed INT4 and run the resulting model through the same interface.

content/learning-paths/embedded-and-microcontrollers/smolvla-onnx-conversion/quantize-and-run-int4-onnx.md

@@ -0,0 +1,66 @@
+---
+title: Quantize SmolVLA to INT4 and compare it with FP32
+description: Quantize eligible SmolVLA linear weights with TorchAO, then compare FP32 and INT4 outputs and ONNX Runtime latency on Arm.
+weight: 4
+layout: learningpathall
+---
+
+## INT4 quantization scope
+
+TorchAO quantizes eligible constant linear weights across the exported SmolVLA
+model. The converter replaces supported ONNX `MatMul` and `Gemm` operations
+with packed `com.microsoft::MatMulNBits` operations.
+
+This is weight-only quantization, not an entirely INT4 graph.
+
+On supported Arm CPUs, ONNX Runtime can use optimized kernels such as KleidiAI.
+
+## Create the packed INT4 model
+
+Run the TorchAO converter:
+
+```bash
+work/venv/bin/python scripts/quantize_onnx_torchao.py \
+  --input work/onnx/fp32/model.onnx \
+  --output work/onnx/int4/smolvla-int4.onnx
+```
+
+The command creates a packed ONNX file and reports how many eligible linear
+operations were converted. Dynamic or unsupported matrix multiplications
+remain in floating point.
+
+## Compare FP32 and INT4
+
+Run both models with the deterministic reference batch created during export:
+
+```bash
+work/venv/bin/python scripts/compare_onnx_outputs.py \
+  --fp32-model work/onnx/fp32/model.onnx \
+  --int4-model work/onnx/int4/smolvla-int4.onnx \
+  --reference-dir work/onnx/fp32/reference \
+  --output work/comparison/smolvla-action-comparison.png
+```
+
+The script runs both models with ONNX Runtime `CPUExecutionProvider` and
+creates:
+
+```text
+work/comparison/smolvla-action-comparison.png
+work/comparison/smolvla-action-comparison.json
+```
+
+The figure compares all seven normalized output channels and median latency.
+The JSON file records the latency and overall output error.
+
+## Review the O6 result
+
+![Seven plots compare FP32 and TorchAO INT4 normalized SmolVLA outputs across all 50 predicted steps for each of seven channels. A latency panel compares median ONNX Runtime latency on a Radxa Orion O6.#center](smolvla-action-comparison.png "SmolVLA Action Comparison")
+
+On the O6, INT4 reduced median ONNX Runtime latency from 3.33 seconds to 2.06
+seconds, a 1.61x speedup. The normalized outputs had an MAE of 0.153.
+
+## What you've accomplished
+
+You have converted eligible SmolVLA linear weights to packed INT4 in an ONNX
+model, run FP32 and INT4 with identical inputs on an Arm CPU, and compared all
+seven normalized output channels and ONNX Runtime latency.

content/learning-paths/embedded-and-microcontrollers/smolvla-onnx-conversion/scripts/check_assets.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+"""Verify the pinned model, source, and environment used by the Learning Path."""
+
+from __future__ import annotations
+
+import argparse
+from importlib import metadata
+import json
+from pathlib import Path
+import subprocess
+import sys
+
+from workspace import configure_workspace
+
+
+EXPECTED = {
+    "lerobot": "30da8e687a6dfc617fcd94afc367ac7071c376ce",
+    "model_repo": "HuggingFaceVLA/smolvla_libero",
+    "model_revision": "6721902bc4d61e50a3bfdb11dfb4cb626f05d102",
+    "base_model_repo": "HuggingFaceTB/SmolVLM2-500M-Instruct",
+    "base_model_revision": "7b375e1b73b11138ff12fe22c8f2822d8fe03467",
+}
+
+EXPECTED_PACKAGES = {
+    "torch": "2.11.0+cpu",
+    "torchvision": "0.26.0+cpu",
+    "torchao": "0.18.0",
+    "transformers": "5.5.4",
+    "onnx": "1.22.0",
+    "onnxruntime": "1.29.0",
+}
+
+POLICY_FILES = (
+    "config.json",
+    "model.safetensors",
+    "policy_preprocessor.json",
+    "policy_preprocessor_step_5_normalizer_processor.safetensors",
+    "policy_postprocessor.json",
+    "policy_postprocessor_step_1_unnormalizer_processor.safetensors",
+)
+BASE_MODEL_FILES = ("config.json", "model.safetensors", "tokenizer.json")
+
+
+def verify_snapshot(root: Path, files: tuple[str, ...], revision: str) -> None:
+    """Verify required snapshot files and their Hugging Face revisions."""
+
+    for relative in files:
+        asset = root / relative
+        metadata_file = root / ".cache/huggingface/download" / f"{relative}.metadata"
+        if not asset.is_file():
+            raise FileNotFoundError(f"Missing public asset: {asset}")
+        if not metadata_file.is_file():
+            raise FileNotFoundError(f"Missing Hugging Face metadata: {metadata_file}")
+        lines = metadata_file.read_text(encoding="utf-8").splitlines()
+        if not lines or lines[0] != revision:
+            raise RuntimeError(f"Unexpected Hugging Face revision for {asset}")
+
+
+def verify_environment(work_root: Path) -> None:
+    """Verify the Python version, pinned packages, and environment manifest."""
+
+    if sys.version_info[:2] != (3, 12):
+        raise RuntimeError("Python 3.12 is required")
+    environment = work_root / "environment.freeze.txt"
+    if not environment.is_file():
+        raise FileNotFoundError(f"Missing environment manifest: {environment}")
+
+    for package, expected in EXPECTED_PACKAGES.items():
+        actual = metadata.version(package)
+        if actual != expected:
+            raise RuntimeError(f"Unexpected {package} version: {actual}")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--work-root", type=Path)
+    args = parser.parse_args()
+    work_root = configure_workspace(args.work_root)
+
+    revisions_path = work_root / "revisions.json"
+    revisions = json.loads(revisions_path.read_text(encoding="utf-8"))
+    if revisions != EXPECTED:
+        raise RuntimeError(f"Revision manifest does not match the Learning Path: {revisions_path}")
+
+    actual_lerobot = subprocess.check_output(
+        ["git", "-C", str(work_root / "lerobot"), "rev-parse", "HEAD"],
+        text=True,
+    ).strip()
+    if actual_lerobot != EXPECTED["lerobot"]:
+        raise RuntimeError(f"Unexpected LeRobot revision: {actual_lerobot}")
+    lerobot_status = subprocess.check_output(
+        ["git", "-C", str(work_root / "lerobot"), "status", "--porcelain"],
+        text=True,
+    ).strip()
+    if lerobot_status:
+        raise RuntimeError("LeRobot worktree has local changes")
+
+    verify_snapshot(
+        work_root / "artifacts/smolvla_libero",
+        POLICY_FILES,
+        EXPECTED["model_revision"],
+    )
+    verify_snapshot(
+        work_root / "artifacts/smolvlm_base",
+        BASE_MODEL_FILES,
+        EXPECTED["base_model_revision"],
+    )
+    verify_environment(work_root)
+
+    print("PASS: public policy, base model, LeRobot source, and environment are ready")
+
+
+if __name__ == "__main__":
+    main()