Skip to content

Add vision-only Edge-LLM ExecuTorch delegate integration - #4730

Open
micwill755 wants to merge 1 commit into
exporters-unified-runnerfrom
executorch-edge
Open

micwill755 wants to merge 1 commit into
exporters-unified-runnerfrom
executorch-edge

Conversation

@micwill755

@micwill755 micwill755 commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a vision-only TensorRT-Edge-LLM delegate path for ExecuTorch.

  • Adds edge_llm::vision_tower and EdgeLLMPartitioner.
  • Adds EL01 metadata wrapping existing TR01/TR02 engine payloads.
  • Adds native EdgeLLMBackend and payload parsing.
  • Adds PI0.5 HWC vision export and .pte packaging.
  • Extends the reference runner for FP16 inputs.
  • Preserves the existing generic TensorRTBackend path.

Motivation and context

PI0.5 components require Edge-LLM-specific runtime behavior beyond generic TensorRT engine execution. This change establishes the first component path—vision—while keeping the exported program libtorch-free.

The partitioner maps edge_llm::vision_tower to EdgeLLMBackend. At runtime, the backend creates a stateful Edge vision runner and invokes it through ExecuTorch’s delegate interface.

A fresh PI0.5 vision engine was exported and executed successfully:

input:  [4, 224, 224, 3] FP16
output: [1024, 2048] FP16

Dependencies

  • ExecuTorch 1.4.1 or the repository’s pinned ExecuTorch commit.
  • CUDA 13 and TensorRT 11.2.
  • TensorRT-Edge-LLM companion changes providing libedgellmExecutorch.so.
  • libNvInfer_edgellm_plugin.so available through EDGELLM_PLUGIN_PATH.
  • Edge support is optional and feature-gated; normal Torch-TensorRT builds remain unchanged.

Issue

No tracking issue was provided.

Type of change

  • New feature (non-breaking change which adds functionality)

Testing

  • 7 focused Python tests passed.
  • C++ EL01 parser Bazel test passed.
  • Edge runtime and native backend built successfully.
  • Fresh PI0.5 vision .pte executed successfully.
  • Existing TensorRT .pte runner regression passed.

Checklist

  • My code follows the style guidelines of this project.
  • I have performed a self-review of my own code.
  • I have commented hard-to-understand areas and integration constraints.
  • I have made corresponding documentation changes.
  • I have added tests to verify the feature.
  • New and existing relevant unit tests pass locally.
  • I have added the relevant PR labels and notified reviewers.

@micwill755
micwill755 added this pull request to stack #4727 September 22, 2026 21:34
@meta-cla meta-cla Bot added the cla signed label Sep 22, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: api [Python] Issues re: Python API component: api [C++] Issues re: C++ API labels Sep 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [C++] Issues re: C++ API component: api [Python] Issues re: Python API component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant