Skip to content

feat(r1pro): add Quest upper-body teleoperation - #3880

Open
TomCC7 wants to merge 5 commits into
cc/feat/unbounded-planar-basefrom
cc/feat/r1pro-teleop
Open

feat(r1pro): add Quest upper-body teleoperation#3880
TomCC7 wants to merge 5 commits into
cc/feat/unbounded-planar-basefrom
cc/feat/r1pro-teleop

Conversation

@TomCC7

@TomCC7 TomCC7 commented Sep 1, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

R1 Pro has planning and Viser support but no Quest teleoperation path for coordinated arm and torso control.

Solution

  • Stream Quest controller and headset poses into one atomic teleoperation IK session.
  • Control the 14 arm joints and four torso joints while leaving the planar base and grippers unclaimed.
  • Constrain the headset target to forward and vertical translation plus pitch and yaw.
  • Add a priority-20 upper-body trajectory task so ManipulationModule.execute() can dispatch planned motion and preempt teleoperation without claiming the base.
  • Follow the existing Quest blueprint split between the robot coordinator and central Quest composition.
  • Reuse the R1 Pro planning groups and enable the Viser panel.

How to Test

Run the feature:

dimos run teleop-quest-r1pro

Validation:

uv run pytest dimos/control/tasks/teleop_ik_task/test_teleop_ik_task.py dimos/control/tasks/test_registry.py dimos/robot/manipulators/r1pro/test_teleop.py dimos/teleop/quest/test_quest_teleop_module.py -q
uv run pytest dimos/robot/manipulators/r1pro/test_teleop.py dimos/manipulation/test_plan_execution.py dimos/control/test_coordinator_routing.py -q
CI=1 uv run pytest dimos/robot/test_all_blueprints_generation.py -q
uv run mypy dimos/robot/manipulators/r1pro/blueprints/teleop.py dimos/robot/manipulators/r1pro/teleop_ik.py
uv run pre-commit run --files dimos/robot/all_blueprints.py dimos/robot/manipulators/r1pro/blueprints/teleop.py dimos/robot/manipulators/r1pro/teleop_ik.py dimos/robot/manipulators/r1pro/test_teleop.py dimos/teleop/quest/README.md docs/capabilities/manipulation/index.md

AI assistance

OpenAI Codex with GPT-5 implemented the feature and tests under developer direction. The resulting changes were reviewed and adjusted interactively.

Checklist

  • I have read and approved the CLA.

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.42060% with 13 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...mos/control/tasks/teleop_ik_task/teleop_ik_task.py 85.93% 4 Missing and 5 partials ⚠️
dimos/control/_control_test_helpers.py 50.00% 2 Missing ⚠️
dimos/teleop/quest/quest_teleop_module.py 60.00% 1 Missing and 1 partial ⚠️
@@                        Coverage Diff                        @@
##           cc/feat/unbounded-planar-base    #3880      +/-   ##
=================================================================
+ Coverage                          78.01%   78.04%   +0.02%     
=================================================================
  Files                               1334     1337       +3     
  Lines                             127828   128036     +208     
  Branches                           11106    11119      +13     
=================================================================
+ Hits                               99724    99921     +197     
- Misses                             24920    24924       +4     
- Partials                            3184     3191       +7     
Flag Coverage Δ
OS-ubuntu-24.04-arm 74.67% <94.42%> (+0.03%) ⬆️
OS-ubuntu-latest 75.26% <94.42%> (+0.03%) ⬆️
Py-3.10 75.25% <94.42%> (+0.03%) ⬆️
Py-3.11 75.25% <94.42%> (+0.03%) ⬆️
Py-3.12 75.25% <94.42%> (+0.03%) ⬆️
Py-3.13 75.25% <94.42%> (+0.02%) ⬆️
Py-3.14 75.25% <94.42%> (+0.03%) ⬆️
Py-3.14t 75.25% <94.42%> (+0.03%) ⬆️
SelfHosted-Large 30.21% <28.32%> (-0.01%) ⬇️
SelfHosted-Linux 34.83% <28.32%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/control/tasks/teleop_ik_task/_registry.py 100.00% <ø> (ø)
...ontrol/tasks/teleop_ik_task/test_teleop_ik_task.py 100.00% <100.00%> (ø)
dimos/control/tasks/test_registry.py 87.19% <ø> (ø)
dimos/control/teleop_coordinator.py 100.00% <100.00%> (ø)
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
dimos/robot/manipulators/common/blueprints.py 90.90% <100.00%> (+0.43%) ⬆️
...imos/robot/manipulators/r1pro/blueprints/teleop.py 100.00% <100.00%> (ø)
dimos/robot/manipulators/r1pro/config.py 100.00% <100.00%> (ø)
dimos/robot/manipulators/r1pro/teleop_ik.py 100.00% <100.00%> (ø)
dimos/robot/manipulators/r1pro/test_teleop.py 100.00% <100.00%> (ø)
... and 9 more

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TomCC7
TomCC7 marked this pull request as ready for review September 1, 2026 22:16
@greptile-apps

greptile-apps Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds Quest-based R1 Pro upper-body teleoperation with coordinated left-controller, right-controller, and headset targets. The new configuration limits teleoperation to the torso and arms, leaving planar-base joints outside the command set. It also adds deadman engagement, input expiry, session reset, and release-before-rearm behavior.

No defect was established from the available evidence.

T-Rex validation blocked

The focused R1 Pro runtime probes could not execute because the Python package providing the zenoh module is absent. Both probes stopped during import at dimos/protocol/pubsub/impl/zenohpubsub.py:23 with ModuleNotFoundError: No module named 'zenoh', before they could exercise command generation, stale-input stopping, or rearming.

Confidence Score: 5/5

No confirmed behavior would block merging; the R1 Pro Quest runtime path still needs execution in an environment with its messaging dependency installed.

No reproducible product defect was found. The only executed result established an environment dependency failure before the runtime assertions could run.

Files Needing Attention: Runtime confirmation remains most valuable for dimos/teleop/quest/blueprints.py, dimos/robot/manipulators/r1pro/blueprints/teleop.py, and dimos/control/tasks/teleop_ik_task/teleop_ik_task.py once zenoh is available.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex attempted to run the teleoperation tests before and after the changes using uv run, and both runs terminated with exit code 1 due to a ModuleNotFoundError: No module named 'zenoh' at zenohpubsub.py:23.
  • The blocker is confirmed to be the missing zenoh package, and no teleoperation defect is established from these runs.

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "revert(r1pro): drop hands-only teleop mo..." | Re-trigger Greptile

@TomCC7
TomCC7 force-pushed the cc/feat/r1pro-teleop branch from 7dcb186 to a602b43 Compare September 1, 2026 22:30
@TomCC7
TomCC7 changed the base branch from feat/r1pro-mobile-manip to cc/feat/unbounded-planar-base September 1, 2026 22:31
@TomCC7
TomCC7 force-pushed the cc/feat/r1pro-teleop branch from a602b43 to 786d9fb Compare September 1, 2026 23:07
Add atomic Quest control for both arms and the four-joint torso while leaving the planar base and grippers unclaimed. Stream the headset pose into anisotropic Pink head targets and expose the existing Viser planning panel.
@TomCC7
TomCC7 force-pushed the cc/feat/r1pro-teleop branch from 786d9fb to 5dca547 Compare September 1, 2026 23:12
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant