Skip to content

Migrate blob consumers of coordinaotor_joint_state topic to per robot joint topic - #3277

Merged
mustafab0 merged 5 commits into
mainfrom
migrate-blob-consumers
Jul 30, 2026
Merged

Migrate blob consumers of coordinaotor_joint_state topic to per robot joint topic#3277
mustafab0 merged 5 commits into
mainfrom
migrate-blob-consumers

Conversation

@mustafab0

Copy link
Copy Markdown
Contributor

Problem

The control coordinator publishes one message with every robot's joints stitched
together. Anything that wants one robot's joints has to take the whole thing.

That merged message is being retired. Five places still read it, and all five only care
about a single robot.

Issue: #


Solution

Each coordinator now also publishes one stream per robot, and those five readers
subscribe to the one they need.

Two halves per deployment: add the output on the coordinator, point the reader at it.
One commit per reader.

Recorder modules are untouched, so recordings keep their old stream names and DataPrep
configs still work.

The merged message still publishes. Retiring it is a later PR, as is the manipulation
module.


Breaking Changes

None. The go2 controllers keep the same LCM topic, so Foxglove and lcm-spy are unaffected.


How to Test

  1. pytest dimos/control dimos/imitation dimos/perception dimos/robot -m 'not (tool or self_hosted or mujoco or self_hosted_large)'
  2. dimos --simulation mujoco run unitree-g1-groot-wbc — G1 still animates in Rerun, nav still drives it.
  3. None of the five deployments run in CI. Owner runs on hardware:
    • learning-collect-quest-piper — record an episode, DataPrep still builds it
    • xarm6-worldbelief — recording still has coordinator_joint_state
    • unitree-g1-coordinator + g1_replay.py --dry-run — picks up current pose
    • unitree-go2-holonomic-controller and -rpp-controller — tracking unchanged

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

@@           Coverage Diff           @@
##             main    #3277   +/-   ##
=======================================
  Coverage   75.20%   75.21%           
=======================================
  Files        1128     1128           
  Lines      107954   107983   +29     
  Branches     9751     9751           
=======================================
+ Hits        81188    81219   +31     
+ Misses      23955    23954    -1     
+ Partials     2811     2810    -1     
Flag Coverage Δ
OS-ubuntu-24.04-arm 68.86% <100.00%> (+<0.01%) ⬆️
OS-ubuntu-latest 70.93% <100.00%> (+0.01%) ⬆️
Py-3.10 70.93% <100.00%> (+<0.01%) ⬆️
Py-3.11 70.92% <100.00%> (+<0.01%) ⬆️
Py-3.12 70.92% <100.00%> (+0.01%) ⬆️
Py-3.13 70.93% <100.00%> (+0.01%) ⬆️
Py-3.14 70.93% <100.00%> (+0.01%) ⬆️
Py-3.14t 70.92% <100.00%> (+<0.01%) ⬆️
SelfHosted-Large 29.08% <100.00%> (ø)
SelfHosted-Linux 35.66% <100.00%> (+<0.01%) ⬆️
SelfHosted-macOS 34.72% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/imitation/collection/blueprint.py 88.88% <100.00%> (+0.65%) ⬆️
dimos/imitation/collection/recorder.py 100.00% <ø> (ø)
dimos/robot/manipulators/common/blueprints.py 88.88% <100.00%> (ø)
...imos/robot/manipulators/piper/blueprints/teleop.py 100.00% <100.00%> (ø)
dimos/robot/manipulators/xarm/blueprints/teleop.py 100.00% <100.00%> (ø)
.../robot/manipulators/xarm/blueprints/worldbelief.py 88.23% <100.00%> (+2.52%) ⬆️
...tree/g1/blueprints/basic/unitree_g1_coordinator.py 100.00% <100.00%> (ø)
...nitree/g1/blueprints/basic/unitree_g1_groot_wbc.py 55.34% <100.00%> (+1.44%) ⬆️
...ueprints/basic/unitree_go2_holonomic_controller.py 100.00% <100.00%> (ø)
...go2/blueprints/basic/unitree_go2_rpp_controller.py 100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mustafab0
mustafab0 force-pushed the migrate-blob-consumers branch from 0fb03ca to 4d7f733 Compare July 29, 2026 21:55
Comment thread dimos/imitation/collection/blueprint.py
paul-nechifor
paul-nechifor previously approved these changes Jul 29, 2026
@mustafab0
mustafab0 enabled auto-merge July 29, 2026 22:05
@mustafab0
mustafab0 added this pull request to the merge queue Jul 29, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 29, 2026
@mustafab0
mustafab0 removed this pull request from the merge queue due to a manual request Jul 29, 2026
@mustafab0
mustafab0 added this pull request to the merge queue Jul 29, 2026
@mustafab0
mustafab0 removed this pull request from the merge queue due to a manual request Jul 29, 2026
The quest teleop coordinators subclass ControlCoordinator to declare
arm_joints and opt into publish_robot_joint_states; the collection
blueprints remap the recorder's joint port onto it. The recorder module is
unchanged, so the db stream name DataPrep reads stays
coordinator_joint_state.

test_blueprints looks the coordinator up with issubclass now that
deployments subclass it.
The xArm6 worldbelief coordinator declares arm_joints and opts into
publish_robot_joint_states; the blueprint remaps the recorder's joint port
onto it. The recorder module is unchanged, so both the db stream name and
its @pose_setter_for still key off the port name.
unitree-g1-coordinator declares g1_joints, opts into
publish_robot_joint_states, and exports it on /g1/joints alongside the
other /g1/* wire topics; g1_replay subscribes there instead of the merged
/coordinator_joint_state. Replay behaviour is otherwise unchanged.
The groot coordinator declares g1_joints and exports it on /g1/joints, so
the Rerun URDF override and its throttle now key off world/g1/joints. The
twist_command remap names the subclass, because remapping_map is keyed by
instance name and a base-class key stops matching once the blueprint
subclasses ControlCoordinator.
The holonomic and RPP controllers declare go2_joints and export it on
/coordinator/joint_state, the same wire topic the merged blob used, so
external readers are unaffected. Also drops the dead ('joint_state',
JointState) transport key — no stream by that name exists in either
blueprint.
@mustafab0
mustafab0 force-pushed the migrate-blob-consumers branch from 4d7f733 to 9cd73dd Compare July 29, 2026 23:25
@mustafab0
mustafab0 enabled auto-merge July 29, 2026 23:25
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Jul 29, 2026
@mustafab0
mustafab0 added this pull request to the merge queue Jul 30, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 30, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 30, 2026
@mustafab0
mustafab0 added this pull request to the merge queue Jul 30, 2026
Merged via the queue into main with commit 0bde111 Jul 30, 2026
33 checks passed
@mustafab0
mustafab0 deleted the migrate-blob-consumers branch July 30, 2026 01:32
mustafab0 added a commit that referenced this pull request Jul 30, 2026
CollectionRecorder and WorldBeliefRecorder capture whatever joints are
present, so the aggregate stream is their intended input. Dropping the
remaps #3277 added lets autoconnect pair them by port name again, with
no module-code changes.

Nothing else consumed either coordinator's `arm_joints`, so the per-robot
opt-in comes off the xarm7 teleop, piper teleop, and xarm6 worldbelief
coordinators too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants