Skip to content

dimos/web: LCM<->WebSocket bridge + Babylon operator console#2813

Open
Nabla7 wants to merge 6 commits into
mainfrom
pim/web/lcm-ws-bridge
Open

dimos/web: LCM<->WebSocket bridge + Babylon operator console#2813
Nabla7 wants to merge 6 commits into
mainfrom
pim/web/lcm-ws-bridge

Conversation

@Nabla7

@Nabla7 Nabla7 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Extracts the sim viewer's browser stack into dimos/web so it works with any blueprint:

  • dimos/web/lcm_bridge — subscribe-all LCM<->WebSocket bridge; browser code is
    just another bus peer (@dimos/msgs decodes the same bytes). Latest-wins buffering
    • per-channel rate caps, tested both directions over a real socket.
  • dimos/web/viewer — the Babylon operator console: scene mesh/splat, robot via
    server-side FK, pointclouds (web worker), nav path, camera, WASD teleop,
    click-to-nav. Renders only, owns no state — the browser-physics/entity-authority
    half stays in the sim tree; entity mirroring is a documented stub in app.js.
  • Wired into the G1 sim blueprint. On by default in sim, DIMOS_ENABLE_BABYLON=0 kills it.

Try it:

dimos --transport lcm --simulation mujoco --scene-package dimos_office run unitree-g1-groot-wbc

Console on :8091. Focus Robot, Enable Drive, WASD walks the G1; the Lidar layer streams /global_map.

--transport lcm matters: the bridge doesn't tap zenoh yet (follow-up: a zenoh
subscribe-all tap — same payload bytes, same browser framing, only the tap changes).
Server-level QoS only, LAN trust. Details in the package READMEs.

Diff is +5.2k/-0, but ~3.4k is the console frontend ported from the sim viewer;
new Python is ~1.6k (bridge, module, FK helpers, tests). Only existing file
touched: the G1 blueprint (+60 lines).

Input for #2710 / #2708 / #2502 — the local "bridge as a DimOS module" case,
alongside hosted teleop (#2745) covering the internet case.

Nabla7 added 6 commits July 7, 2026 02:49
Extract the Babylon scene viewer's /lcm-ws endpoint into a reusable
package, as a working v0 for the Dimos Web discussion (#2710, #2502,
#2708):

- LcmWebSocketBridge core: subscribe-all LCM -> WS fan-out with
  latest-wins per-client buffering, a single drain task per socket, and
  per-channel fnmatch rate caps + allow/blocklists (bus->browser leg
  only). Browser->bus packets republish indistinguishably from any
  other bus peer (LC02 small-message wire format both ways).
- LcmWebSocketBridgeModule: standalone server (default :9669) serving
  /lcm-ws, the @dimos/msgs browser client at /lcm_client.js, and JSON
  debug counters at /.
- Mountable into an existing Starlette app via bridge.routes() (the
  Babylon viewer pattern; the viewer becomes a consumer of this in a
  follow-up).

Per-connection QoS, fragmented LCM (>64KB), WebTransport and brokering
are deliberately out of scope for v0 - see the package README.
Viewer-only port of the simulation scene viewer (the one that has been
driving the sim work, teleoperated over the internet included) into
dimos/web/viewer, composable with any blueprint:

- Renders scene GLB/glTF + gaussian splat, robot (server-side FK from
  joint_state/odom against the same MJCF the sim steps), pointclouds
  (web-worker decode), nav path, camera frames; teleop and
  click-to-navigate publish real bus messages.
- It renders, it never owns state: all display/control topics flow
  through dimos/web/lcm_bridge (first consumer of the extracted
  bridge); the browser-physics / entity-authority half of the original
  module stays in the simulation tree, along with the Havok script.
- Everything optional: no MJCF -> no mujoco import (sim extra only
  needed for the robot layer); no scene -> grid floor. Ports tolerate
  running unwired so a bare script can serve it.
- mujoco dropped from geometry.py (scipy quaternion instead); pimsim
  branding retired.

Verified headless with playwright: G1 (g1_gear_wbc meshes) walking a
circle through the dimos_office scene from synthetic /odom +
/joint_state LCM traffic, console error-free.
dimos --simulation mujoco --scene-package dimos_office run \
    unitree-g1-groot-wbc

now serves the browser console at :8091 alongside the sim: the robot
mirrors live coordinator joint state + odom (g1_29dof robot-only MJCF,
meshes lazily read from LFS g1_urdf at module start), and the viewer
gets the same scene GLB + alignment the sim world was cooked from via
resolve_scene_package. On by default in sim; DIMOS_ENABLE_BABYLON=0
to suppress.

Also: BabylonViewerModule grows mesh_dir (deferred asset read so
blueprint import stays cheap), and Focus Robot now clamps the camera
radius inward - it only ever grew it, so after the scene auto-frame
the camera stayed outside the building staring at the roof.

Verified live: full blueprint boot, G1 rendered inside the office
scene from real sim data, headless playwright screenshots.
Main defaults to the Zenoh transport now; the bridge taps LCM, so
default runs show a posed robot but no pointclouds/teleop. Run with
--transport lcm; a Zenoh subscribe-all twin is the follow-up.
- stream_worker.js now loads with the same cache-bust token as the
  pointcloud worker; a stale cached worker against a newer server is
  the silent-hang failure mode.
- /viewer_debug no longer splats client-controlled JSON keys into
  structlog kwargs (an "event" key collided with the message).
- mesh_dir asset scan skips .DS_Store / AppleDouble junk that rides
  along in LFS-extracted directories.
…irror

The port carried ~1,000 lines that can never execute in a view-only
build: Havok init, the browser character controller (ground snapping,
planar collision, support floor), the robot physics capsule, collision
asset loading, browser-authored sim_odom, and the browser-authoritative
entity world. All removed - this page renders, it never owns state. The
authority build lives in the simulation tree.

Entity mirroring (MuJoCo-owned entities drawn as kinematic meshes) is
reduced to a documented stub: the seam, what to port back, and where
the reference implementation lives. The stub logs once if
/entity_state_batch appears on the bus so operators know the data is
there but undrawn.

index.html loses the Collision/Drop Box/Clear Entities controls and the
Entities HUD line; /config.json now carries exactly the keys the page
reads. app.js: 2857 -> 1830 lines.

Verified headless against the live G1 sim: every remaining control
clicked, zero console errors, viewerReady reached. The sweep caught one
hidden call site (loadSceneAsset -> ensureSupportFloor) that the static
grep missed.
@greptile-apps

greptile-apps Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a browser operator console and an LCM WebSocket bridge. The main changes are:

  • New subscribe-all LCM to WebSocket bridge with browser client helpers.
  • New Babylon viewer server with robot FK, assets, pointclouds, camera, path, and teleop UI.
  • G1 MuJoCo blueprint wiring that launches the viewer by default in sim.
  • Web extra dependency updates and bridge integration tests.

Confidence Score: 4/5

The bridge publish path needs a fix before merging.

  • Browser clients can republish arbitrary LCM channels through /lcm-ws.
  • The viewer starts this bridge in the sim path and exposes command publishing.
  • The main viewer data contracts look consistent from the changed code.

dimos/web/lcm_bridge/bridge.py

Security Review

The new WebSocket bridge allows unauthenticated clients to republish arbitrary LCM packets to the bus. This exposes command channels such as /cmd_vel to any host that can reach the bridge listener.

Important Files Changed

Filename Overview
dimos/web/lcm_bridge/bridge.py Adds the core LCM subscribe-all bridge, with a publish-side channel control gap and a small cross-thread client-state race.
dimos/web/lcm_bridge/module.py Adds the standalone bridge server and status/static routes.
dimos/web/viewer/module.py Adds the Babylon viewer server, asset/config routes, embedded bridge, and robot-pose websocket stream.
dimos/web/viewer/static/app.js Adds the browser console UI, LCM subscriptions, teleop publishing, and rendering paths.
dimos/robot/unitree/g1/blueprints/basic/unitree_g1_groot_wbc.py Wires the Babylon viewer into the G1 MuJoCo blueprint with environment toggles.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Browser[Browser console] -->|LC02 packets over /lcm-ws| Bridge[LcmWebSocketBridge]
  Bridge -->|publish_packet| LCM[(LCM bus)]
  LCM -->|subscribe-all| Bridge
  Bridge -->|filtered/rate-capped packets| Browser
  Sim[G1 sim modules] -->|joint_state / odom| Viewer[BabylonViewerModule]
  Viewer -->|robot pose over /ws| Browser
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart LR
  Browser[Browser console] -->|LC02 packets over /lcm-ws| Bridge[LcmWebSocketBridge]
  Bridge -->|publish_packet| LCM[(LCM bus)]
  LCM -->|subscribe-all| Bridge
  Bridge -->|filtered/rate-capped packets| Browser
  Sim[G1 sim modules] -->|joint_state / odom| Viewer[BabylonViewerModule]
  Viewer -->|robot pose over /ws| Browser
Loading

Reviews (1): Last reviewed commit: "refactor(web): strip authority/physics f..." | Re-trigger Greptile

logger.warning("lcm-ws bridge: dropping malformed packet from client")
return
channel, payload = decoded
self._lcm.publish(channel, payload)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Arbitrary Bus Publish Path

Any client that can reach /lcm-ws can send a valid LC02 frame for any channel, and this line republishes it without a publish allowlist or blocklist. Since the new servers bind on all interfaces and the viewer uses this path for /cmd_vel, a LAN browser can inject actuator or control messages beyond the console topics.

self._on_bus_message(channel, data)
except Exception:
logger.exception("lcm-ws bridge: on_bus_message tap failed")
if not self._clients:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Cross-Thread Client Gate

The LCM reader thread checks _clients while websocket connect and disconnect mutate it on the Starlette loop. A bus message that arrives during the first client accept can see the set as empty and return before scheduling _enqueue, so the newly connected browser can miss its first update even though the loop-side enqueue path would otherwise handle the client safely.

@Nabla7 Nabla7 mentioned this pull request Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant