dimos/web: LCM<->WebSocket bridge + Babylon operator console#2813
dimos/web: LCM<->WebSocket bridge + Babylon operator console#2813Nabla7 wants to merge 6 commits into
Conversation
Extract the Babylon scene viewer's /lcm-ws endpoint into a reusable package, as a working v0 for the Dimos Web discussion (#2710, #2502, #2708): - LcmWebSocketBridge core: subscribe-all LCM -> WS fan-out with latest-wins per-client buffering, a single drain task per socket, and per-channel fnmatch rate caps + allow/blocklists (bus->browser leg only). Browser->bus packets republish indistinguishably from any other bus peer (LC02 small-message wire format both ways). - LcmWebSocketBridgeModule: standalone server (default :9669) serving /lcm-ws, the @dimos/msgs browser client at /lcm_client.js, and JSON debug counters at /. - Mountable into an existing Starlette app via bridge.routes() (the Babylon viewer pattern; the viewer becomes a consumer of this in a follow-up). Per-connection QoS, fragmented LCM (>64KB), WebTransport and brokering are deliberately out of scope for v0 - see the package README.
Viewer-only port of the simulation scene viewer (the one that has been driving the sim work, teleoperated over the internet included) into dimos/web/viewer, composable with any blueprint: - Renders scene GLB/glTF + gaussian splat, robot (server-side FK from joint_state/odom against the same MJCF the sim steps), pointclouds (web-worker decode), nav path, camera frames; teleop and click-to-navigate publish real bus messages. - It renders, it never owns state: all display/control topics flow through dimos/web/lcm_bridge (first consumer of the extracted bridge); the browser-physics / entity-authority half of the original module stays in the simulation tree, along with the Havok script. - Everything optional: no MJCF -> no mujoco import (sim extra only needed for the robot layer); no scene -> grid floor. Ports tolerate running unwired so a bare script can serve it. - mujoco dropped from geometry.py (scipy quaternion instead); pimsim branding retired. Verified headless with playwright: G1 (g1_gear_wbc meshes) walking a circle through the dimos_office scene from synthetic /odom + /joint_state LCM traffic, console error-free.
dimos --simulation mujoco --scene-package dimos_office run \
unitree-g1-groot-wbc
now serves the browser console at :8091 alongside the sim: the robot
mirrors live coordinator joint state + odom (g1_29dof robot-only MJCF,
meshes lazily read from LFS g1_urdf at module start), and the viewer
gets the same scene GLB + alignment the sim world was cooked from via
resolve_scene_package. On by default in sim; DIMOS_ENABLE_BABYLON=0
to suppress.
Also: BabylonViewerModule grows mesh_dir (deferred asset read so
blueprint import stays cheap), and Focus Robot now clamps the camera
radius inward - it only ever grew it, so after the scene auto-frame
the camera stayed outside the building staring at the roof.
Verified live: full blueprint boot, G1 rendered inside the office
scene from real sim data, headless playwright screenshots.
Main defaults to the Zenoh transport now; the bridge taps LCM, so default runs show a posed robot but no pointclouds/teleop. Run with --transport lcm; a Zenoh subscribe-all twin is the follow-up.
- stream_worker.js now loads with the same cache-bust token as the pointcloud worker; a stale cached worker against a newer server is the silent-hang failure mode. - /viewer_debug no longer splats client-controlled JSON keys into structlog kwargs (an "event" key collided with the message). - mesh_dir asset scan skips .DS_Store / AppleDouble junk that rides along in LFS-extracted directories.
…irror The port carried ~1,000 lines that can never execute in a view-only build: Havok init, the browser character controller (ground snapping, planar collision, support floor), the robot physics capsule, collision asset loading, browser-authored sim_odom, and the browser-authoritative entity world. All removed - this page renders, it never owns state. The authority build lives in the simulation tree. Entity mirroring (MuJoCo-owned entities drawn as kinematic meshes) is reduced to a documented stub: the seam, what to port back, and where the reference implementation lives. The stub logs once if /entity_state_batch appears on the bus so operators know the data is there but undrawn. index.html loses the Collision/Drop Box/Clear Entities controls and the Entities HUD line; /config.json now carries exactly the keys the page reads. app.js: 2857 -> 1830 lines. Verified headless against the live G1 sim: every remaining control clicked, zero console errors, viewerReady reached. The sweep caught one hidden call site (loadSceneAsset -> ensureSupportFloor) that the static grep missed.
Greptile SummaryThis PR adds a browser operator console and an LCM WebSocket bridge. The main changes are:
Confidence Score: 4/5The bridge publish path needs a fix before merging.
dimos/web/lcm_bridge/bridge.py
|
| Filename | Overview |
|---|---|
| dimos/web/lcm_bridge/bridge.py | Adds the core LCM subscribe-all bridge, with a publish-side channel control gap and a small cross-thread client-state race. |
| dimos/web/lcm_bridge/module.py | Adds the standalone bridge server and status/static routes. |
| dimos/web/viewer/module.py | Adds the Babylon viewer server, asset/config routes, embedded bridge, and robot-pose websocket stream. |
| dimos/web/viewer/static/app.js | Adds the browser console UI, LCM subscriptions, teleop publishing, and rendering paths. |
| dimos/robot/unitree/g1/blueprints/basic/unitree_g1_groot_wbc.py | Wires the Babylon viewer into the G1 MuJoCo blueprint with environment toggles. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Browser[Browser console] -->|LC02 packets over /lcm-ws| Bridge[LcmWebSocketBridge]
Bridge -->|publish_packet| LCM[(LCM bus)]
LCM -->|subscribe-all| Bridge
Bridge -->|filtered/rate-capped packets| Browser
Sim[G1 sim modules] -->|joint_state / odom| Viewer[BabylonViewerModule]
Viewer -->|robot pose over /ws| Browser
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart LR
Browser[Browser console] -->|LC02 packets over /lcm-ws| Bridge[LcmWebSocketBridge]
Bridge -->|publish_packet| LCM[(LCM bus)]
LCM -->|subscribe-all| Bridge
Bridge -->|filtered/rate-capped packets| Browser
Sim[G1 sim modules] -->|joint_state / odom| Viewer[BabylonViewerModule]
Viewer -->|robot pose over /ws| Browser
Reviews (1): Last reviewed commit: "refactor(web): strip authority/physics f..." | Re-trigger Greptile
| logger.warning("lcm-ws bridge: dropping malformed packet from client") | ||
| return | ||
| channel, payload = decoded | ||
| self._lcm.publish(channel, payload) |
There was a problem hiding this comment.
Any client that can reach /lcm-ws can send a valid LC02 frame for any channel, and this line republishes it without a publish allowlist or blocklist. Since the new servers bind on all interfaces and the viewer uses this path for /cmd_vel, a LAN browser can inject actuator or control messages beyond the console topics.
| self._on_bus_message(channel, data) | ||
| except Exception: | ||
| logger.exception("lcm-ws bridge: on_bus_message tap failed") | ||
| if not self._clients: |
There was a problem hiding this comment.
The LCM reader thread checks _clients while websocket connect and disconnect mutate it on the Starlette loop. A bus message that arrives during the first client accept can see the set as empty and return before scheduling _enqueue, so the newly connected browser can miss its first update even though the loop-side enqueue path would otherwise handle the client safely.
Extracts the sim viewer's browser stack into dimos/web so it works with any blueprint:
dimos/web/lcm_bridge— subscribe-all LCM<->WebSocket bridge; browser code isjust another bus peer (@dimos/msgs decodes the same bytes). Latest-wins buffering
dimos/web/viewer— the Babylon operator console: scene mesh/splat, robot viaserver-side FK, pointclouds (web worker), nav path, camera, WASD teleop,
click-to-nav. Renders only, owns no state — the browser-physics/entity-authority
half stays in the sim tree; entity mirroring is a documented stub in app.js.
Try it:
Console on :8091. Focus Robot, Enable Drive, WASD walks the G1; the Lidar layer streams /global_map.
--transport lcm matters: the bridge doesn't tap zenoh yet (follow-up: a zenoh
subscribe-all tap — same payload bytes, same browser framing, only the tap changes).
Server-level QoS only, LAN trust. Details in the package READMEs.
Diff is +5.2k/-0, but ~3.4k is the console frontend ported from the sim viewer;
new Python is ~1.6k (bridge, module, FK helpers, tests). Only existing file
touched: the G1 blueprint (+60 lines).
Input for #2710 / #2708 / #2502 — the local "bridge as a DimOS module" case,
alongside hosted teleop (#2745) covering the internet case.