Skip to content

Update scene packages: recooked office/supermarket + lowpoly_tdm#2815

Open
Nabla7 wants to merge 5 commits into
mainfrom
pim/data/scene-packages
Open

Update scene packages: recooked office/supermarket + lowpoly_tdm#2815
Nabla7 wants to merge 5 commits into
mainfrom
pim/data/scene-packages

Conversation

@Nabla7

@Nabla7 Nabla7 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Problem

The scene-cooking PR (#2544) carried the recooked office/supermarket packages inside its diff, coupling ~1.6 GB of data to a code review. And there's a new scene to ship.

Solution

Data-only PR that lands ahead of #2544 (main's runtime already loads this package format — verified with main's loader):

  • dimos_office and supermarket recooked with the scene-cooking pipeline (cook_version 4: per-target visuals, objects.json picking sidecar).
  • New: lowpoly_tdm — a small low-poly test map. Cooked at 2x scale with a hand-tuned sidecar: the floors mesh is split into per-plate flat-top boxes (walkable, no hull seams) and the map's ceiling plate is skipped. Source: "LOWPOLY FPS TDM GAME MAP" by ResoForge.
  • Archive rebuilt per docs/development/large_file_management.md (removed stale data/.lfs/scene_packages.tar.gz, regenerated via ./bin/lfs_push).

#2544 no longer touches this archive.

How to Test

uv run dimos --simulation mujoco --scene-package office --viewer rerun run unitree-g1-groot-wbc -o mujocosimmodule.headless=true
uv run dimos --simulation mujoco --scene-package data/scene_packages/lowpoly_tdm --viewer rerun run unitree-g1-groot-wbc -o mujocosimmodule.headless=true

Contributor License Agreement

  • I have read and approved the CLA.

🤖 Generated with Claude Code

Packages recooked with the scene-cooking pipeline (cook_version 4:
per-target visuals, objects.json). Adds the lowpoly_tdm map: floors
split into flat-top boxes, ceiling skipped, 2x scale. Main's runtime
already loads this format; split out of #2544 so the data lands first.
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/robot/unitree/mujoco_connection.py 0.00% 2 Missing ⚠️
@@            Coverage Diff             @@
##             main    #2815      +/-   ##
==========================================
- Coverage   72.02%   71.98%   -0.05%     
==========================================
  Files         957      963       +6     
  Lines       84684    85582     +898     
  Branches     7689     7832     +143     
==========================================
+ Hits        60991    61603     +612     
- Misses      21648    21930     +282     
- Partials     2045     2049       +4     
Flag Coverage Δ
OS-ubuntu-24.04-arm 64.63% <0.00%> (+0.19%) ⬆️
OS-ubuntu-latest 67.17% <0.00%> (+0.16%) ⬆️
Py-3.10 67.15% <0.00%> (+0.16%) ⬆️
Py-3.11 67.16% <0.00%> (+0.16%) ⬆️
Py-3.12 67.15% <0.00%> (+0.16%) ⬆️
Py-3.13 67.16% <0.00%> (+0.17%) ⬆️
Py-3.14 67.17% <0.00%> (+0.16%) ⬆️
Py-3.14t 67.16% <0.00%> (+0.18%) ⬆️
SelfHosted-Linux 38.06% <ø> (-0.03%) ⬇️
SelfHosted-macOS 36.38% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/simulation/scenes/catalog.py 50.00% <ø> (+20.45%) ⬆️
dimos/robot/unitree/mujoco_connection.py 26.08% <0.00%> (-0.23%) ⬇️

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 8, 2026
The legacy mujoco_process sim (used by go2's --simulation mujoco) could
only load the baked-in data/mujoco_sim rooms. Route --scene-package
through it so the cooked packages this PR ships (office, supermarket,
lowpoly_tdm) work there too, without waiting for the MujocoSimModule
migration.

- model.py: load_scene() resolves a scene package to its wrapper.xml;
  get_assets()/load_model() accept extra asset dirs for the hull meshes
- mujoco_process.py: per-package default spawn (office's default start
  pos lands outside the recooked geometry); --mujoco-start-pos still
  overrides
- catalog.py: register lowpoly_tdm
- mujoco_connection.py: pre-pull scene_packages LFS before the sim
  subprocess so extraction doesn't eat its ready timeout

Verified: go1 walking policy stands 3s sim-time in office (-2.0, 1.6)
and lowpoly_tdm (default spawn); no-package path unchanged.
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Jul 8, 2026
@greptile-apps

greptile-apps Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR ships a data-only archive update (recooked dimos_office, supermarket, and new lowpoly_tdm) ahead of the scene-cooking PR (#2544), decoupling ~1.6 GB of binary data from a code review. Alongside the archive, it adds the runtime plumbing needed to consume cooked scene packages: geom-group visibility for depth/RGB cameras, a named spawn-override table, pre-downloading of the LFS archive before the subprocess starts, and a load_scene helper that selects between the package path and the legacy occupancy/room-XML path.

  • New lowpoly_tdm scene added to the catalog with aliases (tdm, lowpoly-tdm) and a corresponding _PACKAGE_DIRS entry; cooked at 2× scale with hand-tuned hull meshes.
  • scene_option.geomgroup[3] is now enabled for all scene-package renders (RGB, depth, viewer) so hull meshes in geom group 3 are visible to cameras; the comment explicitly acknowledges that the robot's own group-3 collision geoms are also exposed.
  • Pre-download of scene_packages LFS data added to MujocoConnection.__init__ so the subprocess doesn't time out waiting for a slow LFS pull.

Confidence Score: 5/5

Safe to merge; changes are well-scoped data registration and runtime plumbing with no logic gaps in the changed paths.

The code changes are small, focused, and well-documented. The spawn-override table and geom-group enablement are opt-in (only active when a scene package is loaded), and all new code paths are guarded by is not None checks. The only open items — missing lowpoly_tdm spawn override and OBJ-only asset loading — are carried from the existing review thread and don't block the core functionality being shipped.

No files require special attention beyond what the existing review thread already covers.

Important Files Changed

Filename Overview
dimos/simulation/scenes/catalog.py Adds lowpoly_tdm (plus two aliases) to _ALIASES and _PACKAGE_DIRS; straightforward registration consistent with existing pattern.
dimos/simulation/mujoco/model.py Adds extra_asset_dirs parameter to get_assets/load_model for scene-package OBJ meshes, and introduces load_scene to unify package vs. legacy XML selection; logic is clean and the mujoco_scene_path.read_text() path is guarded by a None check before use.
dimos/simulation/mujoco/mujoco_process.py Integrates load_scene, adds spawn-override table (_SCENE_PACKAGE_SPAWNS) and _start_pos_is_default helper, and enables geom group 3 for scene-package renders; lowpoly_tdm is absent from the spawn table (covered by existing review thread).
dimos/robot/unitree/mujoco_connection.py Pre-downloads scene_packages LFS data in the parent process before the subprocess starts, preventing timeout during model load; change is minimal and correct.
data/.lfs/scene_packages.tar.gz LFS pointer updated to include recooked dimos_office, supermarket, and new lowpoly_tdm packages; no code changes.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant U as User CLI
    participant MC as MujocoConnection
    participant SP as catalog.resolve_scene_package
    participant LS as model.load_scene
    participant LM as model.load_model
    participant MP as mujoco_process._run_simulation

    U->>MC: __init__(global_config)
    MC->>MC: get_data("mujoco_sim")
    alt scene_package set
        MC->>MC: get_data("scene_packages") [pre-download LFS]
    end
    MC->>MP: subprocess start (pickled config)

    MP->>LS: load_scene(config)
    alt "scene_package configured & no occupancy override"
        LS->>SP: resolve_scene_package(config.scene_package)
        SP-->>LS: ScenePackage (mujoco_scene_path, package_dir)
        LS-->>MP: (scene_xml_text, ScenePackage)
    else legacy path
        LS-->>MP: (load_scene_xml(config), None)
    end

    MP->>MP: "extra_asset_dirs = [package.mujoco_scene_path.parent]"
    MP->>LM: load_model(scene_xml, extra_asset_dirs)
    LM->>LM: "get_assets(extra_asset_dirs) → merge *.obj from pkg dir"
    LM-->>MP: (MjModel, MjData)

    alt scene_package present AND start_pos is default
        MP->>MP: override start_pos from _SCENE_PACKAGE_SPAWNS
    end
    MP->>MP: "data.qpos[0:3] = [x, y, z]"

    MP->>MP: enable scene_option.geomgroup[3] + m_viewer.opt.geomgroup[3]
    MP->>MP: simulation loop (render with scene_option)
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant U as User CLI
    participant MC as MujocoConnection
    participant SP as catalog.resolve_scene_package
    participant LS as model.load_scene
    participant LM as model.load_model
    participant MP as mujoco_process._run_simulation

    U->>MC: __init__(global_config)
    MC->>MC: get_data("mujoco_sim")
    alt scene_package set
        MC->>MC: get_data("scene_packages") [pre-download LFS]
    end
    MC->>MP: subprocess start (pickled config)

    MP->>LS: load_scene(config)
    alt "scene_package configured & no occupancy override"
        LS->>SP: resolve_scene_package(config.scene_package)
        SP-->>LS: ScenePackage (mujoco_scene_path, package_dir)
        LS-->>MP: (scene_xml_text, ScenePackage)
    else legacy path
        LS-->>MP: (load_scene_xml(config), None)
    end

    MP->>MP: "extra_asset_dirs = [package.mujoco_scene_path.parent]"
    MP->>LM: load_model(scene_xml, extra_asset_dirs)
    LM->>LM: "get_assets(extra_asset_dirs) → merge *.obj from pkg dir"
    LM-->>MP: (MjModel, MjData)

    alt scene_package present AND start_pos is default
        MP->>MP: override start_pos from _SCENE_PACKAGE_SPAWNS
    end
    MP->>MP: "data.qpos[0:3] = [x, y, z]"

    MP->>MP: enable scene_option.geomgroup[3] + m_viewer.opt.geomgroup[3]
    MP->>MP: simulation loop (render with scene_option)
Loading

Reviews (3): Last reviewed commit: "Only unhide geom group 3 for scene packa..." | Re-trigger Greptile

Comment on lines +51 to +53
_SCENE_PACKAGE_SPAWNS: dict[str, tuple[float, float]] = {
"dimos_office": (-2.0, 1.6),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing spawn override for lowpoly_tdm

The lowpoly_tdm map is cooked at 2× scale, meaning world coordinates are shifted relative to the default spawn. If the global mujoco_start_pos default (likely the origin) falls inside the scaled geometry, the robot will spawn clipped into a wall or floor. The test command in the PR description uses lowpoly_tdm without --start-pos, so a missing entry here silently falls back to the default coordinates. Consider adding a known-good entry for lowpoly_tdm (and optionally supermarket) if the default origin doesn't clear all geometry.

Comment on lines +45 to +46
for asset_dir in extra_asset_dirs:
mjx_env.update_assets(assets, epath.Path(str(asset_dir)), "*.obj")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Texture assets not loaded from scene package directories

get_assets only pulls *.obj from extra_asset_dirs. If any cooked MJCF scene file references texture files (e.g. *.png) relative to its own directory, those textures won't be in the assets dict and MuJoCo will fail to resolve them at load time. This is fine for purely collision-hull packages, but worth keeping in mind if future packages include visual textures in the MJCF.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Nabla7 added 2 commits July 8, 2026 22:39
Cooked scene packages put their collision hulls in geom group 3, which
default MjvOption hides — the depth-camera lidar saw only the person
mesh (group 0). Enable group 3 for the renderers and the viewer.
Verified: lidar_front depth coverage in the office goes from 17 to
9760 of 14400 pixels.
The go1/g1 robot MJCFs keep their collision capsules in group 3 too, so
enabling it unconditionally would render them in legacy rooms for no
benefit. Scope it to scene-package runs, which need it to see the scene.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant