Skip to content

feat: blueprint namespaces#2725

Open
paul-nechifor wants to merge 3 commits into
mainfrom
paul/feat/namespaces
Open

feat: blueprint namespaces#2725
paul-nechifor wants to merge 3 commits into
mainfrom
paul/feat/namespaces

Conversation

@paul-nechifor

@paul-nechifor paul-nechifor commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Problem

  • A coordinator could only run one instance of each module class.
  • Stream topics, RPC topics and TF frames were global names, so one blueprint could not control several robots of the same type.

Closes #2491

Solution

  • Add instance name to module instances in the module coordinator, so you can run multiple instances.
  • Add namespace(prefix, *blueprints, expose=...). It prefixes instance names, stream topics, RPC topics and TF frames of everything inside. Streams listed in expose stay global so shared modules can consume them. Module references resolve inside the namespace first, then outward.
  • Per instance config works as -o robot0/go2connection.ip=10.0.0.5 or env ROBOT0_GO2CONNECTION__IP=....
  • Simulation support: with --simulation each robot gets its own MuJoCo process and the IPs are ignored. Fixed the hardcoded TF frames in GO2Connection (base_link, camera_link, camera_optical) to honor the namespace prefix. The parent world frame stays shared so all robots hang off one tree root.

How to Test

ROBOT_IPS=10.0.0.1,10.0.0.2 dimos --simulation run unitree-go2-multi-teleop

Two MuJoCo windows and two teleop windows open. Click one teleop window and drive with WASD. Only that robot moves. dimos topic echo /robot0/odom shows the namespaced topics. On real robots use their IPs and drop --simulation.

Contributor License Agreement

  • I have read and approved the CLA.

@mintlify

mintlify Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
dimensional 🟢 Ready View Preview Jul 4, 2026, 1:35 AM

@greptile-apps

greptile-apps Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds blueprint namespaces for running multiple module instances together. The main changes are:

  • Namespaced instance names, stream topics, RPC topics, and TF frames.
  • Per-instance config keys for CLI and environment overrides.
  • Coordinator and porcelain updates for multiple instances of the same module class.
  • Multi-robot Unitree Go2 blueprints and simulation support.

Confidence Score: 4/5

This is close, but these issues should be fixed before merging.

  • Remote porcelain lookup can fail for class-name access when multiple namespaced instances share a class.
  • The new multi-Go2 blueprints still validate robot IPs during import.
  • Missing fleet config can surface as an import-time crash instead of a normal CLI configuration error.

dimos/porcelain/remote_module_source.py; dimos/robot/unitree/go2/blueprints/basic/unitree_go2_multi.py; dimos/robot/unitree/go2/blueprints/basic/unitree_go2_multi_teleop.py

Important Files Changed

Filename Overview
dimos/porcelain/local_module_source.py Local module lookup now supports exact instance names and reports ambiguous class-name matches.
dimos/porcelain/remote_module_source.py Remote descriptors use instance RPC names, but class-name lookup does not match the local ambiguity behavior.
dimos/robot/unitree/go2/blueprints/basic/unitree_go2_multi.py The multi-Go2 blueprint creates namespaced robot instances, but reads required fleet config during import.
dimos/robot/unitree/go2/blueprints/basic/unitree_go2_multi_teleop.py The teleop variant creates one namespaced teleop stack per robot, but also reads required fleet config during import.

Reviews (2): Last reviewed commit: "use multiple keyboard teleops" | Re-trigger Greptile

Comment thread dimos/porcelain/local_module_source.py Outdated
if cls.__name__ == name:
for instance_key, proxy in self._coordinator._deployed_modules.items():
cls = self._coordinator._instance_classes[instance_key]
if name in (instance_key, cls.__name__):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Class Lookup Picks Arbitrary Instance

When two namespaced instances share a class, get_module("NamedModule") matches both entries through cls.__name__ and returns the first match in coordinator dict order. Local porcelain callers can receive the wrong robot's proxy instead of the same ambiguity error the coordinator raises for class-based lookup.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +33 to +35

unitree_go2_multi = autoconnect(
*[namespace(f"robot{i}", GO2Connection.blueprint(ip=ip)) for i, ip in enumerate(_ips)],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Empty Fleet Crashes Import

When ROBOT_IPS is unset or empty, _ips is [] and the module still evaluates autoconnect(*[]) during blueprint import. That reaches reduce(...) with no blueprints and crashes before the CLI can report a clear configuration error for the new unitree-go2-multi blueprint.

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
2709 2 2707 71
View the full list of 3 ❄️ flaky test(s)
dimos.e2e_tests.test_dimsim_spatial_memory::test_go_to_the_bed

Flake rate in main: 20.18% (Passed 91 times, Failed 23 times)

Stack Traces | 553s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x79a8b6f2cc20>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x79a8b4defba0>
human_input = <function human_input.<locals>.send_human_input at 0x79a8b55e04a0>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x79a8b614e420>
explore_house = <function explore_house.<locals>.explore at 0x79a8b55e0a40>

    @pytest.mark.self_hosted_large
    def test_go_to_the_bed(lcm_spy, start_blueprint, human_input, dim_sim, explore_house) -> None:
        start_blueprint(
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        explore_house()
    
        human_input("go to the bed")
    
>       lcm_spy.wait_until_odom_position(-3.567, -1.332, threshold=2, timeout=180)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x79a8b614e420>
explore_house = <function explore_house.<locals>.explore at 0x79a8b55e0a40>
human_input = <function human_input.<locals>.send_human_input at 0x79a8b55e04a0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x79a8b6f2cc20>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x79a8b4defba0>

dimos/e2e_tests/test_dimsim_spatial_memory.py:32: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x79a8b4dee5c0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x79a8b6f2cc20>
        threshold  = 2
        timeout    = 180
        x          = -3.567
        y          = -1.332
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x79a8b614f890: unset>
        fail_message = 'Failed to get to position x=-3.567, y=-1.332'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x79a8b55e0c20>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x79a8b4dee5c0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x79a8b6f2cc20>
        timeout    = 180
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x79a8b614f890: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=-3.567, y=-1.332

deadline   = 3581602.917584083
interval   = 0.1
message    = 'Failed to get to position x=-3.567, y=-1.332'
predicate  = <bound method Event.is_set of <threading.Event at 0x79a8b614f890: unset>>
timeout    = 180

.../utils/testing/waiting.py:35: TimeoutError
dimos.e2e_tests.test_dimsim_walk_forward::test_walk_forward

Flake rate in main: 23.64% (Passed 84 times, Failed 26 times)

Stack Traces | 206s run time
lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7412f163b050>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7412dd4307c0>
human_input = <function human_input.<locals>.send_human_input at 0x7412dd4300e0>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7412de962690>

    @pytest.mark.self_hosted_large
    def test_walk_forward(lcm_spy, start_blueprint, human_input, dim_sim) -> None:
        start_blueprint(
            "run",
            "--disable",
            "spatial-memory",
            "--disable",
            "security-module",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        origin_x, origin_y = 1, 2
        dim_sim.set_agent_position(origin_x, origin_y)
    
        human_input("move forward 3 meter")
    
>       lcm_spy.wait_until_odom_position(origin_x + 3, origin_y, threshold=0.4, timeout=120)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7412de962690>
human_input = <function human_input.<locals>.send_human_input at 0x7412dd4300e0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7412f163b050>
origin_x   = 1
origin_y   = 2
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7412dd4307c0>

dimos/e2e_tests/test_dimsim_walk_forward.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7412dd47ede0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7412f163b050>
        threshold  = 0.4
        timeout    = 120
        x          = 4
        y          = 2
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x7412de963ad0: unset>
        fail_message = 'Failed to get to position x=4, y=2'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x7412dd430ae0>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7412dd47ede0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7412f163b050>
        timeout    = 120
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x7412de963ad0: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=4, y=2

deadline   = 3502392.07328155
interval   = 0.1
message    = 'Failed to get to position x=4, y=2'
predicate  = <bound method Event.is_set of <threading.Event at 0x7412de963ad0: unset>>
timeout    = 120

.../utils/testing/waiting.py:35: TimeoutError
dimos.perception.test_image_embedding.TestImageEmbedding::test_clip_embedding_initialization

Flake rate in main: 21.43% (Passed 11 times, Failed 3 times)

Stack Traces | 27.2s run time
request = <SubRequest 'monitor_threads' for <Function test_clip_embedding_initialization>>

    @pytest.fixture(autouse=True)
    def monitor_threads(request):
        # Capture threads before test runs
        test_name = request.node.nodeid
        with _seen_threads_lock:
            _before_test_threads[test_name] = {
                t.ident for t in threading.enumerate() if t.ident is not None
            }
    
        yield
    
        with _seen_threads_lock:
            before = _before_test_threads.get(test_name, set())
    
        # Threads intentionally left running for the whole process and cleaned up on
        # exit, so they don't count as per-test leaks.
        expected_persistent_thread_prefixes = [
            "Dask-Offload",
            # HuggingFace safetensors conversion thread - no user cleanup API
            # https://github..../transformers/issues/29513
            "Thread-auto_conversion",
        ]
    
        def live_new_threads():
            # Threads created during this test that are still running. A thread that
            # has already stopped is not a leak -- it's done, just not yet reaped
            # from threading's registry -- so we key on is_alive(), not presence.
            result = []
            for t in threading.enumerate():
                if t.ident is None or t.ident in before or t.name == "MainThread":
                    continue
                if any(t.name.startswith(prefix) for prefix in expected_persistent_thread_prefixes):
                    continue
                if t.is_alive():
                    result.append(t)
            return result
    
        # Some C extensions tear their callback threads down asynchronously, so a
        # thread can stay alive briefly after the test cleaned up its owner (notably
        # zenoh's pyo3-closure threads, freed shortly after the session is closed).
        # A single snapshot races that teardown and flags a thread that is about to
        # exit. Give new threads a grace period to drain; only ones that stay alive
        # are real leaks (a genuinely leaked thread never exits).
        deadline = time.monotonic() + 5.0
        leaked = live_new_threads()
        while leaked and time.monotonic() < deadline:
            time.sleep(0.02)
            leaked = live_new_threads()
    
        if not leaked:
            return
    
        with _seen_threads_lock:
            # Report each leaked thread only once across the session.
            truly_new = [t for t in leaked if t.ident not in _seen_threads]
            for t in leaked:
                _seen_threads.add(t.ident)
    
        if not truly_new:
            return
    
        thread_names = [t.name for t in truly_new]
    
>       pytest.fail(
            f"Non-closed threads created during this test. Thread names: {thread_names}. "
            "Please look at the first test that fails and fix that."
        )
E       Failed: Non-closed threads created during this test. Thread names: ['Thread-249']. Please look at the first test that fails and fix that.

before     = {126589532243648, 126589540636352, 126589557421760, 126589565814464, 126589582599872, 126589590992576, ...}
deadline   = 4484377.144444904
expected_persistent_thread_prefixes = ['Dask-Offload', 'Thread-auto_conversion']
leaked     = [<TMonitor(Thread-249, started daemon 126589515458240)>]
live_new_threads = <function monitor_threads.<locals>.live_new_threads at 0x73224e523100>
request    = <SubRequest 'monitor_threads' for <Function test_clip_embedding_initialization>>
t          = <TMonitor(Thread-249, started daemon 126589515458240)>
test_name  = 'dimos/perception/test_image_embedding.py::TestImageEmbedding::test_clip_embedding_initialization'
thread_names = ['Thread-249']
truly_new  = [<TMonitor(Thread-249, started daemon 126589515458240)>]

dimos/conftest.py:273: Failed

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Comment thread docs/usage/cli.md
4. Blueprint definition → `.global_config(simulation="mujoco")`
5. CLI flag → `dimos --simulation run ...`

Environment variables and `.env` values must be prefixed with `DIMOS_`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why'd we change this? Seemed like a good pattern to me

@paul-nechifor paul-nechifor force-pushed the paul/feat/namespaces branch from 184dac8 to a283283 Compare July 7, 2026 04:20
@paul-nechifor paul-nechifor changed the title WIP: feat: blueprint namespaces feat: blueprint namespaces Jul 7, 2026
@paul-nechifor paul-nechifor marked this pull request as ready for review July 7, 2026 17:04
descriptors = self._coord.call("list_modules")
self._descriptors = {d.class_name: d for d in descriptors}
# rpc_name is empty when talking to an older daemon.
self._descriptors = {(d.rpc_name or d.class_name): d for d in descriptors}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Remote lookup diverges RemoteModuleSource now keys descriptors only by rpc_name, but it does not keep the class-name fallback and ambiguity check that the local source uses. With remote instances such as robot0/go2connection and robot1/go2connection, a porcelain access like get_module("GO2Connection") raises KeyError instead of reporting that the class name is ambiguous. Remote callers should get the same clear ambiguity behavior as local callers, while exact instance-name lookups continue to work.

Comment on lines +35 to +39
_ips = global_config.processed_robot_ips

unitree_go2_multi = autoconnect(
*[namespace(f"robot{i}", GO2Connection.blueprint(ip=ip)) for i, ip in enumerate(_ips)],
).global_config(n_workers=max(2, 2 * len(_ips)), robot_model="unitree_go2")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Import still fails This blueprint still reads global_config.processed_robot_ips during module import. When dimos run unitree-go2-multi is called without ROBOT_IPS or --robot-ips, the import raises ValueError before the CLI can load the blueprint and report the missing fleet config cleanly. The IP check should run during blueprint loading or build-time config handling instead of at import time.

Comment on lines +34 to +45
_ips = global_config.processed_robot_ips

unitree_go2_multi_teleop = autoconnect(
*[
namespace(
f"robot{i}",
GO2Connection.blueprint(ip=ip),
KeyboardTeleop.blueprint(),
)
for i, ip in enumerate(_ips)
],
).global_config(n_workers=max(2, 2 * len(_ips)), robot_model="unitree_go2")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Teleop import fails The teleop multi blueprint has the same import-time config read. If ROBOT_IPS or --robot-ips is missing, selecting unitree-go2-multi-teleop raises from global_config.processed_robot_ips while importing the module, before the CLI can handle the missing configuration. This makes the registered blueprint unavailable instead of producing a controlled config error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Blueprint Namespaces proposal for multi-embodiment & teleop & multi-blueprint

2 participants