WIP: Add Python runtime environments by TomCC7 · Pull Request #2704 · dimensionalOS/dimos

TomCC7 · 2026-07-02T23:19:14Z

Status update: This PR is kept as a draft/reference for the packaged-runtime exploration. We do not plan to merge this implementation as-is. The next implementation should use the unified Deployment Worker / Runtime Host path discussed in DIM-1123.

Problem

Python modules currently run in the coordinator environment, which forces module-specific dependency stacks into dimos run and makes modules with complex or conflicting dependencies difficult to deploy safely.

Closes N/A

Solution

Adds Python runtime environments for isolated module workers:

Runtime environment registration and class-keyed runtime placements on blueprints.
Runtime reconciliation before deployment slices launch workers, using locked/non-mutating Runtime Projects.
Python Runtime Worker launchers for venv/project-backed workers while preserving normal DimOS Python module semantics.
Contract/implementation split so the coordinator imports dependency-light Module Contracts and workers import Runtime Implementations.
Runtime project example under examples/dimos-demo-worker-module/ with a runnable script showing main venv → isolated worker venv execution.
OpenSpec artifacts and docs for the runtime environment model.

Runtime structure

flowchart TD
    BP[Blueprint] --> MC[ModuleCoordinator]
    BP --> RP[RuntimePlacement<br/>Contract -> runtime + implementation]
    BP --> RR[RuntimeEnvironmentRegistry<br/>runtime name -> Runtime Project]

    MC --> Recon[Runtime Reconciliation<br/>uv/pixi locked sync]
    Recon --> Venv[Prepared Runtime Project venv<br/>.venv/bin/python]

    MC --> WMP[WorkerManagerPython]

    subgraph Existing["Existing default Python module path"]
        WMP --> DefaultPool[Default Python Worker Pool]
        DefaultPool --> Fork[ForkserverWorkerLauncher]
        Fork --> PyWorkerA[python_worker._worker_entrypoint]
        PyWorkerA --> ContractA[Module class instantiated directly]
    end

    subgraph New["New runtime-placed Python module path"]
        WMP --> RuntimePool[Runtime Python Worker Pool<br/>one pool per runtime]
        RuntimePool --> Cmd[CommandWorkerLauncher]
        Cmd --> PreparedPython[prepared .venv/bin/python]
        PreparedPython --> Entrypoint[venv_worker_entrypoint]
        Entrypoint --> PyWorkerB[python_worker._worker_entrypoint]
        PyWorkerB --> ContractB[Module Contract identity]
        PyWorkerB --> Impl[Runtime Implementation subclass]
        Impl --> Deps[Runtime-only dependencies]
    end

    RP --> WMP
    RR --> WMP
    Venv --> Cmd

    ContractB -. "RPC / streams / refs use contract identity" .-> MC

Runtime placements do not replace the existing Python worker path. They add a second dispatch path inside WorkerManagerPython: unplaced modules continue through the default forkserver pool, while placed Module Contracts are routed to a runtime-specific Python Worker Pool launched from the Runtime Project's prepared venv.

Broader direction

This PR is the first concrete backend for a broader runtime-placement model: DimOS keeps a stable Module Contract in the blueprint graph, while execution can move into a backend-specific runtime.

flowchart TD
    Goal[End goal: remotely deployable, mixed-language DimOS modules]

    Seam[Stable seam introduced here<br/>Module Contract + RuntimePlacement + Python Worker Pool]

    Today[This PR<br/>Local locked Python Runtime Project]
    Remote[Future<br/>Remote Python Runtime Project<br/>SSH/container/robot-side venv]
    Native[Future<br/>Native Module Runtime<br/>Rust/C++ process + native SDK]

    Goal --> Remote
    Goal --> Native
    Remote --> Seam
    Native --> Seam
    Seam --> Today

    Today --> Contract[Dependency-light Module Contract]
    Today --> Impl[Runtime Implementation subclass]
    Today --> Venv[Prepared .venv/bin/python]
    Today --> SameSurface[Same DimOS RPC / streams / lifecycle surface]

The immediate feature is local Python dependency isolation. The longer-term value is the separation between what DimOS sees and where the module runs. That same seam is what remote Python workers and native module runtimes can reuse without changing blueprint-level module identity.

Key tradeoffs:

Runtime Implementations must subclass Module Contracts for this extraction; descriptor/structural compatibility is deferred.
Runtime Project reconciliation is automatic on deployment, but source-controlled project files/lockfiles are not mutated.
Native module runtime unification and remote deployment are out of scope.

How to Test

uv run python examples/dimos-demo-worker-module/demo_run_blueprint.py

Expected: prints the repo/main Python executable, a worker Python executable under examples/dimos-demo-worker-module/.venv/, runtime-only dependency output RuntimeDependency, and worker result: demo-runtime:HELLO FROM MAIN VENV.

Contributor License Agreement

I have read and approved the CLA.

mintlify · 2026-07-02T23:19:34Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
dimensional	🟢 Ready	View Preview	Jul 2, 2026, 11:22 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

codecov · 2026-07-02T23:26:02Z

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
2697	2	2695	71

View the full list of 2 ❄️ flaky test(s)

dimos.e2e_tests.test_dimsim_walk_forward::test_walk_forward

Flake rate in main: 22.12% (Passed 81 times, Failed 23 times)

Stack Traces | 208s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7707df16ea50>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7707dc16cb80>
human_input = <function human_input.<locals>.send_human_input at 0x7707dc16c4a0>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7707ddbd4770>

    @pytest.mark.self_hosted_large
    def test_walk_forward(lcm_spy, start_blueprint, human_input, dim_sim) -> None:
        start_blueprint(
            "run",
            "--disable",
            "spatial-memory",
            "--disable",
            "security-module",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        origin_x, origin_y = 1, 2
        dim_sim.set_agent_position(origin_x, origin_y)
    
        human_input("move forward 3 meter")
    
>       lcm_spy.wait_until_odom_position(origin_x + 3, origin_y, threshold=0.4, timeout=120)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x7707ddbd4770>
human_input = <function human_input.<locals>.send_human_input at 0x7707dc16c4a0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7707df16ea50>
origin_x   = 1
origin_y   = 2
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7707dc16cb80>

dimos/e2e_tests/test_dimsim_walk_forward.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7707dc16cae0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7707df16ea50>
        threshold  = 0.4
        timeout    = 120
        x          = 4
        y          = 2
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x7707ddbd5190: unset>
        fail_message = 'Failed to get to position x=4, y=2'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x7707dc16d1c0>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7707dc16cae0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7707df16ea50>
        timeout    = 120
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x7707ddbd5190: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=4, y=2

deadline   = 3492956.351089703
interval   = 0.1
message    = 'Failed to get to position x=4, y=2'
predicate  = <bound method Event.is_set of <threading.Event at 0x7707ddbd5190: unset>>
timeout    = 120

.../utils/testing/waiting.py:35: TimeoutError

dimos.perception.test_image_embedding.TestImageEmbedding::test_clip_embedding_initialization

Flake rate in main: 100.00% (Passed 0 times, Failed 1 times)

Stack Traces | 29.2s run time

request = <SubRequest 'monitor_threads' for <Function test_clip_embedding_initialization>>

    @pytest.fixture(autouse=True)
    def monitor_threads(request):
        # Capture threads before test runs
        test_name = request.node.nodeid
        with _seen_threads_lock:
            _before_test_threads[test_name] = {
                t.ident for t in threading.enumerate() if t.ident is not None
            }
    
        yield
    
        with _seen_threads_lock:
            before = _before_test_threads.get(test_name, set())
    
        # Threads intentionally left running for the whole process and cleaned up on
        # exit, so they don't count as per-test leaks.
        expected_persistent_thread_prefixes = [
            "Dask-Offload",
            # HuggingFace safetensors conversion thread - no user cleanup API
            # https://github..../transformers/issues/29513
            "Thread-auto_conversion",
        ]
    
        def live_new_threads():
            # Threads created during this test that are still running. A thread that
            # has already stopped is not a leak -- it's done, just not yet reaped
            # from threading's registry -- so we key on is_alive(), not presence.
            result = []
            for t in threading.enumerate():
                if t.ident is None or t.ident in before or t.name == "MainThread":
                    continue
                if any(t.name.startswith(prefix) for prefix in expected_persistent_thread_prefixes):
                    continue
                if t.is_alive():
                    result.append(t)
            return result
    
        # Some C extensions tear their callback threads down asynchronously, so a
        # thread can stay alive briefly after the test cleaned up its owner (notably
        # zenoh's pyo3-closure threads, freed shortly after the session is closed).
        # A single snapshot races that teardown and flags a thread that is about to
        # exit. Give new threads a grace period to drain; only ones that stay alive
        # are real leaks (a genuinely leaked thread never exits).
        deadline = time.monotonic() + 5.0
        leaked = live_new_threads()
        while leaked and time.monotonic() < deadline:
            time.sleep(0.02)
            leaked = live_new_threads()
    
        if not leaked:
            return
    
        with _seen_threads_lock:
            # Report each leaked thread only once across the session.
            truly_new = [t for t in leaked if t.ident not in _seen_threads]
            for t in leaked:
                _seen_threads.add(t.ident)
    
        if not truly_new:
            return
    
        thread_names = [t.name for t in truly_new]
    
>       pytest.fail(
            f"Non-closed threads created during this test. Thread names: {thread_names}. "
            "Please look at the first test that fails and fix that."
        )
E       Failed: Non-closed threads created during this test. Thread names: ['Thread-249']. Please look at the first test that fails and fix that.

before     = {126964115330752, 126964123723456, 126964140508864, 126964148901568, 126964165686976, 126964174079680, ...}
deadline   = 4472861.766745627
expected_persistent_thread_prefixes = ['Dask-Offload', 'Thread-auto_conversion']
leaked     = [<TMonitor(Thread-249, started daemon 126963997353664)>]
live_new_threads = <function monitor_threads.<locals>.live_new_threads at 0x7379e9a4a480>
request    = <SubRequest 'monitor_threads' for <Function test_clip_embedding_initialization>>
t          = <TMonitor(Thread-249, started daemon 126963997353664)>
test_name  = 'dimos/perception/test_image_embedding.py::TestImageEmbedding::test_clip_embedding_initialization'
thread_names = ['Thread-249']
truly_new  = [<TMonitor(Thread-249, started daemon 126963997353664)>]

dimos/conftest.py:273: Failed

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

…ixes Auto-fixes for cc/feat/venv-worker

greptile-apps · 2026-07-07T00:13:11Z

Greptile Summary

This PR adds isolated Python runtime environments for module workers. The main changes are:

Runtime environment registration and blueprint runtime placements.
Runtime reconciliation before worker launch.
Runtime-specific Python worker pools using prepared project virtualenvs.
Worker-side runtime implementation imports behind coordinator-visible contracts.
Demo project and usage docs for isolated worker dependencies.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
dimos/core/runtime_environment.py	Defines runtime projects and resolves launches to the prepared project Python.
dimos/core/coordination/worker_launcher.py	Adds subprocess and forkserver launchers for Python workers.
dimos/core/coordination/worker_manager_python.py	Adds runtime-specific worker pools and placement-aware deployment.
dimos/core/coordination/module_coordinator.py	Runs runtime reconciliation before deploying placed modules.
dimos/core/coordination/python_worker.py	Loads runtime implementation classes inside the worker process.

_{Reviews (3): Last reviewed commit: "fix: launch runtime workers from prepare..." | Re-trigger Greptile}

TomCC7 · 2026-07-09T19:45:05Z

+)
+```
+
+The v1 rules are concrete:


Suggested change

The v1 rules are concrete:

TomCC7 · 2026-07-09T22:33:50Z

+WorkerManagerPython     schedules the normal Python worker pool
+PythonWorker            controls one Python worker process
+WorkerManagerExternal   assigns external modules to machine workers
+ExternalWorker          controls one worker process on one machine


this should be controlling all the worker process on the single machine. since all modules it manages owns it's process there's no need to duplicate the worker process

TomCC7 · 2026-07-09T22:39:39Z

+
+### 3.1 Module contract shape
+
+`ExternalModule` is a declarative `Module` subclass. It adds an implementation reference but no build, process, watchdog, or transport behavior.


Suggested change

`ExternalModule` is a declarative `Module` subclass. It adds an implementation reference but no build, process, watchdog, or transport behavior.

`ExternalModule` is a declarative `Module` subclass. It adds an implementation reference but no build, process, watchdog, or transport behavior. It's similar to current `NativeModule` but more "contract only".

TomCC7 · 2026-07-09T22:40:25Z

+```
+
+```python
+class HeavyDetectorImpl(HeavyDetector):


Suggested change

class HeavyDetectorImpl(HeavyDetector):

import heavy_dependency

class HeavyDetectorImpl(HeavyDetector):

TomCC7 · 2026-07-09T22:41:48Z

+
+
+class MLSPlanner(ExternalModule):
+    implementation = FsPath("target/release/mls_planner")


can support both str and path here no issue, and no need to import Path as FsPath

TomCC7 · 2026-07-09T22:44:58Z

+    path: Out[Path]
+```
+
+The implementation type is sufficient:


should not determine with the type itself... we can just scan the module folder and see what's there. if it's python/ then it's python. can also use heuristics to distinguish path and python module, should be fairly simple

TomCC7 · 2026-07-09T22:45:20Z

+
+```text
+str    Python class import reference
+pathlib.Path   native executable


some indentation issue

TomCC7 · 2026-07-09T22:47:33Z

+- `local` always exists as an implicit target.
+- modules absent from `assignments` run on `local`.
+- an in-environment Python module, the most common module kind, cannot be assigned to a non-local target.
+- each target name identifies one execution machine in v1; aliases for the same machine are rejected.


Suggested change

- each target name identifies one execution machine in v1; aliases for the same machine are rejected.

- each target name identifies one execution machine in v1; aliases for the same machine are rejected.

One open question here is should we just extend blueprint to do this deployment assignment, or keep it separate and create a new registration system?

TomCC7 · 2026-07-09T22:49:43Z

+@dataclass(frozen=True)
+class DeploymentSpec:
+    blueprint: Blueprint
+    targets: Mapping[str, ExecutionTarget]


need api shape for execution target as well

TomCC7 · 2026-07-09T22:51:41Z

+    contract: type[ExternalModule]
+    implementation: str | FsPath
+    implementation_dir: Path
+    preset: ConventionPreset


what is this? should not just dump type with no explanation

TomCC7 · 2026-07-09T22:53:32Z

+Planning fails before mutation when:
+
+- an assignment references a target absent from `targets`,
+- an assignment references a module absent from the Blueprint,
+- an in-environment Python module is assigned remotely,
+- an `ExternalModule` has no discoverable implementation folder,
+- zero or multiple Convention Presets match,
+- a required lockfile, project file, source manifest, or build declaration is missing.


Suggested change

Planning fails before mutation when:

- an assignment references a target absent from `targets`,

- an assignment references a module absent from the Blueprint,

- an in-environment Python module is assigned remotely,

- an `ExternalModule` has no discoverable implementation folder,

- zero or multiple Convention Presets match,

- a required lockfile, project file, source manifest, or build declaration is missing.

too verbose

TomCC7 and others added 12 commits June 4, 2026 13:41

spec: openspec init

fe7a7de

chore: revert change to doc folder

76158b2

Merge branch 'main' into cc/feat/openspec

35c8b14

Merge branch 'main' into cc/feat/openspec

12d4346

Merge remote-tracking branch 'origin/main' into cc/feat/openspec

6cd2fd3

Merge branch 'main' into cc/feat/openspec

45f7f73

Merge remote-tracking branch 'origin/main' into cc/feat/openspec

86a600d

Merge branch 'main' into cc/feat/openspec

43fd853

Merge remote-tracking branch 'origin/main' into cc/feat/openspec

8394a61

Merge remote-tracking branch 'origin/main' into cc/feat/openspec

bae46c4

Merge remote-tracking branch 'origin/main' into cc/feat/openspec

4cf815e

feat: add python runtime environments

051c9b2

TomCC7 temporarily deployed to cachix July 2, 2026 23:19 — with GitHub Actions Inactive

TomCC7 changed the title ~~Add Python runtime environments~~ WIP: Add Python runtime environments Jul 2, 2026

mintlify Bot deployed to staging - docs July 2, 2026 23:22 View deployment

TomCC7 added 2 commits July 6, 2026 10:16

chore: slim demo runtime lockfile

08a9620

refactor: require project runtime environments

dd6c41d

TomCC7 commented Jul 6, 2026

View reviewed changes

Comment thread dimos/core/runtime_environment.py Outdated

Comment thread dimos/core/runtime_environment.py Outdated

Comment thread dimos/core/coordination/blueprints.py

Comment thread dimos/core/coordination/module_coordinator.py Outdated

refactor: move runtime routing into python worker manager

639852b

TomCC7 commented Jul 6, 2026

View reviewed changes

Comment thread dimos/core/coordination/module_coordinator.py Outdated

TomCC7 added 2 commits July 6, 2026 15:40

fix: type python worker manager access

9b81bd6

Merge main into venv worker branch

bc7d417

mintlify Bot deployed to staging - docs July 6, 2026 23:21 View deployment

TomCC7 and others added 5 commits July 6, 2026 16:46

fix: clean up worker launch internals

afd4ebc

fix: normalize demo worker imports

6fcc804

fix: isolate demo worker test state

f48aaff

fix: avoid private coordinator access in restart tests

13a9d10

Merge pull request #2751 from dimensionalOS/cc/feat/venv-worker-autof…

dbb484c

…ixes Auto-fixes for cc/feat/venv-worker

TomCC7 added 2 commits July 6, 2026 16:54

fix: make demo runtime script self-contained

9f5f70f

spec: remove

548dd9e

TomCC7 marked this pull request as ready for review July 7, 2026 00:10

TomCC7 requested review from leshy, mustafab0, paul-nechifor and spomichter as code owners July 7, 2026 00:10

greptile-apps Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/core/runtime_environment.py Outdated

Comment thread dimos/core/coordination/worker_launcher.py

fix: stabilize demo runtime worker CI

00e8710

greptile-apps Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/core/runtime_environment.py Outdated

Comment thread dimos/core/coordination/worker_launcher.py

fix: launch runtime workers from prepared venv

6c76868

dimos-bot mentioned this pull request Jul 7, 2026

Proposal: Module deployment as Grounded Blueprints #2758

Open

TomCC7 marked this pull request as draft July 7, 2026 03:27

doc: proposal

2a89ac6

mintlify Bot deployed to staging - docs July 9, 2026 02:28 View deployment

TomCC7 commented Jul 9, 2026

View reviewed changes

doc: update module deployment proposal

8a0dd5a

mintlify Bot deployed to staging - docs July 9, 2026 06:33 View deployment

proposal update

973d713

mintlify Bot deployed to staging - docs July 9, 2026 19:23 View deployment

TomCC7 commented Jul 9, 2026

View reviewed changes

proposal update

be12228

mintlify Bot deployed to staging - docs July 9, 2026 22:31 View deployment

TomCC7 commented Jul 9, 2026

View reviewed changes


		### 3.1 Module contract shape

		`ExternalModule` is a declarative `Module` subclass. It adds an implementation reference but no build, process, watchdog, or transport behavior.

	class HeavyDetectorImpl(HeavyDetector):
	import heavy_dependency
	class HeavyDetectorImpl(HeavyDetector):



		class MLSPlanner(ExternalModule):
		implementation = FsPath("target/release/mls_planner")

Uh oh!

Conversation

TomCC7 commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Runtime structure

Broader direction

How to Test

Contributor License Agreement

Uh oh!

mintlify Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 2 Tests Failed:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

TomCC7 commented Jul 2, 2026 •

edited

Loading

mintlify Bot commented Jul 2, 2026 •

edited

Loading

codecov Bot commented Jul 2, 2026 •

edited

Loading

greptile-apps Bot commented Jul 7, 2026 •

edited

Loading