Harden subagent safety boundaries: symlink rejection, bounded reads, worker loop hardening (continuation of PR #1)#2
Draft
bgoertzel-sing wants to merge 132 commits into
Conversation
…re orchestration ThreadKeeper decouples reasoning quality from reasoning frequency: a cheap persistent local control loop holds the thread, a local worker loop iterates cheaply, and cloud specialists are invoked only for hard subproblems under an explicit token budget. Built as an extension to OmegaClaw-Core (synced to current upstream main), changing none of its existing behavior. Adds: - src/subagent.py + delegate skill (skills.metta): bounded, governed delegation primitive — the "cloud specialist"/"worker" node mechanism. - src/threadkeeper_budget.py: cost-awareness seam — per-loop token accounting and an escalation decision against a configurable budget; auditable trail (memory/usage.jsonl + memory/escalations.jsonl), ISO/IEC 42001-friendly. - threadkeeper.config.yaml: the four-node mesh + budget threshold in one place; models are clearly-labelled examples, every role swappable, keys by env-var name only (no secrets). - docs/architecture.md (+ reproducible architecture.png), docs/subagent-design.md, reference-skills-subagent.md, tutorial-09-subagents.md. - memory/personas-subagent/ scaffolding with researcher.json.example; real configs and runtime logs gitignored. - README rewrite (pitch -> problem -> four-node solution -> "improvement to OmegaClaw" as extension) and HACKATHON.md (BGI Sprint I, team ThreadKeeper). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The working hackathon demo, ported from runtime into the repo so it reproduces:
- channels/local.py: the WebUI dashboard — a live 4-engine mesh (control /
local-worker / cloud-specialist-A / cloud-specialist-B) showing per-engine
token + cost, a counterfactual "all-frontier vs ThreadKeeper" savings
headline, an "OmegaClaw DNA" panel proving the MeTTa/Hyperon substrate is
present + intact, and a live reasoning ("thinking") pane tailing history.metta.
Plus the agent avatar route.
- src/subagent.py: worker dispatch hardened — LOCAL Ollama workers called via
the native /api/chat path with think:false (reasoning models return empty
content via /v1 on current Ollama builds); CLOUD workers via /v1. Worker
token usage now logged to memory/usage.jsonl so delegated work shows on the
mesh's Local Worker tile. No longer depends on the AIProvider class.
- demo/: bulk document-summarization demo — chunks a doc and routes each chunk
to the local granite worker (high token volume, trivial reasoning, $0). The
doc itself explains ThreadKeeper's thesis, so the summaries reinforce the pitch.
- memory/personas-subagent/: example worker (local granite) + specialist
(GLM 5.2 cloud) persona configs. Keys by env-var name only; endpoints
genericized to localhost.
- PITCH.md: the 3-minute pitch script (problem -> mesh -> live demo -> savings).
- docs/disaster-recovery-and-migration.md: memory persistence + cross-provider
migration analysis (the embedding-provider trap; how learned memory stays
portable under ThreadKeeper).
No secrets committed: all credentials referenced by env-var name only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docs/recursive-self-improvement.md + docs/xi-interview.md: a session where the agent (隙) diagnosed its own pin-spam failure, designed the fix (loop-detection governor), had it built to spec in the channel wrapper it insisted on (outside its own control), and watched its own loop quiet — ~183 repetitions → 1 clean message. Plus: it re-derived ThreadKeeper's thesis, critiqued its own free-cloud control loop, and had its hardware pick (Qwen2.5-14B, not 9B) experimentally confirmed. OmegaClaw's stated RSI goal, demonstrated, with a human-in-the-loop governor put first by the agent's own design. README pointer added. - HACKATHON.md: soften "synced to current upstream" → "branched from a recent upstream main (June 2026)" — accurate (the fork is ~36 commits behind current upstream; this avoids an overclaim a reviewer could flag). No secrets: all credentials by env-var name only; transcript scanned clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… $0, hard->GLM+DeepSeek). Shows 'escalate only when needed' with all tiers, fractions of a cent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-min pitch Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Reasoning-engines dashboard now COLLAPSED by default (click to expand) so the chat is the star — for non-technical users who just want to talk to the agent. - Token counters are now ALL-TIME by default (append-only usage.jsonl): tiles only climb, survive restarts, never reset/lose data. - Honest savings: control loop (MiniMax) counts as cloud, not 'free' — shows on-prem%/cloud%/paid%; savings compared at NORMAL (non-promo) rates; the hackathon free tier noted separately. - 5-tile Local/Cloud column layout (Granite + Qwen local workers shown distinctly for reasoning diversity); full-width fills the screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-history ledger - Primary Reasoning Engine tile is now DYNAMIC: reads PRIMARY_LLM_MODEL from the live container env, so it shows whatever holds the thread right now (Qwen2.5-14B today; flips automatically when the control loop is swapped). - LOCAL column now reflects CURRENT configs (Qwen2.5-14B on .248, Gemma 4 12B on .41) instead of stale historical worker labels. - Full model history moved into its own collapsible '📜 Full history' menu below the current-config dashboard: every model ever run, all-time totals, 'created N days ago', current primary row highlighted. - All-time ledger sourced from the append-only usage.jsonl (counts never reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ThreadKeeper's core decision — whether to escalate to a cloud specialist — now lives in src/escalation.metta as MeTTa pattern-matching rules (tk-escalate), evaluated through OmegaClaw's own MeTTa runtime (PeTTa). threadkeeper_budget.py becomes the seam: it supplies live facts (spend, ceiling, soft threshold, local iterations, hard-flag) and executes the verdict, but the routing logic is now symbolic and agent-readable/-rewritable via the existing read-file/write-file skills — the same self-modification property OpenCog Hyperon is built around. - src/escalation.metta: the 4-rule policy as Atomspace rules; numbers stay in threadkeeper.config.yaml (the .metta owns logic only). - threadkeeper_budget.py: _MettaPolicy loads the policy into PeTTa once and evaluates it MeTTa-first; on any failure (host/CI without the runtime) it falls back to the identical Python rules, preserving the never-raises contract. Hardened PeTTa import to self-discover <PeTTa>/python on sys.path. - tests/test_escalation_metta_parity.py: proves the MeTTa verdict equals the Python spec across every branch + boundary (64/64), all served via [metta]; skips cleanly where PeTTa is absent. - README/HACKATHON: 'policy lives in MeTTa, agent-rewritable' — the honest answer to 'it's just Python orchestration.' Behavior is identical (parity-verified); this is an architecture/transparency improvement, not a policy change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The (delegate ...) skill now consults ThreadKeeper's MeTTa budget policy before dispatching to a CLOUD specialist. A denied delegation is refused and returns the [metta]-tagged reason to the parent loop instead of spending; LOCAL delegations (Ollama on .41/.248) are free and pass through ungated. The gate fails OPEN — if the policy can't be evaluated, the call proceeds — so a broken governor never silently halts the agent. subagent.py: - _persona_is_cloud(): classify a delegation as cloud vs local (explicit node_role, else base_url heuristic — the same one _call_subagent_llm uses). - _escalation_gate(): for cloud, construct a BudgetTracker and call should_escalate(hard=True); returns (allowed, reason). Self-discovers the budget module beside the overlay or in repo src/. Never raises. - dispatch(): consult the gate right after persona load; on deny, return the refusal digest before resolving the provider (saves the spend). threadkeeper_budget.py — two fixes the live wiring exposed (without them the gate would deny forever on the real agent log): - _iter_records: records written WITHOUT a thread_id (the worker/loop usage log format + the dashboard's own records) now count toward the default thread, instead of being filtered out (which made spent_tokens always 0). - local_iterations: records without an explicit node_role are classified local-vs-cloud by model name, so the real usage log (which omits node_role) counts cheap iterations correctly. Verified live: a real dispatch() to the cloud 'specialist' persona is gated by the Atomspace policy and the verdict lands in memory/escalations.jsonl; local delegations pass ungated. Parity test still 64/64. Deployment: agent_10 relaunched via the new launch-agent-10.sh (mounts escalation.metta + threadkeeper_budget.py into the agent's src/); env + run spec captured for reboot-survival. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clarify that the mesh dashboard + budget tracker read ONE agent's usage.jsonl: per-agent (not fleet-wide), tokens dominated by re-sent loop context, costs at example rates, savings is a counterfactual. The dashboard answers 'what did this agent route where, at illustrative rates' — the architecture story, not a billing console. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
added 30 commits
July 8, 2026 02:05
_tool_read_file and _tool_append_file used plain open() after _resolve_workspace_path (which uses realpath for containment). Add _open_workspace_file_read helper that opens with O_NOFOLLOW and validates the fd is a regular non-symlink file, closing a TOCTOU symlink-swap gap between path resolution and the actual file read. Add focused tests for the helper, symlink-escape rejection, and regular-file acceptance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a large hardening continuation of the ThreadKeeper subagent safety work. It builds on the foundations in draft PR #1 (
agent/threadkeeper-safety-floor) and adds 112 additional commits on top of that branch.Key changes (123 commits total vs
main)Symlink rejection across all file-bearing paths:
Bounded reads for all untrusted/external file content:
Worker loop safety:
OMEGACLAW_SUBAGENT_DISPATCH_TIMEOUT_S).Subagent call hardening:
Budget-aware orchestration:
threadkeeper_budget.py: configurable hybrid OmegaClaw mesh for budget-aware orchestration.should_escalatewired into the live delegate path with cloud-gating.Verification
186 tests passed(latest run with mock pytest gate)py_compilecleangit diff --checkcleanRelationship to PR #1
PR #1 (
agent/threadkeeper-safety-floor) established the initial safety floor. This branch (agent/threadkeeper-hardening-next) supersedes and extends it with 112 additional commits. If preferred, this PR can replace PR #1.Notes