Runtime reliability for AI agents. Structural and semantic detection, runtime prevention, native root cause, and one-click fixes.

If Dunetrace helps you, consider giving it a ⭐ on top right, it helps others find the project.

AI agents fail silently:

• ✓ API returns 200   ✓ Logs are clean
• ✗ Agent called the same tool 12 times, burned $10, and gave the user a wrong answer

Tracers answer "what happened?" — after you already know it broke. Dunetrace answers "is something breaking right now?" and fires an alert in 15 seconds, using zero-LLM structural checks that run in-path with sub-500μs overhead.

Dunetrace covers the full agent reliability lifecycle, not just one slice of it:

Where tracers fit in: if you already run Langfuse, LangSmith, or Braintrust, Dunetrace pulls their evaluation results in alongside its own (pillar 3) rather than asking you to switch — see docs/integrations/external-evaluation.md. What no tracer does is pillar 4: none of them can stop a run mid-flight, because none of them run in-path.

1. Start the backend

git clone https://github.com/dunetrace/dunetrace
cd dunetrace && cp .env.example .env
docker compose -f docker-compose.ghcr.yml up -d
pip install -r requirements.txt

2. Install the SDK

pip install dunetrace                       # Python
npm install dunetrace                       # Node.js / TypeScript

3. Instrument your agent

Python

from dunetrace import Dunetrace
import openai

dt = Dunetrace()
dt.init(agent_id="support-agent")   # auto-instruments installed clients (OpenAI, Anthropic, LangChain, CrewAI, httpx, requests)

@dt.agent("support-agent", model="gpt-4o")
def my_agent(question: str) -> str:
    resp = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": question}],
    )
    return resp.choices[0].message.content   # LLM + tool calls tracked automatically, no manual hooks

TypeScript / Node.js

import { Dunetrace } from "dunetrace";
import OpenAI from "openai";

const dt     = new Dunetrace();
const openai = dt.wrapOpenAI(new OpenAI());

await dt.run("my-agent", { model: "gpt-4o" }, async (run) => {
  await openai.chat.completions.create({ model: "gpt-4o", messages });
  run.finalAnswer();
});

Try the built-in failure scenarios

cd packages/sdk-py

python examples/basic_agent.py                          # No LLM calls
SCENARIO=tool_loop python examples/langchain_agent.py   # TOOL_LOOP via LangChain
SCENARIO=failures python examples/decorator_agent.py    # TOOL_LOOP, RETRY_STORM, RAG_EMPTY_RETRIEVAL

Open the dashboard: http://localhost:3000

28 detectors run on every completed run — no configuration, no LLM. A few of the main ones:

Each alert includes: what fired, why it matters, a concrete fix, and a rate context line (first occurrence / recurring / systemic).

→ docs/detectors.md for the full list of 28 detectors

Multi-agent systems — instrument each agent as its own dt.run() and Dunetrace auto-links them into a delegation graph (parent_run_id is threaded automatically for nested runs). Two detectors read that graph: DELEGATION_LOOP (agents cycling without converging) and HANDOFF_CONTEXT_LOSS (a handoff dropping the parent's context). → docs/multi-agent.md

Agent memory — instrument what an agent writes to and reads from its own memory (run.memory_written() / memory_read(), or automatically for LangGraph/CrewAI memory via dt.auto_instrument()), and MEMORY_POISONING flags adversarial content persisted into it. → docs/memory.md

Custom detectors — write a detector in plain English. Dunetrace translates it to a structured condition set, runs it in shadow mode against real traffic, and lets you review the fire rate before any alert fires. In the dashboard: Config → Custom detectors → Add detector.

Detector packs — opt-in detector bundles for a specific class of agent, activated per org. The voice pack adds 9 detectors for real-time voice agents (dt.enable_pack("voice")). Built-in detectors always run; packs only add to them and start in shadow mode.

→ detector packs · voice pack · wiring a voice framework

For failure modes no structural check can catch — did the agent hallucinate, did it finish the task, did it solve the wrong task, is the user going in circles. Post-hoc (never in your agent's request path), sampling-based, disabled by default. Ships seven DeepEval-backed evaluators — four run-level (hallucination, task completion, task-understanding failure, off-topic drift) and three conversation-level (user frustration, confusion loops, sycophancy) — plus false-positive management (confidence floors, grouping, feedback loop, second-opinion for high-stakes findings). Each calibrated before ship (see scripts/calibration/).

SEMANTIC_WORKER_ENABLED=true

→ docs/semantic-evaluation.md

Live at http://localhost:3000. Auto-refreshes every 15s.

→ docs/dashboard.md

Slack and generic webhook (PagerDuty, Linear, custom).

SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
SLACK_MIN_SEVERITY=LOW   # LOW | MEDIUM | HIGH | CRITICAL

A weekly digest (Monday 9am UTC) summarises top failure types and systemic patterns. Enable with DIGEST_ENABLED=true.

→ docs/alerts.md

Root-cause analysis is native — no third-party tracer required. Click Explain + on any alert and Dunetrace analyzes the run's own stored events and returns a specific cause and fix. Every fix is one of two kinds:

• Policy fixes (tool loops, retry storms, runaway step counts) → Dunetrace applies a runtime guardrail directly, no code change needed
• Prompt / code fixes → a diff you copy in, or a one-click draft PR on GitHub for code/infra changes

Fix effectiveness is tracked automatically.

Runtime guardrails that fire mid-run — before a failure propagates.

dt.add_policy(
    name="cap tool calls",
    condition={"trigger": "tool_call_count", "operator": "gt", "value": 5},
    action={"type": "stop"},
)
dt.add_policy(
    name="cost cap",
    condition={"trigger": "cost_usd", "operator": "gt", "value": 0.50},
    action={"type": "switch_model", "params": {"model": "gpt-4o-mini"}},
)

Policies can also be created in the dashboard and fetched automatically by the SDK (60s TTL).

Human-in-the-loop approvals — gate a risky tool (wiring money, deleting data) behind human approval. A require_approval policy blocks the tool call until someone approves in Slack or the dashboard, or it times out (fail-closed: a timeout blocks the tool). No agent code changes — the gate fires on the existing tool-call hook.

dt.add_policy(
    name="approve-wires",
    condition={"trigger": "before_tool_call", "operator": "eq", "value": "wire_money"},
    action={"type": "require_approval", "params": {"timeout_s": 300}},
)

→ docs/policies.md · docs/approvals.md

Query agent signals directly from Claude Code, Cursor, or Codex — without leaving your editor.

pip install dunetrace-mcp

Claude Code: registered automatically in ~/.claude.json after pip install dunetrace-mcp. Restart Claude Code to load.

Cursor: add .cursor/mcp.json to your project root:

{
  "mcpServers": {
    "dunetrace": {
      "command": "dunetrace-mcp",
      "env": {
        "DUNETRACE_API_URL": "http://localhost:8002",
        "DUNETRACE_API_KEY": "dt_dev_test"
      }
    }
  }
}

→ docs/mcp-server.md

Agent Code
  └─► Dunetrace SDK        (raw content → ingest events)
        └─► Ingest API      (POST /v1/ingest → Postgres)
                ├─► Detector          (poll → 28 detectors → signals)
                ├─► Semantic Worker   (optional — poll → DeepEval → signals)
                ├─► Integrations      (optional — pull Langfuse/LangSmith/Braintrust)
                ├─► Alerts            (poll → explain → Slack / webhook)
                └─► Customer API      (runs, signals, explanations → dashboard)

→ docs/architecture.md for the full service breakdown

Evaluation & tracing

• External evaluation (Langfuse, LangSmith, Braintrust, generic push)
• Semantic evaluation (Dunetrace's own DeepEval-backed layer)

Fix & workflow

• GitHub App (automated draft PRs)
• Slack & Linear alerts
• Coding agents — Claude Code, Cursor, Codex (MCP)

Fork, branch, change, make test, PR. For larger changes (new integrations, architecture changes), open an issue first.

New here? See CONTRIBUTING.md for setup and workflow, browse the good first issues (most are "add one detector"), and follow the step-by-step Adding a detector guide.

Requires Python 3.11+, Node.js 22+, Docker + Docker Compose.

dunetrace@gmail.com

Apache 2.0