Skip to content

[bot] Anthropic: messages.parse() not instrumented #555

Description

@braintrust-bot

Summary

client.messages.parse() makes a full model inference call to the Anthropic API but produces zero Braintrust spans. This structured-output helper was added to the anthropic Python SDK in v0.87.0 (2026-03-31). The current version pinned as latest in this repo is anthropic==0.116.0.

messages.parse() is the Anthropic equivalent of OpenAI's chat.completions.parse(), which is instrumented in this repo. The gap creates an asymmetry: users who migrate to parse() for structured outputs lose all tracing visibility.

Braintrust docs status: not_found — The Anthropic integration docs do not mention messages.parse() or structured output tracing.

Root cause

The Messages (and AsyncMessages) proxy class in py/src/braintrust/integrations/anthropic/tracing.py explicitly defines only create(), stream(), and a batches property. The Wrapper.__getattr__ delegates all other attribute access to the underlying unwrapped anthropic.resources.messages.Messages object.

When a caller invokes client.messages.parse(...), Python finds no parse method on the proxy and falls through to the real, uninstrumented SDK method. No span is created.

Unlike create() with stream=True (which the Messages proxy intercepts), parse() is an entirely separate code path in the Anthropic SDK that does not call through the proxy's create().

What needs to be instrumented

Method SDK class Already traced?
client.messages.parse(model, messages, output_format, ...) Messages No
await client.messages.parse(model, messages, output_format, ...) AsyncMessages No

parse() returns a ParsedMessage[T] wrapping the raw Message response. A useful span should record:

  • input: messages, system, tools
  • output: the parsed structured content
  • metadata: model, output_format type, provider: "anthropic"
  • metrics: token counts (from the embedded Message.usage), wall-clock latency

Precedent

OpenAI's chat.completions.parse() is already instrumented via _chat_completion_parse_wrapper (see py/src/braintrust/integrations/openai/tracing.py), as is responses.parse() via _responses_parse_wrapper. The same pattern can be applied here.

Upstream sources

Local repo files inspected

  • py/src/braintrust/integrations/anthropic/tracing.pyMessages class defines create() and stream() only; no parse() method; __getattr__ from Wrapper delegates all other access to the unwrapped object
  • py/src/braintrust/integrations/anthropic/_utils.pyWrapper.__getattr__ confirmed as the delegation mechanism
  • py/src/braintrust/integrations/openai/tracing.py_chat_completion_parse_wrapper and _responses_parse_wrapper as precedent for parse instrumentation
  • py/src/braintrust/integrations/openai/patchers.py_cc_parse_sync, _cc_parse_async as precedent patcher pattern
  • py/src/braintrust/integrations/anthropic/integration.pymin_version = "0.48.0", latest resolves to anthropic==0.116.0
  • Full repo grep for messages.parse and def parse across py/src/braintrust/integrations/anthropic/ — zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions