Summary
client.messages.count_tokens() (and its async counterpart await client.messages.count_tokens()) makes a real API call to POST /v1/messages/count_tokens but produces zero Braintrust spans. This method was added to the anthropic Python SDK in v0.84.0 (2026-02-25). The current version pinned as latest in this repo is anthropic==0.116.0.
Braintrust docs status: not_found — The Anthropic integration docs mention Anthropic as a supported provider but do not document count_tokens instrumentation.
Root cause
The AnthropicIntegration wraps Anthropic.__init__ and AsyncAnthropic.__init__ to return proxy objects (TracedAnthropic / TracedAsyncAnthropic). The messages property on these proxies returns a Messages (or AsyncMessages) wrapper class that only defines create(), stream(), and a batches property.
Messages extends Wrapper, whose __getattr__ delegates unknown attribute access to the underlying (unwrapped) self.__messages object. So when a caller invokes client.messages.count_tokens(...), Python finds no count_tokens method on the proxy class and silently falls through to the real, uninstrumented anthropic.resources.messages.Messages.count_tokens. No span is created.
The same applies to AsyncMessages.
What needs to be instrumented
| Method |
SDK class |
Already traced? |
client.messages.count_tokens(model, messages, ...) |
Messages |
No |
await client.messages.count_tokens(model, messages, ...) |
AsyncMessages |
No |
The response is a MessageTokensCount object with an input_tokens field. A useful span would record:
- input:
messages, system, tools, tool_choice
- output:
input_tokens count
- metadata:
model, provider: "anthropic"
- metrics: wall-clock latency
Upstream sources
Local repo files inspected
py/src/braintrust/integrations/anthropic/tracing.py — Messages class defines create() and stream() only; no count_tokens; __getattr__ from Wrapper delegates all other attribute access to the unwrapped objectpy/src/braintrust/integrations/anthropic/_utils.py — Wrapper.__getattr__ confirmed as the delegation mechanismpy/src/braintrust/integrations/anthropic/patchers.py — only patches Anthropic.__init__ and AsyncAnthropic.__init__py/src/braintrust/integrations/anthropic/integration.py — min_version = "0.48.0", latest pin resolves to anthropic==0.116.0 (from py/pyproject.toml line 327)- Full repo grep for
count_tokens across py/src/braintrust/ — zero matches
Summary
client.messages.count_tokens()(and its async counterpartawait client.messages.count_tokens()) makes a real API call toPOST /v1/messages/count_tokensbut produces zero Braintrust spans. This method was added to theanthropicPython SDK in v0.84.0 (2026-02-25). The current version pinned aslatestin this repo isanthropic==0.116.0.Braintrust docs status:
not_found— The Anthropic integration docs mention Anthropic as a supported provider but do not documentcount_tokensinstrumentation.Root cause
The
AnthropicIntegrationwrapsAnthropic.__init__andAsyncAnthropic.__init__to return proxy objects (TracedAnthropic/TracedAsyncAnthropic). Themessagesproperty on these proxies returns aMessages(orAsyncMessages) wrapper class that only definescreate(),stream(), and abatchesproperty.MessagesextendsWrapper, whose__getattr__delegates unknown attribute access to the underlying (unwrapped)self.__messagesobject. So when a caller invokesclient.messages.count_tokens(...), Python finds nocount_tokensmethod on the proxy class and silently falls through to the real, uninstrumentedanthropic.resources.messages.Messages.count_tokens. No span is created.The same applies to
AsyncMessages.What needs to be instrumented
client.messages.count_tokens(model, messages, ...)Messagesawait client.messages.count_tokens(model, messages, ...)AsyncMessagesThe response is a
MessageTokensCountobject with aninput_tokensfield. A useful span would record:messages,system,tools,tool_choiceinput_tokenscountmodel,provider: "anthropic"Upstream sources
POST /v1/messages/count_tokenshttps://platform.claude.com/docs/en/api/messages-count-tokens
Messages.count_tokens()source:https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/resources/messages/messages.py
messages.count_tokens()https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md
Local repo files inspected
py/src/braintrust/integrations/anthropic/tracing.py—Messagesclass definescreate()andstream()only; nocount_tokens;__getattr__fromWrapperdelegates all other attribute access to the unwrapped objectpy/src/braintrust/integrations/anthropic/_utils.py—Wrapper.__getattr__confirmed as the delegation mechanismpy/src/braintrust/integrations/anthropic/patchers.py— only patchesAnthropic.__init__andAsyncAnthropic.__init__py/src/braintrust/integrations/anthropic/integration.py—min_version = "0.48.0",latestpin resolves toanthropic==0.116.0(frompy/pyproject.tomlline 327)count_tokensacrosspy/src/braintrust/— zero matches