[bot] Google GenAI: `models.count_tokens()` not instrumented

[braintrust-bot]

Summary

client.models.count_tokens() and its async counterpart await client.aio.models.count_tokens() make a real API call to the Gemini token-counting endpoint but produce zero Braintrust spans. The method is available in google-genai==2.10.0 (the current latest pin in this repo) and is documented in the official Google GenAI SDK.

This creates a gap alongside the already-instrumented models.generate_content(), models.embed_content(), and models.generate_images(): any user who calls count_tokens to estimate cost or validate prompt size before generation gets no tracing data.

Braintrust docs status: not_found — The Braintrust tracing guide lists Gemini as a supported provider but does not mention count_tokens instrumentation.

Root cause

The GoogleGenAIIntegration registers patchers for Models._generate_content, Models.generate_content_stream, Models.embed_content, and Models.generate_images (and their async counterparts). No patcher exists for Models.count_tokens or its async variant.

Because the integration uses module-level function patching (not a proxy wrapper), adding count_tokens support requires adding a new ModelsCountTokensPatcher and AsyncModelsCountTokensPatcher — the same pattern already used for the four existing model methods.

What needs to be instrumented

SDK method	Sync / Async	Already traced?
client.models.count_tokens(model, contents, ...)	Sync	No
await client.aio.models.count_tokens(model, contents, ...)	Async	No

The response is a CountTokensResponse with total_tokens and, depending on the request, cache_tokens_details. A useful span should record:

• input: contents, generation_config (if provided), system_instruction, tools
• output: total_tokens
• metadata: model, provider: "google_genai"
• metrics: wall-clock latency

Upstream sources

• Google GenAI Python SDK v2.10.0 (current latest pin):
  https://pypi.org/project/google-genai/2.10.0/
• Gemini API token counting reference:
  https://ai.google.dev/api/tokens
• Usage example from SDK docs:

  response = client.models.count_tokens(
      model='gemini-2.5-flash',
      contents='why is the sky blue?'
  )
  print(response.total_tokens)

Local repo files inspected

• py/src/braintrust/integrations/google_genai/patchers.py — defines ModelsGenerateContentPatcher, ModelsGenerateContentStreamPatcher, ModelsEmbedContentPatcher, ModelsGenerateImagesPatcher and async equivalents; no ModelsCountTokensPatcher
• py/src/braintrust/integrations/google_genai/integration.py — GoogleGenAIIntegration.patchers tuple; count_tokens not listed; min_version = "1.30.0", latest resolves to google-genai==2.10.0
• py/src/braintrust/integrations/google_genai/tracing.py — wrapper functions for generate, stream, embed, and image; no count_tokens wrapper
• py/src/braintrust/integrations/google_genai/cassettes/ — no cassette matching *count_tokens* in any version directory
• Full repo grep for count_tokens across py/src/braintrust/ — zero matches