Skip to content

bug: costs() always returns 0 — token usage tracked but costUsd never calculated correctly #67

Description

@shreyas-lyzr

Summary

Token counts are read correctly from the API response, but costs() always returns costUsd: 0 regardless of provider or model. Reported and investigated by @stealthwhizz.


Root cause analysis

There are three distinct bugs across the stack, each contributing to the same symptom.

Bug 1 — EMPTY_USAGE zeros out costs on the error path (agent.ts:333)

EMPTY_USAGE is a module-level constant applied exclusively inside handleRunFailure():

usage: EMPTY_USAGE,   // line 333 of agent.js

On successful runs, msg.usage is fully populated. On error/aborted runs, usage is silently zeroed — meaning costs() returns 0 even when tokens were consumed before the error. This is arguably intentional but is completely undocumented, so callers have no way to distinguish "nothing was billed" from "billing data was lost".

Bug 2 — calculateCost() in pi-ai silently emits 0 for unknown model IDs

calculateCost() depends on model.cost.input / model.cost.output being present in the model manifest. For custom, unrecognized, or manifest-less model IDs, these fields are zero or undefined, so cost.total = 0. sdk.ts faithfully reads that 0:

costUsd: msg.usage.cost?.total ?? 0,

There is no warning, no log, no throw — costs() silently returns 0 with real inputTokens / outputTokens. This is the primary symptom most users will hit.

Bug 3 — No cost calculation for OpenAI completions streaming

The openai-completions provider reads token counts from stream_options.include_usage but never calls calculateCost() or equivalent. Only openai-responses has the applyServiceTierPricing flow. So costUsd is always 0 for the completions API path regardless of model.


Affected files

File Issue
agent.ts (pi-agent-core) EMPTY_USAGE on error path — undocumented, misleading
sdk.ts (gitagent) Should warn/throw when inputTokens > 0 but costUsd == 0
openai-completions provider (pi-ai) Missing calculateCost() call after usage parsing

Proposed fixes

  1. sdk.ts — Add a guard: if inputTokens > 0 && costUsd === 0, emit a warning (or surface it in costs() metadata) so callers know cost data is unavailable rather than literally zero.

  2. pi-ai / calculateCost() — Return null (not 0) when pricing data is missing for a model ID, so callers can distinguish "calculated zero cost" from "no pricing data available". Log a warning with the unrecognized model ID.

  3. openai-completions provider — Call calculateCost() (or equivalent) after parsing usage tokens, consistent with the openai-responses path.

  4. handleRunFailure() — Document that EMPTY_USAGE is intentional on the error path, or accumulate partial usage before the failure and report it.


Reproduction

Any run using a custom model ID, or using the openai-completions API path — check that costs().totalCostUsd is non-zero after a successful multi-turn session. It will be 0.


Reported by @stealthwhizz.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions