Summary
Token counts are read correctly from the API response, but costs() always returns costUsd: 0 regardless of provider or model. Reported and investigated by @stealthwhizz.
Root cause analysis
There are three distinct bugs across the stack, each contributing to the same symptom.
Bug 1 — EMPTY_USAGE zeros out costs on the error path (agent.ts:333)
EMPTY_USAGE is a module-level constant applied exclusively inside handleRunFailure():
usage: EMPTY_USAGE, // line 333 of agent.js
On successful runs, msg.usage is fully populated. On error/aborted runs, usage is silently zeroed — meaning costs() returns 0 even when tokens were consumed before the error. This is arguably intentional but is completely undocumented, so callers have no way to distinguish "nothing was billed" from "billing data was lost".
Bug 2 — calculateCost() in pi-ai silently emits 0 for unknown model IDs
calculateCost() depends on model.cost.input / model.cost.output being present in the model manifest. For custom, unrecognized, or manifest-less model IDs, these fields are zero or undefined, so cost.total = 0. sdk.ts faithfully reads that 0:
costUsd: msg.usage.cost?.total ?? 0,
There is no warning, no log, no throw — costs() silently returns 0 with real inputTokens / outputTokens. This is the primary symptom most users will hit.
Bug 3 — No cost calculation for OpenAI completions streaming
The openai-completions provider reads token counts from stream_options.include_usage but never calls calculateCost() or equivalent. Only openai-responses has the applyServiceTierPricing flow. So costUsd is always 0 for the completions API path regardless of model.
Affected files
| File |
Issue |
agent.ts (pi-agent-core) |
EMPTY_USAGE on error path — undocumented, misleading |
sdk.ts (gitagent) |
Should warn/throw when inputTokens > 0 but costUsd == 0 |
openai-completions provider (pi-ai) |
Missing calculateCost() call after usage parsing |
Proposed fixes
-
sdk.ts — Add a guard: if inputTokens > 0 && costUsd === 0, emit a warning (or surface it in costs() metadata) so callers know cost data is unavailable rather than literally zero.
-
pi-ai / calculateCost() — Return null (not 0) when pricing data is missing for a model ID, so callers can distinguish "calculated zero cost" from "no pricing data available". Log a warning with the unrecognized model ID.
-
openai-completions provider — Call calculateCost() (or equivalent) after parsing usage tokens, consistent with the openai-responses path.
-
handleRunFailure() — Document that EMPTY_USAGE is intentional on the error path, or accumulate partial usage before the failure and report it.
Reproduction
Any run using a custom model ID, or using the openai-completions API path — check that costs().totalCostUsd is non-zero after a successful multi-turn session. It will be 0.
Reported by @stealthwhizz.
Summary
Token counts are read correctly from the API response, but
costs()always returnscostUsd: 0regardless of provider or model. Reported and investigated by @stealthwhizz.Root cause analysis
There are three distinct bugs across the stack, each contributing to the same symptom.
Bug 1 —
EMPTY_USAGEzeros out costs on the error path (agent.ts:333)EMPTY_USAGEis a module-level constant applied exclusively insidehandleRunFailure():On successful runs,
msg.usageis fully populated. On error/aborted runs, usage is silently zeroed — meaningcosts()returns 0 even when tokens were consumed before the error. This is arguably intentional but is completely undocumented, so callers have no way to distinguish "nothing was billed" from "billing data was lost".Bug 2 —
calculateCost()in pi-ai silently emits 0 for unknown model IDscalculateCost()depends onmodel.cost.input/model.cost.outputbeing present in the model manifest. For custom, unrecognized, or manifest-less model IDs, these fields are zero or undefined, socost.total = 0.sdk.tsfaithfully reads that 0:There is no warning, no log, no throw —
costs()silently returns 0 with realinputTokens/outputTokens. This is the primary symptom most users will hit.Bug 3 — No cost calculation for OpenAI completions streaming
The
openai-completionsprovider reads token counts fromstream_options.include_usagebut never callscalculateCost()or equivalent. Onlyopenai-responseshas theapplyServiceTierPricingflow. SocostUsdis always 0 for the completions API path regardless of model.Affected files
agent.ts(pi-agent-core)EMPTY_USAGEon error path — undocumented, misleadingsdk.ts(gitagent)inputTokens > 0butcostUsd == 0openai-completionsprovider (pi-ai)calculateCost()call after usage parsingProposed fixes
sdk.ts— Add a guard: ifinputTokens > 0 && costUsd === 0, emit a warning (or surface it incosts()metadata) so callers know cost data is unavailable rather than literally zero.pi-ai /
calculateCost()— Returnnull(not0) when pricing data is missing for a model ID, so callers can distinguish "calculated zero cost" from "no pricing data available". Log a warning with the unrecognized model ID.openai-completionsprovider — CallcalculateCost()(or equivalent) after parsing usage tokens, consistent with theopenai-responsespath.handleRunFailure()— Document thatEMPTY_USAGEis intentional on the error path, or accumulate partial usage before the failure and report it.Reproduction
Any run using a custom model ID, or using the
openai-completionsAPI path — check thatcosts().totalCostUsdis non-zero after a successful multi-turn session. It will be 0.Reported by @stealthwhizz.