Claude models missing from the pricing table are silently costed at $0

[TLXyloph]

Both benchmarks price Claude runs from a table keyed by exact model id (CLAUDE_PRICING_PER_1M[model] in bench-browser/src/usage.ts and bench-github/src/usage.ts). The table lists a handful of ids — claude-sonnet-4-6, claude-opus-4-6, claude-haiku-4-5-20251001, plus the bare sonnet/opus/haiku aliases.

Any Claude model that isn't literally one of those keys gets no entry, and the lookup returns undefined:

function getClaudePricing(model: string): ModelPricing | undefined {
  const entry = CLAUDE_PRICING_PER_1M[model];
  if (!entry) return undefined;
  ...
}

The caller then just skips costing entirely:

let totalCost = reportedCost;
if (!totalCost && inputTokens > 0) {
  const pricing = opts.model ? getClaudePricing(opts.model) : undefined;
  if (pricing) {
    totalCost = /* ... */;
  }
}

So the run is reported at $0.00 rather than erroring or warning. Token counts are still correct — only the cost column is wrong, which makes it look plausible rather than broken.

Models that hit this today include claude-opus-4-5, any new point release, and vendor-prefixed ids such as us.anthropic.claude-sonnet-4-5-v1:0. Nothing stops you using them: --model is taken as args.model ?? DEFAULT_MODEL with no allowlist and passed straight through to the agent CLI, so the run itself succeeds normally.

Why it's silent

The local pricing table is only consulted when the Claude JSONL has no result event. When a result event is present, total_cost_usd from Claude is used and the table is bypassed entirely — so on the common path the bug is invisible regardless of which model you pass.

The table is the sole cost source exactly when the run has no result event: the agent crashed, hit a timeout, was killed, or otherwise ended early. Those are the runs where you most want the numbers, and they're the ones that silently read $0.

Reproduction

1. Take a Claude JSONL transcript with assistant-message usage but no result event (i.e. a crashed/interrupted run).
2. Call parseClaudeJsonl(raw, { model: "claude-opus-4-5" }).
3. total_cost_usd comes back 0, while input_tokens / output_tokens are populated.

Passing claude-sonnet-4-6 instead gives a correct non-zero cost, which is what makes the failure mode easy to miss.

Why it matters

Cost is one of the headline columns the benchmarks exist to produce. A silent $0 doesn't look like an error — it looks like a cheap run, and it will happily average into published results, making a model appear free next to the ones that happen to be in the table. Since the benchmarks accept any model, this hits anyone benchmarking outside the default set.

Anthropic prices per family tier rather than per point release, so resolving the model id onto its family (opus / sonnet / haiku) and pricing from that would cover aliases, dated and undated ids, future point releases and vendor-prefixed ids, and would need no code change on each new release. Failing loudly on a genuinely unknown model would also beat costing it at $0.

Happy to open a PR if that approach sounds right.

[TLXyloph]

Correction to the last paragraph of my report: "Anthropic prices per family tier rather than per point release" is not accurate, and I don't want it to mislead anyone implementing a fix.

Anthropic has re-priced across point releases within a family. Opus is exactly where:

• Opus 4.1 and earlier: $15 / $75 per 1M input/output
• Opus 4.5, 4.6, 4.7, 4.8: $5 / $25

So a single flat opus tier cannot be correct for both. Two further corrections while I'm here:

• The table's current haiku rate ($0.80 / $4.00) is the retired Haiku 3.5 price. Haiku 4.5 is $1.00 / $5.00.
• Family-word matching alone does not close the hole. claude-fable-5 and claude-mythos-5 are current models whose ids contain no family word, so they would still resolve to no pricing entry and still silently cost $0 — the very bug this issue is about.

None of this changes the bug report itself: the exact-id lookup does silently cost $0 on result-less runs, and that is still worth fixing. It does change the remedy. Resolving by family is a reasonable direction, but it needs the current rates (opus $5/$25, haiku $1/$5), entries for the family-word-less ids, and — most importantly — a model it genuinely cannot price should be loud rather than $0, since that is the actual defect. A family table without that last part just swaps a silent $0 for a plausible-looking wrong number, which is arguably harder to catch.

Apologies for the noise; I'd rather correct it here than have someone implement against a wrong premise.