feat(hooks): PostModelCall telemetry hook + SessionEnd usage totals

.changeset/post-model-call-telemetry.md

@@ -0,0 +1,11 @@
+---
+'@openrouter/agent': minor
+---
+
+Add a `PostModelCall` lifecycle hook and aggregate usage totals on `SessionEnd` — the telemetry primitives for tracing and benchmark consumers.
+
+`PostModelCall` fires once per completed model response, on **every** request the agent loop makes: the initial request, each tool-round follow-up, the empty-final retry, the `allowFinalResponse` final turn, and approval-resume requests. The payload carries `responseId` (the OpenRouter generation id, deep-linkable), `model`, `durationMs` (dispatch to fully materialized response, including stream consumption), `turnType` (`'initial' | 'resume' | 'tool_round' | 'final' | 'retry'`), `turnNumber`, and a normalized `usage` block (`inputTokens`, `outputTokens`, `totalTokens`, `cachedTokens`, `reasoningTokens`, `cost?`) when the server reported usage accounting. Purely observational: handlers cannot mutate or block.
+
+`SessionEnd` now carries an optional `totalUsage` aggregate (`modelCalls` plus the summed usage fields, with `cost` present when any call reported one) whenever at least one model call completed during the run.
+
+New exported types: `PostModelCallPayload`, `ModelCallUsage`, `SessionUsageTotals`.

packages/agent/README.md

@@ -313,14 +313,22 @@ const result = callModel(client, { model, input, tools, hooks });
 | `PermissionRequest` | When a tool requires approval, before pausing for the human gate | `decision: 'allow'` skips the gate (the tool runs once via the normal round), `'deny'` synthesizes a rejected result without executing, `'ask_user'` (default) falls through to the approval flow. Last handler wins. Payload includes a `riskLevel` derived from the approval gate's shape (`'high'` for tool- or call-level functions, `'medium'` for blanket `true`) |
 | `Stop` | When a `stopWhen` condition halts the loop (`reason: 'max_turns'`) | `forceResume: true` continues the loop (capped at 3 consecutive overrides without tool progress — a bare `forceResume` that changes no state will typically re-trigger the stop condition immediately and burn through the cap, so pair it with `appendPrompt` or external state the stop condition observes); `appendPrompt` injects a user message for the next turn (honored independently of `forceResume`). Blocked/rejected tool outputs count as progress for the cap: the model receives that feedback, and each round costs a full request, so the loop cannot spin hot |
 | `SessionStart` | Once per run, before the initial request. `config` summarizes the session (`hasTools`, `hasApproval`, `hasState`) | none (void) |
-| `SessionEnd` | Once per run, on every exit path — completion, approval pause, interruption, error, and the no-tools streaming paths. `reason` is `'complete' \| 'error' \| 'max_turns' \| 'user'` | none (void) |
+| `SessionEnd` | Once per run, on every exit path — completion, approval pause, interruption, error, and the no-tools streaming paths. `reason` is `'complete' \| 'error' \| 'max_turns' \| 'user'`. When at least one model call completed, `totalUsage` aggregates tokens/cost across all of them (`modelCalls`, `inputTokens`, `outputTokens`, `totalTokens`, `cachedTokens`, `reasoningTokens`, and `cost` when the server reported it) | none (void) |
+| `PostModelCall` | Once per completed model response, on **every** request the loop makes — initial, each tool-round follow-up, the empty-final retry, the `allowFinalResponse` final turn, and approval-resume requests. Payload: `responseId` (the OpenRouter generation id), `model`, `durationMs` (dispatch → fully materialized response, including stream consumption), `turnType` (`'initial' \| 'resume' \| 'tool_round' \| 'final' \| 'retry'`), `turnNumber`, and `usage` (`inputTokens`, `outputTokens`, `totalTokens`, `cachedTokens`, `reasoningTokens`, `cost?`) when the server reported usage accounting. Purely observational — the telemetry primitive for tracing/benchmark consumers: one span per model call | none (void) |
 
 Notes on lifecycle pairing: `SessionEnd` only fires when a matching
 `SessionStart` succeeded, and at most once per run. Pending async hook work is
 always drained on teardown — including on paths that skip `SessionStart`,
 such as resuming from a tool approval. A throwing `SessionEnd` handler never
 masks the run's original error (teardown failures are logged as warnings).
 
+On no-tools streaming paths the initial response is only materialized when the
+stream is consumed, so `PostModelCall` for that response fires during session
+teardown (before `SessionEnd`). A stream that fails or errors before producing
+a materialized response emits no `PostModelCall`; a `response.incomplete`
+response (e.g. truncated at `max_output_tokens`) **does** emit — it carries a
+real generation id and consumed tokens. Note `usage.cost` is only present when
+the request had usage accounting enabled server-side.
 Every handler receives `(payload, context)` — `context` carries the
 `sessionId` (the single source of session identity; payloads do not repeat
 it), the `hookName`, and an `AbortSignal` for cooperative cancellation. The

packages/agent/src/index.ts

@@ -127,14 +127,17 @@ export type {
   HooksManagerOptions,
   InlineHookConfig,
   LifecycleHookContext,
+  ModelCallUsage,
   PermissionRequestPayload,
   PermissionRequestResult,
+  PostModelCallPayload,
   PostToolUseFailurePayload,
   PostToolUsePayload,
   PreToolUsePayload,
   PreToolUseResult,
   SessionEndPayload,
   SessionStartPayload,
+  SessionUsageTotals,
   StopPayload,
   StopResult,
   ToolMatcher,

packages/agent/src/lib/hooks-schemas.ts

@@ -18,6 +18,7 @@ export const HookName = {
   PermissionRequest: 'PermissionRequest',
   SessionStart: 'SessionStart',
   SessionEnd: 'SessionEnd',
+  PostModelCall: 'PostModelCall',
 } as const;
 
 export type HookName = (typeof HookName)[keyof typeof HookName];
@@ -113,18 +114,51 @@ export const SessionStartPayloadSchema = z4.object({
   config: z4.record(z4.string(), z4.unknown()).optional(),
 });
 
+const ModelCallUsageSchema = z4.object({
+  inputTokens: z4.number(),
+  outputTokens: z4.number(),
+  totalTokens: z4.number(),
+  cachedTokens: z4.number(),
+  reasoningTokens: z4.number(),
+  cost: z4.number().optional(),
+});
+
+export type ModelCallUsage = Readonly<z4.infer<typeof ModelCallUsageSchema>>;
+
 export type SessionStartPayload = Readonly<z4.infer<typeof SessionStartPayloadSchema>>;
 
+const SessionUsageTotalsSchema = ModelCallUsageSchema.extend({
+  modelCalls: z4.number(),
+});
 export const SessionEndPayloadSchema = z4.object({
   reason: z4.enum([
     'user',
     'error',
     'max_turns',
     'complete',
   ]),
+  totalUsage: SessionUsageTotalsSchema.optional(),
+});
+
+export const PostModelCallPayloadSchema = z4.object({
+  sessionId: z4.string(),
+  responseId: z4.string(),
+  model: z4.string(),
+  durationMs: z4.number(),
+  turnType: z4.enum([
+    'initial',
+    'resume',
+    'tool_round',
+    'final',
+    'retry',
+  ]),
+  turnNumber: z4.number(),
+  usage: ModelCallUsageSchema.optional(),
 });
 
 export type SessionEndPayload = Readonly<z4.infer<typeof SessionEndPayloadSchema>>;
+export type PostModelCallPayload = Readonly<z4.infer<typeof PostModelCallPayloadSchema>>;
+export type SessionUsageTotals = Readonly<z4.infer<typeof SessionUsageTotalsSchema>>;
 
 //#endregion
 
@@ -238,6 +272,10 @@ export const BUILT_IN_HOOKS: Record<HookName, HookDefinition> = {
     payload: SessionEndPayloadSchema,
     result: VoidResultSchema,
   },
+  PostModelCall: {
+    payload: PostModelCallPayloadSchema,
+    result: VoidResultSchema,
+  },
 };
 
 export const BUILT_IN_HOOK_NAMES = new Set(Object.keys(BUILT_IN_HOOKS));

packages/agent/src/lib/hooks-types.ts

@@ -2,6 +2,7 @@ import * as z4 from 'zod/v4';
 import type {
   PermissionRequestPayload,
   PermissionRequestResult,
+  PostModelCallPayload,
   PostToolUseFailurePayload,
   PostToolUsePayload,
   PreToolUsePayload,
@@ -151,14 +152,17 @@ export interface HooksManagerOptions {
 // and the runtime validation is structurally impossible). Re-exported here
 // so the public import surface is unchanged.
 export type {
+  ModelCallUsage,
   PermissionRequestPayload,
   PermissionRequestResult,
+  PostModelCallPayload,
   PostToolUseFailurePayload,
   PostToolUsePayload,
   PreToolUsePayload,
   PreToolUseResult,
   SessionEndPayload,
   SessionStartPayload,
+  SessionUsageTotals,
   StopPayload,
   StopResult,
   UserPromptSubmitPayload,
@@ -206,6 +210,11 @@ export interface BuiltInHookDefinitions {
     /** Observation-only hook: handlers have no meaningful result. */
     result: undefined;
   };
+  PostModelCall: {
+    payload: PostModelCallPayload;
+    /** Observation-only hook: handlers have no meaningful result. */
+    result: undefined;
+  };
 }
 
 //#endregion