Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/ai-sdk-v7-compat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"evlog": minor
---

Add AI SDK v7 compatibility for `evlog/ai`. `createEvlogIntegration()` now implements both v6 hooks (`onToolCallFinish`, `onFinish`) and v7 hooks (`onToolExecutionEnd`, `onEnd`, `onEmbedEnd`, `onAbort`, `onError`). On v7, embeddings are auto-captured via `onEmbedEnd` when telemetry is enabled, and abort/error lifecycle events are written to the wide event. Pass the integration via `telemetry.integrations` (v7) or `experimental_telemetry.integrations` (v6). Exports a new `EvlogTelemetry` type.
3 changes: 1 addition & 2 deletions apps/docs/app/components/features/FeatureAiSdk.vue
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,7 @@ function setView(view: 'without' | 'with') {
<span class="text-violet-400">const</span> result = <span class="text-amber-400">streamText</span>({
<span class="text-sky-400">model</span>: ai.<span class="text-amber-400">wrap</span>(<span class="text-emerald-400">'anthropic/claude-sonnet-4.6'</span>),
messages,
<span class="text-sky-400">experimental_telemetry</span>: {
<span class="text-sky-400">isEnabled</span>: <span class="text-violet-400">true</span>,
<span class="text-sky-400">telemetry</span>: {
<span class="text-sky-400">integrations</span>: [<span class="text-amber-400">createEvlogIntegration</span>(ai)],
},
})</code></pre>
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/content/4.use-cases/2.ai-sdk/01.overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Add AI observability to my app with evlog.
- Create an AI logger with createAILogger(log) where log is your request logger
- Wrap your model with ai.wrap('anthropic/claude-sonnet-4.6') and pass it to generateText, streamText, etc.
- Token usage, tool calls, streaming metrics, and errors are captured automatically into the wide event
- For deeper observability (tool execution timing, total generation wall time), add createEvlogIntegration(ai) to experimental_telemetry.integrations
- For deeper observability (tool execution timing, total generation wall time), add createEvlogIntegration(ai) to telemetry.integrations
- For embedding calls, use ai.captureEmbed({ usage, model, dimensions, count }) after embed() or embedMany()
- For cost estimation, pass a cost map: createAILogger(log, { cost: { 'claude-sonnet-4.6': { input: 3, output: 15 } } })
- Works with all frameworks: Nuxt, Express, Hono, Fastify, NestJS, Elysia, standalone
Expand Down
6 changes: 3 additions & 3 deletions apps/docs/content/4.use-cases/2.ai-sdk/04.metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Subscribe to metadata updates. The callback fires every time the underlying stat
- Once per step in multi-step agent runs
- Once per `captureEmbed` call
- On model errors
- On `createEvlogIntegration`'s `onFinish`
- On `createEvlogIntegration`'s `onEnd` (v7) or `onFinish` (v6)

Each invocation receives a fresh snapshot. Returns an unsubscribe function. Subscriber errors are isolated and never break the AI flow.

Expand Down Expand Up @@ -162,7 +162,7 @@ Every field that may show up under `ai.*`:
| `ai.msToFinish` | Stream timing | Total stream duration (streaming only) |
| `ai.tokensPerSecond` | Computed | Output tokens per second (streaming only) |
| `ai.error` | Error capture | Error message if a model call fails |
| `ai.tools` | `TelemetryIntegration` | Per-tool `{ name, durationMs, success, error? }` (requires `createEvlogIntegration`) |
| `ai.totalDurationMs` | `TelemetryIntegration` | Total generation wall time (requires `createEvlogIntegration`) |
| `ai.tools` | Telemetry integration | Per-tool `{ name, durationMs, success, error? }` (requires `createEvlogIntegration`) |
| `ai.totalDurationMs` | Telemetry integration | Total generation wall time (requires `createEvlogIntegration`) |
| `ai.embedding` | `captureEmbed` | `{ model?, tokens, dimensions?, count? }` β€” embedding metadata |
| `ai.estimatedCost` | Computed | Estimated cost in dollars (requires `cost` option) |
17 changes: 10 additions & 7 deletions apps/docs/content/4.use-cases/2.ai-sdk/05.telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ links:
variant: subtle
---

`createAILogger` covers tokens, model info, and streaming metrics. For deeper observability β€” per-tool execution timing, success/failure tracking, and total generation wall time β€” add `createEvlogIntegration()` on top. It implements the AI SDK's `TelemetryIntegration` interface and captures data middleware alone cannot see.
`createAILogger` covers tokens, model info, and streaming metrics. For deeper observability β€” per-tool execution timing, success/failure tracking, and total generation wall time β€” add `createEvlogIntegration()` on top. It implements the AI SDK telemetry interface (v6 `TelemetryIntegration`, v7 `Telemetry`) and captures data middleware alone cannot see.

On AI SDK v7, the integration also auto-captures embeddings via `onEmbedEnd`, records stream aborts via `onAbort`, and surfaces unrecoverable errors via `onError`.

## Combined with middleware (recommended)

Expand All @@ -34,8 +36,7 @@ export default defineEventHandler(async (event) => {
const result = await generateText({
model: ai.wrap('anthropic/claude-sonnet-4.6'),
tools: { getWeather, searchDB },
experimental_telemetry: {
isEnabled: true,
telemetry: {
integrations: [createEvlogIntegration(ai)],
},
})
Expand Down Expand Up @@ -80,8 +81,7 @@ const integration = createEvlogIntegration(log)

const result = await generateText({
model: somePreWrappedModel,
experimental_telemetry: {
isEnabled: true,
telemetry: {
integrations: [integration],
},
})
Expand All @@ -91,8 +91,11 @@ const result = await generateText({

| Data | Source | Description |
|------|--------|-------------|
| `ai.tools[]` | `onToolCallFinish` | Per-tool `name`, `durationMs`, `success`, and `error` (if failed) |
| `ai.totalDurationMs` | `onStart` β†’ `onFinish` | Total wall time from generation start to completion |
| `ai.tools[]` | `onToolExecutionEnd` (v7) / `onToolCallFinish` (v6) | Per-tool `name`, `durationMs`, `success`, and `error` (if failed) |
| `ai.totalDurationMs` | `onStart` β†’ `onEnd` (v7) / `onFinish` (v6) | Total wall time from generation start to completion |
| `ai.embedding` | `onEmbedEnd` (v7) | Auto-captured from `embed()` / `embedMany()` when telemetry is enabled |
| `ai.finishReason: 'abort'` | `onAbort` (v7) | Set when a streaming generation is aborted |
| `ai.error` | `onAbort` / `onError` (v7) | Abort reason or unrecoverable generation error |

The middleware captures tokens, model info, and streaming metrics. The integration captures tool execution timing. Together, they give you complete AI observability.

Expand Down
5 changes: 2 additions & 3 deletions apps/docs/skills/review-logging-patterns/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -968,8 +968,7 @@ const agent = new ToolLoopAgent({
model: ai.wrap('anthropic/claude-sonnet-4.6'),
tools: { searchWeb, queryDatabase },
stopWhen: stepCountIs(5),
experimental_telemetry: {
isEnabled: true,
telemetry: {
integrations: [createEvlogIntegration(ai)],
},
})
Expand Down Expand Up @@ -1014,7 +1013,7 @@ Anti-patterns to detect:
| Manual token tracking in `onFinish` | `ai.wrap()` β€” middleware captures automatically |
| `console.log('tokens:', result.usage)` | `ai.wrap()` β€” structured `ai.*` fields in wide event |
| No AI observability | Add `createAILogger(log)` + `ai.wrap()` |
| No tool execution timing | Add `createEvlogIntegration(ai)` to `experimental_telemetry.integrations` |
| No tool execution timing | Add `createEvlogIntegration(ai)` to `telemetry.integrations` |
| Manual cost calculation | Use `cost` option in `createAILogger()` |

---
Expand Down
2 changes: 1 addition & 1 deletion apps/nuxthub-playground/.nuxtrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
setups.@evlog/nuxthub="2.0.0"
setups.@evlog/nuxthub="2.0.1"
4 changes: 2 additions & 2 deletions apps/nuxthub-playground/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
"dependencies": {
"@evlog/nuxthub": "workspace:*",
"@libsql/client": "^0.17.3",
"@ai-sdk/vue": "^3.0.168",
"@ai-sdk/vue": "^4.0.7",
"@nuxthub/core": "^0.10.7",
"ai": "^6.0.168",
"ai": "^7.0.0",
"drizzle-kit": "^0.31.10",
"drizzle-orm": "^0.45.2",
"evlog": "workspace:*",
Expand Down
3 changes: 1 addition & 2 deletions apps/nuxthub-playground/server/api/chat.post.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,7 @@ export default defineEventHandler(async (event) => {
instructions: systemPrompt,
tools: { queryEvents },
stopWhen: stepCountIs(5),
experimental_telemetry: {
isEnabled: true,
telemetry: {
integrations: [createEvlogIntegration(ai)],
},
})
Expand Down
3 changes: 1 addition & 2 deletions apps/nuxthub-playground/server/api/test/ai-wrap.get.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ export default defineEventHandler(async (event) => {
model,
prompt: 'Say hello.',
maxOutputTokens: 200,
experimental_telemetry: {
isEnabled: true,
telemetry: {
integrations: [createEvlogIntegration(ai)],
},
})
Expand Down
2 changes: 1 addition & 1 deletion examples/eve/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"@streamdown/mermaid": "1.0.2",
"@tailwindcss/postcss": "4.3.0",
"@vercel/connect": "0.2.2",
"ai": "7.0.0-beta.178",
"ai": "^7.0.0",
"class-variance-authority": "0.7.1",
"clsx": "2.1.1",
"cmdk": "1.1.1",
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,12 @@
"release": "turbo run build --filter='./packages/*' && changeset publish"
},
"devDependencies": {
"@ai-sdk/provider": "^3.0.8",
"@ai-sdk/provider": "^4.0.0",
"@changesets/changelog-github": "^0.6.0",
"@changesets/cli": "^2.31.0",
"@hrcd/eslint-config": "^3.0.3",
"@types/node": "^25.9.1",
"ai": "^6.0.168",
"ai": "^7.0.0",
"automd": "^0.4.3",
"dotenv-cli": "^11.0.0",
"eslint": "^9.39.4",
Expand Down
4 changes: 3 additions & 1 deletion packages/evlog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -820,7 +820,9 @@ See [the Audit Logs guide](https://evlog.dev/use-cases/audit/overview) for compl

## AI SDK Integration

Capture token usage, tool calls, model info, and streaming metrics from the [Vercel AI SDK](https://ai-sdk.dev) into wide events. Requires `ai >= 6.0.0`.
Capture token usage, tool calls, model info, and streaming metrics from the [Vercel AI SDK](https://ai-sdk.dev) into wide events. Compatible with AI SDK v6 and v7 (`ai >= 6.0.168`). AI SDK v7 requires Node.js 22+.

For tool execution timing, abort tracking, and auto embed capture, pass `createEvlogIntegration(ai)` to `telemetry.integrations` (v7) or `experimental_telemetry.integrations` (v6).

```typescript
import { streamText } from 'ai'
Expand Down
4 changes: 2 additions & 2 deletions packages/evlog/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@
"@nuxt/test-utils": "^4.0.3",
"@orpc/client": "^1.14.2",
"@orpc/server": "^1.14.2",
"ai": "^6.0.168",
"ai": "^7.0.0",
Comment thread
coderabbitai[bot] marked this conversation as resolved.
"@stryker-mutator/core": "^9.6.1",
"@stryker-mutator/vitest-runner": "^9.6.1",
"@sveltejs/kit": "^2.59.0",
Expand Down Expand Up @@ -432,7 +432,7 @@
"@orpc/server": ">=1.14.0",
"eve": ">=0.12.0",
"@tanstack/start-client-core": "^1.167.20",
"ai": ">=6.0.168",
"ai": ">=6.0.168 <8.0.0",
"elysia": ">=1.4.28",
"express": ">=5.2.1",
"fastify": ">=5.8.5",
Expand Down
Loading
Loading