diff --git a/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-cost-breakdown.png b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-cost-breakdown.png new file mode 100644 index 000000000..657f16239 Binary files /dev/null and b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-cost-breakdown.png differ diff --git a/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-token-breakdown.png b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-token-breakdown.png new file mode 100644 index 000000000..ced3abe62 Binary files /dev/null and b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-log-token-breakdown.png differ diff --git a/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-logs.png b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-logs.png index 17d85341a..934853442 100644 Binary files a/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-logs.png and b/public/docs-static/img/agent-network/usage-and-logs/agent-network-access-logs.png differ diff --git a/src/pages/agent-network/global-limits.mdx b/src/pages/agent-network/global-limits.mdx index 41414a63b..002de5252 100644 --- a/src/pages/agent-network/global-limits.mdx +++ b/src/pages/agent-network/global-limits.mdx @@ -4,7 +4,7 @@ export const description = # Global Limits Global limits are account-wide caps on token usage and spend that apply across -**every** policy and **every provider** — a backstop independent of any single policy's +**every** policy and **every provider**, a backstop independent of any single policy's limits. They are **limit-only** rules: unlike a policy, a global limit never selects a provider or authorizes traffic, and it isn't tied to one. It caps a caller's total consumption no matter which provider or gateway the request is routed to. @@ -21,9 +21,9 @@ caller's effective limit and never loosen it, adding one is always safe. Each rule targets who it applies to: -- **Target groups** — the rule binds when the caller's groups intersect the rule's groups. -- **Target users** — the rule binds a specific user directly. -- **Untargeted** — a rule with no target groups or users applies to **every** caller (the +- **Target groups**: the rule binds when the caller's groups intersect the rule's groups. +- **Target users**: the rule binds a specific user directly. +- **Untargeted**: a rule with no target groups or users applies to **every** caller (the account-wide default). A request can be bound by several rules at once (for example an account-wide rule plus a @@ -39,7 +39,7 @@ A global limit carries the same cap shape as a [policy limit](/agent-network/pol Windows and counting work exactly as described in [Token & Budget Limits](/agent-network/policies/limits#how-the-window-works): caps apply to a fixed, epoch-aligned window, and the check is run **before** the request against usage -already accumulated — so a request that starts under the cap is allowed even if it crosses +already accumulated, so a request that starts under the cap is allowed even if it crosses it, and the next one is blocked. ## How Enforcement Works diff --git a/src/pages/agent-network/how-it-works.mdx b/src/pages/agent-network/how-it-works.mdx index a45d67fff..0c3749da1 100644 --- a/src/pages/agent-network/how-it-works.mdx +++ b/src/pages/agent-network/how-it-works.mdx @@ -18,7 +18,7 @@ calling: - **Internal resources** such as databases, internal APIs, and self-hosted models, are reached directly over **peer-to-peer WireGuard tunnels**, the same way any NetBird peer reaches another. This traffic is governed by the same identities and access policies - but does not pass through the proxy, so there is no endpoint or key injection — the + but does not pass through the proxy, so there is no endpoint or key injection. The agent connects straight to the resource over the overlay. ## Architecture @@ -30,7 +30,7 @@ service adds an identity-aware control plane for AI traffic. ### LLM APIs and AI Gateways -The diagram below illustrates the **first path** — an LLM request: the agent reaches the +The diagram below illustrates the **first path**, an LLM request: the agent reaches the endpoint over the WireGuard overlay, the proxy enforces identity, policies, limits, and guardrails against the management control plane, injects the provider key, and forwards to the upstream API or gateway. The proxy can also inject the calling agent's identity into the @@ -43,17 +43,17 @@ can apply tag budgets and per-user attribution. agent network LLM request path through the NetBird proxy

-- **NetBird client** — the agent's device joins the overlay as a peer. Its requests to +- **NetBird client**: the agent's device joins the overlay as a peer. Its requests to the endpoint are routed through the WireGuard tunnel, not the public internet. -- **Proxy peer** — handles LLM traffic only. It terminates the request, establishes +- **Proxy peer**: handles LLM traffic only. It terminates the request, establishes the caller's identity, runs the routing and policy pipeline, injects the provider key, and forwards to the upstream API or gateway. -- **Management service** — the control plane. It holds providers, policies, guardrails, +- **Management service**: the control plane. It holds providers, policies, guardrails, and limits; resolves identities against your IdP; answers the proxy's per-request policy checks; and records usage and access logs. -- **Identity provider** — your existing IdP (Okta, Microsoft Entra ID, Google, …) +- **Identity provider**: your existing IdP (Okta, Microsoft Entra ID, Google, …) supplies the identities and group memberships that policies are written against. -- **Upstreams** — for LLM traffic, the proxy forwards to LLM APIs and AI gateways. +- **Upstreams**: for LLM traffic, the proxy forwards to LLM APIs and AI gateways. The endpoint hostname itself (for example `https://sailcloth.netbird.ai`) is generated when you connect your first provider and is only reachable from inside your overlay. @@ -62,14 +62,14 @@ overlay. ### Internal Resources -The **second path** covers everything that isn't an LLM API — internal databases, +The **second path** covers everything that isn't an LLM API: internal databases, internal APIs, and self-hosted models on a GPU host. Here the proxy is not involved at all. The agent connects to the target's overlay address **directly over a peer-to-peer WireGuard tunnel**, exactly the way any NetBird peer reaches another. Access is still identity-based: the agent's peer identity and group membership are matched against your access policies, so it can reach only the resources it is authorized for. Because the traffic never passes through the proxy, this path has no agent network endpoint, no -provider-key injection, and no token, budget, or per-request LLM logging — it is governed +provider-key injection, and no token, budget, or per-request LLM logging. It is governed like standard NetBird peer-to-peer access. This keeps internal traffic fast and private, flowing straight between the two peers. Because NetBird is a peer-to-peer network, this also works in reverse, so a resource can reach back to an agent when needed, such as to @@ -82,7 +82,7 @@ deliver a callback or webhook. ## The Lifecycle of an LLM Request -This pipeline applies to LLM traffic — requests to the agent network endpoint. +This pipeline applies to LLM traffic: requests to the agent network endpoint. Access to internal resources skips it entirely and flows peer-to-peer (see [Internal Resources](#internal-resources)). @@ -90,24 +90,24 @@ The proxy runs each request through an ordered chain of middleware. On the way t upstream: 1. **Establish identity.** The request arrives over the WireGuard tunnel, so the proxy - maps it to the calling NetBird peer and its identity — tied to your IdP for a human - user, or the peer's own NetBird identity for an autonomous agent — together with its + maps it to the calling NetBird peer and its identity, tied to your IdP for a human + user, or the peer's own NetBird identity for an autonomous agent, together with its group membership. See [Identity and Authentication](#identity-and-authentication). 2. **Parse the request.** Read the target model and stream flag from the body, and capture the prompt if prompt collection is enabled. 3. **Route and inject the key.** Match the model to a provider the caller's groups are authorized to use, rewrite the upstream target, strip any client-supplied auth headers, - and inject the provider's key from server-side storage — see + and inject the provider's key from server-side storage. See [Routing](#routing-matching-a-request-to-a-provider) and [Keyless Access](#keyless-access). 4. **Check policy and limits.** Ask management to select the matching policy and evaluate account- and policy-level token and budget caps. If unauthorized or a cap is exhausted, - the request is denied here — see [Policies, Limits, and + the request is denied here. See [Policies, Limits, and Guardrails](#policies-limits-and-guardrails). 5. **Stamp identity for the gateway.** Add the caller's identity to the upstream request (for example into `metadata.tags` and `x-litellm-end-user-id`) for gateways that key their own budgets and attribution off it, or into a provider's own cost-allocation metadata such as AWS Bedrock's `X-Amzn-Bedrock-Request-Metadata`. This is on by default - and can be turned off per provider — see [Identity + and can be turned off per provider. See [Identity Metadata](/agent-network/providers#identity-metadata). 6. **Apply guardrails.** Enforce the model allowlist and the prompt-capture rules. @@ -116,7 +116,7 @@ reverse: 7. **Meter.** Extract token counts from the response and convert them to cost. 8. **Record.** Post the usage back to management to update the limit counters. Usage is - always recorded; a full access-log entry is written when log collection is on — see + always recorded; a full access-log entry is written when log collection is on. See [Usage and Access Logs](#usage-and-access-logs). A denial at any gate returns `403` to the client with a machine-readable reason, and the @@ -129,14 +129,14 @@ comes from the **NetBird tunnel**. Because the request arrives over WireGuard, t maps its source to the enrolled peer and resolves the peer's NetBird identity and group membership: -- For a **human user** — for example someone running Claude Code — the NetBird identity is +- For a **human user**: for example someone running Claude Code, the NetBird identity is tied to your identity provider (Okta, Microsoft Entra ID, Google, …), so the request carries that user and the groups they belong to. - For an **autonomous agent**, the identity is the agent's own NetBird peer identity and the groups assigned to that peer. Either way the request carries a real identity and its **group membership**, captured at -request time. There is no API key or separate login on the client — the tunnel is the +request time. There is no API key or separate login on the client. The tunnel is the credential. Policies are written against those groups, so access to AI follows the same identities your organization already manages. @@ -161,7 +161,7 @@ was selected and which groups authorized it. ## Policies, Limits, and Guardrails Routing decides *where* a request can go; policies decide *whether it may* and *under what -budget*. By default nothing is allowed — a policy must connect a **source group** to one +budget*. By default nothing is allowed: a policy must connect a **source group** to one or more **providers**. At request time, management evaluates, in order: @@ -206,11 +206,11 @@ rotating a provider key is a single server-side change. Agent Network separates lightweight accounting from full audit detail: -- **Usage** is recorded for **every** served request — identity, provider, model, tokens, - and cost — regardless of any logging setting. This always-on stream powers the usage +- **Usage** is recorded for **every** served request: identity, provider, model, tokens, + and cost, regardless of any logging setting. This always-on stream powers the usage dashboards and the limit counters, and is retained indefinitely. -- **Access logs** add the full per-request detail (method, path, status, duration, and — - when prompt capture is on — the prompt and completion). Full access-log entries are +- **Access logs** add the full per-request detail (method, path, status, duration, and, + when prompt capture is on, the prompt and completion). Full access-log entries are written only when **log collection** is enabled for the account, and are swept after a configurable **retention period**. Prompts can be redacted for PII. @@ -227,7 +227,7 @@ This is also where the two paths differ: - **LLM traffic** rides the overlay to reach the **proxy** peer, which then applies the pipeline above and forwards to the upstream API or gateway. -- **Internal resources** — databases, APIs, and self-hosted models — are reached over a +- **Internal resources**: databases, APIs, and self-hosted models, are reached over a **direct peer-to-peer tunnel** between the agent and the target peer, with no proxy in between. Access is governed by the same identities and access policies as any other NetBird peer, so an agent reaches only the resources its identity is allowed to. diff --git a/src/pages/agent-network/index.mdx b/src/pages/agent-network/index.mdx index 8a1730871..4968639c0 100644 --- a/src/pages/agent-network/index.mdx +++ b/src/pages/agent-network/index.mdx @@ -1,7 +1,7 @@ import { Note } from '@/components/mdx' export const description = - 'Agent Network is NetBird\'s control layer for AI agents — a keyless gateway to LLM APIs and scoped, identity-based access to your internal resources, all over the tunnel with per-identity policies, limits, and audit.' + 'Agent Network is NetBird\'s control layer for AI agents, a keyless gateway to LLM APIs and scoped, identity-based access to your internal resources, all over the tunnel with per-identity policies, limits, and audit.' # What is NetBird Agent Network? @@ -79,7 +79,7 @@ their enterprise stack with minimal changes. ## Focused Dashboard View -Some accounts open directly into Agent Network, with the rest of the NetBird Dashboard—such as Peers, Networks, and DNS—hidden to keep the experience focused on agents. This is the default for accounts onboarded specifically for Agent Network. +Some accounts open directly into Agent Network, with the rest of the NetBird Dashboard (such as Peers, Networks, and DNS) hidden to keep the experience focused on agents. This is the default for accounts onboarded specifically for Agent Network. To use the full NetBird platform, open **Settings** and turn off **Agent Network focused view**. You can re-enable it from the same setting at any time. diff --git a/src/pages/agent-network/integrations/bedrock.mdx b/src/pages/agent-network/integrations/bedrock.mdx index 064493339..2eeb351ee 100644 --- a/src/pages/agent-network/integrations/bedrock.mdx +++ b/src/pages/agent-network/integrations/bedrock.mdx @@ -11,7 +11,7 @@ agents keyless access over the tunnel: NetBird holds the Bedrock API key server- every request to a real identity from your IdP, and applies your policies, limits, and audit on the way to Bedrock. -Bedrock authenticates with a **Bedrock API key** — a long-term key you generate in AWS that +Bedrock authenticates with a **Bedrock API key**, a long-term key you generate in AWS that NetBird injects as a bearer token on every request. You create the key once, hand it to NetBird, and it stays server-side. @@ -29,7 +29,7 @@ See [Bedrock API keys](https://docs.aws.amazon.com/bedrock/latest/userguide/api- for details. - The Bedrock API key grants access to models in your AWS account. Treat it as a secret — + The Bedrock API key grants access to models in your AWS account. Treat it as a secret: store it securely, never commit it to source control, and delete the local copy once it's stored in NetBird. @@ -37,15 +37,15 @@ for details. ## Connect the Provider 1. Go to **Agent Network → Providers** and click **Connect Provider**. -2. Select **AWS Bedrock**. Set the **Upstream URL** to your region's Bedrock runtime host — +2. Select **AWS Bedrock**. Set the **Upstream URL** to your region's Bedrock runtime host, for example `https://bedrock-runtime.us-east-1.amazonaws.com`. Bedrock is region-specific, so the host must include the region you enabled model access in. 3. Paste the **Bedrock API key**. NetBird stores it encrypted server-side, injects it as `Authorization: Bearer …` on each request, and never returns it to callers. -4. _(Optional)_ Restrict the **allowed models** — for example `anthropic.claude-opus-4-8`, +4. _(Optional)_ Restrict the **allowed models**, for example `anthropic.claude-opus-4-8`, `anthropic.claude-sonnet-4-6`, `meta.llama3-3-70b-instruct`, or `amazon.nova-pro`. Leaving the list empty allows any catalog model. -5. Save the provider. The key is now held server-side — the next step authorizes who can use +5. Save the provider. The key is now held server-side, the next step authorizes who can use it.

@@ -56,7 +56,7 @@ See [Providers](/agent-network/providers) for details. ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the Bedrock provider +By default nothing is allowed: a policy must connect a source group to the Bedrock provider before anyone can route through it. 1. Go to **Agent Network → Policies** and add a policy. @@ -70,11 +70,11 @@ See [Policies](/agent-network/policies) for details. ## Cost Allocation -NetBird forwards the caller's identity to every provider by default — see +NetBird forwards the caller's identity to every provider by default. See [Identity Metadata](/agent-network/providers#identity-metadata) for the general behavior and how to turn it off. For Bedrock that identity lands in the [`X-Amzn-Bedrock-Request-Metadata`](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-request-metadata.html) -header — the one AWS reads for cost-allocation tags — carrying the caller's **user** and the +header, the one AWS reads for cost-allocation tags, carrying the caller's **user** and the **group that authorized the request**: ``` @@ -94,7 +94,7 @@ The provider's **Mappings** tab shows exactly what NetBird sends: ## Use with Claude Code To route [Claude Code](/agent-network/integrations/claude-code) through this Bedrock provider -instead of the Anthropic API, see [Use Claude on AWS Bedrock](/agent-network/integrations/claude-code#use-claude-on-aws-bedrock) +instead of the Anthropic API. See [Use Claude on AWS Bedrock](/agent-network/integrations/claude-code#use-claude-on-aws-bedrock) on the Claude Code integration page. diff --git a/src/pages/agent-network/integrations/claude-code.mdx b/src/pages/agent-network/integrations/claude-code.mdx index 3b5b6c4a9..238c6c903 100644 --- a/src/pages/agent-network/integrations/claude-code.mdx +++ b/src/pages/agent-network/integrations/claude-code.mdx @@ -1,5 +1,5 @@ export const description = - 'Route Claude Code through NetBird Agent Network by pointing it at your agent network endpoint as the Anthropic base URL — no API key on the client.' + 'Route Claude Code through NetBird Agent Network by pointing it at your agent network endpoint as the Anthropic base URL. No API key on the client.' # Keyless Access to Claude Code @@ -14,7 +14,7 @@ Anthropic key into one your team reaches with their existing identity: any developer's machine. Each person runs Claude Code over the NetBird tunnel, and the request is tied to their real identity from your identity provider (Okta, Microsoft Entra ID, Google, …). Onboarding and offboarding follow the same IdP groups you already - manage — there's no key to hand out, copy, or revoke. + manage. There's no key to hand out, copy, or revoke. - **Usage tracking per developer and group.** Every request is metered by identity, model, tokens, and cost, so you can see exactly who is using Claude Code and how much it costs, broken down per person and aggregated per IdP group (team, department, project) in @@ -30,7 +30,7 @@ your endpoint. 1. Go to **Agent Network → Providers** and click **Connect Provider**. 2. Select **Anthropic** and paste your Anthropic API key. -3. Save the provider. The key is now held server-side — the next step authorizes who can use it. +3. Save the provider. The key is now held server-side, the next step authorizes who can use it.

connect Anthropic provider in NetBird Agent Network @@ -40,7 +40,7 @@ See [Providers](/agent-network/providers) for details. ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the Anthropic +By default nothing is allowed: a policy must connect a source group to the Anthropic provider before anyone can route Claude Code through it. 1. Go to **Agent Network → Policies** and add a policy. @@ -60,7 +60,7 @@ See [Policies](/agent-network/policies) for details. ## Configure with `settings.json` Next to your agent network endpoint in the NetBird dashboard, click **Agent Config**. The -modal shows tabs for Claude Code, Codex, OpenAI SDK, and cURL — pick **Claude Code** and copy +modal shows tabs for Claude Code, Codex, OpenAI SDK, and cURL. Pick **Claude Code** and copy the pre-filled configuration.

@@ -68,7 +68,7 @@ the pre-filled configuration.

Add the following to `~/.claude/settings.json`. The `apiKeyHelper` returns a dummy value so -Claude Code doesn't prompt for a key — NetBird supplies the real one. +Claude Code doesn't prompt for a key. NetBird supplies the real one. ```json { @@ -89,9 +89,9 @@ export ANTHROPIC_API_KEY=none claude ``` -That's it — Claude Code now sends every request over the NetBird tunnel, where it's tied to +That's it. Claude Code now sends every request over the NetBird tunnel, where it's tied to your identity, checked against your policies and limits, and recorded in -[Usage & Logs](/agent-network/usage-and-logs) — broken down per developer and aggregated +[Usage & Logs](/agent-network/usage-and-logs), broken down per developer and aggregated per IdP group.

@@ -103,11 +103,11 @@ per IdP group. If you reach Claude through **Google Vertex AI** instead of the Anthropic API, point Claude Code's Vertex backend at your agent network endpoint. NetBird holds the Google service account credential server-side and mints the Vertex access token, so Claude Code skips -Google authentication entirely — the client stays keyless. +Google authentication entirely. The client stays keyless. First connect a [Google Vertex AI provider](/agent-network/integrations/vertex-ai) in NetBird. -Set its upstream URL to the region-less host `https://aiplatform.googleapis.com` — not the -`-aiplatform.googleapis.com` form — so it matches `CLOUD_ML_REGION=global` below. +Set its upstream URL to the region-less host `https://aiplatform.googleapis.com`, not the +`-aiplatform.googleapis.com` form, so it matches `CLOUD_ML_REGION=global` below. Then add the following to `~/.claude/settings.json`: @@ -124,7 +124,7 @@ Then add the following to `~/.claude/settings.json`: ``` - `CLAUDE_CODE_USE_VERTEX=1` routes Claude Code through the Vertex backend. -- `CLAUDE_CODE_SKIP_VERTEX_AUTH=1` skips Google auth on the client — NetBird injects the +- `CLAUDE_CODE_SKIP_VERTEX_AUTH=1` skips Google auth on the client: NetBird injects the OAuth token server-side. - `ANTHROPIC_VERTEX_BASE_URL` is your agent network endpoint with the `/v1` suffix. - `CLOUD_ML_REGION=global` pairs with the region-less provider URL above. @@ -133,7 +133,7 @@ Then add the following to `~/.claude/settings.json`: If you reach Claude through **AWS Bedrock** instead of the Anthropic API, point Claude Code's Bedrock backend at your agent network endpoint. NetBird holds the Bedrock API key -server-side and injects it, so Claude Code skips AWS authentication entirely — the client +server-side and injects it, so Claude Code skips AWS authentication entirely. The client stays keyless. First connect an [AWS Bedrock provider](/agent-network/integrations/bedrock) in NetBird. @@ -151,13 +151,13 @@ Then add the following to `~/.claude/settings.json`: ``` - `CLAUDE_CODE_USE_BEDROCK=1` routes Claude Code through the Bedrock backend. -- `CLAUDE_CODE_SKIP_BEDROCK_AUTH=1` skips AWS auth on the client — NetBird injects the +- `CLAUDE_CODE_SKIP_BEDROCK_AUTH=1` skips AWS auth on the client: NetBird injects the Bedrock API key server-side. - `ANTHROPIC_BEDROCK_BASE_URL` is your agent network endpoint with the `/bedrock` suffix (the optional gateway-namespace prefix that disambiguates Bedrock from other providers). - `ANTHROPIC_MODEL` is the full Bedrock model ID including the region prefix (e.g. `eu.anthropic.claude-sonnet-4-5-20250929-v1:0`). Some models may not be available in all - regions — if the model above doesn't work, switch to one in your provider's allowed list, + regions. If the model above doesn't work, switch to one in your provider's allowed list, or change it in Claude Code with `/model `. ## Use Kimi (Moonshot AI) diff --git a/src/pages/agent-network/integrations/codex.mdx b/src/pages/agent-network/integrations/codex.mdx index 3351b13c2..5cfacaa8f 100644 --- a/src/pages/agent-network/integrations/codex.mdx +++ b/src/pages/agent-network/integrations/codex.mdx @@ -1,5 +1,5 @@ export const description = - 'Point the Codex CLI at your NetBird Agent Network endpoint with a custom model provider — keyless, over the tunnel.' + 'Point the Codex CLI at your NetBird Agent Network endpoint with a custom model provider. Keyless, over the tunnel.' # Keyless Access to Codex @@ -13,8 +13,8 @@ into one your team reaches with their existing identity: - **Keyless access through your IdP.** No OpenAI API key is distributed to or stored on any developer's machine. Each person runs Codex over the NetBird tunnel, and the request is tied to their real identity from your identity provider (Okta, Microsoft Entra ID, - Google, …). Onboarding and offboarding follow the same IdP groups you already manage — - there's no key to hand out, copy, or revoke. + Google, …). Onboarding and offboarding follow the same IdP groups you already manage. + There's no key to hand out, copy, or revoke. - **Usage tracking per developer and group.** Every request is metered by identity, model, tokens, and cost, so you can see exactly who is using Codex and how much it costs, broken down per person and aggregated per IdP group (team, department, project) in @@ -30,7 +30,7 @@ endpoint. 1. Go to **Agent Network → Providers** and click **Connect Provider**. 2. Select **OpenAI** (or another OpenAI-compatible provider or gateway) and paste its API key. -3. Save the provider. The key is now held server-side — the next step authorizes who can use it. +3. Save the provider. The key is now held server-side, the next step authorizes who can use it.

connect OpenAI provider in NetBird Agent Network @@ -40,7 +40,7 @@ See [Providers](/agent-network/providers) for details. ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the OpenAI provider +By default nothing is allowed: a policy must connect a source group to the OpenAI provider before anyone can route Codex through it. 1. Go to **Agent Network → Policies** and add a policy. diff --git a/src/pages/agent-network/integrations/index.mdx b/src/pages/agent-network/integrations/index.mdx index 0141c6698..21b33d7ce 100644 --- a/src/pages/agent-network/integrations/index.mdx +++ b/src/pages/agent-network/integrations/index.mdx @@ -1,11 +1,11 @@ export const description = - 'Connect specific agent tools and gateways to NetBird Agent Network — Claude Code, Codex, and LiteLLM.' + 'Connect specific agent tools and gateways to NetBird Agent Network: Claude Code, Codex, and LiteLLM.' # Integrations These guides show how to point common AI tools and gateways at your [agent network endpoint](/agent-network/how-it-works#llm-apis-and-ai-gateways). In every -case the client holds no provider API key — NetBird authorizes the request against your +case the client holds no provider API key. NetBird authorizes the request against your [policies](/agent-network/policies) and injects the upstream key server-side. Replace `` in the snippets below with the endpoint shown on the @@ -13,15 +13,15 @@ Replace `` in the snippets below with the endpoint shown on the ## In This Section -- [Claude Code](/agent-network/integrations/claude-code) — route Claude Code through NetBird. -- [Codex](/agent-network/integrations/codex) — point the Codex CLI at the endpoint. -- [LiteLLM](/agent-network/integrations/litellm) — use a LiteLLM gateway with identity-based +- [Claude Code](/agent-network/integrations/claude-code): route Claude Code through NetBird. +- [Codex](/agent-network/integrations/codex): point the Codex CLI at the endpoint. +- [LiteLLM](/agent-network/integrations/litellm): use a LiteLLM gateway with identity-based attribution and budgets. -- [vLLM](/agent-network/integrations/vllm) — connect a self-hosted, OpenAI-compatible vLLM +- [vLLM](/agent-network/integrations/vllm): connect a self-hosted, OpenAI-compatible vLLM server. -- [Google Vertex AI](/agent-network/integrations/vertex-ai) — connect Gemini and Claude on +- [Google Vertex AI](/agent-network/integrations/vertex-ai): connect Gemini and Claude on Vertex AI with a Google Cloud service account. -- [AWS Bedrock](/agent-network/integrations/bedrock) — connect Claude, Llama, and Nova on +- [AWS Bedrock](/agent-network/integrations/bedrock): connect Claude, Llama, and Nova on Bedrock with a Bedrock API key. -- [Kimi (Moonshot AI)](/agent-network/integrations/kimi) — connect Moonshot's Kimi models and +- [Kimi (Moonshot AI)](/agent-network/integrations/kimi): connect Moonshot's Kimi models and point Kimi CLI or Claude Code at the endpoint. diff --git a/src/pages/agent-network/integrations/litellm.mdx b/src/pages/agent-network/integrations/litellm.mdx index da4058f64..7b794a821 100644 --- a/src/pages/agent-network/integrations/litellm.mdx +++ b/src/pages/agent-network/integrations/litellm.mdx @@ -20,7 +20,7 @@ same network as the proxy so the proxy can reach it directly (for example 1. Go to **Agent Network → Providers** and click **Connect Provider**. 2. Select **LiteLLM Proxy** and set the **Upstream URL** to your self-hosted LiteLLM instance. 3. Paste a LiteLLM **virtual key** as the API key. NetBird stores it server-side. -4. Save the provider. The key is now held server-side — the next step authorizes who can use it. +4. Save the provider. The key is now held server-side, the next step authorizes who can use it.

connect a self-hosted LiteLLM Proxy provider in NetBird Agent Network @@ -28,7 +28,7 @@ same network as the proxy so the proxy can reach it directly (for example ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the LiteLLM provider +By default nothing is allowed: a policy must connect a source group to the LiteLLM provider before anyone can route through it. 1. Go to **Agent Network → Policies** and add a policy. @@ -89,7 +89,7 @@ each NetBird IdP group is listed as a tag passed dynamically in the request. ## Result Agents point at the NetBird endpoint with no key. NetBird enforces your policies, limits, -and guardrails first, then LiteLLM applies its own tag and end-user budgets on top — driven +and guardrails first, then LiteLLM applies its own tag and end-user budgets on top, driven by the same NetBird identity. Every call is recorded in [Usage & Logs](/agent-network/usage-and-logs), where each LiteLLM request shows the caller's identity, auth group, model, tokens, cost, and status. diff --git a/src/pages/agent-network/integrations/vertex-ai.mdx b/src/pages/agent-network/integrations/vertex-ai.mdx index afdfbfff4..1221dcad5 100644 --- a/src/pages/agent-network/integrations/vertex-ai.mdx +++ b/src/pages/agent-network/integrations/vertex-ai.mdx @@ -41,7 +41,7 @@ gcloud iam service-accounts create netbird-vertex \ ## Grant IAM Roles -The service account needs two roles — one to call Vertex AI models, and one to consume the +The service account needs two roles, one to call Vertex AI models, and one to consume the project's enabled services: ```bash @@ -62,7 +62,7 @@ gcloud iam service-accounts keys create netbird-vertex-key.json \ ``` - The key file grants access to Vertex AI in your project. Treat it as a secret — store it + The key file grants access to Vertex AI in your project. Treat it as a secret: store it securely, never commit it to source control, and delete the local copy once it's stored in NetBird. @@ -89,17 +89,17 @@ cat netbird-vertex-key.json | base64 -w 0 2. Select **Google Vertex AI**. NetBird pre-fills the upstream URL (`https://aiplatform.googleapis.com`) and the correct auth handling for Vertex. 3. Provide the **service account key** you generated (`netbird-vertex-key.json`). NetBird stores it encrypted server-side and never returns it to callers. -4. _(Optional)_ Restrict the **allowed models** and set per-model pricing — for example +4. _(Optional)_ Restrict the **allowed models** and set per-model pricing, for example `gemini-2.5-pro`, `gemini-2.5-flash`, `claude-sonnet-4-6`, or `claude-opus-4-7`. Leaving the list empty allows any catalog model. -5. Save the provider. The credential is now held server-side — the next step authorizes who +5. Save the provider. The credential is now held server-side, the next step authorizes who can use it. See [Providers](/agent-network/providers) for details. ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the Vertex AI +By default nothing is allowed: a policy must connect a source group to the Vertex AI provider before anyone can route through it. 1. Go to **Agent Network → Policies** and add a policy. @@ -114,7 +114,7 @@ See [Policies](/agent-network/policies) for details. ## Use with Claude Code To route [Claude Code](/agent-network/integrations/claude-code) through this Vertex AI -provider instead of the Anthropic API, see [Use Claude on Vertex AI](/agent-network/integrations/claude-code#use-claude-on-vertex-ai) +provider instead of the Anthropic API. See [Use Claude on Vertex AI](/agent-network/integrations/claude-code#use-claude-on-vertex-ai) on the Claude Code integration page. ## Manage Service Account Keys diff --git a/src/pages/agent-network/integrations/vllm.mdx b/src/pages/agent-network/integrations/vllm.mdx index 4bf3aa27a..3965e027b 100644 --- a/src/pages/agent-network/integrations/vllm.mdx +++ b/src/pages/agent-network/integrations/vllm.mdx @@ -6,8 +6,8 @@ export const description = # vLLM [vLLM](https://docs.vllm.ai) is a high-throughput inference server for self-hosted models -that exposes an **OpenAI-compatible API**. You run it yourself — typically on a GPU host -inside the same network as the NetBird proxy — and connect it as a provider so agents reach +that exposes an **OpenAI-compatible API**. You run it yourself, typically on a GPU host +inside the same network as the NetBird proxy, and connect it as a provider so agents reach it keyless over the tunnel, authorized by your [policies](/agent-network/policies). This gives you keyless access to your private vLLM from anywhere: the endpoint stays @@ -29,7 +29,7 @@ same network as the proxy so the proxy can reach it directly. NetBird stores it server-side and sends it as a bearer token. Leave it empty if the server requires none. 4. _(Optional)_ Enable **Skip TLS Verification** if your vLLM endpoint serves a self-signed - certificate — see [Skip TLS Verification](/agent-network/providers#skip-tls-verification). + certificate. See [Skip TLS Verification](/agent-network/providers#skip-tls-verification). Prefer mounting trusted certificates on your proxy instances for anything beyond testing. 5. _(Optional)_ List the **models** your server exposes with per-model pricing so cost shows up in [Usage & Logs](/agent-network/usage-and-logs). Leaving the list empty accepts any @@ -42,7 +42,7 @@ same network as the proxy so the proxy can reach it directly. ## Create a Policy -By default nothing is allowed — a policy must connect a source group to the vLLM provider +By default nothing is allowed: a policy must connect a source group to the vLLM provider before anyone can route through it. 1. Go to **Agent Network → Policies** and add a policy. diff --git a/src/pages/agent-network/policies/guardrails.mdx b/src/pages/agent-network/policies/guardrails.mdx index 542fc0a1f..a9765aec2 100644 --- a/src/pages/agent-network/policies/guardrails.mdx +++ b/src/pages/agent-network/policies/guardrails.mdx @@ -19,7 +19,7 @@ denied. Optionally store request prompts and response completions on logged requests. Prompt capture only runs when **both** the account-level prompt collection -setting and a policy guardrail enable it — see +setting and a policy guardrail enable it. See [Log Collection & Retention](/agent-network/usage-and-logs/log-collection). ### PII Redaction diff --git a/src/pages/agent-network/policies/index.mdx b/src/pages/agent-network/policies/index.mdx index 4c5af06ec..7f9db034e 100644 --- a/src/pages/agent-network/policies/index.mdx +++ b/src/pages/agent-network/policies/index.mdx @@ -3,7 +3,7 @@ export const description = # Policies -Policies connect users and agents (source groups) to AI providers — controlling +Policies connect users and agents (source groups) to AI providers, controlling which identities can reach which providers and models, with optional limits and guardrails. @@ -13,16 +13,16 @@ guardrails. This page explains how to create and manage access to AI providers and gateways. If you are - looking for a guide on how to manage access to internal resources, see + looking for a guide on how to manage access to internal resources. See [Access Control](/manage/access-control). ## How Policies Work -- **Source groups** — the users/agents the policy applies to. -- **Destination providers** — the providers the policy grants access to. -- **Limits** — optional per-user and per-group token and budget caps. -- **Guardrails** — optional model allowlist and prompt capture. +- **Source groups**: the users/agents the policy applies to. +- **Destination providers**: the providers the policy grants access to. +- **Limits**: optional per-user and per-group token and budget caps. +- **Guardrails**: optional model allowlist and prompt capture. A request is allowed when a policy connects the caller's groups to the resolved provider and no applicable limit is exhausted. diff --git a/src/pages/agent-network/policies/limits.mdx b/src/pages/agent-network/policies/limits.mdx index 9ded33b9e..4a7dc8315 100644 --- a/src/pages/agent-network/policies/limits.mdx +++ b/src/pages/agent-network/policies/limits.mdx @@ -15,15 +15,15 @@ on the token and/or budget limit, and set the per-user and per-group caps and wi ## Token Limits -- **User cap** — maximum tokens per user in the window. -- **Group cap** — maximum tokens per group in the window. -- **Window** — the fixed period the cap applies to (see [below](#how-the-window-works)). +- **User cap**: maximum tokens per user in the window. +- **Group cap**: maximum tokens per group in the window. +- **Window**: the fixed period the cap applies to (see [below](#how-the-window-works)). ## Budget Limits -- **User cap (USD)** — maximum spend per user in the window. -- **Group cap (USD)** — maximum spend per group in the window. -- **Window** — the fixed period the cap applies to (see [below](#how-the-window-works)). +- **User cap (USD)**: maximum spend per user in the window. +- **Group cap (USD)**: maximum spend per group in the window. +- **Window**: the fixed period the cap applies to (see [below](#how-the-window-works)). ## How the Window Works @@ -44,10 +44,10 @@ This alignment is deliberate: because every node derives the same boundaries fro clock, usage adds up consistently across a clustered deployment instead of drifting with each node's first request. -**Example — a 1-hour (3600s) window.** Buckets run from the top of each hour in UTC: +**Example: a 1-hour (3600s) window.** Buckets run from the top of each hour in UTC: `13:00:00–13:59:59`, `14:00:00–14:59:59`, and so on. A request at `13:45` counts toward the `13:00` bucket; a request at `14:02` lands in a fresh `14:00` bucket with the counter back -at zero — regardless of when the user's first request of the day was. +at zero, regardless of when the user's first request of the day was. ## How Enforcement Works @@ -63,13 +63,13 @@ request that crosses the line still completes; the **next** one is blocked. **Example.** A token cap is set to `1000` for a group, and the group has used `999` in the current window: -- The next request is checked: `999 >= 1000` is false, so it **goes through** — even though +- The next request is checked: `999 >= 1000` is false, so it **goes through**: even though it then consumes, say, 250 tokens and pushes the total to `1249`. - The following request is checked: `1249 >= 1000` is true, so it is **blocked** until the window rolls over. -So a cap is a floor for *when blocking starts*, not a hard ceiling on the exact total — plan +So a cap is a floor for *when blocking starts*, not a hard ceiling on the exact total. Plan caps with a little headroom if you need a strict upper bound. -For account-wide caps that apply across all policies, see +For account-wide caps that apply across all policies. See [Global Limits](/agent-network/global-limits). \ No newline at end of file diff --git a/src/pages/agent-network/providers.mdx b/src/pages/agent-network/providers.mdx index bcb688089..f002558fe 100644 --- a/src/pages/agent-network/providers.mdx +++ b/src/pages/agent-network/providers.mdx @@ -1,5 +1,5 @@ export const description = - 'Connect AI providers and gateways — OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Mistral, Kimi (Moonshot AI), LiteLLM, Portkey, Bifrost, Cloudflare, Vercel, OpenRouter, or any OpenAI-compatible endpoint — to NetBird Agent Network and expose a single keyless endpoint.' + 'Connect AI providers and gateways (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Mistral, Kimi (Moonshot AI), LiteLLM, Portkey, Bifrost, Cloudflare, Vercel, OpenRouter, or any OpenAI-compatible endpoint) to NetBird Agent Network and expose a single keyless endpoint.' # Providers @@ -44,7 +44,7 @@ and budgets (see [How It Works](/agent-network/how-it-works#llm-apis-and-ai-gate ### Custom -- Custom / Self-hosted — any OpenAI-compatible endpoint, including local models served by +- Custom / Self-hosted: any OpenAI-compatible endpoint, including local models served by Ollama, vLLM, or a private GPU host. ## Connect a Provider @@ -67,7 +67,7 @@ and budgets (see [How It Works](/agent-network/how-it-works#llm-apis-and-ai-gate ## Custom & Self-hosted Providers Pick **Custom / Self-hosted** for any OpenAI-compatible endpoint that isn't a first-party -vendor or a named gateway — a private inference server, an on-prem deployment, or a local +vendor or a named gateway, a private inference server, an on-prem deployment, or a local model runtime like Ollama or vLLM (vLLM also has its own named entry). NetBird talks to it the same way it talks to OpenAI: you provide the **Upstream URL** where requests are forwarded and, if the endpoint requires one, an **API key** sent as a bearer token. @@ -96,13 +96,13 @@ The switch appears only for custom (self-hosted) providers and is **off by defau By default NetBird stamps the caller's **user** and the **group that authorized the request** onto each upstream request, so the provider or gateway can attribute usage to the real caller -instead of the shared API key. The exact header or field is provider-specific — see the +instead of the shared API key. The exact header or field is provider-specific. See the provider's [integration guide](/agent-network/integrations) for details (for example, AWS Bedrock carries it in a header used for [cost-allocation tags](/agent-network/integrations/bedrock#cost-allocation), and AI gateways receive their own attribution headers). -This is controlled by the **Forward identity metadata** toggle on the provider — **on by -default**, and shown only for providers that support it (first-party APIs such as OpenAI or +This is controlled by the **Forward identity metadata** toggle on the provider, which is **on by +default** and shown only for providers that support it (first-party APIs such as OpenAI or Anthropic have no such metadata channel, so the toggle doesn't appear for them). Turn it off to keep the caller's identity out of the upstream request. diff --git a/src/pages/agent-network/quickstart.mdx b/src/pages/agent-network/quickstart.mdx index 4c5a755ea..d588fcd86 100644 --- a/src/pages/agent-network/quickstart.mdx +++ b/src/pages/agent-network/quickstart.mdx @@ -29,8 +29,8 @@ and manage the platform. ## Software Requirements - Docker with the docker-compose plugin v2 or higher ([Docker installation guide](https://docs.docker.com/engine/install/)) -- [jq](https://jqlang.github.io/jq/) — install with `sudo apt install jq` or `sudo yum install jq` -- [curl](https://curl.se/) — install with `sudo apt install curl` or `sudo yum install curl` +- [jq](https://jqlang.github.io/jq/): install with `sudo apt install jq` or `sudo yum install jq` +- [curl](https://curl.se/): install with `sudo apt install curl` or `sudo yum install curl` ### Installation script @@ -96,7 +96,7 @@ is needed on the client. NetBird authorizes each request against your policies a injects the upstream provider key server-side. 1. Next to your agent network endpoint, click **Agent Config**. -2. Pick the tab that matches your tool — Claude Code, Codex, OpenAI SDK, or cURL. The +2. Pick the tab that matches your tool, Claude Code, Codex, OpenAI SDK, or cURL. The dashboard pre-fills your endpoint for you. 3. Copy the snippet and apply it. Claude Code reads `~/.claude/settings.json` and Codex reads `~/.codex/config.toml`. diff --git a/src/pages/agent-network/usage-and-logs/access-logs.mdx b/src/pages/agent-network/usage-and-logs/access-logs.mdx index bb666e538..a79584877 100644 --- a/src/pages/agent-network/usage-and-logs/access-logs.mdx +++ b/src/pages/agent-network/usage-and-logs/access-logs.mdx @@ -14,26 +14,83 @@ and the reason a request was allowed or denied. ## Columns - **Time** -- **User / Agent** — the resolved caller identity. -- **Auth Group** — the groups that authorized the request. -- **Provider** — resolved provider and model. -- **Tokens** — input and output. -- **Cost** -- **Status** and **Reason** — for denials, the mapped policy reason; for +- **User / Agent**: the resolved caller identity. +- **Auth Group**: the groups that authorized the request. +- **Provider**: resolved provider and model. +- **Tokens**: input and output. Hover for the full breakdown, including + prompt-cache reads and writes when the request used caching. +- **Cost**: hover for the per-bucket breakdown of input, output, cache read, + and cache write, plus the total. +- **Status** and **Reason**: for denials, the mapped policy reason. For allowed requests, a link to the policy that authorized it. +## Prompt caching + +Agents that use provider-side prompt caching (for example Claude Code caches +its system prompt and tools) are billed for cache reads and writes on top of +regular input tokens. Agent Network records each bucket per request, so the +cache share of your spend is visible instead of hidden inside the total. + +Caching applies to the **prompt only**. Output tokens are generated fresh on +every call, so no provider discounts them and there is no cached-output bucket. + +Every request is metered against four separately priced token buckets: input +(prompt tokens not served from cache), output (the completion), cache read +(prompt tokens replayed from the cache, usually about a tenth of the input +rate), and cache write (prompt tokens written into the cache, usually 1.25× the +input rate). + +Because cache writes bill at a premium, the first request of a session can cost +noticeably more than its input and output counts alone suggest, while the +follow-up requests that read that cache cost much less. + +Hovering **Tokens** shows which buckets a request actually used. Here a +follow-up request sent only 16 fresh input tokens but replayed 3,980 tokens from +the cache, wrote none, and produced 192 of output, for 4,188 tokens in total: + +

+ agent network access log token breakdown with prompt-cache reads and writes +

+ +Hovering **Cost** breaks the same request down in dollars. Most of the spend +here is output, with the 3,980 cached tokens costing only $0.0020 at the reduced +cache-read rate: + +

+ agent network access log cost breakdown by token bucket +

+ +All four buckets are listed even when one costs nothing, so a zero cache-write +line tells you this request read the cache rather than populating it. The total +is always the sum of the four, so the breakdown and the headline figure can +never disagree. + + + Providers report cache tokens differently, which affects how the **Tokens** + column reads. Anthropic-shape providers, including Claude on Amazon Bedrock + and Google Vertex AI, report cache reads and writes *in addition to* + `input_tokens`, so the total is input + output + both cache buckets. OpenAI + instead counts cached tokens as a *subset* of the prompt, so they are already + inside `input_tokens`. The cost breakdown normalizes this: the input line is + always the non-cached remainder, so the same tokens are never billed twice. + + +The same buckets are summed per session in the **Sessions** view, and the +[usage overview](/agent-network/usage-and-logs/usage-overview) shows the cache +share of each day's spend. + ## Sessions Use the **Requests / Sessions** switch above the log to change the view. The **Sessions** view groups related requests into a single session and shows one -row per session, most recent activity first — so you can see the latest sessions +row per session, most recent activity first, so you can see the latest sessions at a glance.

agent network access logs grouped by session

-Some agents keep a session across many separate calls — for example, +Some agents keep a session across many separate calls. For example, [Claude Code](/agent-network/integrations/claude-code) tags every request in a coding session with the same session id. Agent Network groups those requests together, so a whole session shows up as one row with its totals instead of @@ -41,18 +98,20 @@ dozens of individual entries. Each session row summarizes: -- **Activity** — the session's time span, from its first request to its last. -- **User / Agent** and **Auth Group** — the caller and the authorizing groups. -- **Provider** — the provider and the model(s) used (e.g. `2 models`; hover to +- **Activity**: the session's time span, from its first request to its last. +- **User / Agent** and **Auth Group**: the caller and the authorizing groups. +- **Provider**: the provider and the model(s) used (e.g. `2 models`; hover to see the full list). -- **Requests** — how many requests the session made, and over what duration. -- **Tokens** and **Cost** — summed across the whole session. -- **Reason** — the session's policy decision; a denial if any request in the +- **Requests**: how many requests the session made, and over what duration. +- **Tokens** and **Cost**: summed across the whole session. Hover for the + breakdown, including the session's prompt-cache tokens and the same four cost + buckets as a single request. +- **Reason**: the session's policy decision, a denial if any request in the session was blocked. -Expand a session to see its individual requests — each with its timestamp, path, -status, duration, model, tokens, and cost — and expand a request for its full -detail (including the prompt and response when prompt capture is on). +Expand a session to see its individual requests, each with its timestamp, path, +status, duration, model, tokens, and cost. Expand a request for its full detail, +including the prompt and response when prompt capture is on. Requests from agents that don't set a session id each appear as their own single-request row. @@ -65,12 +124,12 @@ single-request row. Filter the log by: -- **Date** — defaults to the last 14 days. +- **Date**: defaults to the last 14 days. - **User** - **Group** - **Provider** - **Model** -- **Path** — match requests whose path starts with a given prefix (e.g. `/v1/messages`). +- **Path**: match requests whose path starts with a given prefix (e.g. `/v1/messages`). All filtering is applied server-side and works in both the Requests and Sessions views. @@ -78,5 +137,5 @@ views. ## Availability Access logs are retained only when log collection is enabled for the account. -Usage and cost are still recorded when it's off — see +Usage and cost are still recorded when it's off. See [Log Collection & Retention](/agent-network/usage-and-logs/log-collection). \ No newline at end of file diff --git a/src/pages/agent-network/usage-and-logs/index.mdx b/src/pages/agent-network/usage-and-logs/index.mdx index 7677f7331..65a46b6ca 100644 --- a/src/pages/agent-network/usage-and-logs/index.mdx +++ b/src/pages/agent-network/usage-and-logs/index.mdx @@ -8,11 +8,11 @@ identity, cost attribution, and token usage. ## In This Section -- [Usage Overview](/agent-network/usage-and-logs/usage-overview) — token and +- [Usage Overview](/agent-network/usage-and-logs/usage-overview): token and cost trends over time. -- [Access Logs](/agent-network/usage-and-logs/access-logs) — per-request log +- [Access Logs](/agent-network/usage-and-logs/access-logs): per-request log with server-side filtering. -- [Log Collection & Retention](/agent-network/usage-and-logs/log-collection) — +- [Log Collection & Retention](/agent-network/usage-and-logs/log-collection): enable/disable logging, set retention, and control prompt capture. ## Usage vs. Logs diff --git a/src/pages/agent-network/usage-and-logs/usage-overview.mdx b/src/pages/agent-network/usage-and-logs/usage-overview.mdx index 5e2562f50..e45b949c9 100644 --- a/src/pages/agent-network/usage-and-logs/usage-overview.mdx +++ b/src/pages/agent-network/usage-and-logs/usage-overview.mdx @@ -14,7 +14,7 @@ Tokens / Cost switch, plus a breakdown table. Filter the view by: -- **Date** — defaults to the last 14 days. +- **Date**: defaults to the last 14 days. - **User** - **Group** - **Provider** @@ -28,6 +28,10 @@ Switch between input/output token totals and estimated USD spend. Cost is derived from the per-model pricing configured on each [provider](/agent-network/providers). +Hovering a day, in the chart or in the breakdown table, also shows its +prompt-cache token buckets (cache reads and writes) and the cache share of the +day's cost. + ## Always Collected Usage is recorded on every request independently of access-log collection, so diff --git a/src/pages/ipa/resources/agent-network.mdx b/src/pages/ipa/resources/agent-network.mdx index e75093af1..2f88e113f 100644 --- a/src/pages/ipa/resources/agent-network.mdx +++ b/src/pages/ipa/resources/agent-network.mdx @@ -268,7 +268,10 @@ echo $response; "input_tokens": 1200, "output_tokens": 640, "total_tokens": 1840, + "cached_input_tokens": 0, + "cache_creation_tokens": 30528, "cost_usd": 0.0231, + "cache_cost_usd": 0.1145, "stream": { "type": "boolean", "description": "Whether the request was a streaming completion." @@ -318,7 +321,10 @@ echo $response; "input_tokens": "integer", "output_tokens": "integer", "total_tokens": "integer", + "cached_input_tokens": "integer", + "cache_creation_tokens": "integer", "cost_usd": "number", + "cache_cost_usd": "number", "stream": "boolean", "group_ids": [ "string" @@ -580,7 +586,10 @@ echo $response; "input_tokens": 8400, "output_tokens": 4480, "total_tokens": 12880, + "cached_input_tokens": 0, + "cache_creation_tokens": 30528, "cost_usd": 0.1617, + "cache_cost_usd": 0.1145, "providers": [ { "type": "string" @@ -638,7 +647,10 @@ echo $response; "input_tokens": 1200, "output_tokens": 640, "total_tokens": 1840, + "cached_input_tokens": 0, + "cache_creation_tokens": 30528, "cost_usd": 0.0231, + "cache_cost_usd": 0.1145, "stream": { "type": "boolean", "description": "Whether the request was a streaming completion." @@ -681,7 +693,10 @@ echo $response; "input_tokens": "integer", "output_tokens": "integer", "total_tokens": "integer", + "cached_input_tokens": "integer", + "cache_creation_tokens": "integer", "cost_usd": "number", + "cache_cost_usd": "number", "providers": [ "string" ], @@ -711,7 +726,10 @@ echo $response; "input_tokens": "integer", "output_tokens": "integer", "total_tokens": "integer", + "cached_input_tokens": "integer", + "cache_creation_tokens": "integer", "cost_usd": "number", + "cache_cost_usd": "number", "stream": "boolean", "group_ids": [ "string" @@ -938,7 +956,10 @@ echo $response; "input_tokens": 120000, "output_tokens": 64000, "total_tokens": 184000, - "cost_usd": 2.31 + "cached_input_tokens": 20000, + "cache_creation_tokens": 45000, + "cost_usd": 2.31, + "cache_cost_usd": 0.42 } ] ``` @@ -949,7 +970,10 @@ echo $response; "input_tokens": "integer", "output_tokens": "integer", "total_tokens": "integer", - "cost_usd": "number" + "cached_input_tokens": "integer", + "cache_creation_tokens": "integer", + "cost_usd": "number", + "cache_cost_usd": "number" } ] ```