Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,79 @@ If you need a different layout, keep passing `sourceDir` and `compiledDir` expli

`renderPrompt()` and `validatePrompt()` use the same source-versus-compiled resolution rules as `kit.renderPrompt()`. The existing synchronous `render()` and `validate()` methods still work for already-resolved compiled or inline assets.

## UsageTap LLM Gateway

Select `provider: usagetap` for the first-party OpenAI-compatible gateway. The default base URL is
`https://gateway.usagetap.com/v1` (including `/v1`) and the default managed route is
`usagetap/standard`. `usagetap/premium`, direct canonical `provider/model` IDs, and ordered
`fallback_models` are supported. Customer attribution is optional; without it, calls are attributed
to the authenticated organization. Pass the API key only in runtime options, never prompt assets.

```ts
import OpenAI from 'openai';
import { createUsageTapGatewayOpenAIConfig, usagetapAdapter } from 'promptopskit/usagetap';

const apiKey = process.env.USAGETAP_GATEWAY_API_KEY!;
const openai = new OpenAI(createUsageTapGatewayOpenAIConfig({ apiKey }));
const request = await usagetapAdapter.renderPrompt({ source: `---
id: buffered-gateway
provider: usagetap
model: usagetap/standard
fallback_models: [usagetap/premium, openai/gpt-5-mini]
provider_options:
usagetap:
feature: prompt-tightener
compress: { mode: deterministic, aggressiveness: 0.35 }
---
# Prompt template
Tighten: {{ prompt }}
` }, { variables: { prompt: 'Summarize the report.' }, usagetap: { apiKey, idempotencyKey: 'request-1' } });
if (!('body' in request)) throw new Error(request.returnMessage);
await openai.chat.completions.create(request.body as never);
```

Gateway compression uses `provider_options.usagetap.compress` or `raw.usagetap.compress` and is
separate from PromptOpsKit's local/client compression pipeline. Never wrap gateway calls with
`beginUsageTapCall`, `withUsageTapCall`, or a `runOpenAIWithUsageTap`-style helper: the gateway
meters itself. `createUsageTapClient` and those helpers meter direct calls to other providers.

For manual transport, append `/chat/completions`, producing
`https://gateway.usagetap.com/v1/chat/completions`; do not append a second `/v1`.

```diff
- provider: llmasaservice
- model: group:standard
+ provider: usagetap
+ model: usagetap/standard
raw:
- llmasaservice:
+ usagetap:
llmGateway:
feature: prompt-tightener
```

```diff
- import { llmasaserviceAdapter, LLMASASERVICE_BASE_URL } from "promptopskit";
+ import { usagetapAdapter, USAGETAP_GATEWAY_BASE_URL } from "promptopskit";
- const request = llmasaserviceAdapter.render(asset, {
- llmasaservice: { apiKey },
+ const request = usagetapAdapter.render(asset, {
+ usagetap: { apiKey, idempotencyKey },
runtime: {
- model: "group:standard",
+ model: "usagetap/standard",
provider_options: {
- llmasaservice: { base_url: baseURL, customer },
+ usagetap: { base_url: baseURL, customer },
},
},
});
```

Consumer applications should explicitly rename `LLMASASERVICE_API_KEY` to
`USAGETAP_GATEWAY_API_KEY`, `LLMASASERVICE_BASE_URL` to `USAGETAP_GATEWAY_BASE_URL`, and
`LLMASASERVICE_MODEL` to `USAGETAP_GATEWAY_MODEL`. Migrating downstream consumers is separate.

## Optional UsageTap Tracking

PromptOpsKit can also help you track provider calls with UsageTap.com while keeping the core render API transport-light.
Expand Down
34 changes: 34 additions & 0 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ Provider aliases:
| `google`, `gemini` | `gemini` |
| `openrouter` | `openrouter` |
| `llmasaservice`, `llmasaservice.io`, `llm gateway` | `llmasaservice` |
| `usagetap`, `usagetap gateway`, `gateway.usagetap.com` | `usagetap` |

Behavior:

Expand Down Expand Up @@ -259,6 +260,39 @@ if (!result.request) throw new Error('Prompt rendering did not produce an OpenRo
const completion = await client.chat.completions.create(result.request.body as any);
```

UsageTap gateway example (buffered Chat Completions):

```ts
import OpenAI from 'openai';
import { createUsageTapGatewayOpenAIConfig, usagetapAdapter } from 'promptopskit/usagetap';

const apiKey = process.env.USAGETAP_GATEWAY_API_KEY!;
const client = new OpenAI(createUsageTapGatewayOpenAIConfig({ apiKey }));
const rendered = await usagetapAdapter.renderPrompt({ source: `---
id: usagetap-example
provider: usagetap
model: usagetap/standard
fallback_models: [usagetap/premium, openai/gpt-5-mini]
provider_options:
usagetap:
feature: prompt-tightener
compress: { mode: deterministic, aggressiveness: 0.35 }
---
# Prompt template
Improve {{ text }}
` }, { variables: { text: 'this prompt' }, usagetap: { apiKey } });
if (!('body' in rendered)) throw new Error(rendered.returnMessage);
await client.chat.completions.create(rendered.body as never);
```

The default URL is `https://gateway.usagetap.com/v1`; append only `/chat/completions` for manual
transport. `usagetap/standard` and `usagetap/premium` are managed aliases; direct `provider/model`
IDs and ordered fallbacks work too. The API key is runtime-only and customer attribution is
optional. Gateway compression (`provider_options.usagetap.compress` or `raw.usagetap.compress`) is
separate from local/client compression. Never combine `usagetapAdapter` with client-side UsageTap
begin/end or runner helpers because the gateway meters itself. Generic UsageTap lifecycle requests
still refer to `createUsageTapClient` and direct-provider metering helpers.

LLMAsAService example:

```typescript
Expand Down
18 changes: 18 additions & 0 deletions docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,24 @@ provider_options:

Use `raw.openrouter` for less common OpenRouter body fields that PromptOpsKit does not model yet.

## UsageTap Gateway

Use `provider: usagetap` with `usagetapAdapter` for the first-party UsageTap gateway. It renders
OpenAI Chat Completions bodies with bearer authentication. The default model is
`usagetap/standard` and the default base URL is `https://gateway.usagetap.com/v1`; managed
`usagetap/premium`, canonical `provider/model` IDs, and ordered `fallback_models` are supported.
Customer attribution is optional. Supply `{ usagetap: { apiKey, idempotencyKey? } }` only at render
time. Typed gateway metadata and compression live under `provider_options.usagetap`; unsupported
fields can use `raw.usagetap`, which merges last.

Gateway calls are already authorized and metered by UsageTap. Do not wrap them in client-side
`beginUsageTapCall`/`endUsageTapCall`, `withUsageTapCall`, or provider runner helpers. Those lifecycle
APIs remain for metering direct calls made to other providers. Gateway compression is also distinct
from PromptOpsKit's local/client prompt compression pipeline.

Manual callers append `/chat/completions` to get
`https://gateway.usagetap.com/v1/chat/completions`, without adding another `/v1`.

## LLMAsAService Gateway

Body shape: OpenAI-compatible Chat Completions payloads sent to `https://gateway.llmasaservice.io`. The adapter reuses the OpenAI chat mapping, applies `provider_options.llmasaservice`, and reads `raw.llmasaservice` for gateway-only body fields.
Expand Down
13 changes: 13 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ export type {
RuntimeHistoryMessage,
OpenAIResponsesRuntimeOptions,
LLMAsAServiceRuntimeOptions,
UsageTapGatewayRuntimeOptions,
ProviderAdapter,
ProviderInlinePromptSource,
ProviderPromptInput,
Expand Down Expand Up @@ -91,6 +92,14 @@ export {
createLLMAsAServiceOpenAIConfig,
llmasaserviceAdapter,
} from './providers/llmasaservice.js';
export {
USAGETAP_GATEWAY_BASE_URL,
USAGETAP_GATEWAY_DEFAULT_MODEL,
USAGETAP_GATEWAY_RESPONSE_HEADER_NAMES,
createUsageTapGatewayOpenAIConfig,
usagetapAdapter,
} from './providers/usagetap.js';
export type { UsageTapGatewayOpenAIConfig, UsageTapGatewayOpenAIConfigOptions } from './providers/usagetap.js';
export { PromptAssetSchema, PromptAssetOverridesSchema } from './schema/index.js';
export {
summarizePromptCompression,
Expand Down Expand Up @@ -157,6 +166,8 @@ export interface RenderPromptOptions {
openaiResponses?: RuntimeRenderOptions['openaiResponses'];
/** LLMAsAService gateway credentials */
llmasaservice?: RuntimeRenderOptions['llmasaservice'];
/** UsageTap gateway credentials and optional per-request idempotency key */
usagetap?: RuntimeRenderOptions['usagetap'];
/** TheTokenCompany compression credentials and transport options */
theTokenCompany?: RuntimeRenderOptions['theTokenCompany'];
}
Expand Down Expand Up @@ -268,6 +279,7 @@ export class PromptOpsKit {
const validation = adapter.validate(resolved, {
openaiResponses: options.openaiResponses,
llmasaservice: options.llmasaservice,
usagetap: options.usagetap,
});

if (!validation.valid) {
Expand Down Expand Up @@ -312,6 +324,7 @@ export class PromptOpsKit {
strict: options.strict,
openaiResponses: options.openaiResponses,
llmasaservice: options.llmasaservice,
usagetap: options.usagetap,
theTokenCompany: options.theTokenCompany,
});
const request = adapter.render(prepared.asset, prepared.runtime);
Expand Down
5 changes: 5 additions & 0 deletions src/overrides/apply-overrides.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ function mergeOverride(
google: mergeRecordBlock(result.raw?.google, override.raw.google),
openrouter: mergeRecordBlock(result.raw?.openrouter, override.raw.openrouter),
llmasaservice: mergeRecordBlock(result.raw?.llmasaservice, override.raw.llmasaservice),
usagetap: mergeRecordBlock(result.raw?.usagetap, override.raw.usagetap),
};
}

Expand All @@ -118,6 +119,10 @@ function mergeOverride(
...result.provider_options?.llmasaservice,
...override.provider_options.llmasaservice,
},
usagetap: {
...result.provider_options?.usagetap,
...override.provider_options.usagetap,
},
};
}

Expand Down
2 changes: 2 additions & 0 deletions src/parser/loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,7 @@ function mergeRaw(base: PromptDefaults['raw'], local: PromptDefaults['raw']): Pr
google: mergeRecordBlock(base?.google, local?.google),
openrouter: mergeRecordBlock(base?.openrouter, local?.openrouter),
llmasaservice: mergeRecordBlock(base?.llmasaservice, local?.llmasaservice),
usagetap: mergeRecordBlock(base?.usagetap, local?.usagetap),
};

removeEmptyProviderBlocks(merged);
Expand All @@ -311,6 +312,7 @@ function mergeProviderOptions(
gemini: mergeRecordBlock(base?.gemini, local?.gemini),
openrouter: mergeRecordBlock(base?.openrouter, local?.openrouter),
llmasaservice: mergeRecordBlock(base?.llmasaservice, local?.llmasaservice),
usagetap: mergeRecordBlock(base?.usagetap, local?.usagetap),
};

removeEmptyProviderBlocks(merged);
Expand Down
11 changes: 11 additions & 0 deletions src/providers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export type {
ValidationResult,
RuntimeRenderOptions,
LLMAsAServiceRuntimeOptions,
UsageTapGatewayRuntimeOptions,
} from './types.js';
export { openaiAdapter } from './openai.js';
export { openaiResponsesAdapter } from './openai-responses.js';
Expand All @@ -20,6 +21,14 @@ export {
createLLMAsAServiceOpenAIConfig,
llmasaserviceAdapter,
} from './llmasaservice.js';
export {
USAGETAP_GATEWAY_BASE_URL,
USAGETAP_GATEWAY_DEFAULT_MODEL,
USAGETAP_GATEWAY_RESPONSE_HEADER_NAMES,
createUsageTapGatewayOpenAIConfig,
usagetapAdapter,
} from './usagetap.js';
export type { UsageTapGatewayOpenAIConfig, UsageTapGatewayOpenAIConfigOptions } from './usagetap.js';

import type { ProviderAdapter } from './types.js';
import { openaiAdapter } from './openai.js';
Expand All @@ -28,6 +37,7 @@ import { anthropicAdapter } from './anthropic.js';
import { geminiAdapter } from './gemini.js';
import { openrouterAdapter } from './openrouter.js';
import { llmasaserviceAdapter } from './llmasaservice.js';
import { usagetapAdapter } from './usagetap.js';

const adapters: Record<string, ProviderAdapter> = {
openai: openaiAdapter,
Expand All @@ -37,6 +47,7 @@ const adapters: Record<string, ProviderAdapter> = {
gemini: geminiAdapter,
openrouter: openrouterAdapter,
llmasaservice: llmasaserviceAdapter,
usagetap: usagetapAdapter,
};

/**
Expand Down
4 changes: 2 additions & 2 deletions src/providers/raw.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ import type { ResolvedPromptAsset } from '../schema/index.js';
export function applyRawProviderBody(
body: Record<string, unknown>,
asset: ResolvedPromptAsset,
provider: 'openai' | 'openai-responses' | 'anthropic' | 'gemini' | 'openrouter' | 'llmasaservice',
provider: 'openai' | 'openai-responses' | 'anthropic' | 'gemini' | 'openrouter' | 'llmasaservice' | 'usagetap',
): Record<string, unknown> {
const raw = getRawProviderBody(asset, provider);
return raw ? { ...body, ...raw } : body;
}

function getRawProviderBody(
asset: ResolvedPromptAsset,
provider: 'openai' | 'openai-responses' | 'anthropic' | 'gemini' | 'openrouter' | 'llmasaservice',
provider: 'openai' | 'openai-responses' | 'anthropic' | 'gemini' | 'openrouter' | 'llmasaservice' | 'usagetap',
): Record<string, unknown> | undefined {
if (provider === 'openai-responses') {
return asset.raw?.['openai-responses'] ?? asset.raw?.openai_responses;
Expand Down
7 changes: 7 additions & 0 deletions src/providers/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ export interface LLMAsAServiceRuntimeOptions {
apiKey: string;
}

/** Credentials and per-request metadata for the UsageTap gateway. */
export interface UsageTapGatewayRuntimeOptions {
apiKey: string;
idempotencyKey?: string;
}

export interface RuntimeHistoryMessage {
role: string;
content: string;
Expand Down Expand Up @@ -94,6 +100,7 @@ export interface RuntimeRenderOptions {
strict?: boolean;
openaiResponses?: OpenAIResponsesRuntimeOptions;
llmasaservice?: LLMAsAServiceRuntimeOptions;
usagetap?: UsageTapGatewayRuntimeOptions;
theTokenCompany?: TheTokenCompanyRuntimeOptions;
}

Expand Down
Loading
Loading