# A1 API Gateway — Integration Guide (consumer API v1)

> Machine-readable reference for calling A1 API Gateway from another project. Written so an AI coding
> assistant (Claude Code, Cursor, Copilot) can integrate correctly from this file alone. Every request
> and response shape below is exact. A human-readable companion is `docs/integration-guide.html`.
>
> **Verification status (2026-09-08):** the Anthropic path of everything below was exercised by hand against
> the running gateway with real vendor calls — 60 scenarios, all passing, in `tests/manual/anthropic-scenarios.md`.
> The OpenAI / xAI pass-through path has so far been verified against a mock vendor only; treat those rows as
> designed behaviour until a real key has been run through them. The same goes for embeddings, speech and
> transcription (added 2026-09-19): tested end to end against a mock of each vendor, not yet with real keys.

## 1. What this is

A1 API Gateway is an HTTP service that sits between your application and the AI vendors (Anthropic,
OpenAI and xAI for chat; OpenAI for images and embeddings; ElevenLabs, AssemblyAI and OpenAI for speech
and transcription). Your application never holds a vendor API key. It holds a **gateway key** (`sk-gw-…`),
calls the gateway with the vendor's own request format, and the gateway authenticates, enforces IP
rules, rate limits and spending caps, forwards the request with the real vendor key, meters the exact
cost, and returns the vendor's response unchanged plus a few `x-gw-*` headers.

```
your app  --(sk-gw key)-->  A1 API Gateway  --(real vendor key)-->  Anthropic / OpenAI / xAI / ElevenLabs / AssemblyAI
```

**Rules for any project using the gateway**
- Never call `api.anthropic.com`, `api.openai.com`, `api.x.ai`, `api.elevenlabs.io` or `api.assemblyai.com` directly.
- Never put a vendor key (`sk-ant-…`, `sk-proj-…`, `xai-…`, an ElevenLabs or AssemblyAI key) in the project. Only `sk-gw-…` keys.
- Read the base URL and the key from environment variables; never hard-code either.

## 2. Configuration

Two environment variables. The production address is `https://api.a1apigateway.com`.

```bash
AI_GATEWAY_URL=https://api.a1apigateway.com        # local development: http://127.0.0.1:8080
AI_GATEWAY_KEY=sk-gw-xxxxxx-xxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

The key is shown **once** when created in the gateway console. If it is lost, a new one must be issued.

## 3. Quickstart — the two-line change

Point the vendor SDK you already use at the gateway and use the gateway key. Nothing else changes.

### OpenAI SDK (works for every provider, including Claude)

The OpenAI SDK appends `/chat/completions` to `baseURL`, so `baseURL` **must end in `/v1`**.

```typescript
// TypeScript — npm install openai
import OpenAI from "openai";

const ai = new OpenAI({
  apiKey: process.env.AI_GATEWAY_KEY,
  baseURL: `${process.env.AI_GATEWAY_URL}/v1`,
});

const res = await ai.chat.completions.create({
  model: "anthropic/claude-opus-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Summarise this contract clause: …" }],
});
console.log(res.choices[0].message.content);
```

```python
# Python — pip install openai
import os
from openai import OpenAI

ai = OpenAI(
    api_key=os.environ["AI_GATEWAY_KEY"],
    base_url=f"{os.environ['AI_GATEWAY_URL']}/v1",
)

res = ai.chat.completions.create(
    model="anthropic/claude-opus-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this contract clause: …"}],
)
print(res.choices[0].message.content)
```

### Anthropic SDK (Claude models only, native Messages API)

The Anthropic SDK appends `/v1/messages` itself, so `baseURL` is the **bare** gateway URL (no `/v1`).
The SDK sends the key as `x-api-key`, which the gateway accepts.

```typescript
// TypeScript — npm install @anthropic-ai/sdk
import Anthropic from "@anthropic-ai/sdk";

const ai = new Anthropic({
  apiKey: process.env.AI_GATEWAY_KEY,
  baseURL: process.env.AI_GATEWAY_URL,
});

const res = await ai.messages.create({
  model: "anthropic/claude-opus-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Summarise this contract clause: …" }],
});
for (const block of res.content) if (block.type === "text") console.log(block.text);
```

```python
# Python — pip install anthropic
import os
import anthropic

ai = anthropic.Anthropic(
    api_key=os.environ["AI_GATEWAY_KEY"],
    base_url=os.environ["AI_GATEWAY_URL"],
)

res = ai.messages.create(
    model="anthropic/claude-opus-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this contract clause: …"}],
)
print("".join(b.text for b in res.content if b.type == "text"))
```

### curl

```bash
curl "$AI_GATEWAY_URL/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"anthropic/claude-opus-5","max_tokens":256,"messages":[{"role":"user","content":"ping"}]}'
```

## 4. Endpoints

| Method | Path | Purpose | Request/response format |
|---|---|---|---|
| POST | `/v1/chat/completions` | Chat with any provider | OpenAI Chat Completions |
| POST | `/v1/messages` | Chat with Claude models, native | Anthropic Messages (forwarded verbatim) |
| POST | `/v1/messages/count_tokens` | Count input tokens before sending (Claude) | Anthropic count_tokens (forwarded verbatim) |
| POST | `/v1/images/generations` | Generate pictures (OpenAI GPT-Image) | OpenAI Images (forwarded verbatim) — **key must allow images** |
| POST | `/v1/embeddings` | Turn text into vectors (OpenAI embedding models) | OpenAI Embeddings (forwarded verbatim) |
| POST | `/v1/audio/speech` | Text to speech (ElevenLabs, OpenAI `tts-1`) | OpenAI Speech; the reply is raw audio bytes |
| POST | `/v1/audio/transcriptions` | Speech to text (AssemblyAI, ElevenLabs Scribe, OpenAI `whisper-1`) | OpenAI Transcriptions (multipart upload) |
| GET | `/v1/models` | Models this key may use, with capabilities | OpenAI `list` shape + `capabilities` |
| GET | `/v1/key` | This key's limits, budgets, routing | gateway-specific JSON |

There are no other consumer endpoints. See §11 for what is not available.

## 5. Authentication

Either header works on every endpoint:

```
Authorization: Bearer sk-gw-…
x-api-key: sk-gw-…
```

Missing key → `401 missing_api_key`. Wrong key → `401 invalid_api_key`. The gateway never reveals
whether a key exists; both errors look the same to a caller.

## 6. Model names

Format: `provider/model`. The provider prefix selects the vendor; the rest is the vendor's own model id.

```
anthropic/claude-opus-5        anthropic/claude-sonnet-5       anthropic/claude-haiku-4-5
openai/gpt-5.6                 openai/gpt-5.4-mini             openai/text-embedding-3-small
xai/grok-4.3                   xai/grok-4.6
elevenlabs/eleven_flash_v2_5   elevenlabs/scribe_v2            assemblyai/universal-3-5-pro
openai/tts-1                   openai/whisper-1
```

Rules:
- The prefix may be omitted when the name is unambiguous: `claude-*` → anthropic; `gpt-*`, `o*`, `text-embedding-*`,
  `tts-*` and `whisper-*` → openai; `grok-*` → xai.
- ElevenLabs and AssemblyAI models **always need their prefix** (`elevenlabs/…`, `assemblyai/…`). A bare
  `eleven_flash_v2_5` or `universal-2` is `404 model_not_found`.
- The gateway admin can define **aliases** per organisation (e.g. `settlement-default` → `anthropic/claude-sonnet-5`). Prefer an alias in production code so the model can be changed without a deploy.
- `x-gw-model` in every response tells you which real model served the request after alias resolution.
- Unknown name → `404 model_not_found`. Known but not permitted for this key → `403 model_not_allowed`.
- On `/v1/messages`, only Anthropic models are accepted.

`GET /v1/models` returns exactly what this key may use, and for each model **what it may send** — check
`capabilities` instead of guessing:

```json
{"object":"list","data":[
  {"id":"anthropic/claude-opus-5","object":"model","created":1767225600,"owned_by":"anthropic",
   "capabilities":{"streaming":true,"images":true,"documents":true,"tools":true,"structured_output":true,
                   "reasoning":true,"prompt_caching":true,"sampling_params":false,"forced_tool_choice":true,
                   "endpoints":["/v1/chat/completions","/v1/messages","/v1/messages/count_tokens"]}},
  {"id":"openai/gpt-5","object":"model","created":1767225600,"owned_by":"openai",
   "capabilities":{"streaming":true,"images":true,"documents":true,"tools":true,"structured_output":true,
                   "reasoning":true,"prompt_caching":true,"sampling_params":true,"forced_tool_choice":true,
                   "endpoints":["/v1/chat/completions"]}},
  {"id":"openai/text-embedding-3-small","object":"model","created":1767225600,"owned_by":"openai",
   "capabilities":{"streaming":false,"images":false,"documents":false,"tools":false,"structured_output":false,
                   "reasoning":false,"prompt_caching":false,"sampling_params":false,"forced_tool_choice":false,
                   "endpoints":["/v1/embeddings"]}}
]}
```

| Field | Meaning |
|---|---|
| `true` / `false` | asserted by the gateway |
| `null` | the gateway does not know; the vendor decides — try it and handle a `400` |
| `documents` | PDFs and text documents as `file` content parts (§8.3b) |
| `prompt_caching` | repeated context billed at the cached rate — automatic, see §8.3c |
| `sampling_params` | `temperature` / `top_p` / `stop` accepted (newer Claude models: `false`, they are dropped with a warning) |
| `forced_tool_choice` | `tool_choice: "required"` / a named tool honoured (`false` → downgraded to `auto` with a warning) |
| `endpoints` | the consumer endpoints that accept this model — exactly one for image, embedding, speech and transcription models |

Embedding, speech and transcription models report every other flag as `false` (nothing streams through the
gateway on those endpoints), except `sampling_params`, which is `true` for transcription models (`temperature`).

`capabilities` is an extra field on the standard OpenAI list entry; OpenAI-shaped clients ignore it.

### One model, one door

Every model in the catalogue does one kind of thing, and answers on the endpoint for that thing only:

| Kind | Endpoint | Models |
|---|---|---|
| chat | `POST /v1/chat/completions` (Claude also on `/v1/messages`) | Claude, GPT, o-series, Grok |
| image | `POST /v1/images/generations` | `gpt-image-*` |
| embedding | `POST /v1/embeddings` | `text-embedding-*` |
| speech | `POST /v1/audio/speech` | `elevenlabs/eleven_*`, `tts-1`, `tts-1-hd` |
| transcription | `POST /v1/audio/transcriptions` | `assemblyai/*`, `elevenlabs/scribe_v2`, `whisper-1` |

A model sent to the wrong endpoint is refused **before** any vendor call with `400 invalid_request`, and the
message names the right one:

```
Model 'text-embedding-3-small' is an embedding model and answers on POST /v1/embeddings, not /v1/chat/completions.
```

This is a change for image models: until 2026-09-19 an image model at the chat door (or a chat model at the
image door) was `404 model_not_found`. Code that branched on that 404 should branch on the 400 instead.
`GET /v1/models` → `capabilities.endpoints` tells you which endpoint each model answers on.

### Prices worth knowing (catalogue re-read from the vendors on 2026-09-19)

- **Long context is billed at the higher rate for the whole request.** xAI charges every text model double
  (input and output) once the prompt is over 200K tokens; OpenAI charges 2× input and 1.5× output over 272K
  prompt tokens on its newer models (`gpt-6-astra`, `gpt-5.6`, `gpt-5.6-sol`/`-terra`/`-luna`, `gpt-5.5`, `gpt-5.4`).
  The gateway picks the rate from the vendor's own prompt count (input + cached + cache-write tokens), and
  reserves at the higher rate when the estimated prompt is already over the line, so a large prompt can meet
  `402 request_too_expensive` sooner than its short-context price suggests.
- **`gpt-5.6` is $4 input / $0.40 cached / $20 output per 1M tokens** (it was priced at $5 / $0.50 / $30).
  It is OpenAI's alias for GPT-5.6 Sol, at a promotional price OpenAI states runs at least until 21 November 2026.
- **Retired xAI names bill as `grok-4.3`.** xAI retired `grok-4`, `grok-4-fast` and `grok-3` on 2026-05-15 and
  now serves and bills them as `grok-4.3` ($1.25 / $0.20 / $2.50 per 1M tokens), so that is what they cost here.
  Move to `xai/grok-4.3` or a newer model by name.
- **`grok-3-mini` is switched off** — it is on no current xAI price list. Requests for it are `404 model_not_found`.

Corrections apply from 2026-09-19; requests already made keep the price they were charged. `x-gw-cost-usd`
is always what you were actually charged.

## 7. Which endpoint to use

**Use `/v1/chat/completions`.** It is the standard door for every project: one format for every
provider, PDFs and documents, prompt caching (automatic), tools, structured output, reasoning, images,
streaming — and every request is validated and priced by the gateway before a vendor is called, so
spending caps are exact.

`/v1/messages` exists for the rare Claude-only feature the OpenAI format cannot express (citations,
`anthropic-beta` features, vendor server tools). It forwards the body untouched, which also means
price-changing fields reach the vendor unpriced. **Ask the gateway admin before using it.**

For Claude models, `/v1/chat/completions` is translated; §11 lists the few OpenAI parameters that do
not map, and every ignored field is reported in `x-gw-warnings` — nothing is dropped silently.

Pictures, embeddings, speech and transcription each have a door of their own, below. A model sent to the
wrong one is a `400 invalid_request` that names the right one (§6, "One model, one door").

| You want | Endpoint | Section |
|---|---|---|
| a picture | `POST /v1/images/generations` | Image generation |
| vectors for search or clustering | `POST /v1/embeddings` | Embeddings |
| audio from text | `POST /v1/audio/speech` | Speech |
| text from audio | `POST /v1/audio/transcriptions` | Transcription |

### Image generation

`POST /v1/images/generations` — OpenAI's own path and body, forwarded as sent. The reply is the vendor's,
unchanged: each picture is in `data[].b64_json`.

```bash
curl "$AI_GATEWAY_URL/v1/images/generations" \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" \
  -H "content-type: application/json" \
  -d '{
        "model": "openai/gpt-image-2.5-flare",
        "prompt": "a single ripe lemon on a white marble counter, morning light",
        "n": 1,
        "size": "1024x1024",
        "quality": "low"
      }'
```

The OpenAI SDK works unchanged — `client.images.generate({...})` with `baseURL` pointing at the gateway.

**Models.** `gpt-image-2.5-flare` (fast, the sensible default) and `gpt-image-2.5-sunburst` (highest
quality); `gpt-image-2` is still priced. Prefixed (`openai/gpt-image-2.5-flare`) or bare both work.
DALL·E is **not** available — OpenAI removed `dall-e-2` and `dall-e-3` from the API on 2026-05-12.

**Parameters.**

| Field | Values | Notes |
|---|---|---|
| `prompt` | text, required | |
| `n` | 1–10 | bounded by your key; each picture is billed |
| `size` | `1024x1024`, `1536x1024`, `1024x1536`, `auto`, or any `WIDTHxHEIGHT` | bounded by your key, compared by area |
| `quality` | `low`, `medium`, `high`, `xhigh`, `max`, `auto` | bounded by your key |
| `output_format` | `png`, `jpeg`, `webp` | default `png` |
| `background` | `transparent`, `opaque`, `auto` | `transparent` needs `png` or `webp` |
| `stream` | — | **refused**, `400 unsupported_parameter` |

#### Your key's image settings

Image generation is **off by default on every key**. An admin switches it on per key, along with ceilings
on how many pictures per request, how large, at what quality, which image models, and an optional cost cap.

Read your own from `GET /v1/key`, so you can check at startup instead of discovering a limit in production:

```json
"images": {
  "allowed": true,
  "max_images_per_request": 2,
  "max_size": "1536x1024",
  "max_quality": "high",
  "max_cost_per_request_usd": null,
  "allowed_models": []
}
```

`allowed_models: []` means every image model the key may already use. A request over any ceiling is refused
**before** a vendor is called, so it costs nothing.

#### What you will get back when it is refused

| Situation | Status and code |
|---|---|
| Images not switched on for this key | `403 service_not_allowed` |
| A chat model sent here | `400 invalid_request` — tells you to use `/v1/chat/completions` (was `404 model_not_found` before 2026-09-19) |
| An image model sent to `/v1/chat/completions` | `400 invalid_request` — tells you to come here (was `404 model_not_found`) |
| `n`, `size` or `quality` above your key's ceiling | `400 invalid_request`, naming the ceiling |
| Worst case above a per-request cost cap | `402 request_too_expensive`, naming estimate and cap |
| `stream: true` | `400 unsupported_parameter` |
| Budget or credit exhausted | `402`, as anywhere else |
| The prompt itself refused by OpenAI | `400 invalid_request`, prefixed `The provider rejected the request:` |

#### Cost and timing

Billed in tokens like everything else: the prompt at the model's input rate, the pictures at its output
rate. `x-gw-cost-usd` is exact. Unlike chat, the worst case is known before the call — how many pictures,
how large and at what quality are all in the body — so the reservation is tight rather than a guess.

At 1024x1024 a `low` picture is about **$0.006** and `max` about **$0.21** — roughly forty times. Pick
`quality` deliberately rather than leaving it to `auto`; `auto` is reserved at the most your key allows,
because you have not said.

Expect **10–60 seconds**. There is no streaming, so set your client's timeout accordingly.

### Embeddings

`POST /v1/embeddings` — OpenAI's own path and body. The body goes to OpenAI exactly as you sent it, with only
`model` rewritten to the vendor's id, and OpenAI's reply comes back unchanged.

```bash
curl "$AI_GATEWAY_URL/v1/embeddings" \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "openai/text-embedding-3-small",
        "input": ["Clause 4 limits liability to the fees paid.", "Payment is due within 30 days."]
      }'
```

```json
{"object":"list",
 "data":[{"object":"embedding","index":0,"embedding":[-0.0061,0.0142,…]},
         {"object":"embedding","index":1,"embedding":[0.0217,-0.0093,…]}],
 "model":"text-embedding-3-small",
 "usage":{"prompt_tokens":19,"total_tokens":19}}
```

With the SDKs (`ai` is the client from §3):

```typescript
const res = await ai.embeddings.create({
  model: "openai/text-embedding-3-small",
  input: ["Clause 4 limits liability to the fees paid.", "Payment is due within 30 days."],
});
const vectors = res.data.map((d) => d.embedding); // number[] each, 1536 long
```

```python
res = ai.embeddings.create(
    model="openai/text-embedding-3-small",
    input=["Clause 4 limits liability to the fees paid.", "Payment is due within 30 days."],
)
vectors = [d.embedding for d in res.data]  # list[float] each, 1536 long
```

**Models and prices** — per 1M input tokens. Nothing is generated, so there is no output price.

| Model | Price | Vector length (default) |
|---|---|---|
| `openai/text-embedding-3-small` | $0.02 | 1536 |
| `openai/text-embedding-3-large` | $0.13 | 3072 |
| `openai/text-embedding-ada-002` | $0.10 | 1536 |

The prefix is optional (`text-embedding-3-small` works too).

**Parameters.**

| Field | Values | Notes |
|---|---|---|
| `input` | a string, an array of strings, an array of token ids, or an array of token-id arrays | required; an empty string or empty array is `400 invalid_request` before OpenAI is called |
| `encoding_format` | `float` (OpenAI's default), `base64` | passed through as sent — see below |
| `dimensions` | integer | `text-embedding-3-*` only; shortens the vector. OpenAI decides |
| `user` | string | passed through |

**Floats or base64.** The gateway sends `encoding_format` only if you did: ask for nothing and you get arrays of
floats, ask for `base64` and you get base64 strings, untouched. The gateway never switches to base64 behind your
back. (The OpenAI SDKs do so themselves: when you leave `encoding_format` out, `embeddings.create` asks for
base64 on the wire and decodes it back into floats for you. That works unchanged through the gateway. With curl
or a plain HTTP client you get floats unless you ask.)

**What is billed.** `usage.prompt_tokens` from OpenAI's reply, at the model's input rate, and nothing else.
Before the call the gateway reserves an over-estimate — text at one token per 3 bytes, token-id arrays at their
exact length — which counts against your per-request cap, then settles to the real
count. `x-gw-cost-usd` is exact.

**Limits OpenAI enforces.** 8,192 tokens per input, 2,048 inputs per request, 300,000 tokens across all inputs
in one request. The gateway does not pre-check these; OpenAI's refusal comes back as `400 invalid_request`
prefixed `The provider rejected the request:`. The gateway's own body limit is 20 MB (`413 request_too_large`).
Split large batches into several requests.

**Retries.** The gateway retries once itself on a transient vendor failure (a 5xx or a dropped connection).
That is safe because an embedding is idempotent.

**Prompt log.** On a key that logs at `full`, the input is kept but the vectors are not (`"[1536 dimensions not stored]"`).

#### What you will get back when it is refused

| Situation | Status and code |
|---|---|
| A chat, image, speech or transcription model sent here | `400 invalid_request` — names the endpoint to use |
| An embedding model sent to `/v1/chat/completions` | `400 invalid_request` — tells you to come here |
| `input` missing, empty, or not one of the four shapes | `400 invalid_request`, before any vendor call |
| Over one of OpenAI's limits above | `400 invalid_request`, prefixed `The provider rejected the request:` |
| Estimate above your key's per-request cost cap | `402 request_too_expensive`, naming estimate and cap |
| Budget or credit exhausted | `402`, as anywhere else |
| Model not permitted for this key or organisation | `403 model_not_allowed` |
| A mail key (`"service": "mail"` in `GET /v1/key`) | `403 service_not_allowed` |
| OpenAI rejected the vendor key | `503 credential_unavailable` — the vendor key is marked unhealthy and nothing is charged; tell the admin |

### Speech

`POST /v1/audio/speech` — OpenAI's speech request, answered by ElevenLabs or by OpenAI's `tts-1` / `tts-1-hd`.
The reply is **the audio itself**: raw bytes with the vendor's `Content-Type` (`audio/mpeg` for mp3), not JSON.
Errors are JSON as usual, so check the status before writing the body to a file.

```bash
curl "$AI_GATEWAY_URL/v1/audio/speech" \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "elevenlabs/eleven_flash_v2_5",
        "voice": "narrator",
        "input": "Your settlement has been approved and will be paid on Friday.",
        "response_format": "mp3"
      }' \
  --fail-with-body --output reply.mp3
```

```typescript
import { writeFile } from "node:fs/promises";

const res = await ai.audio.speech.create({
  model: "elevenlabs/eleven_flash_v2_5",
  voice: "narrator", // a name your organisation set up, or an ElevenLabs voice id
  input: "Your settlement has been approved and will be paid on Friday.",
  response_format: "mp3",
});
await writeFile("reply.mp3", Buffer.from(await res.arrayBuffer()));
```

```python
res = ai.audio.speech.create(
    model="elevenlabs/eleven_flash_v2_5",
    voice="narrator",  # a name your organisation set up, or an ElevenLabs voice id
    input="Your settlement has been approved and will be paid on Friday.",
    response_format="mp3",
)
res.write_to_file("reply.mp3")
```

**Models and prices** — per character of `input`.

| Model | Price | Notes |
|---|---|---|
| `elevenlabs/eleven_flash_v2_5` | $0.05 per 1K characters | fastest and cheapest, multilingual |
| `elevenlabs/eleven_flash_v2` | $0.05 per 1K characters | English only |
| `elevenlabs/eleven_multilingual_v2` | $0.10 per 1K characters | |
| `elevenlabs/eleven_v3` | $0.10 per 1K characters | |
| `elevenlabs/eleven_turbo_v2_5`, `elevenlabs/eleven_turbo_v2` | $0.05 per 1K characters | deprecated by ElevenLabs — use Flash |
| `tts-1` (or `openai/tts-1`) | $15 per 1M characters ($0.015 per 1K) | |
| `tts-1-hd` | $30 per 1M characters ($0.03 per 1K) | |

ElevenLabs models need the `elevenlabs/` prefix. `gpt-4o-mini-tts` is not offered (§11).

**The voice.**
- **OpenAI models:** `voice` goes to OpenAI unchanged — `alloy`, `echo`, `nova` and the rest of OpenAI's names.
- **ElevenLabs models:** `voice` is either
  1. a **name your organisation has set up** in the console under Providers → ElevenLabs → Voices (say
     `narrator`, pointing at an ElevenLabs voice). Names are matched exactly, case included. An admin can point
     the name at a different voice later without any change to your code; or
  2. a **raw ElevenLabs voice id** — exactly 20 letters and digits, e.g. `21m00Tcm4TlvDq8ikWAM`.

  Anything else is `400 invalid_request`, and the message lists your organisation's voice names. OpenAI's names
  do **not** carry over: `"voice": "alloy"` with an `elevenlabs/…` model is refused, unless your organisation
  has named one of its voices `alloy`.
- `voice` must be a string. OpenAI's custom-voice object (`{"id": "voice_…"}`) is refused with `400`.
- A voice cloned in an ElevenLabs account can only be used through that account's key. If your organisation
  cloned it, requests must route to the organisation's own ElevenLabs key — `GET /v1/key` → `routing.elevenlabs`
  shows which key is used.

**Parameters.**

| Field | OpenAI models | ElevenLabs models |
|---|---|---|
| `input` | required | required. Each vendor caps the length (OpenAI: 4,096 characters); its refusal comes back as `400 invalid_request` |
| `response_format` | `mp3` (default), `opus`, `aac`, `flac`, `wav`, `pcm` | `mp3` (default; 44.1 kHz, 128 kbit/s), `opus` (48 kHz, 128 kbit/s), `wav` (24 kHz), `pcm` (raw 24 kHz 16-bit mono, the same as OpenAI's). **`aac` and `flac` are `400 invalid_request`**, before the call |
| `speed` | 0.25–4.0 (OpenAI decides) | **0.7–1.2**; outside that, `400 invalid_request` before the call |
| `instructions` | passed to OpenAI | ignored, and named in `x-gw-warnings` |
| `stream`, `stream_format: "sse"` | refused, `400 unsupported_parameter` | refused, `400 unsupported_parameter` |

For OpenAI models the body is forwarded as sent (only `model` rewritten). For ElevenLabs models only the fields
above are used.

**What is billed.** The characters of `input` at the model's rate, counted as Unicode code points (an emoji is
one character). The count is known before the call, so the reservation is exact. For ElevenLabs, the vendor's
own figure wins when it reports one (the `character-cost` response header): a Voice Library voice with a custom
rate costs more characters than the text has. `x-gw-cost-usd` is exact. As a guide, 1,000 characters cost $0.05
on Flash, $0.10 on v3 or Multilingual v2 and $0.015 on `tts-1`.

**Access and limits.** Any AI key whose model allowlist includes the model may use it; unlike images, audio has
no separate switch on the key. Tokens per minute does not apply — there are no `x-ratelimit-*-tokens` headers
— but requests per minute, concurrency, budgets and credit do. On a key that logs at `full`, the text is kept,
never the audio.

#### What you will get back when it is refused

| Situation | Status and code |
|---|---|
| `input` or `voice` missing | `400 invalid_request` |
| ElevenLabs model with a `voice` that is neither an organisation name nor a voice id | `400 invalid_request`, listing your organisation's voice names |
| ElevenLabs model with `aac` / `flac`, or `speed` outside 0.7–1.2 | `400 invalid_request`, before the call |
| `stream: true` or `stream_format: "sse"` | `400 unsupported_parameter` |
| A model that is not a speech model | `400 invalid_request`, naming its endpoint |
| Estimate above your key's per-request cost cap | `402 request_too_expensive` |
| Budget or credit exhausted | `402`, as anywhere else |
| Model not permitted | `403 model_not_allowed` |
| A mail key | `403 service_not_allowed` |
| The vendor rejected its key | `503 credential_unavailable` — nothing charged; tell the admin |

### Transcription

`POST /v1/audio/transcriptions` — OpenAI's transcription upload, answered by AssemblyAI, ElevenLabs Scribe or
OpenAI's `whisper-1`. Send `multipart/form-data` with the audio in `file`, exactly as for OpenAI.

```bash
curl "$AI_GATEWAY_URL/v1/audio/transcriptions" \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" \
  -F model=assemblyai/universal-3-5-pro \
  -F file=@call-recording.mp3 \
  -F response_format=json \
  --max-time 900
```

```json
{"text":"Hello, I'm calling about settlement S-1042…","usage":{"type":"duration","seconds":184}}
```

```typescript
import { createReadStream } from "node:fs";

const t = await ai.audio.transcriptions.create(
  { model: "assemblyai/universal-3-5-pro", file: createReadStream("call-recording.mp3"), language: "en" },
  { timeout: 15 * 60 * 1000 }, // AssemblyAI can take up to 10 minutes: wait longer than the gateway does
);
console.log(t.text);
```

```python
with open("call-recording.mp3", "rb") as f:
    t = ai.with_options(timeout=900).audio.transcriptions.create(  # wait longer than the gateway's 10 minutes
        model="assemblyai/universal-3-5-pro",
        file=f,
        language="en",
    )
print(t.text)
```

**Models and prices** — per second of audio, rounded up to a whole second.

| Model | Price | Notes |
|---|---|---|
| `assemblyai/universal-3-5-pro` | $0.21 per hour | the only one that takes `prompt` at AssemblyAI |
| `assemblyai/universal-2` | $0.15 per hour | |
| `elevenlabs/scribe_v2` | $0.22 per hour | |
| `whisper-1` (or `openai/whisper-1`) | $0.006 per minute ($0.36 per hour) | OpenAI shuts it down on 26 February 2027 |

AssemblyAI and ElevenLabs models need their prefix.

**Fields** (multipart).

| Field | Notes |
|---|---|
| `file` | required. Up to **25 MB**; larger is `413 request_too_large` ("Audio upload exceeds 25 MB (26214400 bytes)."). Common formats (mp3, m4a, wav, ogg, webm, flac) work with all three vendors; the vendor decides |
| `model` | required |
| `language` | optional ISO-639-1 code (`en`, `de`, …). Without it each vendor detects the language |
| `prompt` | sent to `whisper-1` and AssemblyAI; ignored for ElevenLabs Scribe, with an `x-gw-warnings` note |
| `response_format` | `json` (default), `text`, `verbose_json`, `srt`, `vtt` — for every vendor, see below. Anything else (such as `diarized_json`) is `400 invalid_request` |
| `temperature` | sent to every vendor |
| `timestamp_granularities[]` | ignored, with an `x-gw-warnings` note: the gateway always asks for word timings, and `verbose_json` carries word and segment timings whenever the vendor provides them |
| `stream` | `true` is refused, `400 unsupported_parameter` |

**Every format works with every vendor.** The gateway renders the five formats itself from the vendor's answer,
so subtitles from AssemblyAI or ElevenLabs have the same shape as from `whisper-1`.

| `response_format` | `Content-Type` | Body |
|---|---|---|
| `json` | `application/json` | `{"text": "…", "usage": {"type": "duration", "seconds": 184}}` — `seconds` is what you are billed for |
| `text` | `text/plain` | the transcript |
| `verbose_json` | `application/json` | `{"task": "transcribe", "language", "duration", "text", "words": [{"word", "start", "end"}], "segments": [{"id", "start", "end", "text"}]}`, times in seconds |
| `srt` | `text/plain` | numbered SubRip cues |
| `vtt` | `text/vtt` | WebVTT cues |

For AssemblyAI and ElevenLabs, segments and subtitle cues are built from the word timings: a cue ends at the end
of a sentence, at a pause of more than a second, or at 7 seconds or 16 words. `whisper-1`'s own segments are used
as they are. `language` is whatever the vendor reports, so its form differs between vendors.

**It can take a while.** `whisper-1` and Scribe answer in one call. AssemblyAI works asynchronously: the gateway
uploads the file, submits the job and polls until it finishes, all inside your one request, which for a long
recording can take minutes. The gateway waits up to **10 minutes**, upload included. **Set your client's timeout
above that**, as in the examples — the OpenAI SDKs default to exactly 10 minutes, and curl to none. If your client
gives up first, the SDK counts it as a timeout and retries: a second job, charged again, while the first one
still finishes and is charged too.

**What is billed.** Seconds of audio as the vendor reports them, rounded up (at least 1). The length is not known
before the call, so the gateway reserves as if the file were compressed speech at 32 kbit/s (file bytes ÷ 4,000
= seconds), then settles to the real duration. That over-counts almost any real file — uncompressed 16 kHz WAV by
16× — so a 25 MB WAV holds about 6,550 seconds (about $0.38 at Universal-3.5 Pro) until it settles. With a tight
per-request cap or a nearly spent budget, send compressed audio (mp3, opus) so the reservation fits.

**A 504 here is charged.** If AssemblyAI has accepted the job but not finished it within the 10 minutes, you get
`504 upstream_timeout` — and the request **is charged** at the reservation estimate, because AssemblyAI finishes
the job and bills for it regardless. The 504 carries the charge like any billed response — `x-gw-cost-usd`,
`x-gw-billing`, `x-gw-usage-estimated: 1` — and `x-should-retry: false`, and its message says that retrying will
transcribe and charge again. The OpenAI SDKs obey `x-should-retry`, so they do not retry it; **if you have retry
logic of your own, make it honour that header too**. The transcript cannot be fetched through the gateway
afterwards. Disconnecting does not cancel a job either — it is still charged.

**Access and limits.** As for speech: any AI key whose model allowlist includes the model, no per-minute token
limit, requests per minute, concurrency, budgets and credit as usual. On a key that logs at `full`, the transcript
and the form fields are kept, never the audio.

#### What you will get back when it is refused

| Situation | Status and code |
|---|---|
| Not `multipart/form-data`, or no `file` | `400 invalid_request`, before any vendor call |
| `response_format` not one of the five | `400 invalid_request` |
| `stream=true` | `400 unsupported_parameter` |
| A model that is not a transcription model | `400 invalid_request`, naming its endpoint |
| AssemblyAI could not transcribe the file (too short, unreadable) | `400 invalid_request`, prefixed `The provider rejected the request:` — nothing charged |
| Size-based estimate above your key's per-request cost cap | `402 request_too_expensive` |
| Budget or credit exhausted | `402`, as anywhere else |
| Model not permitted / a mail key | `403 model_not_allowed` / `403 service_not_allowed` |
| File over 25 MB | `413 request_too_large`. The gateway refuses on the `Content-Length` header, so a client that streams a larger body may see the connection closed instead of the 413 — check the size before uploading |
| AssemblyAI rate-limiting the gateway's key (it signals this with a 403) | `429 upstream_rate_limited`, with `retry-after` |
| The vendor rejected its key | `503 credential_unavailable` — nothing charged; tell the admin |
| AssemblyAI still working after 10 minutes | `504 upstream_timeout` with `x-should-retry: false` — **charged** at the estimate; do not retry |

## 8. Features, with exact request shapes

### 8.1 System prompt and multi-turn

Both doors are stateless — send the full history each call.

```json
POST /v1/chat/completions
{"model":"anthropic/claude-opus-5","max_tokens":512,
 "messages":[
   {"role":"system","content":"You are a settlement assistant. Answer in one sentence."},
   {"role":"user","content":"My name is Harshad."},
   {"role":"assistant","content":"Hello Harshad."},
   {"role":"user","content":"What is my name?"}]}
```

```json
POST /v1/messages
{"model":"anthropic/claude-opus-5","max_tokens":512,
 "system":"You are a settlement assistant. Answer in one sentence.",
 "messages":[
   {"role":"user","content":"My name is Harshad."},
   {"role":"assistant","content":"Hello Harshad."},
   {"role":"user","content":"What is my name?"}]}
```

### 8.2 Streaming (SSE)

Add `"stream": true`. The response is `Content-Type: text/event-stream`.

- `/v1/chat/completions` emits OpenAI `chat.completion.chunk` objects, then a trailer comment line
  carrying the cost, then `data: [DONE]`.
- `/v1/messages` emits Anthropic's events (`message_start`, `content_block_delta`, `message_delta`,
  `message_stop`) verbatim, then the same trailer comment.

```
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hel"}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"lo"}}]}

: x-gw-cost-usd=0.000146 x-gw-billing=byok

data: [DONE]
```

The trailer is an SSE **comment** (line starts with `:`); SDKs ignore it, so read it only if you want
the cost of a streamed call. `x-gw-usage-estimated=1` is appended when the vendor's final usage event
never arrived and the cost is the worst-case estimate.

Both SDKs' normal streaming helpers work unchanged (`stream: true` / `ai.messages.stream(...)`).

If your client disconnects mid-stream, the gateway still drains the vendor's response (up to 60 s)
and bills the tokens the vendor actually generated. Disconnecting does not refund a request.

### 8.3 Images

Accepted: `image/jpeg`, `image/png`, `image/gif`, `image/webp`. As an `https://` URL or base64.
Image URLs are fetched by the **vendor's** servers, not yours — URLs behind logins, private networks
or hotlink protection fail with `400 invalid_request` (the vendor's message is relayed) — and so do hosts whose
`robots.txt` disallows crawlers, which Anthropic honours (picsum.photos and Wikimedia, for example). Use base64
for anything you do not control.

```json
POST /v1/chat/completions
{"model":"anthropic/claude-opus-5","max_tokens":256,"messages":[{"role":"user","content":[
  {"type":"image_url","image_url":{"url":"data:image/png;base64,iVBORw0KGgo…"}},
  {"type":"text","text":"What is in this image?"}]}]}
```

```json
POST /v1/messages
{"model":"anthropic/claude-opus-5","max_tokens":256,"messages":[{"role":"user","content":[
  {"type":"image","source":{"type":"base64","media_type":"image/png","data":"iVBORw0KGgo…"}},
  {"type":"text","text":"What is in this image?"}]}]}
```

For a URL on `/v1/messages`: `{"type":"image","source":{"type":"url","url":"https://…"}}`.

### 8.3b PDFs and documents

Send a `file` content part with the document inline as a base64 data URL. PDFs (`application/pdf`) and
text types (`text/*`, `application/json`, CSV, YAML, XML) are accepted. The gateway converts it to
Claude's document block (the filename becomes the document title); for OpenAI models it passes through.
Other types (Word, Excel, …) are refused with `400 unsupported_parameter` naming the media type.
`file_id` references are refused — there is no Files API through the gateway; send the bytes inline.

```json
POST /v1/chat/completions
{"model":"anthropic/claude-opus-5","max_tokens":1024,"messages":[{"role":"user","content":[
  {"type":"file","file":{"filename":"contract.pdf","file_data":"data:application/pdf;base64,JVBERi0xLjQK…"}},
  {"type":"text","text":"What is the termination notice period?"}]}]}
```

Cost is reserved **per page** (worst case 4,600 tokens/page: text plus the page image), then settled to
the real usage. Limits: 32 MB per request at Anthropic (the gateway allows 20 MB bodies), 100 pages per
PDF on 200K-context models.

### 8.3c Prompt caching (automatic)

Repeated context — a long system prompt, a big tool list, the earlier turns of a conversation — is
cached by Anthropic and billed at **10 % of the input price** on the next call within the cache TTL
(5 minutes, refreshed on use). The gateway turns this on for every Claude request; you do nothing.
Writing the cache costs 1.25× on the cached prefix once, so it pays for itself on the second call.

- Only prefixes above the model's minimum (1,024–4,096 tokens depending on model) are cached; shorter
  ones are simply not cached, nothing fails.
- Keep the stable part first and the changing part last: system prompt and tools, then history, then
  the new message. Anything before a byte that changes is still cached; anything after it is not.
- Verify with `usage.prompt_tokens_details.cached_tokens` in the response — non-zero means a cache hit.
- Opt out per request with `"cache_control": false`. Pass `{"type":"ephemeral","ttl":"1h"}` for the
  one-hour tier.

### 8.4 Tool calling (function calling)

OpenAI `tools` / `tool_choice` are translated to Anthropic tools. The reply comes back as OpenAI
`tool_calls` with `finish_reason: "tool_calls"`. Return results with `role: "tool"`.

```json
POST /v1/chat/completions
{"model":"anthropic/claude-opus-5","max_tokens":512,
 "tools":[{"type":"function","function":{"name":"get_settlement","description":"Look up a settlement by id",
   "parameters":{"type":"object","properties":{"id":{"type":"string"}},"required":["id"]}}}],
 "messages":[{"role":"user","content":"Status of settlement S-1042?"}]}
```

Response (abridged):

```json
{"choices":[{"finish_reason":"tool_calls","message":{"role":"assistant","content":null,
  "tool_calls":[{"id":"toolu_01…","type":"function","function":{"name":"get_settlement","arguments":"{\"id\":\"S-1042\"}"}}]}}]}
```

Then continue with the assistant message echoed back plus
`{"role":"tool","tool_call_id":"toolu_01…","content":"{\"status\":\"paid\"}"}`.

Notes:
- `strict: true` on a function is honoured: the vendor guarantees the arguments validate against the schema
  (the schema needs `additionalProperties: false` and `required`, as with OpenAI).
- Only `tools[].type: "function"` is supported. Legacy `functions` / `function_call` → `400`.
- `tool_choice: "required"` or `{"type":"function",…}` is honoured where the model allows it; on models
  that only accept `auto` (Fable 5.1+) it is downgraded to `auto` with an `x-gw-warnings` header.
- On `/v1/messages`, use Anthropic's native `tools` / `tool_use` / `tool_result` shapes unchanged.

### 8.5 Structured (JSON) output

`response_format: {"type":"json_schema", "json_schema": {"name": "...", "schema": {...}}}` is mapped to
Anthropic structured output — the shape is guaranteed. `{"type":"json_object"}` has no Claude
equivalent: the gateway appends a system instruction to answer with a single JSON object (warning
header set). That is a request, not a guarantee — use `json_schema` when the shape matters.

```json
{"model":"anthropic/claude-opus-5","max_tokens":256,
 "response_format":{"type":"json_schema","json_schema":{"name":"verdict",
   "schema":{"type":"object","properties":{"approved":{"type":"boolean"},"reason":{"type":"string"}},
            "required":["approved","reason"],"additionalProperties":false}}},
 "messages":[{"role":"user","content":"Should settlement S-1042 be approved? It is 3 days late."}]}
```

### 8.6 Reasoning / extended thinking

`reasoning_effort` on `/v1/chat/completions` is mapped to Claude's adaptive thinking:

| `reasoning_effort` | Effect |
|---|---|
| omitted | model default (adaptive thinking on current Claude models) |
| `none` | thinking disabled |
| `low` `medium` `high` `xhigh` `max` | adaptive thinking at that effort |
| `minimal` | mapped to `low` (warning header) |

On models without adaptive thinking (Haiku 4.5, Sonnet 4.5 and older) the level becomes a fixed thinking
budget instead — `low` 1,024 · `medium` 4,096 · `high` 16,000 · `xhigh` 24,000 · `max` 32,000 tokens, capped
below `max_tokens` — and `temperature` / `top_p` are dropped, since those models cannot combine them with
thinking. Both are reported in `x-gw-warnings`. If `max_tokens` leaves no room for a budget (≤ 1,024), thinking
is skipped with a warning rather than failing the request.

On `/v1/messages`, send `thinking` / `output_config` exactly as the Anthropic API expects.

### 8.7 Sampling parameters

`temperature`, `top_p`, `stop` pass through. Anthropic accepts `temperature` 0–1; values above 1 are
clamped with a warning. Newer Claude models (Opus 4.7+, Sonnet 5+, Fable) do not accept sampling
parameters at all — they are dropped with a warning rather than failing the request.

### 8.8 Token counting (Claude) — `POST /v1/messages/count_tokens`

Free at the vendor, so the gateway charges nothing and reserves nothing; the call still counts against
your requests-per-minute limit and appears in the request log at cost 0. Body is the Anthropic
`count_tokens` shape — the same `model`, `system`, `messages`, `tools` you are about to send — and the
reply is returned untouched. Anthropic models only.

```json
POST /v1/messages/count_tokens
{"model":"anthropic/claude-opus-5","system":"Be brief.","messages":[{"role":"user","content":"How long is this?"}]}
```
```json
{"input_tokens":21}
```

Both SDKs' helpers work as-is: `ai.messages.countTokens({...})` (TypeScript) / `ai.messages.count_tokens(...)` (Python).
Use it to estimate cost before an expensive call: `input_tokens × input price + max_tokens × output price`.

### 8.9 `max_tokens`

- If omitted, the key's configured **default** is used.
- Each key has a **maximum**; asking for more → `400 invalid_request` before anything is sent.
- The gateway reserves budget for the worst case (`max_tokens` × output price) before calling the vendor,
  then settles to the real cost. A large `max_tokens` can therefore trip `402 request_too_expensive` or
  `402 budget_exceeded` even if the actual answer would have been short. Set `max_tokens` to what you need.

## 8b. Sending mail — `POST /v3/mail/send`

SendGrid's own path and wire format, so an existing project moves onto the gateway by changing two lines:

```js
// TypeScript
sgMail.setApiKey(process.env.AI_GATEWAY_KEY);
sgMail.client.setDefaultRequest("baseUrl", process.env.AI_GATEWAY_URL);
```

```python
# Python
sg = SendGridAPIClient(os.environ["AI_GATEWAY_KEY"])
sg.client.host = os.environ["AI_GATEWAY_URL"]
```

```bash
curl -sS $AI_GATEWAY_URL/v3/mail/send \
  -H "Authorization: Bearer $AI_GATEWAY_KEY" -H "Content-Type: application/json" \
  -d '{"personalizations":[{"to":[{"email":"someone@example.com"}]}],
       "from":{"email":"noreply@yourdomain.com"},
       "subject":"Hello","content":[{"type":"text/plain","value":"Hi there"}]}'
# 202 Accepted, empty body — exactly as SendGrid answers
```

The body is forwarded **verbatim**. The gateway authenticates the key, applies its address rules, counts the
recipients and attributes the send; it never rewrites the message.

**A mail key is a different kind of key**: scoped to this endpoint, unable to call a model, and an AI key
cannot send mail. Either mistake is `403 service_not_allowed`.

**Limits count recipients, not requests** — one call can address up to 1,000 people. The key carries ceilings
per minute, per hour and per day plus a maximum per send; over any of them the send is refused with
`429 recipient_limit_exceeded` and nothing reaches SendGrid. The response carries `x-gw-recipients` and
`x-ratelimit-remaining-recipients-minute` / `-hour` / `-day`.

| Situation | Status and code |
|---|---|
| An AI key here, or a mail key at a model endpoint | `403 service_not_allowed` |
| A body whose recipients cannot be counted | `400 invalid_request` — never forwarded uncounted |
| Over a recipient ceiling | `429 recipient_limit_exceeded`, with `retry-after` |
| SendGrid rejected the message | Its own status and body, relayed; the recipients are given back |
| No SendGrid key on the organisation | `503 credential_unavailable` |

## 9. Response headers

Present on every response, success or error, on both endpoints:

| Header | Example | Meaning |
|---|---|---|
| `x-gw-request-id` | `req_01M1S1S2…` | Quote this when asking the gateway admin about a request |
| `x-gw-model` | `anthropic/claude-opus-5` | Model that served the request, after alias resolution |
| `x-gw-cost-usd` | `0.000146` | Exact cost of this request in USD (success only) |
| `x-gw-billing` | `byok` \| `prepaid` | Billed to the organisation's own vendor key, or deducted from prepaid credit |
| `x-gw-budget-remaining-usd` | `9.997445` | Remaining in the tightest budget that applies to this key |
| `x-gw-usage-estimated` | `1` | Present only when the cost is an estimate (vendor usage never arrived, or a transcription was charged its estimate on a `504`) |
| `x-should-retry` | `false` | Only on a charged transcription `504`: do not retry. The OpenAI SDKs obey it |
| `x-gw-warnings` | `frequency_penalty … was dropped` | Semicolon-separated list of parameters altered or dropped in translation |
| `x-ratelimit-limit-requests` | `120` | This key's requests-per-minute limit |
| `x-ratelimit-remaining-requests` | `119` | Requests left in the current minute |
| `x-ratelimit-reset-requests` | `60s` | When the request window resets |
| `retry-after` | `1` | Seconds to wait — on every `429` |

On `POST /v1/audio/speech` the body is audio, but the headers are the same.

## 10. Errors

### 10.1 Envelope

`/v1/chat/completions`, `/v1/images/generations`, `/v1/embeddings`, `/v1/audio/*`, `/v1/models` and `/v1/key`
use the OpenAI envelope. Money errors add a `budget` object.

```json
{"error":{"message":"Monthly budget of $10.000000 for this key would be exceeded (remaining $0.002000, estimate $0.002014). Resets 2026-10-01T00:00:00.000Z.","type":"budget_error","code":"budget_exceeded",
  "request_id":"req_01…","budget":{"scope":"key","period":"monthly","cap_usd":10,"remaining_usd":0.002,"resets_at":"2026-10-01T00:00:00.000Z"}}}
```

`/v1/messages` uses the Anthropic envelope; the gateway code is appended to the message.

```json
{"type":"error","error":{"type":"budget_error","message":"Monthly budget of $10.000000 for this key would be exceeded (remaining $0.002000, estimate $0.002014). Resets 2026-10-01T00:00:00.000Z. (code: budget_exceeded)"},"request_id":"req_01…"}
```

The vendor SDKs raise their normal typed exceptions from these (`openai.RateLimitError`,
`Anthropic.AuthenticationError`, …); the `code` string is the stable identifier to branch on.

### 10.2 Codes

| HTTP | `code` | Meaning | What your code should do |
|---|---|---|---|
| 400 | `invalid_request` | Malformed body, the model answers on a different endpoint (the message names it), or the vendor rejected the request (its message is relayed) | Fix the request. Do not retry as-is |
| 400 | `unsupported_parameter` | A parameter has no equivalent for this provider (§11) | Remove the parameter |
| 401 | `missing_api_key` | No key sent | Configuration bug |
| 401 | `invalid_api_key` | Key unknown or wrong | Configuration bug — check `AI_GATEWAY_KEY` |
| 402 | `budget_exceeded` | This key's daily/monthly/lifetime cap is reached. `budget.resets_at` says when | Stop until reset, or ask the admin to raise the cap |
| 402 | `tenant_ceiling_exceeded` | The organisation's monthly ceiling is reached | Same — organisation-wide |
| 402 | `credit_exhausted` | Prepaid credit is gone, or no vendor key is configured for this provider | Admin action needed |
| 402 | `request_too_expensive` | Worst case for this single request exceeds the per-request cap | Lower `max_tokens` or shorten the input |
| 402 | `vendor_key_cap_reached` | The vendor key this request routes through has reached its monthly cap | Admin action: raise the cap, or let the key fall back to the organisation default |
| 403 | `ip_not_allowed` | Caller address is not on the key's list | Run from an allowed network, or ask the admin to add the address |
| 403 | `key_suspended` | Key temporarily disabled by the admin | Wait for the admin |
| 403 | `key_revoked` | Key permanently disabled | Obtain a new key |
| 403 | `key_expired` | Key past its expiry date | Obtain a new key |
| 403 | `model_not_allowed` | Model exists but this key may not use it — or the vendor key it routes through does not allow it, which the message names | Use a model from `GET /v1/models` |
| 403 | `service_not_allowed` | This key may not use this endpoint — a mail key calling images, embeddings or audio, or images not switched on for the key | Use an AI key; ask the admin to allow images |
| 403 | `tenant_suspended` / `tenant_closed` | The organisation is suspended or closed | Contact the gateway admin |
| 404 | `model_not_found` | Unknown model name | Check spelling / prefix; see `GET /v1/models` |
| 413 | `request_too_large` | Body over the size limit (20 MB; 25 MB for a transcription upload) | Shrink the request |
| 429 | `rate_limit_exceeded` | The key's requests-per-minute or requests-per-day limit was exceeded | Wait `retry-after` seconds, retry |
| 429 | `concurrency_limit_exceeded` | Too many in-flight requests on this key | Wait `retry-after`, retry; reduce parallelism |
| 429 | `tenant_rate_limit_exceeded` | Organisation-wide limit exceeded | Wait `retry-after`, retry |
| 429 | `upstream_rate_limited` | The vendor is rate-limiting the gateway's key | Wait `retry-after`, retry with backoff |
| 502 | `upstream_error` | Vendor returned an error | Retry once with backoff; then surface |
| 504 | `upstream_timeout` | Vendor did not respond in time | Retry once; consider streaming for long outputs. **Not** a transcription 504 carrying `x-should-retry: false` — that one is charged (§7, Transcription) |
| 503 | `provider_unavailable` / `credential_unavailable` / `platform_cap_reached` | No usable vendor key right now | Retry with backoff; alert the admin if persistent |
| 503 | `gateway_unavailable` / `tenant_storage_unavailable` | Gateway dependency down (fails closed by design) | Retry with backoff |

**Guarantee:** any `401`, `402`, `403`, `404`, `413` or `429` from the gateway means the request was
**never sent to the vendor and nothing was charged**.

**Every refusal is recorded.** Whatever the gateway turns away — and whatever the vendor fails — is written to
the organisation's refusal log with the key, the address it came from, the client's `User-Agent`, the reason and
the numbers behind it, and an admin sees it under **Refusals** in the console. Expect a key used from a new
address to be noticed: five distinct new addresses in an hour suspend the key automatically. Wrong secrets never
suspend a key, however many are tried, but they are all recorded. The secret itself is never stored — only the
public key id, which is the part before the last dash.

**Retry policy that works:** retry only `429`, `502`, `503`, `504`, honour `retry-after`, exponential
backoff, max 3 attempts. Never retry `400`/`401`/`402`/`403`. The one exception: a `504` on
`/v1/audio/transcriptions` after AssemblyAI accepted the job is charged and carries `x-should-retry: false`, so a retry
pays twice — surface it instead. Any retry logic of your own should honour `x-should-retry`.

## 11. Not supported

### On `/v1/chat/completions` with Claude models — rejected (`400 unsupported_parameter`)

| Parameter | Note |
|---|---|
| `n` > 1 | one completion per request |
| `logprobs`, `top_logprobs` | |
| `seed` | |
| `modalities: ["audio"]`, `audio`, content part `input_audio` | no audio in or out |
| `file` part with `file_id`, or a media type other than PDF / text | send the bytes inline as a data URL |
| `messages[].role: "function"`, `function_call`, `functions` | legacy API; use `tools` |
| `tools[].type` other than `"function"` | |
| `tool_choice` for custom tools | |

### On `/v1/chat/completions` with Claude models — altered, always reported in `x-gw-warnings`

`frequency_penalty`, `presence_penalty`, `logit_bias` (dropped) · `response_format: json_object` (becomes a
system instruction) · `temperature` > 1 (clamped) · sampling parameters on models that do not accept them
(dropped) · forced `tool_choice` on models that only accept `auto` (downgraded) · **any other field the
translator does not know** (`store`, `metadata`, `prompt_cache_key`, …: ignored, named in the header).

### On `/v1/chat/completions` with OpenAI models — rejected (`400 unsupported_parameter`)

These change what OpenAI charges in ways the gateway cannot price yet, so they are refused rather than
under-billed: `web_search_options`, `prediction`, `audio` / `modalities: ["audio"]`,
`service_tier` other than `auto` / `default`.

### Not available on any endpoint yet

| Capability | Status |
|---|---|
| Files API (`/v1/files`) | not proxied — send documents inline (base64) on `/v1/messages` |
| Message Batches (`/v1/messages/batches`) | not proxied |
| Image **editing** and variations (`/v1/images/edits`, `/v1/images/variations`) | not proxied — generation only |
| Partial-image streaming on `/v1/images/generations` | refused (`400 unsupported_parameter`); send without `stream` |
| Streaming speech or transcription (`stream`, `stream_format: "sse"`) | refused (`400 unsupported_parameter`); the whole audio or transcript comes back in one response |
| OpenAI's token-billed audio models (`gpt-4o-mini-tts`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-transcribe`) | not in the catalogue — the gateway cannot meter them reliably yet. Use `tts-1` / `whisper-1`, ElevenLabs or AssemblyAI |
| AssemblyAI and ElevenLabs features beyond OpenAI's audio shape (speaker labels, `diarized_json`, summaries, redaction, voice settings other than speed) | not available — only what `/v1/audio/speech` and `/v1/audio/transcriptions` carry |
| Vendor-native pass-through (`/v1/vendors/…`) | not available yet |
| Audio translation (`/v1/audio/translations`) | not proxied |
| OpenAI Responses API (`/v1/responses`) | not proxied |
| Vendor server tools (web search, code execution) | refused on `/v1/chat/completions`; pass through on `/v1/messages` but their per-use vendor surcharge is **not** included in `x-gw-cost-usd` |
| Claude's thinking text in the reply | not returned on `/v1/chat/completions` (the OpenAI shape has no field for it) |

## 12. Inspecting your own key

`GET /v1/key` — safe to call at startup to fail fast on misconfiguration and to learn your limits.

```json
{
  "key_id": "muhr4dre", "name": "settlement-worker", "status": "active",
  "tenant": {"id": "ur7xqq", "name": "Softronet", "credit_remaining_usd": null, "monthly_ceiling_usd": null},
  "routing": {"anthropic": {"vendor_key": "own:anthropic-production", "billing": "byok", "environment": "production", "models": ["anthropic/claude-opus-5"]}},
  "allowed_models": [],
  "environment": "production",
  "rate_limits": {"rpm": 120, "max_concurrency": 8, "rpd": null, "rpm_per_ip": null},
  "budgets": [{"period": "monthly", "cap_usd": 10, "spent_usd": 0.000411, "resets_at": "2026-10-01T00:00:00.000Z"}],
  "ip_mode": "allowlist", "ip_allowlist_count": 2, "ip_capture_open": false, "expires_at": null
}
```

`allowed_models: []` means every model the organisation permits. An entry ending `/*` — `"openai/*"` — means
every model the vendor key for that provider allows, whatever that becomes. The response never includes the
allowed addresses, only how many there are: a caller being refused must not learn which address would work.

`routing` has an entry for each provider — `anthropic`, `openai`, `xai`, `elevenlabs`, `assemblyai` — saying
which vendor key a request would use, who pays (`byok` or `prepaid`), which environment that key belongs to,
and `models`: the list that key allows, or `"all"`. **This is the authoritative answer to "why was my model
refused?"** — a model outside it gives `403 model_not_allowed` naming the vendor key. A provider with no usable
key shows `"vendor_key": null` and an `error` code: expect the same refusal from the endpoint.

`ip_capture_open` is true while an admin has opened the ten-minute window in which the next new address to call
is added to the key's list. There is no tokens-per-minute figure in `rate_limits` any more (§9).

## 13. Getting a key

Keys are issued in the gateway console by your organisation's admin (Keys → New key). Ask for:
- a **name** identifying your project,
- the **models** you need,
- a **budget** and **rate limit** appropriate to the workload,
- **address rules** covering where the project runs — and, if your address changes or you do not know it,
  ask the admin to tick *"add the next new address automatically"* and then make one request within ten
  minutes: that address is added to the list, once, and the window closes.

The secret is shown once. Store it as `AI_GATEWAY_KEY` in the project's secret store.

## 14. Snippet for the consuming project's `CLAUDE.md` / `AGENTS.md`

Copy this into the other project so its AI assistant integrates correctly:

```markdown
## AI calls go through A1 API Gateway
- All LLM calls use A1 API Gateway, never a vendor API directly. Never add a vendor API key.
- Base URL: `AI_GATEWAY_URL` env var. Key: `AI_GATEWAY_KEY` env var (`sk-gw-…`). Read both from the environment.
- OpenAI SDK: `baseURL = AI_GATEWAY_URL + "/v1"`. Anthropic SDK: `baseURL = AI_GATEWAY_URL` (no `/v1`).
- Model names are `provider/model`, e.g. `anthropic/claude-opus-5`. Prefer the alias the admin gave us if one exists.
- Always use `/v1/chat/completions` (OpenAI format). It supports PDFs/documents (`file` parts as base64 data URLs),
  images, tools, JSON output, reasoning, streaming, and prompt caching is automatic. `/v1/messages` only with admin approval.
- `GET /v1/models` lists the models we may use, each with a `capabilities` block (images, documents, tools, …) — check it
  rather than guessing. `POST /v1/messages/count_tokens` counts tokens for free before an expensive call.
- Put stable content first (system prompt, tools), the changing message last — that is what gets cached.
- Each model answers on one endpoint; the wrong one is `400 invalid_request` naming the right one.
- Image generation: `POST /v1/images/generations` with `gpt-image-2.5-flare` (or `-sunburst`) — only if our key allows it.
  No streaming, generation only, and quality drives the cost steeply (`low` → `max` is about 40x).
- Embeddings: `POST /v1/embeddings` with `openai/text-embedding-3-small` (or `-3-large`), OpenAI's body and reply unchanged.
- Speech: `POST /v1/audio/speech` returns raw audio bytes. ElevenLabs models need the `elevenlabs/` prefix and a voice
  name our organisation set up (or a 20-character ElevenLabs voice id); OpenAI names like `alloy` only work on `tts-1`.
- Transcription: `POST /v1/audio/transcriptions` (multipart, ≤ 25 MB) with `assemblyai/universal-3-5-pro`,
  `elevenlabs/scribe_v2` or `whisper-1`; all five response formats work. AssemblyAI can take up to 10 minutes: set the
  client timeout above that. A transcription 504 is charged and says `x-should-retry: false` — never retry it.
- Not available: Files API (`file_id`), batches, image editing, streaming audio, Responses API, web search.
- Retry only 429/502/503/504 with `retry-after` and backoff (never a transcription 504). 4xx other than 429 means fix the request or config.
- Every response has `x-gw-cost-usd` (exact cost) and `x-gw-request-id` (quote it in bug reports).
- Full reference: `docs/INTEGRATION.md` in the AIGateway repository.
```

## 15. Changelog

| Date | Change |
|---|---|
| 2026-09-07 | First version of this guide. Describes consumer API v1 as implemented (spec v0.10). |
| 2026-09-07 | Added `POST /v1/messages/count_tokens` (native pass-through, free) and the `capabilities` block on every `GET /v1/models` entry. |
| 2026-09-17 | Added `POST /v1/images/generations` (OpenAI GPT-Image, forwarded verbatim). Opt-in per key, with per-key ceilings on count, size and quality. Generation only: no edits, variations or streaming. |
| 2026-09-08 | `/v1/chat/completions` is the standard door: PDFs and text documents via `file` parts, automatic prompt caching for Claude (`cache_control: false` to opt out), `strict` tools, `json_object` as a system instruction, every ignored field reported in `x-gw-warnings`, and OpenAI fields the gateway cannot price (`web_search_options`, `prediction`, audio, paid `service_tier`) refused. Documents are reserved per page. `prompt_caching` added to capabilities. |
| 2026-09-19 | Added `POST /v1/embeddings` (OpenAI, forwarded verbatim, `encoding_format` passed through), `POST /v1/audio/speech` (ElevenLabs, OpenAI `tts-1`/`tts-1-hd`; organisation voice names) and `POST /v1/audio/transcriptions` (AssemblyAI, ElevenLabs Scribe, OpenAI `whisper-1`; all five response formats for every vendor). Every model answers on one endpoint and the wrong one is `400 invalid_request` naming the right one — **image models at the wrong door were `404 model_not_found` before**. Long-context rows now bill the whole request at the higher rate (xAI over 200K, OpenAI's newer models over 272K prompt tokens). Catalogue re-read: `gpt-5.6` $4/$20, retired xAI names bill as `grok-4.3`, `grok-3-mini` switched off, many models added. Tokens per minute does not apply to audio. A charged transcription `504` carries its cost headers and `x-should-retry: false`. |