API reference

Prompt caching

Reused prompt prefixes - long system prompts, shared context, few-shot examples - can be served from the provider's cache at a steep discount. boldrouter passes caching through and bills cached tokens at the reduced rate automatically.

On by default

Caching is enabled on every request. Providers that cache automatically (OpenAI, Google) need nothing from you; for Anthropic, the gateway adds a cache breakpoint on your system prompt so a reused prefix is cached.

Opt out per request with the x-bold-cache: off header (or "cache": false in the body). Every response echoes x-bold-cache: on / off.

How cached tokens are billed

Cached prompt tokens are billed at a fraction of the normal input rate - each provider's published cache-read price. The rest of the prompt and the completion bill as usual.

Field	Type	Description
OpenAI	openai	Cached input billed at 0.25× the normal input rate (75% off).
Anthropic	anthropic	Cached input billed at 0.1× the normal input rate (90% off).
Google	google	Cached input billed at 0.25× the normal input rate (75% off).
Mistral	mistral	No prompt caching.
xAI	xai	Cached input billed at 0.16× the normal input rate (84% off).
DeepSeek	deepseek	Cached input billed at 0.02× the normal input rate (98% off).
Perplexity	perplexity	No prompt caching.
Z.ai	zai	Cached input billed at 0.25× the normal input rate (75% off).
MiniMax	minimax	Cached input billed at 0.2× the normal input rate (80% off).
NVIDIA	nvidia	Cached input billed at 0.25× the normal input rate (75% off).
Qwen	qwen	Cached input billed at 0.25× the normal input rate (75% off).
Xiaomi	xiaomi	Cached input billed at 0.01× the normal input rate (99% off).
StreamLake	streamlake	Cached input billed at 0.2× the normal input rate (80% off).
W&B	wandb	Cached input billed at 0.2× the normal input rate (80% off).
Cohere	cohere	Cached input billed at 0.25× the normal input rate (75% off).
Inception	inceptionlabs	Cached input billed at 0.1× the normal input rate (90% off).
Moonshot	moonshot	Cached input billed at 0.17× the normal input rate (83% off).
Black Forest Labs	bfl	Cached input billed at 0.25× the normal input rate (75% off).
H Company	hcompany	Cached input billed at 0.25× the normal input rate (75% off).
Reka	reka	Cached input billed at 0.25× the normal input rate (75% off).
Google Vertex AI	google-vertex	Cached input billed at 0.25× the normal input rate (75% off).
Arcee	arcee	Cached input billed at 0.25× the normal input rate (75% off).
Writer	writer	Cached input billed at 0.25× the normal input rate (75% off).
ElevenLabs	elevenlabs	Cached input billed at 0.25× the normal input rate (75% off).
DeepL	deepl	Cached input billed at 0.25× the normal input rate (75% off).
LightOn	lighton	Cached input billed at 0.25× the normal input rate (75% off).

Savings show up in your Usage dashboard ("Cache savings") and feed into the same prepaid balance - see Pricing & billing.

Reading cache hits

The response usage reports cached tokens under prompt_tokens_details.cached_tokens, exactly where OpenAI clients expect them:

json

"usage": {
  "prompt_tokens": 4561,
  "completion_tokens": 12,
  "total_tokens": 4573,
  "prompt_tokens_details": { "cached_tokens": 4552 }
}

Getting cache hits

Put the stable, reused content first (system prompt, instructions, shared context) and the variable part last. Caches match on a shared prefix.
Keep that prefix byte-identical across requests - any change before the cache breakpoint misses the cache.
Caches are short-lived (minutes) and have provider minimum sizes (e.g. Anthropic caches only prefixes above a token threshold), so caching helps most for large, frequently-reused prompts.

The first request with a cacheable prefix is a cache write (no discount yet); subsequent requests within the cache window are reads at the discounted rate. You'll see cached_tokens: 0 on the first call and a large value on the next.