Prompt caching

Overview

60db caches work you repeat, so repeated requests are cheaper and faster. Cached input is billed at a discounted cached-input rate. Check your plan’s current rates with GET /plans or pricing.cached_input_rate_per_million in GET /v1/usage.

Chat: automatic prefix caching

Caching is automatic. When consecutive requests share the same prompt prefix (system prompt, tool definitions, reference documents), the shared prefix is served from cache. You do not need to change your requests. Read the result from the response:

{
  "usage": {
    "prompt_tokens": 2048,
    "completion_tokens": 120,
    "total_tokens": 2168,
    "prompt_tokens_details": { "cached_tokens": 1792 }
  }
}

For streaming, send stream_options: {"include_usage": true} to receive the final usage chunk.

Order your prompt for caching

Only a prefix can be cached. Put static content first and variable content last:

System prompt
Tool definitions
Reference documents / few-shot examples
Conversation history
The latest user message

Changing any token early in the prompt invalidates the cache from that point on.

Optional hints: `cache_control` and `prompt_cache_key`

Hints are optional. Caching happens with or without them, and hints give no TTL guarantee.

cache_control: {"type": "ephemeral"}, with optional "ttl": "5m" or "1h". Accepted on message content parts, on whole messages, and on tools[] entries. Maximum 4 breakpoints per request. Any other shape returns 400.
prompt_cache_key: string up to 256 characters. Use the same key for requests that share a prefix.

curl https://api.60db.ai/v1/chat/completions \
  -H "Authorization: Bearer $SIXTYDB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "60db-tiny",
    "prompt_cache_key": "support-bot-v3",
    "messages": [
      {
        "role": "system",
        "content": [
          { "type": "text", "text": "You are a support agent. <long policy text>", "cache_control": { "type": "ephemeral" } }
        ]
      },
      { "role": "user", "content": "How do I reset my password?" }
    ]
  }'

Speech-to-text: re-upload cache

POST /stt returns the stored transcript when the same workspace uploads byte-identical audio with the same options (language(s), diarize, timestamps, keywords, context, speaker counts, script_correction) within 30 days. The audio is not re-processed, and the request is billed at the STT cached rate.

Field	Type	Default	Description
`cache`	boolean	`true`	Set `false` to opt out of the re-upload cache
`cache_control`	JSON string	none	`{"type":"ephemeral"}`. Implies `cache` on

The response includes cached: true when served from cache, otherwise false. The cache is scoped per workspace and does not apply to WebSocket or streaming STT.

Where cached usage appears

Chat response: usage.prompt_tokens_details.cached_tokens
Usage logs: metadata.cached_tokens on LLM rows
Analytics: llm_cached_tokens
Realtime: llm_tokens_cached in usage events, and input_token_details.cached_tokens in response.done

Billing

Cached input tokens (Chat) and cached STT re-uploads are billed at a discounted cached rate. When a cached rate is lower than the base rate, it appears in credit_costs from GET /plans.

Get Started

SDKs

Core Features

Prompt caching

Overview

Chat: automatic prefix caching

Order your prompt for caching

Optional hints: `cache_control` and `prompt_cache_key`

Speech-to-text: re-upload cache

Where cached usage appears

Billing

​Overview

​Chat: automatic prefix caching

​Order your prompt for caching

​Optional hints: cache_control and prompt_cache_key

​Speech-to-text: re-upload cache

​Where cached usage appears

​Billing

Overview

Chat: automatic prefix caching

Order your prompt for caching

Optional hints: `cache_control` and `prompt_cache_key`

Speech-to-text: re-upload cache

Where cached usage appears

Billing