` — Optional display title * `-c, --collection <id>` — Target collection (defaults to personal) * `--type <type>` — `user` | `knowledge` | `hive` (default: `user`) * `--no-infer` — Skip LLM-based fact extraction (default: infer is on) **Cost**: \$0.0001 per 1,000 characters. ## Upload a document Upload a file — PDF, DOCX, XLSX, PPTX, EML, MSG, HTML, TXT, Markdown, CSV, PNG, JPG, TIFF, and 70+ other formats — with built-in OCR for scanned content: ```bash theme={null} 60db memory:upload \ --file ~/Documents/quarterly-report.pdf \ --collection company_reports \ --type knowledge \ --title "Q4 2026 Report" ``` **Options:** * `-f, --file <path>` — Absolute or relative path to the document (required) * `-c, --collection <id>` — Target collection * `--type <type>` — `user` | `knowledge` | `hive` (default: `knowledge`) * `--title <title>` — Display title (defaults to filename) * `--chunk-size <n>` — Max characters per chunk, 200-8000 (default: 1500) * `--chunk-overlap <n>` — Character overlap between chunks (default: 200) **Max file size**: 200 MB. **Max chunks per document**: 100. **Cost** (two-stage): * Extract fee: \$0.003 per MB * Ingest fee: \$0.0001 per 1,000 extracted characters Both fees are automatically refunded on any failure. ### Tuning tips | Document type | `--chunk-size` | `--chunk-overlap` | | ------------------------- | -------------- | ----------------- | | Technical docs / API refs | 1500 | 200 (default) | | Long-form prose / books | 2500 | 300 | | FAQs / short snippets | 800 | 100 | | Spreadsheet exports | 3000 | 0 | | Scanned PDFs (OCR) | 2000 | 250 | ## Search memories Hybrid semantic + keyword recall with optional cross-encoder reranking: ```bash theme={null} 60db memory:search \ --query "What is the refund policy?" \ --mode fast \ --limit 5 \ --alpha 0.8 ``` **Options:** * `-q, --query <text>` — Search query (required, max 2,000 chars) * `-c, --collection <id>` — Collection to search * `--mode <mode>` — `fast` (\~100-200ms) or `thinking` (wider candidate pool + cross-encoder rerank, \~200-400ms) * `--limit <n>` — Max results (1-50, default: 10) * `--alpha <n>` — 0 (keyword only) to 1 (semantic only), default 0.8 * `--recency-bias <n>` — Weight for newer memories (0-1, default: 0) * `--graph` — Include knowledge-graph relationships **Advanced reranker knobs** (override server defaults): * `--rerank-top-k <n>` — Max candidates the cross-encoder reranks * `--rerank-timeout <ms>` — Hard timeout for rerank call * `--min-rerank-score <n>` — Drop results below this rerank score (0-1) * `--fetch-multiplier <n>` — In thinking mode, fetch N x limit candidates **Cost**: flat \$0.0003 per query. When the cross-encoder reranker is active, results include both a **dense score** (vector similarity) and a **rerank score** (cross-encoder confidence). The rerank score is the more reliable signal for ranking quality. ### Example — tune by query type ```bash theme={null} # Exact phrase match (keyword-heavy) 60db memory:search -q "invoice #INV-2026-042" --alpha 0.2 # Conceptual question (semantic-heavy) 60db memory:search -q "how do I configure SSO?" --alpha 0.9 # Complex multi-faceted question (thinking mode + reranking) 60db memory:search -q "what are common escalation patterns?" --mode thinking --alpha 0.7 # Recent events focus 60db memory:search -q "product updates" --alpha 0.6 --recency-bias 0.3 # Override reranker settings for this query 60db memory:search -q "detailed compliance requirements" --mode thinking --min-rerank-score 0.3 --fetch-multiplier 5 ``` ## Assemble LLM context Purpose-built for RAG — returns a pre-formatted string ready to prepend to an LLM system message: ```bash theme={null} 60db memory:context \ --query "Tell me about customer preferences" \ --top-k 8 \ --max-context-length 2000 ``` **Options:** * `-q, --query <text>` — Query driving retrieval (required) * `--session-id <id>` — Chat session ID for hierarchical context * `--top-k <n>` — Max memories to pull (default: 10) * `--max-context-length <n>` — Max assembled tokens (default: 4000) * `--graph` — Include graph relationships * `--no-timeline` — Exclude recent events **Graceful degradation**: returns an empty prompt on outage — chat keeps working. **Cost**: flat \$0.0005 per query. ## Manage collections ### List collections ```bash theme={null} 60db memory:collections ``` Shows every collection visible to you in the current workspace with kind (personal/team/knowledge/hive) and shared status. ### Create a collection **Admin/owner only.** Personal collections are auto-created per user — you don't create those manually. ```bash theme={null} 60db memory:create-collection \ --id customer_support \ --label "Customer Support KB" \ --kind knowledge ``` **Options:** * `-i, --id <id>` — Collection ID (required, lowercase alphanumeric + underscores) * `-l, --label <label>` — Human-readable label (required) * `-k, --kind <kind>` — `team` | `knowledge` | `hive` (default: `team`) * `--no-shared` — Don't share with workspace members Both commands are **unbilled**. ## Monitor spend ```bash theme={null} 60db memory:usage 60db memory:usage --period last_30_days 60db memory:usage --period all_time ``` Shows net spend, operation count, refund count, and a per-service-type breakdown (ingest / extract / recall / context). Always free. Works even when the wallet is empty. **Options:** * `--period <period>` — `current_month` (default) | `last_30_days` | `all_time` ## Poll a memory's status Memories are processed asynchronously. Use this to watch a specific ID: ```bash theme={null} 60db memory:status --id mem_01HV8K2X3N4P5Q6R7S8T9U ``` Statuses: `pending` → `processing` → `ready` (or `failed`). Unbilled. ## Delete a memory ```bash theme={null} 60db memory:delete --id mem_01HV8K2X3N4P5Q6R7S8T9U ``` Soft-delete with 24-hour undo grace period. You can delete memories you created; admins/owners can delete any memory. ## Agent-friendly JSON output All memory commands respect the global `--json` flag for machine-readable output: ```bash theme={null} 60db --json memory:search --query "user preferences" --limit 5 ``` Returns the same data as the API but with a structured envelope that includes a `billing: {charged, balance, txId}` object on billable commands. ## Handling insufficient credits When the wallet balance is below the operation cost, commands return a `402`-style error with structured details: ```bash theme={null} $ 60db memory:ingest --text "hello" --type user ✗ Insufficient credits in workspace wallet required: $0.000100 available: $0.000010 shortfall: $0.000090 ℹ Top up your workspace wallet via the Dashboard: https://app.60db.ai ``` The `--json` output surfaces the same details as: ```json theme={null} { "success": false, "error": "INSUFFICIENT_CREDITS", "details": { "required": 0.0001, "available": 0.00001, "shortfall": 0.00009 } } ``` ## Related * [Memory pricing & billing](/api-reference/memory/pricing) * [Memory feature overview](/features/memory) * [CLI: authz](/cli-reference/commands/authz) * [MCP: memory tools](/mcp-server/memory) # Speech-to-Text Source: https://docs.60db.ai/cli-reference/commands/stt STT commands for transcribing audio files # Speech-to-Text (STT) ## Commands ### Transcribe Audio File ```bash theme={null} 60db stt:transcribe --file audio.wav ``` By default the language is auto-detected. Equivalent to passing `--language auto`. ### Specify Language ```bash theme={null} 60db stt:transcribe --file audio.wav --language hi ``` Pass an ISO 639-1 code from the supported set (see `60db stt:languages`). Supported codes include `en`, `hi`, `bn`, `mr`, `pa`, `gu`, `or`, `as`, `ne`, `ta`, `te`, `kn`, `ml`, `sa`, `ar`, and 25 European languages. ### Auto-detect (explicit) ```bash theme={null} 60db stt:transcribe --file meeting.wav --language auto ``` `--language auto` is treated as "omit" — the CLI strips it before sending so the server runs its language identification across all 39 supported languages. You can also simply omit the `--language` flag entirely for the same behavior. ### Enable Speaker Diarization ```bash theme={null} 60db stt:transcribe --file meeting.wav --diarize true ``` Each response segment will include a `speakers` array with labels like `SPEAKER_00`, `SPEAKER_01`. Adds \~50–150 ms of processing latency. ### Add Context (optional refinement) ```bash theme={null} 60db stt:transcribe --file visit.wav \ --context "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique." ``` When `--context` is supplied, the server runs a background LLM refinement pass and the returned `text` is polished for proper nouns, filler removal, and punctuation. Omit `--context` to skip refinement. <Note> `--context` takes a **plain string** — the REST `/stt` endpoint shape. The WebSocket `/ws/stt` endpoint accepts a structured `{general, text, terms}` **object** instead; see the [WebSocket STT reference](/api-reference/websocket/stt). </Note> ### List Available Languages ```bash theme={null} 60db stt:languages ``` Returns the 39-language catalog plus the `auto` entry, sourced from `GET /stt/languages`. ## Options * `-f, --file <path>` — Audio file path (required; max 25 MB; WAV / MP3 / M4A / OGG / FLAC / WebM) * `-l, --language <code>` — ISO 639-1 language code (e.g. `en`, `hi`, `ar`). Omit or pass `auto` for auto-detection across the 39 supported languages. * `--diarize <boolean>` — Enable speaker diarization (default: `false`) * `--context <text>` — Free-form paragraph describing the session (domain, speakers, jargon). Enables server-side LLM refinement of the response text. ## Examples ### STT — Transcribe Audio ```bash theme={null} # Auto-detect language (simplest) 60db stt:transcribe --file meeting.wav # Auto-detect, explicit form 60db stt:transcribe --file meeting.wav --language auto # Force Hindi — skips language identification for lowest latency 60db stt:transcribe --file recording.wav --language hi # Multi-speaker meeting with diarization + auto-detect 60db stt:transcribe --file board-call.wav --diarize true # List supported languages 60db stt:languages ``` ## Notes * **Do not pass unsupported language codes.** The server explicitly rejects `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he` (and Arabic dialect tags like `ar-eg`) with an `unsupported_language` error. Use `auto` or omit the flag for these. * **Auto-detect on REST vs WebSocket.** The REST (`stt:transcribe`) form flow accepts `--language auto` as a convenience. The streaming WebSocket form requires `languages: null`, not the literal string `"auto"`. If you write your own WebSocket client, don't forward the string `"auto"` — send `null`. * A successful response with empty `text` and `warning_codes: ["no_speech_detected"]` means the audio contained no speech (silence / music / noise). This is not an error — do not retry. # Text-to-Speech Source: https://docs.60db.ai/cli-reference/commands/tts TTS commands for generating speech from text # Text-to-Speech (TTS) ## Commands ### Synthesize Speech from Text ```bash theme={null} 60db tts:synthesize --text "Hello this is devendra" --voice-id "voice_id_here" ``` ### Specify Output File ```bash theme={null} 60db tts:synthesize --text "Hello world" --voice-id "abc123" --output speech.mp3 ``` ### Adjust Speech Parameters ```bash theme={null} 60db tts:synthesize --text "Hello" --voice-id "abc123" --speed 1.2 --stability 50 --similarity 75 ``` ### List Available Voices ```bash theme={null} 60db tts:voices ``` ## Options * `-t, --text <text>` - Text to synthesize (required) * `-v, --voice-id <id>` - Voice ID to use (required) * `-o, --output <file>` - Output audio file path (default: `tts_<timestamp>.mp3`) * `--speed <number>` - Speech speed (default: 1) * `--stability <number>` - Voice stability 0-100 (default: 50) * `--similarity <number>` - Voice similarity 0-100 (default: 75) ## Examples ### TTS - Generate Speech ```bash theme={null} # Generate speech from text 60db tts:synthesize --text "Hello, this is devendra" --voice-id "1a8c6331-c79b-47d3-9893-09160a245a3e" # List available voices first 60db tts:voices # Save to specific file 60db tts:synthesize --text "Welcome to our service" --voice-id "abc123" --output welcome.mp3 ``` # Users Source: https://docs.60db.ai/cli-reference/commands/users User management commands # Users ## Commands ### List All Users ```bash theme={null} 60db users --list --page 1 --limit 10 ``` ### Get User Details ```bash theme={null} 60db users --get 123 ``` ### Search Users ```bash theme={null} 60db users --search "john@example.com" ``` ### Create New User ```bash theme={null} 60db user:create --email user@example.com --name "John Doe" --password secret123 ``` ### Update User ```bash theme={null} 60db user:update --id 123 --name "Jane Doe" --active true ``` ### Delete User ```bash theme={null} 60db user:delete --id 123 ``` ## Options ### List Users * `--page <number>` - Page number * `--limit <number>` - Items per page ### Get User * `--id <id>` - User ID (required) ### Search Users * `--search <query>` - Search query (email or name) ### Create User * `--email <email>` - User email (required) * `--name <name>` - Full name * `--password <password>` - Password * `--role <role>` - System role (default: user) ### Update User * `--id <id>` - User ID (required) * `--email <email>` - New email * `--name <name>` - New name * `--active <boolean>` - Active status * `--role <role>` - System role ### Delete User * `--id <id>` - User ID (required) ## Examples ### Export User Data to JSON ```bash theme={null} 60db --json users --list > users.json ``` # Voices Source: https://docs.60db.ai/cli-reference/commands/voices Voice management commands for TTS # Voices ## List Available Voices ```bash theme={null} 60db tts:voices ``` This command returns all available voices that can be used for text-to-speech synthesis. ## Examples ### Get All Voices ```bash theme={null} 60db tts:voices ``` ### Export Voices to JSON ```bash theme={null} 60db --json tts:voices > voices.json ``` ### Use Voice in Synthesis ```bash theme={null} # First get voice ID 60db tts:voices # Use the voice ID in synthesis 60db tts:synthesize --text "Hello" --voice-id "voice_id_here" ``` ## Voice Properties Each voice has the following properties: * `id` - Unique voice identifier * `name` - Voice name * `language` - Language code * `gender` - Voice gender (male, female, neutral) * `description` - Voice description Use these properties when selecting a voice for synthesis. # Workspaces Source: https://docs.60db.ai/cli-reference/commands/workspaces Workspace management commands # Workspaces ## Commands ### List All Workspaces ```bash theme={null} 60db workspaces --list ``` ### Get Workspace Details ```bash theme={null} 60db workspaces --get 456 ``` ### Filter by User ```bash theme={null} 60db workspaces --list --user-id 123 ``` ### Create Workspace ```bash theme={null} 60db workspace:create --name "My Workspace" --owner-id 123 --description "Project workspace" ``` ## Options ### List Workspaces * `--user-id <id>` - Filter by user ID ### Get Workspace * `--id <id>` - Workspace ID (required) ### Create Workspace * `--name <name>` - Workspace name (required) * `--owner-id <id>` - Owner user ID (required) * `--description <text>` - Workspace description # Configuration Source: https://docs.60db.ai/cli-reference/configuration Configure 60db CLI with API keys and settings # Configuration ## Commands ### Show Current Configuration ```bash theme={null} 60db config ``` ### Set API URL ```bash theme={null} 60db config --set apiBaseUrl=https://api.60db.ai ``` ### Set API Key ```bash theme={null} 60db config --set apiKey=your_api_key_here ``` ### List All Config ```bash theme={null} 60db config --list ``` ### Clear Config ```bash theme={null} 60db config --clear ``` # Examples Source: https://docs.60db.ai/cli-reference/examples Practical examples for using 60db CLI # Examples ## Authentication & Setup ```bash theme={null} # Login to get token 60db auth:login --email your@email.com --password yourpassword # Check session 60db auth:session # Or manually set API key 60db config --set apiKey=your_token_here ``` ## AI Chat Completion ```bash theme={null} # Simple chat 60db ai:chat --prompt "Explain quantum computing" # With system prompt for specific behavior 60db ai:chat --prompt "Review this code" --system "You are an expert code reviewer. Be concise and actionable." # Higher temperature for creative responses 60db ai:chat --prompt "Write a poem about AI" --temperature 0.9 ``` ## TTS - Generate Speech ```bash theme={null} # Generate speech from text 60db tts:synthesize --text "Hello, this is devendra" --voice-id "1a8c6331-c79b-47d3-9893-09160a245a3e" # List available voices first 60db tts:voices # Save to specific file 60db tts:synthesize --text "Welcome to our service" --voice-id "abc123" --output welcome.mp3 ``` ## STT - Transcribe Audio ```bash theme={null} # Transcribe audio file 60db stt:transcribe --file meeting.wav # Transcribe Hindi audio 60db stt:transcribe --file recording.wav --language hi # List available languages 60db stt:languages ``` ## AI Meeting Notes Analysis ```bash theme={null} # Analyze meeting transcript 60db ai:meeting --transcript "John: We need to finish by Friday. Jane: I'll handle frontend." --title "Sprint Planning" # Get structured JSON output 60db --json ai:meeting --transcript "$TRANSCRIPT" > meeting-notes.json ``` ## Add Credits to Multiple Users ```bash theme={null} for user_id in 123 456 789; do 60db credits:add --user-id $user_id --amount 10 --description "Monthly bonus" done ``` ## Export User Data to JSON ```bash theme={null} 60db --json users --list > users.json ``` ## Monitor Transactions in Real-Time ```bash theme={null} watch -n 5 '60db --json billing:transactions --limit 5' ``` # Global Options Source: https://docs.60db.ai/cli-reference/global-options Global options available for all 60db CLI commands # Global Options These options can be used with any command: ## Available Options ### `-j, --json` Output in JSON format (for agent consumption) ```bash theme={null} 60db --json users --list ``` ### `-v, --verbose` Enable verbose output ```bash theme={null} 60db --verbose users --list ``` ### `-V, --version` Output the version number ```bash theme={null} 60db --version ``` ### `-h, --help` Display help for command ```bash theme={null} 60db --help 60db users --help ``` ### `--api-url <url>` API base URL (overrides config) ```bash theme={null} 60db --api-url https://api.60db.ai users --list ``` ### `--api-key <key>` API key for authentication (overrides config) ```bash theme={null} 60db --api-key your_key users --list ``` # Installation Source: https://docs.60db.ai/cli-reference/installation How to install 60db CLI globally or locally # Installation **Source code:** [github.com/60db-ai/cli](https://github.com/60db-ai/cli) · **npm:** [`60db-cli`](https://www.npmjs.com/package/60db-cli) ## Prerequisites * Node.js 16 or higher * npm or yarn package manager ## Global Installation Install globally to use the `60db` command anywhere: ```bash theme={null} npm install -g 60db-cli ``` Verify installation: ```bash theme={null} 60db --version ``` ## Local Installation Install as a dependency in your project: ```bash theme={null} npm install 60db-cli ``` Use with npx: ```bash theme={null} npx 60db users --list ``` ## Install from Source Clone the public GitHub repo and link the binary locally — useful when you want to build from a specific branch or commit: ```bash theme={null} git clone https://github.com/60db-ai/cli.git cd cli npm install npm link # exposes the `60db` binary on your PATH 60db --version ``` ## Post-Installation After installation, configure your API credentials: ```bash theme={null} # Set API URL 60db config --set apiBaseUrl=https://api.60db.ai # Set API key 60db config --set apiKey=your_api_key_here # Or login with email/password 60db auth:login --email your@email.com --password yourpassword ``` ## Troubleshooting ### Command not found If you get `command not found` after global installation: 1. Check npm global bin directory: ```bash theme={null} npm config get prefix ``` 2. Add it to your PATH (if not already): ```bash theme={null} export PATH=$(npm config get prefix)/bin:$PATH ``` 3. Or reinstall with sudo (Linux/Mac): ```bash theme={null} sudo npm install -g 60db-cli ``` # 60db CLI Source: https://docs.60db.ai/cli-reference/introduction Agent-native command-line interface for the full 60db platform — AI completions, TTS, STT, voice cloning, memory/RAG, Cerbos policy checks, billing, and workspaces with structured JSON output. # 60db CLI Agent-native command-line interface for the full 60db platform. Covers AI completions, TTS, STT, voice cloning, persistent memory with document upload (PDF, DOCX, XLSX, scanned images with OCR), Cerbos policy checks, billing, and workspace management — all with structured JSON output for scripts and AI agents. **Source code:** [github.com/60db-ai/cli](https://github.com/60db-ai/cli) · **npm:** [`60db-cli`](https://www.npmjs.com/package/60db-cli) · **License:** MIT ## Features * **Dual Mode**: Interactive REPL + subcommand interface * **Agent-Native**: Structured JSON output with `--json` flag * **AI Completions**: Chat, meeting notes analysis, and text completion * **TTS (Text-to-Speech)**: Generate speech from text with multiple voices * **STT (Speech-to-Text)**: Transcribe audio files with multiple languages * **Memory & RAG**: Ingest memories, upload documents (91+ formats with OCR), hybrid semantic search, LLM-ready context assembly, per-month spend tracking * **Authorization (Cerbos)**: Inspect effective permissions and probe `(resource, action)` checks before attempting gated operations * **Authentication**: Secure login and token management * **Complete Coverage**: Users, Credits, Billing, Workspaces, Categories, Memory, Authz * **Configuration**: Persistent config with `60db config` ## Installation ### Global Installation ```bash theme={null} npm install -g 60db-cli ``` ### Local Installation ```bash theme={null} npm install 60db-cli ``` ### From Source ```bash theme={null} git clone https://github.com/60db-ai/cli.git cd cli npm install npm link # exposes the `60db` binary globally ``` ## Quick Start ### Interactive Mode (REPL) ```bash theme={null} 60db 60db --version ``` Enter the interactive shell with tab completion and persistent session: ``` 60db> users 60db> credits:add --user-id 123 --amount 50 60db> config --list 60db> exit ``` ### Command Mode ```bash theme={null} # List all users 60db users --list # Add credits to user 60db credits:add --user-id 123 --amount 50 --currency USD # Upload a document to memory with OCR 60db memory:upload --file ~/Documents/handbook.pdf --type knowledge # Search with hybrid semantic + keyword recall 60db memory:search --query "refund policy" --limit 5 # Check if current user can ingest memories 60db authz:check --resource memory --action create # Get JSON output for agent consumption 60db --json users --list ``` ## Environment Variables ```bash theme={null} # Set API URL export SIXTYDB_API_URL=https://api.60db.ai # Set API Key export SIXTYDB_API_KEY=your_api_key_here ``` Legacy `X60DB_*` variable names are still honored for backward compatibility. ## JSON Output (for AI Agents) All commands support `--json` flag for structured output: ```bash theme={null} 60db --json users --list ``` Response: ```json theme={null} { "success": true, "users": [ { "id": 123, "email": "user@example.com", "full_name": "John Doe", "system_role": "user", "is_active": true, "is_verify_email": true, "created_at": "2024-01-15T10:30:00Z" } ], "pagination": { "total_users": 1, "total_pages": 1, "current_page": 1, "limit": 10 } } ``` ## Error Handling All commands return structured error responses: ```json theme={null} { "success": false, "error": "User not found" } ``` ## Requirements * Node.js >= 16.0.0 * npm or yarn ## Support For issues or questions, please visit: * GitHub: [https://github.com/60db-ai/cli](https://github.com/60db-ai/cli) * Issues: [https://github.com/60db-ai/cli/issues](https://github.com/60db-ai/cli/issues) ## License MIT # Chat Source: https://docs.60db.ai/features/llm-chat Powerful AI chat with text correction, streaming, and function calling ## Overview 60db's LM (Large Language Model) API provides intelligent chat completions with advanced text correction capabilities, streaming responses, and function calling support. Our Small Language Model (SLM) is optimized for fast, efficient responses. ## Features <CardGroup> <Card title="OpenAI Compatible" icon="code"> Drop-in compatible with OpenAI's chat completion format </Card> <Card title="Text Correction" icon="edit"> Smart text correction with dictionary and style options </Card> <Card title="Real-time Streaming" icon="signal-stream"> Server-Sent Events for instant response streaming </Card> <Card title="Function Calling" icon="zap"> Built-in tool/function calling support </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const response = await client.chat.completions.create({ model: '60db-tiny', messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'How can I improve my English?' } ], stream: true }); for await (const chunk of response) { console.log(chunk.choices[0]?.delta?.content); } ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') response = client.chat.completion( model='60db-tiny', messages=[ {'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'How can I improve my English?'} ], stream=True ) print(response['choices'][0]['message']['content']) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl --location 'https://api.60db.ai/v1/chat/completions' \ --header 'Content-Type: application/json' \ --data '{ "model": "60db-tiny", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello! How are you?"} ], "stream": true }' ``` </Tab> </Tabs> ## Streaming Responses Real-time streaming with Server-Sent Events: ```javascript theme={null} const response = await fetch("https://api.60db.ai/v1/chat/completions", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ model: "60db-tiny", messages: [{ role: "user", content: "Tell me a story" }], stream: true, }), }); const reader = response.body.getReader(); const decoder = new TextDecoder(); while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = decoder.decode(value); const lines = chunk.split("\n"); for (const line of lines) { if (line.startsWith("data: ")) { const data = JSON.parse(line.slice(6)); if (data.choices?.[0]?.delta?.content) { console.log(data.choices[0].delta.content); } } } } ``` ## Function Calling (Tools) Define tools the model can use: ```javascript theme={null} const response = await client.chat.completions.create({ model: "60db-tiny", messages: [ { role: "user", content: "What is the weather in San Francisco?" }, ], tool: [ { name: "get_weather", description: "Get the current weather for a location", parameters: { type: "object", properties: { location: { type: "string", description: "The city and state, e.g. San Francisco, CA", }, unit: { type: "string", enum: ["celsius", "fahrenheit"], }, }, required: ["location"], }, }, ], }); ``` ### Streaming Response (SSE) ``` data: {"chat_id": "550e8400-e29b-41d4-a716-446655440000", "type": "chat_id"} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": "I'm"}}]} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": " doing"}}]} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": " well!"}}]} data: {"type": "done", "response_time_ms": 1250} data: [DONE] ``` ## Application Contexts Supported contexts for text correction: | Context | Description | | ---------- | -------------------------------- | | `email` | Professional email communication | | `chat` | Casual chat/messaging | | `document` | Formal documents/reports | | `message` | Short messages/notifications | | `social` | Social media posts | ```javascript theme={null} const response = await client.chat.completions.create({ text: "hey can u send me the report pls", appContext: "email", style: { tone: "professional", autoCapitalize: true }, }); // Output: "Hey, can you send me the report, please?" ``` ## Best Practices <AccordionGroup> <Accordion title="Prompt Engineering"> * Be specific in your system prompt * Provide examples for few-shot learning * Use appropriate tone for your use case * Test different prompts for optimal results </Accordion> <Accordion title="Streaming"> * Use streaming for better UX - Handle connection errors gracefully - Buffer chunks for smooth display - Implement timeout handling </Accordion> <Accordion title="Cost Optimization"> * Cache common responses - Use shorter prompts when possible - Enable chat history for context - Monitor token usage </Accordion> <Accordion title="Text Correction"> * Use dictionary for domain-specific terms * Set appropriate app context * Test style options for your use case * Combine multiple style options </Accordion> </AccordionGroup> ## Use Cases ### Customer Support Chatbot ```javascript theme={null} async function handleCustomerMessage(message) { const response = await client.chat.completions.create({ model: "60db-tiny", messages: [ { role: "system", content: "You are a helpful customer support assistant. Be friendly and professional.", }, { role: "user", content: message }, ], stream: true, save_chat: true, chat_id: customerId, }); return response; } ``` ### Text Correction Service ```javascript theme={null} async function correctText(text, context = "email") { const response = await client.chat.completions.create({ text: text, appContext: context, dictionary: commonTypos, style: { tone: "professional", autoCapitalize: true, autoPunctuate: true, }, }); return response.choices[0].message.content; } ``` ### AI Assistant with Tools ```javascript theme={null} async function aiAssistant(userQuery) { const response = await client.chat.completions.create({ messages: [{ role: "user", content: userQuery }], tool: [getWeatherTool, searchDatabaseTool, sendEmailTool], }); // Handle tool calls and return results return response; } ``` ## Pricing Charges are based on token usage: | Metric | Cost | | --------- | ----------- | | Per Token | \~\$0.00002 | Token usage is tracked and billed to your workspace. Monitor usage via the Analytics API. ## API Reference For detailed API documentation, see: <CardGroup> <Card title="Chat" icon="code" href="/api-reference/llm/chat-completion"> Complete API reference with all parameters and examples </Card> </CardGroup> # Memory & RAG Source: https://docs.60db.ai/features/memory Persistent memory, semantic recall, and retrieval-augmented generation (RAG) for AI chat ## Overview 60db's **Memory** system gives your AI applications a long-term, searchable memory. It stores user preferences, conversation history, knowledge base documents, and arbitrary facts, then retrieves the most relevant ones on demand using hybrid semantic + keyword search. Built on a multi-layer retrieval architecture combining vector search with a knowledge graph, Memory enables: * **Personalized AI chat** — the SLM remembers user preferences across sessions * **Knowledge base Q\&A** — ingest docs and retrieve grounded answers * **Multi-user collaboration** — shared "team" memory collections * **Graph-aware recall** — find related concepts through knowledge-graph traversal <CardGroup> <Card title="Hybrid Search" icon="magnifying-glass"> Semantic (vector) + keyword (BM25) scoring with configurable weights </Card> <Card title="Context Assembly" icon="brain"> One-shot endpoint returns an LLM-ready context string for any query </Card> <Card title="Graph Relationships" icon="diagram-project"> Extracted facts link together as a knowledge graph </Card> <Card title="Multi-Collection" icon="folders"> Personal, team, knowledge, and hive (cross-collection) memory types </Card> <Card title="Document Upload" icon="file-arrow-up"> Upload PDFs, Office docs, and scanned images — text extraction + OCR built in </Card> <Card title="91+ Formats" icon="files"> PDF, DOCX, XLSX, PPTX, EML, MSG, HTML, scanned images with built-in OCR </Card> </CardGroup> ## Core concepts ### Memory collections Memories live in **collections** scoped to your workspace. Each collection is one of: | Kind | Visibility | Who can write | | ------------- | --------------------------------- | ------------------------------------- | | **personal** | Only the owning user | Owner + admin (automatic per-user) | | **team** | All members of the workspace | Owner/admin create; all members write | | **knowledge** | All members (read-only reference) | Owner/admin only | | **hive** | Cross-collection shared facts | Owner/admin only | Your personal collection is created automatically the first time you use Memory. Team/knowledge/hive collections are created by owners/admins. ### Memory types When you store a memory, you specify its type: * **`user`** — private, user-scoped facts. Auto-extracted from conversations or manually entered. * **`knowledge`** — reference content (docs, policies, FAQs). Read by all members. * **`hive`** — workspace-wide shared facts that appear in every user's search results. ### Two search modes * **Fast** — single-query hybrid search. Returns in \~100ms. Default. * **Thinking** — multi-query expansion with reranking. Better recall quality, slower (\~1-2s). ## Storing memories <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Store a personal memory await client.memory.ingest({ text: "I prefer vegetarian food and am lactose intolerant", type: "user", infer: true, // extract structured facts via LLM }); // Store a knowledge base entry await client.memory.ingest({ text: "Our refund policy allows returns within 30 days of purchase.", title: "Refund Policy", type: "knowledge", collection: "company_handbook", }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient(api_key="your-api-key") # Store a personal memory client.memory.ingest( text="I prefer vegetarian food and am lactose intolerant", type="user", infer=True, ) # Store a knowledge base entry client.memory.ingest( text="Our refund policy allows returns within 30 days of purchase.", title="Refund Policy", type="knowledge", collection="company_handbook", ) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/ingest \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "text": "I prefer vegetarian food and am lactose intolerant", "type": "user", "infer": true }' ``` </Tab> </Tabs> ### Response ```json theme={null} { "success": true, "data": { "collection_id": "user_abc123", "collection_label": "Personal Memories", "memory_type": "user", "total_queued": 1, "results": [ { "id": "mem_01HV...", "status": "pending", "message": "Queued for processing" } ] } } ``` Memories are processed asynchronously. Poll `GET /memory/:id/status` to check if ingestion is complete. ## Uploading documents For longer-form content — PDFs, Word docs, spreadsheets, scanned pages, emails — use `POST /memory/documents/extract`. The server handles **format detection, OCR, and chunking** for you, so you just upload the raw file and 60db does the rest. Under the hood, 60db runs your file through its document extraction engine (91+ formats with built-in OCR for scanned documents), splits the extracted text into overlapping chunks, and ingests each chunk as a `knowledge`-type memory in one batch. <Tabs> <Tab title="JavaScript"> ```javascript theme={null} // Upload a PDF from a file input const form = new FormData(); form.append('file', pdfFile); // File or Blob form.append('collection', 'company_handbook'); form.append('type', 'knowledge'); form.append('title', 'Employee Handbook 2026'); const res = await fetch('https://api.60db.com/memory/documents/extract', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key' }, body: form, // do NOT set Content-Type — browser adds multipart boundary }); const { data } = await res.json(); console.log(`Uploaded ${data.filename} → ${data.chunks} chunks (${data.characters} chars)`); ``` </Tab> <Tab title="Python"> ```python theme={null} import requests with open('handbook.pdf', 'rb') as f: res = requests.post( 'https://api.60db.com/memory/documents/extract', headers={'Authorization': 'Bearer sk_your_api_key'}, files={'file': ('handbook.pdf', f, 'application/pdf')}, data={ 'collection': 'company_handbook', 'type': 'knowledge', 'title': 'Employee Handbook 2026', }, ) data = res.json()['data'] print(f"Uploaded {data['filename']} → {data['chunks']} chunks") ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/documents/extract \ -H "Authorization: Bearer sk_your_api_key" \ -F "file=@handbook.pdf" \ -F "collection=company_handbook" \ -F "type=knowledge" \ -F "title=Employee Handbook 2026" ``` </Tab> </Tabs> ### What you can upload | Category | Formats | | --------------------- | --------------------------------------------- | | **Documents** | PDF, DOCX, DOC, ODT, RTF, TXT, MD, HTML, EPUB | | **Spreadsheets** | XLSX, XLS, CSV, ODS | | **Presentations** | PPTX, PPT, ODP | | **Email** | EML, MSG, PST, MBOX | | **Images (OCR)** | PNG, JPG, JPEG, TIFF, BMP, GIF | | **Code & structured** | JSON, XML, YAML, LaTeX | | **Archives** | ZIP, TAR, GZIP, 7Z (extracted recursively) | Max file size: **200 MB**. Max chunks per document: **100** (tune `chunk_size` for larger docs). ### Response ```json theme={null} { "success": true, "data": { "collection_id": "company_handbook", "collection_label": "Company Handbook", "filename": "handbook.pdf", "chunks": 18, "characters": 24680, "memory_type": "knowledge", "total_queued": 18, "results": [ /* one {id, status, message} per chunk */ ], "metadata": { "source": "document_upload", "mime_type": "application/pdf", "page_count": 24, "detected_languages": ["eng"], "total_chunks": 18 } } } ``` <Tip> The returned `metadata.page_count` and `detected_languages` come from the document extraction engine and are useful for displaying upload progress or filtering by source language. Scanned PDFs will list the OCR-detected language codes (`eng`, `fra`, `spa`, etc.). </Tip> ### Chunking controls Two optional form fields tune how text is split: | Field | Default | Description | | --------------- | ------- | ----------------------------------------------------- | | `chunk_size` | 1500 | Max characters per chunk (200–8000) | | `chunk_overlap` | 200 | Characters of overlap between chunks (\< chunk\_size) | See the [API reference](/api-reference/memory/extract-document) for a tuning table by document type. ## Searching memories <Tabs> <Tab title="JavaScript"> ```javascript theme={null} const results = await client.memory.search({ query: "What are my dietary preferences?", mode: "fast", max_results: 10, alpha: 0.8, // 0.8 semantic / 0.2 keyword recency_bias: 0.1, }); results.data.sources.forEach(source => { console.log(source.title, source.text, source.score); }); ``` </Tab> <Tab title="Python"> ```python theme={null} results = client.memory.search( query="What are my dietary preferences?", mode="fast", max_results=10, alpha=0.8, recency_bias=0.1, ) for source in results["data"]["sources"]: print(source["title"], source["text"], source["score"]) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/search \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "query": "What are my dietary preferences?", "mode": "fast", "max_results": 10, "alpha": 0.8 }' ``` </Tab> </Tabs> ### Tuning search | Parameter | Default | Description | | --------------- | ------- | -------------------------------------------------------------- | | `mode` | `fast` | `fast` (single-query) or `thinking` (multi-query reranked) | | `alpha` | 0.8 | Weight of semantic search: 0 = keyword only, 1 = semantic only | | `recency_bias` | 0.0 | Weight given to newer memories (0-1) | | `max_results` | 10 | Max results returned (capped at 50) | | `graph_context` | false | Include knowledge-graph relationships in response | ## Context assembly (RAG for SLM chat) The `/memory/context` endpoint is purpose-built for retrieval-augmented generation. Given a user query, it fetches the most relevant memories, recent events, and graph relationships, and returns a pre-formatted context string ready to prepend to your LLM prompt. <Tabs> <Tab title="JavaScript"> ```javascript theme={null} // Before calling the SLM, assemble context const ctx = await client.memory.context({ query: userMessage, session_id: chatSessionId, top_k: 8, max_context_length: 2000, include_timeline: true, include_graph: false, }); // Prepend to system message const systemMessage = `You are a helpful assistant. ## Relevant memories ${ctx.data.prompt_ready}`; // Call SLM chat with enriched context const response = await client.chat.completions.create({ model: '60db-tiny', messages: [ { role: 'system', content: systemMessage }, { role: 'user', content: userMessage }, ], }); ``` </Tab> <Tab title="Python"> ```python theme={null} # Before calling the SLM, assemble context ctx = client.memory.context( query=user_message, session_id=chat_session_id, top_k=8, max_context_length=2000, include_timeline=True, ) system_message = f"""You are a helpful assistant. ## Relevant memories {ctx['data']['prompt_ready']} """ response = client.chat.completions.create( model="60db-tiny", messages=[ {"role": "system", "content": system_message}, {"role": "user", "content": user_message}, ], ) ``` </Tab> </Tabs> ## Built-in SLM Chat integration When using 60db's UI at `/app/slm-chat`, there's a **Memory** toggle next to Auto-clear. When enabled, every message you send is pre-processed: 1. Your message is sent to `/memory/context` with your session ID 2. Relevant memories and recent events are fetched (semantic + keyword + temporal) 3. The returned `prompt_ready` string is prepended to the system message 4. The enriched prompt goes to the SLM This means your AI chat remembers context across sessions automatically. Toggle it off if you want a fresh, memoryless conversation. ## Collections management ```javascript theme={null} // List your collections const collections = await client.memory.collections.list(); // Create a team collection (admin/owner only) await client.memory.collections.create({ collection_id: "customer_support", label: "Customer Support KB", kind: "team", shared: true, }); ``` ## Role-based access Memory operations are gated by your workspace role: | Action | Owner | Admin | Developer | Member | Viewer | | ---------------------- | :---: | :---: | :-------: | :----: | :----: | | Search memories | ✓ | ✓ | ✓ | ✓ | ✓ | | Create personal memory | ✓ | ✓ | ✓ | ✓ | — | | Delete own memory | ✓ | ✓ | ✓ | ✓ | — | | Delete any memory | ✓ | ✓ | — | — | — | | Create team collection | ✓ | ✓ | — | — | — | | Create knowledge/hive | ✓ | ✓ | — | — | — | | Export memories | ✓ | ✓ | ✓ | ✓ | — | ## API key access To use Memory via an API key (for programmatic access), the key must have the `memory` scope. When creating an API key in Settings → Developers, check the "Memory & RAG" box. ```javascript theme={null} // API key with memory scope const response = await fetch('https://api.60db.com/memory/search', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query: 'user preferences', max_results: 5 }), }); ``` ## Pricing Memory is **pay-as-you-go** — no subscription, no seat pricing, no minimum commitment. You pay only for the operations you run, deducted from a single workspace wallet you top up via Stripe, Razorpay, or Dodo Payments. | Operation | Rate | | ---------------------------- | ------------------------------------------- | | Ingest a memory | **\$0.0001 per 1,000 characters** | | Upload a document (extract) | **\$0.003 per MB** | | Upload a document (ingest) | **\$0.0001 per 1,000 extracted characters** | | Search (hybrid recall) | **\$0.0003 per query** | | Context assembly (LLM-ready) | **\$0.0005 per query** | **Real-world cost examples:** * A knowledge base with 100 MB of docs + 10,000 searches/month → **\~\$23/month** * A personal assistant with 1,000 user memories + 500 searches/day → **\~\$5/month** * A support bot with 1 GB of docs + 100,000 searches/month → **\~\$53/month** <Tip> Compared to proprietary memory services at $249–$5,000/month flat, 60db Memory is 5–50x cheaper for most workloads — and you only pay for what you actually use. </Tip> Every billable request returns these response headers so you can track spend without polling: ``` x-credit-balance: 9.465200 ← wallet balance after this charge x-credit-charged: 0.000300 ← amount charged for this request x-billing-tx: 84ffd09e-... ← audit row UUID ``` **Automatic refunds** — if a request fails after being charged (upstream outage, corrupt file, etc.), the charge is reversed automatically and logged as a compensating row in `transaction_log`. No support tickets required. **Never billed** — listing collections, creating collections, checking memory status, deleting memories, and `GET /memory/usage` are **always free** so you can still manage your data when the wallet is empty. See the full [Pricing & Billing reference](/api-reference/memory/pricing) for rate details, refund policy, and the complete header/error reference. ## Handling insufficient credits When the wallet runs out, billable endpoints return **HTTP 402**: ```json theme={null} { "success": false, "message": "Insufficient credits", "error_code": "INSUFFICIENT_CREDITS", "details": { "required": 0.0003, "available": 0.00001, "shortfall": 0.00029 } } ``` Your client should catch `error_code === "INSUFFICIENT_CREDITS"` and prompt the user to top up. Here's a pattern for the search endpoint: ```javascript theme={null} async function searchWithTopUpPrompt(query) { const res = await fetch('https://api.60db.com/memory/search', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query, max_results: 10 }), }); if (res.status === 402) { const err = await res.json(); showTopUpModal({ shortfall: err.details.shortfall, available: err.details.available, topUpUrl: 'https://app.60db.com/app/billing', }); return null; } const data = await res.json(); console.log(`Balance: $${res.headers.get('x-credit-balance')}`); return data; } ``` ## Tracking usage Call [`GET /memory/usage`](/api-reference/memory/usage) to get a monthly spend breakdown by operation type. This is what powers the **Spend this month** card on the 60db Memory dashboard. ```javascript theme={null} const res = await fetch('https://api.60db.com/memory/usage?period=current_month', { headers: { 'Authorization': 'Bearer sk_your_api_key' }, }); const { data } = await res.json(); console.log(`Spent $${data.total.net_spend_usd.toFixed(4)} this month`); console.log(`Wallet: $${data.billing_owner.current_balance_usd.toFixed(4)}`); ``` ## Failure handling The Memory service is designed to **degrade gracefully**: * If the memory layer is temporarily unreachable, `POST /memory/ingest` queues your memory in a retry table, returns `202 Accepted`, and **automatically refunds** the charge so you aren't billed for work that didn't happen. * `POST /memory/context` returns an empty prompt on outage — your SLM chat still works, just without memory context. No charge when context is empty. * `POST /memory/search` returns `503` — the UI shows a "Memory temporarily unavailable" banner without blocking other features. Auto-refunded. * `POST /memory/documents/extract` auto-refunds the extract fee if extraction fails (corrupt file, empty PDF, OCR error). If extraction succeeds but the wallet can't cover the post-extraction ingest fee, the extract fee is refunded and a `402` is returned. ## Limits * **Ingest batch**: Up to 100 memories per request * **Memory text**: Max 100,000 characters per entry * **Query length**: Max 2,000 characters * **Results**: Max 50 per search (refine query for more precise results) * **Context length**: Max 16,000 tokens assembled per request * **Document upload**: Max 200 MB per file, max 100 chunks per document * **Rate limit**: 30 ingests/min per workspace ## Further reading * [API Reference: Pricing & Billing](/api-reference/memory/pricing) * [API Reference: Get Memory Usage](/api-reference/memory/usage) * [API Reference: Ingest memory](/api-reference/memory/ingest) * [API Reference: Upload document](/api-reference/memory/extract-document) * [API Reference: Search memories](/api-reference/memory/search) * [API Reference: Context assembly](/api-reference/memory/context) * [API Reference: Collections](/api-reference/memory/collections) # Speech-to-Text Source: https://docs.60db.ai/features/speech-to-text Transcribe audio to text with high accuracy ## Overview 60db's Speech-to-Text (STT) API converts spoken audio into written text with high accuracy across 39 languages, including code-switched Indic+English. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack) — non-hallucinating models that don't invent text on silent or noisy input. ## Features <CardGroup> <Card title="Multi-Language" icon="globe"> 39 languages with auto-detection and Indic+English code-switching </Card> <Card title="Speaker Diarization" icon="users"> Opt-in pyannote speaker diarization via `diarize: true` </Card> <Card title="Timestamps" icon="clock"> Word-level timestamps included automatically </Card> <Card title="Non-hallucinating" icon="shield-check"> Non-hallucinating backend that emits blank tokens on silence — no phantom text </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const file = document.querySelector('input[type="file"]').files[0]; const result = await client.speechToText(file, { language: 'en' }); console.log('Transcription:', result.text); console.log('Confidence:', result.confidence); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') with open('recording.mp3', 'rb') as audio_file: result = client.speech_to_text(audio_file, language='en') print(f"Transcription: {result['text']}") print(f"Confidence: {result['confidence']}") ``` </Tab> </Tabs> ## Supported Formats | Format | Max Size | Max Duration | Quality | | ------ | -------- | ------------ | --------- | | MP3 | 25MB | 10 min | Good | | WAV | 25MB | 10 min | Excellent | | FLAC | 25MB | 10 min | Lossless | | OGG | 25MB | 10 min | Good | | M4A | 25MB | 10 min | Good | ## Language Support ### Auto-Detection Let the API automatically identify the language. **Omit the `language` field** (or pass the string `"auto"`) and the server runs language identification across all 39 supported languages: ```javascript theme={null} // Simplest form — omit language entirely const result = await client.speechToText(file); // Equivalent — the REST endpoint treats "auto" as omission. // Note: on the WebSocket streaming endpoint you must use `languages: null`, // not the literal string "auto". const result2 = await client.speechToText(file, { language: 'auto' }); console.log('Detected language:', result.language); ``` ### Specify Language For lowest latency, pass a single ISO 639-1 code and the server skips language identification entirely: ```javascript theme={null} const result = await client.speechToText(file, { language: 'hi' // ISO 639-1 code }); ``` **Unsupported codes** (`ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`) and Arabic dialect tags (`ar-eg`, `ar-lv`, …) return an `unsupported_language` error. For non-MSA Arabic audio, pass `ar` for best-effort MSA transcription. ### Get Supported Languages ```javascript theme={null} const languages = await client.getLanguages(); languages.forEach(lang => { console.log(`${lang.name} (${lang.code})`); }); ``` ## Advanced Features ### Word-Level Timestamps Word timings are always included in the response — no flag needed: ```javascript theme={null} const result = await client.speechToText(file); // Word-level timing lives inside each segment result.segments.forEach(segment => { segment.words.forEach(word => { console.log(`${word.word}: ${word.start}s - ${word.end}s (${word.confidence})`); }); }); ``` ### Speaker Diarization Identify different speakers with `diarize: true`: ```javascript theme={null} const result = await client.speechToText(file, { diarize: true }); // Each segment carries a `speakers` array when diarization is enabled. // Raw labels are SPEAKER_00, SPEAKER_01, … result.segments.forEach(segment => { const speaker = segment.speakers?.[0]?.speaker ?? 'unknown'; console.log(`[${speaker}] ${segment.text}`); }); ``` Client applications typically re-label the raw `SPEAKER_NN` IDs as "Speaker 1", "Speaker 2" in order of first appearance for readability. ## Best Practices <AccordionGroup> <Accordion title="Audio Quality"> * Use high-quality recordings (16kHz+ sample rate) * Minimize background noise * Ensure clear speech * Avoid audio compression when possible </Accordion> <Accordion title="Accuracy Tips"> * Specify the language when known * Use appropriate model for your use case * Provide clean audio without music * Split very long recordings </Accordion> <Accordion title="Performance"> * Keep files under 25MB * Use appropriate format (WAV for quality, MP3 for size) * Process in batches for multiple files — but stay under the 8-concurrent STT cap per user </Accordion> </AccordionGroup> ## Use Cases ### Meeting Transcription ```python theme={null} with open('meeting.mp3', 'rb') as audio: result = client.speech_to_text( audio, diarize=True, ) # Generate meeting notes for segment in result['segments']: speaker = (segment.get('speakers') or [{}])[0].get('speaker', 'unknown') print(f"[{segment['start']:.1f}s] {speaker}: {segment['text']}") ``` ### Voice Commands ```javascript theme={null} async function processVoiceCommand(audioBlob) { const result = await client.speechToText(audioBlob, { language: 'en', }); const command = parseCommand(result.text); executeCommand(command); } ``` ### Subtitle Generation ```javascript theme={null} const result = await client.speechToText(videoAudio); const subtitles = generateSRT(result.words); saveFile('subtitles.srt', subtitles); ``` ## API Reference <Card title="Speech to Text API" icon="microphone" href="/api-reference/stt/speech-to-text"> View complete API documentation </Card> # Streaming Source: https://docs.60db.ai/features/streaming Real-time audio streaming for low-latency applications ## Overview Streaming allows you to receive audio chunks in real-time as they're generated, enabling immediate playback without waiting for the entire audio file to be created. This is essential for interactive applications like voice assistants and chatbots. ## Benefits <CardGroup> <Card title="Low Latency" icon="bolt"> Start playing audio within milliseconds </Card> <Card title="Memory Efficient" icon="memory"> Process chunks instead of loading entire file </Card> <Card title="Better UX" icon="smile"> Progressive audio playback feels more responsive </Card> <Card title="Long Content" icon="book"> Handle unlimited text length efficiently </Card> </CardGroup> ## How It Works 1. **Send Request**: Submit text to the streaming endpoint 2. **Receive Chunks**: Get audio chunks as they're generated 3. **Play Immediately**: Start playing the first chunk 4. **Continue Streaming**: Receive and play subsequent chunks 5. **Complete**: Receive completion notification ## Implementation <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Audio player setup const audioContext = new AudioContext(); const audioQueue = []; await client.textToSpeechStream( { text: 'This is a longer text that will be streamed in real-time for immediate playback.', voice_id: 'default-voice' }, { onChunk: async (chunk) => { // Convert chunk to audio buffer const audioBuffer = await audioContext.decodeAudioData(chunk.buffer); // Add to queue and play audioQueue.push(audioBuffer); if (audioQueue.length === 1) { playNextChunk(); } }, onComplete: () => { console.log('Streaming complete'); }, onError: (error) => { console.error('Streaming error:', error); } } ); function playNextChunk() { if (audioQueue.length === 0) return; const buffer = audioQueue[0]; const source = audioContext.createBufferSource(); source.buffer = buffer; source.connect(audioContext.destination); source.onended = () => { audioQueue.shift(); playNextChunk(); }; source.start(); } ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient import pyaudio client = SixtyDBClient('your-api-key') # Audio player setup p = pyaudio.PyAudio() stream = p.open( format=pyaudio.paInt16, channels=1, rate=24000, output=True ) def handle_chunk(chunk): # Play audio chunk immediately stream.write(chunk) def handle_complete(): print("Streaming complete") stream.stop_stream() stream.close() p.terminate() def handle_error(error): print(f"Error: {error}") # Stream text to speech client.text_to_speech_stream( text='This is a longer text that will be streamed in real-time.', on_chunk=handle_chunk, on_complete=handle_complete, on_error=handle_error, voice_id='default-voice' ) ``` </Tab> </Tabs> ## Advanced Usage ### React Component ```jsx theme={null} import { useState } from 'react'; import { SixtyDBClient } from '60db'; function StreamingTTS() { const [isStreaming, setIsStreaming] = useState(false); const [text, setText] = useState(''); const client = new SixtyDBClient(process.env.REACT_APP_API_KEY); const streamAudio = async () => { setIsStreaming(true); await client.textToSpeechStream( { text, voice_id: 'default-voice' }, { onChunk: (chunk) => { // Play chunk playAudioChunk(chunk); }, onComplete: () => { setIsStreaming(false); }, onError: (error) => { console.error(error); setIsStreaming(false); } } ); }; return ( <div> <textarea value={text} onChange={(e) => setText(e.target.value)} placeholder="Enter text to speak..." /> <button onClick={streamAudio} disabled={isStreaming}> {isStreaming ? 'Streaming...' : 'Speak'} </button> </div> ); } ``` ### Voice Assistant ```javascript theme={null} class VoiceAssistant { constructor(apiKey) { this.client = new SixtyDBClient(apiKey); this.audioContext = new AudioContext(); } async speak(text) { return new Promise((resolve, reject) => { this.client.textToSpeechStream( { text, voice_id: 'assistant-voice' }, { onChunk: (chunk) => this.playChunk(chunk), onComplete: () => resolve(), onError: (error) => reject(error) } ); }); } async playChunk(chunk) { const audioBuffer = await this.audioContext.decodeAudioData(chunk.buffer); const source = this.audioContext.createBufferSource(); source.buffer = audioBuffer; source.connect(this.audioContext.destination); source.start(); } } // Usage const assistant = new VoiceAssistant('your-api-key'); await assistant.speak('Hello! How can I help you today?'); ``` ## Performance Optimization ### Buffering Strategy ```javascript theme={null} class AudioStreamer { constructor() { this.chunks = []; this.isPlaying = false; this.minBufferSize = 3; // Wait for 3 chunks before playing } addChunk(chunk) { this.chunks.push(chunk); if (!this.isPlaying && this.chunks.length >= this.minBufferSize) { this.startPlayback(); } } async startPlayback() { this.isPlaying = true; while (this.chunks.length > 0) { const chunk = this.chunks.shift(); await this.playChunk(chunk); } this.isPlaying = false; } async playChunk(chunk) { // Play audio chunk return new Promise((resolve) => { // Audio playback logic setTimeout(resolve, chunkDuration); }); } } ``` ## Best Practices <AccordionGroup> <Accordion title="Buffer Management"> * Maintain a small buffer (2-3 chunks) for smooth playback * Handle network interruptions gracefully * Implement retry logic for failed chunks </Accordion> <Accordion title="Error Handling"> * Always implement onError callback * Provide user feedback during streaming * Have fallback for streaming failures </Accordion> <Accordion title="Performance"> * Reuse AudioContext instances * Clean up resources after playback * Monitor memory usage for long streams </Accordion> <Accordion title="User Experience"> * Show loading indicator before first chunk * Allow users to stop streaming * Provide playback controls </Accordion> </AccordionGroup> ## Use Cases ### Real-time Chat ```javascript theme={null} async function sendMessage(message) { // Display user message displayMessage('user', message); // Get AI response const response = await getAIResponse(message); displayMessage('assistant', response); // Stream audio response await client.textToSpeechStream( { text: response }, { onChunk: (chunk) => playAudioChunk(chunk), onComplete: () => enableInput() } ); } ``` ### Audiobook Player ```javascript theme={null} async function playChapter(chapterText) { let currentPosition = 0; await client.textToSpeechStream( { text: chapterText }, { onChunk: (chunk) => { playChunk(chunk); currentPosition += chunk.duration; updateProgress(currentPosition); }, onComplete: () => { moveToNextChapter(); } } ); } ``` ## API Reference <Card title="Streaming API" icon="signal-stream" href="/api-reference/tts/text-to-speech-stream"> View complete streaming API documentation </Card> # Text-to-Speech Source: https://docs.60db.ai/features/text-to-speech Convert text to natural-sounding speech ## Overview 60db's Text-to-Speech (TTS) API converts written text into natural-sounding speech using advanced AI models. Our TTS engine supports multiple voices, languages, and customization options. ## Features <CardGroup> <Card title="Multiple Voices" icon="users"> Choose from 50+ pre-built voices or create custom voices </Card> <Card title="Voice Customization" icon="sliders"> Adjust speed, stability, and similarity </Card> <Card title="High Quality" icon="star"> Crystal-clear audio with natural intonation </Card> <Card title="Multiple Formats" icon="file-audio"> Support for MP3, WAV, OGG, and FLAC output formats </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice', enhance: true, speed: 1.0 }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') audio = client.text_to_speech( text='Hello, world!', voice_id='default-voice', enhance=True, speed=1.0 ) with open('output.mp3', 'wb') as f: f.write(audio) ``` </Tab> </Tabs> ## Voice Parameters ### Speed Control the speaking rate of the generated audio: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'This will be spoken faster', speed: 1.5 // Range: 0.5 to 2.0 }); ``` * `0.5`: Half speed (slow) * `1.0`: Normal speed (default) * `2.0`: Double speed (fast) ### Stability Control how expressive versus consistent the generated voice sounds: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'More consistent delivery', stability: 50 // Range: 0 to 100 (default 50) }); ``` * Lower values produce more expressive, varied output * Higher values produce more consistent, stable output ### Similarity Control how closely the output matches the source voice: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Close clone match', similarity: 75 // Range: 0 to 100 (default 75) }); ``` ### Enhancement Enable audio enhancement for better quality: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Enhanced audio quality', enhance: true // Default: true }); ``` ## Output Formats Supported audio formats: | Format | Quality | File Size | Use Case | | ------ | --------- | --------- | --------------------- | | MP3 | Good | Small | Web, mobile apps | | WAV | Excellent | Large | Professional audio | | OGG | Good | Small | Web streaming | | FLAC | Lossless | Medium | High-quality archival | ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Hello, world!', output_format: 'wav' // mp3, wav, ogg, flac }); ``` ## Best Practices <AccordionGroup> <Accordion title="Text Formatting"> * Use proper punctuation for natural pauses * Break long texts into paragraphs * Use SSML tags for advanced control (coming soon) </Accordion> <Accordion title="Voice Selection"> * Test multiple voices for your use case * Consider accent and gender for your audience * Use custom voices for brand consistency </Accordion> <Accordion title="Performance"> * Cache frequently used audio * Batch requests when possible * Use appropriate audio format for your use case </Accordion> <Accordion title="Quality"> * Enable enhancement for production use * Use WAV format for highest quality * Test with different speed settings </Accordion> </AccordionGroup> ## Use Cases ### Voice Assistants ```javascript theme={null} // Voice assistant async function speakResponse(text) { const audio = await client.textToSpeech({ text: text, voice_id: 'assistant-voice', enhance: true }); playAudio(audio); } ``` ### Content Narration ```javascript theme={null} // Generate audiobook chapter const audio = await client.textToSpeech({ text: chapterText, voice_id: 'narrator-voice', speed: 0.95, output_format: 'mp3' }); saveToFile(`chapter-${chapterNum}.mp3`, audio); ``` ### Accessibility ```javascript theme={null} // Make web content accessible async function readAloud(element) { const text = element.textContent; const audio = await client.textToSpeech({ text, voice_id: 'clear-voice', enhance: true }); playAudio(audio); } ``` ## API Reference For detailed API documentation, see: <Card title="Text to Speech" icon="microphone" href="/api-reference/tts/text-to-speech"> Standard TTS endpoint </Card> # Custom Voices Source: https://docs.60db.ai/features/voices Create and manage custom voice profiles ## Overview Create custom voice profiles that match your brand identity or clone specific voices for personalized experiences. Our voice cloning technology requires just a few minutes of audio to create a high-quality custom voice. ## Creating a Custom Voice ### Requirements <Card title="Audio Samples" icon="microphone"> * **Minimum**: 3 audio files * **Maximum**: 10 audio files * **Duration**: 10-60 seconds per file * **Total**: At least 2 minutes combined * **Format**: MP3, WAV, or FLAC * **Quality**: 44.1kHz+ sample rate recommended </Card> ### Step-by-Step Guide <Steps> <Step title="Prepare Audio Samples"> Record or collect 3-10 high-quality audio samples of the voice you want to clone </Step> <Step title="Upload Samples"> Use the API or dashboard to upload your audio files </Step> <Step title="Wait for Processing"> Voice cloning typically takes 10-15 minutes </Step> <Step title="Test Your Voice"> Generate test audio to verify quality </Step> <Step title="Use in Production"> Start using your custom voice in your applications </Step> </Steps> ## Code Examples <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Create custom voice const files = [ document.querySelector('#file1').files[0], document.querySelector('#file2').files[0], document.querySelector('#file3').files[0] ]; const voice = await client.createVoice({ name: 'My Brand Voice', description: 'Professional voice for customer service', language: 'en', gender: 'female', files: files }); console.log('Voice ID:', voice.id); console.log('Status:', voice.status); // Use the custom voice const audio = await client.textToSpeech({ text: 'Hello from my custom voice!', voice_id: voice.id }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') # Create custom voice files = [ open('sample1.mp3', 'rb'), open('sample2.mp3', 'rb'), open('sample3.mp3', 'rb') ] voice = client.create_voice( name='My Brand Voice', files=files, description='Professional voice for customer service', language='en', gender='female' ) print(f"Voice ID: {voice['id']}") print(f"Status: {voice['status']}") # Close files for f in files: f.close() # Use the custom voice audio = client.text_to_speech( text='Hello from my custom voice!', voice_id=voice['id'] ) with open('output.mp3', 'wb') as f: f.write(audio) ``` </Tab> </Tabs> ## Audio Sample Guidelines ### Content Recommendations <AccordionGroup> <Accordion title="Variety"> * Include different sentence types (questions, statements, exclamations) * Cover various emotions and tones * Use different speaking speeds * Include both short and long sentences </Accordion> <Accordion title="Quality"> * Record in a quiet environment * Use a good quality microphone * Maintain consistent volume * Avoid background music or noise * No echo or reverb </Accordion> <Accordion title="Technical"> * Sample rate: 44.1kHz or higher * Bit depth: 16-bit or higher * Format: WAV (lossless) preferred * Mono or stereo both acceptable </Accordion> <Accordion title="Content"> * Natural, conversational speech * Clear pronunciation * Consistent accent * Avoid reading in monotone * Include natural pauses </Accordion> </AccordionGroup> ## Managing Custom Voices ### List Your Voices ```javascript theme={null} const voices = await client.getVoices(); // Filter custom voices const customVoices = voices.filter(v => v.is_custom); customVoices.forEach(voice => { console.log(`${voice.name} (${voice.id})`); }); ``` ### Update Voice Metadata ```javascript theme={null} await client.updateVoice('voice-id', { name: 'Updated Voice Name', description: 'Updated description' }); ``` ### Delete a Voice ```javascript theme={null} await client.deleteVoice('voice-id'); ``` ## Voice Quality Tips <CardGroup> <Card title="Recording Environment" icon="location-dot"> Record in a quiet room with minimal echo and background noise </Card> <Card title="Microphone Quality" icon="microphone"> Use a quality microphone for best results </Card> <Card title="Speaking Style" icon="comment"> Speak naturally and expressively </Card> <Card title="Audio Length" icon="clock"> Provide at least 2 minutes of total audio </Card> </CardGroup> ## Use Cases ### Brand Voice Create a consistent voice for all your brand communications: ```javascript theme={null} const brandVoice = await client.createVoice({ name: 'Acme Brand Voice', description: 'Official voice for Acme Corporation', files: brandAudioSamples }); // Use in all customer touchpoints const greeting = await client.textToSpeech({ text: 'Welcome to Acme Corporation. How can we help you today?', voice_id: brandVoice.id }); ``` ### Personal Assistant Clone your own voice for a personalized assistant: ```javascript theme={null} const myVoice = await client.createVoice({ name: 'My Personal Voice', description: 'My voice for personal assistant', files: myRecordings }); // Personal reminders in your own voice const reminder = await client.textToSpeech({ text: 'Remember to call mom at 3 PM', voice_id: myVoice.id }); ``` ### Character Voices Create unique voices for game characters or audiobooks: ```javascript theme={null} const characterVoice = await client.createVoice({ name: 'Wizard Character', description: 'Mystical wizard voice for fantasy game', files: characterSamples }); ``` ## Pricing Custom voice creation is available on Pro and Enterprise plans: | Plan | Custom Voices | Processing Time | | ---------- | ------------- | ------------------- | | Free | 0 | - | | Starter | 0 | - | | Pro | 5 | 10-15 min | | Enterprise | Unlimited | Priority (5-10 min) | ## API Reference <CardGroup> <Card title="Create Voice" icon="plus" href="/api-reference/voices/create-voice"> Create a new custom voice </Card> <Card title="Manage Voices" icon="gear" href="/api-reference/voices/get-voices"> List, update, and delete voices </Card> </CardGroup> # LiveKit Integration Source: https://docs.60db.ai/integrations/livekit Use 60db STT, TTS, and LLM inside a LiveKit Agents voice pipeline. # LiveKit Integration Build real-time voice agents with [LiveKit Agents](https://github.com/livekit/agents) powered by 60db's speech and language services — STT, TTS, and LLM all in one plugin. <CardGroup> <Card title="Speech-to-Text" icon="microphone"> Real-time streaming transcription via WebSocket with interim results </Card> <Card title="Text-to-Speech" icon="volume-high"> Low-latency streaming synthesis with chunked audio delivery </Card> <Card title="LLM Chat" icon="brain"> OpenAI-compatible chat completions with tool-call support </Card> </CardGroup> *** ## Installation <Steps> <Step title="Install the plugin"> Requires Python **3.10+**. ```bash theme={null} pip install livekit-plugins-60db ``` </Step> <Step title="Set your API key"> Choose one of the following methods: **Option A — Environment variable (recommended):** ```bash theme={null} export SIXTY_DB_API_KEY="your-api-key" ``` **Option B — `.env.local` file:** ``` SIXTY_DB_API_KEY=your-api-key ``` **Option C — Pass it directly in code:** ```python theme={null} from livekit.plugins._60db import _60dbClient client = _60dbClient("your-api-key") # sets global default ``` </Step> </Steps> *** ## Quick Start Wire all three services together in a `VoicePipelineAgent`: ```python theme={null} import asyncio from livekit.agents import VoicePipelineAgent, cli, WorkerType from livekit.plugins import silero from livekit.plugins._60db import _60dbClient, STT, TTS, LLM client = _60dbClient("your-api-key") # sets global default async def entrypoint(ctx): await ctx.connect() agent = VoicePipelineAgent( vad=silero.VAD.load(), # voice activity detection stt=STT(), # 60db speech-to-text llm=LLM(), # 60db language model tts=TTS(), # 60db text-to-speech ) agent.start(ctx.room) if __name__ == "__main__": cli.run_app(worker_type=WorkerType.ROOM, entrypoint_fnc=entrypoint) ``` *** ## Configuration ### Environment Variables All services read from the same environment variables by default. You only need to override them if you use a custom deployment. | Variable | Default | Description | | ------------------ | ----------------------------------------- | ---------------------------- | | `SIXTY_DB_API_KEY` | — | Your 60db API key (required) | | `SIXTY_DB_STT_URL` | `wss://api.60db.ai/ws/stt` | STT WebSocket endpoint | | `SIXTY_DB_TTS_URL` | `wss://api.60db.ai/ws/tts` | TTS WebSocket endpoint | | `SIXTY_DB_LLM_URL` | `https://api.60db.ai/v1/chat/completions` | LLM HTTP endpoint | Each service also accepts a direct `ws_url` (or `api_url`) constructor argument which takes precedence over environment variables. *** ## Services <Tabs> <Tab title="STT"> ### Speech-to-Text The STT service streams audio to 60db over WebSocket and returns transcriptions in real time — including interim (partial) results as the speaker is still talking. #### Parameters | Name | Type | Default | Description | | ----------------- | ------------------- | ----------------------- | ------------------------------------------------ | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `ws_url` | `str \| None` | env var | WebSocket endpoint URL | | `languages` | `list[str] \| None` | `["en"]` | Language codes for recognition | | `encoding` | `str` | `"mulaw"` | Audio encoding format | | `sample_rate` | `int` | `8000` | Audio sample rate in Hz | | `continuous_mode` | `bool` | `True` | Keep the session open for continuous recognition | #### Example ```python theme={null} from livekit.plugins._60db import STT stt = STT() # picks up api_key from global client or env var async with stt.stream() as stream: stream.push_frame(audio_frame) # push raw audio frames async for event in stream: if event.type == stt.SpeechEventType.FINAL_TRANSCRIPT: print(event.alternatives[0].text) ``` #### Audio Formats The plugin automatically handles audio conversion for you — no manual preprocessing needed. | Step | What happens | | ------------- | --------------------------------------------------------------------- | | Stereo → Mono | Downmix via `audioop.tomono` | | Resampling | Any input rate → target rate via `audioop.ratecv` | | Encoding | LINEAR16 PCM → mulaw via `audioop.lin2ulaw` (when `encoding="mulaw"`) | | Parameter | Default | Supported | | ----------- | --------- | ------------------------- | | Encoding | `mulaw` | `mulaw`, `LINEAR16` | | Sample rate | `8000 Hz` | Any rate (auto-resampled) | </Tab> <Tab title="TTS"> ### Text-to-Speech The TTS service converts text to speech over WebSocket. It supports both **one-shot** synthesis and **streaming** mode for low-latency incremental output. #### Parameters | Name | Type | Default | Description | | ------------- | ------------- | ---------------------------------------- | ------------------------ | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `ws_url` | `str \| None` | env var | WebSocket endpoint URL | | `voice_id` | `str` | `"fbb75ed2-975a-40c7-9e06-38e30524a9a1"` | Voice identifier | | `encoding` | `str` | `"LINEAR16"` | Output audio encoding | | `sample_rate` | `int` | `16000` | Output sample rate in Hz | #### Examples **One-shot synthesis** — best for short, complete responses: ```python theme={null} from livekit.plugins._60db import TTS tts = TTS() async for chunk in tts.synthesize("Hello, how can I help you?"): # chunk.data contains raw PCM audio bytes print(f"Received {len(chunk.data)} bytes") ``` **Streaming synthesis** — best for incremental LLM output: ```python theme={null} async with tts.stream() as stream: stream.push_text("Hello, ") stream.push_text("how can I help you?") stream.end_input() async for chunk in stream: print(f"Received {len(chunk.data)} bytes") ``` #### Audio Format | Parameter | Default | Notes | | ----------- | ---------- | ------------------------------ | | Encoding | `LINEAR16` | Raw PCM | | Sample rate | `16000 Hz` | Configurable via `sample_rate` | Audio is returned as base64-encoded PCM chunks by the server, decoded automatically by the plugin. </Tab> <Tab title="LLM"> ### Language Model The LLM service provides OpenAI-compatible chat completions with SSE streaming and tool-call support. #### Parameters | Name | Type | Default | Description | | ---------------------- | --------------- | ----------------------- | ----------------------------------- | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `api_url` | `str \| None` | env var | Chat completions endpoint URL | | `model` | `str` | `"qcall/slm-3b-int4"` | Model identifier | | `temperature` | `float \| None` | `None` | Sampling temperature (0–2) | | `top_p` | `float \| None` | `None` | Nucleus sampling probability | | `top_k` | `int \| None` | `None` | Top-K sampling parameter | | `max_tokens` | `int \| None` | `None` | Maximum tokens in the response | | `chat_template_kwargs` | `dict \| None` | `None` | Extra template kwargs for the model | #### Example ```python theme={null} from livekit.plugins._60db import LLM from livekit.agents import llm model = LLM(temperature=0.7) chat_ctx = llm.ChatContext() chat_ctx.append(role="system", text="You are a helpful voice assistant.") chat_ctx.append(role="user", text="What is the capital of France?") async for chunk in model.chat(chat_ctx=chat_ctx): print(chunk.choices[0].delta.content, end="") ``` #### Tool Calls The LLM supports OpenAI-style function tool calls. When the model calls a tool, the plugin accumulates the streamed argument fragments and emits a complete `FunctionToolCall` once the stream finishes. </Tab> </Tabs> *** ## Timeouts Control timeouts per-request by passing `APIConnectOptions`: ```python theme={null} from livekit.agents import APIConnectOptions opts = APIConnectOptions(timeout=30.0) # STT async with stt.stream(conn_options=opts) as s: ... # TTS async for chunk in tts.synthesize("Hello", conn_options=opts): ... # LLM async for chunk in model.chat(chat_ctx=ctx, conn_options=opts): ... ``` *** ## Error Handling All three services raise standard LiveKit Agents exceptions: | Exception | When it's raised | | -------------------- | ----------------------------------------------------------------------- | | `APIConnectionError` | WebSocket handshake failure, HTTP error, or unexpected protocol message | | `APITimeoutError` | Request exceeds the configured timeout | | `ValueError` | Missing API key or service URL at construction time | ### Common HTTP Error Codes (LLM) | Status | Meaning | | ------ | -------------------------- | | `401` | Invalid or missing API key | | `429` | Rate limit exceeded | | `500` | Server error | | `503` | Service unavailable | ### Retry Tips * **STT**: If you receive an `error` message during the handshake, reconnect with a short delay. A `connecting` status before `connection_established` is normal. * **TTS**: Retry connection failures with exponential backoff. * **LLM**: `httpx.TimeoutException` → `APITimeoutError`; `httpx.HTTPStatusError` → `APIConnectionError`. *** ## WebSocket Protocol Reference <Accordion title="STT WebSocket Protocol"> **Connect:** ``` wss://api.60db.ai/ws/stt?apiKey={API_KEY} ``` **Handshake:** Server → `{"connection_established": true}` Client → start command: ```json theme={null} { "type": "start", "languages": ["en"], "config": { "encoding": "mulaw", "sample_rate": 8000, "continuous_mode": true } } ``` Server → `{"type": "connected"}` **Audio:** Send raw audio as **binary WebSocket frames**. **Transcription response:** ```json theme={null} { "type": "transcription", "text": "Hello, world", "is_final": false, "language": "en" } ``` **Stop:** Client → `{"type": "stop"}` Server → ```json theme={null} { "type": "session_stopped", "billing_summary": { "total_cost": "..." } } ``` </Accordion> <Accordion title="TTS WebSocket Protocol"> **Connect:** ``` wss://api.60db.ai/ws/tts?apiKey={API_KEY} ``` **Create context:** ```json theme={null} { "create_context": { "context_id": "unique-id", "voice_id": "fbb75ed2-975a-40c7-9e06-38e30524a9a1", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 } } } ``` **Send text:** ```json theme={null} { "send_text": { "context_id": "unique-id", "text": "Hello!" } } ``` **Flush** (trigger audio generation): ```json theme={null} { "flush_context": { "context_id": "unique-id" } } ``` **Audio response:** ```json theme={null} { "audio_chunk": { "audioContent": "<base64 PCM>" } } ``` Followed by `{"flush_completed": true}` when all audio has been delivered. **Close context:** ```json theme={null} { "close_context": { "context_id": "unique-id" } } ``` </Accordion> # Introduction Source: https://docs.60db.ai/introduction Welcome to 60db - The Modern Voice AI Platform ## What is 60db? 60db is a modern AI platform that combines state-of-the-art voice capabilities (TTS, STT, custom voices) with a hosted **LLM chat** and a **Memory / RAG layer** for building assistants that remember context and retrieve knowledge on demand. Developers can integrate natural-sounding voice, accurate speech recognition, and persistent memory into their applications with just a few lines of code. ## Key Features <CardGroup> <Card title="Text-to-Speech" icon="microphone" href="/features/text-to-speech"> Convert text to natural-sounding speech with multiple voice options </Card> <Card title="Speech-to-Text" icon="waveform" href="/features/speech-to-text"> Transcribe audio to text with high accuracy across multiple languages </Card> <Card title="Custom Voices" icon="user-voice" href="/features/voices"> Create and manage custom voice profiles for your brand </Card> <Card title="Real-time Streaming" icon="signal-stream" href="/features/streaming"> Stream audio in real-time for low-latency applications </Card> <Card title="LLM Chat" icon="comments" href="/features/llm-chat"> OpenAI-compatible chat completions with 60db's hosted SLM models </Card> <Card title="Memory & RAG" icon="brain" href="/features/memory"> Persistent memory, document upload (PDF/DOCX/OCR), and semantic recall for AI chat </Card> </CardGroup> ## Why Choose 60db? <AccordionGroup> <Accordion title="High Quality Audio"> Our TTS engine produces natural-sounding speech with proper intonation, emotion, and clarity that rivals human speech. </Accordion> <Accordion title="Multiple Languages"> Support for 50+ languages and dialects, making your application globally accessible. </Accordion> <Accordion title="Easy Integration"> Simple SDKs for JavaScript/TypeScript and Python make integration straightforward and quick. </Accordion> <Accordion title="Scalable Infrastructure"> Built on robust infrastructure that scales automatically to handle your growing needs. </Accordion> <Accordion title="Real-time Processing"> Low-latency streaming capabilities for real-time voice applications. </Accordion> </AccordionGroup> ## Use Cases * **Voice Assistants**: Build intelligent voice-enabled applications * **Content Creation**: Generate voiceovers for videos and podcasts * **Accessibility**: Make your content accessible to visually impaired users * **Customer Service**: Create automated voice response systems * **E-learning**: Develop interactive educational content with voice * **Gaming**: Add realistic character voices to your games ## Getting Started Ready to get started? Check out our [Quickstart Guide](/quickstart) to integrate 60db into your application in minutes. <CardGroup> <Card title="Quickstart" icon="rocket" href="/quickstart"> Get up and running in under 5 minutes </Card> <Card title="API Reference" icon="code" href="/api-reference/introduction"> Explore our comprehensive API documentation </Card> </CardGroup> # Claude desktop Source: https://docs.60db.ai/mcp-server/claude-desktop # Claude Desktop Integration Integrate the 60db MCP Server with Claude Desktop. ## Configuration ### Step 1: Build the Server ```bash theme={null} cd /path/to/mcp-server npm install npm run build ``` ### Step 2: Locate Claude Desktop Config **macOS**: ``` ~/Library/Application Support/Claude/claude_desktop_config.json ``` **Windows**: ``` %APPDATA%/Claude/claude_desktop_config.json ``` ### Step 3: Add Configuration ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/absolute/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` <Callout type="warning"> Use absolute paths. Relative paths may not work. </Callout> ### Step 4: Restart Claude Desktop Quit and restart Claude Desktop. ## Usage Examples ### Generate Speech ``` You: Convert "Hello, world!" to speech ``` ### Transcribe Audio ``` You: Transcribe this audio: https://example.com/meeting.mp3 ``` ### List Voices ``` You: What English voices are available? ``` ### Create Voice ``` You: Create a voice named "Brand Voice" from this sample ``` ## Troubleshooting ### Server Not Appearing 1. Check the config file path 2. Verify the server path is absolute 3. Check the API key is set 4. Restart Claude Desktop ### Authentication Error ```bash theme={null} export SIXTYDB_API_KEY=sk_your_api_key_here ``` ## Next Steps * [MCP Inspector](/mcp-server/mcp-inspector) - Test the server # Configuration Source: https://docs.60db.ai/mcp-server/configuration # Configuration Configure the 60db MCP Server for your environment. ## Environment Variables ### Required ```bash theme={null} # 60db API Key (required) export SIXTYDB_API_KEY=sk_your_api_key_here ``` ### Optional ```bash theme={null} # API Base URL (default: https://api.60db.ai) export SIXTYDB_API_BASE_URL=https://api.60db.ai # Debug mode (default: false) export SIXTYDB_DEBUG=true ``` ## Claude Desktop Configuration Add to your Claude Desktop config: **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` **Windows**: `%APPDATA%/Claude/claude_desktop_config.json` ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/absolute/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` <Callout type="warning"> Use absolute paths for the args array. </Callout> ## Custom Application Integration ```typescript theme={null} import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; const transport = new StdioClientTransport({ command: "node", args: ["/path/to/mcp-server/dist/index.js"], env: { SIXTYDB_API_KEY: "sk_your_api_key_here", SIXTYDB_API_BASE_URL: "https://api.60db.ai" } }); const client = new Client({ name: "my-app", version: "1.0.0" }, { capabilities: {} }); await client.connect(transport); ``` ## Testing Test with MCP Inspector: ```bash theme={null} npm install -g @modelcontextprotocol/inspector mcp-inspector node dist/index.js ``` ## Next Steps * [Quick Start](/mcp-server/quickstart) - Test your setup # Installation Source: https://docs.60db.ai/mcp-server/installation # Installation Install and set up the 60db MCP Server. **Source code:** [github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) · **npm:** [`60db-mcp-server`](https://www.npmjs.com/package/60db-mcp-server) ## Prerequisites * **Node.js 18+** - [Download](https://nodejs.org/) * **60db API Key** - Get from [https://app.60db.ai](https://app.60db.ai) ## Install from npm ```bash theme={null} npm install -g 60db-mcp-server ``` Verify the binary is on your PATH: ```bash theme={null} 60db-mcp-server --help ``` ## Install from Source ```bash theme={null} # Clone the public repository git clone https://github.com/60db-ai/mcp.git cd mcp # Install dependencies npm install # Build the server (TypeScript → dist/) npm run build # Run it npm start ``` ## Verify Installation ```bash theme={null} # Check if server is installed 60db-mcp-server --version # Or if installed from source npm start ``` ## Environment Setup Set your API key: ```bash theme={null} # Set 60db API key export SIXTYDB_API_KEY=sk_your_api_key_here # Set API base URL (optional, defaults to https://api.60db.ai) export SIXTYDB_API_BASE_URL=https://api.60db.ai ``` Alternatively, authenticate with a JWT: ```bash theme={null} export SIXTYDB_JWT_TOKEN=your_jwt_token_here ``` <Note> Legacy `QLABS_API_KEY`, `QLABS_JWT_TOKEN`, and `QLABS_API_BASE_URL` env vars are still honored for backward compatibility — existing configs will continue to work after the rename, but new deployments should use the `SIXTYDB_*` names. </Note> ### Making it Permanent Add to your `~/.bashrc`, `~/.zshrc`, or `~/.profile`: ```bash theme={null} echo 'export SIXTYDB_API_KEY=sk_your_api_key_here' >> ~/.bashrc source ~/.bashrc ``` ## Development Setup For development with auto-reload: ```bash theme={null} npm run dev ``` ## Production Build ```bash theme={null} npm run build npm start ``` ## Next Steps * [Configuration](/mcp-server/configuration) - Complete the setup * [Claude Desktop](/mcp-server/claude-desktop) - Integrate with Claude # Introduction Source: https://docs.60db.ai/mcp-server/introduction # MCP Server The 60db MCP (Model Context Protocol) Server enables AI assistants like Claude to directly interact with the full 60db platform — voice AI, persistent memory, and access-control checks. **Source code:** [github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) · **npm:** [`60db-mcp-server`](https://www.npmjs.com/package/60db-mcp-server) · **License:** MIT ## What is the MCP Server? The MCP Server provides a standardized interface for AI assistants to: * **Generate speech** from text using any voice * **Transcribe audio** files with speaker diarization * **Create custom voices** from audio samples * **Manage workspaces** and team collaboration * **Store and recall memories** — persistent memory and RAG across sessions * **Upload documents** — PDF, DOCX, XLSX, PPTX, EML, scanned images with built-in OCR * **Assemble LLM-ready context** — purpose-built for retrieval-augmented generation * **Check permissions** — query Cerbos policy before attempting gated actions * **Track usage** and analytics * **View monthly spend** across every billable operation ## Quick Start ### 1. Install the Server ```bash theme={null} # From npm npm install -g 60db-mcp-server # Or clone from source git clone https://github.com/60db-ai/mcp.git cd mcp npm install npm run build ``` ### 2. Configure Claude Desktop Add to your Claude Desktop config: **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` **Windows**: `%APPDATA%/Claude/claude_desktop_config.json` ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` ### 3. Use with Claude Restart Claude Desktop and start using 60db voice AI: ``` You: Can you convert "Hello, world!" to speech using a female English voice? Claude: I'll use the 60db TTS tool to generate speech for you. ``` ## Available Tools | Category | Tools | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **Voices** | List voices, get voice details, create custom voices | | **TTS** | Synthesize speech, view generation history | | **STT** | Transcribe audio, view transcription history | | **Workspaces** | Manage team workspaces and members | | **Memory & RAG** | Ingest memories, upload documents (91+ formats with OCR), hybrid semantic search, context assembly, manage collections, track spend (10 tools total) | | **Authz** | Fetch effective permissions map, probe single `(resource, action)` checks — useful before attempting gated actions | | **Analytics** | Track usage statistics | | **Billing** | View plans, subscriptions, and invoices | ## Features ### Text-to-Speech Generate natural speech in 40+ languages: ```typescript theme={null} // Synthesize speech { "text": "Hello, world!", "voice_id": "voice_abc123", "speed": 1.0, "output_format": "mp3" } ``` ### Speech-to-Text Transcribe audio with speaker identification: ```typescript theme={null} // Transcribe audio { "audio_url": "https://example.com/meeting.mp3", "language": "en-US", "diarization": true, "timestamps": true } ``` ### Voice Cloning Create custom voices from audio samples: ```typescript theme={null} // Create custom voice { "name": "Brand Voice", "sample_url": "https://example.com/sample.mp3", "language": "en-US" } ``` ### Memory & RAG Give your AI assistant a persistent, searchable long-term memory: ```typescript theme={null} // Store a user preference { "tool": "sixtydb_memory_ingest", "arguments": { "text": "User prefers dark mode and metric units", "type": "user" } } // Upload a document (PDF, DOCX, scanned image, etc.) { "tool": "sixtydb_memory_upload_document", "arguments": { "file_path": "/Users/me/Documents/handbook.pdf", "collection": "company_handbook", "type": "knowledge" } } // Search with hybrid semantic + keyword recall { "tool": "sixtydb_memory_search", "arguments": { "query": "What is the refund policy?", "alpha": 0.8, "mode": "fast" } } // Assemble LLM-ready context for RAG { "tool": "sixtydb_memory_context", "arguments": { "query": "Tell me about my customer's recent orders" } } ``` All memory operations are [pay-as-you-go](/api-reference/memory/pricing) — the tool response surfaces `charged`, `balance`, and `tx` details in the billing footer on every call. ### Authorization checks Before attempting a gated action, query the Cerbos permission map so the agent can fail fast on disallowed operations: ```typescript theme={null} // Fetch the full permission map (once per session) { "tool": "sixtydb_get_permissions", "arguments": {} } // Probe a specific action { "tool": "sixtydb_check_permission", "arguments": { "resource": "memory", "action": "create" } } ``` This lets the agent hide or disable features that the user's role can't use, instead of watching for `403 POLICY_DENY` responses mid-flow. ## Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ Claude (AI Assistant) │ └─────────────────────────┬───────────────────────────────────┘ │ │ MCP Protocol (stdio) │ ┌─────────────────────────▼───────────────────────────────────┐ │ 60db MCP Server │ ├─────────────────────────────────────────────────────────────┤ │ Voice: sixtydb_tts_synthesize, sixtydb_stt_transcribe │ │ sixtydb_list_voices, sixtydb_create_voice │ │ Memory: sixtydb_memory_ingest, sixtydb_memory_search │ │ sixtydb_memory_upload_document │ │ sixtydb_memory_context, sixtydb_memory_get_usage │ │ Authz: sixtydb_get_permissions, sixtydb_check_permission│ │ Admin: sixtydb_list_workspaces, sixtydb_get_usage_stats │ │ sixtydb_list_invoices │ │ ... 30+ tools total │ └─────────────────────────┬───────────────────────────────────┘ │ │ HTTPS (JWT or API key) │ ┌─────────────────────────▼───────────────────────────────────┐ │ 60db API │ │ (api.60db.ai) │ │ │ │ Every request passes through: │ │ Cerbos policy check → Memory billing → Controller │ └─────────────────────────────────────────────────────────────┘ ``` ## Next Steps * [Installation](/mcp-server/installation) - Install and configure the server * [Configuration](/mcp-server/configuration) - Set up authentication * [Quick Start](/mcp-server/quickstart) - Run your first request * [Claude Desktop](/mcp-server/claude-desktop) - Integration guide ## Requirements * **Node.js**: 18 or higher * **API Key**: 60db API key from dashboard * **MCP Client**: Claude Desktop or custom MCP client ## Support * **Documentation**: [https://docs.60db.ai](https://docs.60db.ai) * **GitHub**: [https://github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) * **Issues**: [https://github.com/60db-ai/mcp/issues](https://github.com/60db-ai/mcp/issues) # Mcp inspector Source: https://docs.60db.ai/mcp-server/mcp-inspector # MCP Inspector Test and debug the 60db MCP Server with the MCP Inspector. ## Installation ```bash theme={null} npm install -g @modelcontextprotocol/inspector ``` ## Usage ### Start the Inspector ```bash theme={null} # From the server directory mcp-inspector node dist/index.js # With environment variables SIXTYDB_API_KEY=sk_your_key mcp-inspector node dist/index.js ``` The Inspector opens in your browser at `http://localhost:5173`. ## Features * **View all tools**: Browse 20+ available tools * **Execute tools**: Test tools with parameters * **View responses**: See formatted responses * **Debug logging**: View server logs in real-time ## Example Workflows ### List Voices 1. Click `60db_list_voices` 2. Enter: `{"language": "en-US", "limit": 5}` 3. Click "Execute" 4. View available voices ### Synthesize Speech 1. Click `60db_tts_synthesize` 2. Enter: `{"text": "Hello!", "voice_id": "voice_abc123"}` 3. Click "Execute" 4. Receive audio URL ### Transcribe Audio 1. Click `60db_stt_transcribe` 2. Enter: `{"audio_url": "https://example.com/audio.mp3"}` 3. Click "Execute" 4. View transcription ## Debugging ### Enable Debug Logging ```bash theme={null} SIXTYDB_DEBUG=true mcp-inspector node dist/index.js ``` ### View Server Logs The Inspector displays real-time logs: ``` 60db MCP Server starting... API URL: https://api.60db.ai Auth: API Key 60db MCP Server running via stdio ``` ## Best Practices 1. **Test before integrating**: Always test in Inspector first 2. **Use sample data**: Start with simple examples 3. **Monitor response times**: Check tool execution times 4. **Validate responses**: Verify response formats ## Next Steps * [Claude Desktop](/mcp-server/claude-desktop) - Integrate with Claude # Memory Source: https://docs.60db.ai/mcp-server/memory # Memory & RAG Tools The 60db MCP Server exposes the full Memory/RAG layer to MCP clients like Claude Desktop. These tools let an AI assistant persist user memories, ingest knowledge base documents (91+ formats with built-in OCR), run hybrid semantic recall, and assemble LLM-ready context for retrieval-augmented generation. <Info> All memory tools are **pay-as-you-go** from the workspace wallet. Every billable response surfaces the new balance, the charge amount, and the transaction ID in the formatted output so the agent can reason about cost. See the [pricing reference](/api-reference/memory/pricing) for rates and refund policy. </Info> ## Tool summary | Tool | Billed | Purpose | | ---------------------------------- | ------ | ----------------------------------------------------------- | | `sixtydb_memory_ingest` | ✓ | Store a single memory | | `sixtydb_memory_ingest_batch` | ✓ | Batch-store up to 100 memories | | `sixtydb_memory_upload_document` | ✓✓ | Extract + ingest a document (PDF, DOCX, XLSX, images, etc.) | | `sixtydb_memory_search` | ✓ | Hybrid semantic + keyword recall | | `sixtydb_memory_context` | ✓ | Assemble LLM-ready context for RAG | | `sixtydb_memory_list_collections` | — | List collections in the workspace | | `sixtydb_memory_create_collection` | — | Create a team/knowledge/hive collection | | `sixtydb_memory_get_usage` | — | Monthly spend breakdown + wallet balance | | `sixtydb_memory_get_status` | — | Poll a memory's ingestion state | | `sixtydb_memory_delete` | — | Soft-delete a memory (24h undo) | ## Ingest a memory Store a single fact, preference, or conversation snippet: ```json theme={null} { "tool": "sixtydb_memory_ingest", "arguments": { "text": "User prefers vegetarian food and is lactose intolerant.", "title": "Dietary preferences", "type": "user", "infer": true } } ``` **Parameters:** * `text` (required) — content to store, max 100,000 characters * `title` (optional) — display title * `collection` (optional) — target collection ID (defaults to personal) * `type` — `user`, `knowledge`, or `hive` (default: `user`) * `infer` — if true, the memory service extracts structured facts via LLM **Cost:** \$0.0001 per 1,000 characters. ## Upload a document The most powerful memory tool — give the agent a file path and 60db handles format detection, OCR, chunking, and ingestion: ```json theme={null} { "tool": "sixtydb_memory_upload_document", "arguments": { "file_path": "/Users/me/Documents/quarterly-report.pdf", "collection": "company_reports", "type": "knowledge", "title": "Q4 2026 Quarterly Report" } } ``` **Parameters:** * `file_path` (required) — absolute path on the agent's local filesystem * `collection` (optional) — target collection * `type` — `user` | `knowledge` | `hive` (default: `knowledge` — right choice for docs) * `title` (optional) — display title, defaults to filename * `chunk_size` (optional) — characters per chunk (200–8000, default 1500) * `chunk_overlap` (optional) — character overlap between chunks (default 200) **Supported formats:** PDF, DOCX, DOC, ODT, RTF, TXT, MD, HTML, EPUB, XLSX, XLS, CSV, ODS, PPTX, PPT, ODP, EML, MSG, PNG, JPG, TIFF, BMP, and 70+ more. OCR is applied automatically to scanned PDFs and images. **Max file size:** 200 MB. **Max chunks per document:** 100. **Cost (two-stage):** * Extract fee: \$0.003 per MB (pre-charged) * Ingest fee: \$0.0001 per 1,000 extracted characters (post-charged) Both fees are automatically refunded on any failure. ## Search memories Hybrid semantic + keyword recall with optional cross-encoder reranking, across user memories AND knowledge documents in one call: ```json theme={null} { "tool": "sixtydb_memory_search", "arguments": { "query": "What are my dietary preferences?", "mode": "thinking", "max_results": 10, "alpha": 0.8 } } ``` **Parameters:** * `query` (required) — search text, max 2,000 chars * `collection` (optional) — collection to search * `mode` — `fast` (dense retrieval, \~100-200ms) or `thinking` (wider pool + cross-encoder rerank, \~200-400ms) * `max_results` — 1–50, default 10 * `alpha` — 0 (keyword only) to 1 (semantic only), default 0.8 * `recency_bias` — weight for newer memories (0–1), default 0 * `graph_context` — include knowledge-graph relationships **Advanced reranker knobs** (optional, override server defaults): * `rerank_top_k` — max candidates the cross-encoder reranks (1-500) * `rerank_timeout_ms` — hard timeout for rerank call (50-5000ms) * `min_rerank_score` — drop results below this score (0-1) * `fetch_multiplier` — in thinking mode, fetch N x max\_results candidates (1-10) When the reranker is active, each result includes a `rerank_score` (cross-encoder confidence, 0-1) alongside the regular `score` (dense similarity). The rerank score is the more reliable ranking signal. **Tuning by query type:** | Query type | Recommended | | --------------------- | ------------------------------- | | Exact phrase match | `alpha: 0.2, mode: fast` | | Conceptual question | `alpha: 0.9, mode: fast` | | Complex multi-faceted | `alpha: 0.7, mode: thinking` | | Latest-events focus | `alpha: 0.6, recency_bias: 0.3` | **Cost:** flat \$0.0003 per query regardless of parameters. ## Assemble context (RAG) Purpose-built for retrieval-augmented generation. Returns a `prompt_ready` string you can prepend directly to an LLM system message: ```json theme={null} { "tool": "sixtydb_memory_context", "arguments": { "query": "What should I tell the customer about their order?", "session_id": "chat_abc123", "top_k": 8, "max_context_length": 2000 } } ``` **Graceful degradation** — if the memory layer is unreachable, the tool returns an empty `prompt_ready` string and the charge is automatically refunded so the chat flow keeps working. **Cost:** flat \$0.0005 per query. ## Collections List everything the caller can see: ```json theme={null} { "tool": "sixtydb_memory_list_collections", "arguments": {} } ``` Create a team/knowledge/hive collection (admin/owner only): ```json theme={null} { "tool": "sixtydb_memory_create_collection", "arguments": { "collection_id": "customer_support", "label": "Customer Support KB", "kind": "knowledge", "shared": true } } ``` Personal collections are auto-created per user on first use and cannot be created via this tool. **Both are unbilled.** ## Monitor spend Track usage and wallet balance without affecting the wallet: ```json theme={null} { "tool": "sixtydb_memory_get_usage", "arguments": { "period": "current_month" } } ``` Returns net spend, operation count, refund count, and a per-service-type breakdown, plus the billing owner's current wallet balance. Always free, works even when the wallet is empty. **Periods:** `current_month` (default), `last_30_days`, `all_time`. ## Handling insufficient credits When the wallet runs out, billable tools return an error with `INSUFFICIENT_CREDITS` structure. Agents should: 1. Catch the 402 case 2. Call `sixtydb_memory_get_usage` to show the shortfall to the user 3. Prompt the user to top up via the 60db dashboard (link: `/app/billing`) 4. Retry the original operation once the wallet is funded ## Billing transparency in every response Every billable tool response includes a **Billing** footer in the formatted output: ``` **Memory queued** in collection `user_abc123` Type: user Total queued: 1 - `mem_01HV...` — pending: Memory queued for processing _Billing: charged **$0.000013** · balance **$9.464587** · tx `84ffd09e`_ ``` The JSON response format also embeds a `billing: { balance, charged, txId }` object for programmatic consumers. ## Related * [Memory pricing & billing](/api-reference/memory/pricing) * [Get memory usage endpoint](/api-reference/memory/usage) * [Memory feature overview](/features/memory) * [Authz MCP tools](/mcp-server/authz) # Quickstart Source: https://docs.60db.ai/mcp-server/quickstart # Quick Start Get started with the 60db MCP Server in minutes. ## Step 1: Install the Server ```bash theme={null} npm install @60db ``` ## Step 2: Set Your API Key ```bash theme={null} export SIXTYDB_API_KEY=sk_your_api_key_here ``` ## Step 2: Start the Server ```bash theme={null} npm start ``` You should see: ``` 60db MCP Server starting... API URL: https://api.60db.ai Auth: API Key 60db MCP Server running via stdio ``` ## Step 3: Test with Claude Desktop Configure Claude Desktop and restart. Then ask: ``` Can you list available English female voices? ``` Claude will use the MCP server to fetch voices. ## Example Usage ### Text-to-Speech ``` You: Convert "Hello, world!" to speech using voice "Sarah" ``` ### Speech-to-Text ``` You: Transcribe this meeting audio: https://example.com/meeting.mp3 ``` ### Create Custom Voice ``` You: Create a custom voice named "Brand Voice" from this sample ``` ## Next Steps * [Voices](/mcp-server/voices) - Learn about voice tools * [TTS](/mcp-server/tts) - Text-to-speech guide * [STT](/mcp-server/stt) - Speech-to-text guide # Stt Source: https://docs.60db.ai/mcp-server/stt # Speech-to-Text (STT) Transcribe audio files to text with optional speaker diarization. ## Available Tools | Tool | Description | | ------------------------ | ---------------------------------- | | `sixtydb_stt_transcribe` | Transcribe audio to text | | `sixtydb_stt_logs` | View transcription history | | `sixtydb_stt_get` | Get specific transcription details | ## Transcribe Audio Convert audio to text via the `sixtydb_stt_transcribe` tool: ```json theme={null} { "audio_url": "https://example.com/meeting.mp3", "language": "auto", "diarize": true } ``` ### Auto-detect vs explicit language * **Omit `language`** (or pass `"auto"`) to enable auto-detection across all 39 supported languages. The MCP shim strips `"auto"` before forwarding so the server's language identification runs. * **Pass a single ISO 639-1 code** (e.g. `"hi"`, `"en"`, `"ar"`) to skip language identification and run the fast path for that language. * Do **not** pass unsupported codes (`ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`) or Arabic dialect tags (`ar-eg`, `ar-lv`, …) — they return an `unsupported_language` error. For non-MSA Arabic audio pass `"ar"` for best-effort MSA transcription. ## Parameters | Parameter | Type | Default | Description | | ----------------- | ------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `audio_url` | string | Required | URL to audio file (downloaded and forwarded as multipart to `POST /stt`; max 25 MB) | | `language` | string | *auto* | ISO 639-1 code, or `"auto"` / omit for auto-detect | | `diarize` | boolean | `false` | Enable pyannote speaker diarization — adds a `speakers` array to each segment with `SPEAKER_00`, `SPEAKER_01`, … labels | | `context` | string | — | Free-form paragraph describing the session (domain, speakers, jargon). When supplied, the server runs a background LLM refinement pass; response text is polished for proper nouns, filler removal, and punctuation. Omit to skip refinement. | | `response_format` | string | `"markdown"` | `"markdown"` or `"json"` | ### Context string example ``` Cricket coaching session. Players: Arjun Mehta, Ishaan Verma, Aryan Khan, Rohan. Discussing batting technique, stamina, running between wickets, off-side balls. ``` <Note> `context` is a **plain string** on the REST `POST /stt` tool (this page). The WebSocket `/ws/stt` endpoint takes a structured `{general, text, terms}` **object** instead — see the [WebSocket STT reference](/api-reference/websocket/stt). </Note> ## Response shape (JSON) ```json theme={null} { "request_id": "req_...", "text": "Hello, thanks for joining...", "language": "en", "language_name": "English", "duration_sec": 5.2, "segments": [ { "start": 0.0, "end": 3.1, "text": "Hello, thanks for joining the call today.", "confidence": 0.92, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 } ], "speakers": [ { "speaker": "SPEAKER_00", "start": 0.0, "end": 3.1 } ] } ], "warning_codes": [] } ``` **Empty-speech signal**: A successful response with `text: ""` and `warning_codes: ["no_speech_detected"]` means the audio contained no speech. This is not an error — do not retry. ## Usage in Claude ``` You: Transcribe this meeting audio You: Transcribe this with speaker identification You: Transcribe only in Hindi (use language "hi") ``` ## Supported Audio Formats * WAV, MP3, M4A, OGG, FLAC, WebM, MP4 audio track * Max file size: 25 MB * Max duration: 1 hour * Recommended: 16 kHz+ sample rate ## Supported Languages 39 languages total — fetch the live catalog from the `sixtydb_stt_languages` tool (or the REST `GET /stt/languages` endpoint). Includes 25 European languages, 13 Indic languages with English code-switching, and Arabic MSA. See the [`Get STT Languages`](/api-reference/stt/get-languages) reference for the full list. ## Related * [STT Models](/api-reference/models/get-stt-models) — exposed STT model catalog (currently `60db-stt-v01`) * [WebSocket STT](/api-reference/websocket/stt) — real-time streaming variant (note: the WS form uses `languages: null` for auto-detect, not `"auto"`) * [TTS](/mcp-server/tts) — Text-to-speech # Tts Source: https://docs.60db.ai/mcp-server/tts # Text-to-Speech (TTS) Generate natural speech from text using the MCP server. ## Available Tools | Tool | Description | | --------------------- | ------------------------------- | | `60db_tts_synthesize` | Convert text to speech | | `60db_tts_logs` | View TTS generation history | | `60db_tts_get` | Get specific generation details | ## Synthesize Speech Convert text to audio: ```typescript theme={null} { "text": "Hello, world!", "voice_id": "voice_abc123", "speed": 1.0, "output_format": "mp3" } ``` **Response**: ```markdown theme={null} # TTS Generation Complete **Audio URL**: https://cdn.60db.ai/audio/tts_123456.mp3 **Duration**: 2.3 seconds **Voice**: Sarah (en-US) ``` ## Parameters | Parameter | Type | Default | Description | | --------------- | ------ | -------- | ----------------------------- | | `text` | string | Required | Text to synthesize | | `voice_id` | string | Required | Voice to use | | `speed` | number | 1 | Speech speed (0.25-2.0) | | `stability` | number | 50 | Voice stability (0-100) | | `similarity` | number | 75 | Voice similarity (0-100) | | `output_format` | string | mp3 | Output format (mp3, wav, ogg) | ## Usage in Claude ``` You: Convert "Welcome to our podcast" to speech You: Generate speech with faster speed You: Use a Spanish voice for this text ``` ## Supported Languages * English (US, UK, Australian) * Spanish, French, German, Italian * Portuguese, Japanese, Korean, Chinese * And 30+ more languages ## Related * [Voices](/mcp-server/voices) - Browse available voices * [STT](/mcp-server/stt) - Speech-to-text # Voices Source: https://docs.60db.ai/mcp-server/voices # Voices Manage and use voices for text-to-speech synthesis through the MCP server. ## Available Tools | Tool | Description | | ------------------- | --------------------------------------- | | `60db_list_voices` | List available voices with filtering | | `60db_get_voice` | Get detailed voice information | | `60db_create_voice` | Create a cloned voice from audio sample | ## List Voices Browse available voices by language, gender, and type: ```typescript theme={null} // List English female voices { "language": "en-US", "gender": "Female", "limit": 10 } ``` **Response**: ```markdown theme={null} # Available Voices | Voice ID | Name | Language | Gender | |----------|------|----------|--------| | voice_abc123 | Sarah | en-US | Female | | voice_def456 | Emma | en-US | Female | ``` ## Get Voice Details Retrieve detailed information about a voice: ```typescript theme={null} { "voice_id": "voice_abc123" } ``` ## Create Custom Voice Create a cloned voice from an audio sample: ```typescript theme={null} { "name": "Brand Voice", "sample_url": "https://example.com/sample.mp3", "language": "en-US", "gender": "Female" } ``` <Callout type="info"> Voice cloning requires a 30+ second audio sample with clear speech. </Callout> ## Usage in Claude ``` You: List all available English voices You: Get details for voice "Sarah" You: Create a custom voice from this sample ``` ## Related * [TTS](/mcp-server/tts) - Use voices for speech synthesis # Workspaces Source: https://docs.60db.ai/mcp-server/workspaces # Workspaces Manage team workspaces and collaboration through the MCP server. ## Available Tools | Tool | Description | | ---------------------------- | ---------------------- | | `60db_list_workspaces` | List all workspaces | | `60db_get_workspace` | Get workspace details | | `60db_create_workspace` | Create a new workspace | | `60db_get_workspace_members` | List workspace members | ## List Workspaces View all workspaces: ```typescript theme={null} { "limit": 20 } ``` ## Create Workspace Create a new team workspace: ```typescript theme={null} { "name": "Engineering Team", "description": "Main engineering workspace" } ``` ## Get Workspace Members List members of a workspace: ```typescript theme={null} { "workspace_id": "workspace_abc123" } ``` ## Usage in Claude ``` You: List all my workspaces You: Create a new workspace for the design team You: Who are the members of the Engineering workspace? ``` ## Related * [Analytics](/mcp-server/analytics) - Track usage # Quickstart Source: https://docs.60db.ai/quickstart Start using 60db in under 5 minutes ## Get Your API Key First, you'll need an API key to authenticate your requests: <Steps> <Step title="Sign Up"> Create an account at [app.60db.ai](https://app.60db.ai) </Step> <Step title="Navigate to API Keys"> Go to Settings → API Keys in your dashboard </Step> <Step title="Create New Key"> Click "Create API Key" and give it a descriptive name </Step> <Step title="Copy Your Key"> Copy your API key and store it securely </Step> </Steps> <Warning> Keep your API key secret! Never commit it to version control or expose it in client-side code. </Warning> ## Choose Your SDK <Tabs> <Tab title="JavaScript/TypeScript"> ### Installation ```bash theme={null} npm install 60db ``` ### Basic Usage ```typescript theme={null} import { SixtyDBClient } from '60db'; // Initialize the client const client = new SixtyDBClient('your-api-key'); // Text to Speech const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice' }); // Get all voices const voices = await client.getVoices(); console.log(voices); ``` </Tab> <Tab title="Python"> ### Installation ```bash theme={null} pip install 60db ``` ### Basic Usage ```python theme={null} from sixtydb import SixtyDBClient # Initialize the client client = SixtyDBClient('your-api-key') # Text to Speech audio = client.text_to_speech('Hello, world!', voice_id='default-voice') # Save audio to file with open('output.mp3', 'wb') as f: f.write(audio) # Get all voices voices = client.get_voices() print(voices) ``` </Tab> </Tabs> ## Next Steps <CardGroup> <Card title="Explore Features" icon="compass" href="/features/text-to-speech"> Learn about all available features </Card> <Card title="API Reference" icon="book" href="/api-reference/introduction"> Dive into the complete API documentation </Card> <Card title="Custom Voices" icon="user-voice" href="/features/voices"> Create your own custom voice profiles </Card> <Card title="Memory & RAG" icon="brain" href="/features/memory"> Give your AI persistent memory and upload documents (PDF/DOCX/OCR) </Card> <Card title="LLM Chat" icon="comments" href="/features/llm-chat"> OpenAI-compatible chat completions with 60db models </Card> <Card title="Webhooks" icon="webhook" href="/platform/webhooks"> Set up webhooks for event notifications </Card> </CardGroup> # JavaScript/TypeScript SDK Source: https://docs.60db.ai/sdks/javascript Complete guide to the 60db JavaScript/TypeScript SDK ## Installation ```bash theme={null} npm install 60db ``` ## Initialization ```typescript theme={null} import { SixtyDBClient } from "60db"; // Simple initialization const client = new SixtyDBClient("your-api-key"); ``` ## Text-to-Speech ### Basic TTS ```typescript theme={null} const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice', enhance: true, speed: 1.0, language:"en-us" }); ### Get all voices const voices = await client.getVoices(); ### Get all lanuages const lanuages = await client.getLanguages(); ### audio is an ArrayBuffer containing the audio data ``` ## Speech-to-Text ### Transcribe Audio ```typescript theme={null} const file = document.querySelector('input[type="file"]').files[0]; const result = await client.speechToText(file, { language: "en", }); console.log(result.text); ``` ### With Context Refinement Supply a `context` string to enable server-side LLM polishing — proper nouns, filler removal, and punctuation are cleaned on the response text: ```typescript theme={null} const result = await client.speechToText(file, { language: "hi", diarize: true, context: "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.", }); ``` Omit `context` entirely to skip refinement. Note: this is the REST shape — the WebSocket streaming endpoint takes a `{general, text, terms}` object instead. ### Get Supported Languages ```typescript theme={null} const languages = await client.getLanguages(); console.log(languages); ``` ## Voice Management ### List All Voices ```typescript theme={null} const voices = await client.getVoices(); voices.forEach((voice) => { console.log(`${voice.name} (${voice.id})`); }); ``` ### Get Specific Voice ```typescript theme={null} const voice = await client.getVoice("voice-id"); console.log(voice); ``` ### Create Custom Voice ```typescript theme={null} const files = [file1, file2, file3]; // File objects const newVoice = await client.createVoice({ name: "My Custom Voice", files: files, description: "A custom voice for my brand", }); console.log("Created voice:", newVoice.id); ``` ### Update Voice ```typescript theme={null} await client.updateVoice("voice-id", { name: "Updated Voice Name", description: "Updated description", }); ``` ### Delete Voice ```typescript theme={null} await client.deleteVoice("voice-id"); ``` ## Authentication ### Sign Up ```typescript theme={null} const user = await client.signUp({ email: "user@example.com", password: "secure-password", name: "John Doe", }); ``` ### Sign In ```typescript theme={null} const session = await client.signIn({ email: "user@example.com", password: "secure-password", }); console.log("Token:", session.token); ``` ### Get Profile ```typescript theme={null} const profile = await client.getProfile(); console.log(profile); ``` ### Update Profile ```typescript theme={null} await client.updateProfile({ name: "Jane Doe", company: "Acme Inc", }); ``` ## Workspace Management ### List Workspaces ```typescript theme={null} const workspaces = await client.getWorkspaces(); ``` ### Create Workspace ```typescript theme={null} const workspace = await client.createWorkspace({ name: "My Workspace", description: "Team workspace", }); ``` ## Billing ### Get Available Plans ```typescript theme={null} const plans = await client.getPlans(); plans.forEach((plan) => { console.log(`${plan.name}: $${plan.price}/month`); }); ``` ### Get Current Plan ```typescript theme={null} const currentPlan = await client.getCurrentPlan(); console.log("Current plan:", currentPlan.name); ``` ### Subscribe to Plan ```typescript theme={null} await client.subscribe("plan-id"); ``` ## Analytics ### Get Usage Statistics ```typescript theme={null} const usage = await client.getUsage(); console.log("Characters used:", usage.characters); console.log("API calls:", usage.api_calls); ``` ## API Key Management ### List API Keys ```typescript theme={null} const apiKeys = await client.getApiKeys(); ``` ### Create API Key ```typescript theme={null} const newKey = await client.createApiKey("Production Key"); console.log("New API key:", newKey.key); ``` ### Delete API Key ```typescript theme={null} await client.deleteApiKey("key-id"); ``` ## Webhooks ### List Webhooks ```typescript theme={null} const webhooks = await client.getWebhooks(); ``` ### Create Webhook ```typescript theme={null} const webhook = await client.createWebhook({ url: "https://example.com/webhook", events: ["tts.completed", "stt.completed"], }); ``` ### Delete Webhook ```typescript theme={null} await client.deleteWebhook("webhook-id"); ``` ## Error Handling ```typescript theme={null} try { const audio = await client.textToSpeech({ text: "Hello, world!", voice_id: "invalid-voice", }); } catch (error) { if (error.response) { // API error console.error("API Error:", error.response.status); console.error("Message:", error.response.data.message); } else { // Network or other error console.error("Error:", error.message); } } ``` ## TypeScript Types The SDK includes full TypeScript type definitions: ```typescript theme={null} import { SixtyDBClient, SixtyDBConfig, TTSResponse } from "60db"; ``` # Python SDK Source: https://docs.60db.ai/sdks/python Complete guide to the 60db Python SDK ## Installation ```bash theme={null} pip install 60db ``` ## Initialization ```python theme={null} from sixtydb import SixtyDBClient # Simple initialization client = SixtyDBClient('your-api-key') # With custom configuration ### Get all voices voices = await client.getVoices(); ### Get all lanuages lanuages = await client.getLanguages(); ### audio is an ArrayBuffer containing the audio data ``` ## Text-to-Speech ### Basic TTS ```python theme={null} audio = client.text_to_speech( text='Hello, world!', voice_id='default-voice', enhance=True, speed=1.0, language='en-us' ) # Save to file with open('output.mp3', 'wb') as f: f.write(audio) ``` ## Speech-to-Text ### Transcribe Audio ```python theme={null} with open('audio.mp3', 'rb') as audio_file: result = client.speech_to_text(audio_file, language='en') print(result['text']) ``` ### With Context Refinement Supply a `context` string to enable server-side LLM polishing — proper nouns, filler removal, and punctuation are cleaned on the response text: ```python theme={null} with open('audio.mp3', 'rb') as audio_file: result = client.speech_to_text( audio_file, language='hi', diarize=True, context='Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.', ) print(result['text']) ``` Omit `context` entirely to skip refinement. Note: this is the REST shape — the WebSocket streaming endpoint takes a `{general, text, terms}` object instead. ### Get Supported Languages ```python theme={null} languages = client.get_languages() for lang in languages: print(f"{lang['name']} ({lang['code']})") ``` ## Voice Management ### List All Voices ```python theme={null} voices = client.get_voices() for voice in voices: print(f"{voice['name']} ({voice['id']})") ``` ### Get Specific Voice ```python theme={null} voice = client.get_voice('voice-id') print(voice) ``` ### Create Custom Voice ```python theme={null} files = [ open('sample1.mp3', 'rb'), open('sample2.mp3', 'rb'), open('sample3.mp3', 'rb') ] new_voice = client.create_voice( name='My Custom Voice', files=files, description='A custom voice for my brand' ) print(f"Created voice: {new_voice['id']}") # Close files for f in files: f.close() ``` ### Update Voice ```python theme={null} client.update_voice( voice_id='voice-id', name='Updated Voice Name', description='Updated description' ) ``` ### Delete Voice ```python theme={null} client.delete_voice('voice-id') ``` ## Authentication ### Sign Up ```python theme={null} user = client.sign_up( email='user@example.com', password='secure-password', name='John Doe' ) ``` ### Sign In ```python theme={null} session = client.sign_in( email='user@example.com', password='secure-password' ) print(f"Token: {session['token']}") ``` ### Get Profile ```python theme={null} profile = client.get_profile() print(profile) ``` ### Update Profile ```python theme={null} client.update_profile( name='Jane Doe', company='Acme Inc' ) ``` ## Workspace Management ### List Workspaces ```python theme={null} workspaces = client.get_workspaces() ``` ### Create Workspace ```python theme={null} workspace = client.create_workspace( name='My Workspace', description='Team workspace' ) ``` ## Billing ### Get Available Plans ```python theme={null} plans = client.get_plans() for plan in plans: print(f"{plan['name']}: ${plan['price']}/month") ``` ## Analytics ### Get Usage Statistics ```python theme={null} usage = client.get_usage() print(f"Characters used: {usage['characters']}") print(f"API calls: {usage['api_calls']}") ``` ## API Key Management ### List API Keys ```python theme={null} api_keys = client.get_api_keys() ``` ### Create API Key ```python theme={null} new_key = client.create_api_key('Production Key') print(f"New API key: {new_key['key']}") ``` ### Delete API Key ```python theme={null} client.delete_api_key('key-id') ``` ## Webhooks ### List Webhooks ```python theme={null} webhooks = client.get_webhooks() ``` ### Create Webhook ```python theme={null} webhook = client.create_webhook( url='https://example.com/webhook', events=['tts.completed', 'stt.completed'] ) ``` ### Delete Webhook ```python theme={null} client.delete_webhook('webhook-id') ``` ## Error Handling ```python theme={null} from requests.exceptions import HTTPError try: audio = client.text_to_speech( text='Hello, world!', voice_id='invalid-voice' ) except HTTPError as e: print(f"HTTP Error: {e.response.status_code}") print(f"Message: {e.response.json()['message']}") except Exception as e: print(f"Error: {str(e)}") ``` ## Type Hints The SDK includes type hints for better IDE support: ```python theme={null} from sixtydb import SixtyDBClient from typing import Dict, List, Any client: SixtyDBClient = SixtyDBClient('your-api-key') voices: List[Dict[str, Any]] = client.get_voices() ``` ## Async Support For async applications, you can use the SDK with asyncio: ```python theme={null} import asyncio from sixtydb import SixtyDBClient async def main(): client = SixtyDBClient('your-api-key') # Run in executor for async compatibility loop = asyncio.get_event_loop() audio = await loop.run_in_executor( None, client.text_to_speech, 'Hello, world!', 'default-voice' ) with open('output.mp3', 'wb') as f: f.write(audio) asyncio.run(main()) ``` # Codex Skills Source: https://docs.60db.ai/skills/introduction Drop 60db voice into Claude Code, OpenAI Codex, OpenCode, or any agent with a skills directory — triggered by /60db. # 60db Agent Skills Bring 60db's full voice stack — **text-to-speech, speech-to-text, voice cloning, voice management, and the LLM core** — directly into your coding agent. Install it once and call it with `/60db` from [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [OpenAI Codex](https://developers.openai.com/codex), [OpenCode](https://opencode.ai), or any agent that reads a skills directory. It's a single **zero-dependency** Python CLI (stdlib only, Python 3.8+) — no SDK, no `pip install`, no version drift. <CardGroup> <Card title="Claude Code" icon="comments"> Native skill — `/60db` and `/setup-60db` </Card> <Card title="OpenAI Codex" icon="robot"> Same skill folders, copied into `~/.codex/skills` </Card> <Card title="OpenCode & others" icon="terminal"> Plain markdown + stdlib Python runs anywhere </Card> </CardGroup> <Info> Source repository: [github.com/60db-ai/60db.ai-skills](https://github.com/60db-ai/60db.ai-skills) — Apache-2.0. </Info> *** ## Why a Skill, Not Just the SDK The skill **encodes the behavior that's actually true** against the live API, so you never relearn the gotchas. It's stdlib-only, so it keeps working regardless of SDK or doc drift. The five mistakes it pins for you: <AccordionGroup> <Accordion title="REST /tts-synthesize returns NDJSON, not one JSON blob"> Concatenate each line's `result.audioContent` (base64 LINEAR16 PCM). The CLI does this for you. </Accordion> <Accordion title="Only 8000 / 16000 / 24000 / 48000 Hz are accepted"> `44100` is rejected with a `400`. </Accordion> <Accordion title="The output_format flag is ignored"> The API always returns raw PCM — wrap it as WAV yourself (the CLI does). </Accordion> <Accordion title="Sample rate ≠ fidelity"> The model is band-limited at \~8 kHz, so 48 kHz is upsampled headroom. `enhance` and `60db-quality` do not raise the ceiling. </Accordion> <Accordion title="The voice id field is voice_id, not id"> Language and gender live under `labels` — confirmed by live inspection. </Accordion> </AccordionGroup> *** ## How It Works One CLI — `skills/60db/scripts/sixtydb.py` — fronts every 60db use case against `https://api.60db.ai`. ``` /60db 1. Setup once → user runs `init`: hidden key prompt → config (mode 600) 2. Ask if unclear → prompt for: use case · voice · model · kHz · format 3. Route → tts / stt / voices / clone / chat / langs / doctor 4. Verify → `doctor` reports key presence + resolved defaults (never the key) ``` Every setting resolves the same way, so you can override per-call or store defaults once: ``` CLI flag > env SIXTYDB_* > config file > built-in default ``` <Note> **Setup is the user's job.** The API key is never typed by the agent, never passed on the command line, never printed, never committed. The user runs `init` (a hidden prompt) or sets `SIXTYDB_API_KEY`. In a chat session where a hidden prompt isn't reachable, the agent scaffolds defaults with `init --no-key` and hands the key step back to the user. </Note> *** ## Use Cases What you can build with `/60db` without leaving your coding agent: <CardGroup> <Card title="Narration & voiceovers" icon="microphone"> Turn a script, blog post, or `.txt` file into studio-quality audio for videos, podcasts, or demos — `tts script.txt --voice <id> --out vo.wav`. </Card> <Card title="Transcription & subtitles" icon="closed-captioning"> Convert recordings, calls, or meetings into text with speaker labels and word timings, then build SRT/VTT — `stt rec.mp4 --diarize --timestamps --json`. </Card> <Card title="Voice agents & bots" icon="robot"> Wire `stt` (ears) + `chat` (brain) + `tts` (mouth) into a conversational loop for support, reception, or IVR-style flows. </Card> <Card title="Custom brand voices" icon="user-pen"> Clone a voice from a few samples and reuse it across all your audio — `clone --name "Brand VO" --sample a.wav --sample b.wav`. </Card> <Card title="Accessibility" icon="universal-access"> Add read-aloud audio to docs, articles, and apps for visually impaired or on-the-go users. </Card> <Card title="Multilingual content" icon="language"> Generate speech and transcribe audio across 30+ languages — check `langs` / `langs --stt` for coverage. </Card> </CardGroup> ### Who it's for <AccordionGroup> <Accordion title="Developers building voice features"> Add TTS/STT to an app without learning the SDK or fighting the docs — the skill drives one stdlib-only CLI that already pins the API's real behavior. </Accordion> <Accordion title="Creators & content teams"> Produce voiceovers and dubbed audio straight from a script file, with a consistent brand voice via cloning. </Accordion> <Accordion title="Support & operations teams"> Prototype phone/voice bots from the `stt + chat + tts` parts, and transcribe call recordings for QA or summaries. </Accordion> <Accordion title="Anyone in a coding agent"> If you live in Claude Code, Codex, or OpenCode, `/60db` lets you ship audio without context-switching to a dashboard or another tool. </Accordion> </AccordionGroup> ### A typical workflow <Steps> <Step title="Set up once"> Run `/setup-60db` to store your API key and defaults (voice, model, sample rate). </Step> <Step title="Pick a voice"> Run `voices` to list available voice IDs (add `--mine` for your cloned voices). </Step> <Step title="Generate or transcribe"> Call `tts` to make audio or `stt` to make text — the skill asks for anything it needs. </Step> <Step title="Iterate or chain"> Adjust `--speed` / `--stability`, or chain `stt → chat → tts` for a full voice-agent turn. </Step> </Steps> *** ## The Two Skills | Skill | Trigger | What it does | | -------------- | ------------- | -------------------------------------------------------------------------- | | **60db** | `/60db` | The worker — routes every voice use case to the CLI | | **setup-60db** | `/setup-60db` | First-run onboarding: collects the key privately, picks defaults, verifies | The worker drives one CLI with **9 subcommands**: | Subcommand | What it does | Status | | -------------- | ---------------------------------------------------------------------- | ------------------------ | | `init` | Hidden key prompt → config (mode 600); `--no-key` stores defaults only | ✅ | | `doctor` | Diagnose setup — key presence (never revealed), resolved defaults | ✅ | | `tts` | Text or `.txt` → WAV (REST NDJSON; `--ws` for legacy WebSocket) | ✅ verified | | `stt` | Audio → transcript (`--diarize`, `--timestamps`, `--json`) | ✅ verified | | `voices` | List voice ids (built-in + your cloned), `--mine`, `--json` | ✅ verified | | `clone` | Train a new voice from samples (`--sample` / `--sample-url`) | ⚠️ doc-only — test first | | `delete-voice` | Hard-delete one of your custom voices | ⚠️ doc-only | | `langs` | Supported languages (`--stt` for STT's 39) | ✅ | | `chat` | LLM core (`60db-tiny`) you pair with stt+tts for a voice agent | documented | *** ## Install <Steps> <Step title="Add the skill"> **Claude Code (recommended):** ```bash theme={null} npx skills add 60db-ai/60db.ai-skills ``` **Codex / OpenCode / other agents** — copy the two skill folders into that agent's skills directory: ```bash theme={null} git clone https://github.com/60db-ai/60db.ai-skills.git cp -r 60db.ai-skills/skills/60db 60db.ai-skills/skills/setup-60db ~/.codex/skills/ ``` </Step> <Step title="Restart your agent session"> Skills are only picked up on session start — this is an agent-platform limitation, not a bug. </Step> <Step title="Run setup once"> Run `/setup-60db`, or directly: ```bash theme={null} E=skills/60db/scripts/sixtydb.py python3 $E init # hidden prompt for your API key → config (mode 600) python3 $E doctor # verify (reports key presence without revealing it) ``` Get a key at **app.60db.ai → Settings → Developer → API Keys**. </Step> </Steps> <Warning> The API key is **yours to enter** — never paste it into the agent chat (chat history is retained by the platform). It's stored locally at `~/.config/60db/config.json` (mode 600) or read from `SIXTYDB_API_KEY`, and `.gitignore` keeps it out of git. </Warning> *** ## Run It ```bash theme={null} E=skills/60db/scripts/sixtydb.py python3 $E tts "Hello there." --out out/hello.wav # text → WAV (48 kHz, 60db-quality) python3 $E tts script.txt --voice <id> --out out/vo.wav python3 $E stt recording.mp3 --diarize --timestamps # audio → transcript python3 $E voices # list voice ids python3 $E langs --stt # 39 STT languages python3 $E chat "One-line summary of attachment theory." # LLM core for agents python3 $E doctor # diagnose setup ``` Store defaults once so you stop repeating flags: ```bash theme={null} python3 $E init --no-key --voice <id> --model 60db-quality --sample-rate 48000 ``` *** *** ## Quick Decision Guide | I want to... | Use | | ---------------------------------------- | ---------------------------------------------------------------------- | | Narration / voiceover from text | `tts "..." --out out/vo.wav` | | Read a script file aloud | `tts script.txt --voice <id> --out out/vo.wav` | | Transcribe audio with speaker labels | `stt rec.mp3 --diarize --timestamps` | | See which voices I can use | `voices` (then `voices --mine`) | | Use a specific language | `langs` / `langs --stt` | | Clone my own brand voice | `clone --name "Brand VO" --sample a.wav --sample b.wav` *(test first)* | | Build a phone/voice bot | `chat` as the brain + `stt` ears + `tts` mouth | | Figure out why audio sounds "compressed" | `doctor` → troubleshooting | | Set up for the first time | `/setup-60db` | *** ## Examples <CodeGroup> ```bash Voiceover from text theme={null} python3 $E tts "Welcome to the show." --voice <id> --out out/intro.wav # wrote out/intro.wav (412160 B PCM, ~4.3s @ 48000Hz, REST, model=60db-quality) ``` ```bash Narrate a script file theme={null} python3 $E tts episode.txt --voice <id> --out out/ep.wav --speed 0.95 --stability 65 ``` ```bash Transcribe with speakers + timings theme={null} python3 $E stt interview.m4a --diarize --timestamps --out interview.txt # [English, 42.7s] So the first thing we noticed was the latency... ``` ```bash Structured JSON for subtitles theme={null} python3 $E stt talk.mp4 --timestamps --confidence --json > talk.json # talk.json → words[] each with {word, start, end} → build SRT/VTT ``` ```bash One voice-agent turn: ears → brain → mouth theme={null} USER=$(python3 $E stt caller.wav --json | python3 -c 'import json,sys;print(json.load(sys.stdin)["text"])') REPLY=$(python3 $E chat "$USER" --system "Concise support agent." --chat-id call-42 | head -1) python3 $E tts "$REPLY" --voice <id> --out out/reply.wav ``` </CodeGroup> *** ## Gotchas That Bite Everyone | Symptom | Cause | Fix | | --------------------------------- | --------------------------------------------------- | ------------------------------------------------------------------ | | Audio sounds a touch "compressed" | Model is \~16 kHz-native, band-limited to \~8 kHz | Expected; `enhance` / `60db-quality` don't lift it | | Garbled / truncated audio | REST returns **NDJSON**, not one JSON blob | Concatenate every line's `result.audioContent` (the CLI does this) | | `400` on synthesize | Sample rate `44100` (or anything off-menu) | Use `8000 / 16000 / 24000 / 48000` only | | Format flag seems ignored | `output_format` **is** ignored — always raw PCM | Wrap the PCM as WAV yourself (the CLI does) | | Long script cut off | TTS caps at **5000 chars** per request | Split, synthesize, concatenate WAVs | | `voices` prints `None` for the id | Field is `voice_id`; language/gender under `labels` | The CLI already handles this | | `no API key found` | Key not in config or env | Run `init`, or export `SIXTYDB_API_KEY` | *** ## Coverage | Use case | Command | Status | | ---------------------------------- | -------------- | ---------------------------------- | | Text-to-speech (REST + WS) | `tts` | ✅ verified | | Speech-to-text (batch + streaming) | `stt` | ✅ endpoint verified | | List voices | `voices` | ✅ verified (`voice_id` / `labels`) | | Voice cloning | `clone` | ⚠️ doc-only — test first | | Update / delete voice | `delete-voice` | ⚠️ doc-only | | Languages | `langs` | ✅ | | LLM chat (voice-agent core) | `chat` | documented; no native calling API | *** ## How It Helps <CardGroup> <Card title="Zero setup friction" icon="bolt"> Stdlib-only Python — no SDK, no `requests`, no version drift. Only `python3` 3.8+ required. </Card> <Card title="Truth over docs" icon="shield-check"> Every endpoint is annotated verified vs. doc-only; live-API findings win over the published docs. </Card> <Card title="Voice agents from parts" icon="phone"> No native telephony API — build a voice agent from `stt` (ears) + `chat` (brain) + `tts` (mouth). </Card> <Card title="Secure by design" icon="lock"> The key is never typed by the agent, never printed, never committed. </Card> </CardGroup> *** ## FAQ <AccordionGroup> <Accordion title="How do I install it the fastest way?"> `npx skills add 60db-ai/60db.ai-skills`, then restart your agent and run `/setup-60db`. </Accordion> <Accordion title="Where does my API key go? Is it safe?"> Into a local config at `~/.config/60db/config.json` (mode 600), or the `SIXTYDB_API_KEY` env var. It's never passed on the command line, never printed, and `.gitignore` keeps it out of git. The agent never types it for you. </Accordion> <Accordion title="My voice sounds slightly compressed at 48 kHz. Bug?"> No. The model is \~16 kHz-native and band-limited to \~8 kHz, so 48 kHz is upsampled headroom. `enhance` and `60db-quality` don't raise the ceiling. </Accordion> <Accordion title="Which sample rates are allowed?"> `8000`, `16000`, `24000`, `48000` only. `44100` is rejected. </Accordion> <Accordion title="Can 60db make or receive phone calls?"> There is no native calling/telephony API. You build a voice agent from the parts: `stt` (ears) + `chat` (brain, `60db-tiny`) + `tts` (mouth). </Accordion> <Accordion title="Does it work outside Claude Code?"> Yes — it's markdown + stdlib Python. Copy the two skill folders into Codex's or OpenCode's skills directory. Only the onboarding prompt is Claude-specific and degrades gracefully. </Accordion> <Accordion title="Do I need the SDK or requests?"> No. Only `python3` 3.8+. The single optional dependency is `websockets`, used solely by the legacy `--ws` TTS path. </Accordion> </AccordionGroup> <Note> Full command reference, recipes, and the complete troubleshooting list live in the repo's [GUIDE.md](https://github.com/60db-ai/60db.ai-skills/blob/main/GUIDE.md) and [references/](https://github.com/60db-ai/60db.ai-skills/tree/main/skills/60db/references). </Note> # WebSocket API Source: https://docs.60db.ai/websocket-api Real-time Speech-to-Text (STT) and Text-to-Speech (TTS) streaming API # WebSocket API Real-time bidirectional streaming API for Speech-to-Text (STT) and Text-to-Speech (TTS) services. ## Overview The 60db WebSocket API provides real-time streaming capabilities for: * **Speech-to-Text (STT)**: Convert audio to text with 99+ language support * **Text-to-Speech (TTS)**: Synthesize natural-sounding speech with multiple voices ## Base URLs | Environment | STT URL | TTS URL | | ------------------- | -------------------------- | -------------------------- | | Production | `ws://api.60db.ai/ws/stt` | `ws://api.60db.ai/ws/tts` | | Production (Secure) | `wss://api.60db.ai/ws/stt` | `wss://api.60db.ai/ws/tts` | ## Authentication WebSocket connections require authentication using API key via query parameter: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key_here ``` **Getting Your API Key:** 1. Go to [app.60db.ai](https://app.60db.ai) 2. Navigate to **Settings → Developer → API Keys** 3. Click **Create API Key** 4. Copy and store your API key securely *** ## 🎤 STT WebSocket (Speech-to-Text) **What it does:** You send audio → You get text back **Use for:** Transcribing speech, voice commands, call center analytics, meeting transcription ### Quick Start ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`wss://api.60db.ai/ws/stt?apiKey=${API_KEY}`); ws.onopen = () => console.log('✅ Connected'); ws.onmessage = (data) => { const msg = JSON.parse(data); // STT tags this frame with `type` and keeps the fields top-level // (TTS nests them under `connection_established` — the shapes differ). if (msg.type === 'connection_established') { console.log('✅ Authenticated!'); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true } })); } // Wait for `session_started`, not `connected` — `connected` fires before // the session exists and arrives twice (proxy frame, then upstream frame). if (msg.type === 'session_started') { console.log('✅ Ready! Send audio now'); // Send audio chunks const interval = setInterval(() => { ws.send(audioBuffer); // Your audio data }, 60); // Stop after 5 seconds setTimeout(() => { clearInterval(interval); ws.send(JSON.stringify({ type: 'stop' })); }, 5000); } if (msg.type === 'transcription') { console.log('📝 Text:', msg.text); } if (msg.type === 'session_stopped') { console.log('✅ Done! Cost:', msg.billing_summary.total_cost); ws.close(); } }; ``` ### How STT Works ``` 1. Connect → 2. Authenticate → 3. Start session → 4. Send audio → 5. Get text → 6. Stop ``` *** ## 🔊 TTS WebSocket (Text-to-Speech) **What it does:** You send text → You get audio back **Use for:** Voice assistants, audiobooks, accessibility, chatbots ### Quick Start ```javascript theme={null} const WebSocket = require('ws'); const fs = require('fs'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`wss://api.60db.ai/ws/tts?apiKey=${API_KEY}`); const contextId = 'my-session-' + Date.now(); const audioChunks = []; ws.onopen = () => console.log('✅ Connected'); ws.onmessage = (data) => { const msg = JSON.parse(data); if (msg.connection_established) { console.log('✅ Authenticated!'); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 } } })); } if (msg.context_created) { console.log('✅ Context created!'); // Send text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, this is a test of the text to speech service.' } })); // Flush ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log('🔊 Audio chunk received'); } if (msg.flush_completed) { console.log('✅ All audio received!'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } if (msg.context_closed) { console.log('✅ Done!'); // Save audio const audio = Buffer.concat(audioChunks); fs.writeFileSync('output.pcm', audio); console.log('💾 Saved output.pcm'); ws.close(); } }; ``` ### How TTS Works ``` 1. Connect → 2. Authenticate → 3. Create context → 4. Send text → 5. Flush → 6. Get audio → 7. Close ``` *** ## 📋 STT vs TTS Comparison | Feature | STT (Speech-to-Text) | TTS (Text-to-Speech) | | ----------------- | ------------------------ | --------------------------- | | **Input** | Audio (binary) | Text (string) | | **Output** | Text (string) | Audio (binary) | | **Use Case** | Transcribe speech | Generate speech | | **Direction** | Audio → Text | Text → Audio | | **First Message** | `{ type: "start", ... }` | `{ create_context: {...} }` | | **Session End** | `{ type: "stop" }` | `{ close_context: {...} }` | | **Pricing** | \$0.00000833/second | \$0.00002/character | *** ## 📚 Full Documentation * 🎤 [STT WebSocket](/websocket-api/stt) - Complete STT guide with all parameters * 🔊 [TTS WebSocket](/websocket-api/tts) - Complete TTS guide with all parameters *** ## 💡 Key Concepts ### STT Messages ```javascript theme={null} // Start session { type: "start", languages: ["en"], config: {...} } // Stop session { type: "stop" } ``` ### TTS Messages ```javascript theme={null} // Create context { create_context: { context_id, voice_id, audio_config } } // Send text { send_text: { context_id, text: "..." } } // Get audio { flush_context: { context_id } } // Close session { close_context: { context_id } } ``` *** ## 📊 Pricing | Service | Rate | Minimum | | ------- | ------------------- | ------- | | STT | \$0.00000833/second | \$0.01 | | TTS | \$0.00002/character | \$0.01 | *** ## 🆘 Support * **Email**: [support@60db.ai](mailto:support@60db.ai) * **Documentation**: [https://docs.60db.ai](https://docs.60db.ai) * **Status**: [https://status.60db.ai](https://status.60db.ai) # STT WebSocket Source: https://docs.60db.ai/websocket-api/stt WebSocket /ws/stt Real-time Speech-to-Text WebSocket API for streaming audio transcription # STT WebSocket API Real-time Speech-to-Text transcription via WebSocket streaming with support for 39 languages (including code-switched Indic+English) and telephony integration. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack). ## 🚀 Quick Start (Copy & Paste) ```javascript theme={null} const WebSocket = require('ws'); // 1. Your API key const API_KEY = 'sk_live_your_api_key'; // 2. Connect const ws = new WebSocket(`wss://api.60db.ai/ws/stt?apiKey=${API_KEY}`); // 3. Handle messages ws.on('message', (data) => { const msg = JSON.parse(data); // Authenticated? Start session! // NOTE: on STT the fields are top-level and the frame is tagged by `type`. if (msg.type === 'connection_established') { console.log('✅ Authenticated'); ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true } })); } // Session ready? Send audio! // Gate on `session_started`, not `connected` — `connected` fires before the // session exists and arrives twice (proxy frame, then upstream frame). if (msg.type === 'session_started') { console.log('✅ Ready! Send audio now'); // Send dummy audio (480 bytes every 60ms) let count = 0; const interval = setInterval(() => { ws.send(Buffer.alloc(480, 0xff)); if (++count >= 83) { // 5 seconds clearInterval(interval); ws.send(JSON.stringify({ type: 'stop' })); } }, 60); } // Got text! if (msg.type === 'transcription' && msg.is_final) { console.log('📝', msg.text); } // Done! if (msg.type === 'session_stopped') { console.log('✅ Complete! Cost:', msg.billing_summary.total_cost); ws.close(); } }); ``` **That's it!** You'll see: * ✅ Authenticated * ✅ Ready! Send audio now * 📝 Hello world (transcribed text) * ✅ Complete! Cost: \$0.000043 *** ## 📖 How It Works (5 Simple Steps) 1. **Connect** with your API key 2. **Send** `{ type: "start", ... }` to begin session 3. **Stream** audio data (binary chunks) 4. **Receive** text transcriptions in real-time 5. **Stop** with `{ type: "stop" }` when done *** ## Endpoint <ParamField name="url" type="string"> `ws://api.60db.ai/ws/stt` or `wss://api.60db.ai/ws/stt` </ParamField> ## Authentication Query parameter authentication: <ParamField name="apiKey" type="string"> Your API key for authentication </ParamField> Example: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key ``` ## Connection Details | Property | Value | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Protocol | WebSocket (RFC 6455) | | Frame types | Binary (telephony) or Text/JSON (browser) | | Ping/keepalive | The server answers client-initiated WebSocket pings with a pong; it does not ping first. Send a ping every 20–30 s so intermediaries do not idle the socket out. | ## Session Lifecycle ``` Client Server │ │ │──── TCP/TLS connect ─────────►│ │◄─── {"type":"connecting"} ────│ authenticating... │◄─── {"type":"connected"} ─────│ proxy wired to upstream │◄─── connection_established ──│ authenticated — safe to send `start` │ │ │──── {"type":"start", ...} ───►│ │◄─── {"type":"session_started"}│ safe to send audio │ │ │──── audio frames / messages ─►│ │◄─── {"type":"speech_started"} │ VAD detected voice │◄─── {"type":"transcription"} │ is_final=true, speech_final=false (first emit, context only) │◄─── {"type":"transcription"} │ is_final=true, speech_final=true (canonical answer) │ │ │──── {"type":"stop"} ─────────►│ │◄─── {"type":"session_stopped"}│ ``` **Two-phase finals (context-gated LLM refinement).** When you supply a `context` object on `start`, every utterance produces **two** `transcription` events sharing a `sentence_id`: 1. **First emit** — `is_final: true, speech_final: false` — fast dict-corrected text. Use for low-latency UI paint and barge-in. 2. **Canonical** — `is_final: true, speech_final: true` — definitive LLM-refined answer. Always arrives. When `context` is **omitted**, every utterance produces a single `transcription` event with `is_final: true, speech_final: true` (no first emit). Simple consumers can gate exclusively on `speech_final: true` and ignore the rest — that gives them exactly one canonical event per utterance regardless of whether refinement is on. ## Client → Server Messages ### `start` — Begin session Sent once after connection is established. Must be sent before any audio. <RequestExample> ```json Request theme={null} { "type": "start", "languages": ["en", "hi"], "context": { "general": [ { "key": "domain", "value": "Healthcare" }, { "key": "doctor", "value": "Dr. Martha Smith" } ], "text": "Routine diabetes follow-up consultation.", "terms": ["Celebrex", "Zyrtec", "Metformin", "HbA1c"] }, "config": { "encoding": "mulaw", "sample_rate": 8000, "utterance_end_ms": 1000, "continuous_mode": true, "interim_results_frequency": 300, "audio_enhancement": "adaptive", "diarize": false, "remove_fillers": false } } ``` </RequestExample> **Parameters:** <ParamField name="languages" type="array"> Array of ISO 639-1 codes from the supported set (see `GET /stt/languages`), e.g. `["en", "hi"]`. Max 5 per session. Omit or send `null` to auto-detect across all 39 v1 languages. Arabic dialect tags (`ar-eg`, …) are rejected. Unsupported: `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`. **Important:** never send `languages: "auto"` or `["auto"]`. The auto entry in `GET /stt/languages` is a convenience for the REST `/stt` form-upload flow; on WebSocket you must use `null` instead. Sending the literal string `"auto"` returns `language 'auto' is not in the v1 supported`. The 60db `/ws/stt` proxy strips it automatically as a safety net, but your client should send `null` directly. </ParamField> <ParamField name="context" type="object"> Optional hint object `{general, text, terms}` that opens the server-side LLM refinement gate. When supplied, each utterance emits **two** `transcription` events sharing a `sentence_id` — a fast first emit (`speech_final: false`) with dict-corrected text, followed \~300–700 ms later by a canonical emit (`speech_final: true`) with LLM-refined text (proper nouns corrected, fillers removed, punctuation added, script consistency enforced). See [Canonical-answer semantics](#canonical-answer-semantics-speech-final) below. * `general` — array of `{key, value}` pairs. Free-form metadata (`domain`, `topic`, speaker names) surfaced to the LLM verbatim as hint lines. * `text` — background paragraph describing the session. Useful for narrative context. * `terms` — array of proper nouns, acronyms, and domain-specific jargon to preserve in the transcript. All three fields are optional; at least one should be populated. Omit `context` entirely to disable refinement for the session — each utterance then arrives as a single `speech_final: true` event (no first emit). </ParamField> <ParamField name="config.encoding" type="string"> Audio encoding format. Use `"mulaw"` for telephony/Twilio, `"linear"` (Int16 PCM) for browser capture. Options: `"mulaw"`, `"linear"`. The first raw binary frame auto-selects `mulaw`. </ParamField> <ParamField name="config.sample_rate" type="integer"> Actual sample rate of the audio being sent — 8000 for telephony, 48000 for typical browser capture. Server resamples to 16 kHz internally. Options: `8000`, `16000`, `24000`, `44100`, `48000`. Must match the real capture rate; a mismatch produces garbled audio with no error. </ParamField> <ParamField name="config.utterance_end_ms" type="integer"> Silence duration (ms) after last speech chunk before finalizing the utterance. **Minimum 1000 ms** — the upstream server rejects anything lower, so the 60db proxy clamps sub-1000 values up to 1000 ms (on `start` and on mid-session `config`) rather than letting the session die. Sending `500` is accepted but behaves as `1000`. Recommended 1000–1500 ms for voicebots; for barge-in, react to `speech_started` and interim results instead of lowering this. Range: `≥ 1000` </ParamField> <ParamField name="config.continuous_mode" type="boolean"> Keep session alive between utterances instead of stopping after the first transcription. Required for voicebot / phone call use cases. Send `false` for single-shot transcription. </ParamField> <ParamField name="config.interim_results_frequency" type="integer"> How often (ms) to emit interim (partial) transcription results during speech. Use 300ms for barge-in, 500ms otherwise. Disabled by default. Range: `≥ 300` </ParamField> <ParamField name="config.diarize" type="boolean"> Run pyannote speaker diarization on each finalized utterance and attach a `speakers` array. Requires `HF_TOKEN` on the server. Adds \~50–150 ms latency per final. </ParamField> <ParamField name="config.min_speakers" type="integer | null"> Lower bound on diarization speaker count. Only read when `diarize=true`. </ParamField> <ParamField name="config.max_speakers" type="integer | null"> Upper bound on diarization speaker count. Only read when `diarize=true`. </ParamField> <ParamField name="config.audio_enhancement" type="string"> Real-time audio enhancement to improve transcription quality on noisy input. There are **3 options** available: | Value | Description | | ------------ | -------------------------------------------------------------------------------------------------- | | `"off"` | No audio processing (default) | | `"light"` | Noise reduction only — best for mildly noisy environments | | `"adaptive"` | Automatic noise reduction + gain control based on input levels — best for variable/telephony audio | Use `"adaptive"` for telephony or noisy environments, `"light"` when you only need mild cleanup, and `"off"` when the input is already clean. </ParamField> <ParamField name="config.remove_fillers" type="boolean"> Ask the LLM refinement pass to strip filler words (`um`, `uh`, `like`, `you know`, …) from the canonical transcript. Only takes effect when `context` is set — refinement is gated on context, and the raw first-emit still contains the fillers. Non-boolean values are coerced to `false` by the 60db proxy. </ParamField> <ParamField name="config.no_speech_threshold" type="float"> Reserved for legacy client compatibility. **Ignored by the 60db STT backend** — the non-hallucinating backends don't emit a `no_speech_prob`. </ParamField> ### `audio` — JSON audio chunk (browser mode) <RequestExample> ```json Request theme={null} { "type": "audio", "audio": "<base64-encoded Int16 PCM or μ-law bytes>", "encoding": "linear", "sample_rate": 48000, "timestamp": 1700000000000 } ``` </RequestExample> **Fields:** <ParamField name="type" type="string"> Must be `"audio"` </ParamField> <ParamField name="audio" type="string"> Base64-encoded audio bytes (Int16 PCM or μ-law) </ParamField> <ParamField name="encoding" type="string"> `"linear"` or `"mulaw"` </ParamField> <ParamField name="sample_rate" type="integer"> Actual sample rate of the audio </ParamField> <ParamField name="timestamp" type="integer"> Unix ms timestamp — useful for latency measurement </ParamField> ### Binary frame — raw μ-law audio (telephony mode) Send a raw WebSocket binary frame with μ-law bytes, no JSON wrapper. The server auto-detects this as telephony mode on the first binary frame. ``` Recommended chunk size: 480 bytes = 60ms at 8kHz Twilio default: 160 bytes = 20ms — batch 3 chunks into 60ms before sending ``` ### `config` — Change language mid-session <RequestExample> ```json Request theme={null} { "type": "config", "languages": ["hi"], "continuous_mode": true } ``` </RequestExample> Both `languages` and `continuous_mode` are optional; include only fields you want to change. Send `"languages": null` to revert to auto-detect. ### `stop` — End session <RequestExample> ```json Request theme={null} { "type": "stop" } ``` </RequestExample> Server processes any remaining audio buffer, sends `session_stopped`, then closes. ### `test` — Ping / latency check <RequestExample> ```json Request theme={null} { "type": "test", "message": "ping", "timestamp": 1700000000000 } ``` </RequestExample> Server echoes `test_response` with the same `timestamp` for round-trip measurement. ## Server → Client Messages ### `connecting` — Authentication in progress <ResponseExample> ```json Response theme={null} { "type": "connecting", "message": "Authenticating...", "timestamp": 1775465918269 } ``` </ResponseExample> ### `connection_established` — Authentication successful <ResponseExample> ```json Response theme={null} { "type": "connection_established", "service": "stt", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } ``` </ResponseExample> <Warning> On STT these fields are **top-level** and the frame is tagged by `type`. Check `msg.type === "connection_established"` — `msg.connection_established` is the TTS frame's shape, and a client written against it will never start its STT session. </Warning> **Fields:** <ResponseField name="service" type="string"> Service name: `"stt"` </ResponseField> <ResponseField name="user_id" type="integer"> Your user ID </ResponseField> <ResponseField name="credit_balance" type="number"> Available credits </ResponseField> <ResponseField name="workspace" type="string"> Workspace name </ResponseField> ### `connected` — After `start` message is processed <ResponseExample> ```json Response theme={null} { "type": "connected", "server_info": { "server_type": "60db STT", "device": "cuda", "model": "60db-stt-v01", "processing_mode": "sentence_based_modular", "supported_languages": { "en": "English", "hi": "Hindi" }, "total_languages": 40, "features": { "sentence_based_processing": true, "real_time_streaming": true, "telephony_support": true, "unicode_support": true, "mixed_language_support": true } } } ``` </ResponseExample> <Note> Expect **two** `connected` frames: the proxy's own handshake (`session_id` plus a small `server_info`), then the upstream capability frame shown above. Neither means "audio is accepted" — wait for `session_started`. Keep any `connected` handler idempotent so audio capture does not start twice. </Note> ### `session_started` — `start` accepted, audio is now allowed <ResponseExample> ```json Response theme={null} { "type": "session_started", "session_id": "sess_8c3d1a9f4b7e2c51", "language": "Multi-language: EN, HI", "languages": ["en", "hi"], "model": "60db-stt-v01", "processing_mode": "sentence_based_continuous", "continuous_mode": true, "interim_frequency": 300, "diarize": false, "llm_refinement": true } ``` </ResponseExample> Send audio only after this event — earlier frames are answered with `unknown message type: audio`. `llm_refinement: true` confirms your `context` opened the refinement gate, so every utterance will arrive as a two-phase pair; `false` means one canonical event per utterance. `languages` is the resolved candidate list after normalization. ### `speech_started` — VAD detected voice activity <ResponseExample> ```json Response theme={null} { "type": "speech_started", "timestamp": 1700000000.123 } ``` </ResponseExample> Use this for barge-in: interrupt TTS playback when this arrives. Fired after 2 consecutive VAD-positive chunks (\~64ms of confirmed speech). ### `transcription` — Transcription result All results (interim and final) share the same `transcription` type — differentiate with flags. **Final result** (`is_final=true`, `speech_final=true`): <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "Hello, how are you?", "confidence": 0.87, "language": "en", "language_name": "EN", "is_final": true, "speech_final": true, "is_partial": false, "sentence_id": 3, "processing_mode": "sentence_complete", "duration": 1.82, "latency": 0.43, "timestamp": 1700000000.456, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 }, { "word": "how", "start": 0.35, "end": 0.52, "confidence": 0.92 } ], "utterance_end_ms": 1820 } ``` </ResponseExample> **Empty speech\_final signal** (`text=""`, `is_final=true`, `speech_final=true`): Sent when audio was detected but transcription was rejected (silence, hallucination, low confidence, wrong language). Client should reset its state on this message and not treat it as an error. <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "", "confidence": 0.0, "is_final": true, "speech_final": true, "processing_mode": "speech_end_no_result", "timestamp": 1700000000.789 } ``` </ResponseExample> **Interim result** (`is_final=false`, `speech_final=false`) — only sent when `interim_results_frequency` is set: <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "Hello how", "confidence": 0.72, "language": "en", "is_final": false, "speech_final": false, "is_partial": true } ``` </ResponseExample> Use interims only for barge-in word-count checks. Never send interim text to the LLM — a final with `is_final=true, speech_final=true` will follow. **Response Fields:** <ResponseField name="text" type="string"> Transcribed text. Empty string = speech-end-no-result signal. </ResponseField> <ResponseField name="confidence" type="number"> 0.0–1.0. Telephony typically 0.35–0.75; browser 0.55–0.95. </ResponseField> <ResponseField name="language" type="string"> Detected language code e.g. `"en"`. </ResponseField> <ResponseField name="language_name" type="string"> Uppercase language code e.g. `"EN"`. </ResponseField> <ResponseField name="is_final" type="boolean"> `true` = end of speech reached. May still be followed by a canonical upgrade if LLM refinement is active. </ResponseField> <ResponseField name="speech_final" type="boolean"> `true` = canonical answer, will not be revised. When LLM refinement is on, one `is_final: true, speech_final: false` event is followed by one `is_final: true, speech_final: true`. When refinement is off, every final is `speech_final: true`. See [Canonical-answer semantics](#canonical-answer-semantics-speech-final). </ResponseField> <ResponseField name="is_partial" type="boolean"> `true` for interim results only. </ResponseField> <ResponseField name="sentence_id" type="integer"> Monotonically increasing counter per session. </ResponseField> <ResponseField name="duration" type="number"> Duration (seconds) of the audio segment transcribed. </ResponseField> <ResponseField name="latency" type="number"> Seconds from processing start to result ready (excludes queue time). </ResponseField> <ResponseField name="words" type="array"> Word-level timestamps `[{word, start, end, confidence}]`. Note: the field is **`confidence`**, not `probability`. Present on finals; empty on interims. </ResponseField> <ResponseField name="utterance_end_ms" type="integer"> Timestamp (ms) of last word in the utterance. </ResponseField> <ResponseField name="processing_mode" type="string"> Marker for utterances the consumer should skip. Omitted on ordinary finals. **None of these are billed.** | Value | Meaning | | ------------------------ | ------------------------------------------------------------------------- | | `speech_end_no_result` | Speech ended but the recognizer produced no text. | | `speech_end_too_short` | Utterance below the minimum duration to recognize. | | `hallucination_rejected` | Word-rate guard rejected the output as a likely hallucination. | | `low_snr_dropped` | Audio dropped before language ID / ASR — signal-to-noise below the floor. | </ResponseField> <ResponseField name="llm_applied" type="boolean"> On canonical emits where refinement ran: `true` = `text` is the LLM-refined version, `false` = the LLM was skipped or failed and the fast first-emit text was promoted unchanged. `llm_reason` carries the why, `llm_latency_ms` the round-trip time. The canonical always arrives either way. </ResponseField> <ResponseField name="tentative" type="boolean"> Added by the 60db proxy when the upstream rejected the utterance as a suspected hallucination but interim text was available — the proxy sends that text instead of an empty final, with `tentative_reason: "hallucination_rejected"`. Safe to route on; flag it for review. </ResponseField> ### Canonical-answer semantics: `speech_final` `is_final` and `speech_final` are **NOT identical** when LLM refinement is active — they split into two distinct meanings: | `is_final` | `speech_final` | Meaning | | ---------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `false` | `false` | Interim partial — text may still change as more audio arrives. | | `true` | `false` | End-of-speech reached, dict-corrected text. **The LLM is still processing.** A follow-up event with `speech_final: true` will arrive shortly with the canonical text. Only emitted when refinement is active for this utterance. | | `true` | `true` | **Canonical answer.** Definitive, will not be revised. LLM-refined text when context was supplied, otherwise the original ASR text. | The same `sentence_id` is echoed across both phases so clients can reconcile. **Canonical event example (after LLM refinement):** <ResponseExample> ```json Response theme={null} { "type": "transcription", "sentence_id": 3, "text": "डॉक्टर साहब, मेरा sugar level बहुत high है। Metformin की dose बढ़ाओ।", "confidence": 0.87, "language": "hi", "language_name": "HI", "is_final": true, "speech_final": true, "is_partial": false, "duration": 1.82, "words": [ { "word": "डॉक्टर", "start": 0.0, "end": 0.32, "confidence": 0.94 } ], "speakers": null, "timestamp": 1700000000.789 } ``` </ResponseExample> **Guarantees:** * **Exactly one canonical event per utterance.** When refinement is on, you get two `transcription` events per utterance (first emit + canonical). When refinement is off, you get one (`speech_final: true`). Never zero, never three. * **Same `sentence_id`** across both phases. Reconcile on that key. * **The canonical always arrives.** Consumers waiting on `speech_final: true` never hang. * **`sentence_id` ordering is preserved per session**, but canonicals are NOT guaranteed to arrive in `sentence_id` order when LLM is on — two utterances finalizing close in time may complete refinement out of order. Key on `sentence_id`, not arrival order. * **`words[]` corresponds to the original ASR output** on both phases — the LLM does not realign tokens. Use `words[]` for word-level timing, `text` for display. **Recommended client patterns:** *Simplest — don't care about the first-emit optimization:* ```js theme={null} function onMessage(msg) { if (msg.type === 'transcription' && msg.speech_final) { // Canonical — render and forget render(msg); } // Ignore is_final && !speech_final (intermediate, will be replaced) // Ignore is_partial (interim, handle separately if needed) } ``` *With fast first-emit (UX-aware):* ```js theme={null} const livePainted = new Map(); // sentence_id → line slot function onMessage(msg) { if (msg.type !== 'transcription') return; const sid = msg.sentence_id; if (msg.is_partial) { renderPartial(msg); return; } let entry = livePainted.get(sid); if (!entry) { entry = createLine(); livePainted.set(sid, entry); } entry.text = msg.text; entry.pending = msg.is_final && !msg.speech_final; // dim while LLM runs render(entry); if (msg.speech_final) { finalize(entry); livePainted.delete(sid); } } ``` **For voicebot NLU routing**: feed the first-emit text (`speech_final: false`) to NLU immediately for fast intent dispatch — don't wait for canonical. If your NLU benefits from proper-noun accuracy (name-spelling slots, drug-name lookup), run a second-pass call on the canonical (`speech_final: true`) text and reconcile on `sentence_id`. <Note> **Legacy `refined` event.** Earlier builds emitted a separate `refined` event \~400 ms after the final instead of a second `transcription`. The 60db `/ws/stt` proxy transparently handles both shapes — if you're still seeing `refined` events in the wire trace, upstream workers haven't been restarted onto the two-phase build yet. New client code should target the two-phase flow only; `refined` is accepted but deprecated. </Note> ### `language_changed` — After `config` message changes language <ResponseExample> ```json Response theme={null} { "type": "language_changed", "language": "Multi-language: HI", "language_code": ["hi"] } ``` </ResponseExample> ### `mode_changed` — After `config` message changes `continuous_mode` <ResponseExample> ```json Response theme={null} { "type": "mode_changed", "continuous_mode": true, "mode_name": "continuous", "silence_threshold": 0.5 } ``` </ResponseExample> ### `session_stopped` — After `stop` is processed <ResponseExample> ```json Response theme={null} { "type": "session_stopped", "billing_summary": { "total_duration_seconds": 5.2, "total_cost": 0.000043, "characters_transcribed": 42, "client_estimated_seconds": 5.24 } } ``` </ResponseExample> Only canonical finals (`is_final: true` **and** `speech_final: true`) with a `duration` are charged — first emits are previews, so turning on LLM refinement does not double your bill, and the four skip modes above cost nothing. `client_estimated_seconds` is a diagnostic estimate of what the client sent; never show it as a billed figure. ### `error` — Processing error <ResponseExample> ```json Response theme={null} { "type": "error", "error": "Audio processing error: ...", "timestamp": 1700000000.0 } ``` </ResponseExample> ### `test_response` — Reply to `test` ping <ResponseExample> ```json Response theme={null} { "type": "test_response", "message": "pong - qlabs-stt-proxy ready", "timestamp": 1700000000000, "processing_mode": "proxy" } ``` Answered by the 60db proxy itself — it never reaches the STT server, so a reply confirms the proxy hop only, and the echoed `timestamp` measures round-trip to the proxy. </ResponseExample> ## Complete Example <Tabs> <TabItem label="JavaScript (Node.js)"> ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/stt?apiKey=${API_KEY}`); ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (data) => { const msg = JSON.parse(data); console.log('←', msg.type || Object.keys(msg)[0]); if (msg.type === 'connection_established') { console.log(' User ID:', msg.user_id); console.log(' Credits:', msg.credit_balance); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en', 'hi'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true, utterance_end_ms: 1000, interim_results_frequency: 300 } })); } else if (msg.type === 'session_started') { console.log('✓ Session started! Send audio now...'); // Send audio chunks (480 bytes = ~60ms at 8kHz) const audioInterval = setInterval(() => { const audioChunk = getAudioChunk(); ws.send(audioChunk); }, 60); // Stop after 5 seconds setTimeout(() => { clearInterval(audioInterval); ws.send(JSON.stringify({ type: 'stop' })); }, 5000); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected - barge-in opportunity'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓ Final:', msg.text, `(confidence: ${msg.confidence})`); } else { console.log(' Partial:', msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); console.log(' Duration:', msg.billing_summary.total_duration_seconds, 's'); console.log(' Cost: $', msg.billing_summary.total_cost); ws.close(); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` </TabItem> <TabItem label="Python"> ```python theme={null} import asyncio import json import websockets async def stt_websocket(): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/stt?apiKey={API_KEY}" async with websockets.connect(url) as ws: # Wait for authentication: `connecting`, then `connected`, then this while True: msg = json.loads(await ws.recv()) if msg.get("type") == "connection_established": print(f"✓ Connected (User: {msg['user_id']})") print(f" Credits: ${msg['credit_balance']}") break # Start session await ws.send(json.dumps({ "type": "start", "languages": ["en", "hi"], "config": { "encoding": "mulaw", "sample_rate": 8000, "continuous_mode": True, "utterance_end_ms": 1000, "interim_results_frequency": 300 } })) # Audio is accepted only once the session is started while True: msg = json.loads(await ws.recv()) if msg.get("type") == "session_started": break print("✓ Session started!") # Send audio for 5 seconds for _ in range(83): # ~5000ms / 60ms audio_chunk = get_audio_chunk() await ws.send(audio_chunk) # Send as binary await asyncio.sleep(0.06) # Stop session await ws.send(json.dumps({"type": "stop"})) # Process remaining messages while True: msg = json.loads(await ws.recv()) msg_type = msg.get("type") if msg_type == "speech_started": print("🎤 Speech detected") elif msg_type == "transcription": if msg.get("is_final"): print(f"✓ {msg['text']} (confidence: {msg['confidence']})") else: print(f" {msg['text']}...") elif msg_type == "session_stopped": print(f"✓ Session stopped") print(f" Duration: {msg['billing_summary']['total_duration_seconds']}s") print(f" Cost: ${msg['billing_summary']['total_cost']}") break asyncio.run(stt_websocket()) ``` </TabItem> <TabItem label="Browser"> ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key'); let mediaRecorder; let audioContext; ws.onopen = () => { console.log('✓ Connected'); }; ws.onmessage = (event) => { const msg = JSON.parse(event.data); if (msg.type === 'connection_established') { console.log('✓ Authenticated!'); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'linear', sample_rate: 48000, continuous_mode: true, interim_results_frequency: 300 } })); } else if (msg.type === 'session_started') { console.log('✓ Session started! Start speaking...'); startAudioCapture(); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓', msg.text); updateTranscriptDisplay(msg.text); } else { console.log('...', msg.text); updateInterimDisplay(msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); stopAudioCapture(); ws.close(); } }; async function startAudioCapture() { const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); audioContext = new AudioContext({ sampleRate: 48000 }); const source = audioContext.createMediaStreamSource(stream); const processor = audioContext.createScriptProcessor(4096, 1, 1); processor.onaudioprocess = (e) => { const inputData = e.inputBuffer.getChannelData(0); const pcm16 = new Int16Array(inputData.length); for (let i = 0; i < inputData.length; i++) { pcm16[i] = Math.max(-32768, Math.min(32767, inputData[i] * 32768)); } ws.send(JSON.stringify({ type: 'audio', audio: btoa(String.fromCharCode(...new Uint8Array(pcm16.buffer))), encoding: 'linear', sample_rate: 48000 })); }; source.connect(processor); processor.connect(audioContext.destination); mediaRecorder = { stream, processor, source }; } function stopAudioCapture() { if (mediaRecorder) { mediaRecorder.stream.getTracks().forEach(track => track.stop()); mediaRecorder.processor.disconnect(); audioContext.close(); } } ``` </TabItem> </Tabs> ## Audio Requirements | Property | Telephony (μ-law) | Browser (PCM) | | ----------- | ----------------- | ----------------------------- | | Encoding | `mulaw` (8-bit) | `linear` (16-bit) | | Sample Rate | 8000 Hz | 16000, 24000, 44100, 48000 Hz | | Chunk Size | 480 bytes (60ms) | 960-1920 bytes (60-120ms) | | Channels | Mono (1 channel) | Mono (1 channel) | ## Supported Languages 39 languages total, backed by Parakeet-TDT (25 European), Vaani-FastConformer (13 Indic + Hinglish), and FC-Arabic (MSA). Fetch the full catalog from `GET /stt/languages`. **Not supported** (explicit rejection, no silent aliasing): `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`. Arabic dialect tags (`ar-eg`, `ar-lv`, …) return `dialect_not_supported` — pass `ar` for best-effort MSA. Common languages: | Code | Language | Code | Language | | ---- | --------- | ---- | -------- | | `en` | English | `hi` | Hindi | | `bn` | Bengali | `es` | Spanish | | `fr` | French | `de` | German | | `gu` | Gujarati | `ta` | Tamil | | `te` | Telugu | `kn` | Kannada | | `ml` | Malayalam | `mr` | Marathi | | `pa` | Punjabi | `ar` | Arabic | ## Pricing * **Rate**: \$0.00000833 per second * **Minimum**: \$0.01 per session * **Billing**: Per second of audio processed ## Related * [TTS WebSocket](/websocket-api/tts) - Text-to-Speech endpoint * [WebSocket Quick Start](/websocket-api/quickstart) - Get started guide * [WebSocket Playground](/websocket-playground) - Test in browser # TTS WebSocket Source: https://docs.60db.ai/websocket-api/tts WebSocket /ws/tts Real-time Text-to-Speech WebSocket API for streaming audio synthesis # TTS WebSocket API Real-time Text-to-Speech synthesis via WebSocket streaming with full-duplex bidirectional communication. ## 🚀 Quick Start (Copy & Paste) ```javascript theme={null} const WebSocket = require('ws'); const fs = require('fs'); // 1. Your API key const API_KEY = 'sk_live_your_api_key'; // 2. Connect const ws = new WebSocket(`wss://api.60db.ai/ws/tts?apiKey=${API_KEY}`); // 3. Unique context ID const contextId = 'my-session-' + Date.now(); const audioChunks = []; // 4. Handle messages ws.on('message', (data) => { const msg = JSON.parse(data); // Authenticated? Create context! if (msg.connection_established) { console.log('✅ Authenticated'); ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 } } })); } // Context ready? Send text! if (msg.context_created) { console.log('✅ Context created'); // Send your text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, world!' } })); // Flush to get audio ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } // Got audio? Save it! if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log('🔊 Audio chunk:', audioData.length, 'bytes'); } // All audio received? if (msg.flush_completed) { console.log('✅ All audio received!'); console.log(' Total:', audioChunks.length, 'chunks'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } // Done! Save audio if (msg.context_closed) { console.log('✅ Complete!'); // Save to file const completeAudio = Buffer.concat(audioChunks); fs.writeFileSync('output.pcm', completeAudio); console.log('💾 Saved: output.pcm'); console.log(' Size:', completeAudio.length, 'bytes'); ws.close(); } }); ``` **That's it!** You'll see: * ✅ Authenticated * ✅ Context created * 🔊 Audio chunk: 1024 bytes (multiple times) * ✅ All audio received! * ✅ Complete! * 💾 Saved: output.pcm *** ## 📖 How It Works (5 Simple Steps) 1. **Connect** with your API key 2. **Create** a context with voice settings 3. **Send** your text message 4. **Flush** to trigger synthesis 5. **Close** when done (receive audio file) *** ## Endpoint <ParamField name="url" type="string"> `ws://api.60db.ai/ws/tts` </ParamField> ## Authentication Query parameter authentication: <ParamField name="apiKey" type="string"> Your API key for authentication </ParamField> Example: ``` ws://api.60db.ai/ws/tts?apiKey=sk_live_your_api_key ``` ## Protocol Overview ``` Client Server | | |─── create_context ──────────────────▶ | |◀── context_created ───────────────── | | | |─── send_text ───────────────────────▶ | |─── flush_context ───────────────────▶ | |◀── audio_chunk #1 ────────────────── | |◀── audio_chunk #N ────────────────── | |◀── flush_completed ───────────────── | | | |─── close_context ───────────────────▶ | |◀── context_closed ────────────────── | ``` ## Connection Sequence ### 1. Connect ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); ``` ### 2. Receive Authentication Message <ResponseExample> ```json Response theme={null} { "connecting": true, "message": "Authenticating...", "timestamp": 1775465918269 } ``` </ResponseExample> ### 3. Receive Connection Established <ResponseExample> ```json Response theme={null} { "connection_established": { "service": "tts", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } } ``` </ResponseExample> **Fields:** <ResponseField name="service" type="string"> Service name: `"tts"` </ResponseField> <ResponseField name="user_id" type="integer"> Your user ID </ResponseField> <ResponseField name="credit_balance" type="number"> Available credits </ResponseField> <ResponseField name="workspace" type="string"> Workspace name </ResponseField> ## Client → Server Messages ### 1. create\_context **Must be the first message.** Initializes the TTS session with voice and audio settings. <RequestExample> ```json Request theme={null} { "create_context": { "context_id": "my-session-123", "voice_id": "7911a3e8", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } } ``` </RequestExample> **Parameters:** <ParamField name="context_id" type="string"> Unique session identifier. Default: auto-generated UUID </ParamField> <ParamField name="voice_id" type="string"> Voice ID to use for synthesis </ParamField> <ParamField name="audio_config.audio_encoding" type="string"> Audio encoding. Options: `LINEAR16`, `PCM`, `MULAW`, `ULAW`, `OGG_OPUS` </ParamField> <ParamField name="audio_config.sample_rate_hertz" type="integer"> Sample rate in Hz. Options: `8000`, `16000`, `24000`, `48000` </ParamField> <ParamField name="speed" type="number"> Speech speed multiplier (0.5 – 2.0). </ParamField> <ParamField name="stability" type="number"> Voice stability (0-100). Lower = more expressive, higher = more consistent. </ParamField> <ParamField name="similarity" type="number"> Voice similarity (0-100). How closely the output matches the source voice. </ParamField> **Supported encoding + sample rate combinations:** | `audio_encoding` | Supported `sample_rate_hertz` | Output format | | ---------------- | ------------------------------------------- | ------------------------------------------ | | `LINEAR16` | `8000`, `16000` (default), `24000`, `48000` | Raw PCM, 16-bit signed little-endian, mono | | `PCM` | `8000`, `16000` (default), `24000`, `48000` | Same as LINEAR16 | | `MULAW` | `8000` | G.711 μ-law encoded, mono | | `ULAW` | `8000` | Same as MULAW | | `OGG_OPUS` | `24000` | Ogg Opus compressed audio | > **Note:** `MULAW`/`ULAW` only works at `8000` Hz. `OGG_OPUS` only works at `24000` Hz. **Limits:** | Parameter | Min | Max | Default | | ------------------------- | ------ | ------------ | ------- | | `speed` | 0.5 | 2.0 | 1 | | `stability` | 0 | 100 | 50 | | `similarity` | 0 | 100 | 75 | | `text` (per send\_text) | 1 char | — | — | | text buffer (accumulated) | — | 50,000 chars | — | ### 2. send\_text Append text to the internal buffer. Text is accumulated until a `flush_context` or `close_context` is received. <RequestExample> ```json Request theme={null} { "send_text": { "context_id": "my-session-123", "text": "Hello, how are you doing today?" } } ``` </RequestExample> **Fields:** <ParamField name="context_id" type="string"> Session identifier </ParamField> <ParamField name="text" type="string"> Text to append to buffer (max cumulative 50,000 characters) </ParamField> You can send multiple `send_text` messages to build up text incrementally (e.g., from an LLM token stream): ```json theme={null} {"send_text": {"context_id": "ctx-1", "text": "Hello, "}} {"send_text": {"context_id": "ctx-1", "text": "how are you "}} {"send_text": {"context_id": "ctx-1", "text": "doing today?"}} ``` ### 3. flush\_context Triggers synthesis of all accumulated text. The server responds with `audio_chunk` messages followed by `flush_completed`. <RequestExample> ```json Request theme={null} { "flush_context": { "context_id": "my-session-123" } } ``` </RequestExample> ### 4. close\_context Flushes any remaining text, sends final audio, and closes the WebSocket connection. <RequestExample> ```json Request theme={null} { "close_context": { "context_id": "my-session-123" } } ``` </RequestExample> ## Server → Client Messages ### context\_created Confirms the session was initialized successfully. <ResponseExample> ```json Response theme={null} { "context_created": { "context_id": "my-session-123" } } ``` </ResponseExample> ### audio\_chunk Contains a chunk of synthesized audio. Multiple chunks are sent per flush. <ResponseExample> ```json Response theme={null} { "audio_chunk": { "context_id": "my-session-123", "audioContent": "SGVsbG8gd29ybGQ..." } } ``` </ResponseExample> **Fields:** <ResponseField name="context_id" type="string"> Session identifier </ResponseField> <ResponseField name="audioContent" type="string"> Base64-encoded audio bytes </ResponseField> The audio encoding and chunk format depend on `audio_config`: | Encoding | Chunk format | Notes | | ------------------ | ------------------------------- | ---------------------------------------------------------------------------------- | | `LINEAR16` / `PCM` | Raw PCM, 16-bit signed LE, mono | Chunks can be concatenated directly | | `MULAW` / `ULAW` | G.711 μ-law, 8-bit, mono | Chunks can be concatenated directly | | `OGG_OPUS` | Independent Ogg Opus files | Each chunk is a self-contained OGG file. **Chunks cannot be naively concatenated** | ### flush\_completed Signals that all audio for the flushed text has been sent. <ResponseExample> ```json Response theme={null} { "flush_completed": { "context_id": "my-session-123" } } ``` </ResponseExample> ### context\_closed Confirms the session is closed. The WebSocket connection closes after this message. <ResponseExample> ```json Response theme={null} { "context_closed": { "context_id": "my-session-123" } } ``` </ResponseExample> ### error Sent if synthesis fails or a protocol violation occurs. <ResponseExample> ```json Response theme={null} { "error": { "context_id": "my-session-123", "message": "voice_id required" } } ``` </ResponseExample> **Common errors:** | Message | Cause | | -------------------------------------------- | ------------------------------------------ | | `voice_id required` | `create_context` sent without `voice_id` | | `text_buffer exceeded 50000 character limit` | Too much text accumulated without flushing | | `Unsupported audio_encoding: X` | Invalid encoding value | | `Unsupported sample_rate_hertz: X` | Invalid sample rate | ## Complete Example <Tabs> <TabItem label="JavaScript (Node.js)"> ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/tts?apiKey=${API_KEY}`); const contextId = 'test-' + Date.now(); const audioChunks = []; ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (data) => { const msg = JSON.parse(data); const msgType = Object.keys(msg)[0]; console.log('←', msgType); if (msg.connection_established) { console.log(' Credits:', msg.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (msg.context_created) { console.log('✓ Context created!'); // Send text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you doing today?' } })); // Flush ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log(' Received audio chunk:', audioData.length, 'bytes'); } else if (msg.flush_completed) { console.log('✓ Flush completed!'); console.log(' Total audio size:', audioChunks.reduce((sum, chunk) => sum + chunk.length, 0), 'bytes'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } else if (msg.context_closed) { console.log('✓ Context closed'); // Save audio const audio = Buffer.concat(audioChunks); require('fs').writeFileSync('output.pcm', audio); console.log('Saved output.pcm'); ws.close(); } if (msg.error) { console.error('TTS Error:', msg.error.message); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` </TabItem> <TabItem label="Python"> ```python theme={null} import asyncio import json import base64 import websockets async def tts_websocket(): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/tts?apiKey={API_KEY}" context_id = f"session-{int(asyncio.get_event_loop().time())}" async with websockets.connect(url) as ws: # Wait for connection resp = json.loads(await ws.recv()) if resp.get('connection_established'): print(f"✓ Connected (Credits: ${resp['connection_established']['credit_balance']})") # Create context await ws.send(json.dumps({ "create_context": { "context_id": context_id, "voice_id": "fbb75ed2-975a-40c7-9e06-38e30524a9a1", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } })) # Wait for context_created resp = json.loads(await ws.recv()) assert "context_created" in resp print("✓ Context created!") # Send text + flush await ws.send(json.dumps({ "send_text": { "context_id": context_id, "text": "Hello, how are you doing today?" } })) await ws.send(json.dumps({ "flush_context": {"context_id": context_id} })) # Receive audio chunks until flush_completed audio_data = b"" chunk_count = 0 while True: msg = json.loads(await ws.recv()) if "audio_chunk" in msg: audio_data += base64.b64decode(msg["audio_chunk"]["audioContent"]) chunk_count += 1 print(f" Received chunk #{chunk_count}") elif "flush_completed" in msg: print("✓ Flush completed!") break elif "error" in msg: print(f"Error: {msg['error']['message']}") break # Close context await ws.send(json.dumps({ "close_context": {"context_id": context_id} })) resp = json.loads(await ws.recv()) assert "context_closed" in resp # Save audio with open("output.pcm", "wb") as f: f.write(audio_data) print(f"✓ Saved {len(audio_data)} bytes to output.pcm") asyncio.run(tts_websocket()) ``` </TabItem> <TabItem label="Browser"> ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); const contextId = 'test-' + Date.now(); const audioChunks = []; let audioCtx; ws.onopen = () => { console.log('✓ Connected'); }; ws.onmessage = (event) => { const data = JSON.parse(event.data); const msgType = Object.keys(data)[0]; console.log('←', msgType); if (data.connection_established) { console.log(' Credits:', data.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (data.context_created) { console.log('✓ Context created!'); // Send text + flush ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you?' } })); ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (data.audio_chunk) { // Collect base64 audio const binary = atob(data.audio_chunk.audioContent); const bytes = new Uint8Array(binary.length); for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); audioChunks.push(bytes); console.log(' Received audio chunk'); } else if (data.flush_completed) { console.log('✓ Flush completed!'); // Play collected audio playPCM16(audioChunks, 16000); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } if (data.error) { console.error('TTS Error:', data.error.message); } if (data.context_closed) { console.log('✓ Context closed'); } }; function playPCM16(chunks, sampleRate) { const totalBytes = chunks.reduce((s, c) => s + c.length, 0); const merged = new Uint8Array(totalBytes); let offset = 0; for (const chunk of chunks) { merged.set(chunk, offset); offset += chunk.length; } const int16 = new Int16Array(merged.buffer); const float32 = new Float32Array(int16.length); for (let i = 0; i < int16.length; i++) float32[i] = int16[i] / 32768; audioCtx = audioCtx || new AudioContext({ sampleRate }); const buf = audioCtx.createBuffer(1, float32.length, sampleRate); buf.getChannelData(0).set(float32); const src = audioCtx.createBufferSource(); src.buffer = buf; src.connect(audioCtx.destination); src.start(); } ``` </TabItem> </Tabs> ## Audio Configuration | Encoding | Sample Rates | Description | | ---------- | ------------------------- | ----------------------- | | `LINEAR16` | 8000, 16000, 24000, 48000 | PCM 16-bit signed | | `MULAW` | 8000 | G.711 μ-law (telephony) | | `OGG_OPUS` | 24000 | Compressed audio | For telephony integration (Twilio, etc.), use `MULAW` at `8000` Hz. ## Default Voice The default voice ID is: ``` fbb75ed2-975a-40c7-9e06-38e30524a9a1 ``` To get more voices, use the [Voices API](/api-reference/voices/get-voices). ## Context Management ### Reuse Context Keep a context open for multiple syntheses: ```javascript theme={null} // Create once ws.send(JSON.stringify({ create_context: { context_id, voice_id, audio_config } })); // Send multiple texts ws.send(JSON.stringify({ send_text: { context_id, text: "Hello" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); ws.send(JSON.stringify({ send_text: { context_id, text: "World" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); // Close when done ws.send(JSON.stringify({ close_context: { context_id } })); ``` ### Multiple Contexts You can create multiple contexts in one connection: ```javascript theme={null} const context1 = 'ctx-1'; const context2 = 'ctx-2'; // Create both contexts ws.send(JSON.stringify({ create_context: { context_id: context1, voice_id: voice1, audio_config } })); ws.send(JSON.stringify({ create_context: { context_id: context2, voice_id: voice2, audio_config } })); ``` ## Supported Languages The TTS model supports synthesis in multiple Indic languages and English. The language is auto-detected from the input text. | Language | ID | | --------- | -- | | English | en | | Hindi | hi | | Bengali | bn | | Gujarati | gu | | Kannada | kn | | Malayalam | ml | | Marathi | mr | | Punjabi | pa | | Tamil | ta | | Telugu | te | ## Pricing * **Rate**: \$0.00002 per character * **Minimum**: \$0.01 per context * **Billing**: Per character synthesized ## Related * [STT WebSocket](/websocket-api/stt) - Speech-to-Text endpoint * [WebSocket Quick Start](/websocket-api/quickstart) - Get started guide * [WebSocket Playground](/websocket-playground) - Test in browser * [Voices API](/api-reference/voices/get-voices)

# Get Usage Source: https://docs.60db.ai/api-reference/analytics/get-usage GET /analytics/usage Get usage statistics and analytics for all services ## Request ### Headers Bearer token with your API key ### Query Parameters Start date (ISO 8601 format, e.g. `2026-04-01`) End date (ISO 8601 format, e.g. `2026-04-13`) Aggregation period: `day`, `week`, `month`, `year` ## Response Returns usage data for all services — TTS, STT, LLM, and Memory — aggregated by period. Time-series data with per-period breakdowns Date string for the period TTS characters used in this period STT minutes used in this period TTS + STT cost in this period LLM tokens consumed in this period LLM cost in USD for this period Memory operations (chars, bytes, queries) in this period Memory cost in USD for this period Aggregated totals across all periods Total cost across all services (TTS + STT + LLM + Memory) ```bash cURL theme={null} curl "https://api.60db.ai/analytics/usage?date_from=2026-04-01&date_to=2026-04-13" \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const usage = await client.getUsage({ date_from: '2026-04-01', date_to: '2026-04-13' }); ``` ```python Python theme={null} usage = client.get_usage( date_from="2026-04-01", date_to="2026-04-13" ) ``` ```json Response theme={null} { "success": true, "usage_by_period": [ { "period": "2026-04-10", "tts_characters": 5000, "stt_minutes": 12.5, "cost_usd": 0.15, "llm_tokens": 8500, "llm_cost_usd": 0.017, "memory_units": 3200, "memory_cost_usd": 0.0003 } ], "summary": { "tts_characters": 125000, "stt_minutes": 180, "llm_tokens": 45000, "llm_cost_usd": 0.09, "memory_units": 15000, "memory_cost_usd": 0.0015, "total_cost_usd": 3.59, "plan": "Starter", "limits": { "tts_characters": 30000, "stt_minutes": 120 } } } ``` # Create API Key Source: https://docs.60db.ai/api-reference/api-keys/create-api-key POST /developer/api Create a new API key ## Request ### Headers Bearer token with your API key application/json ### Body Name for the API key ## Response API key ID API key name The actual API key (only shown once) Creation timestamp The API key is only shown once. Store it securely! ```bash cURL theme={null} curl -X POST https://api.60db.ai/developer/api \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "name": "Production Key" }' ``` ```javascript JavaScript theme={null} const newKey = await client.createApiKey("Production Key"); console.log("New API key:", newKey.key); ``` ```python Python theme={null} new_key = client.create_api_key('Production Key') print(f"New API key: {new_key['key']}") ``` ```json Response theme={null} { "id": "key-456", "name": "Production Key", "key": "sk_live_1234567890abcdefghijklmnopqrstuvwxyz", "created_at": "2026-01-29T11:35:00Z" } ``` # Delete API Key Source: https://docs.60db.ai/api-reference/api-keys/delete-api-key DELETE /developer/api/:id Delete an API key ## Request ### Path Parameters The ID of the API key to delete ### Headers Bearer token with your API key This action cannot be undone. Any applications using this key will lose access immediately. ## Response Indicates successful deletion Confirmation message ```bash cURL theme={null} curl -X DELETE https://api.60db.ai/developer/api/key-123 \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} await client.deleteApiKey("key-123"); ``` ```python Python theme={null} client.delete_api_key('key-123') ``` ```json Response theme={null} { "success": true, "message": "API key deleted successfully" } ``` # Get API Keys Source: https://docs.60db.ai/api-reference/api-keys/get-api-keys GET /developer/api List all API keys ## Request ### Headers Bearer token with your API key ## Response Array of API key objects API key ID API key name API key (masked) Creation timestamp Last used timestamp ```bash cURL theme={null} curl https://api.60db.ai/developer/api \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const apiKeys = await client.getApiKeys(); ``` ```python Python theme={null} api_keys = client.get_api_keys() ``` ```json Response theme={null} { "api_keys": [ { "id": "key-123", "name": "Production Key", "key": "sk_live_**********************abc", "created_at": "2026-01-15T10:00:00Z", "last_used": "2026-01-29T11:30:00Z" } ] } ``` # Payment Transactions Source: https://docs.60db.ai/api-reference/billing/transactions GET /billing/transactions List payment transactions (subscriptions, wallet top-ups) ## Request ### Headers Bearer token (JWT or API key) ### Query Parameters Page number Items per page (max 100) Filter: `one_time`, `subscription`, `admin_bonus` Filter: `succeeded`, `pending`, `failed` ## Response Returns payment history (wallet top-ups, subscription payments). For per-operation service usage (TTS/STT/LLM/Memory deductions), use [`GET /billing/usage-logs`](/api-reference/billing/usage-logs) instead. Only the **workspace owner** can view transactions. Protected by `billing:manage` permission. ```bash cURL theme={null} curl "https://api.60db.ai/billing/transactions?limit=10" \ -H "Authorization: Bearer your-api-key" ``` ```json Response theme={null} { "success": true, "data": { "transactions": [ { "hash_id": "abc123", "amount": 50.00, "currency": "USD", "payment_type": "one_time", "status": "succeeded", "balance_before": 0.50, "balance_after": 50.50, "description": "Workspace wallet top-up", "created_at": "2026-04-13T10:00:00.000Z" } ], "pagination": { "current_page": 1, "per_page": 10, "total_records": 5, "total_pages": 1 } } } ``` # Usage Logs Source: https://docs.60db.ai/api-reference/billing/usage-logs GET /billing/usage-logs View per-operation deduction history for all services (TTS, STT, LLM, Memory) ## Request ### Headers Bearer token (JWT or API key) ### Query Parameters Page number for pagination Items per page (max 100) Filter by service type. Options: `TTS`, `STT`, `LLM`, `MEMORY_INGEST`, `MEMORY_EXTRACT`, `MEMORY_RECALL`, `MEMORY_CONTEXT` ## Response Array of transaction log entries Unique transaction ID (UUID) Service that was charged (TTS, STT, LLM, MEMORY\_INGEST, etc.) USD amount deducted (negative values indicate refunds) Units consumed (characters for TTS, minutes for STT, tokens for LLM, bytes/queries for Memory) Workspace balance before this operation Workspace balance after this operation Name of the user who performed the operation ISO 8601 timestamp Aggregated totals grouped by service type Only the **workspace owner** can view usage logs. This endpoint is protected by `billing:manage` permission. Every API operation (TTS, STT, LLM, Memory) that costs money is logged here — this is your complete billing audit trail. ```bash cURL theme={null} curl "https://api.60db.ai/billing/usage-logs?limit=10&service_type=TTS" \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const response = await fetch( 'https://api.60db.ai/billing/usage-logs?limit=10&service_type=LLM', { headers: { 'Authorization': 'Bearer sk_your_api_key' } } ); const data = await response.json(); ``` ```json Response theme={null} { "success": true, "message": "Usage logs fetched successfully", "data": { "logs": [ { "hash_id": "84ffd09e-f5a4-42ea-a8fc-f50038392652", "service_type": "TTS", "amount_deducted": "0.00024000", "units_used": "12", "previous_balance": "9.50024000", "new_balance": "9.50000000", "metadata": {}, "created_at": "2026-04-13T12:30:00.000Z", "user_name": "Kapil Karda", "user_email": "kapil@example.com" }, { "hash_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "service_type": "LLM", "amount_deducted": "0.00150000", "units_used": "1500", "previous_balance": "9.50174000", "new_balance": "9.50024000", "metadata": {}, "created_at": "2026-04-13T12:25:00.000Z", "user_name": "Kapil Karda", "user_email": "kapil@example.com" } ], "pagination": { "current_page": 1, "per_page": 10, "total_records": 42, "total_pages": 5 }, "summary": [ { "service_type": "TTS", "count": 15, "total_cost": 0.0036, "total_units": 180 }, { "service_type": "LLM", "count": 20, "total_cost": 0.03, "total_units": 30000 }, { "service_type": "MEMORY_INGEST", "count": 7, "total_cost": 0.0007, "total_units": 7000 } ] } } ``` # API Reference Source: https://docs.60db.ai/api-reference/introduction Complete API reference for 60db ## Base URL ``` https://api.60db.ai ``` ## Authentication All API requests require authentication using an API key in the `Authorization` header: ```bash theme={null} Authorization: Bearer your-api-key ``` ## Response Format All API responses are returned in JSON format unless otherwise specified. ### Success Response ```json theme={null} { "success": true, "data": { ... } } ``` ### Error Response ```json theme={null} { "success": false, "error": "Error type", "message": "Detailed error message" } ``` ## HTTP Status Codes | Status Code | Description | | ----------- | ------------------------------------------------------- | | 200 | Success | | 201 | Created | | 202 | Accepted (queued for async processing) | | 400 | Bad Request | | 401 | Unauthorized | | 402 | Insufficient credits — workspace wallet balance too low | | 403 | Forbidden | | 404 | Not Found | | 413 | Payload Too Large | | 422 | Unprocessable Entity (validation or extraction fail) | | 429 | Too Many Requests | | 500 | Internal Server Error | | 503 | Service Unavailable | ## Rate Limiting API requests are rate-limited based on your subscription plan. Rate limit information is included in response headers: * `X-RateLimit-Limit`: Maximum requests allowed * `X-RateLimit-Remaining`: Remaining requests in current window * `X-RateLimit-Reset`: Time when the rate limit resets (Unix timestamp) ## Pagination List endpoints support pagination using query parameters: * `page`: Page number (default: 1) * `limit`: Items per page (default: 20, max: 100) ```bash theme={null} GET /voices?page=2&limit=50 ``` Response includes pagination metadata: ```json theme={null} { "data": [...], "pagination": { "page": 2, "limit": 50, "total": 150, "pages": 3 } } ``` ## Endpoints Overview ### Text-to-Speech * `POST /tts-synthesize` - Convert text to speech ### Speech-to-Text * `POST /stt` - Transcribe audio to text * `GET /stt/languages` - Get supported languages ### Voices * `GET /voices` - List all voices ### Memory & RAG * `POST /memory/ingest` - Store a single memory (user/knowledge/hive) * `POST /memory/ingest/batch` - Batch-store up to 100 memories * `POST /memory/documents/extract` - Upload a document (PDF, DOCX, XLSX, images) for extraction + OCR + chunked ingest * `POST /memory/search` - Hybrid semantic + keyword recall * `POST /memory/context` - Assemble an LLM-ready context string for RAG * `GET /memory/collections` - List collections in the workspace * `POST /memory/collections` - Create a team/knowledge/hive collection * `GET /memory/usage` - Monthly spend breakdown and wallet balance Memory endpoints are billed **pay-as-you-go** from the workspace wallet (see [Memory Pricing](/api-reference/memory/pricing) for rates). Every billable response includes `x-credit-balance`, `x-credit-charged`, and `x-billing-tx` headers so you can track spend in real time. When the wallet is empty, billable endpoints return `402 INSUFFICIENT_CREDITS` with a structured `details.shortfall` body so your client can prompt for top-up. ### LLM Chat * `POST /v1/chat/completions` - OpenAI-compatible chat completions ### Billing & Usage Each workspace has its own **USD wallet**. All service operations (TTS, STT, LLM, Memory) deduct from the workspace wallet. Funds are added via the [Dashboard](https://app.60db.ai). Only workspace owners can access billing endpoints. * `GET /billing/usage-logs` - Per-operation deduction history (TTS, STT, LLM, Memory) * `GET /billing/transactions` - Payment history (top-ups, subscriptions) All billing endpoints require `billing:manage` permission (workspace owner only). Non-owners receive `403 Forbidden`. ## SDKs We provide official SDKs for popular programming languages: npm install 60db pip install 60db # Chat Source: https://docs.60db.ai/api-reference/llm/chat-completion POST /v1/chat/completions Generate AI responses using our Small Language Model (SLM) with support for streaming, text correction, and function calling ## Request ### Headers Bearer token with your API key application/json ### Body - Direct Messages Mode (OpenAI Compatible) The model to use for completion Array of message objects with `role` and `content` Enable streaming response (Server-Sent Events) Top-k sampling parameter for response generation Template configuration options Enable thinking mode in the model Array of tool/function definitions for function calling ### Body - Enhanced Text Correction Mode The text to correct/improve Array of term-replacement pairs for custom corrections The term to find and replace The replacement text Style configuration for text correction Tone to apply (e.g., "professional", "casual", "friendly") Automatically capitalize sentences Add proper punctuation Whether to use contractions (false to expand them) Expand abbreviations to full form Application context (e.g., "email", "chat", "document") Save conversation to chat history Existing chat ID to continue conversation ## Response Array of completion choices The generated message with role and content Streaming delta with incremental content (stream mode only) ID of the chat session (for new chats) Token usage information Total tokens used in the request Response time in milliseconds ```bash cURL theme={null} # Direct Messages Mode (OpenAI Compatible) curl --location 'https://api.60db.ai/v1/chat/completions' \ --header 'Authorization: Bearer your-api-key' \ --header 'Content-Type: application/json' \ --data '{ "model": "60db-tiny", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "How are you? How can I improve my English communication?" } ], "top_k": 20, "chat_template_kwargs": { "enable_thinking": false }, "stream": true }' ``` ```bash cURL theme={null} # With Function Calling (Tools) curl --location 'https://api.60db.ai/v1/chat/completions' \ --header 'Authorization: Bearer your-api-key' \ --header 'Content-Type: application/json' \ --data '{ "model": "60db-tiny", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What'\''s the weather like in San Francisco?" } ], "stream": true, "tool": [ { "name": "get_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } ] }' ``` ```bash cURL theme={null} # Enhanced Text Correction Mode curl --location 'https://api.60db.ai/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer your-api-key' \ --data '{ "model": "60db-tiny", "text": "hello how r u i need help with english", "dictionary": [ {"term": "r", "replacement": "are"}, {"term": "u", "replacement": "you"} ], "style": { "tone": "professional", "autoCapitalize": true, "autoPunctuate": true, "useContractions": false, "expandAbbreviations": true }, "appContext": "email", "stream": true, "save_chat": true }' ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); // Direct messages mode const response = await client.chat.completions.create({ model: "60db-tiny", messages: [ { role: "system", content: "You are a helpful assistant." }, { role: "user", content: "How can I improve my English?" }, ], stream: true, }); // Handle streaming response for await (const chunk of response) { console.log(chunk.choices[0]?.delta?.content); } ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') # Text correction mode response = client.chat.completion( text='hello how r u i need help with english', dictionary=[ {'term': 'r', 'replacement': 'are'}, {'term': 'u', 'replacement': 'you'} ], style={ 'tone': 'professional', 'auto_capitalize': True, 'auto_punctuate': True, 'use_contractions': False, 'expand_abbreviations': True }, app_context='email', stream=True ) print(response['choices'][0]['message']['content']) ``` ```json Response (Non-Streaming) theme={null} { "id": "chatcmpl-123", "object": "chat.completion", "created": 1677652288, "model": "60db-tiny", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "I'm doing well, thank you! Here are some tips to improve your English communication skills..." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 25, "completion_tokens": 150, "total_tokens": 175 }, "chat_id": "550e8400-e29b-41d4-a716-446655440000", "response_time_ms": 1250 } ``` ```json Response (Streaming - SSE) theme={null} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":"I'm"}}]} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":" doing"}}]} data: {"id":"chatcmpl-123","choices":[{"delta":{"content":" well!"}}]} data: {"type":"done","response_time_ms":1250} data: [DONE] ``` ## Streaming Response When `stream: true`, the response is sent as Server-Sent Events (SSE): 1. **chat\_id event** - Sent first for new chats 2. **content chunks** - Delta updates with incremental content 3. **done event** - Signals completion with response time 4. **\[DONE]** - Final termination signal ```javascript theme={null} // Handling streaming in JavaScript const response = await fetch('https://api.60db.ai/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer your-api-key' }, body: JSON.stringify({ messages, stream: true }) }); const reader = response.body.getReader(); const decoder = new TextDecoder(); let buffer = ''; while (true) { const { done, value } = await reader.read(); if (done) break; buffer += decoder.decode(value, { stream: true }); const lines = buffer.split('\n'); buffer = lines.pop() || ''; for (const line of lines) { if (line.startsWith('data: ')) { const data = JSON.parse(line.slice(6)); if (data.type === 'chat_id') { console.log('Chat ID:', data.chat_id); } else if (data.type === 'done') { console.log('Response time:', data.response_time_ms); } else if (data.choices?.[0]?.delta?.content) { console.log('Content:', data.choices[0].delta.content); } } } } ``` Define tools/functions that the model can call during conversation: ```json theme={null} { "tool": [ { "name": "get_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } } ] } ``` The model will respond with tool calls that you can execute and send back the results. ## Text Correction Features Define custom term replacements that will always be applied: ```json theme={null} { "dictionary": [ {"term": "pls", "replacement": "please"}, {"term": "thx", "replacement": "thanks"}, {"term": "ASAP", "replacement": "as soon as possible"} ] } ``` Up to 100 dictionary entries are supported, each with a maximum length of 200 characters. Configure how the text should be corrected and styled: | Option | Type | Description | | --------------------- | ------- | ---------------------------------------------------- | | `tone` | string | Target tone: "professional", "casual", "friendly" | | `autoCapitalize` | boolean | Automatically capitalize first letter of sentences | | `autoPunctuate` | boolean | Add proper punctuation marks | | `useContractions` | boolean | Set to false to expand contractions (can't → cannot) | | `expandAbbreviations` | boolean | Expand common abbreviations | Provide context to help the model adjust its corrections: ```json theme={null} { "appContext": "email" } ``` Supported contexts: "email", "chat", "document", "message", "social" Token costs are calculated based on usage. The current rate is approximately \$0.00002 per token. Costs are deducted from your workspace billing balance. Chat history is automatically saved when `save_chat: true`. Use `chat_id` to continue existing conversations or create new chats by omitting this parameter. # Collections Source: https://docs.60db.ai/api-reference/memory/collections Manage memory collections (personal, team, knowledge, hive) A **collection** groups related memories. Each 60db workspace has automatic per-user personal collections, plus any team/knowledge/hive collections created by owners/admins. ## List collections `GET /memory/collections` Returns all collections the caller can access in the current workspace. ```bash theme={null} curl https://api.60db.com/memory/collections \ -H "Authorization: Bearer sk_abc123" ``` ```json Response theme={null} { "success": true, "data": [ { "hash_id": "uuid-1", "collection_id": "user_abc123", "label": "Personal Memories", "kind": "personal", "owner_user_id": 42, "shared": false, "created_at": "2026-04-01T10:00:00Z" }, { "hash_id": "uuid-2", "collection_id": "customer_support", "label": "Customer Support KB", "kind": "team", "owner_user_id": null, "shared": true, "created_at": "2026-04-02T14:30:00Z" } ] } ``` ## Create collection `POST /memory/collections` **Permission**: Owner or admin only. ### Body Unique ID within the workspace. Lowercase, alphanumeric, underscore, hyphen. Human-readable display name. Collection type. One of: `team`, `knowledge`, `hive`. If true, all workspace members can read memories in this collection. Arbitrary metadata attached to the collection. ```bash theme={null} curl -X POST https://api.60db.com/memory/collections \ -H "Authorization: Bearer sk_abc123" \ -H "Content-Type: application/json" \ -d '{ "collection_id": "customer_support", "label": "Customer Support KB", "kind": "knowledge", "shared": true }' ``` ## Collection kinds | Kind | Use case | Who can read | Who can write | | ----------- | ------------------------------------------------------- | ------------ | ---------------- | | `personal` | Per-user private memories | Owner only | Owner only | | `team` | Shared team-scoped memories | All members | All members | | `knowledge` | Reference documents, policies | All members | Owner/admin only | | `hive` | Cross-collection shared facts appearing in every search | All members | Owner/admin only | # Assemble Context (RAG) Source: https://docs.60db.ai/api-reference/memory/context POST /memory/context One-shot context assembly for LLM prompts — memories + timeline + graph Purpose-built for retrieval-augmented generation (RAG). Given a user query, the endpoint retrieves the most relevant memories, recent events, and graph relationships, then returns a **pre-formatted context string** ready to prepend to your LLM prompt. This is the easiest way to add memory to your AI chat. One call → formatted context. ## Request ### Headers Bearer token with your API key application/json ### Body The user's query. This drives retrieval. Chat session ID for hierarchical session context. Number of memories to retrieve. Max 100. Maximum assembled context length in tokens. Older chunks truncated if exceeded. Include knowledge-graph relationships. Include recent events from EventStoreDB. ## Response **The key field** — a pre-formatted context string you can prepend directly to your LLM system message. Structured context with separate `chunks`, `sources`, `graph_context`, and `timeline` sections. Status message (e.g., "Context assembled from 8 memories and 3 recent events"). ## Complete RAG example ```bash cURL theme={null} curl -X POST https://api.60db.ai/memory/context \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "query": "What do you know about my preferences?", "top_k": 8, "max_context_length": 2000, "include_timeline": true }' ``` ```javascript JavaScript theme={null} const response = await fetch('https://api.60db.ai/memory/context', { method: 'POST', headers: { 'Authorization': 'Bearer your-api-key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query: "What do you know about my preferences?", top_k: 8, max_context_length: 2000, include_timeline: true, }), }); const data = await response.json(); console.log(data.data.prompt_ready); ``` ```python Python theme={null} import requests response = requests.post( "https://api.60db.ai/memory/context", headers={ "Authorization": "Bearer your-api-key", "Content-Type": "application/json", }, json={ "query": "What do you know about my preferences?", "top_k": 8, "max_context_length": 2000, "include_timeline": True, }, ) data = response.json() print(data["data"]["prompt_ready"]) ``` ```json Response theme={null} { "success": true, "data": { "context": { "chunks": [ { "chunk_id": "c_01HV...", "source_id": "mem_01HV8K...", "text": "User prefers vegetarian food, lactose intolerant", "score": 0.92 } ], "sources": [ { "source_id": "mem_01HV8K...", "title": "Dietary preferences", "text": "User prefers vegetarian food, lactose intolerant", "score": 0.92 } ] }, "prompt_ready": "## User Memories\n- User prefers vegetarian food, lactose intolerant", "message": "Context assembled from 1 memories and 0 recent events" } } ``` ## Billing Flat \*\*$0.0005 per query** — slightly higher than `/memory/search` because context assembly also formats an LLM-ready prompt with recent events and optional graph context. A chat application doing 1,000 turns/day costs about $15/month. Response headers: | Header | Meaning | | ------------------ | -------------------------------- | | `x-credit-balance` | Wallet balance after this charge | | `x-credit-charged` | `0.000500` | | `x-billing-tx` | Audit row UUID | If the memory service is unreachable and the endpoint returns the graceful-degradation empty context (see below), you are **not** charged — the auto-refund guard reverses the deduction. See [Pricing & Billing](/api-reference/memory/pricing) for the full rate card. ## Graceful degradation If the Memory service is unavailable, `/memory/context` returns `prompt_ready: ""` instead of failing. Your application can continue with no context — the SLM chat will still work, just without memory-grounded responses. ```json No-context response theme={null} { "success": true, "data": { "context": { "chunks": [], "sources": [] }, "prompt_ready": "", "message": "Memory system not ready — no context assembled" } } ``` **Best practice**: Always check if `prompt_ready` is non-empty before prepending it. # Upload Document Source: https://docs.60db.ai/api-reference/memory/extract-document POST /memory/documents/extract Extract text from a document (PDF, DOCX, XLSX, scanned images...) with built-in OCR and ingest it into a memory collection in a single request Upload a document to have it extracted, chunked, and ingested into a memory collection in a single request. 60db's document extraction engine handles 91+ file formats and includes built-in OCR for scanned PDFs and images, so you can send the raw file and let the server do the rest. The browser just posts the file and 60db handles format detection, OCR, chunking, and ingestion. **Supported formats** (partial list — 91 total): * **Documents**: PDF, DOCX, DOC, ODT, RTF, TXT, MD, HTML, EPUB * **Spreadsheets**: XLSX, XLS, CSV, ODS * **Presentations**: PPTX, PPT, ODP * **Email**: EML, MSG, PST, MBOX * **Images (OCR)**: PNG, JPG, JPEG, TIFF, BMP, GIF * **Code & structured**: JSON, XML, YAML, LaTeX, Markdown variants * **Archives**: ZIP, TAR, GZIP, 7Z (extracted recursively) ## Request ### Headers Bearer token with your API key multipart/form-data ### Body (multipart/form-data) The document to extract. Max 200 MB per file. Collection ID to store the extracted chunks in. Defaults to the caller's personal collection. Memory type for the ingested chunks. One of: `user`, `knowledge`, `hive`. For document uploads, `knowledge` is almost always the right choice. Display title for the document. Defaults to the uploaded filename. When the document produces multiple chunks, each chunk is labeled `"{title} (part N/M)"`. Maximum characters per chunk. Larger chunks preserve more context but are less precise for recall. Minimum 200, maximum 8000. Characters of overlap between adjacent chunks. Helps preserve sentences that span chunk boundaries. Must be less than `chunk_size`. ## Response `true` on success. The collection the chunks were stored in. Human-readable collection name. Original filename of the uploaded document. Number of chunks produced from the extracted text. Total characters of extracted text. The memory type used for ingest (`user`, `knowledge`, or `hive`). Number of memories queued for processing (equals `chunks`). Array of `{id, status, message}` — one entry per chunk ingested. Use the IDs with `GET /memory/:id/status` to poll for processing completion. Extracted document metadata, including `mime_type`, `filename`, `page_count` (for PDFs), `detected_languages`, and `total_chunks`. ## Examples ```bash cURL theme={null} curl -X POST https://api.60db.ai/memory/documents/extract \ -H "Authorization: Bearer your-api-key" \ -F "file=@quarterly-report.pdf" \ -F "collection=company_handbook" \ -F "type=knowledge" \ -F "title=Q4 2026 Report" ``` ```javascript JavaScript theme={null} const form = new FormData(); form.append('file', pdfFile); form.append('collection', 'company_handbook'); form.append('type', 'knowledge'); form.append('title', 'Q4 2026 Report'); const response = await fetch('https://api.60db.ai/memory/documents/extract', { method: 'POST', headers: { 'Authorization': 'Bearer your-api-key', }, body: form, }); const { success, data } = await response.json(); console.log(`Ingested ${data.chunks} chunks from ${data.filename}`); ``` ```python Python theme={null} import requests with open('quarterly-report.pdf', 'rb') as f: response = requests.post( 'https://api.60db.ai/memory/documents/extract', headers={'Authorization': 'Bearer your-api-key'}, files={'file': ('quarterly-report.pdf', f, 'application/pdf')}, data={ 'collection': 'company_handbook', 'type': 'knowledge', 'title': 'Q4 2026 Report', }, ) result = response.json() print(f"Ingested {result['data']['chunks']} chunks") ``` ```json Response theme={null} { "success": true, "data": { "collection_id": "company_handbook", "collection_label": "Company Handbook", "filename": "quarterly-report.pdf", "chunks": 18, "characters": 24680, "memory_type": "knowledge", "total_queued": 18, "results": [ { "id": "mem_01HV8K...", "status": "pending", "message": "Queued for processing" }, { "id": "mem_01HV8L...", "status": "pending", "message": "Queued for processing" } ], "metadata": { "source": "document_upload", "filename": "quarterly-report.pdf", "mime_type": "application/pdf", "page_count": 24, "detected_languages": ["eng"], "total_chunks": 18 } } } ``` ## Pipeline When you POST a file, 60db runs it through this pipeline: 1. **Validate** — file present, type allowed, under 200 MB, collection accessible. 2. **Extract** — the document extraction engine detects the format (PDF, DOCX, image, etc.) and returns plain text plus metadata (`mime_type`, `page_count`, `tables`, `quality_score`). OCR is applied automatically for scanned PDFs and images. 3. **Chunk** — split the extracted text into overlapping segments of `chunk_size` characters with `chunk_overlap` character overlap. 4. **Register collection** — ensure the target collection is ready (idempotent, cached). 5. **Ingest** — stream all chunks into the memory layer in a single batch. 6. **Return** — the response includes one `{id, status, message}` entry per chunk. Processing continues asynchronously. For very large documents, prefer a higher `chunk_size` (e.g. 3000) to keep the per-chunk memory count low. The endpoint rejects uploads that produce more than 100 chunks with a `TOO_MANY_CHUNKS` error — split such files before upload or use a larger chunk size. ## Tuning | Document type | Recommended `chunk_size` | `chunk_overlap` | Notes | | ------------------------- | ------------------------ | --------------- | ------------------------------------------------------- | | Technical docs / API refs | 1500 | 200 | Default. Balances recall precision and context. | | Long-form prose / books | 2500 | 300 | Fewer chunks, more context per result. | | FAQs / short snippets | 800 | 100 | Higher precision — each Q\&A becomes its own chunk. | | Spreadsheet exports | 3000 | 0 | Tables should stay contiguous; overlap hurts. | | Scanned PDFs (OCR) | 2000 | 250 | OCR adds whitespace noise; slightly longer chunks help. | ## Billing Document upload is **two-stage billing** — you pay for the extraction and for the resulting ingest. | Stage | Rate | When charged | Refund on failure | | --------------- | --------------------------------------- | ------------------------------- | ----------------- | | **Extract fee** | \$0.003 per MB | Before extraction runs | Yes, auto | | **Ingest fee** | \$0.0001 per 1,000 extracted characters | After extraction, before ingest | Yes, auto | The two charges are separate rows in `transaction_log` (`MEMORY_EXTRACT` and `MEMORY_INGEST`) so you can distinguish extraction cost from storage cost in your reporting. **Example** — uploading a 2 MB PDF that extracts to 50,000 characters of text: ``` Extract fee: 2 × $0.003 = $0.006 Ingest fee: (50,000 / 1000) × $0.0001 = $0.005 Total: $0.011 ``` **Response headers** on success: | Header | Meaning | | ------------------------ | ------------------------------------------------------------------------------ | | `x-credit-balance` | Wallet balance after the **extract fee** was deducted (set before ingest runs) | | `x-credit-charged` | Just the extract fee | | `x-credit-charged-total` | Extract fee + ingest fee combined | | `x-billing-tx` | UUID of the **extract** audit row (the ingest row is linked via metadata) | **Special failure case** — if extraction succeeds but your wallet can't cover the post-extraction ingest charge, the extract fee is automatically refunded and the response is `402 INSUFFICIENT_CREDITS` with `details.extract_fee_refunded` populated. You pay nothing for the failed attempt. See [Pricing & Billing](/api-reference/memory/pricing) for the full policy. ## Error responses | Status | Code | Meaning | | ------ | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 400 | `NO_FILE` | No file was attached to the request. | | 400 | `INVALID_TYPE` | `type` must be `user`, `knowledge`, or `hive`. | | 402 | `INSUFFICIENT_CREDITS` | Wallet cannot cover the extract fee (pre-charge) OR the post-extraction ingest fee (in which case `details.extract_fee_refunded` is populated). | | 403 | `POLICY_DENY` | Your role (e.g. `viewer`) is not allowed to create memories. | | 404 | `COLLECTION_NOT_FOUND` | The specified `collection` doesn't exist or you can't access it. | | 413 | `TOO_MANY_CHUNKS` | Document produced more than 100 chunks. Increase `chunk_size` or split the file. Full auto-refund. | | 422 | `EMPTY_EXTRACTION` | Document contains no extractable text. Full auto-refund. | | 422 | `EMPTY_CHUNKS` | Chunking produced zero segments. Full auto-refund. | | 422 | `EXTRACTION_FAILED` | Extraction engine rejected the file. Full auto-refund. | | 503 | `MEMORY_INFRA_NOT_READY` | The workspace's memory layer is still provisioning. Retry in \~10s. Full auto-refund. | | 503 | `EXTRACTION_SERVICE_UNAVAILABLE` | Document extraction is temporarily unavailable. Full auto-refund. | | 202 | `MEMORY_QUEUED` | Memory layer is temporarily unreachable; chunks were queued and will retry automatically. Both extract and ingest fees are refunded because the work won't actually happen. | ## Checking ingestion status The endpoint returns immediately once chunks are queued — full embedding/indexing happens asynchronously. Poll `GET /memory/:id/status` with any of the returned chunk IDs to check progress: ```bash theme={null} curl https://api.60db.com/memory/mem_01HV8K.../status \ -H "Authorization: Bearer sk_abc123" ``` Statuses: `pending` → `processing` → `ready` (or `failed`). ## Size limits * **Per file**: 200 MB * **Chunks per document**: 100 (use a larger `chunk_size` to fit bigger files) * **Chunk text length**: 100,000 characters * **Supported languages for OCR**: 100+ languages including English, Spanish, French, German, Chinese, Japanese, Arabic, Hindi, and more * **Rate limit**: Same as `POST /memory/ingest/batch` (30 uploads/min per workspace on default plans) # Ingest Memory Source: https://docs.60db.ai/api-reference/memory/ingest POST /memory/ingest Store a memory (user, knowledge, or hive) in a collection Store a new memory in your workspace. Memories are processed asynchronously — the response returns immediately with a pending memory ID that you can poll for status. ## Request ### Headers Bearer token with your API key application/json ### Body The memory content to store. Max 100,000 characters. Optional display title for the memory. Memory type. One of: `user`, `knowledge`, `hive`. * `user`: Personal memory for the calling user * `knowledge`: Shared knowledge base entry (admin/owner only) * `hive`: Workspace-wide shared memory (admin/owner only) Collection ID to store the memory in. Defaults to the caller's personal collection. For team collections, pass the collection\_id returned from `POST /memory/collections`. If true, the memory service extracts structured facts and preferences via LLM inference. Optional metadata to attach to the memory. Filterable at search time. ## Response True on success The collection the memory was stored in Type: user, knowledge, or hive Number of memories queued for processing List of `{id, status, message}` — one per memory ingested ## Example ```bash cURL theme={null} curl -X POST https://api.60db.ai/memory/ingest \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "text": "User prefers vegetarian food, lactose intolerant", "title": "Dietary preferences", "type": "user", "infer": true }' ``` ```javascript JavaScript theme={null} const response = await fetch('https://api.60db.ai/memory/ingest', { method: 'POST', headers: { 'Authorization': 'Bearer your-api-key', 'Content-Type': 'application/json', }, body: JSON.stringify({ text: "User prefers vegetarian food, lactose intolerant", title: "Dietary preferences", type: "user", infer: true, }), }); const data = await response.json(); ``` ```python Python theme={null} import requests response = requests.post( "https://api.60db.ai/memory/ingest", headers={ "Authorization": "Bearer your-api-key", "Content-Type": "application/json", }, json={ "text": "User prefers vegetarian food, lactose intolerant", "title": "Dietary preferences", "type": "user", "infer": True, }, ) data = response.json() ``` ```json Response theme={null} { "success": true, "data": { "collection_id": "user_abc123", "collection_label": "Personal Memories", "memory_type": "user", "total_queued": 1, "results": [ { "id": "mem_01HV8K2X3N4P5Q6R7S8T9U", "status": "pending", "message": "Memory queued for processing" } ] } } ``` ## Billing This endpoint is billed at \*\*$0.0001 per 1,000 characters** of `text`. A 500-char memory costs $0.00005; a 5,000-char memory costs \$0.0005. The charge is deducted upfront from the workspace owner's wallet, and automatically refunded if the request fails. **Response headers** — every successful request returns: | Header | Meaning | | ------------------ | ------------------------------------------- | | `x-credit-balance` | Your wallet balance **after** this charge | | `x-credit-charged` | Amount charged for this specific request | | `x-billing-tx` | UUID of the audit row (for refunds/support) | See [Pricing & Billing](/api-reference/memory/pricing) for the full rate card and refund policy. ## Error responses | Status | Code | Meaning | | ------ | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | | 400 | — | Missing or invalid `text` field | | 402 | `INSUFFICIENT_CREDITS` | Wallet balance is lower than the ingest charge. Response body includes `details.required`, `details.available`, and `details.shortfall`. | | 403 | `POLICY_DENY` | Role is not allowed to create memories (viewer) | | 503 | `MEMORY_INFRA_NOT_READY` | The memory layer is still being provisioned for this workspace (usually only on first use). Retry in \~10s. | | 503 | `MEMORY_UNREACHABLE` | The memory service is temporarily unavailable. Request is queued for retry and the charge is automatically refunded. | ## Checking status Poll `GET /memory/:id/status` to check if a memory has been fully processed. Statuses: `pending`, `processing`, `ready`, `failed`. # Pricing & Billing Source: https://docs.60db.ai/api-reference/memory/pricing Pay-as-you-go pricing for 60db Memory — wallet-based, per-operation, fully transparent 60db Memory is **pay-as-you-go**, not subscription-based. Every operation deducts a precise amount from your workspace's wallet, and every charge is logged in a transaction audit trail you can read via [`GET /memory/usage`](/api-reference/memory/usage). No minimum commitment, no seat pricing, no overage surprises. You pay only for the operations you run, and when you run out of credits the service returns `402 Insufficient Credits` until you top up your wallet. ## Rates | Operation | Unit | Rate | Example | | --------------------------------------- | -------------------- | ------------ | ------------------------------------------- | | **Ingest** — store a memory | per 1,000 characters | **\$0.0001** | 5,000-char memory = \$0.0005 | | **Document upload** — extract fee | per megabyte | **\$0.003** | 2 MB PDF = \$0.006 (plus ingest cost below) | | **Document upload** — ingest fee | per 1,000 characters | **\$0.0001** | 50,000 chars extracted = \$0.005 | | **Search** — hybrid recall | per query | **\$0.0003** | 1,000 searches = \$0.30 | | **Context assembly** — LLM-ready prompt | per query | **\$0.0005** | 1,000 queries = \$0.50 | ### What these add up to in practice | Scenario | Monthly cost | | --------------------------------------------------------------------------------------------- | --------------- | | Knowledge base of **100 MB** uploaded + **10,000 searches/mo** | **\~\$23/mo** | | Personal assistant — **1,000 user memories stored** (\~200 chars each) + **500 searches/day** | **\~\$4.70/mo** | | Enterprise support bot — **1 GB docs** + **100,000 searches/mo** | **\~\$53/mo** | Compare against proprietary memory services that charge $249–$5,000/month flat subscriptions regardless of usage. For most workloads, 60db's pay-as-you-go model is 5–50x cheaper. ## How the wallet works Your 60db workspace has its own **USD wallet**. Every billable operation (Memory, TTS, STT, LLM) deducts from the workspace wallet atomically: 1. **Top up** via the [Dashboard billing page](https://app.60db.ai) using Dodo Payments. 2. **Use the API** — every billable request is charged upfront from the workspace wallet. 3. **Automatic refund** on failure — if a request fails after being charged (e.g. the upstream service is unreachable or a document is corrupt), the charge is automatically reversed and logged with a `*_REFUND` transaction. 4. **Monitor spend** via [`GET /billing/usage-logs`](/api-reference/billing/usage-logs) for all services, or [`GET /memory/usage`](/api-reference/memory/usage) for Memory-specific breakdown. ## Response headers (every billable request) Every successful memory operation returns three custom headers so you can track spend without polling: | Header | Meaning | | ------------------------ | ------------------------------------------------------------------------------------------------- | | `x-credit-balance` | Your wallet balance **after** this charge (USD, 6 decimal places) | | `x-credit-charged` | Amount charged **for this specific request** (USD, 6 decimal places) | | `x-credit-charged-total` | Present only on `/memory/documents/extract` — sum of the extract fee + post-extraction ingest fee | | `x-billing-tx` | UUID of the `transaction_log` row (useful for support tickets and manual refunds) | ### Example ```bash theme={null} curl -v -X POST https://api.60db.com/memory/search \ -H "Authorization: Bearer sk_abc123" \ -H "Content-Type: application/json" \ -d '{"query": "user preferences"}' 2>&1 | grep "^< x-" ``` ``` < x-credit-balance: 9.465200 < x-credit-charged: 0.000300 < x-billing-tx: 84ffd09e-f5a4-42ea-a8fc-f50038392652 ``` ## Handling `402 Insufficient Credits` When your wallet runs out, all billable memory endpoints return **HTTP 402** with a structured error body: ```json theme={null} { "success": false, "message": "Insufficient credits", "error_code": "INSUFFICIENT_CREDITS", "details": { "required": 0.0003, "available": 0.00001, "shortfall": 0.00029 } } ``` Your client should catch this and either prompt the user to top up the wallet (via the dashboard) or surface the shortfall in your own UI. Administrative endpoints (`GET /memory/collections`, `GET /memory/:id/status`, `DELETE /memory/:id`, `GET /memory/usage`) are **never** billed and never return 402 — they stay available even when the wallet is empty so customers can still inspect, clean up, and check usage. ## Automatic refund on failure 60db charges **upfront** but refunds automatically whenever the downstream work fails. You don't need to open a ticket — the refund lands in `transaction_log` as a negative row linked to the original charge via `reference_hash_id`. Refunds are triggered on: | Scenario | Refund behavior | | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | Controller throws an unexpected error (HTTP 5xx) | Full auto-refund via response guard | | Client validation error (HTTP 4xx) | Full auto-refund (nothing actually ran) | | Memory service unreachable — request queued (`202 MEMORY_QUEUED`) | Full auto-refund (work won't happen) | | Document extraction fails (corrupt/unsupported file, `422 EXTRACTION_FAILED`) | Full auto-refund of the extract fee | | Extraction succeeds but wallet can't cover the post-extract ingest charge | Extract fee refunded, `402` returned with `extract_fee_refunded` in details | You can verify a refund by calling `GET /memory/usage` — the `refunds` counter increments, and `net_spend_usd` reflects the reversal. ## What is NOT billed To give you visibility into your data without forcing charges: * Listing collections (`GET /memory/collections`) * Creating collections (`POST /memory/collections`) * Checking memory status (`GET /memory/:id/status`) * Deleting a memory (`DELETE /memory/:id`) * Service health (`GET /memory/health`) * Reconcile (`POST /memory/reconcile`) * Usage breakdown (`GET /memory/usage`) ## Setting spend limits The pay-as-you-go model means you cap spending by capping your wallet balance. Top up only what you want to risk this period. We recommend: * **Small teams**: top up $10–$50 at a time via the [Dashboard](https://app.60db.ai) billing page and let the wallet drain naturally. * **Production deployments**: set a cron that pings `GET /memory/usage` daily and alerts your team when net spend crosses a threshold. * **Enterprise**: contact sales for committed-use pricing with higher rate limits and dedicated support. ## Related * [`GET /memory/usage`](/api-reference/memory/usage) — check your current spend breakdown * [Memory feature overview](/features/memory) — what you can build with 60db Memory * [Dashboard billing page](https://app.60db.ai) — top up your workspace wallet # Search Memories Source: https://docs.60db.ai/api-reference/memory/search POST /memory/search Hybrid semantic + keyword search with cross-encoder reranking Search memories in a collection using hybrid retrieval. Combines vector similarity (semantic) with BM25 keyword scoring, with optional cross-encoder reranking for higher precision. Optionally returns graph relationships. ## Request ### Headers Bearer token with your API key application/json ### Body Search query text. Max 2,000 characters. Collection to search. Defaults to the caller's personal collection. Search mode: * `fast` — single-query dense retrieval (\~100-200ms). Best for simple lookups. * `thinking` — fetches a wider candidate pool and applies cross-encoder reranking for higher precision (\~200-400ms). Best for complex or multi-faceted questions. Maximum number of results. Capped at 50. Weight of semantic search (0-1). `0` = keyword only, `1` = semantic only. Weight given to newer memories (0-1). Include knowledge-graph relationships in the response. ### Advanced reranker knobs These parameters override server-side defaults for the cross-encoder reranker. Omit to use the deployment default. Max candidates the cross-encoder reranks (1-500). Default: server setting (30). Hard timeout for the rerank call in milliseconds (50-5000). Default: server setting (500). Drop results with rerank score below this threshold (0-1). Default: server setting (0.25). In `thinking` mode, fetch N x `max_results` candidates before reranking (1-10). Default: server setting (3). ## Response Raw chunk-level search results with scores. Each chunk includes: * `score` — dense vector similarity score (0-1) * `rerank_score` — cross-encoder rerank score (0-1, present when reranker is active, null otherwise) Deduplicated source memories (one per unique memory\_id) Graph nodes, edges, and triplets (only if `graph_context: true`) Total number of chunks returned Search latency in milliseconds Per-query diagnostic trace including stage timings, reranker meta, and active flag snapshot. Useful for debugging search quality. ## Example ```bash cURL theme={null} curl -X POST https://api.60db.ai/memory/search \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "query": "What are my dietary preferences?", "mode": "thinking", "max_results": 5, "alpha": 0.8, "recency_bias": 0.1 }' ``` ```javascript JavaScript theme={null} const response = await fetch('https://api.60db.ai/memory/search', { method: 'POST', headers: { 'Authorization': 'Bearer your-api-key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query: "What are my dietary preferences?", mode: "thinking", max_results: 5, alpha: 0.8, recency_bias: 0.1, }), }); const data = await response.json(); ``` ```python Python theme={null} import requests response = requests.post( "https://api.60db.ai/memory/search", headers={ "Authorization": "Bearer your-api-key", "Content-Type": "application/json", }, json={ "query": "What are my dietary preferences?", "mode": "thinking", "max_results": 5, "alpha": 0.8, "recency_bias": 0.1, }, ) data = response.json() ``` ```json Response theme={null} { "success": true, "data": { "query": "What are my dietary preferences?", "chunks": [ { "chunk_id": "c_01HV...", "source_id": "mem_01HV8K...", "text": "User prefers vegetarian food, lactose intolerant", "score": 0.912, "rerank_score": 0.9485, "metadata": { "title": "Dietary preferences" } } ], "sources": [ { "source_id": "mem_01HV8K...", "title": "Dietary preferences", "text": "User prefers vegetarian food, lactose intolerant", "score": 0.912, "rerank_score": 0.9485 } ], "total_chunks": 1, "total_sources": 1, "latency_ms": 287.4, "mode": "thinking", "alpha": 0.8, "trace": { "timings_ms": { "embed_ms": 112, "vector_search_ms": 52, "total_ms": 287 }, "rerank": { "mode": "on", "ok": true, "latency_ms": 103, "top_score": 0.9485 } } } } ``` ## Billing Flat \*\*$0.0003 per query**, regardless of `max_results` or `mode`. A workload of 10,000 searches per month costs $3. Every successful request returns: | Header | Meaning | | ------------------ | -------------------------------- | | `x-credit-balance` | Wallet balance after this charge | | `x-credit-charged` | `0.000300` | | `x-billing-tx` | Audit row UUID | On `402 INSUFFICIENT_CREDITS`, the response includes `details.shortfall` so you can prompt the user to top up. See [Pricing & Billing](/api-reference/memory/pricing). ## Tuning **Query types and recommended settings**: | Query type | Recommended settings | | ------------------------------ | ------------------------------- | | Exact phrase match | `alpha: 0.2, mode: fast` | | Conceptual question | `alpha: 0.9, mode: fast` | | Complex multi-faceted question | `alpha: 0.7, mode: thinking` | | Latest-events focus | `alpha: 0.6, recency_bias: 0.3` | # Get Memory Usage Source: https://docs.60db.ai/api-reference/memory/usage GET /memory/usage Monthly (or custom-period) spend breakdown for the workspace's memory operations Return the wallet owner's spend on memory operations over a given period, broken down by operation type. Unbilled — calling this endpoint is always free and works even when the wallet is empty. Use this endpoint to: * Power a "Memory spend" widget in your own dashboard * Alert on daily/monthly spend thresholds * Reconcile refunds against original charges ## Request ### Headers Bearer token with your API key ### Query parameters Time window to aggregate over. One of: * `current_month` — from midnight on the 1st of the current month * `last_30_days` — rolling 30-day window ending now * `all_time` — all history (since the transaction log was created) ## Response `true` on success. Echoes back the requested `period`. The workspace this usage belongs to. The user whose wallet funds memory operations in this workspace. Live wallet balance in USD (8-decimal precision). Summed across all memory service types. Total spend after refunds (USD). Refunds subtract from the total. Count of positive-amount operations (ingests + extracts + searches + contexts). Count of refund rows issued in this period. Per-service-type breakdown. Keys are `MEMORY_INGEST`, `MEMORY_EXTRACT`, `MEMORY_RECALL`, `MEMORY_CONTEXT`, and the corresponding `*_REFUND` entries when refunds have occurred. Each value contains: * `net_spend_usd` — after refunds * `gross_units` — total units used (chars for ingest, bytes for extract, queries for recall/context) * `operation_count` — positive operations only * `refund_count` — refunded operations only ## Example ```bash cURL theme={null} curl https://api.60db.ai/memory/usage?period=current_month \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const res = await fetch('https://api.60db.ai/memory/usage?period=current_month', { headers: { 'Authorization': 'Bearer your-api-key' }, }); const { data } = await res.json(); ``` ```python Python theme={null} import requests res = requests.get( 'https://api.60db.ai/memory/usage', params={'period': 'current_month'}, headers={'Authorization': 'Bearer your-api-key'}, ) data = res.json()['data'] ``` ```json Response theme={null} { "success": true, "data": { "period": "current_month", "workspace_id": 24, "billing_owner": { "id": 33, "name": "Kapil Karda", "current_balance_usd": 9.46304021 }, "total": { "net_spend_usd": 0.002779, "operations": 10, "refunds": 1 }, "by_service": { "MEMORY_EXTRACT": { "net_spend_usd": 0.00000169, "gross_units": 592, "operation_count": 2, "refund_count": 0 }, "MEMORY_EXTRACT_REFUND": { "net_spend_usd": -0.00000010, "gross_units": 0, "operation_count": 0, "refund_count": 1 }, "MEMORY_INGEST": { "net_spend_usd": 0.00007110, "gross_units": 711, "operation_count": 2, "refund_count": 0 }, "MEMORY_RECALL": { "net_spend_usd": 0.00270000, "gross_units": 9, "operation_count": 9, "refund_count": 0 } } } } ``` ## Notes * Refunds appear as separate service types (`MEMORY_EXTRACT_REFUND`, `MEMORY_INGEST_REFUND`, etc.) with negative `net_spend_usd`. This preserves an auditable distinction between "a charge that happened and was reversed" vs "no charge at all". * The `billing_owner.current_balance_usd` reflects the live wallet — it is **not** period-scoped, it's always the current balance regardless of which `period` you requested. * For a full operation-level audit trail (every single row in `transaction_log`), use the dashboard — this endpoint is an aggregated summary. ## Related * [Pricing & Billing](/api-reference/memory/pricing) — rates, refund policy, header reference * [Memory feature overview](/features/memory) # Get TTS Models Source: https://docs.60db.ai/api-reference/models/get-models GET /tts/models Retrieve the list of available Text-to-Speech synthesis models Returns the catalog of TTS voice-synthesis models exposed by 60db. For Speech-to-Text models, use [`GET /stt/models`](/api-reference/models/get-stt-models). The legacy alias `GET /models` returns the same payload as `GET /tts/models` for backwards compatibility. ## Request ### Headers Bearer token with your API key ## Response Indicates whether the request was successful Status message Array of TTS model entries Unique model identifier used when invoking TTS endpoints (e.g. `60db-fast-v01`, `60db-quality-v01`) Human-readable display name Short description of the model's intended use Model tier — `"cloned"` for fast voice-clone models or `"professional"` for high-quality studio voice models Always `"tts"` for this endpoint ```bash cURL theme={null} curl https://api.60db.ai/tts/models \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const { data: models } = await client.getTTSModels(); for (const model of models) { console.log(`${model.id} — ${model.model_name} (${model.category})`); } ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') res = client.get_tts_models() for model in res['data']: print(f"{model['id']} — {model['model_name']} ({model['category']})") ``` ```json Response theme={null} { "success": true, "message": "TTS models fetched successfully", "data": [ { "id": "60db-fast-v01", "model_name": "60db Fast", "description": "Fast voice cloning model for quick voice creation", "category": "cloned", "type": "tts" }, { "id": "60db-quality-v01", "model_name": "60db Quality", "description": "High quality professional voice model for production use", "category": "professional", "type": "tts" } ] } ``` # Get STT Models Source: https://docs.60db.ai/api-reference/models/get-stt-models GET /stt/models Retrieve the list of available Speech-to-Text models Returns the catalog of STT (transcription) models exposed by 60db. For Text-to-Speech models, use [`GET /tts/models`](/api-reference/models/get-models). ## Request ### Headers Bearer token with your API key ## Response Indicates whether the request was successful Status message Array of STT model entries Unique model identifier (e.g. `60db-stt-v01`) Human-readable display name Short description of the model's capabilities Always `"speech-to-text"` for STT models Always `"stt"` for this endpoint Number of supported transcription languages Feature flags advertised by this model: `auto_language_detection`, `speaker_diarization`, `word_timestamps`, `code_switching_indic_english`, `continuous_mode`, `telephony_mulaw`, `websocket_streaming` ```bash cURL theme={null} curl https://api.60db.ai/stt/models \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const { data: models } = await client.getSTTModels(); for (const model of models) { console.log(`${model.id} — ${model.model_name}`); console.log(` Languages: ${model.languages}`); console.log(` Features: ${model.features.join(", ")}`); } ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') res = client.get_stt_models() for model in res['data']: print(f"{model['id']} — {model['model_name']}") print(f" Languages: {model['languages']}") print(f" Features: {', '.join(model['features'])}") ``` ```json Response theme={null} { "success": true, "message": "STT models fetched successfully", "data": [ { "id": "60db-stt-v01", "model_name": "60db STT v01", "description": "Non-hallucinating speech recognition supporting 39 languages with auto-detection, speaker diarization, and real-time streaming", "category": "speech-to-text", "type": "stt", "languages": 39, "features": [ "auto_language_detection", "speaker_diarization", "word_timestamps", "code_switching_indic_english", "continuous_mode", "telephony_mulaw", "websocket_streaming" ] } ] } ``` ## Notes * The `id` value is the same string reported by the `session_started` WebSocket event (see [`/ws/stt`](/api-reference/websocket/stt)) and is stable across requests. * Currently only one STT model is exposed. Future versions (`60db-stt-v02`, domain-tuned variants, etc.) will appear as additional entries in this array without any breaking changes to the response shape. # Get Languages Source: https://docs.60db.ai/api-reference/stt/get-languages GET /stt/languages Get the list of supported speech-to-text languages ## Request ### Headers Bearer token with your API key ## Response Always `true` on 200 responses Status message (`"Languages retrieved successfully"`) Language catalog envelope Ordered list of supported languages. The **first entry is always `auto`**, representing auto-detect. Use the `code` value when submitting `/stt` requests. ISO 639-1 language code, or `"auto"` for the auto-detect entry Human-readable language name (e.g. `"English"`, `"Hindi"`, `"Arabic (MSA)"`) Same as `name` for compatibility with clients that read the `native` key `false` for European languages, or an array like `["en"]` for Indic languages that support inline code-switching with the listed language(s) Target word error rate for this language at normal audio quality. Omitted for `auto`. Total number of language entries (including the `auto` option) Backend feature flags (e.g. `word_timestamps`, `speaker_diarization`, `srt_export`, `vtt_export`, `code_switching_indic_english`, `multi_language_per_request`, `max_languages_per_request`, `auto_language_detection`, `websocket_streaming`, `telephony_mulaw`, `continuous_mode`) ```bash cURL theme={null} curl https://api.60db.ai/stt/languages \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const res = await client.getLanguages(); const languages = res.data.languages; // languages[0] is always { code: "auto", name: "Auto-detect", ... } for (const lang of languages) { console.log(`${lang.name} (${lang.code})`); } ``` ```python Python theme={null} res = client.get_languages() languages = res['data']['languages'] # languages[0] is always {'code': 'auto', 'name': 'Auto-detect', ...} for lang in languages: print(f"{lang['name']} ({lang['code']})") ``` ```json Response theme={null} { "success": true, "message": "Languages retrieved successfully", "data": { "languages": [ { "code": "auto", "name": "Auto-detect", "native": "Auto", "code_switching": false }, { "code": "en", "name": "English", "native": "English", "code_switching": false, "wer_target": 0.08 }, { "code": "hi", "name": "Hindi", "native": "Hindi", "code_switching": ["en"], "wer_target": 0.28 }, { "code": "ar", "name": "Arabic (MSA)", "native": "Arabic (MSA)", "code_switching": false, "wer_target": 0.20 }, { "code": "bn", "name": "Bengali", "native": "Bengali", "code_switching": ["en"], "wer_target": 0.25 }, { "code": "fr", "name": "French", "native": "French", "code_switching": false, "wer_target": 0.12 } ], "total": 40, "features": { "word_timestamps": true, "speaker_diarization": true, "srt_export": true, "vtt_export": true, "confidence_scoring": true, "code_switching_indic_english": [ "hi+en", "bn+en", "mr+en", "pa+en", "gu+en", "or+en", "as+en", "ne+en", "te+en", "kn+en", "ta+en", "ml+en" ], "multi_language_per_request": true, "max_languages_per_request": 5, "auto_language_detection": true, "websocket_streaming": true, "telephony_mulaw": true, "continuous_mode": true } } } ``` ## Notes * The `auto` entry is synthesized by this API and is **not** one of the backend's native language codes. To submit a request with auto-detection, either omit the `language` field from `POST /stt` or explicitly pass `language=auto`. * Supported languages total 39 real languages + 1 `auto` entry. * **Unsupported languages** (explicitly rejected): Urdu (`ur`), Japanese (`ja`), Korean (`ko`), Chinese (`zh`), Thai (`th`), Vietnamese (`vi`), Indonesian (`id`), Tagalog (`tl`), Swahili (`sw`), Turkish (`tr`), Persian (`fa`), Hebrew (`he`). * **Arabic dialect tags** (e.g. `ar-eg`, `ar-lv`) are rejected — pass `language=ar` for best-effort MSA transcription. # Speech to Text Source: https://docs.60db.ai/api-reference/stt/speech-to-text POST /stt Transcribe audio to text with auto language detection and optional speaker diarization ## Request ### Headers Bearer token with your API key multipart/form-data ### Form Data Audio file to transcribe. * Supported formats: WAV, MP3, M4A, OGG, FLAC, WebM, MP4 (audio track) * Max file size: 10 MB * Max duration: 1 hour ISO 639-1 language code (e.g. `en`, `hi`, `ar`, `fr`). **Omit this field or pass `auto`** to enable language auto-detection across the 39 supported languages. When specified and valid, skips language identification entirely for lowest latency. Enable speaker diarization. When `true`, each segment of the response includes a `speakers` array identifying distinct speakers (`SPEAKER_00`, `SPEAKER_01`, …). Adds \~50–150 ms of processing latency per request. Free-form paragraph describing the session — domain, speakers, jargon, proper nouns you want preserved. When supplied, the server runs a background LLM refinement pass and the response text is polished for proper nouns, filler removal, and punctuation. Omit to skip refinement. Example: `"Cricket coaching session. Players: Arjun Mehta, Ishaan Verma, Aryan Khan, Rohan. Discussing batting technique, stamina, running between wickets, off-side balls."` On the **Free** plan, `context` is silently stripped server-side and the response includes `warning_codes: ["llm_refinement_not_in_plan"]`. The transcript is produced without refinement. Upgrade to enable the gate. The REST `POST /stt` form takes `context` as a **plain string**. The WebSocket `/ws/stt` endpoint takes a structured `{general, text, terms}` **object** instead — see the [WebSocket STT reference](/api-reference/websocket/stt). Custom vocabulary boost. CSV with optional `:weight` per term, e.g. `"Acme:5,XYZ Pharma:8,off-side"`. Default weight `1.5`, max `10`. Used to bias the recognizer toward acoustically-similar but spelled-differently words (brand names, jargon). Up to 30 entries are surfaced to the LLM hint; all entries participate in fuzzy / phonetic matching. Words replaced by the boost appear in the response with `boosted: true` and `original`. Constrain language-ID candidates. CSV of ISO 639-1 codes, e.g. `"en,hi"`. Narrower lists are faster. When omitted, the full supported set is used. Diarization tuning. Lower bound on detected speaker count. Read only when `diarize=true`. Values `<= 0` are clamped to `null`. Diarization tuning. Upper bound on detected speaker count. Read only when `diarize=true`. `"none"` | `"word"`. Set to `"word"` to add `start` / `end` to each entry in `words` and `segments[].words`. When `true`, adds a per-word `confidence` (0-1) to the response. Devanagari / Latin script normalization for code-mixed audio. LID flapping detector tuning — how aggressively to split on language change. Default safe; expose only for advanced users. ## Response The endpoint passes through the response shape from the 60db STT backend. Key fields: Unique request identifier Full normalized transcript (digit, entity, and bidi normalization applied) Detected or specified ISO 639-1 language code (e.g. `"en"`). `null` when no speech was detected. Full English language name (e.g. `"English"`) How the language was resolved: `"fast_path"` (caller specified a single language), `"lid_per_segment"` (auto-detected per segment), `"long_audio_chunked"` (file > 90 s ran through the chunker — debug-only), or `"mixed"`. Audio signal-to-noise ratio in decibels. Useful as an audio-quality indicator: `>= 15` good, `0–15` fair, `< 0` poor. When the recording is too noisy, the audio is dropped before LID/ASR — the response has empty `text` and `warning_codes` includes `low_snr_dropped` (no charge). Audio duration in seconds Server processing time in milliseconds Real-time factor (processing\_ms / (duration\_sec × 1000)) Array of utterance-level segments. Each segment has `{start, end, language, language_name, text, confidence, words[]}`. When `diarize=true`, segments also include a `speakers` array. When the request ran through the long-audio chunker (`language_source == "long_audio_chunked"`), each segment also includes a debug-only `chunk_idx` integer (zero-based index of the chunk this segment came from). Flat word-level list across all segments. Each word has `{word, start, end, confidence?, boosted?, original?}`. `confidence` is included when `include_confidence=true`. `boosted: true` and `original` are present when the keyword/context-terms boost replaced this word — the segment-level `text` is already rebuilt from boosted words upstream so no client-side stitching is required. Non-fatal warnings. Each item has `{code, message, affected_segments}`. Common codes include `no_speech_detected`, `inline_code_switch_partial`, `low_snr_dropped`, `llm_refinement_not_in_plan`. Flat list of the `code` values from `warnings`, for quick checks. | Code | Meaning | | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `no_speech_detected` | Audio processed but contained no speech (silence / music). Not an error. | | `low_snr_dropped` | Audio was dropped before LID/ASR because SNR was below the floor. Response `text` is empty. **No credits charged for this request.** | | `llm_refinement_not_in_plan` | The `context` field was provided but the active plan does not include LLM refinement. The field was ignored; transcription proceeded without refinement. | | `inline_code_switch_partial` | Some words inside a segment were transcribed in a different language than the segment label. | Internal language detection metadata: `{mode, candidates[], segment_count, lid_calls}` ```bash cURL theme={null} curl -X POST https://api.60db.ai/stt \ -H "Authorization: Bearer your-api-key" \ -F "file=@recording.mp3" \ -F "language=auto" \ -F "diarize=true" \ -F "context=Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique." ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const file = document.querySelector('input[type="file"]').files[0]; const result = await client.speechToText(file, { // Omit language or pass "auto" for auto-detect language: "auto", diarize: true, // Optional: free-form context string enables LLM refinement on // proper nouns, filler removal, and punctuation. context: "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.", }); console.log("Transcription:", result.text); console.log("Detected language:", result.language); for (const seg of result.segments || []) { const speaker = seg.speakers?.[0]?.speaker ?? "unknown"; console.log(`[${speaker}] ${seg.text}`); } ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') with open('recording.mp3', 'rb') as audio_file: result = client.speech_to_text( audio_file, language='auto', # omit or 'auto' for auto-detect diarize=True, # Optional: free-form context string enables LLM refinement on # proper nouns, filler removal, and punctuation. context='Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.', ) print(f"Transcription: {result['text']}") print(f"Language: {result['language']} ({result['language_name']})") print(f"Duration: {result['duration_sec']}s") ``` ```json Response theme={null} { "request_id": "req_6822626d028743f5942526e0c08fa60c", "language": "en", "language_name": "English", "languages": null, "language_source": "fast_path", "duration_sec": 5.2, "processing_ms": 185, "rtf": 0.036, "text": "Hello, this is a test of the speech to text API. It works great!", "segments": [ { "start": 0.0, "end": 3.1, "language": "en", "language_name": "English", "text": "Hello, this is a test of the speech to text API.", "confidence": 0.92, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 }, { "word": "this", "start": 0.35, "end": 0.52, "confidence": 0.93 } ], "speakers": [ { "speaker": "SPEAKER_00", "start": 0.0, "end": 3.1 } ] }, { "start": 3.1, "end": 5.2, "language": "en", "language_name": "English", "text": "It works great!", "confidence": 0.89, "words": [], "speakers": [ { "speaker": "SPEAKER_01", "start": 3.1, "end": 5.2 } ] } ], "words": [], "warnings": [], "warning_codes": [], "language_detection": { "mode": "fast_path", "candidates": ["en"], "segment_count": 2, "lid_calls": 0 } } ``` ```json No speech detected theme={null} { "request_id": "req_...", "language": null, "language_name": null, "languages": null, "language_source": "fast_path", "duration_sec": 0.5, "processing_ms": 185, "rtf": 0.37, "text": "", "segments": [{ "start": 0.0, "end": 0.5, "text": "", "confidence": 0.85, "words": [] }], "words": [], "warnings": [ { "code": "no_speech_detected", "message": "No speech detected in audio", "affected_segments": [] } ], "warning_codes": ["no_speech_detected"], "language_detection": { "mode": "fast_path", "candidates": ["en"], "segment_count": 1, "lid_calls": 0 } } ``` ## Errors | Status | `error_code` | When | Retry guidance | | ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | 401 | `UNAUTHENTICATED` | Auth missing or invalid | Don't retry without credentials | | 402 | `INSUFFICIENT_CREDITS` / `ZERO_BALANCE` | Workspace wallet | Top up | | 429 | `STT_CONCURRENCY_LIMIT` | Per-user concurrency cap reached (counted across REST + WS combined). `details.limit` carries the active cap. | Retry after an in-flight request finishes; do not auto-retry without backoff | | 429 | `STT_UPSTREAM_RATE_LIMIT` | Upstream STT service is rate-limiting | Honor the **`Retry-After`** HTTP header (seconds) rather than the JSON body | | 499 | `STT_CLIENT_CANCELLED` | Client closed the connection before the response was returned. The upstream call was aborted. | Intentional; no retry. **No charge.** | | 503 | `STT_UPSTREAM_UNAVAILABLE` | Upstream STT service returned a 5xx | Retry with exponential backoff (1s → 2s → 4s …) | ```json 429 STT_CONCURRENCY_LIMIT theme={null} { "success": false, "error_code": "STT_CONCURRENCY_LIMIT", "message": "Too many concurrent STT requests for this user", "details": { "limit": 8, "retry_hint": "Wait for an in-flight request to complete" } } ``` ```http 429 STT_UPSTREAM_RATE_LIMIT theme={null} HTTP/1.1 429 Too Many Requests Retry-After: 30 Content-Type: application/json { "success": false, "error_code": "STT_UPSTREAM_RATE_LIMIT", "message": "Upstream STT rate-limited", "details": { "retry_after": "30" } } ``` ```json 499 STT_CLIENT_CANCELLED theme={null} { "success": false, "error_code": "STT_CLIENT_CANCELLED", "message": "Request cancelled by client" } ``` ```json 503 STT_UPSTREAM_UNAVAILABLE theme={null} { "success": false, "error_code": "STT_UPSTREAM_UNAVAILABLE", "message": "Upstream STT service unavailable" } ``` `499` is non-standard but is used to signal "client closed connection before response was sent" (nginx convention). Treat as expected when the user cancels — wire `AbortController.abort()` to your cancel button. ## Notes * **Auto-detect**: The most reliable way to enable auto-detect is to **omit the `language` field entirely**. Passing `"auto"` is also accepted and treated identically. * **Speaker labels**: When `diarize=true`, raw speaker IDs look like `SPEAKER_00`, `SPEAKER_01`. Client UIs typically re-label these as "Speaker 1", "Speaker 2" in order of first appearance for readability. * **Empty transcript**: A successful response with `text: ""` and `warning_codes: ["no_speech_detected"]` means the upload was processed but contained no speech (silence, noise, or music only). This is **not** an error — do not retry. * **Low-SNR drop**: A response with `text: ""` and `warning_codes: ["low_snr_dropped"]` means the audio was rejected upstream before LID/ASR because it was too noisy. **No credits are charged.** Surface "Audio too noisy — try recording in a quieter environment" rather than treating as a real result. * **Diarization surcharge**: When `diarize=true` is passed (or speakers are detected in the response), the request incurs a +30% surcharge on top of the base STT rate to cover GPU diarization cost. * **Refunds**: Empty transcripts caused by `low_snr_dropped`, client-aborted requests (`499`), and upstream errors (`503`) are not charged. # API Playground Troubleshooting Source: https://docs.60db.ai/api-reference/troubleshooting Common issues and solutions when using the API playground # API Playground Troubleshooting The API playground lets you test endpoints directly in the browser. Here are solutions to common issues. ## "Missing required fields to send a playground request" This error occurs when required parameters haven't been filled in. Here's how to fix it: ### For GET Requests GET requests typically only need your API key: 1. **Enter your API key** in the Authorization field 2. Click **Send** Example working GET requests: * `GET /models` * `GET /voices` * `GET /developer/api` ### For POST Requests POST requests need additional body parameters: 1. **Enter your API key** in the Authorization field 2. **Fill in all required body parameters** (marked with `required`) 3. Click **Send** Example: For `POST /tts-synthesize`, you must fill in: * `text` (required) - The text to convert to speech * `voice_id` (optional) - Voice to use * `speed` (optional) - Speech speed ### For Path Parameters Some endpoints have path parameters like `{id}`: 1. **Enter your API key** in the Authorization field 2. **Fill in the path parameter** (e.g., the API key ID) 3. Click \*\*Send\` Example: `DELETE /developer/api/{id}` requires the API key ID in the URL. ## Billing & Credits Issues ### Issue: "402 Insufficient credits" or "ZERO\_BALANCE" **Cause**: The workspace wallet has no funds **Solution**: * Check your workspace balance in the [Dashboard](https://app.60db.ai) billing page * Add funds via the Dashboard billing page * Each workspace has its own wallet — make sure you're checking the right workspace ### Issue: "403 Forbidden" on billing endpoints **Cause**: Only workspace **owners** can access billing endpoints **Solution**: * Verify you are the workspace owner (not admin/developer/member) * Contact the workspace owner to manage billing ### Issue: "Workspace not found" on API calls **Cause**: The API key or JWT token is not associated with a valid workspace **Solution**: * If using API keys: verify the key is active and linked to a workspace * If using JWT: include `workspace_id` in the request body or query params * Check that the workspace hasn't been deleted ## STT / TTS Limits & Errors ### Issue: "429 STT\_CONCURRENCY\_LIMIT" or "429 TTS\_CONCURRENCY\_LIMIT" **Cause**: You have reached your per-user concurrency cap (counted across REST + WS combined). Defaults: STT 8, TTS 5. **Solution**: * Wait for an in-flight request or session to finish, then retry. * Do **not** auto-retry without backoff — the limit only releases when an in-flight call completes. * The error body includes `details.limit` with the active cap value. ### Issue: "429 STT\_UPSTREAM\_RATE\_LIMIT" **Cause**: The upstream STT service is rate-limiting all 60db traffic. **Solution**: * Read the **`Retry-After`** HTTP header (seconds) and back off for that duration before retrying. The header takes precedence over the JSON body's `details.retry_after`. ### Issue: "503 STT\_UPSTREAM\_UNAVAILABLE" **Cause**: The upstream STT service returned a 5xx. **Solution**: * Retry with exponential backoff (1s → 2s → 4s …). Treat as transient, not a user-input error. * No credits are charged for upstream errors. ### Issue: "499 STT\_CLIENT\_CANCELLED" **Cause**: The client closed the connection before the response was returned. The upstream call was aborted. **Solution**: * Intentional — wire `AbortController.abort()` to your cancel button. **No credits are charged.** Don't retry; don't show an error toast. ### Issue: WebSocket closed with code `1008` and `STT_CONCURRENCY_LIMIT` / `TTS_CONCURRENCY_LIMIT` **Cause**: Concurrency cap reached on the WebSocket path. **Solution**: * Wait for an in-flight session to finish, then reconnect. Do not auto-reconnect immediately on `1008`. ## Common Issues ### Issue: "401 Unauthorized" **Cause**: Invalid or missing API key **Solution**: * Verify your API key is correct * Get a new API key from [app.60db.ai](https://app.60db.ai) * Make sure you're including the key in the Authorization field ### Issue: "CORS error" **Cause**: Browser security blocking the request **Solution**: * Make sure you're testing from the documentation site * Try refreshing the page * Clear your browser cache ### Issue: Request times out **Cause**: Network issues or server problems **Solution**: * Check your internet connection * Try again in a few moments * Check if the API is operational ## Getting Your API Key 1. Go to [app.60db.ai](https://app.60db.ai) 2. Navigate to **Settings → Developer → API Keys** 3. Click **Create API Key** 4. Copy and store your API key securely ## Tips for Using the Playground 1. **Start with GET requests** - They're simpler and only need an API key 2. **Check required fields** - All required fields must be filled before sending 3. **Use example values** - The playground shows example values to help you get started 4. **Test incrementally** - Try one endpoint at a time to understand the API ## Still Having Issues? If you're still experiencing problems: * Check the [Authentication](/authentication) documentation * Contact support at [support@60db.com](mailto:support@60db.com) * Open an issue on [GitHub](https://github.com/60db-ai) # Text to Speech Source: https://docs.60db.ai/api-reference/tts/text-to-speech POST /tts-synthesize Convert text to natural-sounding speech ## Request ### Headers Bearer token with your API key application/json ### Body The text to convert to speech (max 5000 characters) ID of the voice to use. Fetch available voices from `GET /voices`. Nested audio configuration block (matches the upstream Inworld schema). Audio encoding for the streamed response. Options: `LINEAR16`, `OGG_OPUS`. Output sample rate. Options: `16000`, `24000`, `48000`. Speaking rate. Range `0.5` (slow) – `2.0` (fast). `1.0` = normal. Voice consistency `0`–`100`. `0` = expressive, `50` = balanced, `100` = consistent. Voice match fidelity `0`–`100`. `0` = loose, `75` = strong, `100` = exact clone. Set to `"WORD"` to receive per-word timestamps in the final NDJSON chunk (`timestampInfo`). Omit or set to `"NONE"` to skip. Optional cross-lingual synthesis hint (e.g. `"en"`, `"hi"`, `"ar"`). Required when the voice's reference audio is in a different language than the input text. Auto-detected from voice metadata when omitted. **Legacy flat keys (`sample_rate`, `audio_encoding`) are still accepted** for backwards-compatibility, but new integrations should send the nested `audio_config` object. ## Response Indicates if the request was successful Status message Base64-encoded audio data Audio sample rate in Hz Duration of the audio in seconds Audio encoding format Audio output format ```bash cURL theme={null} curl -X POST https://api.60db.ai/tts-synthesize \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "text": "Hello, this is a test of the Inworld-compatible text-to-speech API with streaming.", "voice_id": "038cf0d1-eef8-45a6-81b0-99c5e57a33d2", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 24000 }, "speed": 1, "stability": 50, "similarity": 75, "timestamp_type": "WORD" }' ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const audio = await client.textToSpeech({ text: "Hello, this is a test of the Inworld-compatible text-to-speech API with streaming.", voice_id: "038cf0d1-eef8-45a6-81b0-99c5e57a33d2", // model_id: "indic_tts_v1", audio_config: { audio_encoding: "LINEAR16", sample_rate_hertz: 24000, }, speed: 1, stability: 50, similarity: 75, timestamp_type: "WORD", }); // audio is an ArrayBuffer ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') audio = client.text_to_speech( text='Hello, this is a test of the Inworld-compatible text-to-speech API with streaming.', voice_id='038cf0d1-eef8-45a6-81b0-99c5e57a33d2', # model_id='indic_tts_v1', audio_config={ 'audio_encoding': 'LINEAR16', 'sample_rate_hertz': 24000, }, speed=1, stability=50, similarity=75, timestamp_type='WORD', ) # Save to file with open('output.wav', 'wb') as f: f.write(audio) ``` ```json Response theme={null} { "success": true, "message": "Audio generated successfully", "audio_base64": "SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4Ljc2LjEwMAAAAAAAAAAAAAAA...", "sample_rate": 24000, "duration_seconds": 3.5, "encoding": "mp3", "output_format": "mp3" } ``` # Get TTS Languages Source: https://docs.60db.ai/api-reference/tts/text-to-speech-languages GET /tts/languages Retrieve the list of voice languages supported by the Text-to-Speech service Returns the catalog of voice languages available for TTS synthesis. This list is distinct from the STT transcription languages returned by [`GET /stt/languages`](/api-reference/stt/get-languages) — TTS supports a smaller set of voice-tuned languages (\~30) whereas STT supports 39 transcription languages plus an `auto` auto-detect option. Use the `language_id` values from this endpoint when you filter voices by language or when you create custom / professional cloned voices. ## Request ### Headers Bearer token with your API key ## Response Always `true` on 200 responses Status message (`"TTS languages retrieved successfully"`) Flat array of supported TTS voice languages ISO 639-1 language code (e.g. `"en"`, `"hi"`, `"fr"`). Use this value when passing a `language` field to any voice-creation or voice-filter endpoint. Human-readable language name (e.g. `"English"`, `"Hindi"`) ```bash cURL theme={null} curl https://api.60db.ai/tts/languages \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const res = await client.getTTSLanguages(); for (const lang of res.data) { console.log(`${lang.language_id} — ${lang.name}`); } ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') res = client.get_tts_languages() for lang in res['data']: print(f"{lang['language_id']} — {lang['name']}") ``` ```json Response theme={null} { "success": true, "message": "TTS languages retrieved successfully", "data": [ { "language_id": "en", "name": "English" }, { "language_id": "hi", "name": "Hindi" }, { "language_id": "as", "name": "Assamese" }, { "language_id": "bn", "name": "Bengali" }, { "language_id": "gu", "name": "Gujarati" }, { "language_id": "kn", "name": "Kannada" }, { "language_id": "ml", "name": "Malayalam" }, { "language_id": "mr", "name": "Marathi" }, { "language_id": "ne", "name": "Nepali" }, { "language_id": "or", "name": "Odia" }, { "language_id": "pa", "name": "Punjabi" }, { "language_id": "ta", "name": "Tamil" }, { "language_id": "te", "name": "Telugu" }, { "language_id": "de", "name": "German" }, { "language_id": "nl", "name": "Dutch" }, { "language_id": "fr", "name": "French" }, { "language_id": "es", "name": "Spanish" }, { "language_id": "it", "name": "Italian" }, { "language_id": "pt", "name": "Portuguese" }, { "language_id": "pl", "name": "Polish" }, { "language_id": "ar", "name": "Arabic" }, { "language_id": "ja", "name": "Japanese" }, { "language_id": "ko", "name": "Korean" }, { "language_id": "tr", "name": "Turkish" }, { "language_id": "vi", "name": "Vietnamese" }, { "language_id": "ru", "name": "Russian" }, { "language_id": "id", "name": "Indonesian" }, { "language_id": "th", "name": "Thai" }, { "language_id": "fil", "name": "Filipino" }, { "language_id": "uk", "name": "Ukrainian" } ] } ``` ## Notes * **TTS ≠ STT language list.** TTS voice synthesis supports a curated set of voice-tuned languages. If you need to transcribe audio in a language that is not in this list, use the STT API — the supported language sets are deliberately different. * The endpoint is a transparent wrapper around the upstream TTS service's `/v1/languages`, so the `data` array is the raw upstream response. Do not rely on the order being alphabetical — sort on the client side if needed. * Languages returned here can be passed as the `language` field when creating a cloned voice via [`POST /voices`](/api-reference/voices/create-voice) or a professional voice via [`POST /voices/professional`](/api-reference/voices/create-voice). # Text to Speech Stream Source: https://docs.60db.ai/api-reference/tts/text-to-speech-stream POST /tts-stream Stream text to speech with real-time audio chunks ## Request ### Headers Bearer token with your API key application/json ### Body The text to convert to speech (max 5000 characters) ID of the voice to use Enable audio enhancement Speech speed multiplier (0.5 to 2.0) Voice stability 0-100 (lower = more expressive, higher = more consistent) Voice similarity 0-100 (how closely the output matches the source voice) ## Response The response is streamed as newline-delimited JSON (NDJSON). Each line contains a JSON object: ### Chunk Object Type of message: "chunk", "complete", or "error" Contains the audio chunk data Base64-encoded audio chunk Error message (only for error type) ```bash cURL theme={null} curl -X POST https://api.60db.ai/tts-stream \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "text": "This is a longer text that will be streamed in real-time.", "voice_id": "default-voice", "speed": 1, "stability": 50, "similarity": 75 }' \ --no-buffer ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); await client.textToSpeechStream( { text: "This is a longer text that will be streamed in real-time.", voice_id: "default-voice", speed: 1, stability: 50, similarity: 75, }, { onChunk: (chunk) => { console.log("Received chunk:", chunk.length, "bytes"); // Play or process the audio chunk }, onComplete: () => { console.log("Streaming complete"); }, onError: (error) => { console.error("Error:", error); }, }, ); ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') def handle_chunk(chunk): print(f"Received {len(chunk)} bytes") # Play or process the audio chunk def handle_complete(): print("Streaming complete") def handle_error(error): print(f"Error: {error}") client.text_to_speech_stream( text='This is a longer text that will be streamed in real-time.', on_chunk=handle_chunk, on_complete=handle_complete, on_error=handle_error, voice_id='default-voice', speed=1, stability=50, similarity=75 ) ``` ```json Chunk Response theme={null} {"type":"chunk","result":{"audioContent":"SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4..."}} {"type":"chunk","result":{"audioContent":"//uQxAAAAAAAAAAAAAAASW5mbwAAAA8AAAAGAAA..."}} {"type":"complete"} ``` ```json Error Response theme={null} { "type": "error", "message": "Invalid voice_id" } ``` ## Use Cases Streaming is ideal for: * **Real-time applications**: Voice assistants, chatbots * **Long-form content**: Articles, books, documents * **Low latency**: Start playing audio before generation completes * **Progressive enhancement**: Display text while generating audio # Create Voice Source: https://docs.60db.ai/api-reference/voices/create-voice POST /voices Create a custom voice from audio samples ## Request ### Headers Bearer token with your API key multipart/form-data ### Form Data Name for the custom voice Audio files for voice cloning (minimum 3, maximum 10 files). Each file should be: - Format: MP3, WAV, or FLAC - Duration: 10-60 seconds each - Quality: Clear speech, minimal background noise - Total duration: At least 2 minutes combined Description of the voice Primary language code (e.g., "en", "es") Voice gender: "male", "female", or "neutral" ## Response Unique identifier for the created voice Voice name Processing status: "processing", "ready", "failed" ISO 8601 timestamp Estimated time for voice processing to complete ```bash cURL theme={null} curl -X POST https://api.60db.ai/voices \ -H "Authorization: Bearer your-api-key" \ -F "name=My Custom Voice" \ -F "description=Professional voice for my brand" \ -F "language=en" \ -F "gender=female" \ -F "files=@sample1.mp3" \ -F "files=@sample2.mp3" \ -F "files=@sample3.mp3" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const files = [ document.querySelector("#file1").files[0], document.querySelector("#file2").files[0], document.querySelector("#file3").files[0], ]; const voice = await client.createVoice({ name: "My Custom Voice", description: "Professional voice for my brand", language: "en", gender: "female", files: files, }); console.log("Voice ID:", voice.id); ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') files = [ open('sample1.mp3', 'rb'), open('sample2.mp3', 'rb'), open('sample3.mp3', 'rb') ] voice = client.create_voice( name='My Custom Voice', files=files, description='Professional voice for my brand', language='en', gender='female' ) print(f"Voice ID: {voice['id']}") # Close files for f in files: f.close() ``` ```json Response theme={null} { "id": "voice-custom-123", "name": "My Custom Voice", "description": "Professional voice for my brand", "language": "en", "gender": "female", "status": "processing", "is_custom": true, "created_at": "2026-01-29T11:30:00Z", "estimated_completion": "2026-01-29T11:45:00Z" } ``` ## Voice Cloning Best Practices * Use high-quality recordings (at least 44.1kHz sample rate) - Ensure minimal background noise - Avoid music or sound effects - Use consistent recording environment * Include varied sentence structures - Cover different emotions and tones - Include questions, statements, and exclamations - Avoid repetitive content * Minimum 3 files, maximum 10 files - Each file: 10-60 seconds - Total duration: At least 2 minutes - Supported formats: MP3, WAV, FLAC Voice processing typically takes 10-15 minutes. You'll receive a webhook notification when your voice is ready to use. # Delete Voice Source: https://docs.60db.ai/api-reference/voices/delete-voice DELETE /voices/:id Delete a custom voice ## Request ### Path Parameters The unique identifier of the voice to delete ### Headers Bearer token with your API key This action cannot be undone. Only custom voices can be deleted. ## Response Indicates successful deletion Confirmation message ```bash cURL theme={null} curl -X DELETE https://api.60db.ai/voices/voice-custom-123 \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} await client.deleteVoice("voice-custom-123"); ``` ```python Python theme={null} client.delete_voice('voice-custom-123') ``` ```json Response theme={null} { "success": true, "message": "Voice deleted successfully" } ``` # Get My Voices Source: https://docs.60db.ai/api-reference/voices/get-my-voices GET /myvoices Retrieve all voices created by the authenticated user ## Request ### Headers Bearer token with your API key ## Response Indicates whether the request was successful Response message describing the result Array of voice objects created by the authenticated user. Returns an empty array if no voices have been created. Unique voice identifier (UUID format) Display name of the voice Voice category: `"cloned"` or `"professional"` TTS model used by the voice: `"60db Fast"` or `"60db Quality"` Metadata labels describing the voice attributes ISO language code (e.g., `"en"`, `"hi"`, `"ar"`, `"fr"`, `"de"`, `"es"`, `"it"`, `"nl"`, `"pl"`, `"pt"`, `"bn"`, `"gu"`, `"kn"`, `"ml"`, `"mr"`, `"pa"`, `"ta"`, `"te"`) Full language name (e.g., `"English"`, `"Hindi"`, `"Arabic"`) Voice gender: `"male"` or `"female"` Voice accent: `"American"`, `"British"`, `"Indian"`, or `"Neutral"` Optional description of the voice. `null` if not provided. Whether the voice is a native/platform-provided voice. Always `false` for user-created voices. Tier availability list. Currently returns an empty array for all voices. Recommended use case categories for the voice. Possible values include: `"Entertainment/TV"`, `"IVR/Call Center"`, `"Finance/Banking"`, `"Conversational"`, `"Medical/Healthcare"`, `"Religious/Spiritual"`, `"Corporate/Business"`, `"Government/Public Sector"`, `"Social Media"`, `"Documentary"`, `"Travel/Tourism"`, `"Kids/Children"`, `"Legal"`, `"Audiobook"`, `"Gaming"`, `"Podcast"`, `"Advertisement"`, `"E-Learning"`, `"News/Journalism"`, `"Sports"`, `"Animation"`, `"Professional Cloning"` ```bash cURL theme={null} curl https://api.60db.ai/myvoices \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); const response = await client.getMyVoices(); console.log(response.data); ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') response = client.get_my_voices() for voice in response['data']: print(f"{voice['name']} - {voice['labels']['language_name']} ({voice['model']})") ``` ```json Response theme={null} { "success": true, "message": "User voices fetched successfully", "data": [ { "voice_id": "7c32dd99-d61d-493b-9601-37e9464be6f4", "name": "myvoice", "model": "60db Fast", "labels": { "language": "en", "language_name": "English", "gender": "male", "accent": "Indian" }, "description": "this is my voice", "categories": [ "Entertainment/TV" ] }, { "voice_id": "b859ebca-f7e9-4fc0-a934-2b32ded9a4c4", "name": "Monika - Natural and Calm", "model": "60db Quality", "labels": { "language": "en", "language_name": "English", "gender": "female", "accent": "Indian" }, "description": null, "categories": [ "Professional Cloning", "Entertainment/TV", "IVR/Call Center", "Corporate/Business", "Social Media", "Documentary", "Medical/Healthcare" ] } ] } ``` # Get Voice by ID Source: https://docs.60db.ai/api-reference/voices/get-voice GET /voices/:id Retrieve details of a specific voice by its ID ## Request ### Path Parameters The unique identifier of the voice ### Headers Bearer token with your API key ## Response Unique voice identifier Voice name Voice description Language code Voice gender Whether this is a custom voice URL to voice sample ISO 8601 timestamp ```bash cURL theme={null} curl https://api.60db.ai/voices/voice-001 \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const voice = await client.getVoice("voice-001"); console.log(voice); ``` ```python Python theme={null} voice = client.get_voice('voice-001') print(voice) ``` ```json Response theme={null} { "id": "voice-001", "name": "Sarah", "description": "Professional female voice with American accent", "language": "en-US", "gender": "female", "age": "middle", "accent": "American", "use_case": ["narration", "customer-service"], "sample_url": "https://cdn.60db.com/samples/voice-001.mp3", "is_custom": false, "created_at": "2026-01-15T10:00:00Z" } ``` # Get Voices Source: https://docs.60db.ai/api-reference/voices/get-voices GET /voices Retrieve voices for the authenticated workspace, filtered by model tier ## Request ### Headers Bearer token with your API key ### Query Parameters Voice model tier to return. Accepted values: * `"quality"` (default) — returns `60db Quality` professional voices * `"fast"` — returns `60db Fast` cloned voices Any other value falls back to `"quality"`. ## Response Indicates whether the request was successful Response message describing the result Array of voice objects matching the requested model tier Resolved model tier applied to the response: `"quality"` or `"fast"` Unique voice identifier (UUID format) Display name of the voice TTS model used by the voice: `"60db Fast"` or `"60db Quality"` Metadata labels describing the voice attributes ISO language code (e.g., `"en"`, `"hi"`, `"ar"`, `"fr"`, `"de"`, `"es"`, `"it"`, `"nl"`, `"pl"`, `"pt"`, `"bn"`, `"gu"`, `"kn"`, `"ml"`, `"mr"`, `"pa"`, `"ta"`, `"te"`) Full language name (e.g., `"English"`, `"Hindi"`, `"Arabic"`) Voice gender: `"male"` or `"female"` Voice accent: `"American"`, `"British"`, `"Indian"`, or `"Neutral"` Optional description of the voice. `null` if not provided. Recommended use case categories for the voice. Possible values include: `"Entertainment/TV"`, `"IVR/Call Center"`, `"Finance/Banking"`, `"Conversational"`, `"Medical/Healthcare"`, `"Religious/Spiritual"`, `"Corporate/Business"`, `"Government/Public Sector"`, `"Social Media"`, `"Documentary"`, `"Travel/Tourism"`, `"Kids/Children"`, `"Legal"`, `"Audiobook"`, `"Gaming"`, `"Podcast"`, `"Advertisement"`, `"E-Learning"`, `"News/Journalism"`, `"Sports"`, `"Animation"`, `"Professional Cloning"` ```bash cURL theme={null} # Default (quality) curl https://api.60db.ai/voices \ -H "Authorization: Bearer your-api-key" # Fast tier curl "https://api.60db.ai/voices?model=fast" \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} import { SixtyDBClient } from "60db"; const client = new SixtyDBClient("your-api-key"); // Defaults to quality const quality = await client.getVoices(); // Fast tier const fast = await client.getVoices({ model: "fast" }); ``` ```python Python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') # Defaults to quality response = client.get_voices() # Fast tier response = client.get_voices(model='fast') for voice in response['data']: print(f"{voice['name']} - {voice['labels']['language_name']} ({voice['model']})") ``` ```json Quality (default) theme={null} { "success": true, "message": "Voices fetched successfully", "data": [ { "voice_id": "84982e79-c596-4883-b6fa-9af334767aef", "name": "Simmi", "model": "60db Quality", "labels": { "language": "hi", "language_name": "Hindi", "gender": "female", "accent": "Indian" }, "description": null, "categories": [ "Entertainment/TV", "IVR/Call Center", "Documentary" ] } ], "meta": { "model": "quality" } } ``` ```json Fast theme={null} { "success": true, "message": "Voices fetched successfully", "data": [ { "voice_id": "fbb75ed2-975a-40c7-9e06-38e30524a9a1", "name": "Zara", "model": "60db Fast", "labels": { "language": "hi", "language_name": "Hindi", "gender": "female", "accent": "Indian" }, "description": null, "categories": [ "Entertainment/TV", "IVR/Call Center", "Finance/Banking", "Conversational" ] } ], "meta": { "model": "fast" } } ``` # Update Voice Source: https://docs.60db.ai/api-reference/voices/update-voice PUT /voices/:id Update a custom voice's metadata ## Request ### Path Parameters The unique identifier of the voice ### Headers Bearer token with your API key application/json ### Body Updated voice name Updated description Only custom voices can be updated. System voices cannot be modified. ## Response Voice identifier Updated voice name Updated description ISO 8601 timestamp ```bash cURL theme={null} curl -X PUT https://api.60db.ai/voices/voice-custom-123 \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "name": "Updated Voice Name", "description": "Updated description" }' ``` ```javascript JavaScript theme={null} await client.updateVoice("voice-custom-123", { name: "Updated Voice Name", description: "Updated description", }); ``` ```python Python theme={null} client.update_voice( voice_id='voice-custom-123', name='Updated Voice Name', description='Updated description' ) ``` ```json Response theme={null} { "id": "voice-custom-123", "name": "Updated Voice Name", "description": "Updated description", "updated_at": "2026-01-29T11:35:00Z" } ``` # Create Webhook Source: https://docs.60db.ai/api-reference/webhooks/create-webhook POST /webhooks Create a new webhook ## Request ### Headers Bearer token with your API key application/json ### Body Webhook URL (must be HTTPS) Array of event types to subscribe to Optional secret for webhook signature verification ## Available Events * `tts.completed` - Text-to-speech generation completed * `tts.failed` - Text-to-speech generation failed * `stt.completed` - Speech-to-text transcription completed * `stt.failed` - Speech-to-text transcription failed * `voice.created` - Custom voice created * `voice.ready` - Custom voice processing completed * `voice.failed` - Custom voice processing failed ## Response Webhook ID Webhook URL Subscribed events Webhook status Creation timestamp ```bash cURL theme={null} curl -X POST https://api.60db.ai/webhooks \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/webhook", "events": ["tts.completed", "stt.completed"], "secret": "your-webhook-secret" }' ``` ```javascript JavaScript theme={null} const webhook = await client.createWebhook({ url: "https://example.com/webhook", events: ["tts.completed", "stt.completed"], secret: "your-webhook-secret", }); ``` ```python Python theme={null} webhook = client.create_webhook( url='https://example.com/webhook', events=['tts.completed', 'stt.completed'], secret='your-webhook-secret' ) ``` ```json Response theme={null} { "id": "wh-456", "url": "https://example.com/webhook", "events": ["tts.completed", "stt.completed"], "status": "active", "created_at": "2026-01-29T11:35:00Z" } ``` ## Webhook Payload When an event occurs, we'll send a POST request to your webhook URL: ```json theme={null} { "event": "tts.completed", "timestamp": "2026-01-29T11:35:00Z", "data": { "id": "tts-123", "text": "Hello, world!", "voice_id": "default-voice", "audio_url": "https://cdn.60db.com/audio/tts-123.mp3", "duration": 2.5 } } ``` # Delete Webhook Source: https://docs.60db.ai/api-reference/webhooks/delete-webhook DELETE /webhooks/:id Delete a webhook ## Request ### Path Parameters The ID of the webhook to delete ### Headers Bearer token with your API key ## Response Indicates successful deletion Confirmation message ```bash cURL theme={null} curl -X DELETE https://api.60db.ai/webhooks/wh-123 \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} await client.deleteWebhook("wh-123"); ``` ```python Python theme={null} client.delete_webhook('wh-123') ``` ```json Response theme={null} { "success": true, "message": "Webhook deleted successfully" } ``` # Get Webhooks Source: https://docs.60db.ai/api-reference/webhooks/get-webhooks GET /webhooks List all webhooks ## Request ### Headers Bearer token with your API key ## Response Array of webhook objects Webhook ID Webhook URL Subscribed events Webhook status: "active" or "inactive" Creation timestamp ```bash cURL theme={null} curl https://api.60db.ai/webhooks \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const webhooks = await client.getWebhooks(); ``` ```python Python theme={null} webhooks = client.get_webhooks() ``` ```json Response theme={null} { "webhooks": [ { "id": "wh-123", "url": "https://example.com/webhook", "events": ["tts.completed", "stt.completed"], "status": "active", "created_at": "2026-01-15T10:00:00Z" } ] } ``` # WebSocket Introduction Source: https://docs.60db.ai/api-reference/websocket/introduction Overview of 60db WebSocket API for real-time STT and TTS # WebSocket API Introduction The 60db WebSocket API provides real-time, bidirectional streaming for Speech-to-Text (STT) and Text-to-Speech (TTS) services. ## Base URL ``` ws://api.60db.ai/ws ``` ## Available Endpoints | Endpoint | Description | | ------------ | ------------------------ | | `/ws/stt` | Speech-to-Text streaming | | `/ws/tts` | Text-to-Speech streaming | | `/ws/health` | Health check endpoint | ## Why WebSocket? WebSocket provides several advantages over REST API for audio processing: * **Real-time streaming**: Low-latency bidirectional communication * **Continuous audio**: Stream audio chunks as they're recorded * **Live transcription**: Get partial results while speaking * **Efficient**: No need to send complete audio files * **Interactive**: Natural conversation experience ## Quick Start ### 1. Connect with API Key ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key'); ``` ### 2. Handle Connection Events ```javascript theme={null} ws.onopen = () => { console.log('Connected!'); }; ws.onmessage = (event) => { const data = JSON.parse(event.data); console.log('Received:', data); }; ``` ### 3. Send Messages ```javascript theme={null} // Start STT session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'mulaw', sample_rate: 8000 } })); ``` ### 4. Send Binary Data ```javascript theme={null} // Send audio bytes ws.send(audioBuffer); ``` ## Connection Flow ``` Client Server | | |----- WebSocket Connect ------->| | | |<-- Authentication Message ----| | | |----- Start Session Message --->| | | |----- Binary Audio Data ------->| |----- Binary Audio Data ------->| |----- Binary Audio Data ------->| | | |<--- Transcription Results ----| |<--- Transcription Results ----| | | |----- Stop Session Message ---->| | | |<-- Billing Summary -----------| ``` ## Authentication WebSocket connections authenticate via query parameter: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key ``` Or with JWT token: ``` ws://api.60db.ai/ws/stt?token=your_jwt_token ``` ## Message Format All control messages are JSON formatted: ```json theme={null} { "type": "message_type", "data": "message_data" } ``` Binary audio data is sent as raw bytes without JSON encoding. ## Next Steps * [STT WebSocket Guide](/api-reference/websocket/stt) - Learn Speech-to-Text * [TTS WebSocket Guide](/api-reference/websocket/tts) - Learn Text-to-Speech * [Complete WebSocket Reference](/websocket-api) - Full API documentation # STT WebSocket Source: https://docs.60db.ai/api-reference/websocket/stt WebSocket /ws/stt Speech-to-Text WebSocket endpoint for real-time transcription # STT WebSocket Real-time Speech-to-Text transcription via WebSocket streaming with support for 39 languages (including code-switched Indic+English) and telephony integration. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack). ## Endpoint `ws://api.60db.ai/ws/stt` or `wss://api.60db.ai/ws/stt` ## Authentication Query parameter authentication: Your API key for authentication JWT token (alternative to API key) Workspace ID for billing. Required when using JWT auth. API keys are automatically pinned to their workspace. Examples: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key ws://api.60db.ai/ws/stt?token=eyJ...&workspace_id=24 ``` The WebSocket connection checks **workspace wallet balance** before starting a session. If the workspace has insufficient credits, the connection is closed with a `1008` status code and an `INSUFFICIENT_CREDITS` error. ## Connection Details | Property | Value | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Protocol | WebSocket (RFC 6455) | | Frame types | Binary (telephony) or Text/JSON (browser) | | Ping/keepalive | The server answers client-initiated WebSocket pings with a pong; it does not ping first. Send a ping every 20–30 s so intermediaries do not idle the socket out. | The same port accepts plain HTTP GET requests and responds `200 ok` — safe for load-balancer health checks. ## Session Lifecycle ``` Client Server │ │ │──── TCP/TLS connect ───────────────────►│ │◄─── {"type":"connecting", ...} ─────────│ authenticating... │◄─── {"type":"connected", server_info} ──│ proxy wired to upstream │◄─── {"type":"connection_established"} ──│ ready — safe to send `start` │ │ │──── {"type":"start", ...} ─────────────►│ │◄─── {"type":"session_started", ...} ────│ safe to send audio │ │ │──── audio frames / JSON audio ─────────►│ │◄─── {"type":"speech_started"} ──────────│ VAD detected voice │◄─── {"type":"transcription", ...} ──────│ is_final=true, speech_final=false (first emit, context only) │◄─── {"type":"transcription", ...} ──────│ is_final=true, speech_final=true (canonical answer) │ │ │──── {"type":"stop"} ────────────────────►│ │◄─── {"type":"session_stopped"} ─────────│ ``` **Two-phase finals (context-gated LLM refinement).** When you supply a `context` object on `start`, every utterance produces **two** `transcription` events sharing a `sentence_id`: 1. **First emit** — `is_final: true, speech_final: false` — fast dict-corrected text. Use for low-latency UI paint and barge-in. 2. **Canonical** — `is_final: true, speech_final: true` — definitive LLM-refined answer. Always arrives. When `context` is **omitted**, every utterance produces a single `transcription` event with `is_final: true, speech_final: true` (no first emit). Simple consumers can gate exclusively on `speech_final: true` regardless of whether refinement is on. **Do not send the `start` message until `connection_established` is received.** The backend proxy attaches its client-message listener only after authenticating and opening the upstream connection. Messages sent earlier will be silently dropped. Likewise, **do not send `audio` messages until `session_started` is received** — the upstream returns `unknown message type: audio` if it arrives before the session is ready. ## Client → Server Messages ### `start` — Begin session Sent once after connection is established. Must be sent before any audio. ```json Request theme={null} { "type": "start", "languages": ["en", "hi"], "context": { "general": [ { "key": "domain", "value": "Healthcare" }, { "key": "doctor", "value": "Dr. Martha Smith" } ], "text": "Routine diabetes follow-up consultation.", "terms": ["Celebrex", "Zyrtec", "Metformin", "HbA1c"] }, "config": { "encoding": "linear", "sample_rate": 48000, "utterance_end_ms": 1000, "continuous_mode": true, "interim_results_frequency": 300, "audio_enhancement": "adaptive", "diarize": false, "remove_fillers": false } } ``` **Parameters:** Array of ISO 639-1 language codes from the supported set (see `GET /stt/languages`), e.g. `["en", "hi"]`. * `["en"]` — fast path, lowest latency * `["en","hi"]` — shared-backend multi-language (the Indic+English pipeline handles both natively), no LID overhead * `null` or omit — auto-detect across all 39 v1 languages * Arabic dialect tags (`ar-eg`, `ar-lv`, …) are rejected; pass `ar` for best-effort MSA * Max 5 languages per session **Unsupported:** `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he` — these return an `unsupported_language` error. **Never send `languages: "auto"` or `languages: ["auto"]`.** The auto-detect entry in `GET /stt/languages` is a convenience for the REST `/stt` form-upload flow only. On WebSocket, the server only accepts real ISO codes and uses `null` as the auto-detect signal. Sending `"auto"` returns `language 'auto' is not in the v1 supported`. The 60db WebSocket proxy (`/ws/stt`) strips the string `"auto"` from incoming `start` and `config` messages as a safety net, but your client should send `null` directly. Optional hint object `{general, text, terms}` that opens the server-side LLM refinement gate. When supplied, each utterance emits **two** `transcription` events sharing a `sentence_id` — a fast first emit (`speech_final: false`) with dict-corrected text, followed \~300–700 ms later by a canonical emit (`speech_final: true`) with LLM-refined text (proper nouns corrected, fillers removed, punctuation added, script consistency enforced). See [Canonical-answer semantics](#canonical-answer-semantics-speech-final) below. * `general` — array of `{key, value}` pairs. Free-form metadata (`domain`, `topic`, speaker names) surfaced to the LLM verbatim as hint lines. * `text` — background paragraph describing the session. Useful for narrative context. * `terms` — array of proper nouns, acronyms, and domain-specific jargon to preserve in the transcript. All three fields are optional; at least one should be populated. Omit `context` entirely to disable refinement for the session — finals then arrive once as plain `transcription` events. Audio encoding format. Use `"linear"` for browser Int16 PCM, `"mulaw"` for G.711 telephony. Options: `"linear"`, `"mulaw"`. The first raw binary frame auto-selects `mulaw`. Actual sample rate of the audio being sent. Server resamples to 16 kHz internally with stateful `audioop.ratecv`. Options: `8000`, `16000`, `24000`, `44100`, `48000`. Must match the real capture rate — do not hardcode 16000 on 48 kHz browser input. Silence duration (ms) after last speech chunk before finalizing the utterance. **Minimum 1000 ms.** The upstream STT server *rejects* a `start` carrying a lower value (`utterance_end_ms must be at least 1000`), which would kill the session — so the 60db `/ws/stt` proxy clamps anything below 1000 ms up to 1000 ms before forwarding, on both `start` and mid-session `config`. Sending `500` is therefore accepted, but you get 1000 ms behaviour; there is no error and no way to go faster. Recommended 1000–1500 ms for voicebots; raise it for thoughtful long-form speakers. For fast barge-in, act on `speech_started` and interim results rather than lowering this value. Omit the key to take the server default of 1000 ms. Keep session alive between utterances. Required for voicebot / phone call use cases. `false` = single-shot STT that stops after the first final. How often (ms) to emit interim partial results during speech. **Values below 300 ms are silently clamped to 300 ms.** Use 300 for barge-in, 500 otherwise, omit to disable. Run pyannote speaker diarization on each finalized utterance and attach a `speakers` array. Requires `HF_TOKEN` on the server. Adds \~50–150 ms latency per final. Check `session_started.diarize` to confirm the request was accepted. Lower bound on diarization speaker count. `0` and negative values are silently clamped to `null`. Only read when `diarize=true`. Upper bound on diarization speaker count. Only read when `diarize=true`. Real-time audio enhancement to improve transcription quality on noisy input. Options: | Value | Description | | ------------ | -------------------------------------------------------------------------------------------------- | | `"off"` | No audio processing (default) | | `"light"` | Noise reduction only — best for mildly noisy environments | | `"adaptive"` | Automatic noise reduction + gain control based on input levels — best for variable/telephony audio | Use `"adaptive"` for telephony or noisy environments, `"light"` when you only need mild cleanup, and `"off"` when the input is already clean. Ask the LLM refinement pass to strip filler words (`um`, `uh`, `like`, `you know`, …) from the canonical transcript. Only takes effect when `context` is set — refinement is gated on context, and the raw first-emit still contains the fillers. Non-boolean values are coerced to `false` by the 60db proxy. Reserved for legacy client compatibility. **Ignored by 60db STT** — the non-hallucinating backend never emits a `no_speech_prob`. Safe to pass for migration convenience. ### `audio` — JSON audio chunk (browser mode) ```json Request theme={null} { "type": "audio", "audio": "", "encoding": "linear", "sample_rate": 48000, "timestamp": 1700000000000 } ``` **Fields:** Must be `"audio"` Base64-encoded audio bytes (Int16 PCM or μ-law) `"linear"` or `"mulaw"` Actual sample rate of the audio Unix ms timestamp — useful for latency measurement ### Binary frame — raw μ-law audio (telephony mode) Send a raw WebSocket binary frame with μ-law bytes, no JSON wrapper. The server auto-detects this as telephony mode on the first binary frame. ``` Recommended chunk size: 480 bytes = 60ms at 8kHz Twilio default: 160 bytes = 20ms — batch 3 chunks into 60ms before sending ``` ### `config` — Change language mid-session ```json Request theme={null} { "type": "config", "languages": ["hi"], "continuous_mode": true } ``` Both `languages` and `continuous_mode` are optional; include only fields you want to change. Send `"languages": null` to revert to auto-detect. The proxy applies the same sanitising here as on `start`: the literal string `"auto"` is stripped from `languages`, and a `config.utterance_end_ms` below 1000 ms is clamped up to 1000 ms. ### `stop` — End session ```json Request theme={null} { "type": "stop" } ``` Server processes any remaining audio buffer, sends `session_stopped`, then closes. ### `test` — Ping / latency check ```json Request theme={null} { "type": "test", "message": "ping", "timestamp": 1700000000000 } ``` Server echoes `test_response` with the same `timestamp` for round-trip measurement. ## Server → Client Messages ### `connecting` — Authentication in progress ```json Response theme={null} { "type": "connecting", "message": "Authenticating...", "timestamp": 1775465918269 } ``` ### `connection_established` — Authentication successful ```json Response theme={null} { "type": "connection_established", "service": "stt", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } ``` On **STT** these fields are **top-level** and the frame is identified by `type`. Branch on `msg.type === "connection_established"` — not on `msg.connection_established`, which is the **TTS** frame's shape (`/ws/tts` nests the same fields under a `connection_established` key). A client written against the TTS shape will silently never start its STT session. **Fields:** Service name: `"stt"` Your user ID Available credits Workspace name ### `connected` — Proxy wired to upstream STT server Sent by the backend proxy after it has opened the upstream connection and attached its client-message listener. **After this point, client messages are no longer dropped.** **You will normally see two `connected` frames**, and they carry different payloads. The proxy emits its own handshake frame first (`session_id` plus a small `server_info`), then forwards the upstream server's capability frame (the one below, with `features` and `timing`). Both are informational — treat `connection_established` as "safe to send `start`" and `session_started` as "safe to send audio", and make any `connected` handler idempotent so it does not start audio capture twice. ```json theme={null} { "type": "connected", "session_id": "0f5a4d7c-9d0f-4a5e-9d0b-1f2c3d4e5f60", "server_info": { "server_type": "qlabs-stt-proxy", "backend_url": "wss://stt.60db.ai/v1/stream" } } ``` `server_info` on the proxy frame is diagnostic; do not branch on its contents. ```json Response theme={null} { "type": "connected", "server_info": { "server_type": "60db STT", "ready": true, "total_languages": 40, "features": { "vad_segmentation": true, "multi_language_per_session": true, "max_languages_per_session": 5, "telephony_mulaw": true, "browser_pcm": true, "continuous_mode": true, "interim_results": true, "speaker_diarization": true, "code_switching_indic_english": true, "non_hallucinating": true }, "timing": { "min_utterance_end_ms": 1000, "default_utterance_end_ms": 1000, "max_utterance_seconds": 30 } } } ``` ### `session_started` — `start` message processed, audio is now accepted Sent by the upstream STT server after a `start` message is received and validated. It is safe to begin sending `audio` frames immediately after this event. ```json Response theme={null} { "type": "session_started", "session_id": "sess_8c3d1a9f4b7e2c51", "language": "Multi-language: EN, HI", "languages": ["en", "hi"], "model": "60db-stt-v01", "processing_mode": "sentence_based_continuous", "continuous_mode": true, "interim_frequency": 300, "diarize": false, "llm_refinement": true } ``` **Fields worth branching on:** | Field | Notes | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `languages` | The authoritative resolved candidate list after normalization and shared-backend collapse. `language` is a human-readable label for logs only. | | `llm_refinement` | `true` when you supplied a valid `context` **and** refinement is enabled server-side. `true` means every utterance arrives as a two-phase pair (see [Canonical-answer semantics](#canonical-answer-semantics-speech-final)); `false` or absent means one canonical event per utterance. Check this rather than assuming refinement is on because you sent a `context`. | | `diarize` | Echoes whether diarization was actually accepted. | | `model` | Always `60db-stt-v01`. The proxy rebrands the upstream label and strips the upstream `device` field, so neither reveals backend internals. | ### `speech_started` — VAD detected voice activity ```json Response theme={null} { "type": "speech_started", "timestamp": 1700000000.123 } ``` Use this for barge-in: interrupt TTS playback when this arrives. Fired after 2 consecutive VAD-positive chunks (\~64ms of confirmed speech). ### `transcription` — Transcription result All results (interim and final) share the same `transcription` type — differentiate with flags. **Final result** (`is_final=true`, `speech_final=true`): ```json Response theme={null} { "type": "transcription", "text": "Hello, how are you?", "confidence": 0.87, "language": "en", "language_name": "EN", "is_final": true, "speech_final": true, "is_partial": false, "sentence_id": 3, "duration": 1.82, "latency": 0.43, "timestamp": 1700000000.456, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 }, { "word": "how", "start": 0.35, "end": 0.52, "confidence": 0.92 } ], "speakers": [ { "speaker": "SPEAKER_00", "start": 0.0, "end": 1.82 } ] } ``` **Empty speech\_final signal** (`text=""`, `is_final=true`, `speech_final=true`): Sent when audio was detected but transcription was rejected (silence, hallucination, low confidence, wrong language). Client should reset its state on this message and not treat it as an error. ```json Response theme={null} { "type": "transcription", "text": "", "confidence": 0.0, "is_final": true, "speech_final": true, "processing_mode": "speech_end_no_result", "timestamp": 1700000000.789 } ``` **Interim result** (`is_final=false`, `speech_final=false`) — only sent when `interim_results_frequency` is set: ```json Response theme={null} { "type": "transcription", "text": "Hello how", "confidence": 0.72, "language": "en", "is_final": false, "speech_final": false, "is_partial": true } ``` Use interims only for barge-in word-count checks. Never send interim text to the LLM — a final with `is_final=true, speech_final=true` will follow. **Response Fields:** Transcribed text. Empty string = speech-end-no-result signal. 0.0–1.0. Telephony typically 0.35–0.75; browser 0.55–0.95. Detected language code e.g. `"en"`. Uppercase language code (e.g. `"EN"`) in the WS shape. Note: REST `/stt` returns the full English name (`"English"`) here — WS preserves the legacy uppercase-code shape for client compatibility. `true` = end of speech reached. May still be followed by a canonical upgrade if LLM refinement is active. `true` = canonical answer, will not be revised. When LLM refinement is on, one `is_final: true, speech_final: false` event is followed by one `is_final: true, speech_final: true`. When refinement is off, every final is `speech_final: true`. See [Canonical-answer semantics](#canonical-answer-semantics-speech-final). `true` for interim results only. Monotonically increasing counter per session. Duration (seconds) of the audio segment transcribed. Seconds from processing start to result ready (excludes queue time). Word-level timestamps `[{word, start, end, confidence, boosted?, original?}]`. Note: the field name is **`confidence`**, not `probability` (60db STT convention — different from legacy Whisper docs). Present on finals; empty on interims. When the keyword/context-terms boost replaced a word, the entry includes `boosted: true` and `original` (the pre-boost word): ```json theme={null} { "word": "Acme", "start": 1.5, "end": 2.0, "confidence": 0.85, "boosted": true, "original": "akmie" } ``` The segment-level `text` is already rebuilt from boosted words upstream — no client-side stitching required. Recommended UI: subtle underline on `boosted: true` words, with `original` shown on hover. List of `[{speaker, start, end}]` diarization turns when `config.diarize=true`. Omitted or `null` otherwise. Raw speaker IDs look like `SPEAKER_00`, `SPEAKER_01`; clients typically re-label these as "Speaker 1", "Speaker 2" in order of first appearance. Marker for utterances that should be skipped by the consumer (no useful text, **never billed**). Omitted on ordinary finals. | Value | Meaning | | ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | | `speech_end_no_result` | Speech ended but the recognizer produced no text. | | `speech_end_too_short` | Utterance below the minimum duration to recognize. | | `hallucination_rejected` | Word-rate guard rejected output as likely hallucination. | | `low_snr_dropped` | Audio dropped before LID/ASR; SNR below floor. **No credits charged.** Treat as "skip this utterance — no useful text". | Present on canonical emits when refinement was attempted. `true` = `text` is the LLM-refined version. `false` = the LLM was skipped or failed and the first-emit text was promoted unchanged — the event still arrives, so consumers waiting on `speech_final: true` never hang. Why `llm_applied` is `false`. Upstream values include `gate_closed` and `error:*` (timeout, HTTP error). The 60db proxy adds `dropped_too_many_words` when it rolled a refinement back — see the guardrail note below. Diagnostic; safe to ignore in display logic. Round-trip time to the LLM endpoint, in ms. Present when the refinement call actually ran — including runs the proxy later rolled back, so SLA dashboards keep the timing. Added by the 60db proxy when the upstream rejected an utterance as a suspected hallucination (`processing_mode: "hallucination_rejected"`) but usable text was available from the interim or first emit. The proxy substitutes that text rather than delivering an empty final. Route on it, but flag it for review — accuracy is not guaranteed. Absent on ordinary events. Why the event is tentative. Currently only `hallucination_rejected`. Audio signal-to-noise ratio (dB) for this utterance, when measured. Optional. Surface as a "good / fair / poor" badge: `>= 15` good, `0–15` fair, `< 0` poor. ### Canonical-answer semantics: `speech_final` `is_final` and `speech_final` are **NOT identical** when LLM refinement is active — they split into two distinct meanings: | `is_final` | `speech_final` | Meaning | | ---------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `false` | `false` | Interim partial — text may still change as more audio arrives. | | `true` | `false` | End-of-speech reached, dict-corrected text. **The LLM is still processing.** A follow-up event with `speech_final: true` will arrive shortly with the canonical text. Only emitted when refinement is active for this utterance. | | `true` | `true` | **Canonical answer.** Definitive, will not be revised. LLM-refined text when context was supplied, otherwise the original ASR text. | The same `sentence_id` is echoed across both phases so clients can reconcile. **Canonical event example (after LLM refinement):** ```json Response theme={null} { "type": "transcription", "sentence_id": 3, "text": "डॉक्टर साहब, मेरा sugar level बहुत high है। Metformin की dose बढ़ाओ।", "confidence": 0.87, "language": "hi", "language_name": "HI", "is_final": true, "speech_final": true, "is_partial": false, "duration": 1.82, "words": [ { "word": "डॉक्टर", "start": 0.0, "end": 0.32, "confidence": 0.94 } ], "speakers": null, "timestamp": 1700000000.789 } ``` **Guarantees:** * **Exactly one canonical event per utterance.** When refinement is on, you get two `transcription` events per utterance (first emit + canonical). When refinement is off, you get one (`speech_final: true`). Never zero, never three. * **Same `sentence_id`** across both phases. Reconcile on that key. * **The canonical always arrives.** Consumers waiting on `speech_final: true` never hang. * **`sentence_id` ordering is preserved per session**, but canonicals are NOT guaranteed to arrive in `sentence_id` order when LLM is on — two utterances finalizing close in time may complete refinement out of order. Key on `sentence_id`, not arrival order. * **`words[]` corresponds to the original ASR output** on both phases — the LLM does not realign tokens. Use `words[]` for word-level timing, `text` for display. **Recommended client patterns:** *Simplest — don't care about the first-emit optimization:* ```js theme={null} function onMessage(msg) { if (msg.type === 'transcription' && msg.speech_final) { // Canonical — render and forget render(msg); } // Ignore is_final && !speech_final (intermediate, will be replaced) // Ignore is_partial (interim, handle separately if needed) } ``` *With fast first-emit (UX-aware):* ```js theme={null} const livePainted = new Map(); // sentence_id → line slot function onMessage(msg) { if (msg.type !== 'transcription') return; const sid = msg.sentence_id; if (msg.is_partial) { renderPartial(msg); return; } let entry = livePainted.get(sid); if (!entry) { entry = createLine(); livePainted.set(sid, entry); } entry.text = msg.text; entry.pending = msg.is_final && !msg.speech_final; // dim while LLM runs render(entry); if (msg.speech_final) { finalize(entry); livePainted.delete(sid); } } ``` **For voicebot NLU routing**: feed the first-emit text (`speech_final: false`) to NLU immediately for fast intent dispatch — don't wait for canonical. If your NLU benefits from proper-noun accuracy (name-spelling slots, drug-name lookup), run a second-pass call on the canonical (`speech_final: true`) text and reconcile on `sentence_id`. **Word-preservation guardrail (60db proxy).** Refinement is allowed to polish, not to delete. If the canonical text keeps less than 40% of the first emit's words, the proxy restores the first-emit text and re-flags the event as `llm_applied: false` with `llm_reason: "dropped_too_many_words"` (`llm_latency_ms` is kept). The same rule is applied to legacy `refined` events, which are dropped outright when they fail it, so the transcription you already rendered stands. You never receive a canonical that silently lost most of what was said. **Legacy `refined` event.** Earlier builds emitted a separate `refined` event \~400 ms after the final instead of a second `transcription`. The 60db `/ws/stt` proxy transparently handles both shapes — if you're still seeing `refined` events in the wire trace, upstream workers haven't been restarted onto the two-phase build yet. New client code should target the two-phase flow only; `refined` is accepted but deprecated. ### `language_changed` — After `config` message changes language ```json Response theme={null} { "type": "language_changed", "language": "Multi-language: HI", "language_code": ["hi"] } ``` ### `mode_changed` — After `config` message changes `continuous_mode` ```json Response theme={null} { "type": "mode_changed", "continuous_mode": true, "mode_name": "continuous", "silence_threshold": 0.5 } ``` ### `session_stopped` — After `stop` is processed ```json Response theme={null} { "type": "session_stopped", "billing_summary": { "total_duration_seconds": 12.40, "total_cost": 0.000620, "characters_transcribed": 188, "client_estimated_seconds": 12.42 } } ``` **`billing_summary` fields:** | Field | Type | Notes | | -------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `total_duration_seconds` | number | The audio actually billed — sum of canonical finals that passed the billable predicate. **Sessions that previously double-billed** when the client sent `stop` are now billed correctly; expect \~50% lower numbers for affected sessions vs historical values. | | `total_cost` | number | USD cost for the session. | | `characters_transcribed` | number | Total characters across canonical transcriptions. | | `client_estimated_seconds` | number | **Diagnostic only** — rough estimate of audio the client sent. Useful for debugging duration drift; never display as a billed number. | **What gets billed:** only canonical finals (`is_final: true` **and** `speech_final: true`) that carry a `duration`. First emits (`speech_final: false`) are previews and are never charged, so two-phase refinement costs the same as a single-phase session. Utterances marked `speech_end_no_result`, `speech_end_too_short`, `hallucination_rejected` or `low_snr_dropped` are dropped upstream and never charged either — silence, noise and rejected audio cost nothing. ### `error` — Processing error ```json Response theme={null} { "type": "error", "error": "Audio processing error: ...", "timestamp": 1700000000.0 } ``` #### Concurrency-limit error frame When a user has reached their per-user STT session cap (counted across REST + WS combined), the server sends an error frame and closes with code `1008`: ```json Response theme={null} { "type": "error", "error": "Too many concurrent STT sessions for this user", "error_code": "STT_CONCURRENCY_LIMIT", "details": { "limit": 8 } } ``` Existing sessions are unaffected; the cap releases when an in-flight session ends. **Do not auto-reconnect on `1008`** — the limit only frees when an in-flight session completes. The frontend should distinguish concurrency-limit closes from auth/network failures so the UI message is correct. ### `connection_closed` — Upstream dropped the session Sent when the upstream STT service closes its side. Billing is finalised at this point and the client socket is closed straight after, so treat it as terminal and reconnect if you still have audio to send. ```json Response theme={null} { "type": "connection_closed", "reason": "60db service disconnected", "code": 1006 } ``` ### `test_response` — Reply to `test` ping ```json Response theme={null} { "type": "test_response", "message": "pong - qlabs-stt-proxy ready", "timestamp": 1700000000000, "processing_mode": "proxy" } ``` Answered by the 60db proxy itself — it never reaches the STT server, so a reply confirms the proxy hop only, and the echoed `timestamp` measures round-trip to the proxy. ## Complete Example ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/stt?apiKey=${API_KEY}`); ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (data) => { const msg = JSON.parse(data); console.log('←', msg.type || Object.keys(msg)[0]); if (msg.type === 'connection_established') { console.log(' User ID:', msg.user_id); console.log(' Credits:', msg.credit_balance); // Start session with config ws.send(JSON.stringify({ type: 'start', languages: ['en', 'hi'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true, utterance_end_ms: 1000, interim_results_frequency: 300 } })); } else if (msg.type === 'session_started') { // Audio is only accepted from here on — `connected` arrives earlier and // can arrive twice (proxy frame + upstream frame). console.log('✓ Session started! Send audio now...'); // Send audio chunks (480 bytes = ~60ms at 8kHz) const audioInterval = setInterval(() => { const audioChunk = getAudioChunk(); // Your audio capture function ws.send(audioChunk); }, 60); // Stop after 5 seconds setTimeout(() => { clearInterval(audioInterval); ws.send(JSON.stringify({ type: 'stop' })); }, 5000); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected - barge-in opportunity'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓ Final:', msg.text, `(confidence: ${msg.confidence})`); } else { console.log(' Partial:', msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); console.log(' Duration:', msg.billing_summary.total_duration_seconds, 's'); console.log(' Cost: $', msg.billing_summary.total_cost); ws.close(); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` ```python theme={null} import asyncio import json import websockets API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/stt?apiKey={API_KEY}" async def stt_websocket(): async with websockets.connect(url) as ws: # Wait for authentication to finish. `connecting` arrives first, then # `connected` (proxy), then `connection_established`. while True: msg = json.loads(await ws.recv()) if msg.get("type") == "connection_established": print(f"✓ Connected (User: {msg['user_id']})") print(f" Credits: ${msg['credit_balance']}") break # Start session await ws.send(json.dumps({ "type": "start", "languages": ["en", "hi"], "config": { "encoding": "mulaw", "sample_rate": 8000, "continuous_mode": True, "utterance_end_ms": 1000, "interim_results_frequency": 300 } })) # Audio is accepted only after `session_started` while True: msg = json.loads(await ws.recv()) if msg.get("type") == "session_started": break print("✓ Session started!") # Send audio for 5 seconds for _ in range(83): # ~5000ms / 60ms audio_chunk = get_audio_chunk() # Your audio capture function await ws.send(audio_chunk) # Send as binary await asyncio.sleep(0.06) # Stop session await ws.send(json.dumps({"type": "stop"})) # Process remaining messages while True: msg = json.loads(await ws.recv()) msg_type = msg.get("type") if msg_type == "speech_started": print("🎤 Speech detected") elif msg_type == "transcription": if msg.get("is_final"): print(f"✓ {msg['text']} (confidence: {msg['confidence']})") else: print(f" {msg['text']}...") elif msg_type == "session_stopped": print(f"✓ Session stopped") print(f" Duration: {msg['billing_summary']['total_duration_seconds']}s") print(f" Cost: ${msg['billing_summary']['total_cost']}") break asyncio.run(stt_websocket()) ``` ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key'); let mediaRecorder; let audioContext; ws.onopen = () => { console.log('✓ Connected'); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'linear', sample_rate: 48000, continuous_mode: true, interim_results_frequency: 300 } })); }; ws.onmessage = (event) => { const msg = JSON.parse(event.data); if (msg.type === 'connected') { console.log('✓ Session started! Start speaking...'); startAudioCapture(); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓', msg.text); updateTranscriptDisplay(msg.text); } else { console.log('...', msg.text); updateInterimDisplay(msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); stopAudioCapture(); ws.close(); } }; async function startAudioCapture() { const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); audioContext = new AudioContext({ sampleRate: 48000 }); const source = audioContext.createMediaStreamSource(stream); const processor = audioContext.createScriptProcessor(4096, 1, 1); processor.onaudioprocess = (e) => { const inputData = e.inputBuffer.getChannelData(0); const pcm16 = new Int16Array(inputData.length); for (let i = 0; i < inputData.length; i++) { pcm16[i] = Math.max(-32768, Math.min(32767, inputData[i] * 32768)); } ws.send(JSON.stringify({ type: 'audio', audio: btoa(String.fromCharCode(...new Uint8Array(pcm16.buffer))), encoding: 'linear', sample_rate: 48000 })); }; source.connect(processor); processor.connect(audioContext.destination); mediaRecorder = { stream, processor, source }; } function stopAudioCapture() { if (mediaRecorder) { mediaRecorder.stream.getTracks().forEach(track => track.stop()); mediaRecorder.processor.disconnect(); audioContext.close(); } } ``` ## Audio Requirements | Property | Telephony (μ-law) | Browser (PCM) | | ----------- | ----------------- | ----------------------------- | | Encoding | `mulaw` (8-bit) | `linear` (16-bit) | | Sample Rate | 8000 Hz | 16000, 24000, 44100, 48000 Hz | | Chunk Size | 480 bytes (60ms) | 960-1920 bytes (60-120ms) | | Channels | Mono (1 channel) | Mono (1 channel) | ## Supported Languages 39 transcription languages total (25 European, 13 Indic with Hinglish code-switching, and Arabic MSA). Fetch the full catalog from `GET /stt/languages`. | Code | Language | Code | Language | | ---- | ------------ | ---- | --------- | | `en` | English | `hi` | Hindi | | `es` | Spanish | `bn` | Bengali | | `fr` | French | `mr` | Marathi | | `de` | German | `pa` | Punjabi | | `it` | Italian | `gu` | Gujarati | | `pt` | Portuguese | `ta` | Tamil | | `nl` | Dutch | `te` | Telugu | | `pl` | Polish | `kn` | Kannada | | `ru` | Russian | `ml` | Malayalam | | `uk` | Ukrainian | `or` | Odia | | `cs` | Czech | `as` | Assamese | | `sv` | Swedish | `ne` | Nepali | | `ar` | Arabic (MSA) | `sa` | Sanskrit | **Code-switching (Indic + English):** `hi+en`, `bn+en`, `mr+en`, `pa+en`, `gu+en`, `or+en`, `as+en`, `ne+en`, `te+en`, `kn+en`, `ta+en`, `ml+en` — collapses to the fast path when both languages share the same pipeline. **Not supported** (explicit rejection): `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`. These return an `unsupported_language` error — there is **no silent aliasing**. Arabic dialect tags (`ar-eg`, `ar-lv`, `ar-gu`, `ar-ma`) return `dialect_not_supported` — pass `ar` for best-effort MSA transcription of dialectal audio. ## Limitations and Best Practices **Handshake ordering (most common mistake)** Do **not** send `start` in `ws.onopen`. Wait for the proxy's `connection_established` message first — it marks the point at which the proxy has attached its client-message listener. Likewise, wait for `session_started` before sending `audio` frames, otherwise the upstream server returns `unknown message type: audio`. **`utterance_end_ms` floor** The minimum is **1000 ms**, and it is enforced twice: the upstream server rejects a `start` below it, and the 60db proxy clamps sub-1000 values up to 1000 ms before forwarding so an older client is silently corrected instead of disconnected. Shorter silences fragment utterances and make the non-hallucinating backends drop short segments. If you need faster turn-taking, drive barge-in from `speech_started` and interim results — not from this value. **Language count** Max 5 languages per session. Cross-backend multi-language (e.g. `["en","ar"]`) runs per-utterance LID which adds \~20–50 ms per utterance. Same-backend multi-language (e.g. `["en","hi"]`) collapses to a single backend's fast path with zero LID overhead — use this whenever possible. **Telephony confidence** 8 kHz μ-law → 16 kHz resampling reduces backend confidence by \~0.10–0.15 compared to native wideband input. Use a client-side threshold of 0.35 for telephony vs 0.55 for browser. **Buffer limits** * Pre-speech ring buffer: 1.0 s (captures first word before VAD fires) * Minimum coalesce before VAD: 160 ms (μ-law 1280 bytes, linear `sample_rate × 2 × 160 / 1000` bytes) * Maximum utterance duration: 30 s (anything longer force-finalizes) **VAD** * Speech start: Silero probability > `STT_VAD_THRESHOLD` (default 0.5, server-configurable) * Silence → utterance end: `utterance_end_ms` of consecutive sub-threshold audio * No separate continuation threshold **Non-hallucinating architecture** The 60db STT backend uses non-hallucinating architectures that do not output `no_speech_prob` — the CTC/RNN-T topology emits blank tokens on non-speech. Hallucination guard is a word-rate sanity check (> 5 words/second → rejected). **Diarization** `diarize=true` requires `HF_TOKEN` on the server plus gated model approval for `pyannote/speaker-diarization-3.1`. Without both, the request silently falls back to a deterministic mock. Check `session_started.diarize` to confirm the request was accepted. ## Pricing * **Rate**: \$0.00000833 per second * **Minimum**: \$0.01 per session * **Billing**: Per second of audio processed ## Error Codes | Close code | `error_code` (in preceding error frame) | Description | | ---------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | 1008 | `AUTH_FAILED` | API key / JWT rejected, or missing `workspace_id` on JWT auth | | 1008 | `INSUFFICIENT_CREDITS` | Workspace wallet has no funds | | 1008 | `STT_CONCURRENCY_LIMIT` | Per-user concurrency cap reached. `details.limit` carries the active cap. Do not auto-reconnect; cap releases when an in-flight session ends. | | 1011 | — | Internal server error | | 1006 | — | Connection lost | `error_code` sits alongside `error` on the frame — for example `{"type":"error","error":"Insufficient credits to start STT session","error_code":"INSUFFICIENT_CREDITS","details":{"required":0.01,"available":0}}`. One exception: when the wallet runs out **mid-session**, the deduction failure is reported as `{"type":"error","error":{"code":"INSUFFICIENT_CREDITS","message":"…"}}` — a nested object — before the socket closes with `1008`. Handle both shapes if you parse credit errors. ## Testing ```bash theme={null} # Install wscat npm install -g wscat # Test STT wscat -c "ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key" ``` Then send: ```json theme={null} {"type":"start","languages":["en"],"config":{"encoding":"mulaw","sample_rate":8000,"continuous_mode":true}} ``` ## Related * [TTS WebSocket](/api-reference/websocket/tts) - Text-to-Speech endpoint * [WebSocket API Reference](/websocket-api) - Complete documentation * [Voices API](/api-reference/voices/get-voices) - Get available voices # TTS WebSocket Source: https://docs.60db.ai/api-reference/websocket/tts WebSocket /ws/tts Text-to-Speech WebSocket endpoint for real-time audio synthesis # TTS WebSocket Real-time Text-to-Speech synthesis via WebSocket streaming with full-duplex bidirectional communication. ## Endpoint `ws://api.60db.ai/ws/tts` ## Authentication Query parameter authentication: Your API key for authentication JWT token (alternative to API key) Workspace ID for billing. Required when using JWT auth. API keys are automatically pinned to their workspace. Examples: ``` ws://api.60db.ai/ws/tts?apiKey=sk_live_your_api_key ws://api.60db.ai/ws/tts?token=eyJ...&workspace_id=24 ``` The WebSocket connection checks **workspace wallet balance** before starting a session. If the workspace has insufficient credits, the connection is closed with a `1008` status code and an `INSUFFICIENT_CREDITS` error. ## Protocol Overview ``` Client Server | | |─── create_context ──────────────────▶ | |◀── context_created ───────────────── | | | |─── send_text ───────────────────────▶ | |─── send_text ───────────────────────▶ | |─── flush_context ───────────────────▶ | |◀── audio_chunk #1 ────────────────── | |◀── audio_chunk #2 ────────────────── | |◀── audio_chunk #N ────────────────── | |◀── flush_completed ───────────────── | | | |─── close_context ───────────────────▶ | |◀── context_closed ────────────────── | | (connection closes) | ``` ## Connection Sequence ### 1. Connect ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); ``` ### 2. Receive Authentication Message ```json Response theme={null} { "connecting": true, "message": "Authenticating...", "timestamp": 1775465918269 } ``` ### 3. Receive Connection Established ```json Response theme={null} { "connection_established": { "service": "tts", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } } ``` **Fields:** Service name: `"tts"` Your user ID Available credits Workspace name ## Client → Server Messages ### 1. create\_context **Must be the first message.** Initializes the TTS session with voice and audio settings. ```json Request theme={null} { "create_context": { "context_id": "my-session-123", "voice_id": "7911a3e8", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } } ``` **Parameters:** Unique session identifier. Default: auto-generated UUID Voice ID to use for synthesis Audio encoding. Options: `LINEAR16`, `PCM`, `MULAW`, `ULAW`, `OGG_OPUS` Sample rate in Hz. Options: `8000`, `16000`, `24000`, `48000` Speech speed multiplier (0.5 – 2.0). Voice stability (0-100). Lower = more expressive, higher = more consistent. Voice similarity (0-100). How closely the output matches the source voice. **Supported encoding + sample rate combinations:** Not all combinations are valid. The table below shows which pairs are supported. Unsupported combinations silently fall back to `LINEAR16` at `16000` Hz. | `audio_encoding` | Supported `sample_rate_hertz` | Output format | | ---------------- | ------------------------------------------- | ------------------------------------------ | | `LINEAR16` | `8000`, `16000` (default), `24000`, `48000` | Raw PCM, 16-bit signed little-endian, mono | | `PCM` | `8000`, `16000` (default), `24000`, `48000` | Same as LINEAR16 | | `MULAW` | `8000` | G.711 μ-law encoded, mono | | `ULAW` | `8000` | Same as MULAW | | `OGG_OPUS` | `24000` | Ogg Opus compressed audio | > **Note:** `MULAW`/`ULAW` only works at `8000` Hz. Using other sample rates with MULAW falls back to LINEAR16 @ 16kHz. Similarly, `OGG_OPUS` only works at `24000` Hz. **Limits:** | Parameter | Min | Max | Default | Behavior when out of range | | ------------------------- | ------ | ------------ | ------- | -------------------------- | | `speed` | 0.5 | 2.0 | 1 | Silently clamped | | `stability` | 0 | 100 | 50 | Silently clamped | | `similarity` | 0 | 100 | 75 | Silently clamped | | `text` (per send\_text) | 1 char | — | — | Empty text is ignored | | text buffer (accumulated) | — | 50,000 chars | — | Error returned if exceeded | ### 2. send\_text Append text to the internal buffer. Text is accumulated until a `flush_context` or `close_context` is received. ```json Request theme={null} { "send_text": { "context_id": "my-session-123", "text": "Hello, how are you doing today?" } } ``` **Fields:** Session identifier Text to append to buffer (max cumulative 50,000 characters) You can send multiple `send_text` messages to build up text incrementally (e.g., from an LLM token stream): ```json theme={null} {"send_text": {"context_id": "ctx-1", "text": "Hello, "}} {"send_text": {"context_id": "ctx-1", "text": "how are you "}} {"send_text": {"context_id": "ctx-1", "text": "doing today?"}} ``` **Text chunking behavior:** Long text is automatically split into sentence-based chunks for reliable synthesis. The model works best with 3–30 second utterances. The server handles: * Sentence boundary detection for natural chunk splits * Newline characters (`\n`) are treated as hard paragraph boundaries * Mixed-language text (e.g., English + Hindi) is chunked per paragraph to prevent early EOS ### 3. flush\_context Triggers synthesis of all accumulated text. The server responds with `audio_chunk` messages followed by `flush_completed` (only on success — if synthesis fails, an `error` message is sent instead with no `flush_completed`). ```json Request theme={null} { "flush_context": { "context_id": "my-session-123" } } ``` ### 4. close\_context Flushes any remaining text, sends final audio, and closes the WebSocket connection. ```json Request theme={null} { "close_context": { "context_id": "my-session-123" } } ``` ## Server → Client Messages ### context\_created Confirms the session was initialized successfully. ```json Response theme={null} { "context_created": { "context_id": "my-session-123" } } ``` ### audio\_chunk Contains a chunk of synthesized audio. Multiple chunks are sent per flush. Each chunk is streamed as soon as it's decoded for minimum latency. ```json Response theme={null} { "audio_chunk": { "context_id": "my-session-123", "audioContent": "SGVsbG8gd29ybGQ..." } } ``` **Fields:** Session identifier Base64-encoded audio bytes The audio encoding and chunk format depend on `audio_config`: | Encoding | Chunk format | Notes | | ------------------ | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `LINEAR16` / `PCM` | Raw PCM, 16-bit signed LE, mono | Chunks can be concatenated directly | | `MULAW` / `ULAW` | G.711 μ-law, 8-bit, mono | Chunks can be concatenated directly | | `OGG_OPUS` | Independent Ogg Opus files | Each chunk is a self-contained OGG file. **Chunks cannot be naively concatenated** — decode each independently or use LINEAR16 for concatenatable streaming | ### flush\_completed Signals that all audio for the flushed text has been sent. Only sent on successful synthesis — if synthesis fails, an `error` is sent instead. ```json Response theme={null} { "flush_completed": { "context_id": "my-session-123" } } ``` ### context\_closed Confirms the session is closed. The WebSocket connection closes after this message. ```json Response theme={null} { "context_closed": { "context_id": "my-session-123" } } ``` ### error Sent if synthesis fails or a protocol violation occurs. ```json Response theme={null} { "error": { "context_id": "my-session-123", "message": "voice_id required" } } ``` **Common errors:** | Message | Cause | | -------------------------------------------- | ------------------------------------------ | | `voice_id required` | `create_context` sent without `voice_id` | | `text_buffer exceeded 50000 character limit` | Too much text accumulated without flushing | | `Unsupported audio_encoding: X` | Invalid encoding value | | `Unsupported sample_rate_hertz: X` | Invalid sample rate | #### Concurrency-limit error frame When a user has reached their per-user TTS session cap (counted across REST + WS combined), the server sends an error frame and closes with code `1008`: ```json Response theme={null} { "error": { "message": "Too many concurrent TTS sessions for this user", "code": "TTS_CONCURRENCY_LIMIT", "details": { "limit": 5 } } } ``` **STT vs TTS error-frame shape mismatch.** The TTS WS uses the legacy shape `{error: {message, code, details}}`. The STT WS uses `{type: "error", error, error_code, details}`. SDK / client code that reads both connections must branch on the connection it's reading from. This inconsistency is pre-existing in the WS handlers, not new. Existing sessions are unaffected; the cap releases when an in-flight session ends. **Do not auto-reconnect on `1008`** — the limit only frees when an in-flight session completes. ## Complete Example ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/tts?apiKey=${API_KEY}`); const contextId = 'test-' + Date.now(); // Store audio chunks const audioChunks = []; ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (raw) => { const data = JSON.parse(raw); const msgType = Object.keys(data)[0]; console.log('←', msgType); if (data.connection_established) { console.log(' Credits:', data.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: '7911a3e8', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (data.context_created) { console.log('✓ Context created!'); // Send text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you doing today?' } })); // Flush ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (data.audio_chunk) { console.log(' Received audio chunk'); const audioData = Buffer.from(data.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); } else if (data.flush_completed) { console.log('✓ Flush completed!'); console.log(' Total audio size:', audioChunks.reduce((sum, chunk) => sum + chunk.length, 0), 'bytes'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } else if (data.context_closed) { console.log('✓ Context closed'); // Save audio const audio = Buffer.concat(audioChunks); require('fs').writeFileSync('output.pcm', audio); console.log('Saved output.pcm'); ws.close(); } if (data.error) { console.error('TTS Error:', data.error.message); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` ```python theme={null} import asyncio import json import base64 import websockets async def tts_websocket(): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/tts?apiKey={API_KEY}" context_id = f"session-{int(asyncio.get_event_loop().time())}" async with websockets.connect(url) as ws: # Wait for connection resp = json.loads(await ws.recv()) if resp.get('connection_established'): print(f"✓ Connected (Credits: ${resp['connection_established']['credit_balance']})") # Create context await ws.send(json.dumps({ "create_context": { "context_id": context_id, "voice_id": "7911a3e8", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } })) # Wait for context_created resp = json.loads(await ws.recv()) assert "context_created" in resp print("✓ Context created!") # Send text + flush await ws.send(json.dumps({ "send_text": { "context_id": context_id, "text": "Hello, how are you doing today?" } })) await ws.send(json.dumps({ "flush_context": {"context_id": context_id} })) # Receive audio chunks until flush_completed audio_data = b"" while True: msg = json.loads(await ws.recv()) if "audio_chunk" in msg: audio_data += base64.b64decode(msg["audio_chunk"]["audioContent"]) print(" Received audio chunk") elif "flush_completed" in msg: print("✓ Flush completed!") break elif "error" in msg: print(f"Error: {msg['error']['message']}") break # Close context await ws.send(json.dumps({ "close_context": {"context_id": context_id} })) resp = json.loads(await ws.recv()) assert "context_closed" in resp # Save raw PCM (16-bit, 16kHz, mono) with open("output.pcm", "wb") as f: f.write(audio_data) print(f"Saved {len(audio_data)} bytes to output.pcm") asyncio.run(tts_websocket()) ``` ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); const contextId = 'test-' + Date.now(); const audioChunks = []; let audioCtx; ws.onopen = () => { console.log('✓ Connected'); }; ws.onmessage = (event) => { const data = JSON.parse(event.data); const msgType = Object.keys(data)[0]; console.log('←', msgType); if (data.connection_established) { console.log(' Credits:', data.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: '7911a3e8', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (data.context_created) { console.log('✓ Context created!'); // Send text + flush ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you?' } })); ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (data.audio_chunk) { // Collect base64 audio const binary = atob(data.audio_chunk.audioContent); const bytes = new Uint8Array(binary.length); for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); audioChunks.push(bytes); console.log(' Received audio chunk'); } else if (data.flush_completed) { console.log('✓ Flush completed!'); // Play collected audio playPCM16(audioChunks, 16000); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } if (data.error) { console.error('TTS Error:', data.error.message); } if (data.context_closed) { console.log('✓ Context closed'); } }; function playPCM16(chunks, sampleRate) { const totalBytes = chunks.reduce((s, c) => s + c.length, 0); const merged = new Uint8Array(totalBytes); let offset = 0; for (const chunk of chunks) { merged.set(chunk, offset); offset += chunk.length; } const int16 = new Int16Array(merged.buffer); const float32 = new Float32Array(int16.length); for (let i = 0; i < int16.length; i++) float32[i] = int16[i] / 32768; audioCtx = audioCtx || new AudioContext({ sampleRate }); const buf = audioCtx.createBuffer(1, float32.length, sampleRate); buf.getChannelData(0).set(float32); const src = audioCtx.createBufferSource(); src.buffer = buf; src.connect(audioCtx.destination); src.start(); } ``` ## Real-time Playback (Browser) For low-latency playback as chunks arrive (instead of waiting for all chunks), use the Web Audio API with scheduled `AudioBufferSourceNode`: ```javascript theme={null} let audioCtx; let nextPlayTime = 0; function onAudioChunk(base64Audio, sampleRate) { const binary = atob(base64Audio); const bytes = new Uint8Array(binary.length); for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); // Decode PCM int16 LE → Float32 const int16 = new Int16Array(bytes.buffer); const float32 = new Float32Array(int16.length); for (let i = 0; i < int16.length; i++) float32[i] = int16[i] / 32768; if (!audioCtx) audioCtx = new AudioContext({ sampleRate }); const buf = audioCtx.createBuffer(1, float32.length, sampleRate); buf.getChannelData(0).set(float32); const source = audioCtx.createBufferSource(); source.buffer = buf; source.connect(audioCtx.destination); const now = audioCtx.currentTime; // First chunk: 150ms pre-buffer to absorb network jitter // Late chunks: schedule 20ms ahead for minimal gap if (nextPlayTime <= now) { nextPlayTime = now + (nextPlayTime === 0 ? 0.15 : 0.02); } source.start(nextPlayTime); nextPlayTime += float32.length / sampleRate; } ``` ## LLM Integration Pattern When streaming tokens from an LLM into TTS: ```python theme={null} import json import base64 import websockets async def llm_to_tts(llm_stream, voice_id): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/tts?apiKey={API_KEY}" async with websockets.connect(url) as ws: # Wait for connection await ws.recv() # connection_established # Create context await ws.send(json.dumps({ "create_context": { "context_id": "llm-session", "voice_id": voice_id, "audio_config": {"audio_encoding": "MULAW", "sample_rate_hertz": 8000} } })) await ws.recv() # context_created # Stream LLM tokens as text chunks async for token in llm_stream: await ws.send(json.dumps({ "send_text": {"context_id": "llm-session", "text": token} })) # Flush + close when LLM is done await ws.send(json.dumps({ "flush_context": {"context_id": "llm-session"} })) audio = b"" while True: msg = json.loads(await ws.recv()) if "audio_chunk" in msg: audio += base64.b64decode(msg["audio_chunk"]["audioContent"]) elif "flush_completed" in msg: break elif "error" in msg: raise RuntimeError(msg["error"]["message"]) await ws.send(json.dumps({ "close_context": {"context_id": "llm-session"} })) await ws.recv() # context_closed return audio ``` ## Audio Format Notes | Encoding | Format | Chunk behavior | Best for | | ---------- | ------------------------------- | ------------------------------------------------------------ | ----------------------------------- | | `LINEAR16` | Raw PCM, 16-bit signed LE, mono | Concatenatable | General purpose, highest quality | | `MULAW` | G.711 μ-law, 8kHz, mono | Concatenatable | Telephony (Twilio, SIP) | | `OGG_OPUS` | Ogg Opus compressed, 24kHz | **NOT concatenatable** — each chunk is a standalone OGG file | Web playback, bandwidth-constrained | For telephony integration (Twilio, etc.), use `MULAW` at `8000` Hz: ```json theme={null} "audio_config": { "audio_encoding": "MULAW", "sample_rate_hertz": 8000 } ``` For web playback with low bandwidth, use `OGG_OPUS` at `24000` Hz: ```json theme={null} "audio_config": { "audio_encoding": "OGG_OPUS", "sample_rate_hertz": 24000 } ``` > **Important:** OGG\_OPUS chunks are individually wrapped OGG files. To merge for download, decode each chunk independently (e.g., via `AudioContext.decodeAudioData()`) and concatenate the PCM output. Do not concatenate raw OGG bytes. ## Supported Languages The TTS model supports synthesis in multiple Indic languages and English. The language is auto-detected from the input text — no explicit language parameter is needed. | Language | ID | | --------- | -- | | English | en | | Hindi | hi | | Bengali | bn | | Gujarati | gu | | Kannada | kn | | Malayalam | ml | | Marathi | mr | | Punjabi | pa | | Tamil | ta | | Telugu | te | | Assamese | as | | Odia | or | Mixed-language text is supported. Use newlines (`\n`) to separate paragraphs in different languages for best results. ## Default Voice The default voice ID is: ``` fbb75ed2-975a-40c7-9e06-38e30524a9a1 ``` To get more voices, use the [Voices API](/api-reference/voices/get-voices). ## Context Management ### Reuse Context Keep a context open for multiple syntheses: ```javascript theme={null} // Create once ws.send(JSON.stringify({ create_context: { context_id, voice_id, audio_config } })); // Send multiple texts ws.send(JSON.stringify({ send_text: { context_id, text: "Hello" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); ws.send(JSON.stringify({ send_text: { context_id, text: "World" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); // Close when done ws.send(JSON.stringify({ close_context: { context_id } })); ``` ### Multiple Contexts You can create multiple contexts in one connection: ```javascript theme={null} const context1 = 'ctx-1'; const context2 = 'ctx-2'; // Create both contexts ws.send(JSON.stringify({ create_context: { context_id: context1, voice_id: voice1, audio_config } })); ws.send(JSON.stringify({ create_context: { context_id: context2, voice_id: voice2, audio_config } })); ``` ## Pricing * **Rate**: \$0.00002 per character * **Minimum**: \$0.01 per context * **Billing**: Per character synthesized ## Error Codes | Close code | `code` (in preceding error frame) | Description | | ---------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | 1008 | `UNAUTHENTICATED` | Authentication failed | | 1008 | `INSUFFICIENT_CREDITS` | Workspace wallet has no funds | | 1008 | `TTS_CONCURRENCY_LIMIT` | Per-user concurrency cap reached. `details.limit` carries the active cap. Do not auto-reconnect; cap releases when an in-flight session ends. | | 1011 | — | Voice not found / invalid audio config | | 1006 | — | Connection lost | ## Testing ```bash theme={null} # Install wscat npm install -g wscat # Test TTS wscat -c "ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key" ``` Then send: ```json theme={null} {"create_context":{"context_id":"test-123","voice_id":"fbb75ed2-975a-40c7-9e06-38e30524a9a1","audio_config":{"audio_encoding":"LINEAR16","sample_rate_hertz":16000}}} {"send_text":{"context_id":"test-123","text":"Hello"}} {"flush_context":{"context_id":"test-123"}} ``` ## Related * [STT WebSocket](/api-reference/websocket/stt) - Speech-to-Text endpoint * [Voices API](/api-reference/voices/get-voices) - Get available voices * [WebSocket API Reference](/websocket-api) - Complete documentation # Create Workspace Source: https://docs.60db.ai/api-reference/workspaces/create-workspace POST /workspaces Create a new workspace ## Request ### Headers Bearer token with your API key application/json ### Body Workspace name Workspace description ## Response Workspace ID Workspace name Workspace description Creation timestamp ```bash cURL theme={null} curl -X POST https://api.60db.ai/workspaces \ -H "Authorization: Bearer your-api-key" \ -H "Content-Type: application/json" \ -d '{ "name": "Development", "description": "Development workspace" }' ``` ```javascript JavaScript theme={null} const workspace = await client.createWorkspace({ name: "Development", description: "Development workspace", }); ``` ```python Python theme={null} workspace = client.create_workspace( name='Development', description='Development workspace' ) ``` ```json Response theme={null} { "id": "ws-456", "name": "Development", "description": "Development workspace", "created_at": "2026-01-29T11:35:00Z" } ``` # Get Workspaces Source: https://docs.60db.ai/api-reference/workspaces/get-workspaces GET /workspaces List all workspaces ## Request ### Headers Bearer token with your API key ## Response Array of workspace objects Workspace ID Workspace name Workspace description Creation timestamp ```bash cURL theme={null} curl https://api.60db.ai/workspaces \ -H "Authorization: Bearer your-api-key" ``` ```javascript JavaScript theme={null} const workspaces = await client.getWorkspaces(); ``` ```python Python theme={null} workspaces = client.get_workspaces() ``` ```json Response theme={null} { "workspaces": [ { "id": "ws-123", "name": "Production", "description": "Production workspace", "created_at": "2026-01-15T10:00:00Z" } ] } ``` # Authentication Source: https://docs.60db.ai/authentication Learn how to authenticate with the 60db API ## API Key Authentication 60db uses API key authentication to secure all API requests. Your API key should be included in the `Authorization` header of every request. ### Getting Your API Key Navigate to [app.60db.ai](https://app.60db.ai) and log in Go to Settings → Developer → API Keys Click "Create API Key" and provide a descriptive name Copy and store your API key in a secure location ## Using Your API Key ### With SDKs ```typescript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Text to Speech const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice', speed: 1.0 // 0.5 to 2.0 (default 1.0) }); // Get all voices const voices = await client.getVoices(); // Get all languages const languages = await client.getLanguages(); ``` You can also specify a custom base URL: ```typescript theme={null} const client = new SixtyDBClient({ apiKey: 'your-api-key', baseUrl: 'https://custom-api.60db.com' }); ``` ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') # Text to Speech audio = await client.text_to_speech( text='Hello, world!', voice_id='default-voice', speed=1.0 # 0.5 to 2.0 (default 1.0) ) # Get all voices voices = await client.get_voices() # Get all languages languages = await client.get_languages() ``` You can also specify a custom base URL: ```python theme={null} client = SixtyDBClient( api_key='your-api-key', base_url='https://custom-api.60db.com' ) ``` ### Direct API Calls If you're making direct HTTP requests, include your API key in the `Authorization` header: ```bash theme={null} curl https://api.60db.ai/voices \ -H "Authorization: Bearer your-api-key" ``` ## Security Best Practices Never expose your API key in client-side code, public repositories, or version control systems. ### Environment Variables Store your API key in environment variables: Create a `.env` file: ```bash theme={null} SIXTYDB_API_KEY=your-api-key ``` Use it in your code: ```typescript theme={null} const client = new SixtyDBClient(process.env.SIXTYDB_API_KEY); ``` Create a `.env` file: ```bash theme={null} SIXTYDB_API_KEY=your-api-key ``` Use it in your code: ```python theme={null} import os from sixtydb import SixtyDBClient client = SixtyDBClient(os.getenv('SIXTYDB_API_KEY')) ``` ### Key Rotation Regularly rotate your API keys for enhanced security: 1. Create a new API key 2. Update your application to use the new key 3. Test that everything works correctly 4. Delete the old API key ## Rate Limiting API requests are rate-limited based on your subscription plan: | Plan | Requests per Minute | Requests per Day | | ---------- | ------------------- | ---------------- | | Free | 10 | 1,000 | | Starter | 60 | 10,000 | | Pro | 300 | 100,000 | | Enterprise | Custom | Custom | Rate limit headers are included in every API response: - `X-RateLimit-Limit`: Maximum requests allowed - `X-RateLimit-Remaining`: Remaining requests in current window - `X-RateLimit-Reset`: Time when the rate limit resets ## Error Handling When authentication fails, you'll receive a `401 Unauthorized` response: ```json theme={null} { "error": "Unauthorized", "message": "Invalid API key" } ``` Common authentication errors: | Status Code | Error | Description | | ----------- | ----------------- | ----------------------------------------- | | 401 | Unauthorized | Invalid or missing API key | | 403 | Forbidden | API key doesn't have required permissions | | 429 | Too Many Requests | Rate limit exceeded | # Changelog Source: https://docs.60db.ai/changelog Release notes for the 60db platform — APIs, SDKs, CLI, MCP server, and WebSocket protocol. Every product surface shares this page. Dated entries flag the surfaces they touched (REST, WebSocket, SDK, CLI, MCP, Proxy) so you can skim by what you integrate with. Surfaces: **WebSocket**, **Proxy**, **Docs** ## Changed **`config.utterance_end_ms` minimum and default are now 1000 ms** (were 300 ms minimum / 500 ms default). * The upstream STT server **rejects** a `start` carrying a lower value instead of clamping it, which killed the session outright. * The 60db `/ws/stt` proxy now clamps any value below 1000 ms up to 1000 ms before forwarding — on `start` **and** on mid-session `config` — so existing clients sending `500` keep working and simply get 1000 ms behaviour. * Need faster turn-taking? Drive barge-in from `speech_started` and interim results rather than lowering this value. ## Fixed (docs) **STT handshake frames were documented with the TTS shape.** `/ws/stt` sends `{"type":"connecting"}` and `{"type":"connection_established","service":"stt","user_id":…}` — tagged by `type`, fields top-level. The examples previously branched on `msg.connection_established`, the nested `/ws/tts` shape, which never matches on STT. Examples now also wait for `session_started` before streaming audio, since `connected` arrives before the session exists and is emitted twice (proxy frame, then upstream capability frame). ## Documented * `session_started.llm_refinement` — whether your `context` actually opened the refinement gate. * `llm_applied`, `llm_reason`, `llm_latency_ms` on canonical transcription emits. * `tentative` / `tentative_reason` — proxy-substituted text when upstream rejects an utterance as a hallucination. * The proxy's word-preservation guardrail: a refinement that keeps under 40% of the first emit's words is rolled back to the original text. * `connection_closed`, the `AUTH_FAILED` error code, and the nested mid-session `INSUFFICIENT_CREDITS` error shape. * Billing: only canonical finals are charged; `speech_end_no_result`, `speech_end_too_short`, `hallucination_rejected` and `low_snr_dropped` never are. See [WebSocket STT reference](/api-reference/websocket/stt). ## Changed **`POST /tts-synthesize` now accepts the full Inworld schema.** * `audio_config: { audio_encoding, sample_rate_hertz }` — nested block matching the upstream schema. * `timestamp_type` (default `"WORD"`) — enables per-word timestamps in the final NDJSON chunk. * Legacy flat keys (`sample_rate`, `audio_encoding`) still work; new clients should use `audio_config`. Default `sample_rate_hertz` moved from **48000 → 24000** to match the upstream default and reduce bandwidth. Callers that need 48 kHz must send it explicitly. See [Text to Speech reference](/api-reference/tts/text-to-speech). ## Added **`config.remove_fillers` on WebSocket `start` (STT).** * New boolean field inside `start.config`. Default `false`. * When `true` **and** `context` is set (LLM refinement gate is open), the canonical `transcription` emit (`speech_final: true`) drops filler words — `um`, `uh`, `like`, `you know`, … * No effect when `context` is omitted (refinement stays off). * The 60db `/ws/stt` proxy coerces non-boolean values to `false` rather than rejecting the session. See [WebSocket STT reference](/api-reference/websocket/stt#config-remove-fillers). Surfaces: **REST**, **Email** ## Added **Onboarding fields (`POST /user/onboarding`).** * Now accepts `default_voice_id` (string), `marketing_consent` (boolean), `consent_timestamp`, and `consent_source` — all optional and backward compatible. * Marketing consent is **server-stamped**: `consent_timestamp` / `consent_source` are set on the server; any client-sent values are ignored. * Consent is persisted **best-effort after the onboarding commit**, so a consent-write failure can never fail the onboarding save. **One-click unsubscribe (`/unsubscribe`).** * Public, HMAC-token-verified endpoint for lifecycle email. Sends a `List-Unsubscribe` (one-click) header for bulk-sender compliance. ## Changed **Email delivery now uses Cloudflare Email Service.** * Transactional and lifecycle mail send via Cloudflare (`smtp.mx.cloudflare.net`, implicit TLS on 465) authenticated with a Cloudflare API token; DKIM/ARC are applied automatically. Configure `CLOUDFLARE_API_TOKEN` (used as the SMTP password) and `SMTP_EMAIL` (a sender on a domain onboarded for Email Sending). * Signup verification email is now sent **after** the account is committed, so a transient email/transport failure can no longer roll back account creation. ## Fixed **`GET /workspaces` duplicate rows.** * A workspace with more than one active membership row no longer appears multiple times in the response; the list is de-duplicated per workspace. **Onboarding 500 on consent.** * Submitting onboarding no longer returns `500` when the marketing-consent columns are absent from the database (consent persistence is best-effort — apply the consent migration to enable it). **Email credential safety.** * SMTP / Cloudflare credentials are scrubbed from any thrown or logged email error. ## Security * Cerbos authorization checks added across additional routes (STT, TTS, SLM, Studio, webhooks, sixtydb). ## Migrations * New environments must apply the consent columns on `users` (`marketing_consent`, `consent_timestamp`, `consent_source`); plus the events / lifecycle-email tables if the lifecycle email program is enabled. Surfaces: **REST** ## Added * New STT transcribe API powering the STT editor. * Audio playback in the STT editor. ## Fixed * Resolved an audio-URL leak. * Fixed audio not playing after a redirect. Surfaces: **REST**, **Billing** ## Added * **Coupon system** for wallet top-ups — percentage / fixed discounts and deposit multipliers (e.g. 2× / 3×). Redemptions are recorded for audit (`coupon_redemptions`, referencing the top-up's `payment_transaction_id`). ## Added **Longer request timeout (REST `POST /stt`).** * Server-side request timeout raised from 120 s → **600 s**; clients should match. The 25 MB / \~1-hour file-size cap remains. **Custom vocabulary boost.** * New `keywords` form field on `POST /stt`. CSV with optional per-term weights, e.g. `Acme:5,XYZ Pharma:8`. * The same boost runs on the WebSocket path via the existing `context.terms` field — no client change required to benefit; existing `terms` payloads now produce `boosted` / `original` markers automatically. * Words replaced by the boost appear in the response with `boosted: true` and `original: ""`. **Word-level timestamps and confidence.** * `return_timestamps=word` adds per-word `start` / `end` to the REST response. * `include_confidence=true` adds per-word `confidence` (0-1). **Diarization controls.** * `min_speakers` / `max_speakers` form fields on `POST /stt`. **Script correction.** * `script_correction=true` for code-mixed Devanagari / Latin audio. **Audio quality indicator.** * `snr_db` is now a top-level field on every STT response and a per-message field on WS `transcription` events. Surface as a "good / fair / poor" badge (`>= 15` good, `0–15` fair, `< 0` poor). ## Limits **Per-user concurrency cap.** Counted across REST + WS combined. | Service | Default | | ------- | ------- | | STT | 8 | | TTS | 5 | Excess returns: * **REST** — `429` with `error_code: STT_CONCURRENCY_LIMIT` / `TTS_CONCURRENCY_LIMIT` and `details.limit`. * **WebSocket** — error frame followed by close code `1008`. The cap releases when an in-flight request completes; no server-side queueing. ## New error codes | Status | `error_code` | When | Retry guidance | | ------ | ------------------------------------------------- | --------------------------------------- | ----------------------------------------- | | 429 | `STT_CONCURRENCY_LIMIT` / `TTS_CONCURRENCY_LIMIT` | Per-user concurrency cap | Retry after an in-flight request finishes | | 429 | `STT_UPSTREAM_RATE_LIMIT` | Upstream rate-limit pass-through | Honor the **`Retry-After`** HTTP header | | 499 | `STT_CLIENT_CANCELLED` | Client closed the connection mid-flight | Intentional; no retry, no charge | | 503 | `STT_UPSTREAM_UNAVAILABLE` | Upstream STT 5xx | Exponential backoff | `499` is non-standard but used (nginx convention) to signal the client closed the connection before the response was sent. ## New warning codes | Code | Meaning | | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `low_snr_dropped` | Audio dropped before LID/ASR; SNR below floor. Response `text` is empty. **No credits charged.** | | `llm_refinement_not_in_plan` | The `context` field was provided but the active plan does not include LLM refinement. The field was ignored; transcription proceeded without refinement. | The same `low_snr_dropped` value appears as a `processing_mode` on WS `transcription` events — treat as "skip this utterance, no useful text". ## Billing changes * **WS double-bill fix.** Sessions that previously double-billed when the client sent `stop` are now billed correctly. `billing_summary.total_duration_seconds` will show as \~50 % of historical values for affected sessions. * **`billing_summary.client_estimated_seconds` (new, diagnostic)** — rough estimate of audio the client sent. Useful only for debugging duration drift; never display as a billed number. * **Low-SNR refunds.** REST sessions where upstream dropped audio for low SNR no longer charge the user. WS already handled this. * **Cancellation propagation.** Client disconnects mid-request now correctly abort the upstream call. **No charge for cancelled requests** (`499 STT_CLIENT_CANCELLED`). * **Diarization surcharge documented.** When `diarize=true`, requests incur a documented **+30 %** surcharge on top of the base STT rate to cover GPU diarization (pyannote). ## Plan-tier gate `context` (LLM refinement) is now a paid feature. On the **Free** plan the field is silently stripped server-side and the response includes `warning_codes: ["llm_refinement_not_in_plan"]`. Surface an upgrade prompt or hide / disable the input on Free plans for better UX. ## Surface summary * **REST** — `POST /stt` accepts new form fields (`keywords`, `languages`, `min_speakers`, `max_speakers`, `return_timestamps`, `include_confidence`, `script_correction`, `min_split_sec`); returns new fields (`snr_db`, `language_source: "long_audio_chunked"`, per-word `boosted`/`original`/`confidence`, per-segment `chunk_idx`). * **REST `POST /tts`, `/tts-synthesize`, `/tts-stream`** — may now return `429 TTS_CONCURRENCY_LIMIT`. * **WebSocket `/ws/stt`** — adds `low_snr_dropped` to `processing_mode`; per-word `boosted`/`original`; message-level `snr_db`; concurrency-limit error frame + close `1008`; corrected `billing_summary` with new `client_estimated_seconds` field. * **WebSocket `/ws/tts`** — concurrency-limit error frame (legacy `{error: {message, code, details}}` shape) + close `1008`. * **Web UI** — axios timeout raised to 600 s, error-code-aware toasts, `warning_codes` surfaced, WS `1008` distinguished from auth/network failures. The 25 MB upload cap is unchanged. ## Added — Context-gated LLM refinement Speech-to-Text now accepts an optional **context hint** that opens a server-side LLM refinement gate. When supplied, the response transcript is polished for proper-noun accuracy, filler removal, punctuation, and script consistency in mixed-language audio. The shape differs per transport: | Transport | `context` shape | Enables | | ------------------------------------------- | ------------------------------------------ | --------------------------------- | | **REST** `POST /stt` (and `/v1/transcribe`) | plain `string` — free-form paragraph | LLM refinement of response `text` | | **WebSocket** `/v1/stream` (and `/ws/stt`) | structured object `{general, text, terms}` | Two-phase canonical final flow | ### REST — `context: string` Free-form paragraph describing the session (domain, speakers, jargon). Serialized as a multipart form field. ```bash cURL theme={null} curl -X POST https://api.60db.ai/stt \ -H "Authorization: Bearer $API_KEY" \ -F "file=@meeting.wav" \ -F "context=Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique, stamina, running between wickets." ``` ```javascript JavaScript theme={null} await client.speechToText(file, { language: "auto", diarize: true, context: "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma.", }); ``` ```python Python theme={null} client.speech_to_text( audio_file, language='auto', diarize=True, context='Cricket coaching session. Players: Arjun Mehta, Ishaan Verma.', ) ``` ```bash CLI theme={null} 60db stt:transcribe --file meeting.wav \ --context "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma." ``` ```json MCP theme={null} { "audio_url": "https://example.com/meeting.wav", "context": "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma." } ``` ### WebSocket — structured `{general, text, terms}` ```json Start message theme={null} { "type": "start", "languages": ["en", "hi"], "context": { "general": [ { "key": "domain", "value": "Cricket coaching" }, { "key": "players", "value": "Arjun Mehta, Ishaan Verma, Aryan Khan, Rohan" } ], "text": "Coach reviewing a batting practice session.", "terms": ["Arjun Mehta", "Ishaan Verma", "off-side", "stamina", "wickets"] }, "config": { "encoding": "linear", "sample_rate": 48000, "continuous_mode": true } } ``` ## Changed — WebSocket two-phase canonical flow When `context` is supplied, each utterance now produces **two** `transcription` events sharing a `sentence_id`: 1. **First emit** — `is_final: true, speech_final: false` — fast dict-corrected text. Use for low-latency UI paint and voicebot barge-in / NLU. 2. **Canonical** — `is_final: true, speech_final: true` — definitive answer. Either LLM-refined (when `llm_applied: true`) or the original text re-emitted (when LLM was skipped or failed). New canonical-only fields: | Field | Type | Description | | ---------------- | --------- | ------------------------------------------------------------------------------------- | | `llm_applied` | `boolean` | `true` if the LLM ran, `false` if skipped / failed. | | `llm_latency_ms` | `number` | Round-trip to the LLM endpoint (SLA monitoring). | | `llm_reason` | `string` | Diagnostic when `llm_applied: false` (`gate_closed`, `error:TimeoutException`, etc.). | Simple consumers can gate exclusively on `speech_final: true` and ignore the rest — one canonical event per utterance regardless of whether refinement is on. See the [WebSocket STT Reference](/api-reference/websocket/stt#canonical-answer-semantics-speech-final) for the full table and reconciliation patterns. ## Added — Word-preservation guardrails (proxy + UI) To defend against over-aggressive LLM refinement and fast-speech hallucination rejection: * **Refined word-retention guardrail.** If the canonical text drops more than \~60% of the first-emit's token count, the proxy rolls back to the first-emit text and marks `llm_applied: false, llm_reason: "dropped_too_many_words"`. Tuned so legitimate polish (filler removal, 10–40% compression) passes through untouched. * **Hallucination-rejected fallback.** When the upstream server's word-rate guard emits an empty final (`processing_mode: "hallucination_rejected"`, common on fast-paced Indic/English mixed audio), the proxy upgrades it into a tentative canonical using the cached first-emit / last-interim text plus `tentative: true` and `tentative_reason: "hallucination_rejected"` so words never silently disappear. Both guardrails run in the `/ws/stt` proxy, so voicebots and web clients inherit them automatically. ## Added — Backward-compat `refined` event handling Pre-migration upstream builds emit LLM refinement as a separate `refined` event rather than a second `transcription`. The proxy and web client transparently accept both shapes, so existing integrations keep working through the rollout. If you see `refined` events in the wire trace, upstream workers haven't been restarted onto the two-phase build yet — it's still fully functional; the `refined` event is deprecated but accepted. ## Surface summary * **REST** — `POST /stt`, `POST /v1/transcribe` — `context: string` form field. * **WebSocket** — `/v1/stream`, `/ws/stt` — `context: {general, text, terms}` on `start`; two-phase canonical flow; `llm_applied` / `llm_latency_ms` / `llm_reason` on canonical. * **JavaScript SDK** — `client.speechToText(audio, { context })`. * **Python SDK** — `client.speech_to_text(audio_file, context=...)`. * **CLI** — `60db stt:transcribe --context "..."`. * **MCP Server** — `sixtydb_stt_transcribe` accepts `context`. * **Proxy** — word-retention guardrail, hallucination fallback, first-emit caching, legacy `refined` shim. * **Web UI** — context input in Speech-to-Text page and realtime demo; segments rendered with `tentative` marker when flagged. ## Environment knobs (server-side) | Variable | Default | Purpose | | -------------------------- | ----------- | --------------------------------------------------------------------------- | | `STT_LLM_ENABLED` | `true` | Master kill-switch for refinement. | | `STT_LLM_MODEL` | `60db-tiny` | OpenAI-compatible model identifier. | | `STT_LLM_TIMEOUT_SEC` | `10.0` | Per-call timeout — on timeout, canonical falls back to original. | | `STT_LLM_MIN_WORDS` | `4` | Skip refinement for tiny utterances (`"Yeah"`, `"Okay"`). | | `STT_WS_HALLUCINATION_WPS` | `8.0` | Word-rate ceiling; finals above it are flagged as `hallucination_rejected`. | # AI Completions Source: https://docs.60db.ai/cli-reference/commands/ai AI chat, meeting analysis, and text completion commands # AI Completions ## Commands ### AI Chat Completions ```bash theme={null} 60db ai:chat --prompt "Hello, how are you?" ``` ### With Custom System Prompt ```bash theme={null} 60db ai:chat --prompt "Summarize this text" --system "You are a helpful assistant." ``` ### With Specific Model ```bash theme={null} 60db ai:chat --prompt "Generate code" --model "60db-tiny" --temperature 0.7 ``` ### AI Meeting Notes Analysis ```bash theme={null} 60db ai:meeting --transcript "Meeting transcript here..." --title "Weekly Standup" ``` ### AI Text Completion ```bash theme={null} 60db ai:complete --prompt "Once upon a time" --max-tokens 100 ``` ### With JSON Output for Agents ```bash theme={null} 60db --json ai:chat --prompt "Analyze this data" ``` ## Options ### AI Chat Options * `-p, --prompt ` - User prompt/message * `-m, --model ` - Model name (default: 60db-tiny) * `-s, --system ` - System prompt * `--messages ` - Messages as JSON array * `--max-tokens ` - Max tokens (default: 2048) * `--temperature ` - Temperature (default: 0.1) * `--top-k ` - Top K (default: 5) * `--top-p ` - Top P (default: 0.9) * `--stream ` - Enable streaming (default: false) * `--enable-thinking ` - Enable thinking (default: false) ### AI Meeting Notes Options * `-p, --prompt ` or `-t, --transcript ` - Meeting transcript * `--title ` - Meeting title * `-m, --model ` - Model name * `-s, --system ` - Custom system prompt * All other AI options supported ### AI Text Completion Options * `-p, --prompt ` - Prompt text (required) * `--max-tokens ` - Max tokens to generate ## Examples ### AI Chat Completion ```bash theme={null} # Simple chat 60db ai:chat --prompt "Explain quantum computing" # With system prompt for specific behavior 60db ai:chat --prompt "Review this code" --system "You are an expert code reviewer. Be concise and actionable." # Higher temperature for creative responses 60db ai:chat --prompt "Write a poem about AI" --temperature 0.9 ``` ### AI Meeting Notes Analysis ```bash theme={null} # Analyze meeting transcript 60db ai:meeting --transcript "John: We need to finish by Friday. Jane: I'll handle frontend." --title "Sprint Planning" # Get structured JSON output 60db --json ai:meeting --transcript "$TRANSCRIPT" > meeting-notes.json ``` # Analytics Source: https://docs.60db.ai/cli-reference/commands/analytics Analytics and usage monitoring commands # Analytics ## Get Usage Statistics ```bash theme={null} 60db analytics:usage ``` ### Options * `--user-id ` - Filter by user ID * `--start-date ` - Start date (YYYY-MM-DD) * `--end-date ` - End date (YYYY-MM-DD) * `--limit ` - Number of records to return ## Examples ### Get Overall Usage ```bash theme={null} 60db analytics:usage ``` ### Get Usage for Specific User ```bash theme={null} 60db analytics:usage --user-id 123 ``` ### Get Usage for Date Range ```bash theme={null} 60db analytics:usage --start-date 2024-01-01 --end-date 2024-01-31 ``` ### Export Usage Data ```bash theme={null} 60db --json analytics:usage > usage.json ``` ### Monitor Usage in Real-Time ```bash theme={null} watch -n 10 '60db --json analytics:usage --limit 10' ``` # Authentication Source: https://docs.60db.ai/cli-reference/commands/authentication Authentication commands for 60db CLI # Authentication ## Commands ### Login with Email and Password (Recommended) ```bash theme={null} 60db auth:login --email your@email.com --password yourpassword ``` ### Show Current Session Info ```bash theme={null} 60db auth:session ``` ### Get Current Authentication Token ```bash theme={null} 60db auth:token ``` ### Logout and Clear Token ```bash theme={null} 60db auth:logout ``` ## Examples ### Authentication & Setup ```bash theme={null} # Login to get token 60db auth:login --email your@email.com --password yourpassword # Check session 60db auth:session # Or manually set API key 60db config --set apiKey=your_token_here ``` # Billing Source: https://docs.60db.ai/cli-reference/commands/billing Billing and payment commands # Billing ## Commands ### List Invoices ```bash theme={null} 60db billing:invoices --limit 20 ``` ### Filter by User ```bash theme={null} 60db billing:invoices --user-id 123 ``` ### Filter by Status ```bash theme={null} 60db billing:invoices --status paid ``` ### List Payment Transactions ```bash theme={null} 60db billing:transactions --limit 20 ``` ### Filter by Type ```bash theme={null} 60db billing:transactions --type one_time ``` ## Options ### List Invoices * `--limit ` - Number of invoices to return * `--user-id ` - Filter by user ID * `--status ` - Filter by status (paid, pending, failed) ### List Transactions * `--limit ` - Number of transactions to return * `--type ` - Filter by type (one\_time, subscription) ## Examples ### Monitor Transactions in Real-Time ```bash theme={null} watch -n 5 '60db --json billing:transactions --limit 5' ``` # Categories Source: https://docs.60db.ai/cli-reference/commands/categories Category management commands # Categories ## Commands ### List Available Categories ```bash theme={null} 60db categories ``` # Credits Source: https://docs.60db.ai/cli-reference/commands/credits Credit management commands # Credits ## Workspace Wallet All credits and wallet balances are managed at the **workspace level**. Each workspace has its own USD wallet. When you add credits, they go to the specified workspace — not to a user account. ## Commands ### Add Bonus Credits ```bash theme={null} 60db credits:add --user-id 123 --workspace-id 24 --tts 1000 --stt 60 --voice 10 ``` ### Add Wallet Amount (to workspace) ```bash theme={null} 60db credits:add --user-id 123 --workspace-id 24 --amount 50.00 --description "Refund" ``` ### Add Both Bonus Credits and Amount ```bash theme={null} 60db credits:add --user-id 123 --workspace-id 24 --tts 1000 --amount 25.00 ``` ### Get Workspace Balance ```bash theme={null} 60db credits:balance --user-id 123 ``` ### Get Transaction History ```bash theme={null} 60db credits:history --user-id 123 --limit 20 ``` ## Options ### Add Credits * `--user-id ` - User ID (required) * `--workspace-id ` - Workspace ID (if omitted, uses user's primary workspace) * `--tts ` - TTS bonus credits (characters) * `--stt ` - STT bonus credits (in seconds) * `--voice ` - Voice cloning bonus credits * `--amount ` - Wallet amount in USD * `--description ` - Transaction description ### Get Balance * `--user-id ` - User ID (required) ### Get History * `--user-id ` - User ID (required) * `--limit ` - Number of transactions to return * `--offset ` - Offset for pagination All wallet amounts are in **USD only**. Multi-currency is not supported. ## Examples ### Add Credits to a Workspace ```bash theme={null} 60db credits:add --user-id 123 --workspace-id 24 --amount 10 --description "Monthly bonus" ``` ### Add Credits to Multiple Workspaces ```bash theme={null} for ws_id in 24 25 26; do 60db credits:add --user-id 123 --workspace-id $ws_id --amount 10 --description "Monthly bonus" done ``` # Memory & RAG Source: https://docs.60db.ai/cli-reference/commands/memory CLI commands for 60db's pay-as-you-go memory layer — ingest, upload, search, context assembly, and usage tracking # Memory & RAG The 60db CLI exposes the full Memory/RAG layer for scripting and automation. Every billable command surfaces the wallet balance and charge in the output footer so you can watch spend in real time. All memory commands are pay-as-you-go. See the [pricing reference](/api-reference/memory/pricing) for rates. When the wallet runs out, commands return `INSUFFICIENT_CREDITS` with a structured shortfall — top up via the [Dashboard](https://app.60db.ai) billing page to continue. ## Commands | Command | Billed | Purpose | | ------------------------------- | ------ | -------------------------------------------------------- | | `60db memory:ingest` | ✓ | Store a single memory | | `60db memory:upload` | ✓✓ | Extract + ingest a document (PDF, DOCX, XLSX, images...) | | `60db memory:search` | ✓ | Hybrid semantic search | | `60db memory:context` | ✓ | Assemble LLM-ready context | | `60db memory:collections` | — | List collections | | `60db memory:create-collection` | — | Create team/knowledge/hive collection | | `60db memory:usage` | — | Monthly spend breakdown | | `60db memory:status` | — | Poll a memory's ingestion status | | `60db memory:delete` | — | Soft-delete a memory (24h undo) | ## Ingest a memory Store a single fact, preference, or text snippet: ```bash theme={null} 60db memory:ingest \ --text "User prefers dark mode and metric units" \ --type user \ --title "UI preferences" ``` **Options:** * `-t, --text ` — Memory content (required, max 100,000 chars) * `--title ` — Optional display title * `-c, --collection <id>` — Target collection (defaults to personal) * `--type <type>` — `user` | `knowledge` | `hive` (default: `user`) * `--no-infer` — Skip LLM-based fact extraction (default: infer is on) **Cost**: \$0.0001 per 1,000 characters. ## Upload a document Upload a file — PDF, DOCX, XLSX, PPTX, EML, MSG, HTML, TXT, Markdown, CSV, PNG, JPG, TIFF, and 70+ other formats — with built-in OCR for scanned content: ```bash theme={null} 60db memory:upload \ --file ~/Documents/quarterly-report.pdf \ --collection company_reports \ --type knowledge \ --title "Q4 2026 Report" ``` **Options:** * `-f, --file <path>` — Absolute or relative path to the document (required) * `-c, --collection <id>` — Target collection * `--type <type>` — `user` | `knowledge` | `hive` (default: `knowledge`) * `--title <title>` — Display title (defaults to filename) * `--chunk-size <n>` — Max characters per chunk, 200-8000 (default: 1500) * `--chunk-overlap <n>` — Character overlap between chunks (default: 200) **Max file size**: 200 MB. **Max chunks per document**: 100. **Cost** (two-stage): * Extract fee: \$0.003 per MB * Ingest fee: \$0.0001 per 1,000 extracted characters Both fees are automatically refunded on any failure. ### Tuning tips | Document type | `--chunk-size` | `--chunk-overlap` | | ------------------------- | -------------- | ----------------- | | Technical docs / API refs | 1500 | 200 (default) | | Long-form prose / books | 2500 | 300 | | FAQs / short snippets | 800 | 100 | | Spreadsheet exports | 3000 | 0 | | Scanned PDFs (OCR) | 2000 | 250 | ## Search memories Hybrid semantic + keyword recall with optional cross-encoder reranking: ```bash theme={null} 60db memory:search \ --query "What is the refund policy?" \ --mode fast \ --limit 5 \ --alpha 0.8 ``` **Options:** * `-q, --query <text>` — Search query (required, max 2,000 chars) * `-c, --collection <id>` — Collection to search * `--mode <mode>` — `fast` (\~100-200ms) or `thinking` (wider candidate pool + cross-encoder rerank, \~200-400ms) * `--limit <n>` — Max results (1-50, default: 10) * `--alpha <n>` — 0 (keyword only) to 1 (semantic only), default 0.8 * `--recency-bias <n>` — Weight for newer memories (0-1, default: 0) * `--graph` — Include knowledge-graph relationships **Advanced reranker knobs** (override server defaults): * `--rerank-top-k <n>` — Max candidates the cross-encoder reranks * `--rerank-timeout <ms>` — Hard timeout for rerank call * `--min-rerank-score <n>` — Drop results below this rerank score (0-1) * `--fetch-multiplier <n>` — In thinking mode, fetch N x limit candidates **Cost**: flat \$0.0003 per query. When the cross-encoder reranker is active, results include both a **dense score** (vector similarity) and a **rerank score** (cross-encoder confidence). The rerank score is the more reliable signal for ranking quality. ### Example — tune by query type ```bash theme={null} # Exact phrase match (keyword-heavy) 60db memory:search -q "invoice #INV-2026-042" --alpha 0.2 # Conceptual question (semantic-heavy) 60db memory:search -q "how do I configure SSO?" --alpha 0.9 # Complex multi-faceted question (thinking mode + reranking) 60db memory:search -q "what are common escalation patterns?" --mode thinking --alpha 0.7 # Recent events focus 60db memory:search -q "product updates" --alpha 0.6 --recency-bias 0.3 # Override reranker settings for this query 60db memory:search -q "detailed compliance requirements" --mode thinking --min-rerank-score 0.3 --fetch-multiplier 5 ``` ## Assemble LLM context Purpose-built for RAG — returns a pre-formatted string ready to prepend to an LLM system message: ```bash theme={null} 60db memory:context \ --query "Tell me about customer preferences" \ --top-k 8 \ --max-context-length 2000 ``` **Options:** * `-q, --query <text>` — Query driving retrieval (required) * `--session-id <id>` — Chat session ID for hierarchical context * `--top-k <n>` — Max memories to pull (default: 10) * `--max-context-length <n>` — Max assembled tokens (default: 4000) * `--graph` — Include graph relationships * `--no-timeline` — Exclude recent events **Graceful degradation**: returns an empty prompt on outage — chat keeps working. **Cost**: flat \$0.0005 per query. ## Manage collections ### List collections ```bash theme={null} 60db memory:collections ``` Shows every collection visible to you in the current workspace with kind (personal/team/knowledge/hive) and shared status. ### Create a collection **Admin/owner only.** Personal collections are auto-created per user — you don't create those manually. ```bash theme={null} 60db memory:create-collection \ --id customer_support \ --label "Customer Support KB" \ --kind knowledge ``` **Options:** * `-i, --id <id>` — Collection ID (required, lowercase alphanumeric + underscores) * `-l, --label <label>` — Human-readable label (required) * `-k, --kind <kind>` — `team` | `knowledge` | `hive` (default: `team`) * `--no-shared` — Don't share with workspace members Both commands are **unbilled**. ## Monitor spend ```bash theme={null} 60db memory:usage 60db memory:usage --period last_30_days 60db memory:usage --period all_time ``` Shows net spend, operation count, refund count, and a per-service-type breakdown (ingest / extract / recall / context). Always free. Works even when the wallet is empty. **Options:** * `--period <period>` — `current_month` (default) | `last_30_days` | `all_time` ## Poll a memory's status Memories are processed asynchronously. Use this to watch a specific ID: ```bash theme={null} 60db memory:status --id mem_01HV8K2X3N4P5Q6R7S8T9U ``` Statuses: `pending` → `processing` → `ready` (or `failed`). Unbilled. ## Delete a memory ```bash theme={null} 60db memory:delete --id mem_01HV8K2X3N4P5Q6R7S8T9U ``` Soft-delete with 24-hour undo grace period. You can delete memories you created; admins/owners can delete any memory. ## Agent-friendly JSON output All memory commands respect the global `--json` flag for machine-readable output: ```bash theme={null} 60db --json memory:search --query "user preferences" --limit 5 ``` Returns the same data as the API but with a structured envelope that includes a `billing: {charged, balance, txId}` object on billable commands. ## Handling insufficient credits When the wallet balance is below the operation cost, commands return a `402`-style error with structured details: ```bash theme={null} $ 60db memory:ingest --text "hello" --type user ✗ Insufficient credits in workspace wallet required: $0.000100 available: $0.000010 shortfall: $0.000090 ℹ Top up your workspace wallet via the Dashboard: https://app.60db.ai ``` The `--json` output surfaces the same details as: ```json theme={null} { "success": false, "error": "INSUFFICIENT_CREDITS", "details": { "required": 0.0001, "available": 0.00001, "shortfall": 0.00009 } } ``` ## Related * [Memory pricing & billing](/api-reference/memory/pricing) * [Memory feature overview](/features/memory) * [CLI: authz](/cli-reference/commands/authz) * [MCP: memory tools](/mcp-server/memory) # Speech-to-Text Source: https://docs.60db.ai/cli-reference/commands/stt STT commands for transcribing audio files # Speech-to-Text (STT) ## Commands ### Transcribe Audio File ```bash theme={null} 60db stt:transcribe --file audio.wav ``` By default the language is auto-detected. Equivalent to passing `--language auto`. ### Specify Language ```bash theme={null} 60db stt:transcribe --file audio.wav --language hi ``` Pass an ISO 639-1 code from the supported set (see `60db stt:languages`). Supported codes include `en`, `hi`, `bn`, `mr`, `pa`, `gu`, `or`, `as`, `ne`, `ta`, `te`, `kn`, `ml`, `sa`, `ar`, and 25 European languages. ### Auto-detect (explicit) ```bash theme={null} 60db stt:transcribe --file meeting.wav --language auto ``` `--language auto` is treated as "omit" — the CLI strips it before sending so the server runs its language identification across all 39 supported languages. You can also simply omit the `--language` flag entirely for the same behavior. ### Enable Speaker Diarization ```bash theme={null} 60db stt:transcribe --file meeting.wav --diarize true ``` Each response segment will include a `speakers` array with labels like `SPEAKER_00`, `SPEAKER_01`. Adds \~50–150 ms of processing latency. ### Add Context (optional refinement) ```bash theme={null} 60db stt:transcribe --file visit.wav \ --context "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique." ``` When `--context` is supplied, the server runs a background LLM refinement pass and the returned `text` is polished for proper nouns, filler removal, and punctuation. Omit `--context` to skip refinement. <Note> `--context` takes a **plain string** — the REST `/stt` endpoint shape. The WebSocket `/ws/stt` endpoint accepts a structured `{general, text, terms}` **object** instead; see the [WebSocket STT reference](/api-reference/websocket/stt). </Note> ### List Available Languages ```bash theme={null} 60db stt:languages ``` Returns the 39-language catalog plus the `auto` entry, sourced from `GET /stt/languages`. ## Options * `-f, --file <path>` — Audio file path (required; max 25 MB; WAV / MP3 / M4A / OGG / FLAC / WebM) * `-l, --language <code>` — ISO 639-1 language code (e.g. `en`, `hi`, `ar`). Omit or pass `auto` for auto-detection across the 39 supported languages. * `--diarize <boolean>` — Enable speaker diarization (default: `false`) * `--context <text>` — Free-form paragraph describing the session (domain, speakers, jargon). Enables server-side LLM refinement of the response text. ## Examples ### STT — Transcribe Audio ```bash theme={null} # Auto-detect language (simplest) 60db stt:transcribe --file meeting.wav # Auto-detect, explicit form 60db stt:transcribe --file meeting.wav --language auto # Force Hindi — skips language identification for lowest latency 60db stt:transcribe --file recording.wav --language hi # Multi-speaker meeting with diarization + auto-detect 60db stt:transcribe --file board-call.wav --diarize true # List supported languages 60db stt:languages ``` ## Notes * **Do not pass unsupported language codes.** The server explicitly rejects `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he` (and Arabic dialect tags like `ar-eg`) with an `unsupported_language` error. Use `auto` or omit the flag for these. * **Auto-detect on REST vs WebSocket.** The REST (`stt:transcribe`) form flow accepts `--language auto` as a convenience. The streaming WebSocket form requires `languages: null`, not the literal string `"auto"`. If you write your own WebSocket client, don't forward the string `"auto"` — send `null`. * A successful response with empty `text` and `warning_codes: ["no_speech_detected"]` means the audio contained no speech (silence / music / noise). This is not an error — do not retry. # Text-to-Speech Source: https://docs.60db.ai/cli-reference/commands/tts TTS commands for generating speech from text # Text-to-Speech (TTS) ## Commands ### Synthesize Speech from Text ```bash theme={null} 60db tts:synthesize --text "Hello this is devendra" --voice-id "voice_id_here" ``` ### Specify Output File ```bash theme={null} 60db tts:synthesize --text "Hello world" --voice-id "abc123" --output speech.mp3 ``` ### Adjust Speech Parameters ```bash theme={null} 60db tts:synthesize --text "Hello" --voice-id "abc123" --speed 1.2 --stability 50 --similarity 75 ``` ### List Available Voices ```bash theme={null} 60db tts:voices ``` ## Options * `-t, --text <text>` - Text to synthesize (required) * `-v, --voice-id <id>` - Voice ID to use (required) * `-o, --output <file>` - Output audio file path (default: `tts_<timestamp>.mp3`) * `--speed <number>` - Speech speed (default: 1) * `--stability <number>` - Voice stability 0-100 (default: 50) * `--similarity <number>` - Voice similarity 0-100 (default: 75) ## Examples ### TTS - Generate Speech ```bash theme={null} # Generate speech from text 60db tts:synthesize --text "Hello, this is devendra" --voice-id "1a8c6331-c79b-47d3-9893-09160a245a3e" # List available voices first 60db tts:voices # Save to specific file 60db tts:synthesize --text "Welcome to our service" --voice-id "abc123" --output welcome.mp3 ``` # Users Source: https://docs.60db.ai/cli-reference/commands/users User management commands # Users ## Commands ### List All Users ```bash theme={null} 60db users --list --page 1 --limit 10 ``` ### Get User Details ```bash theme={null} 60db users --get 123 ``` ### Search Users ```bash theme={null} 60db users --search "john@example.com" ``` ### Create New User ```bash theme={null} 60db user:create --email user@example.com --name "John Doe" --password secret123 ``` ### Update User ```bash theme={null} 60db user:update --id 123 --name "Jane Doe" --active true ``` ### Delete User ```bash theme={null} 60db user:delete --id 123 ``` ## Options ### List Users * `--page <number>` - Page number * `--limit <number>` - Items per page ### Get User * `--id <id>` - User ID (required) ### Search Users * `--search <query>` - Search query (email or name) ### Create User * `--email <email>` - User email (required) * `--name <name>` - Full name * `--password <password>` - Password * `--role <role>` - System role (default: user) ### Update User * `--id <id>` - User ID (required) * `--email <email>` - New email * `--name <name>` - New name * `--active <boolean>` - Active status * `--role <role>` - System role ### Delete User * `--id <id>` - User ID (required) ## Examples ### Export User Data to JSON ```bash theme={null} 60db --json users --list > users.json ``` # Voices Source: https://docs.60db.ai/cli-reference/commands/voices Voice management commands for TTS # Voices ## List Available Voices ```bash theme={null} 60db tts:voices ``` This command returns all available voices that can be used for text-to-speech synthesis. ## Examples ### Get All Voices ```bash theme={null} 60db tts:voices ``` ### Export Voices to JSON ```bash theme={null} 60db --json tts:voices > voices.json ``` ### Use Voice in Synthesis ```bash theme={null} # First get voice ID 60db tts:voices # Use the voice ID in synthesis 60db tts:synthesize --text "Hello" --voice-id "voice_id_here" ``` ## Voice Properties Each voice has the following properties: * `id` - Unique voice identifier * `name` - Voice name * `language` - Language code * `gender` - Voice gender (male, female, neutral) * `description` - Voice description Use these properties when selecting a voice for synthesis. # Workspaces Source: https://docs.60db.ai/cli-reference/commands/workspaces Workspace management commands # Workspaces ## Commands ### List All Workspaces ```bash theme={null} 60db workspaces --list ``` ### Get Workspace Details ```bash theme={null} 60db workspaces --get 456 ``` ### Filter by User ```bash theme={null} 60db workspaces --list --user-id 123 ``` ### Create Workspace ```bash theme={null} 60db workspace:create --name "My Workspace" --owner-id 123 --description "Project workspace" ``` ## Options ### List Workspaces * `--user-id <id>` - Filter by user ID ### Get Workspace * `--id <id>` - Workspace ID (required) ### Create Workspace * `--name <name>` - Workspace name (required) * `--owner-id <id>` - Owner user ID (required) * `--description <text>` - Workspace description # Configuration Source: https://docs.60db.ai/cli-reference/configuration Configure 60db CLI with API keys and settings # Configuration ## Commands ### Show Current Configuration ```bash theme={null} 60db config ``` ### Set API URL ```bash theme={null} 60db config --set apiBaseUrl=https://api.60db.ai ``` ### Set API Key ```bash theme={null} 60db config --set apiKey=your_api_key_here ``` ### List All Config ```bash theme={null} 60db config --list ``` ### Clear Config ```bash theme={null} 60db config --clear ``` # Examples Source: https://docs.60db.ai/cli-reference/examples Practical examples for using 60db CLI # Examples ## Authentication & Setup ```bash theme={null} # Login to get token 60db auth:login --email your@email.com --password yourpassword # Check session 60db auth:session # Or manually set API key 60db config --set apiKey=your_token_here ``` ## AI Chat Completion ```bash theme={null} # Simple chat 60db ai:chat --prompt "Explain quantum computing" # With system prompt for specific behavior 60db ai:chat --prompt "Review this code" --system "You are an expert code reviewer. Be concise and actionable." # Higher temperature for creative responses 60db ai:chat --prompt "Write a poem about AI" --temperature 0.9 ``` ## TTS - Generate Speech ```bash theme={null} # Generate speech from text 60db tts:synthesize --text "Hello, this is devendra" --voice-id "1a8c6331-c79b-47d3-9893-09160a245a3e" # List available voices first 60db tts:voices # Save to specific file 60db tts:synthesize --text "Welcome to our service" --voice-id "abc123" --output welcome.mp3 ``` ## STT - Transcribe Audio ```bash theme={null} # Transcribe audio file 60db stt:transcribe --file meeting.wav # Transcribe Hindi audio 60db stt:transcribe --file recording.wav --language hi # List available languages 60db stt:languages ``` ## AI Meeting Notes Analysis ```bash theme={null} # Analyze meeting transcript 60db ai:meeting --transcript "John: We need to finish by Friday. Jane: I'll handle frontend." --title "Sprint Planning" # Get structured JSON output 60db --json ai:meeting --transcript "$TRANSCRIPT" > meeting-notes.json ``` ## Add Credits to Multiple Users ```bash theme={null} for user_id in 123 456 789; do 60db credits:add --user-id $user_id --amount 10 --description "Monthly bonus" done ``` ## Export User Data to JSON ```bash theme={null} 60db --json users --list > users.json ``` ## Monitor Transactions in Real-Time ```bash theme={null} watch -n 5 '60db --json billing:transactions --limit 5' ``` # Global Options Source: https://docs.60db.ai/cli-reference/global-options Global options available for all 60db CLI commands # Global Options These options can be used with any command: ## Available Options ### `-j, --json` Output in JSON format (for agent consumption) ```bash theme={null} 60db --json users --list ``` ### `-v, --verbose` Enable verbose output ```bash theme={null} 60db --verbose users --list ``` ### `-V, --version` Output the version number ```bash theme={null} 60db --version ``` ### `-h, --help` Display help for command ```bash theme={null} 60db --help 60db users --help ``` ### `--api-url <url>` API base URL (overrides config) ```bash theme={null} 60db --api-url https://api.60db.ai users --list ``` ### `--api-key <key>` API key for authentication (overrides config) ```bash theme={null} 60db --api-key your_key users --list ``` # Installation Source: https://docs.60db.ai/cli-reference/installation How to install 60db CLI globally or locally # Installation **Source code:** [github.com/60db-ai/cli](https://github.com/60db-ai/cli) · **npm:** [`60db-cli`](https://www.npmjs.com/package/60db-cli) ## Prerequisites * Node.js 16 or higher * npm or yarn package manager ## Global Installation Install globally to use the `60db` command anywhere: ```bash theme={null} npm install -g 60db-cli ``` Verify installation: ```bash theme={null} 60db --version ``` ## Local Installation Install as a dependency in your project: ```bash theme={null} npm install 60db-cli ``` Use with npx: ```bash theme={null} npx 60db users --list ``` ## Install from Source Clone the public GitHub repo and link the binary locally — useful when you want to build from a specific branch or commit: ```bash theme={null} git clone https://github.com/60db-ai/cli.git cd cli npm install npm link # exposes the `60db` binary on your PATH 60db --version ``` ## Post-Installation After installation, configure your API credentials: ```bash theme={null} # Set API URL 60db config --set apiBaseUrl=https://api.60db.ai # Set API key 60db config --set apiKey=your_api_key_here # Or login with email/password 60db auth:login --email your@email.com --password yourpassword ``` ## Troubleshooting ### Command not found If you get `command not found` after global installation: 1. Check npm global bin directory: ```bash theme={null} npm config get prefix ``` 2. Add it to your PATH (if not already): ```bash theme={null} export PATH=$(npm config get prefix)/bin:$PATH ``` 3. Or reinstall with sudo (Linux/Mac): ```bash theme={null} sudo npm install -g 60db-cli ``` # 60db CLI Source: https://docs.60db.ai/cli-reference/introduction Agent-native command-line interface for the full 60db platform — AI completions, TTS, STT, voice cloning, memory/RAG, Cerbos policy checks, billing, and workspaces with structured JSON output. # 60db CLI Agent-native command-line interface for the full 60db platform. Covers AI completions, TTS, STT, voice cloning, persistent memory with document upload (PDF, DOCX, XLSX, scanned images with OCR), Cerbos policy checks, billing, and workspace management — all with structured JSON output for scripts and AI agents. **Source code:** [github.com/60db-ai/cli](https://github.com/60db-ai/cli) · **npm:** [`60db-cli`](https://www.npmjs.com/package/60db-cli) · **License:** MIT ## Features * **Dual Mode**: Interactive REPL + subcommand interface * **Agent-Native**: Structured JSON output with `--json` flag * **AI Completions**: Chat, meeting notes analysis, and text completion * **TTS (Text-to-Speech)**: Generate speech from text with multiple voices * **STT (Speech-to-Text)**: Transcribe audio files with multiple languages * **Memory & RAG**: Ingest memories, upload documents (91+ formats with OCR), hybrid semantic search, LLM-ready context assembly, per-month spend tracking * **Authorization (Cerbos)**: Inspect effective permissions and probe `(resource, action)` checks before attempting gated operations * **Authentication**: Secure login and token management * **Complete Coverage**: Users, Credits, Billing, Workspaces, Categories, Memory, Authz * **Configuration**: Persistent config with `60db config` ## Installation ### Global Installation ```bash theme={null} npm install -g 60db-cli ``` ### Local Installation ```bash theme={null} npm install 60db-cli ``` ### From Source ```bash theme={null} git clone https://github.com/60db-ai/cli.git cd cli npm install npm link # exposes the `60db` binary globally ``` ## Quick Start ### Interactive Mode (REPL) ```bash theme={null} 60db 60db --version ``` Enter the interactive shell with tab completion and persistent session: ``` 60db> users 60db> credits:add --user-id 123 --amount 50 60db> config --list 60db> exit ``` ### Command Mode ```bash theme={null} # List all users 60db users --list # Add credits to user 60db credits:add --user-id 123 --amount 50 --currency USD # Upload a document to memory with OCR 60db memory:upload --file ~/Documents/handbook.pdf --type knowledge # Search with hybrid semantic + keyword recall 60db memory:search --query "refund policy" --limit 5 # Check if current user can ingest memories 60db authz:check --resource memory --action create # Get JSON output for agent consumption 60db --json users --list ``` ## Environment Variables ```bash theme={null} # Set API URL export SIXTYDB_API_URL=https://api.60db.ai # Set API Key export SIXTYDB_API_KEY=your_api_key_here ``` Legacy `X60DB_*` variable names are still honored for backward compatibility. ## JSON Output (for AI Agents) All commands support `--json` flag for structured output: ```bash theme={null} 60db --json users --list ``` Response: ```json theme={null} { "success": true, "users": [ { "id": 123, "email": "user@example.com", "full_name": "John Doe", "system_role": "user", "is_active": true, "is_verify_email": true, "created_at": "2024-01-15T10:30:00Z" } ], "pagination": { "total_users": 1, "total_pages": 1, "current_page": 1, "limit": 10 } } ``` ## Error Handling All commands return structured error responses: ```json theme={null} { "success": false, "error": "User not found" } ``` ## Requirements * Node.js >= 16.0.0 * npm or yarn ## Support For issues or questions, please visit: * GitHub: [https://github.com/60db-ai/cli](https://github.com/60db-ai/cli) * Issues: [https://github.com/60db-ai/cli/issues](https://github.com/60db-ai/cli/issues) ## License MIT # Chat Source: https://docs.60db.ai/features/llm-chat Powerful AI chat with text correction, streaming, and function calling ## Overview 60db's LM (Large Language Model) API provides intelligent chat completions with advanced text correction capabilities, streaming responses, and function calling support. Our Small Language Model (SLM) is optimized for fast, efficient responses. ## Features <CardGroup> <Card title="OpenAI Compatible" icon="code"> Drop-in compatible with OpenAI's chat completion format </Card> <Card title="Text Correction" icon="edit"> Smart text correction with dictionary and style options </Card> <Card title="Real-time Streaming" icon="signal-stream"> Server-Sent Events for instant response streaming </Card> <Card title="Function Calling" icon="zap"> Built-in tool/function calling support </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const response = await client.chat.completions.create({ model: '60db-tiny', messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'How can I improve my English?' } ], stream: true }); for await (const chunk of response) { console.log(chunk.choices[0]?.delta?.content); } ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') response = client.chat.completion( model='60db-tiny', messages=[ {'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'How can I improve my English?'} ], stream=True ) print(response['choices'][0]['message']['content']) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl --location 'https://api.60db.ai/v1/chat/completions' \ --header 'Content-Type: application/json' \ --data '{ "model": "60db-tiny", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello! How are you?"} ], "stream": true }' ``` </Tab> </Tabs> ## Streaming Responses Real-time streaming with Server-Sent Events: ```javascript theme={null} const response = await fetch("https://api.60db.ai/v1/chat/completions", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ model: "60db-tiny", messages: [{ role: "user", content: "Tell me a story" }], stream: true, }), }); const reader = response.body.getReader(); const decoder = new TextDecoder(); while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = decoder.decode(value); const lines = chunk.split("\n"); for (const line of lines) { if (line.startsWith("data: ")) { const data = JSON.parse(line.slice(6)); if (data.choices?.[0]?.delta?.content) { console.log(data.choices[0].delta.content); } } } } ``` ## Function Calling (Tools) Define tools the model can use: ```javascript theme={null} const response = await client.chat.completions.create({ model: "60db-tiny", messages: [ { role: "user", content: "What is the weather in San Francisco?" }, ], tool: [ { name: "get_weather", description: "Get the current weather for a location", parameters: { type: "object", properties: { location: { type: "string", description: "The city and state, e.g. San Francisco, CA", }, unit: { type: "string", enum: ["celsius", "fahrenheit"], }, }, required: ["location"], }, }, ], }); ``` ### Streaming Response (SSE) ``` data: {"chat_id": "550e8400-e29b-41d4-a716-446655440000", "type": "chat_id"} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": "I'm"}}]} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": " doing"}}]} data: {"id": "chatcmpl-123", "choices": [{"delta": {"content": " well!"}}]} data: {"type": "done", "response_time_ms": 1250} data: [DONE] ``` ## Application Contexts Supported contexts for text correction: | Context | Description | | ---------- | -------------------------------- | | `email` | Professional email communication | | `chat` | Casual chat/messaging | | `document` | Formal documents/reports | | `message` | Short messages/notifications | | `social` | Social media posts | ```javascript theme={null} const response = await client.chat.completions.create({ text: "hey can u send me the report pls", appContext: "email", style: { tone: "professional", autoCapitalize: true }, }); // Output: "Hey, can you send me the report, please?" ``` ## Best Practices <AccordionGroup> <Accordion title="Prompt Engineering"> * Be specific in your system prompt * Provide examples for few-shot learning * Use appropriate tone for your use case * Test different prompts for optimal results </Accordion> <Accordion title="Streaming"> * Use streaming for better UX - Handle connection errors gracefully - Buffer chunks for smooth display - Implement timeout handling </Accordion> <Accordion title="Cost Optimization"> * Cache common responses - Use shorter prompts when possible - Enable chat history for context - Monitor token usage </Accordion> <Accordion title="Text Correction"> * Use dictionary for domain-specific terms * Set appropriate app context * Test style options for your use case * Combine multiple style options </Accordion> </AccordionGroup> ## Use Cases ### Customer Support Chatbot ```javascript theme={null} async function handleCustomerMessage(message) { const response = await client.chat.completions.create({ model: "60db-tiny", messages: [ { role: "system", content: "You are a helpful customer support assistant. Be friendly and professional.", }, { role: "user", content: message }, ], stream: true, save_chat: true, chat_id: customerId, }); return response; } ``` ### Text Correction Service ```javascript theme={null} async function correctText(text, context = "email") { const response = await client.chat.completions.create({ text: text, appContext: context, dictionary: commonTypos, style: { tone: "professional", autoCapitalize: true, autoPunctuate: true, }, }); return response.choices[0].message.content; } ``` ### AI Assistant with Tools ```javascript theme={null} async function aiAssistant(userQuery) { const response = await client.chat.completions.create({ messages: [{ role: "user", content: userQuery }], tool: [getWeatherTool, searchDatabaseTool, sendEmailTool], }); // Handle tool calls and return results return response; } ``` ## Pricing Charges are based on token usage: | Metric | Cost | | --------- | ----------- | | Per Token | \~\$0.00002 | Token usage is tracked and billed to your workspace. Monitor usage via the Analytics API. ## API Reference For detailed API documentation, see: <CardGroup> <Card title="Chat" icon="code" href="/api-reference/llm/chat-completion"> Complete API reference with all parameters and examples </Card> </CardGroup> # Memory & RAG Source: https://docs.60db.ai/features/memory Persistent memory, semantic recall, and retrieval-augmented generation (RAG) for AI chat ## Overview 60db's **Memory** system gives your AI applications a long-term, searchable memory. It stores user preferences, conversation history, knowledge base documents, and arbitrary facts, then retrieves the most relevant ones on demand using hybrid semantic + keyword search. Built on a multi-layer retrieval architecture combining vector search with a knowledge graph, Memory enables: * **Personalized AI chat** — the SLM remembers user preferences across sessions * **Knowledge base Q\&A** — ingest docs and retrieve grounded answers * **Multi-user collaboration** — shared "team" memory collections * **Graph-aware recall** — find related concepts through knowledge-graph traversal <CardGroup> <Card title="Hybrid Search" icon="magnifying-glass"> Semantic (vector) + keyword (BM25) scoring with configurable weights </Card> <Card title="Context Assembly" icon="brain"> One-shot endpoint returns an LLM-ready context string for any query </Card> <Card title="Graph Relationships" icon="diagram-project"> Extracted facts link together as a knowledge graph </Card> <Card title="Multi-Collection" icon="folders"> Personal, team, knowledge, and hive (cross-collection) memory types </Card> <Card title="Document Upload" icon="file-arrow-up"> Upload PDFs, Office docs, and scanned images — text extraction + OCR built in </Card> <Card title="91+ Formats" icon="files"> PDF, DOCX, XLSX, PPTX, EML, MSG, HTML, scanned images with built-in OCR </Card> </CardGroup> ## Core concepts ### Memory collections Memories live in **collections** scoped to your workspace. Each collection is one of: | Kind | Visibility | Who can write | | ------------- | --------------------------------- | ------------------------------------- | | **personal** | Only the owning user | Owner + admin (automatic per-user) | | **team** | All members of the workspace | Owner/admin create; all members write | | **knowledge** | All members (read-only reference) | Owner/admin only | | **hive** | Cross-collection shared facts | Owner/admin only | Your personal collection is created automatically the first time you use Memory. Team/knowledge/hive collections are created by owners/admins. ### Memory types When you store a memory, you specify its type: * **`user`** — private, user-scoped facts. Auto-extracted from conversations or manually entered. * **`knowledge`** — reference content (docs, policies, FAQs). Read by all members. * **`hive`** — workspace-wide shared facts that appear in every user's search results. ### Two search modes * **Fast** — single-query hybrid search. Returns in \~100ms. Default. * **Thinking** — multi-query expansion with reranking. Better recall quality, slower (\~1-2s). ## Storing memories <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Store a personal memory await client.memory.ingest({ text: "I prefer vegetarian food and am lactose intolerant", type: "user", infer: true, // extract structured facts via LLM }); // Store a knowledge base entry await client.memory.ingest({ text: "Our refund policy allows returns within 30 days of purchase.", title: "Refund Policy", type: "knowledge", collection: "company_handbook", }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient(api_key="your-api-key") # Store a personal memory client.memory.ingest( text="I prefer vegetarian food and am lactose intolerant", type="user", infer=True, ) # Store a knowledge base entry client.memory.ingest( text="Our refund policy allows returns within 30 days of purchase.", title="Refund Policy", type="knowledge", collection="company_handbook", ) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/ingest \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "text": "I prefer vegetarian food and am lactose intolerant", "type": "user", "infer": true }' ``` </Tab> </Tabs> ### Response ```json theme={null} { "success": true, "data": { "collection_id": "user_abc123", "collection_label": "Personal Memories", "memory_type": "user", "total_queued": 1, "results": [ { "id": "mem_01HV...", "status": "pending", "message": "Queued for processing" } ] } } ``` Memories are processed asynchronously. Poll `GET /memory/:id/status` to check if ingestion is complete. ## Uploading documents For longer-form content — PDFs, Word docs, spreadsheets, scanned pages, emails — use `POST /memory/documents/extract`. The server handles **format detection, OCR, and chunking** for you, so you just upload the raw file and 60db does the rest. Under the hood, 60db runs your file through its document extraction engine (91+ formats with built-in OCR for scanned documents), splits the extracted text into overlapping chunks, and ingests each chunk as a `knowledge`-type memory in one batch. <Tabs> <Tab title="JavaScript"> ```javascript theme={null} // Upload a PDF from a file input const form = new FormData(); form.append('file', pdfFile); // File or Blob form.append('collection', 'company_handbook'); form.append('type', 'knowledge'); form.append('title', 'Employee Handbook 2026'); const res = await fetch('https://api.60db.com/memory/documents/extract', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key' }, body: form, // do NOT set Content-Type — browser adds multipart boundary }); const { data } = await res.json(); console.log(`Uploaded ${data.filename} → ${data.chunks} chunks (${data.characters} chars)`); ``` </Tab> <Tab title="Python"> ```python theme={null} import requests with open('handbook.pdf', 'rb') as f: res = requests.post( 'https://api.60db.com/memory/documents/extract', headers={'Authorization': 'Bearer sk_your_api_key'}, files={'file': ('handbook.pdf', f, 'application/pdf')}, data={ 'collection': 'company_handbook', 'type': 'knowledge', 'title': 'Employee Handbook 2026', }, ) data = res.json()['data'] print(f"Uploaded {data['filename']} → {data['chunks']} chunks") ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/documents/extract \ -H "Authorization: Bearer sk_your_api_key" \ -F "file=@handbook.pdf" \ -F "collection=company_handbook" \ -F "type=knowledge" \ -F "title=Employee Handbook 2026" ``` </Tab> </Tabs> ### What you can upload | Category | Formats | | --------------------- | --------------------------------------------- | | **Documents** | PDF, DOCX, DOC, ODT, RTF, TXT, MD, HTML, EPUB | | **Spreadsheets** | XLSX, XLS, CSV, ODS | | **Presentations** | PPTX, PPT, ODP | | **Email** | EML, MSG, PST, MBOX | | **Images (OCR)** | PNG, JPG, JPEG, TIFF, BMP, GIF | | **Code & structured** | JSON, XML, YAML, LaTeX | | **Archives** | ZIP, TAR, GZIP, 7Z (extracted recursively) | Max file size: **200 MB**. Max chunks per document: **100** (tune `chunk_size` for larger docs). ### Response ```json theme={null} { "success": true, "data": { "collection_id": "company_handbook", "collection_label": "Company Handbook", "filename": "handbook.pdf", "chunks": 18, "characters": 24680, "memory_type": "knowledge", "total_queued": 18, "results": [ /* one {id, status, message} per chunk */ ], "metadata": { "source": "document_upload", "mime_type": "application/pdf", "page_count": 24, "detected_languages": ["eng"], "total_chunks": 18 } } } ``` <Tip> The returned `metadata.page_count` and `detected_languages` come from the document extraction engine and are useful for displaying upload progress or filtering by source language. Scanned PDFs will list the OCR-detected language codes (`eng`, `fra`, `spa`, etc.). </Tip> ### Chunking controls Two optional form fields tune how text is split: | Field | Default | Description | | --------------- | ------- | ----------------------------------------------------- | | `chunk_size` | 1500 | Max characters per chunk (200–8000) | | `chunk_overlap` | 200 | Characters of overlap between chunks (\< chunk\_size) | See the [API reference](/api-reference/memory/extract-document) for a tuning table by document type. ## Searching memories <Tabs> <Tab title="JavaScript"> ```javascript theme={null} const results = await client.memory.search({ query: "What are my dietary preferences?", mode: "fast", max_results: 10, alpha: 0.8, // 0.8 semantic / 0.2 keyword recency_bias: 0.1, }); results.data.sources.forEach(source => { console.log(source.title, source.text, source.score); }); ``` </Tab> <Tab title="Python"> ```python theme={null} results = client.memory.search( query="What are my dietary preferences?", mode="fast", max_results=10, alpha=0.8, recency_bias=0.1, ) for source in results["data"]["sources"]: print(source["title"], source["text"], source["score"]) ``` </Tab> <Tab title="cURL"> ```bash theme={null} curl -X POST https://api.60db.com/memory/search \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "query": "What are my dietary preferences?", "mode": "fast", "max_results": 10, "alpha": 0.8 }' ``` </Tab> </Tabs> ### Tuning search | Parameter | Default | Description | | --------------- | ------- | -------------------------------------------------------------- | | `mode` | `fast` | `fast` (single-query) or `thinking` (multi-query reranked) | | `alpha` | 0.8 | Weight of semantic search: 0 = keyword only, 1 = semantic only | | `recency_bias` | 0.0 | Weight given to newer memories (0-1) | | `max_results` | 10 | Max results returned (capped at 50) | | `graph_context` | false | Include knowledge-graph relationships in response | ## Context assembly (RAG for SLM chat) The `/memory/context` endpoint is purpose-built for retrieval-augmented generation. Given a user query, it fetches the most relevant memories, recent events, and graph relationships, and returns a pre-formatted context string ready to prepend to your LLM prompt. <Tabs> <Tab title="JavaScript"> ```javascript theme={null} // Before calling the SLM, assemble context const ctx = await client.memory.context({ query: userMessage, session_id: chatSessionId, top_k: 8, max_context_length: 2000, include_timeline: true, include_graph: false, }); // Prepend to system message const systemMessage = `You are a helpful assistant. ## Relevant memories ${ctx.data.prompt_ready}`; // Call SLM chat with enriched context const response = await client.chat.completions.create({ model: '60db-tiny', messages: [ { role: 'system', content: systemMessage }, { role: 'user', content: userMessage }, ], }); ``` </Tab> <Tab title="Python"> ```python theme={null} # Before calling the SLM, assemble context ctx = client.memory.context( query=user_message, session_id=chat_session_id, top_k=8, max_context_length=2000, include_timeline=True, ) system_message = f"""You are a helpful assistant. ## Relevant memories {ctx['data']['prompt_ready']} """ response = client.chat.completions.create( model="60db-tiny", messages=[ {"role": "system", "content": system_message}, {"role": "user", "content": user_message}, ], ) ``` </Tab> </Tabs> ## Built-in SLM Chat integration When using 60db's UI at `/app/slm-chat`, there's a **Memory** toggle next to Auto-clear. When enabled, every message you send is pre-processed: 1. Your message is sent to `/memory/context` with your session ID 2. Relevant memories and recent events are fetched (semantic + keyword + temporal) 3. The returned `prompt_ready` string is prepended to the system message 4. The enriched prompt goes to the SLM This means your AI chat remembers context across sessions automatically. Toggle it off if you want a fresh, memoryless conversation. ## Collections management ```javascript theme={null} // List your collections const collections = await client.memory.collections.list(); // Create a team collection (admin/owner only) await client.memory.collections.create({ collection_id: "customer_support", label: "Customer Support KB", kind: "team", shared: true, }); ``` ## Role-based access Memory operations are gated by your workspace role: | Action | Owner | Admin | Developer | Member | Viewer | | ---------------------- | :---: | :---: | :-------: | :----: | :----: | | Search memories | ✓ | ✓ | ✓ | ✓ | ✓ | | Create personal memory | ✓ | ✓ | ✓ | ✓ | — | | Delete own memory | ✓ | ✓ | ✓ | ✓ | — | | Delete any memory | ✓ | ✓ | — | — | — | | Create team collection | ✓ | ✓ | — | — | — | | Create knowledge/hive | ✓ | ✓ | — | — | — | | Export memories | ✓ | ✓ | ✓ | ✓ | — | ## API key access To use Memory via an API key (for programmatic access), the key must have the `memory` scope. When creating an API key in Settings → Developers, check the "Memory & RAG" box. ```javascript theme={null} // API key with memory scope const response = await fetch('https://api.60db.com/memory/search', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query: 'user preferences', max_results: 5 }), }); ``` ## Pricing Memory is **pay-as-you-go** — no subscription, no seat pricing, no minimum commitment. You pay only for the operations you run, deducted from a single workspace wallet you top up via Stripe, Razorpay, or Dodo Payments. | Operation | Rate | | ---------------------------- | ------------------------------------------- | | Ingest a memory | **\$0.0001 per 1,000 characters** | | Upload a document (extract) | **\$0.003 per MB** | | Upload a document (ingest) | **\$0.0001 per 1,000 extracted characters** | | Search (hybrid recall) | **\$0.0003 per query** | | Context assembly (LLM-ready) | **\$0.0005 per query** | **Real-world cost examples:** * A knowledge base with 100 MB of docs + 10,000 searches/month → **\~\$23/month** * A personal assistant with 1,000 user memories + 500 searches/day → **\~\$5/month** * A support bot with 1 GB of docs + 100,000 searches/month → **\~\$53/month** <Tip> Compared to proprietary memory services at $249–$5,000/month flat, 60db Memory is 5–50x cheaper for most workloads — and you only pay for what you actually use. </Tip> Every billable request returns these response headers so you can track spend without polling: ``` x-credit-balance: 9.465200 ← wallet balance after this charge x-credit-charged: 0.000300 ← amount charged for this request x-billing-tx: 84ffd09e-... ← audit row UUID ``` **Automatic refunds** — if a request fails after being charged (upstream outage, corrupt file, etc.), the charge is reversed automatically and logged as a compensating row in `transaction_log`. No support tickets required. **Never billed** — listing collections, creating collections, checking memory status, deleting memories, and `GET /memory/usage` are **always free** so you can still manage your data when the wallet is empty. See the full [Pricing & Billing reference](/api-reference/memory/pricing) for rate details, refund policy, and the complete header/error reference. ## Handling insufficient credits When the wallet runs out, billable endpoints return **HTTP 402**: ```json theme={null} { "success": false, "message": "Insufficient credits", "error_code": "INSUFFICIENT_CREDITS", "details": { "required": 0.0003, "available": 0.00001, "shortfall": 0.00029 } } ``` Your client should catch `error_code === "INSUFFICIENT_CREDITS"` and prompt the user to top up. Here's a pattern for the search endpoint: ```javascript theme={null} async function searchWithTopUpPrompt(query) { const res = await fetch('https://api.60db.com/memory/search', { method: 'POST', headers: { 'Authorization': 'Bearer sk_your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ query, max_results: 10 }), }); if (res.status === 402) { const err = await res.json(); showTopUpModal({ shortfall: err.details.shortfall, available: err.details.available, topUpUrl: 'https://app.60db.com/app/billing', }); return null; } const data = await res.json(); console.log(`Balance: $${res.headers.get('x-credit-balance')}`); return data; } ``` ## Tracking usage Call [`GET /memory/usage`](/api-reference/memory/usage) to get a monthly spend breakdown by operation type. This is what powers the **Spend this month** card on the 60db Memory dashboard. ```javascript theme={null} const res = await fetch('https://api.60db.com/memory/usage?period=current_month', { headers: { 'Authorization': 'Bearer sk_your_api_key' }, }); const { data } = await res.json(); console.log(`Spent $${data.total.net_spend_usd.toFixed(4)} this month`); console.log(`Wallet: $${data.billing_owner.current_balance_usd.toFixed(4)}`); ``` ## Failure handling The Memory service is designed to **degrade gracefully**: * If the memory layer is temporarily unreachable, `POST /memory/ingest` queues your memory in a retry table, returns `202 Accepted`, and **automatically refunds** the charge so you aren't billed for work that didn't happen. * `POST /memory/context` returns an empty prompt on outage — your SLM chat still works, just without memory context. No charge when context is empty. * `POST /memory/search` returns `503` — the UI shows a "Memory temporarily unavailable" banner without blocking other features. Auto-refunded. * `POST /memory/documents/extract` auto-refunds the extract fee if extraction fails (corrupt file, empty PDF, OCR error). If extraction succeeds but the wallet can't cover the post-extraction ingest fee, the extract fee is refunded and a `402` is returned. ## Limits * **Ingest batch**: Up to 100 memories per request * **Memory text**: Max 100,000 characters per entry * **Query length**: Max 2,000 characters * **Results**: Max 50 per search (refine query for more precise results) * **Context length**: Max 16,000 tokens assembled per request * **Document upload**: Max 200 MB per file, max 100 chunks per document * **Rate limit**: 30 ingests/min per workspace ## Further reading * [API Reference: Pricing & Billing](/api-reference/memory/pricing) * [API Reference: Get Memory Usage](/api-reference/memory/usage) * [API Reference: Ingest memory](/api-reference/memory/ingest) * [API Reference: Upload document](/api-reference/memory/extract-document) * [API Reference: Search memories](/api-reference/memory/search) * [API Reference: Context assembly](/api-reference/memory/context) * [API Reference: Collections](/api-reference/memory/collections) # Speech-to-Text Source: https://docs.60db.ai/features/speech-to-text Transcribe audio to text with high accuracy ## Overview 60db's Speech-to-Text (STT) API converts spoken audio into written text with high accuracy across 39 languages, including code-switched Indic+English. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack) — non-hallucinating models that don't invent text on silent or noisy input. ## Features <CardGroup> <Card title="Multi-Language" icon="globe"> 39 languages with auto-detection and Indic+English code-switching </Card> <Card title="Speaker Diarization" icon="users"> Opt-in pyannote speaker diarization via `diarize: true` </Card> <Card title="Timestamps" icon="clock"> Word-level timestamps included automatically </Card> <Card title="Non-hallucinating" icon="shield-check"> Non-hallucinating backend that emits blank tokens on silence — no phantom text </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const file = document.querySelector('input[type="file"]').files[0]; const result = await client.speechToText(file, { language: 'en' }); console.log('Transcription:', result.text); console.log('Confidence:', result.confidence); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') with open('recording.mp3', 'rb') as audio_file: result = client.speech_to_text(audio_file, language='en') print(f"Transcription: {result['text']}") print(f"Confidence: {result['confidence']}") ``` </Tab> </Tabs> ## Supported Formats | Format | Max Size | Max Duration | Quality | | ------ | -------- | ------------ | --------- | | MP3 | 25MB | 10 min | Good | | WAV | 25MB | 10 min | Excellent | | FLAC | 25MB | 10 min | Lossless | | OGG | 25MB | 10 min | Good | | M4A | 25MB | 10 min | Good | ## Language Support ### Auto-Detection Let the API automatically identify the language. **Omit the `language` field** (or pass the string `"auto"`) and the server runs language identification across all 39 supported languages: ```javascript theme={null} // Simplest form — omit language entirely const result = await client.speechToText(file); // Equivalent — the REST endpoint treats "auto" as omission. // Note: on the WebSocket streaming endpoint you must use `languages: null`, // not the literal string "auto". const result2 = await client.speechToText(file, { language: 'auto' }); console.log('Detected language:', result.language); ``` ### Specify Language For lowest latency, pass a single ISO 639-1 code and the server skips language identification entirely: ```javascript theme={null} const result = await client.speechToText(file, { language: 'hi' // ISO 639-1 code }); ``` **Unsupported codes** (`ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`) and Arabic dialect tags (`ar-eg`, `ar-lv`, …) return an `unsupported_language` error. For non-MSA Arabic audio, pass `ar` for best-effort MSA transcription. ### Get Supported Languages ```javascript theme={null} const languages = await client.getLanguages(); languages.forEach(lang => { console.log(`${lang.name} (${lang.code})`); }); ``` ## Advanced Features ### Word-Level Timestamps Word timings are always included in the response — no flag needed: ```javascript theme={null} const result = await client.speechToText(file); // Word-level timing lives inside each segment result.segments.forEach(segment => { segment.words.forEach(word => { console.log(`${word.word}: ${word.start}s - ${word.end}s (${word.confidence})`); }); }); ``` ### Speaker Diarization Identify different speakers with `diarize: true`: ```javascript theme={null} const result = await client.speechToText(file, { diarize: true }); // Each segment carries a `speakers` array when diarization is enabled. // Raw labels are SPEAKER_00, SPEAKER_01, … result.segments.forEach(segment => { const speaker = segment.speakers?.[0]?.speaker ?? 'unknown'; console.log(`[${speaker}] ${segment.text}`); }); ``` Client applications typically re-label the raw `SPEAKER_NN` IDs as "Speaker 1", "Speaker 2" in order of first appearance for readability. ## Best Practices <AccordionGroup> <Accordion title="Audio Quality"> * Use high-quality recordings (16kHz+ sample rate) * Minimize background noise * Ensure clear speech * Avoid audio compression when possible </Accordion> <Accordion title="Accuracy Tips"> * Specify the language when known * Use appropriate model for your use case * Provide clean audio without music * Split very long recordings </Accordion> <Accordion title="Performance"> * Keep files under 25MB * Use appropriate format (WAV for quality, MP3 for size) * Process in batches for multiple files — but stay under the 8-concurrent STT cap per user </Accordion> </AccordionGroup> ## Use Cases ### Meeting Transcription ```python theme={null} with open('meeting.mp3', 'rb') as audio: result = client.speech_to_text( audio, diarize=True, ) # Generate meeting notes for segment in result['segments']: speaker = (segment.get('speakers') or [{}])[0].get('speaker', 'unknown') print(f"[{segment['start']:.1f}s] {speaker}: {segment['text']}") ``` ### Voice Commands ```javascript theme={null} async function processVoiceCommand(audioBlob) { const result = await client.speechToText(audioBlob, { language: 'en', }); const command = parseCommand(result.text); executeCommand(command); } ``` ### Subtitle Generation ```javascript theme={null} const result = await client.speechToText(videoAudio); const subtitles = generateSRT(result.words); saveFile('subtitles.srt', subtitles); ``` ## API Reference <Card title="Speech to Text API" icon="microphone" href="/api-reference/stt/speech-to-text"> View complete API documentation </Card> # Streaming Source: https://docs.60db.ai/features/streaming Real-time audio streaming for low-latency applications ## Overview Streaming allows you to receive audio chunks in real-time as they're generated, enabling immediate playback without waiting for the entire audio file to be created. This is essential for interactive applications like voice assistants and chatbots. ## Benefits <CardGroup> <Card title="Low Latency" icon="bolt"> Start playing audio within milliseconds </Card> <Card title="Memory Efficient" icon="memory"> Process chunks instead of loading entire file </Card> <Card title="Better UX" icon="smile"> Progressive audio playback feels more responsive </Card> <Card title="Long Content" icon="book"> Handle unlimited text length efficiently </Card> </CardGroup> ## How It Works 1. **Send Request**: Submit text to the streaming endpoint 2. **Receive Chunks**: Get audio chunks as they're generated 3. **Play Immediately**: Start playing the first chunk 4. **Continue Streaming**: Receive and play subsequent chunks 5. **Complete**: Receive completion notification ## Implementation <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Audio player setup const audioContext = new AudioContext(); const audioQueue = []; await client.textToSpeechStream( { text: 'This is a longer text that will be streamed in real-time for immediate playback.', voice_id: 'default-voice' }, { onChunk: async (chunk) => { // Convert chunk to audio buffer const audioBuffer = await audioContext.decodeAudioData(chunk.buffer); // Add to queue and play audioQueue.push(audioBuffer); if (audioQueue.length === 1) { playNextChunk(); } }, onComplete: () => { console.log('Streaming complete'); }, onError: (error) => { console.error('Streaming error:', error); } } ); function playNextChunk() { if (audioQueue.length === 0) return; const buffer = audioQueue[0]; const source = audioContext.createBufferSource(); source.buffer = buffer; source.connect(audioContext.destination); source.onended = () => { audioQueue.shift(); playNextChunk(); }; source.start(); } ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient import pyaudio client = SixtyDBClient('your-api-key') # Audio player setup p = pyaudio.PyAudio() stream = p.open( format=pyaudio.paInt16, channels=1, rate=24000, output=True ) def handle_chunk(chunk): # Play audio chunk immediately stream.write(chunk) def handle_complete(): print("Streaming complete") stream.stop_stream() stream.close() p.terminate() def handle_error(error): print(f"Error: {error}") # Stream text to speech client.text_to_speech_stream( text='This is a longer text that will be streamed in real-time.', on_chunk=handle_chunk, on_complete=handle_complete, on_error=handle_error, voice_id='default-voice' ) ``` </Tab> </Tabs> ## Advanced Usage ### React Component ```jsx theme={null} import { useState } from 'react'; import { SixtyDBClient } from '60db'; function StreamingTTS() { const [isStreaming, setIsStreaming] = useState(false); const [text, setText] = useState(''); const client = new SixtyDBClient(process.env.REACT_APP_API_KEY); const streamAudio = async () => { setIsStreaming(true); await client.textToSpeechStream( { text, voice_id: 'default-voice' }, { onChunk: (chunk) => { // Play chunk playAudioChunk(chunk); }, onComplete: () => { setIsStreaming(false); }, onError: (error) => { console.error(error); setIsStreaming(false); } } ); }; return ( <div> <textarea value={text} onChange={(e) => setText(e.target.value)} placeholder="Enter text to speak..." /> <button onClick={streamAudio} disabled={isStreaming}> {isStreaming ? 'Streaming...' : 'Speak'} </button> </div> ); } ``` ### Voice Assistant ```javascript theme={null} class VoiceAssistant { constructor(apiKey) { this.client = new SixtyDBClient(apiKey); this.audioContext = new AudioContext(); } async speak(text) { return new Promise((resolve, reject) => { this.client.textToSpeechStream( { text, voice_id: 'assistant-voice' }, { onChunk: (chunk) => this.playChunk(chunk), onComplete: () => resolve(), onError: (error) => reject(error) } ); }); } async playChunk(chunk) { const audioBuffer = await this.audioContext.decodeAudioData(chunk.buffer); const source = this.audioContext.createBufferSource(); source.buffer = audioBuffer; source.connect(this.audioContext.destination); source.start(); } } // Usage const assistant = new VoiceAssistant('your-api-key'); await assistant.speak('Hello! How can I help you today?'); ``` ## Performance Optimization ### Buffering Strategy ```javascript theme={null} class AudioStreamer { constructor() { this.chunks = []; this.isPlaying = false; this.minBufferSize = 3; // Wait for 3 chunks before playing } addChunk(chunk) { this.chunks.push(chunk); if (!this.isPlaying && this.chunks.length >= this.minBufferSize) { this.startPlayback(); } } async startPlayback() { this.isPlaying = true; while (this.chunks.length > 0) { const chunk = this.chunks.shift(); await this.playChunk(chunk); } this.isPlaying = false; } async playChunk(chunk) { // Play audio chunk return new Promise((resolve) => { // Audio playback logic setTimeout(resolve, chunkDuration); }); } } ``` ## Best Practices <AccordionGroup> <Accordion title="Buffer Management"> * Maintain a small buffer (2-3 chunks) for smooth playback * Handle network interruptions gracefully * Implement retry logic for failed chunks </Accordion> <Accordion title="Error Handling"> * Always implement onError callback * Provide user feedback during streaming * Have fallback for streaming failures </Accordion> <Accordion title="Performance"> * Reuse AudioContext instances * Clean up resources after playback * Monitor memory usage for long streams </Accordion> <Accordion title="User Experience"> * Show loading indicator before first chunk * Allow users to stop streaming * Provide playback controls </Accordion> </AccordionGroup> ## Use Cases ### Real-time Chat ```javascript theme={null} async function sendMessage(message) { // Display user message displayMessage('user', message); // Get AI response const response = await getAIResponse(message); displayMessage('assistant', response); // Stream audio response await client.textToSpeechStream( { text: response }, { onChunk: (chunk) => playAudioChunk(chunk), onComplete: () => enableInput() } ); } ``` ### Audiobook Player ```javascript theme={null} async function playChapter(chapterText) { let currentPosition = 0; await client.textToSpeechStream( { text: chapterText }, { onChunk: (chunk) => { playChunk(chunk); currentPosition += chunk.duration; updateProgress(currentPosition); }, onComplete: () => { moveToNextChapter(); } } ); } ``` ## API Reference <Card title="Streaming API" icon="signal-stream" href="/api-reference/tts/text-to-speech-stream"> View complete streaming API documentation </Card> # Text-to-Speech Source: https://docs.60db.ai/features/text-to-speech Convert text to natural-sounding speech ## Overview 60db's Text-to-Speech (TTS) API converts written text into natural-sounding speech using advanced AI models. Our TTS engine supports multiple voices, languages, and customization options. ## Features <CardGroup> <Card title="Multiple Voices" icon="users"> Choose from 50+ pre-built voices or create custom voices </Card> <Card title="Voice Customization" icon="sliders"> Adjust speed, stability, and similarity </Card> <Card title="High Quality" icon="star"> Crystal-clear audio with natural intonation </Card> <Card title="Multiple Formats" icon="file-audio"> Support for MP3, WAV, OGG, and FLAC output formats </Card> </CardGroup> ## Basic Usage <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice', enhance: true, speed: 1.0 }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') audio = client.text_to_speech( text='Hello, world!', voice_id='default-voice', enhance=True, speed=1.0 ) with open('output.mp3', 'wb') as f: f.write(audio) ``` </Tab> </Tabs> ## Voice Parameters ### Speed Control the speaking rate of the generated audio: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'This will be spoken faster', speed: 1.5 // Range: 0.5 to 2.0 }); ``` * `0.5`: Half speed (slow) * `1.0`: Normal speed (default) * `2.0`: Double speed (fast) ### Stability Control how expressive versus consistent the generated voice sounds: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'More consistent delivery', stability: 50 // Range: 0 to 100 (default 50) }); ``` * Lower values produce more expressive, varied output * Higher values produce more consistent, stable output ### Similarity Control how closely the output matches the source voice: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Close clone match', similarity: 75 // Range: 0 to 100 (default 75) }); ``` ### Enhancement Enable audio enhancement for better quality: ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Enhanced audio quality', enhance: true // Default: true }); ``` ## Output Formats Supported audio formats: | Format | Quality | File Size | Use Case | | ------ | --------- | --------- | --------------------- | | MP3 | Good | Small | Web, mobile apps | | WAV | Excellent | Large | Professional audio | | OGG | Good | Small | Web streaming | | FLAC | Lossless | Medium | High-quality archival | ```javascript theme={null} const audio = await client.textToSpeech({ text: 'Hello, world!', output_format: 'wav' // mp3, wav, ogg, flac }); ``` ## Best Practices <AccordionGroup> <Accordion title="Text Formatting"> * Use proper punctuation for natural pauses * Break long texts into paragraphs * Use SSML tags for advanced control (coming soon) </Accordion> <Accordion title="Voice Selection"> * Test multiple voices for your use case * Consider accent and gender for your audience * Use custom voices for brand consistency </Accordion> <Accordion title="Performance"> * Cache frequently used audio * Batch requests when possible * Use appropriate audio format for your use case </Accordion> <Accordion title="Quality"> * Enable enhancement for production use * Use WAV format for highest quality * Test with different speed settings </Accordion> </AccordionGroup> ## Use Cases ### Voice Assistants ```javascript theme={null} // Voice assistant async function speakResponse(text) { const audio = await client.textToSpeech({ text: text, voice_id: 'assistant-voice', enhance: true }); playAudio(audio); } ``` ### Content Narration ```javascript theme={null} // Generate audiobook chapter const audio = await client.textToSpeech({ text: chapterText, voice_id: 'narrator-voice', speed: 0.95, output_format: 'mp3' }); saveToFile(`chapter-${chapterNum}.mp3`, audio); ``` ### Accessibility ```javascript theme={null} // Make web content accessible async function readAloud(element) { const text = element.textContent; const audio = await client.textToSpeech({ text, voice_id: 'clear-voice', enhance: true }); playAudio(audio); } ``` ## API Reference For detailed API documentation, see: <Card title="Text to Speech" icon="microphone" href="/api-reference/tts/text-to-speech"> Standard TTS endpoint </Card> # Custom Voices Source: https://docs.60db.ai/features/voices Create and manage custom voice profiles ## Overview Create custom voice profiles that match your brand identity or clone specific voices for personalized experiences. Our voice cloning technology requires just a few minutes of audio to create a high-quality custom voice. ## Creating a Custom Voice ### Requirements <Card title="Audio Samples" icon="microphone"> * **Minimum**: 3 audio files * **Maximum**: 10 audio files * **Duration**: 10-60 seconds per file * **Total**: At least 2 minutes combined * **Format**: MP3, WAV, or FLAC * **Quality**: 44.1kHz+ sample rate recommended </Card> ### Step-by-Step Guide <Steps> <Step title="Prepare Audio Samples"> Record or collect 3-10 high-quality audio samples of the voice you want to clone </Step> <Step title="Upload Samples"> Use the API or dashboard to upload your audio files </Step> <Step title="Wait for Processing"> Voice cloning typically takes 10-15 minutes </Step> <Step title="Test Your Voice"> Generate test audio to verify quality </Step> <Step title="Use in Production"> Start using your custom voice in your applications </Step> </Steps> ## Code Examples <Tabs> <Tab title="JavaScript"> ```javascript theme={null} import { SixtyDBClient } from '60db'; const client = new SixtyDBClient('your-api-key'); // Create custom voice const files = [ document.querySelector('#file1').files[0], document.querySelector('#file2').files[0], document.querySelector('#file3').files[0] ]; const voice = await client.createVoice({ name: 'My Brand Voice', description: 'Professional voice for customer service', language: 'en', gender: 'female', files: files }); console.log('Voice ID:', voice.id); console.log('Status:', voice.status); // Use the custom voice const audio = await client.textToSpeech({ text: 'Hello from my custom voice!', voice_id: voice.id }); ``` </Tab> <Tab title="Python"> ```python theme={null} from sixtydb import SixtyDBClient client = SixtyDBClient('your-api-key') # Create custom voice files = [ open('sample1.mp3', 'rb'), open('sample2.mp3', 'rb'), open('sample3.mp3', 'rb') ] voice = client.create_voice( name='My Brand Voice', files=files, description='Professional voice for customer service', language='en', gender='female' ) print(f"Voice ID: {voice['id']}") print(f"Status: {voice['status']}") # Close files for f in files: f.close() # Use the custom voice audio = client.text_to_speech( text='Hello from my custom voice!', voice_id=voice['id'] ) with open('output.mp3', 'wb') as f: f.write(audio) ``` </Tab> </Tabs> ## Audio Sample Guidelines ### Content Recommendations <AccordionGroup> <Accordion title="Variety"> * Include different sentence types (questions, statements, exclamations) * Cover various emotions and tones * Use different speaking speeds * Include both short and long sentences </Accordion> <Accordion title="Quality"> * Record in a quiet environment * Use a good quality microphone * Maintain consistent volume * Avoid background music or noise * No echo or reverb </Accordion> <Accordion title="Technical"> * Sample rate: 44.1kHz or higher * Bit depth: 16-bit or higher * Format: WAV (lossless) preferred * Mono or stereo both acceptable </Accordion> <Accordion title="Content"> * Natural, conversational speech * Clear pronunciation * Consistent accent * Avoid reading in monotone * Include natural pauses </Accordion> </AccordionGroup> ## Managing Custom Voices ### List Your Voices ```javascript theme={null} const voices = await client.getVoices(); // Filter custom voices const customVoices = voices.filter(v => v.is_custom); customVoices.forEach(voice => { console.log(`${voice.name} (${voice.id})`); }); ``` ### Update Voice Metadata ```javascript theme={null} await client.updateVoice('voice-id', { name: 'Updated Voice Name', description: 'Updated description' }); ``` ### Delete a Voice ```javascript theme={null} await client.deleteVoice('voice-id'); ``` ## Voice Quality Tips <CardGroup> <Card title="Recording Environment" icon="location-dot"> Record in a quiet room with minimal echo and background noise </Card> <Card title="Microphone Quality" icon="microphone"> Use a quality microphone for best results </Card> <Card title="Speaking Style" icon="comment"> Speak naturally and expressively </Card> <Card title="Audio Length" icon="clock"> Provide at least 2 minutes of total audio </Card> </CardGroup> ## Use Cases ### Brand Voice Create a consistent voice for all your brand communications: ```javascript theme={null} const brandVoice = await client.createVoice({ name: 'Acme Brand Voice', description: 'Official voice for Acme Corporation', files: brandAudioSamples }); // Use in all customer touchpoints const greeting = await client.textToSpeech({ text: 'Welcome to Acme Corporation. How can we help you today?', voice_id: brandVoice.id }); ``` ### Personal Assistant Clone your own voice for a personalized assistant: ```javascript theme={null} const myVoice = await client.createVoice({ name: 'My Personal Voice', description: 'My voice for personal assistant', files: myRecordings }); // Personal reminders in your own voice const reminder = await client.textToSpeech({ text: 'Remember to call mom at 3 PM', voice_id: myVoice.id }); ``` ### Character Voices Create unique voices for game characters or audiobooks: ```javascript theme={null} const characterVoice = await client.createVoice({ name: 'Wizard Character', description: 'Mystical wizard voice for fantasy game', files: characterSamples }); ``` ## Pricing Custom voice creation is available on Pro and Enterprise plans: | Plan | Custom Voices | Processing Time | | ---------- | ------------- | ------------------- | | Free | 0 | - | | Starter | 0 | - | | Pro | 5 | 10-15 min | | Enterprise | Unlimited | Priority (5-10 min) | ## API Reference <CardGroup> <Card title="Create Voice" icon="plus" href="/api-reference/voices/create-voice"> Create a new custom voice </Card> <Card title="Manage Voices" icon="gear" href="/api-reference/voices/get-voices"> List, update, and delete voices </Card> </CardGroup> # LiveKit Integration Source: https://docs.60db.ai/integrations/livekit Use 60db STT, TTS, and LLM inside a LiveKit Agents voice pipeline. # LiveKit Integration Build real-time voice agents with [LiveKit Agents](https://github.com/livekit/agents) powered by 60db's speech and language services — STT, TTS, and LLM all in one plugin. <CardGroup> <Card title="Speech-to-Text" icon="microphone"> Real-time streaming transcription via WebSocket with interim results </Card> <Card title="Text-to-Speech" icon="volume-high"> Low-latency streaming synthesis with chunked audio delivery </Card> <Card title="LLM Chat" icon="brain"> OpenAI-compatible chat completions with tool-call support </Card> </CardGroup> *** ## Installation <Steps> <Step title="Install the plugin"> Requires Python **3.10+**. ```bash theme={null} pip install livekit-plugins-60db ``` </Step> <Step title="Set your API key"> Choose one of the following methods: **Option A — Environment variable (recommended):** ```bash theme={null} export SIXTY_DB_API_KEY="your-api-key" ``` **Option B — `.env.local` file:** ``` SIXTY_DB_API_KEY=your-api-key ``` **Option C — Pass it directly in code:** ```python theme={null} from livekit.plugins._60db import _60dbClient client = _60dbClient("your-api-key") # sets global default ``` </Step> </Steps> *** ## Quick Start Wire all three services together in a `VoicePipelineAgent`: ```python theme={null} import asyncio from livekit.agents import VoicePipelineAgent, cli, WorkerType from livekit.plugins import silero from livekit.plugins._60db import _60dbClient, STT, TTS, LLM client = _60dbClient("your-api-key") # sets global default async def entrypoint(ctx): await ctx.connect() agent = VoicePipelineAgent( vad=silero.VAD.load(), # voice activity detection stt=STT(), # 60db speech-to-text llm=LLM(), # 60db language model tts=TTS(), # 60db text-to-speech ) agent.start(ctx.room) if __name__ == "__main__": cli.run_app(worker_type=WorkerType.ROOM, entrypoint_fnc=entrypoint) ``` *** ## Configuration ### Environment Variables All services read from the same environment variables by default. You only need to override them if you use a custom deployment. | Variable | Default | Description | | ------------------ | ----------------------------------------- | ---------------------------- | | `SIXTY_DB_API_KEY` | — | Your 60db API key (required) | | `SIXTY_DB_STT_URL` | `wss://api.60db.ai/ws/stt` | STT WebSocket endpoint | | `SIXTY_DB_TTS_URL` | `wss://api.60db.ai/ws/tts` | TTS WebSocket endpoint | | `SIXTY_DB_LLM_URL` | `https://api.60db.ai/v1/chat/completions` | LLM HTTP endpoint | Each service also accepts a direct `ws_url` (or `api_url`) constructor argument which takes precedence over environment variables. *** ## Services <Tabs> <Tab title="STT"> ### Speech-to-Text The STT service streams audio to 60db over WebSocket and returns transcriptions in real time — including interim (partial) results as the speaker is still talking. #### Parameters | Name | Type | Default | Description | | ----------------- | ------------------- | ----------------------- | ------------------------------------------------ | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `ws_url` | `str \| None` | env var | WebSocket endpoint URL | | `languages` | `list[str] \| None` | `["en"]` | Language codes for recognition | | `encoding` | `str` | `"mulaw"` | Audio encoding format | | `sample_rate` | `int` | `8000` | Audio sample rate in Hz | | `continuous_mode` | `bool` | `True` | Keep the session open for continuous recognition | #### Example ```python theme={null} from livekit.plugins._60db import STT stt = STT() # picks up api_key from global client or env var async with stt.stream() as stream: stream.push_frame(audio_frame) # push raw audio frames async for event in stream: if event.type == stt.SpeechEventType.FINAL_TRANSCRIPT: print(event.alternatives[0].text) ``` #### Audio Formats The plugin automatically handles audio conversion for you — no manual preprocessing needed. | Step | What happens | | ------------- | --------------------------------------------------------------------- | | Stereo → Mono | Downmix via `audioop.tomono` | | Resampling | Any input rate → target rate via `audioop.ratecv` | | Encoding | LINEAR16 PCM → mulaw via `audioop.lin2ulaw` (when `encoding="mulaw"`) | | Parameter | Default | Supported | | ----------- | --------- | ------------------------- | | Encoding | `mulaw` | `mulaw`, `LINEAR16` | | Sample rate | `8000 Hz` | Any rate (auto-resampled) | </Tab> <Tab title="TTS"> ### Text-to-Speech The TTS service converts text to speech over WebSocket. It supports both **one-shot** synthesis and **streaming** mode for low-latency incremental output. #### Parameters | Name | Type | Default | Description | | ------------- | ------------- | ---------------------------------------- | ------------------------ | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `ws_url` | `str \| None` | env var | WebSocket endpoint URL | | `voice_id` | `str` | `"fbb75ed2-975a-40c7-9e06-38e30524a9a1"` | Voice identifier | | `encoding` | `str` | `"LINEAR16"` | Output audio encoding | | `sample_rate` | `int` | `16000` | Output sample rate in Hz | #### Examples **One-shot synthesis** — best for short, complete responses: ```python theme={null} from livekit.plugins._60db import TTS tts = TTS() async for chunk in tts.synthesize("Hello, how can I help you?"): # chunk.data contains raw PCM audio bytes print(f"Received {len(chunk.data)} bytes") ``` **Streaming synthesis** — best for incremental LLM output: ```python theme={null} async with tts.stream() as stream: stream.push_text("Hello, ") stream.push_text("how can I help you?") stream.end_input() async for chunk in stream: print(f"Received {len(chunk.data)} bytes") ``` #### Audio Format | Parameter | Default | Notes | | ----------- | ---------- | ------------------------------ | | Encoding | `LINEAR16` | Raw PCM | | Sample rate | `16000 Hz` | Configurable via `sample_rate` | Audio is returned as base64-encoded PCM chunks by the server, decoded automatically by the plugin. </Tab> <Tab title="LLM"> ### Language Model The LLM service provides OpenAI-compatible chat completions with SSE streaming and tool-call support. #### Parameters | Name | Type | Default | Description | | ---------------------- | --------------- | ----------------------- | ----------------------------------- | | `api_key` | `str \| None` | global client / env var | Your 60db API key | | `api_url` | `str \| None` | env var | Chat completions endpoint URL | | `model` | `str` | `"qcall/slm-3b-int4"` | Model identifier | | `temperature` | `float \| None` | `None` | Sampling temperature (0–2) | | `top_p` | `float \| None` | `None` | Nucleus sampling probability | | `top_k` | `int \| None` | `None` | Top-K sampling parameter | | `max_tokens` | `int \| None` | `None` | Maximum tokens in the response | | `chat_template_kwargs` | `dict \| None` | `None` | Extra template kwargs for the model | #### Example ```python theme={null} from livekit.plugins._60db import LLM from livekit.agents import llm model = LLM(temperature=0.7) chat_ctx = llm.ChatContext() chat_ctx.append(role="system", text="You are a helpful voice assistant.") chat_ctx.append(role="user", text="What is the capital of France?") async for chunk in model.chat(chat_ctx=chat_ctx): print(chunk.choices[0].delta.content, end="") ``` #### Tool Calls The LLM supports OpenAI-style function tool calls. When the model calls a tool, the plugin accumulates the streamed argument fragments and emits a complete `FunctionToolCall` once the stream finishes. </Tab> </Tabs> *** ## Timeouts Control timeouts per-request by passing `APIConnectOptions`: ```python theme={null} from livekit.agents import APIConnectOptions opts = APIConnectOptions(timeout=30.0) # STT async with stt.stream(conn_options=opts) as s: ... # TTS async for chunk in tts.synthesize("Hello", conn_options=opts): ... # LLM async for chunk in model.chat(chat_ctx=ctx, conn_options=opts): ... ``` *** ## Error Handling All three services raise standard LiveKit Agents exceptions: | Exception | When it's raised | | -------------------- | ----------------------------------------------------------------------- | | `APIConnectionError` | WebSocket handshake failure, HTTP error, or unexpected protocol message | | `APITimeoutError` | Request exceeds the configured timeout | | `ValueError` | Missing API key or service URL at construction time | ### Common HTTP Error Codes (LLM) | Status | Meaning | | ------ | -------------------------- | | `401` | Invalid or missing API key | | `429` | Rate limit exceeded | | `500` | Server error | | `503` | Service unavailable | ### Retry Tips * **STT**: If you receive an `error` message during the handshake, reconnect with a short delay. A `connecting` status before `connection_established` is normal. * **TTS**: Retry connection failures with exponential backoff. * **LLM**: `httpx.TimeoutException` → `APITimeoutError`; `httpx.HTTPStatusError` → `APIConnectionError`. *** ## WebSocket Protocol Reference <Accordion title="STT WebSocket Protocol"> **Connect:** ``` wss://api.60db.ai/ws/stt?apiKey={API_KEY} ``` **Handshake:** Server → `{"connection_established": true}` Client → start command: ```json theme={null} { "type": "start", "languages": ["en"], "config": { "encoding": "mulaw", "sample_rate": 8000, "continuous_mode": true } } ``` Server → `{"type": "connected"}` **Audio:** Send raw audio as **binary WebSocket frames**. **Transcription response:** ```json theme={null} { "type": "transcription", "text": "Hello, world", "is_final": false, "language": "en" } ``` **Stop:** Client → `{"type": "stop"}` Server → ```json theme={null} { "type": "session_stopped", "billing_summary": { "total_cost": "..." } } ``` </Accordion> <Accordion title="TTS WebSocket Protocol"> **Connect:** ``` wss://api.60db.ai/ws/tts?apiKey={API_KEY} ``` **Create context:** ```json theme={null} { "create_context": { "context_id": "unique-id", "voice_id": "fbb75ed2-975a-40c7-9e06-38e30524a9a1", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 } } } ``` **Send text:** ```json theme={null} { "send_text": { "context_id": "unique-id", "text": "Hello!" } } ``` **Flush** (trigger audio generation): ```json theme={null} { "flush_context": { "context_id": "unique-id" } } ``` **Audio response:** ```json theme={null} { "audio_chunk": { "audioContent": "<base64 PCM>" } } ``` Followed by `{"flush_completed": true}` when all audio has been delivered. **Close context:** ```json theme={null} { "close_context": { "context_id": "unique-id" } } ``` </Accordion> # Introduction Source: https://docs.60db.ai/introduction Welcome to 60db - The Modern Voice AI Platform ## What is 60db? 60db is a modern AI platform that combines state-of-the-art voice capabilities (TTS, STT, custom voices) with a hosted **LLM chat** and a **Memory / RAG layer** for building assistants that remember context and retrieve knowledge on demand. Developers can integrate natural-sounding voice, accurate speech recognition, and persistent memory into their applications with just a few lines of code. ## Key Features <CardGroup> <Card title="Text-to-Speech" icon="microphone" href="/features/text-to-speech"> Convert text to natural-sounding speech with multiple voice options </Card> <Card title="Speech-to-Text" icon="waveform" href="/features/speech-to-text"> Transcribe audio to text with high accuracy across multiple languages </Card> <Card title="Custom Voices" icon="user-voice" href="/features/voices"> Create and manage custom voice profiles for your brand </Card> <Card title="Real-time Streaming" icon="signal-stream" href="/features/streaming"> Stream audio in real-time for low-latency applications </Card> <Card title="LLM Chat" icon="comments" href="/features/llm-chat"> OpenAI-compatible chat completions with 60db's hosted SLM models </Card> <Card title="Memory & RAG" icon="brain" href="/features/memory"> Persistent memory, document upload (PDF/DOCX/OCR), and semantic recall for AI chat </Card> </CardGroup> ## Why Choose 60db? <AccordionGroup> <Accordion title="High Quality Audio"> Our TTS engine produces natural-sounding speech with proper intonation, emotion, and clarity that rivals human speech. </Accordion> <Accordion title="Multiple Languages"> Support for 50+ languages and dialects, making your application globally accessible. </Accordion> <Accordion title="Easy Integration"> Simple SDKs for JavaScript/TypeScript and Python make integration straightforward and quick. </Accordion> <Accordion title="Scalable Infrastructure"> Built on robust infrastructure that scales automatically to handle your growing needs. </Accordion> <Accordion title="Real-time Processing"> Low-latency streaming capabilities for real-time voice applications. </Accordion> </AccordionGroup> ## Use Cases * **Voice Assistants**: Build intelligent voice-enabled applications * **Content Creation**: Generate voiceovers for videos and podcasts * **Accessibility**: Make your content accessible to visually impaired users * **Customer Service**: Create automated voice response systems * **E-learning**: Develop interactive educational content with voice * **Gaming**: Add realistic character voices to your games ## Getting Started Ready to get started? Check out our [Quickstart Guide](/quickstart) to integrate 60db into your application in minutes. <CardGroup> <Card title="Quickstart" icon="rocket" href="/quickstart"> Get up and running in under 5 minutes </Card> <Card title="API Reference" icon="code" href="/api-reference/introduction"> Explore our comprehensive API documentation </Card> </CardGroup> # Claude desktop Source: https://docs.60db.ai/mcp-server/claude-desktop # Claude Desktop Integration Integrate the 60db MCP Server with Claude Desktop. ## Configuration ### Step 1: Build the Server ```bash theme={null} cd /path/to/mcp-server npm install npm run build ``` ### Step 2: Locate Claude Desktop Config **macOS**: ``` ~/Library/Application Support/Claude/claude_desktop_config.json ``` **Windows**: ``` %APPDATA%/Claude/claude_desktop_config.json ``` ### Step 3: Add Configuration ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/absolute/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` <Callout type="warning"> Use absolute paths. Relative paths may not work. </Callout> ### Step 4: Restart Claude Desktop Quit and restart Claude Desktop. ## Usage Examples ### Generate Speech ``` You: Convert "Hello, world!" to speech ``` ### Transcribe Audio ``` You: Transcribe this audio: https://example.com/meeting.mp3 ``` ### List Voices ``` You: What English voices are available? ``` ### Create Voice ``` You: Create a voice named "Brand Voice" from this sample ``` ## Troubleshooting ### Server Not Appearing 1. Check the config file path 2. Verify the server path is absolute 3. Check the API key is set 4. Restart Claude Desktop ### Authentication Error ```bash theme={null} export SIXTYDB_API_KEY=sk_your_api_key_here ``` ## Next Steps * [MCP Inspector](/mcp-server/mcp-inspector) - Test the server # Configuration Source: https://docs.60db.ai/mcp-server/configuration # Configuration Configure the 60db MCP Server for your environment. ## Environment Variables ### Required ```bash theme={null} # 60db API Key (required) export SIXTYDB_API_KEY=sk_your_api_key_here ``` ### Optional ```bash theme={null} # API Base URL (default: https://api.60db.ai) export SIXTYDB_API_BASE_URL=https://api.60db.ai # Debug mode (default: false) export SIXTYDB_DEBUG=true ``` ## Claude Desktop Configuration Add to your Claude Desktop config: **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` **Windows**: `%APPDATA%/Claude/claude_desktop_config.json` ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/absolute/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` <Callout type="warning"> Use absolute paths for the args array. </Callout> ## Custom Application Integration ```typescript theme={null} import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; const transport = new StdioClientTransport({ command: "node", args: ["/path/to/mcp-server/dist/index.js"], env: { SIXTYDB_API_KEY: "sk_your_api_key_here", SIXTYDB_API_BASE_URL: "https://api.60db.ai" } }); const client = new Client({ name: "my-app", version: "1.0.0" }, { capabilities: {} }); await client.connect(transport); ``` ## Testing Test with MCP Inspector: ```bash theme={null} npm install -g @modelcontextprotocol/inspector mcp-inspector node dist/index.js ``` ## Next Steps * [Quick Start](/mcp-server/quickstart) - Test your setup # Installation Source: https://docs.60db.ai/mcp-server/installation # Installation Install and set up the 60db MCP Server. **Source code:** [github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) · **npm:** [`60db-mcp-server`](https://www.npmjs.com/package/60db-mcp-server) ## Prerequisites * **Node.js 18+** - [Download](https://nodejs.org/) * **60db API Key** - Get from [https://app.60db.ai](https://app.60db.ai) ## Install from npm ```bash theme={null} npm install -g 60db-mcp-server ``` Verify the binary is on your PATH: ```bash theme={null} 60db-mcp-server --help ``` ## Install from Source ```bash theme={null} # Clone the public repository git clone https://github.com/60db-ai/mcp.git cd mcp # Install dependencies npm install # Build the server (TypeScript → dist/) npm run build # Run it npm start ``` ## Verify Installation ```bash theme={null} # Check if server is installed 60db-mcp-server --version # Or if installed from source npm start ``` ## Environment Setup Set your API key: ```bash theme={null} # Set 60db API key export SIXTYDB_API_KEY=sk_your_api_key_here # Set API base URL (optional, defaults to https://api.60db.ai) export SIXTYDB_API_BASE_URL=https://api.60db.ai ``` Alternatively, authenticate with a JWT: ```bash theme={null} export SIXTYDB_JWT_TOKEN=your_jwt_token_here ``` <Note> Legacy `QLABS_API_KEY`, `QLABS_JWT_TOKEN`, and `QLABS_API_BASE_URL` env vars are still honored for backward compatibility — existing configs will continue to work after the rename, but new deployments should use the `SIXTYDB_*` names. </Note> ### Making it Permanent Add to your `~/.bashrc`, `~/.zshrc`, or `~/.profile`: ```bash theme={null} echo 'export SIXTYDB_API_KEY=sk_your_api_key_here' >> ~/.bashrc source ~/.bashrc ``` ## Development Setup For development with auto-reload: ```bash theme={null} npm run dev ``` ## Production Build ```bash theme={null} npm run build npm start ``` ## Next Steps * [Configuration](/mcp-server/configuration) - Complete the setup * [Claude Desktop](/mcp-server/claude-desktop) - Integrate with Claude # Introduction Source: https://docs.60db.ai/mcp-server/introduction # MCP Server The 60db MCP (Model Context Protocol) Server enables AI assistants like Claude to directly interact with the full 60db platform — voice AI, persistent memory, and access-control checks. **Source code:** [github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) · **npm:** [`60db-mcp-server`](https://www.npmjs.com/package/60db-mcp-server) · **License:** MIT ## What is the MCP Server? The MCP Server provides a standardized interface for AI assistants to: * **Generate speech** from text using any voice * **Transcribe audio** files with speaker diarization * **Create custom voices** from audio samples * **Manage workspaces** and team collaboration * **Store and recall memories** — persistent memory and RAG across sessions * **Upload documents** — PDF, DOCX, XLSX, PPTX, EML, scanned images with built-in OCR * **Assemble LLM-ready context** — purpose-built for retrieval-augmented generation * **Check permissions** — query Cerbos policy before attempting gated actions * **Track usage** and analytics * **View monthly spend** across every billable operation ## Quick Start ### 1. Install the Server ```bash theme={null} # From npm npm install -g 60db-mcp-server # Or clone from source git clone https://github.com/60db-ai/mcp.git cd mcp npm install npm run build ``` ### 2. Configure Claude Desktop Add to your Claude Desktop config: **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` **Windows**: `%APPDATA%/Claude/claude_desktop_config.json` ```json theme={null} { "mcpServers": { "60db": { "command": "node", "args": ["/path/to/mcp-server/dist/index.js"], "env": { "SIXTYDB_API_KEY": "sk_your_api_key_here", "SIXTYDB_API_BASE_URL": "https://api.60db.ai" } } } } ``` ### 3. Use with Claude Restart Claude Desktop and start using 60db voice AI: ``` You: Can you convert "Hello, world!" to speech using a female English voice? Claude: I'll use the 60db TTS tool to generate speech for you. ``` ## Available Tools | Category | Tools | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **Voices** | List voices, get voice details, create custom voices | | **TTS** | Synthesize speech, view generation history | | **STT** | Transcribe audio, view transcription history | | **Workspaces** | Manage team workspaces and members | | **Memory & RAG** | Ingest memories, upload documents (91+ formats with OCR), hybrid semantic search, context assembly, manage collections, track spend (10 tools total) | | **Authz** | Fetch effective permissions map, probe single `(resource, action)` checks — useful before attempting gated actions | | **Analytics** | Track usage statistics | | **Billing** | View plans, subscriptions, and invoices | ## Features ### Text-to-Speech Generate natural speech in 40+ languages: ```typescript theme={null} // Synthesize speech { "text": "Hello, world!", "voice_id": "voice_abc123", "speed": 1.0, "output_format": "mp3" } ``` ### Speech-to-Text Transcribe audio with speaker identification: ```typescript theme={null} // Transcribe audio { "audio_url": "https://example.com/meeting.mp3", "language": "en-US", "diarization": true, "timestamps": true } ``` ### Voice Cloning Create custom voices from audio samples: ```typescript theme={null} // Create custom voice { "name": "Brand Voice", "sample_url": "https://example.com/sample.mp3", "language": "en-US" } ``` ### Memory & RAG Give your AI assistant a persistent, searchable long-term memory: ```typescript theme={null} // Store a user preference { "tool": "sixtydb_memory_ingest", "arguments": { "text": "User prefers dark mode and metric units", "type": "user" } } // Upload a document (PDF, DOCX, scanned image, etc.) { "tool": "sixtydb_memory_upload_document", "arguments": { "file_path": "/Users/me/Documents/handbook.pdf", "collection": "company_handbook", "type": "knowledge" } } // Search with hybrid semantic + keyword recall { "tool": "sixtydb_memory_search", "arguments": { "query": "What is the refund policy?", "alpha": 0.8, "mode": "fast" } } // Assemble LLM-ready context for RAG { "tool": "sixtydb_memory_context", "arguments": { "query": "Tell me about my customer's recent orders" } } ``` All memory operations are [pay-as-you-go](/api-reference/memory/pricing) — the tool response surfaces `charged`, `balance`, and `tx` details in the billing footer on every call. ### Authorization checks Before attempting a gated action, query the Cerbos permission map so the agent can fail fast on disallowed operations: ```typescript theme={null} // Fetch the full permission map (once per session) { "tool": "sixtydb_get_permissions", "arguments": {} } // Probe a specific action { "tool": "sixtydb_check_permission", "arguments": { "resource": "memory", "action": "create" } } ``` This lets the agent hide or disable features that the user's role can't use, instead of watching for `403 POLICY_DENY` responses mid-flow. ## Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ Claude (AI Assistant) │ └─────────────────────────┬───────────────────────────────────┘ │ │ MCP Protocol (stdio) │ ┌─────────────────────────▼───────────────────────────────────┐ │ 60db MCP Server │ ├─────────────────────────────────────────────────────────────┤ │ Voice: sixtydb_tts_synthesize, sixtydb_stt_transcribe │ │ sixtydb_list_voices, sixtydb_create_voice │ │ Memory: sixtydb_memory_ingest, sixtydb_memory_search │ │ sixtydb_memory_upload_document │ │ sixtydb_memory_context, sixtydb_memory_get_usage │ │ Authz: sixtydb_get_permissions, sixtydb_check_permission│ │ Admin: sixtydb_list_workspaces, sixtydb_get_usage_stats │ │ sixtydb_list_invoices │ │ ... 30+ tools total │ └─────────────────────────┬───────────────────────────────────┘ │ │ HTTPS (JWT or API key) │ ┌─────────────────────────▼───────────────────────────────────┐ │ 60db API │ │ (api.60db.ai) │ │ │ │ Every request passes through: │ │ Cerbos policy check → Memory billing → Controller │ └─────────────────────────────────────────────────────────────┘ ``` ## Next Steps * [Installation](/mcp-server/installation) - Install and configure the server * [Configuration](/mcp-server/configuration) - Set up authentication * [Quick Start](/mcp-server/quickstart) - Run your first request * [Claude Desktop](/mcp-server/claude-desktop) - Integration guide ## Requirements * **Node.js**: 18 or higher * **API Key**: 60db API key from dashboard * **MCP Client**: Claude Desktop or custom MCP client ## Support * **Documentation**: [https://docs.60db.ai](https://docs.60db.ai) * **GitHub**: [https://github.com/60db-ai/mcp](https://github.com/60db-ai/mcp) * **Issues**: [https://github.com/60db-ai/mcp/issues](https://github.com/60db-ai/mcp/issues) # Mcp inspector Source: https://docs.60db.ai/mcp-server/mcp-inspector # MCP Inspector Test and debug the 60db MCP Server with the MCP Inspector. ## Installation ```bash theme={null} npm install -g @modelcontextprotocol/inspector ``` ## Usage ### Start the Inspector ```bash theme={null} # From the server directory mcp-inspector node dist/index.js # With environment variables SIXTYDB_API_KEY=sk_your_key mcp-inspector node dist/index.js ``` The Inspector opens in your browser at `http://localhost:5173`. ## Features * **View all tools**: Browse 20+ available tools * **Execute tools**: Test tools with parameters * **View responses**: See formatted responses * **Debug logging**: View server logs in real-time ## Example Workflows ### List Voices 1. Click `60db_list_voices` 2. Enter: `{"language": "en-US", "limit": 5}` 3. Click "Execute" 4. View available voices ### Synthesize Speech 1. Click `60db_tts_synthesize` 2. Enter: `{"text": "Hello!", "voice_id": "voice_abc123"}` 3. Click "Execute" 4. Receive audio URL ### Transcribe Audio 1. Click `60db_stt_transcribe` 2. Enter: `{"audio_url": "https://example.com/audio.mp3"}` 3. Click "Execute" 4. View transcription ## Debugging ### Enable Debug Logging ```bash theme={null} SIXTYDB_DEBUG=true mcp-inspector node dist/index.js ``` ### View Server Logs The Inspector displays real-time logs: ``` 60db MCP Server starting... API URL: https://api.60db.ai Auth: API Key 60db MCP Server running via stdio ``` ## Best Practices 1. **Test before integrating**: Always test in Inspector first 2. **Use sample data**: Start with simple examples 3. **Monitor response times**: Check tool execution times 4. **Validate responses**: Verify response formats ## Next Steps * [Claude Desktop](/mcp-server/claude-desktop) - Integrate with Claude # Memory Source: https://docs.60db.ai/mcp-server/memory # Memory & RAG Tools The 60db MCP Server exposes the full Memory/RAG layer to MCP clients like Claude Desktop. These tools let an AI assistant persist user memories, ingest knowledge base documents (91+ formats with built-in OCR), run hybrid semantic recall, and assemble LLM-ready context for retrieval-augmented generation. <Info> All memory tools are **pay-as-you-go** from the workspace wallet. Every billable response surfaces the new balance, the charge amount, and the transaction ID in the formatted output so the agent can reason about cost. See the [pricing reference](/api-reference/memory/pricing) for rates and refund policy. </Info> ## Tool summary | Tool | Billed | Purpose | | ---------------------------------- | ------ | ----------------------------------------------------------- | | `sixtydb_memory_ingest` | ✓ | Store a single memory | | `sixtydb_memory_ingest_batch` | ✓ | Batch-store up to 100 memories | | `sixtydb_memory_upload_document` | ✓✓ | Extract + ingest a document (PDF, DOCX, XLSX, images, etc.) | | `sixtydb_memory_search` | ✓ | Hybrid semantic + keyword recall | | `sixtydb_memory_context` | ✓ | Assemble LLM-ready context for RAG | | `sixtydb_memory_list_collections` | — | List collections in the workspace | | `sixtydb_memory_create_collection` | — | Create a team/knowledge/hive collection | | `sixtydb_memory_get_usage` | — | Monthly spend breakdown + wallet balance | | `sixtydb_memory_get_status` | — | Poll a memory's ingestion state | | `sixtydb_memory_delete` | — | Soft-delete a memory (24h undo) | ## Ingest a memory Store a single fact, preference, or conversation snippet: ```json theme={null} { "tool": "sixtydb_memory_ingest", "arguments": { "text": "User prefers vegetarian food and is lactose intolerant.", "title": "Dietary preferences", "type": "user", "infer": true } } ``` **Parameters:** * `text` (required) — content to store, max 100,000 characters * `title` (optional) — display title * `collection` (optional) — target collection ID (defaults to personal) * `type` — `user`, `knowledge`, or `hive` (default: `user`) * `infer` — if true, the memory service extracts structured facts via LLM **Cost:** \$0.0001 per 1,000 characters. ## Upload a document The most powerful memory tool — give the agent a file path and 60db handles format detection, OCR, chunking, and ingestion: ```json theme={null} { "tool": "sixtydb_memory_upload_document", "arguments": { "file_path": "/Users/me/Documents/quarterly-report.pdf", "collection": "company_reports", "type": "knowledge", "title": "Q4 2026 Quarterly Report" } } ``` **Parameters:** * `file_path` (required) — absolute path on the agent's local filesystem * `collection` (optional) — target collection * `type` — `user` | `knowledge` | `hive` (default: `knowledge` — right choice for docs) * `title` (optional) — display title, defaults to filename * `chunk_size` (optional) — characters per chunk (200–8000, default 1500) * `chunk_overlap` (optional) — character overlap between chunks (default 200) **Supported formats:** PDF, DOCX, DOC, ODT, RTF, TXT, MD, HTML, EPUB, XLSX, XLS, CSV, ODS, PPTX, PPT, ODP, EML, MSG, PNG, JPG, TIFF, BMP, and 70+ more. OCR is applied automatically to scanned PDFs and images. **Max file size:** 200 MB. **Max chunks per document:** 100. **Cost (two-stage):** * Extract fee: \$0.003 per MB (pre-charged) * Ingest fee: \$0.0001 per 1,000 extracted characters (post-charged) Both fees are automatically refunded on any failure. ## Search memories Hybrid semantic + keyword recall with optional cross-encoder reranking, across user memories AND knowledge documents in one call: ```json theme={null} { "tool": "sixtydb_memory_search", "arguments": { "query": "What are my dietary preferences?", "mode": "thinking", "max_results": 10, "alpha": 0.8 } } ``` **Parameters:** * `query` (required) — search text, max 2,000 chars * `collection` (optional) — collection to search * `mode` — `fast` (dense retrieval, \~100-200ms) or `thinking` (wider pool + cross-encoder rerank, \~200-400ms) * `max_results` — 1–50, default 10 * `alpha` — 0 (keyword only) to 1 (semantic only), default 0.8 * `recency_bias` — weight for newer memories (0–1), default 0 * `graph_context` — include knowledge-graph relationships **Advanced reranker knobs** (optional, override server defaults): * `rerank_top_k` — max candidates the cross-encoder reranks (1-500) * `rerank_timeout_ms` — hard timeout for rerank call (50-5000ms) * `min_rerank_score` — drop results below this score (0-1) * `fetch_multiplier` — in thinking mode, fetch N x max\_results candidates (1-10) When the reranker is active, each result includes a `rerank_score` (cross-encoder confidence, 0-1) alongside the regular `score` (dense similarity). The rerank score is the more reliable ranking signal. **Tuning by query type:** | Query type | Recommended | | --------------------- | ------------------------------- | | Exact phrase match | `alpha: 0.2, mode: fast` | | Conceptual question | `alpha: 0.9, mode: fast` | | Complex multi-faceted | `alpha: 0.7, mode: thinking` | | Latest-events focus | `alpha: 0.6, recency_bias: 0.3` | **Cost:** flat \$0.0003 per query regardless of parameters. ## Assemble context (RAG) Purpose-built for retrieval-augmented generation. Returns a `prompt_ready` string you can prepend directly to an LLM system message: ```json theme={null} { "tool": "sixtydb_memory_context", "arguments": { "query": "What should I tell the customer about their order?", "session_id": "chat_abc123", "top_k": 8, "max_context_length": 2000 } } ``` **Graceful degradation** — if the memory layer is unreachable, the tool returns an empty `prompt_ready` string and the charge is automatically refunded so the chat flow keeps working. **Cost:** flat \$0.0005 per query. ## Collections List everything the caller can see: ```json theme={null} { "tool": "sixtydb_memory_list_collections", "arguments": {} } ``` Create a team/knowledge/hive collection (admin/owner only): ```json theme={null} { "tool": "sixtydb_memory_create_collection", "arguments": { "collection_id": "customer_support", "label": "Customer Support KB", "kind": "knowledge", "shared": true } } ``` Personal collections are auto-created per user on first use and cannot be created via this tool. **Both are unbilled.** ## Monitor spend Track usage and wallet balance without affecting the wallet: ```json theme={null} { "tool": "sixtydb_memory_get_usage", "arguments": { "period": "current_month" } } ``` Returns net spend, operation count, refund count, and a per-service-type breakdown, plus the billing owner's current wallet balance. Always free, works even when the wallet is empty. **Periods:** `current_month` (default), `last_30_days`, `all_time`. ## Handling insufficient credits When the wallet runs out, billable tools return an error with `INSUFFICIENT_CREDITS` structure. Agents should: 1. Catch the 402 case 2. Call `sixtydb_memory_get_usage` to show the shortfall to the user 3. Prompt the user to top up via the 60db dashboard (link: `/app/billing`) 4. Retry the original operation once the wallet is funded ## Billing transparency in every response Every billable tool response includes a **Billing** footer in the formatted output: ``` **Memory queued** in collection `user_abc123` Type: user Total queued: 1 - `mem_01HV...` — pending: Memory queued for processing _Billing: charged **$0.000013** · balance **$9.464587** · tx `84ffd09e`_ ``` The JSON response format also embeds a `billing: { balance, charged, txId }` object for programmatic consumers. ## Related * [Memory pricing & billing](/api-reference/memory/pricing) * [Get memory usage endpoint](/api-reference/memory/usage) * [Memory feature overview](/features/memory) * [Authz MCP tools](/mcp-server/authz) # Quickstart Source: https://docs.60db.ai/mcp-server/quickstart # Quick Start Get started with the 60db MCP Server in minutes. ## Step 1: Install the Server ```bash theme={null} npm install @60db ``` ## Step 2: Set Your API Key ```bash theme={null} export SIXTYDB_API_KEY=sk_your_api_key_here ``` ## Step 2: Start the Server ```bash theme={null} npm start ``` You should see: ``` 60db MCP Server starting... API URL: https://api.60db.ai Auth: API Key 60db MCP Server running via stdio ``` ## Step 3: Test with Claude Desktop Configure Claude Desktop and restart. Then ask: ``` Can you list available English female voices? ``` Claude will use the MCP server to fetch voices. ## Example Usage ### Text-to-Speech ``` You: Convert "Hello, world!" to speech using voice "Sarah" ``` ### Speech-to-Text ``` You: Transcribe this meeting audio: https://example.com/meeting.mp3 ``` ### Create Custom Voice ``` You: Create a custom voice named "Brand Voice" from this sample ``` ## Next Steps * [Voices](/mcp-server/voices) - Learn about voice tools * [TTS](/mcp-server/tts) - Text-to-speech guide * [STT](/mcp-server/stt) - Speech-to-text guide # Stt Source: https://docs.60db.ai/mcp-server/stt # Speech-to-Text (STT) Transcribe audio files to text with optional speaker diarization. ## Available Tools | Tool | Description | | ------------------------ | ---------------------------------- | | `sixtydb_stt_transcribe` | Transcribe audio to text | | `sixtydb_stt_logs` | View transcription history | | `sixtydb_stt_get` | Get specific transcription details | ## Transcribe Audio Convert audio to text via the `sixtydb_stt_transcribe` tool: ```json theme={null} { "audio_url": "https://example.com/meeting.mp3", "language": "auto", "diarize": true } ``` ### Auto-detect vs explicit language * **Omit `language`** (or pass `"auto"`) to enable auto-detection across all 39 supported languages. The MCP shim strips `"auto"` before forwarding so the server's language identification runs. * **Pass a single ISO 639-1 code** (e.g. `"hi"`, `"en"`, `"ar"`) to skip language identification and run the fast path for that language. * Do **not** pass unsupported codes (`ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`) or Arabic dialect tags (`ar-eg`, `ar-lv`, …) — they return an `unsupported_language` error. For non-MSA Arabic audio pass `"ar"` for best-effort MSA transcription. ## Parameters | Parameter | Type | Default | Description | | ----------------- | ------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `audio_url` | string | Required | URL to audio file (downloaded and forwarded as multipart to `POST /stt`; max 25 MB) | | `language` | string | *auto* | ISO 639-1 code, or `"auto"` / omit for auto-detect | | `diarize` | boolean | `false` | Enable pyannote speaker diarization — adds a `speakers` array to each segment with `SPEAKER_00`, `SPEAKER_01`, … labels | | `context` | string | — | Free-form paragraph describing the session (domain, speakers, jargon). When supplied, the server runs a background LLM refinement pass; response text is polished for proper nouns, filler removal, and punctuation. Omit to skip refinement. | | `response_format` | string | `"markdown"` | `"markdown"` or `"json"` | ### Context string example ``` Cricket coaching session. Players: Arjun Mehta, Ishaan Verma, Aryan Khan, Rohan. Discussing batting technique, stamina, running between wickets, off-side balls. ``` <Note> `context` is a **plain string** on the REST `POST /stt` tool (this page). The WebSocket `/ws/stt` endpoint takes a structured `{general, text, terms}` **object** instead — see the [WebSocket STT reference](/api-reference/websocket/stt). </Note> ## Response shape (JSON) ```json theme={null} { "request_id": "req_...", "text": "Hello, thanks for joining...", "language": "en", "language_name": "English", "duration_sec": 5.2, "segments": [ { "start": 0.0, "end": 3.1, "text": "Hello, thanks for joining the call today.", "confidence": 0.92, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 } ], "speakers": [ { "speaker": "SPEAKER_00", "start": 0.0, "end": 3.1 } ] } ], "warning_codes": [] } ``` **Empty-speech signal**: A successful response with `text: ""` and `warning_codes: ["no_speech_detected"]` means the audio contained no speech. This is not an error — do not retry. ## Usage in Claude ``` You: Transcribe this meeting audio You: Transcribe this with speaker identification You: Transcribe only in Hindi (use language "hi") ``` ## Supported Audio Formats * WAV, MP3, M4A, OGG, FLAC, WebM, MP4 audio track * Max file size: 25 MB * Max duration: 1 hour * Recommended: 16 kHz+ sample rate ## Supported Languages 39 languages total — fetch the live catalog from the `sixtydb_stt_languages` tool (or the REST `GET /stt/languages` endpoint). Includes 25 European languages, 13 Indic languages with English code-switching, and Arabic MSA. See the [`Get STT Languages`](/api-reference/stt/get-languages) reference for the full list. ## Related * [STT Models](/api-reference/models/get-stt-models) — exposed STT model catalog (currently `60db-stt-v01`) * [WebSocket STT](/api-reference/websocket/stt) — real-time streaming variant (note: the WS form uses `languages: null` for auto-detect, not `"auto"`) * [TTS](/mcp-server/tts) — Text-to-speech # Tts Source: https://docs.60db.ai/mcp-server/tts # Text-to-Speech (TTS) Generate natural speech from text using the MCP server. ## Available Tools | Tool | Description | | --------------------- | ------------------------------- | | `60db_tts_synthesize` | Convert text to speech | | `60db_tts_logs` | View TTS generation history | | `60db_tts_get` | Get specific generation details | ## Synthesize Speech Convert text to audio: ```typescript theme={null} { "text": "Hello, world!", "voice_id": "voice_abc123", "speed": 1.0, "output_format": "mp3" } ``` **Response**: ```markdown theme={null} # TTS Generation Complete **Audio URL**: https://cdn.60db.ai/audio/tts_123456.mp3 **Duration**: 2.3 seconds **Voice**: Sarah (en-US) ``` ## Parameters | Parameter | Type | Default | Description | | --------------- | ------ | -------- | ----------------------------- | | `text` | string | Required | Text to synthesize | | `voice_id` | string | Required | Voice to use | | `speed` | number | 1 | Speech speed (0.25-2.0) | | `stability` | number | 50 | Voice stability (0-100) | | `similarity` | number | 75 | Voice similarity (0-100) | | `output_format` | string | mp3 | Output format (mp3, wav, ogg) | ## Usage in Claude ``` You: Convert "Welcome to our podcast" to speech You: Generate speech with faster speed You: Use a Spanish voice for this text ``` ## Supported Languages * English (US, UK, Australian) * Spanish, French, German, Italian * Portuguese, Japanese, Korean, Chinese * And 30+ more languages ## Related * [Voices](/mcp-server/voices) - Browse available voices * [STT](/mcp-server/stt) - Speech-to-text # Voices Source: https://docs.60db.ai/mcp-server/voices # Voices Manage and use voices for text-to-speech synthesis through the MCP server. ## Available Tools | Tool | Description | | ------------------- | --------------------------------------- | | `60db_list_voices` | List available voices with filtering | | `60db_get_voice` | Get detailed voice information | | `60db_create_voice` | Create a cloned voice from audio sample | ## List Voices Browse available voices by language, gender, and type: ```typescript theme={null} // List English female voices { "language": "en-US", "gender": "Female", "limit": 10 } ``` **Response**: ```markdown theme={null} # Available Voices | Voice ID | Name | Language | Gender | |----------|------|----------|--------| | voice_abc123 | Sarah | en-US | Female | | voice_def456 | Emma | en-US | Female | ``` ## Get Voice Details Retrieve detailed information about a voice: ```typescript theme={null} { "voice_id": "voice_abc123" } ``` ## Create Custom Voice Create a cloned voice from an audio sample: ```typescript theme={null} { "name": "Brand Voice", "sample_url": "https://example.com/sample.mp3", "language": "en-US", "gender": "Female" } ``` <Callout type="info"> Voice cloning requires a 30+ second audio sample with clear speech. </Callout> ## Usage in Claude ``` You: List all available English voices You: Get details for voice "Sarah" You: Create a custom voice from this sample ``` ## Related * [TTS](/mcp-server/tts) - Use voices for speech synthesis # Workspaces Source: https://docs.60db.ai/mcp-server/workspaces # Workspaces Manage team workspaces and collaboration through the MCP server. ## Available Tools | Tool | Description | | ---------------------------- | ---------------------- | | `60db_list_workspaces` | List all workspaces | | `60db_get_workspace` | Get workspace details | | `60db_create_workspace` | Create a new workspace | | `60db_get_workspace_members` | List workspace members | ## List Workspaces View all workspaces: ```typescript theme={null} { "limit": 20 } ``` ## Create Workspace Create a new team workspace: ```typescript theme={null} { "name": "Engineering Team", "description": "Main engineering workspace" } ``` ## Get Workspace Members List members of a workspace: ```typescript theme={null} { "workspace_id": "workspace_abc123" } ``` ## Usage in Claude ``` You: List all my workspaces You: Create a new workspace for the design team You: Who are the members of the Engineering workspace? ``` ## Related * [Analytics](/mcp-server/analytics) - Track usage # Quickstart Source: https://docs.60db.ai/quickstart Start using 60db in under 5 minutes ## Get Your API Key First, you'll need an API key to authenticate your requests: <Steps> <Step title="Sign Up"> Create an account at [app.60db.ai](https://app.60db.ai) </Step> <Step title="Navigate to API Keys"> Go to Settings → API Keys in your dashboard </Step> <Step title="Create New Key"> Click "Create API Key" and give it a descriptive name </Step> <Step title="Copy Your Key"> Copy your API key and store it securely </Step> </Steps> <Warning> Keep your API key secret! Never commit it to version control or expose it in client-side code. </Warning> ## Choose Your SDK <Tabs> <Tab title="JavaScript/TypeScript"> ### Installation ```bash theme={null} npm install 60db ``` ### Basic Usage ```typescript theme={null} import { SixtyDBClient } from '60db'; // Initialize the client const client = new SixtyDBClient('your-api-key'); // Text to Speech const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice' }); // Get all voices const voices = await client.getVoices(); console.log(voices); ``` </Tab> <Tab title="Python"> ### Installation ```bash theme={null} pip install 60db ``` ### Basic Usage ```python theme={null} from sixtydb import SixtyDBClient # Initialize the client client = SixtyDBClient('your-api-key') # Text to Speech audio = client.text_to_speech('Hello, world!', voice_id='default-voice') # Save audio to file with open('output.mp3', 'wb') as f: f.write(audio) # Get all voices voices = client.get_voices() print(voices) ``` </Tab> </Tabs> ## Next Steps <CardGroup> <Card title="Explore Features" icon="compass" href="/features/text-to-speech"> Learn about all available features </Card> <Card title="API Reference" icon="book" href="/api-reference/introduction"> Dive into the complete API documentation </Card> <Card title="Custom Voices" icon="user-voice" href="/features/voices"> Create your own custom voice profiles </Card> <Card title="Memory & RAG" icon="brain" href="/features/memory"> Give your AI persistent memory and upload documents (PDF/DOCX/OCR) </Card> <Card title="LLM Chat" icon="comments" href="/features/llm-chat"> OpenAI-compatible chat completions with 60db models </Card> <Card title="Webhooks" icon="webhook" href="/platform/webhooks"> Set up webhooks for event notifications </Card> </CardGroup> # JavaScript/TypeScript SDK Source: https://docs.60db.ai/sdks/javascript Complete guide to the 60db JavaScript/TypeScript SDK ## Installation ```bash theme={null} npm install 60db ``` ## Initialization ```typescript theme={null} import { SixtyDBClient } from "60db"; // Simple initialization const client = new SixtyDBClient("your-api-key"); ``` ## Text-to-Speech ### Basic TTS ```typescript theme={null} const audio = await client.textToSpeech({ text: 'Hello, world!', voice_id: 'default-voice', enhance: true, speed: 1.0, language:"en-us" }); ### Get all voices const voices = await client.getVoices(); ### Get all lanuages const lanuages = await client.getLanguages(); ### audio is an ArrayBuffer containing the audio data ``` ## Speech-to-Text ### Transcribe Audio ```typescript theme={null} const file = document.querySelector('input[type="file"]').files[0]; const result = await client.speechToText(file, { language: "en", }); console.log(result.text); ``` ### With Context Refinement Supply a `context` string to enable server-side LLM polishing — proper nouns, filler removal, and punctuation are cleaned on the response text: ```typescript theme={null} const result = await client.speechToText(file, { language: "hi", diarize: true, context: "Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.", }); ``` Omit `context` entirely to skip refinement. Note: this is the REST shape — the WebSocket streaming endpoint takes a `{general, text, terms}` object instead. ### Get Supported Languages ```typescript theme={null} const languages = await client.getLanguages(); console.log(languages); ``` ## Voice Management ### List All Voices ```typescript theme={null} const voices = await client.getVoices(); voices.forEach((voice) => { console.log(`${voice.name} (${voice.id})`); }); ``` ### Get Specific Voice ```typescript theme={null} const voice = await client.getVoice("voice-id"); console.log(voice); ``` ### Create Custom Voice ```typescript theme={null} const files = [file1, file2, file3]; // File objects const newVoice = await client.createVoice({ name: "My Custom Voice", files: files, description: "A custom voice for my brand", }); console.log("Created voice:", newVoice.id); ``` ### Update Voice ```typescript theme={null} await client.updateVoice("voice-id", { name: "Updated Voice Name", description: "Updated description", }); ``` ### Delete Voice ```typescript theme={null} await client.deleteVoice("voice-id"); ``` ## Authentication ### Sign Up ```typescript theme={null} const user = await client.signUp({ email: "user@example.com", password: "secure-password", name: "John Doe", }); ``` ### Sign In ```typescript theme={null} const session = await client.signIn({ email: "user@example.com", password: "secure-password", }); console.log("Token:", session.token); ``` ### Get Profile ```typescript theme={null} const profile = await client.getProfile(); console.log(profile); ``` ### Update Profile ```typescript theme={null} await client.updateProfile({ name: "Jane Doe", company: "Acme Inc", }); ``` ## Workspace Management ### List Workspaces ```typescript theme={null} const workspaces = await client.getWorkspaces(); ``` ### Create Workspace ```typescript theme={null} const workspace = await client.createWorkspace({ name: "My Workspace", description: "Team workspace", }); ``` ## Billing ### Get Available Plans ```typescript theme={null} const plans = await client.getPlans(); plans.forEach((plan) => { console.log(`${plan.name}: $${plan.price}/month`); }); ``` ### Get Current Plan ```typescript theme={null} const currentPlan = await client.getCurrentPlan(); console.log("Current plan:", currentPlan.name); ``` ### Subscribe to Plan ```typescript theme={null} await client.subscribe("plan-id"); ``` ## Analytics ### Get Usage Statistics ```typescript theme={null} const usage = await client.getUsage(); console.log("Characters used:", usage.characters); console.log("API calls:", usage.api_calls); ``` ## API Key Management ### List API Keys ```typescript theme={null} const apiKeys = await client.getApiKeys(); ``` ### Create API Key ```typescript theme={null} const newKey = await client.createApiKey("Production Key"); console.log("New API key:", newKey.key); ``` ### Delete API Key ```typescript theme={null} await client.deleteApiKey("key-id"); ``` ## Webhooks ### List Webhooks ```typescript theme={null} const webhooks = await client.getWebhooks(); ``` ### Create Webhook ```typescript theme={null} const webhook = await client.createWebhook({ url: "https://example.com/webhook", events: ["tts.completed", "stt.completed"], }); ``` ### Delete Webhook ```typescript theme={null} await client.deleteWebhook("webhook-id"); ``` ## Error Handling ```typescript theme={null} try { const audio = await client.textToSpeech({ text: "Hello, world!", voice_id: "invalid-voice", }); } catch (error) { if (error.response) { // API error console.error("API Error:", error.response.status); console.error("Message:", error.response.data.message); } else { // Network or other error console.error("Error:", error.message); } } ``` ## TypeScript Types The SDK includes full TypeScript type definitions: ```typescript theme={null} import { SixtyDBClient, SixtyDBConfig, TTSResponse } from "60db"; ``` # Python SDK Source: https://docs.60db.ai/sdks/python Complete guide to the 60db Python SDK ## Installation ```bash theme={null} pip install 60db ``` ## Initialization ```python theme={null} from sixtydb import SixtyDBClient # Simple initialization client = SixtyDBClient('your-api-key') # With custom configuration ### Get all voices voices = await client.getVoices(); ### Get all lanuages lanuages = await client.getLanguages(); ### audio is an ArrayBuffer containing the audio data ``` ## Text-to-Speech ### Basic TTS ```python theme={null} audio = client.text_to_speech( text='Hello, world!', voice_id='default-voice', enhance=True, speed=1.0, language='en-us' ) # Save to file with open('output.mp3', 'wb') as f: f.write(audio) ``` ## Speech-to-Text ### Transcribe Audio ```python theme={null} with open('audio.mp3', 'rb') as audio_file: result = client.speech_to_text(audio_file, language='en') print(result['text']) ``` ### With Context Refinement Supply a `context` string to enable server-side LLM polishing — proper nouns, filler removal, and punctuation are cleaned on the response text: ```python theme={null} with open('audio.mp3', 'rb') as audio_file: result = client.speech_to_text( audio_file, language='hi', diarize=True, context='Cricket coaching session. Players: Arjun Mehta, Ishaan Verma. Discussing batting technique.', ) print(result['text']) ``` Omit `context` entirely to skip refinement. Note: this is the REST shape — the WebSocket streaming endpoint takes a `{general, text, terms}` object instead. ### Get Supported Languages ```python theme={null} languages = client.get_languages() for lang in languages: print(f"{lang['name']} ({lang['code']})") ``` ## Voice Management ### List All Voices ```python theme={null} voices = client.get_voices() for voice in voices: print(f"{voice['name']} ({voice['id']})") ``` ### Get Specific Voice ```python theme={null} voice = client.get_voice('voice-id') print(voice) ``` ### Create Custom Voice ```python theme={null} files = [ open('sample1.mp3', 'rb'), open('sample2.mp3', 'rb'), open('sample3.mp3', 'rb') ] new_voice = client.create_voice( name='My Custom Voice', files=files, description='A custom voice for my brand' ) print(f"Created voice: {new_voice['id']}") # Close files for f in files: f.close() ``` ### Update Voice ```python theme={null} client.update_voice( voice_id='voice-id', name='Updated Voice Name', description='Updated description' ) ``` ### Delete Voice ```python theme={null} client.delete_voice('voice-id') ``` ## Authentication ### Sign Up ```python theme={null} user = client.sign_up( email='user@example.com', password='secure-password', name='John Doe' ) ``` ### Sign In ```python theme={null} session = client.sign_in( email='user@example.com', password='secure-password' ) print(f"Token: {session['token']}") ``` ### Get Profile ```python theme={null} profile = client.get_profile() print(profile) ``` ### Update Profile ```python theme={null} client.update_profile( name='Jane Doe', company='Acme Inc' ) ``` ## Workspace Management ### List Workspaces ```python theme={null} workspaces = client.get_workspaces() ``` ### Create Workspace ```python theme={null} workspace = client.create_workspace( name='My Workspace', description='Team workspace' ) ``` ## Billing ### Get Available Plans ```python theme={null} plans = client.get_plans() for plan in plans: print(f"{plan['name']}: ${plan['price']}/month") ``` ## Analytics ### Get Usage Statistics ```python theme={null} usage = client.get_usage() print(f"Characters used: {usage['characters']}") print(f"API calls: {usage['api_calls']}") ``` ## API Key Management ### List API Keys ```python theme={null} api_keys = client.get_api_keys() ``` ### Create API Key ```python theme={null} new_key = client.create_api_key('Production Key') print(f"New API key: {new_key['key']}") ``` ### Delete API Key ```python theme={null} client.delete_api_key('key-id') ``` ## Webhooks ### List Webhooks ```python theme={null} webhooks = client.get_webhooks() ``` ### Create Webhook ```python theme={null} webhook = client.create_webhook( url='https://example.com/webhook', events=['tts.completed', 'stt.completed'] ) ``` ### Delete Webhook ```python theme={null} client.delete_webhook('webhook-id') ``` ## Error Handling ```python theme={null} from requests.exceptions import HTTPError try: audio = client.text_to_speech( text='Hello, world!', voice_id='invalid-voice' ) except HTTPError as e: print(f"HTTP Error: {e.response.status_code}") print(f"Message: {e.response.json()['message']}") except Exception as e: print(f"Error: {str(e)}") ``` ## Type Hints The SDK includes type hints for better IDE support: ```python theme={null} from sixtydb import SixtyDBClient from typing import Dict, List, Any client: SixtyDBClient = SixtyDBClient('your-api-key') voices: List[Dict[str, Any]] = client.get_voices() ``` ## Async Support For async applications, you can use the SDK with asyncio: ```python theme={null} import asyncio from sixtydb import SixtyDBClient async def main(): client = SixtyDBClient('your-api-key') # Run in executor for async compatibility loop = asyncio.get_event_loop() audio = await loop.run_in_executor( None, client.text_to_speech, 'Hello, world!', 'default-voice' ) with open('output.mp3', 'wb') as f: f.write(audio) asyncio.run(main()) ``` # Codex Skills Source: https://docs.60db.ai/skills/introduction Drop 60db voice into Claude Code, OpenAI Codex, OpenCode, or any agent with a skills directory — triggered by /60db. # 60db Agent Skills Bring 60db's full voice stack — **text-to-speech, speech-to-text, voice cloning, voice management, and the LLM core** — directly into your coding agent. Install it once and call it with `/60db` from [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [OpenAI Codex](https://developers.openai.com/codex), [OpenCode](https://opencode.ai), or any agent that reads a skills directory. It's a single **zero-dependency** Python CLI (stdlib only, Python 3.8+) — no SDK, no `pip install`, no version drift. <CardGroup> <Card title="Claude Code" icon="comments"> Native skill — `/60db` and `/setup-60db` </Card> <Card title="OpenAI Codex" icon="robot"> Same skill folders, copied into `~/.codex/skills` </Card> <Card title="OpenCode & others" icon="terminal"> Plain markdown + stdlib Python runs anywhere </Card> </CardGroup> <Info> Source repository: [github.com/60db-ai/60db.ai-skills](https://github.com/60db-ai/60db.ai-skills) — Apache-2.0. </Info> *** ## Why a Skill, Not Just the SDK The skill **encodes the behavior that's actually true** against the live API, so you never relearn the gotchas. It's stdlib-only, so it keeps working regardless of SDK or doc drift. The five mistakes it pins for you: <AccordionGroup> <Accordion title="REST /tts-synthesize returns NDJSON, not one JSON blob"> Concatenate each line's `result.audioContent` (base64 LINEAR16 PCM). The CLI does this for you. </Accordion> <Accordion title="Only 8000 / 16000 / 24000 / 48000 Hz are accepted"> `44100` is rejected with a `400`. </Accordion> <Accordion title="The output_format flag is ignored"> The API always returns raw PCM — wrap it as WAV yourself (the CLI does). </Accordion> <Accordion title="Sample rate ≠ fidelity"> The model is band-limited at \~8 kHz, so 48 kHz is upsampled headroom. `enhance` and `60db-quality` do not raise the ceiling. </Accordion> <Accordion title="The voice id field is voice_id, not id"> Language and gender live under `labels` — confirmed by live inspection. </Accordion> </AccordionGroup> *** ## How It Works One CLI — `skills/60db/scripts/sixtydb.py` — fronts every 60db use case against `https://api.60db.ai`. ``` /60db 1. Setup once → user runs `init`: hidden key prompt → config (mode 600) 2. Ask if unclear → prompt for: use case · voice · model · kHz · format 3. Route → tts / stt / voices / clone / chat / langs / doctor 4. Verify → `doctor` reports key presence + resolved defaults (never the key) ``` Every setting resolves the same way, so you can override per-call or store defaults once: ``` CLI flag > env SIXTYDB_* > config file > built-in default ``` <Note> **Setup is the user's job.** The API key is never typed by the agent, never passed on the command line, never printed, never committed. The user runs `init` (a hidden prompt) or sets `SIXTYDB_API_KEY`. In a chat session where a hidden prompt isn't reachable, the agent scaffolds defaults with `init --no-key` and hands the key step back to the user. </Note> *** ## Use Cases What you can build with `/60db` without leaving your coding agent: <CardGroup> <Card title="Narration & voiceovers" icon="microphone"> Turn a script, blog post, or `.txt` file into studio-quality audio for videos, podcasts, or demos — `tts script.txt --voice <id> --out vo.wav`. </Card> <Card title="Transcription & subtitles" icon="closed-captioning"> Convert recordings, calls, or meetings into text with speaker labels and word timings, then build SRT/VTT — `stt rec.mp4 --diarize --timestamps --json`. </Card> <Card title="Voice agents & bots" icon="robot"> Wire `stt` (ears) + `chat` (brain) + `tts` (mouth) into a conversational loop for support, reception, or IVR-style flows. </Card> <Card title="Custom brand voices" icon="user-pen"> Clone a voice from a few samples and reuse it across all your audio — `clone --name "Brand VO" --sample a.wav --sample b.wav`. </Card> <Card title="Accessibility" icon="universal-access"> Add read-aloud audio to docs, articles, and apps for visually impaired or on-the-go users. </Card> <Card title="Multilingual content" icon="language"> Generate speech and transcribe audio across 30+ languages — check `langs` / `langs --stt` for coverage. </Card> </CardGroup> ### Who it's for <AccordionGroup> <Accordion title="Developers building voice features"> Add TTS/STT to an app without learning the SDK or fighting the docs — the skill drives one stdlib-only CLI that already pins the API's real behavior. </Accordion> <Accordion title="Creators & content teams"> Produce voiceovers and dubbed audio straight from a script file, with a consistent brand voice via cloning. </Accordion> <Accordion title="Support & operations teams"> Prototype phone/voice bots from the `stt + chat + tts` parts, and transcribe call recordings for QA or summaries. </Accordion> <Accordion title="Anyone in a coding agent"> If you live in Claude Code, Codex, or OpenCode, `/60db` lets you ship audio without context-switching to a dashboard or another tool. </Accordion> </AccordionGroup> ### A typical workflow <Steps> <Step title="Set up once"> Run `/setup-60db` to store your API key and defaults (voice, model, sample rate). </Step> <Step title="Pick a voice"> Run `voices` to list available voice IDs (add `--mine` for your cloned voices). </Step> <Step title="Generate or transcribe"> Call `tts` to make audio or `stt` to make text — the skill asks for anything it needs. </Step> <Step title="Iterate or chain"> Adjust `--speed` / `--stability`, or chain `stt → chat → tts` for a full voice-agent turn. </Step> </Steps> *** ## The Two Skills | Skill | Trigger | What it does | | -------------- | ------------- | -------------------------------------------------------------------------- | | **60db** | `/60db` | The worker — routes every voice use case to the CLI | | **setup-60db** | `/setup-60db` | First-run onboarding: collects the key privately, picks defaults, verifies | The worker drives one CLI with **9 subcommands**: | Subcommand | What it does | Status | | -------------- | ---------------------------------------------------------------------- | ------------------------ | | `init` | Hidden key prompt → config (mode 600); `--no-key` stores defaults only | ✅ | | `doctor` | Diagnose setup — key presence (never revealed), resolved defaults | ✅ | | `tts` | Text or `.txt` → WAV (REST NDJSON; `--ws` for legacy WebSocket) | ✅ verified | | `stt` | Audio → transcript (`--diarize`, `--timestamps`, `--json`) | ✅ verified | | `voices` | List voice ids (built-in + your cloned), `--mine`, `--json` | ✅ verified | | `clone` | Train a new voice from samples (`--sample` / `--sample-url`) | ⚠️ doc-only — test first | | `delete-voice` | Hard-delete one of your custom voices | ⚠️ doc-only | | `langs` | Supported languages (`--stt` for STT's 39) | ✅ | | `chat` | LLM core (`60db-tiny`) you pair with stt+tts for a voice agent | documented | *** ## Install <Steps> <Step title="Add the skill"> **Claude Code (recommended):** ```bash theme={null} npx skills add 60db-ai/60db.ai-skills ``` **Codex / OpenCode / other agents** — copy the two skill folders into that agent's skills directory: ```bash theme={null} git clone https://github.com/60db-ai/60db.ai-skills.git cp -r 60db.ai-skills/skills/60db 60db.ai-skills/skills/setup-60db ~/.codex/skills/ ``` </Step> <Step title="Restart your agent session"> Skills are only picked up on session start — this is an agent-platform limitation, not a bug. </Step> <Step title="Run setup once"> Run `/setup-60db`, or directly: ```bash theme={null} E=skills/60db/scripts/sixtydb.py python3 $E init # hidden prompt for your API key → config (mode 600) python3 $E doctor # verify (reports key presence without revealing it) ``` Get a key at **app.60db.ai → Settings → Developer → API Keys**. </Step> </Steps> <Warning> The API key is **yours to enter** — never paste it into the agent chat (chat history is retained by the platform). It's stored locally at `~/.config/60db/config.json` (mode 600) or read from `SIXTYDB_API_KEY`, and `.gitignore` keeps it out of git. </Warning> *** ## Run It ```bash theme={null} E=skills/60db/scripts/sixtydb.py python3 $E tts "Hello there." --out out/hello.wav # text → WAV (48 kHz, 60db-quality) python3 $E tts script.txt --voice <id> --out out/vo.wav python3 $E stt recording.mp3 --diarize --timestamps # audio → transcript python3 $E voices # list voice ids python3 $E langs --stt # 39 STT languages python3 $E chat "One-line summary of attachment theory." # LLM core for agents python3 $E doctor # diagnose setup ``` Store defaults once so you stop repeating flags: ```bash theme={null} python3 $E init --no-key --voice <id> --model 60db-quality --sample-rate 48000 ``` *** *** ## Quick Decision Guide | I want to... | Use | | ---------------------------------------- | ---------------------------------------------------------------------- | | Narration / voiceover from text | `tts "..." --out out/vo.wav` | | Read a script file aloud | `tts script.txt --voice <id> --out out/vo.wav` | | Transcribe audio with speaker labels | `stt rec.mp3 --diarize --timestamps` | | See which voices I can use | `voices` (then `voices --mine`) | | Use a specific language | `langs` / `langs --stt` | | Clone my own brand voice | `clone --name "Brand VO" --sample a.wav --sample b.wav` *(test first)* | | Build a phone/voice bot | `chat` as the brain + `stt` ears + `tts` mouth | | Figure out why audio sounds "compressed" | `doctor` → troubleshooting | | Set up for the first time | `/setup-60db` | *** ## Examples <CodeGroup> ```bash Voiceover from text theme={null} python3 $E tts "Welcome to the show." --voice <id> --out out/intro.wav # wrote out/intro.wav (412160 B PCM, ~4.3s @ 48000Hz, REST, model=60db-quality) ``` ```bash Narrate a script file theme={null} python3 $E tts episode.txt --voice <id> --out out/ep.wav --speed 0.95 --stability 65 ``` ```bash Transcribe with speakers + timings theme={null} python3 $E stt interview.m4a --diarize --timestamps --out interview.txt # [English, 42.7s] So the first thing we noticed was the latency... ``` ```bash Structured JSON for subtitles theme={null} python3 $E stt talk.mp4 --timestamps --confidence --json > talk.json # talk.json → words[] each with {word, start, end} → build SRT/VTT ``` ```bash One voice-agent turn: ears → brain → mouth theme={null} USER=$(python3 $E stt caller.wav --json | python3 -c 'import json,sys;print(json.load(sys.stdin)["text"])') REPLY=$(python3 $E chat "$USER" --system "Concise support agent." --chat-id call-42 | head -1) python3 $E tts "$REPLY" --voice <id> --out out/reply.wav ``` </CodeGroup> *** ## Gotchas That Bite Everyone | Symptom | Cause | Fix | | --------------------------------- | --------------------------------------------------- | ------------------------------------------------------------------ | | Audio sounds a touch "compressed" | Model is \~16 kHz-native, band-limited to \~8 kHz | Expected; `enhance` / `60db-quality` don't lift it | | Garbled / truncated audio | REST returns **NDJSON**, not one JSON blob | Concatenate every line's `result.audioContent` (the CLI does this) | | `400` on synthesize | Sample rate `44100` (or anything off-menu) | Use `8000 / 16000 / 24000 / 48000` only | | Format flag seems ignored | `output_format` **is** ignored — always raw PCM | Wrap the PCM as WAV yourself (the CLI does) | | Long script cut off | TTS caps at **5000 chars** per request | Split, synthesize, concatenate WAVs | | `voices` prints `None` for the id | Field is `voice_id`; language/gender under `labels` | The CLI already handles this | | `no API key found` | Key not in config or env | Run `init`, or export `SIXTYDB_API_KEY` | *** ## Coverage | Use case | Command | Status | | ---------------------------------- | -------------- | ---------------------------------- | | Text-to-speech (REST + WS) | `tts` | ✅ verified | | Speech-to-text (batch + streaming) | `stt` | ✅ endpoint verified | | List voices | `voices` | ✅ verified (`voice_id` / `labels`) | | Voice cloning | `clone` | ⚠️ doc-only — test first | | Update / delete voice | `delete-voice` | ⚠️ doc-only | | Languages | `langs` | ✅ | | LLM chat (voice-agent core) | `chat` | documented; no native calling API | *** ## How It Helps <CardGroup> <Card title="Zero setup friction" icon="bolt"> Stdlib-only Python — no SDK, no `requests`, no version drift. Only `python3` 3.8+ required. </Card> <Card title="Truth over docs" icon="shield-check"> Every endpoint is annotated verified vs. doc-only; live-API findings win over the published docs. </Card> <Card title="Voice agents from parts" icon="phone"> No native telephony API — build a voice agent from `stt` (ears) + `chat` (brain) + `tts` (mouth). </Card> <Card title="Secure by design" icon="lock"> The key is never typed by the agent, never printed, never committed. </Card> </CardGroup> *** ## FAQ <AccordionGroup> <Accordion title="How do I install it the fastest way?"> `npx skills add 60db-ai/60db.ai-skills`, then restart your agent and run `/setup-60db`. </Accordion> <Accordion title="Where does my API key go? Is it safe?"> Into a local config at `~/.config/60db/config.json` (mode 600), or the `SIXTYDB_API_KEY` env var. It's never passed on the command line, never printed, and `.gitignore` keeps it out of git. The agent never types it for you. </Accordion> <Accordion title="My voice sounds slightly compressed at 48 kHz. Bug?"> No. The model is \~16 kHz-native and band-limited to \~8 kHz, so 48 kHz is upsampled headroom. `enhance` and `60db-quality` don't raise the ceiling. </Accordion> <Accordion title="Which sample rates are allowed?"> `8000`, `16000`, `24000`, `48000` only. `44100` is rejected. </Accordion> <Accordion title="Can 60db make or receive phone calls?"> There is no native calling/telephony API. You build a voice agent from the parts: `stt` (ears) + `chat` (brain, `60db-tiny`) + `tts` (mouth). </Accordion> <Accordion title="Does it work outside Claude Code?"> Yes — it's markdown + stdlib Python. Copy the two skill folders into Codex's or OpenCode's skills directory. Only the onboarding prompt is Claude-specific and degrades gracefully. </Accordion> <Accordion title="Do I need the SDK or requests?"> No. Only `python3` 3.8+. The single optional dependency is `websockets`, used solely by the legacy `--ws` TTS path. </Accordion> </AccordionGroup> <Note> Full command reference, recipes, and the complete troubleshooting list live in the repo's [GUIDE.md](https://github.com/60db-ai/60db.ai-skills/blob/main/GUIDE.md) and [references/](https://github.com/60db-ai/60db.ai-skills/tree/main/skills/60db/references). </Note> # WebSocket API Source: https://docs.60db.ai/websocket-api Real-time Speech-to-Text (STT) and Text-to-Speech (TTS) streaming API # WebSocket API Real-time bidirectional streaming API for Speech-to-Text (STT) and Text-to-Speech (TTS) services. ## Overview The 60db WebSocket API provides real-time streaming capabilities for: * **Speech-to-Text (STT)**: Convert audio to text with 99+ language support * **Text-to-Speech (TTS)**: Synthesize natural-sounding speech with multiple voices ## Base URLs | Environment | STT URL | TTS URL | | ------------------- | -------------------------- | -------------------------- | | Production | `ws://api.60db.ai/ws/stt` | `ws://api.60db.ai/ws/tts` | | Production (Secure) | `wss://api.60db.ai/ws/stt` | `wss://api.60db.ai/ws/tts` | ## Authentication WebSocket connections require authentication using API key via query parameter: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key_here ``` **Getting Your API Key:** 1. Go to [app.60db.ai](https://app.60db.ai) 2. Navigate to **Settings → Developer → API Keys** 3. Click **Create API Key** 4. Copy and store your API key securely *** ## 🎤 STT WebSocket (Speech-to-Text) **What it does:** You send audio → You get text back **Use for:** Transcribing speech, voice commands, call center analytics, meeting transcription ### Quick Start ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`wss://api.60db.ai/ws/stt?apiKey=${API_KEY}`); ws.onopen = () => console.log('✅ Connected'); ws.onmessage = (data) => { const msg = JSON.parse(data); // STT tags this frame with `type` and keeps the fields top-level // (TTS nests them under `connection_established` — the shapes differ). if (msg.type === 'connection_established') { console.log('✅ Authenticated!'); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true } })); } // Wait for `session_started`, not `connected` — `connected` fires before // the session exists and arrives twice (proxy frame, then upstream frame). if (msg.type === 'session_started') { console.log('✅ Ready! Send audio now'); // Send audio chunks const interval = setInterval(() => { ws.send(audioBuffer); // Your audio data }, 60); // Stop after 5 seconds setTimeout(() => { clearInterval(interval); ws.send(JSON.stringify({ type: 'stop' })); }, 5000); } if (msg.type === 'transcription') { console.log('📝 Text:', msg.text); } if (msg.type === 'session_stopped') { console.log('✅ Done! Cost:', msg.billing_summary.total_cost); ws.close(); } }; ``` ### How STT Works ``` 1. Connect → 2. Authenticate → 3. Start session → 4. Send audio → 5. Get text → 6. Stop ``` *** ## 🔊 TTS WebSocket (Text-to-Speech) **What it does:** You send text → You get audio back **Use for:** Voice assistants, audiobooks, accessibility, chatbots ### Quick Start ```javascript theme={null} const WebSocket = require('ws'); const fs = require('fs'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`wss://api.60db.ai/ws/tts?apiKey=${API_KEY}`); const contextId = 'my-session-' + Date.now(); const audioChunks = []; ws.onopen = () => console.log('✅ Connected'); ws.onmessage = (data) => { const msg = JSON.parse(data); if (msg.connection_established) { console.log('✅ Authenticated!'); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 } } })); } if (msg.context_created) { console.log('✅ Context created!'); // Send text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, this is a test of the text to speech service.' } })); // Flush ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log('🔊 Audio chunk received'); } if (msg.flush_completed) { console.log('✅ All audio received!'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } if (msg.context_closed) { console.log('✅ Done!'); // Save audio const audio = Buffer.concat(audioChunks); fs.writeFileSync('output.pcm', audio); console.log('💾 Saved output.pcm'); ws.close(); } }; ``` ### How TTS Works ``` 1. Connect → 2. Authenticate → 3. Create context → 4. Send text → 5. Flush → 6. Get audio → 7. Close ``` *** ## 📋 STT vs TTS Comparison | Feature | STT (Speech-to-Text) | TTS (Text-to-Speech) | | ----------------- | ------------------------ | --------------------------- | | **Input** | Audio (binary) | Text (string) | | **Output** | Text (string) | Audio (binary) | | **Use Case** | Transcribe speech | Generate speech | | **Direction** | Audio → Text | Text → Audio | | **First Message** | `{ type: "start", ... }` | `{ create_context: {...} }` | | **Session End** | `{ type: "stop" }` | `{ close_context: {...} }` | | **Pricing** | \$0.00000833/second | \$0.00002/character | *** ## 📚 Full Documentation * 🎤 [STT WebSocket](/websocket-api/stt) - Complete STT guide with all parameters * 🔊 [TTS WebSocket](/websocket-api/tts) - Complete TTS guide with all parameters *** ## 💡 Key Concepts ### STT Messages ```javascript theme={null} // Start session { type: "start", languages: ["en"], config: {...} } // Stop session { type: "stop" } ``` ### TTS Messages ```javascript theme={null} // Create context { create_context: { context_id, voice_id, audio_config } } // Send text { send_text: { context_id, text: "..." } } // Get audio { flush_context: { context_id } } // Close session { close_context: { context_id } } ``` *** ## 📊 Pricing | Service | Rate | Minimum | | ------- | ------------------- | ------- | | STT | \$0.00000833/second | \$0.01 | | TTS | \$0.00002/character | \$0.01 | *** ## 🆘 Support * **Email**: [support@60db.ai](mailto:support@60db.ai) * **Documentation**: [https://docs.60db.ai](https://docs.60db.ai) * **Status**: [https://status.60db.ai](https://status.60db.ai) # STT WebSocket Source: https://docs.60db.ai/websocket-api/stt WebSocket /ws/stt Real-time Speech-to-Text WebSocket API for streaming audio transcription # STT WebSocket API Real-time Speech-to-Text transcription via WebSocket streaming with support for 39 languages (including code-switched Indic+English) and telephony integration. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack). ## 🚀 Quick Start (Copy & Paste) ```javascript theme={null} const WebSocket = require('ws'); // 1. Your API key const API_KEY = 'sk_live_your_api_key'; // 2. Connect const ws = new WebSocket(`wss://api.60db.ai/ws/stt?apiKey=${API_KEY}`); // 3. Handle messages ws.on('message', (data) => { const msg = JSON.parse(data); // Authenticated? Start session! // NOTE: on STT the fields are top-level and the frame is tagged by `type`. if (msg.type === 'connection_established') { console.log('✅ Authenticated'); ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true } })); } // Session ready? Send audio! // Gate on `session_started`, not `connected` — `connected` fires before the // session exists and arrives twice (proxy frame, then upstream frame). if (msg.type === 'session_started') { console.log('✅ Ready! Send audio now'); // Send dummy audio (480 bytes every 60ms) let count = 0; const interval = setInterval(() => { ws.send(Buffer.alloc(480, 0xff)); if (++count >= 83) { // 5 seconds clearInterval(interval); ws.send(JSON.stringify({ type: 'stop' })); } }, 60); } // Got text! if (msg.type === 'transcription' && msg.is_final) { console.log('📝', msg.text); } // Done! if (msg.type === 'session_stopped') { console.log('✅ Complete! Cost:', msg.billing_summary.total_cost); ws.close(); } }); ``` **That's it!** You'll see: * ✅ Authenticated * ✅ Ready! Send audio now * 📝 Hello world (transcribed text) * ✅ Complete! Cost: \$0.000043 *** ## 📖 How It Works (5 Simple Steps) 1. **Connect** with your API key 2. **Send** `{ type: "start", ... }` to begin session 3. **Stream** audio data (binary chunks) 4. **Receive** text transcriptions in real-time 5. **Stop** with `{ type: "stop" }` when done *** ## Endpoint <ParamField name="url" type="string"> `ws://api.60db.ai/ws/stt` or `wss://api.60db.ai/ws/stt` </ParamField> ## Authentication Query parameter authentication: <ParamField name="apiKey" type="string"> Your API key for authentication </ParamField> Example: ``` ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key ``` ## Connection Details | Property | Value | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Protocol | WebSocket (RFC 6455) | | Frame types | Binary (telephony) or Text/JSON (browser) | | Ping/keepalive | The server answers client-initiated WebSocket pings with a pong; it does not ping first. Send a ping every 20–30 s so intermediaries do not idle the socket out. | ## Session Lifecycle ``` Client Server │ │ │──── TCP/TLS connect ─────────►│ │◄─── {"type":"connecting"} ────│ authenticating... │◄─── {"type":"connected"} ─────│ proxy wired to upstream │◄─── connection_established ──│ authenticated — safe to send `start` │ │ │──── {"type":"start", ...} ───►│ │◄─── {"type":"session_started"}│ safe to send audio │ │ │──── audio frames / messages ─►│ │◄─── {"type":"speech_started"} │ VAD detected voice │◄─── {"type":"transcription"} │ is_final=true, speech_final=false (first emit, context only) │◄─── {"type":"transcription"} │ is_final=true, speech_final=true (canonical answer) │ │ │──── {"type":"stop"} ─────────►│ │◄─── {"type":"session_stopped"}│ ``` **Two-phase finals (context-gated LLM refinement).** When you supply a `context` object on `start`, every utterance produces **two** `transcription` events sharing a `sentence_id`: 1. **First emit** — `is_final: true, speech_final: false` — fast dict-corrected text. Use for low-latency UI paint and barge-in. 2. **Canonical** — `is_final: true, speech_final: true` — definitive LLM-refined answer. Always arrives. When `context` is **omitted**, every utterance produces a single `transcription` event with `is_final: true, speech_final: true` (no first emit). Simple consumers can gate exclusively on `speech_final: true` and ignore the rest — that gives them exactly one canonical event per utterance regardless of whether refinement is on. ## Client → Server Messages ### `start` — Begin session Sent once after connection is established. Must be sent before any audio. <RequestExample> ```json Request theme={null} { "type": "start", "languages": ["en", "hi"], "context": { "general": [ { "key": "domain", "value": "Healthcare" }, { "key": "doctor", "value": "Dr. Martha Smith" } ], "text": "Routine diabetes follow-up consultation.", "terms": ["Celebrex", "Zyrtec", "Metformin", "HbA1c"] }, "config": { "encoding": "mulaw", "sample_rate": 8000, "utterance_end_ms": 1000, "continuous_mode": true, "interim_results_frequency": 300, "audio_enhancement": "adaptive", "diarize": false, "remove_fillers": false } } ``` </RequestExample> **Parameters:** <ParamField name="languages" type="array"> Array of ISO 639-1 codes from the supported set (see `GET /stt/languages`), e.g. `["en", "hi"]`. Max 5 per session. Omit or send `null` to auto-detect across all 39 v1 languages. Arabic dialect tags (`ar-eg`, …) are rejected. Unsupported: `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`. **Important:** never send `languages: "auto"` or `["auto"]`. The auto entry in `GET /stt/languages` is a convenience for the REST `/stt` form-upload flow; on WebSocket you must use `null` instead. Sending the literal string `"auto"` returns `language 'auto' is not in the v1 supported`. The 60db `/ws/stt` proxy strips it automatically as a safety net, but your client should send `null` directly. </ParamField> <ParamField name="context" type="object"> Optional hint object `{general, text, terms}` that opens the server-side LLM refinement gate. When supplied, each utterance emits **two** `transcription` events sharing a `sentence_id` — a fast first emit (`speech_final: false`) with dict-corrected text, followed \~300–700 ms later by a canonical emit (`speech_final: true`) with LLM-refined text (proper nouns corrected, fillers removed, punctuation added, script consistency enforced). See [Canonical-answer semantics](#canonical-answer-semantics-speech-final) below. * `general` — array of `{key, value}` pairs. Free-form metadata (`domain`, `topic`, speaker names) surfaced to the LLM verbatim as hint lines. * `text` — background paragraph describing the session. Useful for narrative context. * `terms` — array of proper nouns, acronyms, and domain-specific jargon to preserve in the transcript. All three fields are optional; at least one should be populated. Omit `context` entirely to disable refinement for the session — each utterance then arrives as a single `speech_final: true` event (no first emit). </ParamField> <ParamField name="config.encoding" type="string"> Audio encoding format. Use `"mulaw"` for telephony/Twilio, `"linear"` (Int16 PCM) for browser capture. Options: `"mulaw"`, `"linear"`. The first raw binary frame auto-selects `mulaw`. </ParamField> <ParamField name="config.sample_rate" type="integer"> Actual sample rate of the audio being sent — 8000 for telephony, 48000 for typical browser capture. Server resamples to 16 kHz internally. Options: `8000`, `16000`, `24000`, `44100`, `48000`. Must match the real capture rate; a mismatch produces garbled audio with no error. </ParamField> <ParamField name="config.utterance_end_ms" type="integer"> Silence duration (ms) after last speech chunk before finalizing the utterance. **Minimum 1000 ms** — the upstream server rejects anything lower, so the 60db proxy clamps sub-1000 values up to 1000 ms (on `start` and on mid-session `config`) rather than letting the session die. Sending `500` is accepted but behaves as `1000`. Recommended 1000–1500 ms for voicebots; for barge-in, react to `speech_started` and interim results instead of lowering this. Range: `≥ 1000` </ParamField> <ParamField name="config.continuous_mode" type="boolean"> Keep session alive between utterances instead of stopping after the first transcription. Required for voicebot / phone call use cases. Send `false` for single-shot transcription. </ParamField> <ParamField name="config.interim_results_frequency" type="integer"> How often (ms) to emit interim (partial) transcription results during speech. Use 300ms for barge-in, 500ms otherwise. Disabled by default. Range: `≥ 300` </ParamField> <ParamField name="config.diarize" type="boolean"> Run pyannote speaker diarization on each finalized utterance and attach a `speakers` array. Requires `HF_TOKEN` on the server. Adds \~50–150 ms latency per final. </ParamField> <ParamField name="config.min_speakers" type="integer | null"> Lower bound on diarization speaker count. Only read when `diarize=true`. </ParamField> <ParamField name="config.max_speakers" type="integer | null"> Upper bound on diarization speaker count. Only read when `diarize=true`. </ParamField> <ParamField name="config.audio_enhancement" type="string"> Real-time audio enhancement to improve transcription quality on noisy input. There are **3 options** available: | Value | Description | | ------------ | -------------------------------------------------------------------------------------------------- | | `"off"` | No audio processing (default) | | `"light"` | Noise reduction only — best for mildly noisy environments | | `"adaptive"` | Automatic noise reduction + gain control based on input levels — best for variable/telephony audio | Use `"adaptive"` for telephony or noisy environments, `"light"` when you only need mild cleanup, and `"off"` when the input is already clean. </ParamField> <ParamField name="config.remove_fillers" type="boolean"> Ask the LLM refinement pass to strip filler words (`um`, `uh`, `like`, `you know`, …) from the canonical transcript. Only takes effect when `context` is set — refinement is gated on context, and the raw first-emit still contains the fillers. Non-boolean values are coerced to `false` by the 60db proxy. </ParamField> <ParamField name="config.no_speech_threshold" type="float"> Reserved for legacy client compatibility. **Ignored by the 60db STT backend** — the non-hallucinating backends don't emit a `no_speech_prob`. </ParamField> ### `audio` — JSON audio chunk (browser mode) <RequestExample> ```json Request theme={null} { "type": "audio", "audio": "<base64-encoded Int16 PCM or μ-law bytes>", "encoding": "linear", "sample_rate": 48000, "timestamp": 1700000000000 } ``` </RequestExample> **Fields:** <ParamField name="type" type="string"> Must be `"audio"` </ParamField> <ParamField name="audio" type="string"> Base64-encoded audio bytes (Int16 PCM or μ-law) </ParamField> <ParamField name="encoding" type="string"> `"linear"` or `"mulaw"` </ParamField> <ParamField name="sample_rate" type="integer"> Actual sample rate of the audio </ParamField> <ParamField name="timestamp" type="integer"> Unix ms timestamp — useful for latency measurement </ParamField> ### Binary frame — raw μ-law audio (telephony mode) Send a raw WebSocket binary frame with μ-law bytes, no JSON wrapper. The server auto-detects this as telephony mode on the first binary frame. ``` Recommended chunk size: 480 bytes = 60ms at 8kHz Twilio default: 160 bytes = 20ms — batch 3 chunks into 60ms before sending ``` ### `config` — Change language mid-session <RequestExample> ```json Request theme={null} { "type": "config", "languages": ["hi"], "continuous_mode": true } ``` </RequestExample> Both `languages` and `continuous_mode` are optional; include only fields you want to change. Send `"languages": null` to revert to auto-detect. ### `stop` — End session <RequestExample> ```json Request theme={null} { "type": "stop" } ``` </RequestExample> Server processes any remaining audio buffer, sends `session_stopped`, then closes. ### `test` — Ping / latency check <RequestExample> ```json Request theme={null} { "type": "test", "message": "ping", "timestamp": 1700000000000 } ``` </RequestExample> Server echoes `test_response` with the same `timestamp` for round-trip measurement. ## Server → Client Messages ### `connecting` — Authentication in progress <ResponseExample> ```json Response theme={null} { "type": "connecting", "message": "Authenticating...", "timestamp": 1775465918269 } ``` </ResponseExample> ### `connection_established` — Authentication successful <ResponseExample> ```json Response theme={null} { "type": "connection_established", "service": "stt", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } ``` </ResponseExample> <Warning> On STT these fields are **top-level** and the frame is tagged by `type`. Check `msg.type === "connection_established"` — `msg.connection_established` is the TTS frame's shape, and a client written against it will never start its STT session. </Warning> **Fields:** <ResponseField name="service" type="string"> Service name: `"stt"` </ResponseField> <ResponseField name="user_id" type="integer"> Your user ID </ResponseField> <ResponseField name="credit_balance" type="number"> Available credits </ResponseField> <ResponseField name="workspace" type="string"> Workspace name </ResponseField> ### `connected` — After `start` message is processed <ResponseExample> ```json Response theme={null} { "type": "connected", "server_info": { "server_type": "60db STT", "device": "cuda", "model": "60db-stt-v01", "processing_mode": "sentence_based_modular", "supported_languages": { "en": "English", "hi": "Hindi" }, "total_languages": 40, "features": { "sentence_based_processing": true, "real_time_streaming": true, "telephony_support": true, "unicode_support": true, "mixed_language_support": true } } } ``` </ResponseExample> <Note> Expect **two** `connected` frames: the proxy's own handshake (`session_id` plus a small `server_info`), then the upstream capability frame shown above. Neither means "audio is accepted" — wait for `session_started`. Keep any `connected` handler idempotent so audio capture does not start twice. </Note> ### `session_started` — `start` accepted, audio is now allowed <ResponseExample> ```json Response theme={null} { "type": "session_started", "session_id": "sess_8c3d1a9f4b7e2c51", "language": "Multi-language: EN, HI", "languages": ["en", "hi"], "model": "60db-stt-v01", "processing_mode": "sentence_based_continuous", "continuous_mode": true, "interim_frequency": 300, "diarize": false, "llm_refinement": true } ``` </ResponseExample> Send audio only after this event — earlier frames are answered with `unknown message type: audio`. `llm_refinement: true` confirms your `context` opened the refinement gate, so every utterance will arrive as a two-phase pair; `false` means one canonical event per utterance. `languages` is the resolved candidate list after normalization. ### `speech_started` — VAD detected voice activity <ResponseExample> ```json Response theme={null} { "type": "speech_started", "timestamp": 1700000000.123 } ``` </ResponseExample> Use this for barge-in: interrupt TTS playback when this arrives. Fired after 2 consecutive VAD-positive chunks (\~64ms of confirmed speech). ### `transcription` — Transcription result All results (interim and final) share the same `transcription` type — differentiate with flags. **Final result** (`is_final=true`, `speech_final=true`): <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "Hello, how are you?", "confidence": 0.87, "language": "en", "language_name": "EN", "is_final": true, "speech_final": true, "is_partial": false, "sentence_id": 3, "processing_mode": "sentence_complete", "duration": 1.82, "latency": 0.43, "timestamp": 1700000000.456, "words": [ { "word": "Hello", "start": 0.0, "end": 0.32, "confidence": 0.94 }, { "word": "how", "start": 0.35, "end": 0.52, "confidence": 0.92 } ], "utterance_end_ms": 1820 } ``` </ResponseExample> **Empty speech\_final signal** (`text=""`, `is_final=true`, `speech_final=true`): Sent when audio was detected but transcription was rejected (silence, hallucination, low confidence, wrong language). Client should reset its state on this message and not treat it as an error. <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "", "confidence": 0.0, "is_final": true, "speech_final": true, "processing_mode": "speech_end_no_result", "timestamp": 1700000000.789 } ``` </ResponseExample> **Interim result** (`is_final=false`, `speech_final=false`) — only sent when `interim_results_frequency` is set: <ResponseExample> ```json Response theme={null} { "type": "transcription", "text": "Hello how", "confidence": 0.72, "language": "en", "is_final": false, "speech_final": false, "is_partial": true } ``` </ResponseExample> Use interims only for barge-in word-count checks. Never send interim text to the LLM — a final with `is_final=true, speech_final=true` will follow. **Response Fields:** <ResponseField name="text" type="string"> Transcribed text. Empty string = speech-end-no-result signal. </ResponseField> <ResponseField name="confidence" type="number"> 0.0–1.0. Telephony typically 0.35–0.75; browser 0.55–0.95. </ResponseField> <ResponseField name="language" type="string"> Detected language code e.g. `"en"`. </ResponseField> <ResponseField name="language_name" type="string"> Uppercase language code e.g. `"EN"`. </ResponseField> <ResponseField name="is_final" type="boolean"> `true` = end of speech reached. May still be followed by a canonical upgrade if LLM refinement is active. </ResponseField> <ResponseField name="speech_final" type="boolean"> `true` = canonical answer, will not be revised. When LLM refinement is on, one `is_final: true, speech_final: false` event is followed by one `is_final: true, speech_final: true`. When refinement is off, every final is `speech_final: true`. See [Canonical-answer semantics](#canonical-answer-semantics-speech-final). </ResponseField> <ResponseField name="is_partial" type="boolean"> `true` for interim results only. </ResponseField> <ResponseField name="sentence_id" type="integer"> Monotonically increasing counter per session. </ResponseField> <ResponseField name="duration" type="number"> Duration (seconds) of the audio segment transcribed. </ResponseField> <ResponseField name="latency" type="number"> Seconds from processing start to result ready (excludes queue time). </ResponseField> <ResponseField name="words" type="array"> Word-level timestamps `[{word, start, end, confidence}]`. Note: the field is **`confidence`**, not `probability`. Present on finals; empty on interims. </ResponseField> <ResponseField name="utterance_end_ms" type="integer"> Timestamp (ms) of last word in the utterance. </ResponseField> <ResponseField name="processing_mode" type="string"> Marker for utterances the consumer should skip. Omitted on ordinary finals. **None of these are billed.** | Value | Meaning | | ------------------------ | ------------------------------------------------------------------------- | | `speech_end_no_result` | Speech ended but the recognizer produced no text. | | `speech_end_too_short` | Utterance below the minimum duration to recognize. | | `hallucination_rejected` | Word-rate guard rejected the output as a likely hallucination. | | `low_snr_dropped` | Audio dropped before language ID / ASR — signal-to-noise below the floor. | </ResponseField> <ResponseField name="llm_applied" type="boolean"> On canonical emits where refinement ran: `true` = `text` is the LLM-refined version, `false` = the LLM was skipped or failed and the fast first-emit text was promoted unchanged. `llm_reason` carries the why, `llm_latency_ms` the round-trip time. The canonical always arrives either way. </ResponseField> <ResponseField name="tentative" type="boolean"> Added by the 60db proxy when the upstream rejected the utterance as a suspected hallucination but interim text was available — the proxy sends that text instead of an empty final, with `tentative_reason: "hallucination_rejected"`. Safe to route on; flag it for review. </ResponseField> ### Canonical-answer semantics: `speech_final` `is_final` and `speech_final` are **NOT identical** when LLM refinement is active — they split into two distinct meanings: | `is_final` | `speech_final` | Meaning | | ---------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `false` | `false` | Interim partial — text may still change as more audio arrives. | | `true` | `false` | End-of-speech reached, dict-corrected text. **The LLM is still processing.** A follow-up event with `speech_final: true` will arrive shortly with the canonical text. Only emitted when refinement is active for this utterance. | | `true` | `true` | **Canonical answer.** Definitive, will not be revised. LLM-refined text when context was supplied, otherwise the original ASR text. | The same `sentence_id` is echoed across both phases so clients can reconcile. **Canonical event example (after LLM refinement):** <ResponseExample> ```json Response theme={null} { "type": "transcription", "sentence_id": 3, "text": "डॉक्टर साहब, मेरा sugar level बहुत high है। Metformin की dose बढ़ाओ।", "confidence": 0.87, "language": "hi", "language_name": "HI", "is_final": true, "speech_final": true, "is_partial": false, "duration": 1.82, "words": [ { "word": "डॉक्टर", "start": 0.0, "end": 0.32, "confidence": 0.94 } ], "speakers": null, "timestamp": 1700000000.789 } ``` </ResponseExample> **Guarantees:** * **Exactly one canonical event per utterance.** When refinement is on, you get two `transcription` events per utterance (first emit + canonical). When refinement is off, you get one (`speech_final: true`). Never zero, never three. * **Same `sentence_id`** across both phases. Reconcile on that key. * **The canonical always arrives.** Consumers waiting on `speech_final: true` never hang. * **`sentence_id` ordering is preserved per session**, but canonicals are NOT guaranteed to arrive in `sentence_id` order when LLM is on — two utterances finalizing close in time may complete refinement out of order. Key on `sentence_id`, not arrival order. * **`words[]` corresponds to the original ASR output** on both phases — the LLM does not realign tokens. Use `words[]` for word-level timing, `text` for display. **Recommended client patterns:** *Simplest — don't care about the first-emit optimization:* ```js theme={null} function onMessage(msg) { if (msg.type === 'transcription' && msg.speech_final) { // Canonical — render and forget render(msg); } // Ignore is_final && !speech_final (intermediate, will be replaced) // Ignore is_partial (interim, handle separately if needed) } ``` *With fast first-emit (UX-aware):* ```js theme={null} const livePainted = new Map(); // sentence_id → line slot function onMessage(msg) { if (msg.type !== 'transcription') return; const sid = msg.sentence_id; if (msg.is_partial) { renderPartial(msg); return; } let entry = livePainted.get(sid); if (!entry) { entry = createLine(); livePainted.set(sid, entry); } entry.text = msg.text; entry.pending = msg.is_final && !msg.speech_final; // dim while LLM runs render(entry); if (msg.speech_final) { finalize(entry); livePainted.delete(sid); } } ``` **For voicebot NLU routing**: feed the first-emit text (`speech_final: false`) to NLU immediately for fast intent dispatch — don't wait for canonical. If your NLU benefits from proper-noun accuracy (name-spelling slots, drug-name lookup), run a second-pass call on the canonical (`speech_final: true`) text and reconcile on `sentence_id`. <Note> **Legacy `refined` event.** Earlier builds emitted a separate `refined` event \~400 ms after the final instead of a second `transcription`. The 60db `/ws/stt` proxy transparently handles both shapes — if you're still seeing `refined` events in the wire trace, upstream workers haven't been restarted onto the two-phase build yet. New client code should target the two-phase flow only; `refined` is accepted but deprecated. </Note> ### `language_changed` — After `config` message changes language <ResponseExample> ```json Response theme={null} { "type": "language_changed", "language": "Multi-language: HI", "language_code": ["hi"] } ``` </ResponseExample> ### `mode_changed` — After `config` message changes `continuous_mode` <ResponseExample> ```json Response theme={null} { "type": "mode_changed", "continuous_mode": true, "mode_name": "continuous", "silence_threshold": 0.5 } ``` </ResponseExample> ### `session_stopped` — After `stop` is processed <ResponseExample> ```json Response theme={null} { "type": "session_stopped", "billing_summary": { "total_duration_seconds": 5.2, "total_cost": 0.000043, "characters_transcribed": 42, "client_estimated_seconds": 5.24 } } ``` </ResponseExample> Only canonical finals (`is_final: true` **and** `speech_final: true`) with a `duration` are charged — first emits are previews, so turning on LLM refinement does not double your bill, and the four skip modes above cost nothing. `client_estimated_seconds` is a diagnostic estimate of what the client sent; never show it as a billed figure. ### `error` — Processing error <ResponseExample> ```json Response theme={null} { "type": "error", "error": "Audio processing error: ...", "timestamp": 1700000000.0 } ``` </ResponseExample> ### `test_response` — Reply to `test` ping <ResponseExample> ```json Response theme={null} { "type": "test_response", "message": "pong - qlabs-stt-proxy ready", "timestamp": 1700000000000, "processing_mode": "proxy" } ``` Answered by the 60db proxy itself — it never reaches the STT server, so a reply confirms the proxy hop only, and the echoed `timestamp` measures round-trip to the proxy. </ResponseExample> ## Complete Example <Tabs> <TabItem label="JavaScript (Node.js)"> ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/stt?apiKey=${API_KEY}`); ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (data) => { const msg = JSON.parse(data); console.log('←', msg.type || Object.keys(msg)[0]); if (msg.type === 'connection_established') { console.log(' User ID:', msg.user_id); console.log(' Credits:', msg.credit_balance); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en', 'hi'], config: { encoding: 'mulaw', sample_rate: 8000, continuous_mode: true, utterance_end_ms: 1000, interim_results_frequency: 300 } })); } else if (msg.type === 'session_started') { console.log('✓ Session started! Send audio now...'); // Send audio chunks (480 bytes = ~60ms at 8kHz) const audioInterval = setInterval(() => { const audioChunk = getAudioChunk(); ws.send(audioChunk); }, 60); // Stop after 5 seconds setTimeout(() => { clearInterval(audioInterval); ws.send(JSON.stringify({ type: 'stop' })); }, 5000); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected - barge-in opportunity'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓ Final:', msg.text, `(confidence: ${msg.confidence})`); } else { console.log(' Partial:', msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); console.log(' Duration:', msg.billing_summary.total_duration_seconds, 's'); console.log(' Cost: $', msg.billing_summary.total_cost); ws.close(); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` </TabItem> <TabItem label="Python"> ```python theme={null} import asyncio import json import websockets async def stt_websocket(): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/stt?apiKey={API_KEY}" async with websockets.connect(url) as ws: # Wait for authentication: `connecting`, then `connected`, then this while True: msg = json.loads(await ws.recv()) if msg.get("type") == "connection_established": print(f"✓ Connected (User: {msg['user_id']})") print(f" Credits: ${msg['credit_balance']}") break # Start session await ws.send(json.dumps({ "type": "start", "languages": ["en", "hi"], "config": { "encoding": "mulaw", "sample_rate": 8000, "continuous_mode": True, "utterance_end_ms": 1000, "interim_results_frequency": 300 } })) # Audio is accepted only once the session is started while True: msg = json.loads(await ws.recv()) if msg.get("type") == "session_started": break print("✓ Session started!") # Send audio for 5 seconds for _ in range(83): # ~5000ms / 60ms audio_chunk = get_audio_chunk() await ws.send(audio_chunk) # Send as binary await asyncio.sleep(0.06) # Stop session await ws.send(json.dumps({"type": "stop"})) # Process remaining messages while True: msg = json.loads(await ws.recv()) msg_type = msg.get("type") if msg_type == "speech_started": print("🎤 Speech detected") elif msg_type == "transcription": if msg.get("is_final"): print(f"✓ {msg['text']} (confidence: {msg['confidence']})") else: print(f" {msg['text']}...") elif msg_type == "session_stopped": print(f"✓ Session stopped") print(f" Duration: {msg['billing_summary']['total_duration_seconds']}s") print(f" Cost: ${msg['billing_summary']['total_cost']}") break asyncio.run(stt_websocket()) ``` </TabItem> <TabItem label="Browser"> ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key'); let mediaRecorder; let audioContext; ws.onopen = () => { console.log('✓ Connected'); }; ws.onmessage = (event) => { const msg = JSON.parse(event.data); if (msg.type === 'connection_established') { console.log('✓ Authenticated!'); // Start session ws.send(JSON.stringify({ type: 'start', languages: ['en'], config: { encoding: 'linear', sample_rate: 48000, continuous_mode: true, interim_results_frequency: 300 } })); } else if (msg.type === 'session_started') { console.log('✓ Session started! Start speaking...'); startAudioCapture(); } else if (msg.type === 'speech_started') { console.log('🎤 Speech detected'); } else if (msg.type === 'transcription') { if (msg.is_final) { console.log('✓', msg.text); updateTranscriptDisplay(msg.text); } else { console.log('...', msg.text); updateInterimDisplay(msg.text); } } else if (msg.type === 'session_stopped') { console.log('✓ Session stopped'); stopAudioCapture(); ws.close(); } }; async function startAudioCapture() { const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); audioContext = new AudioContext({ sampleRate: 48000 }); const source = audioContext.createMediaStreamSource(stream); const processor = audioContext.createScriptProcessor(4096, 1, 1); processor.onaudioprocess = (e) => { const inputData = e.inputBuffer.getChannelData(0); const pcm16 = new Int16Array(inputData.length); for (let i = 0; i < inputData.length; i++) { pcm16[i] = Math.max(-32768, Math.min(32767, inputData[i] * 32768)); } ws.send(JSON.stringify({ type: 'audio', audio: btoa(String.fromCharCode(...new Uint8Array(pcm16.buffer))), encoding: 'linear', sample_rate: 48000 })); }; source.connect(processor); processor.connect(audioContext.destination); mediaRecorder = { stream, processor, source }; } function stopAudioCapture() { if (mediaRecorder) { mediaRecorder.stream.getTracks().forEach(track => track.stop()); mediaRecorder.processor.disconnect(); audioContext.close(); } } ``` </TabItem> </Tabs> ## Audio Requirements | Property | Telephony (μ-law) | Browser (PCM) | | ----------- | ----------------- | ----------------------------- | | Encoding | `mulaw` (8-bit) | `linear` (16-bit) | | Sample Rate | 8000 Hz | 16000, 24000, 44100, 48000 Hz | | Chunk Size | 480 bytes (60ms) | 960-1920 bytes (60-120ms) | | Channels | Mono (1 channel) | Mono (1 channel) | ## Supported Languages 39 languages total, backed by Parakeet-TDT (25 European), Vaani-FastConformer (13 Indic + Hinglish), and FC-Arabic (MSA). Fetch the full catalog from `GET /stt/languages`. **Not supported** (explicit rejection, no silent aliasing): `ur`, `ja`, `ko`, `zh`, `th`, `vi`, `id`, `tl`, `sw`, `tr`, `fa`, `he`. Arabic dialect tags (`ar-eg`, `ar-lv`, …) return `dialect_not_supported` — pass `ar` for best-effort MSA. Common languages: | Code | Language | Code | Language | | ---- | --------- | ---- | -------- | | `en` | English | `hi` | Hindi | | `bn` | Bengali | `es` | Spanish | | `fr` | French | `de` | German | | `gu` | Gujarati | `ta` | Tamil | | `te` | Telugu | `kn` | Kannada | | `ml` | Malayalam | `mr` | Marathi | | `pa` | Punjabi | `ar` | Arabic | ## Pricing * **Rate**: \$0.00000833 per second * **Minimum**: \$0.01 per session * **Billing**: Per second of audio processed ## Related * [TTS WebSocket](/websocket-api/tts) - Text-to-Speech endpoint * [WebSocket Quick Start](/websocket-api/quickstart) - Get started guide * [WebSocket Playground](/websocket-playground) - Test in browser # TTS WebSocket Source: https://docs.60db.ai/websocket-api/tts WebSocket /ws/tts Real-time Text-to-Speech WebSocket API for streaming audio synthesis # TTS WebSocket API Real-time Text-to-Speech synthesis via WebSocket streaming with full-duplex bidirectional communication. ## 🚀 Quick Start (Copy & Paste) ```javascript theme={null} const WebSocket = require('ws'); const fs = require('fs'); // 1. Your API key const API_KEY = 'sk_live_your_api_key'; // 2. Connect const ws = new WebSocket(`wss://api.60db.ai/ws/tts?apiKey=${API_KEY}`); // 3. Unique context ID const contextId = 'my-session-' + Date.now(); const audioChunks = []; // 4. Handle messages ws.on('message', (data) => { const msg = JSON.parse(data); // Authenticated? Create context! if (msg.connection_established) { console.log('✅ Authenticated'); ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 } } })); } // Context ready? Send text! if (msg.context_created) { console.log('✅ Context created'); // Send your text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, world!' } })); // Flush to get audio ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } // Got audio? Save it! if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log('🔊 Audio chunk:', audioData.length, 'bytes'); } // All audio received? if (msg.flush_completed) { console.log('✅ All audio received!'); console.log(' Total:', audioChunks.length, 'chunks'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } // Done! Save audio if (msg.context_closed) { console.log('✅ Complete!'); // Save to file const completeAudio = Buffer.concat(audioChunks); fs.writeFileSync('output.pcm', completeAudio); console.log('💾 Saved: output.pcm'); console.log(' Size:', completeAudio.length, 'bytes'); ws.close(); } }); ``` **That's it!** You'll see: * ✅ Authenticated * ✅ Context created * 🔊 Audio chunk: 1024 bytes (multiple times) * ✅ All audio received! * ✅ Complete! * 💾 Saved: output.pcm *** ## 📖 How It Works (5 Simple Steps) 1. **Connect** with your API key 2. **Create** a context with voice settings 3. **Send** your text message 4. **Flush** to trigger synthesis 5. **Close** when done (receive audio file) *** ## Endpoint <ParamField name="url" type="string"> `ws://api.60db.ai/ws/tts` </ParamField> ## Authentication Query parameter authentication: <ParamField name="apiKey" type="string"> Your API key for authentication </ParamField> Example: ``` ws://api.60db.ai/ws/tts?apiKey=sk_live_your_api_key ``` ## Protocol Overview ``` Client Server | | |─── create_context ──────────────────▶ | |◀── context_created ───────────────── | | | |─── send_text ───────────────────────▶ | |─── flush_context ───────────────────▶ | |◀── audio_chunk #1 ────────────────── | |◀── audio_chunk #N ────────────────── | |◀── flush_completed ───────────────── | | | |─── close_context ───────────────────▶ | |◀── context_closed ────────────────── | ``` ## Connection Sequence ### 1. Connect ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); ``` ### 2. Receive Authentication Message <ResponseExample> ```json Response theme={null} { "connecting": true, "message": "Authenticating...", "timestamp": 1775465918269 } ``` </ResponseExample> ### 3. Receive Connection Established <ResponseExample> ```json Response theme={null} { "connection_established": { "service": "tts", "user_id": 43, "credit_balance": 9.97, "workspace": "default" } } ``` </ResponseExample> **Fields:** <ResponseField name="service" type="string"> Service name: `"tts"` </ResponseField> <ResponseField name="user_id" type="integer"> Your user ID </ResponseField> <ResponseField name="credit_balance" type="number"> Available credits </ResponseField> <ResponseField name="workspace" type="string"> Workspace name </ResponseField> ## Client → Server Messages ### 1. create\_context **Must be the first message.** Initializes the TTS session with voice and audio settings. <RequestExample> ```json Request theme={null} { "create_context": { "context_id": "my-session-123", "voice_id": "7911a3e8", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } } ``` </RequestExample> **Parameters:** <ParamField name="context_id" type="string"> Unique session identifier. Default: auto-generated UUID </ParamField> <ParamField name="voice_id" type="string"> Voice ID to use for synthesis </ParamField> <ParamField name="audio_config.audio_encoding" type="string"> Audio encoding. Options: `LINEAR16`, `PCM`, `MULAW`, `ULAW`, `OGG_OPUS` </ParamField> <ParamField name="audio_config.sample_rate_hertz" type="integer"> Sample rate in Hz. Options: `8000`, `16000`, `24000`, `48000` </ParamField> <ParamField name="speed" type="number"> Speech speed multiplier (0.5 – 2.0). </ParamField> <ParamField name="stability" type="number"> Voice stability (0-100). Lower = more expressive, higher = more consistent. </ParamField> <ParamField name="similarity" type="number"> Voice similarity (0-100). How closely the output matches the source voice. </ParamField> **Supported encoding + sample rate combinations:** | `audio_encoding` | Supported `sample_rate_hertz` | Output format | | ---------------- | ------------------------------------------- | ------------------------------------------ | | `LINEAR16` | `8000`, `16000` (default), `24000`, `48000` | Raw PCM, 16-bit signed little-endian, mono | | `PCM` | `8000`, `16000` (default), `24000`, `48000` | Same as LINEAR16 | | `MULAW` | `8000` | G.711 μ-law encoded, mono | | `ULAW` | `8000` | Same as MULAW | | `OGG_OPUS` | `24000` | Ogg Opus compressed audio | > **Note:** `MULAW`/`ULAW` only works at `8000` Hz. `OGG_OPUS` only works at `24000` Hz. **Limits:** | Parameter | Min | Max | Default | | ------------------------- | ------ | ------------ | ------- | | `speed` | 0.5 | 2.0 | 1 | | `stability` | 0 | 100 | 50 | | `similarity` | 0 | 100 | 75 | | `text` (per send\_text) | 1 char | — | — | | text buffer (accumulated) | — | 50,000 chars | — | ### 2. send\_text Append text to the internal buffer. Text is accumulated until a `flush_context` or `close_context` is received. <RequestExample> ```json Request theme={null} { "send_text": { "context_id": "my-session-123", "text": "Hello, how are you doing today?" } } ``` </RequestExample> **Fields:** <ParamField name="context_id" type="string"> Session identifier </ParamField> <ParamField name="text" type="string"> Text to append to buffer (max cumulative 50,000 characters) </ParamField> You can send multiple `send_text` messages to build up text incrementally (e.g., from an LLM token stream): ```json theme={null} {"send_text": {"context_id": "ctx-1", "text": "Hello, "}} {"send_text": {"context_id": "ctx-1", "text": "how are you "}} {"send_text": {"context_id": "ctx-1", "text": "doing today?"}} ``` ### 3. flush\_context Triggers synthesis of all accumulated text. The server responds with `audio_chunk` messages followed by `flush_completed`. <RequestExample> ```json Request theme={null} { "flush_context": { "context_id": "my-session-123" } } ``` </RequestExample> ### 4. close\_context Flushes any remaining text, sends final audio, and closes the WebSocket connection. <RequestExample> ```json Request theme={null} { "close_context": { "context_id": "my-session-123" } } ``` </RequestExample> ## Server → Client Messages ### context\_created Confirms the session was initialized successfully. <ResponseExample> ```json Response theme={null} { "context_created": { "context_id": "my-session-123" } } ``` </ResponseExample> ### audio\_chunk Contains a chunk of synthesized audio. Multiple chunks are sent per flush. <ResponseExample> ```json Response theme={null} { "audio_chunk": { "context_id": "my-session-123", "audioContent": "SGVsbG8gd29ybGQ..." } } ``` </ResponseExample> **Fields:** <ResponseField name="context_id" type="string"> Session identifier </ResponseField> <ResponseField name="audioContent" type="string"> Base64-encoded audio bytes </ResponseField> The audio encoding and chunk format depend on `audio_config`: | Encoding | Chunk format | Notes | | ------------------ | ------------------------------- | ---------------------------------------------------------------------------------- | | `LINEAR16` / `PCM` | Raw PCM, 16-bit signed LE, mono | Chunks can be concatenated directly | | `MULAW` / `ULAW` | G.711 μ-law, 8-bit, mono | Chunks can be concatenated directly | | `OGG_OPUS` | Independent Ogg Opus files | Each chunk is a self-contained OGG file. **Chunks cannot be naively concatenated** | ### flush\_completed Signals that all audio for the flushed text has been sent. <ResponseExample> ```json Response theme={null} { "flush_completed": { "context_id": "my-session-123" } } ``` </ResponseExample> ### context\_closed Confirms the session is closed. The WebSocket connection closes after this message. <ResponseExample> ```json Response theme={null} { "context_closed": { "context_id": "my-session-123" } } ``` </ResponseExample> ### error Sent if synthesis fails or a protocol violation occurs. <ResponseExample> ```json Response theme={null} { "error": { "context_id": "my-session-123", "message": "voice_id required" } } ``` </ResponseExample> **Common errors:** | Message | Cause | | -------------------------------------------- | ------------------------------------------ | | `voice_id required` | `create_context` sent without `voice_id` | | `text_buffer exceeded 50000 character limit` | Too much text accumulated without flushing | | `Unsupported audio_encoding: X` | Invalid encoding value | | `Unsupported sample_rate_hertz: X` | Invalid sample rate | ## Complete Example <Tabs> <TabItem label="JavaScript (Node.js)"> ```javascript theme={null} const WebSocket = require('ws'); const API_KEY = 'sk_live_your_key'; const ws = new WebSocket(`ws://api.60db.ai/ws/tts?apiKey=${API_KEY}`); const contextId = 'test-' + Date.now(); const audioChunks = []; ws.on('open', () => { console.log('✓ Connected'); }); ws.on('message', (data) => { const msg = JSON.parse(data); const msgType = Object.keys(msg)[0]; console.log('←', msgType); if (msg.connection_established) { console.log(' Credits:', msg.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (msg.context_created) { console.log('✓ Context created!'); // Send text ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you doing today?' } })); // Flush ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (msg.audio_chunk) { const audioData = Buffer.from(msg.audio_chunk.audioContent, 'base64'); audioChunks.push(audioData); console.log(' Received audio chunk:', audioData.length, 'bytes'); } else if (msg.flush_completed) { console.log('✓ Flush completed!'); console.log(' Total audio size:', audioChunks.reduce((sum, chunk) => sum + chunk.length, 0), 'bytes'); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } else if (msg.context_closed) { console.log('✓ Context closed'); // Save audio const audio = Buffer.concat(audioChunks); require('fs').writeFileSync('output.pcm', audio); console.log('Saved output.pcm'); ws.close(); } if (msg.error) { console.error('TTS Error:', msg.error.message); } }); ws.on('error', (error) => { console.error('Error:', error); }); ws.on('close', () => { console.log('Connection closed'); }); ``` </TabItem> <TabItem label="Python"> ```python theme={null} import asyncio import json import base64 import websockets async def tts_websocket(): API_KEY = "sk_live_your_key" url = f"ws://api.60db.ai/ws/tts?apiKey={API_KEY}" context_id = f"session-{int(asyncio.get_event_loop().time())}" async with websockets.connect(url) as ws: # Wait for connection resp = json.loads(await ws.recv()) if resp.get('connection_established'): print(f"✓ Connected (Credits: ${resp['connection_established']['credit_balance']})") # Create context await ws.send(json.dumps({ "create_context": { "context_id": context_id, "voice_id": "fbb75ed2-975a-40c7-9e06-38e30524a9a1", "audio_config": { "audio_encoding": "LINEAR16", "sample_rate_hertz": 16000 }, "speed": 1, "stability": 50, "similarity": 75 } })) # Wait for context_created resp = json.loads(await ws.recv()) assert "context_created" in resp print("✓ Context created!") # Send text + flush await ws.send(json.dumps({ "send_text": { "context_id": context_id, "text": "Hello, how are you doing today?" } })) await ws.send(json.dumps({ "flush_context": {"context_id": context_id} })) # Receive audio chunks until flush_completed audio_data = b"" chunk_count = 0 while True: msg = json.loads(await ws.recv()) if "audio_chunk" in msg: audio_data += base64.b64decode(msg["audio_chunk"]["audioContent"]) chunk_count += 1 print(f" Received chunk #{chunk_count}") elif "flush_completed" in msg: print("✓ Flush completed!") break elif "error" in msg: print(f"Error: {msg['error']['message']}") break # Close context await ws.send(json.dumps({ "close_context": {"context_id": context_id} })) resp = json.loads(await ws.recv()) assert "context_closed" in resp # Save audio with open("output.pcm", "wb") as f: f.write(audio_data) print(f"✓ Saved {len(audio_data)} bytes to output.pcm") asyncio.run(tts_websocket()) ``` </TabItem> <TabItem label="Browser"> ```javascript theme={null} const ws = new WebSocket('ws://api.60db.ai/ws/tts?apiKey=sk_live_your_key'); const contextId = 'test-' + Date.now(); const audioChunks = []; let audioCtx; ws.onopen = () => { console.log('✓ Connected'); }; ws.onmessage = (event) => { const data = JSON.parse(event.data); const msgType = Object.keys(data)[0]; console.log('←', msgType); if (data.connection_established) { console.log(' Credits:', data.connection_established.credit_balance); // Create context ws.send(JSON.stringify({ create_context: { context_id: contextId, voice_id: 'fbb75ed2-975a-40c7-9e06-38e30524a9a1', audio_config: { audio_encoding: 'LINEAR16', sample_rate_hertz: 16000 }, speed: 1, stability: 50, similarity: 75 } })); } else if (data.context_created) { console.log('✓ Context created!'); // Send text + flush ws.send(JSON.stringify({ send_text: { context_id: contextId, text: 'Hello, how are you?' } })); ws.send(JSON.stringify({ flush_context: { context_id: contextId } })); } else if (data.audio_chunk) { // Collect base64 audio const binary = atob(data.audio_chunk.audioContent); const bytes = new Uint8Array(binary.length); for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); audioChunks.push(bytes); console.log(' Received audio chunk'); } else if (data.flush_completed) { console.log('✓ Flush completed!'); // Play collected audio playPCM16(audioChunks, 16000); // Close context ws.send(JSON.stringify({ close_context: { context_id: contextId } })); } if (data.error) { console.error('TTS Error:', data.error.message); } if (data.context_closed) { console.log('✓ Context closed'); } }; function playPCM16(chunks, sampleRate) { const totalBytes = chunks.reduce((s, c) => s + c.length, 0); const merged = new Uint8Array(totalBytes); let offset = 0; for (const chunk of chunks) { merged.set(chunk, offset); offset += chunk.length; } const int16 = new Int16Array(merged.buffer); const float32 = new Float32Array(int16.length); for (let i = 0; i < int16.length; i++) float32[i] = int16[i] / 32768; audioCtx = audioCtx || new AudioContext({ sampleRate }); const buf = audioCtx.createBuffer(1, float32.length, sampleRate); buf.getChannelData(0).set(float32); const src = audioCtx.createBufferSource(); src.buffer = buf; src.connect(audioCtx.destination); src.start(); } ``` </TabItem> </Tabs> ## Audio Configuration | Encoding | Sample Rates | Description | | ---------- | ------------------------- | ----------------------- | | `LINEAR16` | 8000, 16000, 24000, 48000 | PCM 16-bit signed | | `MULAW` | 8000 | G.711 μ-law (telephony) | | `OGG_OPUS` | 24000 | Compressed audio | For telephony integration (Twilio, etc.), use `MULAW` at `8000` Hz. ## Default Voice The default voice ID is: ``` fbb75ed2-975a-40c7-9e06-38e30524a9a1 ``` To get more voices, use the [Voices API](/api-reference/voices/get-voices). ## Context Management ### Reuse Context Keep a context open for multiple syntheses: ```javascript theme={null} // Create once ws.send(JSON.stringify({ create_context: { context_id, voice_id, audio_config } })); // Send multiple texts ws.send(JSON.stringify({ send_text: { context_id, text: "Hello" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); ws.send(JSON.stringify({ send_text: { context_id, text: "World" } })); ws.send(JSON.stringify({ flush_context: { context_id } })); // Close when done ws.send(JSON.stringify({ close_context: { context_id } })); ``` ### Multiple Contexts You can create multiple contexts in one connection: ```javascript theme={null} const context1 = 'ctx-1'; const context2 = 'ctx-2'; // Create both contexts ws.send(JSON.stringify({ create_context: { context_id: context1, voice_id: voice1, audio_config } })); ws.send(JSON.stringify({ create_context: { context_id: context2, voice_id: voice2, audio_config } })); ``` ## Supported Languages The TTS model supports synthesis in multiple Indic languages and English. The language is auto-detected from the input text. | Language | ID | | --------- | -- | | English | en | | Hindi | hi | | Bengali | bn | | Gujarati | gu | | Kannada | kn | | Malayalam | ml | | Marathi | mr | | Punjabi | pa | | Tamil | ta | | Telugu | te | ## Pricing * **Rate**: \$0.00002 per character * **Minimum**: \$0.01 per context * **Billing**: Per character synthesized ## Related * [STT WebSocket](/websocket-api/stt) - Speech-to-Text endpoint * [WebSocket Quick Start](/websocket-api/quickstart) - Get started guide * [WebSocket Playground](/websocket-playground) - Test in browser * [Voices API](/api-reference/voices/get-voices) - Get available voices