STT WebSocket

Real-time Speech-to-Text transcription via WebSocket streaming with support for 45 languages (including code-switched Indic+English) and telephony integration. Powered by 60db STT v01 (a non-hallucinating, multi-backend speech recognition stack).

Endpoint

Authentication

Query parameter authentication: Examples:

ws://api.60db.ai/ws/stt?apiKey=sk_live_your_api_key
ws://api.60db.ai/ws/stt?token=eyJ...&workspace_id=24

The WebSocket connection checks workspace wallet balance before starting a session. If the workspace has insufficient credits, the connection is closed with a 1008 status code and an INSUFFICIENT_CREDITS error.

Connection Details

Property	Value
Protocol	WebSocket (RFC 6455)
Frame types	Binary (telephony) or Text/JSON (browser)
Ping/keepalive	The server answers client-initiated WebSocket pings with a pong; it does not ping first. Send a ping every 20–30 s so intermediaries do not idle the socket out.

The same port accepts plain HTTP GET requests and responds 200 ok — safe for load-balancer health checks.

Session Lifecycle

Client                                    Server
  │                                         │
  │──── TCP/TLS connect ───────────────────►│
  │◄─── {"type":"connecting", ...} ─────────│  authenticating...
  │◄─── {"type":"connected", server_info} ──│  proxy wired to upstream
  │◄─── {"type":"connection_established"} ──│  ready — safe to send `start`
  │                                         │
  │──── {"type":"start", ...} ─────────────►│
  │◄─── {"type":"session_started", ...} ────│  safe to send audio
  │                                         │
  │──── audio frames / JSON audio ─────────►│
  │◄─── {"type":"speech_started"} ──────────│  VAD detected voice
  │◄─── {"type":"transcription", ...} ──────│  is_final=false (interim, optional, repeats)
  │◄─── {"type":"turn_complete", ...} ──────│  only with turn_detector=true
  │◄─── {"type":"transcription", ...} ──────│  is_final=true (final text for the turn)
  │                                         │
  │──── {"type":"stop"} ────────────────────►│
  │◄─── {"type":"session_stopped"} ─────────│

Single-phase finals. Each turn produces one committed transcription with is_final: true (and speech_final mirrors is_final — they are always equal). Text is never rewritten after recognition, so there is no second “canonical” emit and no refined event. Interims (is_final: false) are optional live guesses; only finals are committed. A single caller turn can still yield more than one final — a pause can split it, and a monologue longer than max_utterance_sec emits one or more segment_final: true chunks before the closing final. Each final covers a non-overlapping slice of audio, so append them.

Do not send the start message until connection_established is received. The backend proxy attaches its client-message listener only after authenticating and opening the upstream connection. Messages sent earlier will be silently dropped. Likewise, do not send audio messages until session_started is received — the upstream returns unknown message type: audio if it arrives before the session is ready.

Client → Server Messages

`start` — Begin session

Sent once after connection is established. Must be sent before any audio.

{
  "type": "start",
  "languages": ["en", "hi"],
  "context": {
    "general": [
      { "key": "domain",  "value": "Healthcare" },
      { "key": "doctor",  "value": "Dr. Martha Smith" }
    ],
    "text":  "Routine diabetes follow-up consultation.",
    "terms": ["Celebrex", "Zyrtec", "Metformin", "HbA1c"]
  },
  "keywords": ["Metformin", "HbA1c"],
  "config": {
    "encoding": "linear",
    "sample_rate": 48000,
    "utterance_end_ms": 1000,
    "early_final": false,
    "turn_detector": true,
    "interim_results_frequency": 500,
    "audio_enhancement": "off",
    "diarize": false
  }
}

Parameters:

Auto-detect: omit languages, or send null, [], "auto" or ["auto"]. The /ws/stt proxy expands all of these to the full language list the STT server advertises (session_started.language_mode is then candidate_auto; a single code gives selected). The STT server itself rejects the literal "auto" and fails real speech on an empty list, so the proxy does the expansion for you.

Turn-ending profiles

early_final and turn_detector together decide how fast a turn closes and whether a sentence with a pause stays in one final. Pick one profile:

Profile	`turn_detector`	`early_final`	Time to final	Sentences
A. Whole sentences (recommended)	`true`	`false`	~2 s (up to ~3 s)	Kept whole
B. Balanced	`false`	`false`	~1.5 s	A pause over 1 s splits
C. Fastest (server default)	`false`	`true`	~1.0 s	A pause over ~300 ms can split

No words are lost with any profile — only where turns are cut and how fast finals arrive changes. session_started echoes the effective values.

`audio` — JSON audio chunk (browser mode)

{
  "type": "audio",
  "audio": "<base64-encoded Int16 PCM or μ-law bytes>",
  "encoding": "linear",
  "sample_rate": 48000,
  "timestamp": 1700000000000
}

Fields:

Binary frame — raw μ-law audio (telephony mode)

Send a raw WebSocket binary frame with μ-law bytes, no JSON wrapper. The server auto-detects this as telephony mode on the first binary frame.

Recommended chunk size: 480 bytes = 60ms at 8kHz
Twilio default: 160 bytes = 20ms — batch 3 chunks into 60ms before sending

`config` — Change settings mid-session

{
  "type": "config",
  "languages": ["hi"]
}

Send only what changes. Each recognised key is acknowledged with its own event:

Field	Acknowledgement
`languages`	`language_changed`
`utterance_end_ms`	`utterance_end_changed`
`turn_detector`	`turn_detector_changed`
`interim_results_frequency` (`null` turns interims off)	`interim_frequency_changed`
`interim_delta_streaming`	`interim_delta_streaming_changed`
`token_streaming`	`token_streaming_changed`
`max_utterance_sec`	`max_utterance_changed`
`audio_enhancement`	`audio_enhancement_changed`
`encoding` / `sample_rate`	`audio_format_changed`
`keywords`	`keywords_changed`
`context`	`context_changed`
`early_final`	none (applies to the next turn)

If any value is invalid you get an error and nothing in that message is applied. Changing languages mid-sentence first finalizes that sentence, then switches — prefer switching between turns. diarize, min_speakers, and max_speakers can only be set in start. Send "languages": null to revert to auto-detect. The proxy applies the same sanitising as on start: "auto" is stripped, and utterance_end_ms is snapped onto the ≥1000 / step-100 grid.

`finalize` — End the current turn now

{
  "type": "finalize"
}

Closes the current turn immediately and returns its transcription (or no_speech), or nothing if no audio is buffered. The session continues. Use it when your application already knows the caller is done — e.g. after a DTMF key.

`stop` — End session

{
  "type": "stop"
}

Server processes any remaining audio buffer, sends session_stopped, then closes.

`test` — Ping / latency check

{
  "type": "test",
  "message": "ping",
  "timestamp": 1700000000000
}

Server echoes test_response with the same timestamp for round-trip measurement.

Server → Client Messages

`connecting` — Authentication in progress

{
  "type": "connecting",
  "message": "Authenticating...",
  "timestamp": 1775465918269
}

`connection_established` — Authentication successful

{
  "type": "connection_established",
  "service": "stt",
  "user_id": 43,
  "credit_balance": 9.97,
  "workspace": "default"
}

On STT these fields are top-level and the frame is identified by type. Branch on msg.type === "connection_established" — not on msg.connection_established, which is the TTS frame’s shape (/ws/tts nests the same fields under a connection_established key). A client written against the TTS shape will silently never start its STT session.

Fields:

string

Service name: "stt"

integer

Your user ID

number

Available credits

string

Workspace name

`connected` — Proxy wired to upstream STT server

Sent by the backend proxy after it has opened the upstream connection and attached its client-message listener. After this point, client messages are no longer dropped.

You will normally see two connected frames, and they carry different payloads. The proxy emits its own handshake frame first (session_id plus a small server_info), then forwards the upstream server’s capability frame (the one below, with features and timing). Both are informational — treat connection_established as “safe to send start” and session_started as “safe to send audio”, and make any connected handler idempotent so it does not start audio capture twice.

{
  "type": "connected",
  "session_id": "0f5a4d7c-9d0f-4a5e-9d0b-1f2c3d4e5f60",
  "server_info": {
    "server_type": "qlabs-stt-proxy"
  }
}

server_info on the proxy frame is diagnostic; do not branch on its contents.

{
  "type": "connected",
  "server_info": {
    "server_type": "60db STT",
    "ready": true,
    "total_languages": 45,
    "features": {
      "vad_segmentation": true,
      "multi_language_per_session": true,
      "telephony_mulaw": true,
      "browser_pcm": true,
      "turn_detector": true,
      "interim_results": true,
      "speaker_diarization": true,
      "code_switching_indic_english": true,
      "non_hallucinating": true
    },
    "timing": {
      "min_utterance_end_ms": 1000,
      "default_utterance_end_ms": 1000,
      "max_utterance_seconds": 30
    }
  }
}

`session_started` — `start` message processed, audio is now accepted

Sent by the upstream STT server after a start message is received and validated. It is safe to begin sending audio frames immediately after this event.

{
  "type": "session_started",
  "session_id": "sess_8c3d1a9f4b7e2c51",
  "ignored_config_keys": [],
  "languages": ["en", "hi"],
  "model": "60db-stt-v01",
  "encoding": "linear",
  "sample_rate": 48000,
  "audio_enhancement": "off",
  "utterance_end_ms": 1000,
  "early_final": false,
  "turn_detector": true,
  "interim_frequency": 500,
  "interim_min_voiced_ms": null,
  "interim_delta_streaming": false,
  "token_streaming": false,
  "max_utterance_seconds": 30.0,
  "diarize": false
}

Trust this event over your request — it echoes the effective settings. Fields worth branching on:

Field	Notes
`languages`	The authoritative resolved candidate list after normalization and shared-backend collapse.
`ignored_config_keys`	Every key the server did not recognise (for example browser `echoCancellation`). Unknown keys never fail a session — check this list while integrating.
`early_final` / `turn_detector`	The effective turn-shaping profile (see Turn-ending profiles).
`interim_frequency`	Effective `interim_results_frequency` (`null` = interims off). `interim_min_voiced_ms: null` means the default (600).
`diarize`	Echoes whether diarization was actually accepted.
`model`	Always `60db-stt-v01`. The proxy rebrands the upstream label and strips the upstream `device` field, so neither reveals backend internals.

`speech_started` — VAD detected voice activity

{
  "type": "speech_started",
  "timestamp": 1700000000.123
}

Use this for barge-in: interrupt TTS playback when this arrives. Fired after 2 consecutive VAD-positive chunks (~64ms of confirmed speech).

`transcription` — Transcription result

All results (interim and final) share the same transcription type — differentiate with flags. Final result (is_final=true, speech_final=true):

{
  "type": "transcription",
  "text": "Hello, how are you?",
  "confidence": 0.87,
  "language": "en",
  "language_name": "EN",
  "is_final": true,
  "speech_final": true,
  "segment_final": false,
  "is_partial": false,
  "utterance_id": 3,
  "sentence_id": 3,
  "final_id": "aa653fdf0a5c4372aee5d3af4dac2164:3",
  "processing_mode": "sentence_complete",
  "endpoint_reason": "vad_silence",
  "duration": 1.82,
  "latency": 0.43,
  "timestamp": 1700000000.456,
  "words": [
    { "word": "Hello", "start": 0.0, "end": 0.32, "probability": 0.94 },
    { "word": "how",   "start": 0.35, "end": 0.52, "probability": 0.92 }
  ],
  "speakers": [
    { "speaker": "SPEAKER_00", "start": 0.0, "end": 1.82 }
  ]
}

Segment final (segment_final=true) — max_utterance_sec cut a long monologue; the same turn continues with more finals to come. Check segment_final before speech_final. Withheld final (text="", processing_mode="low_confidence_transcript") — the text was too short and too unsure to trust. Only sent when all hold: confidence below 0.55, at most 4 words, at most 14 characters. The event still arrives so you can clear the interim it replaces; it is not billed.

{
  "type": "transcription",
  "text": "",
  "is_final": true,
  "speech_final": true,
  "segment_final": false,
  "processing_mode": "low_confidence_transcript",
  "words": [],
  "timestamp": 1700000000.789
}

Interim result (is_final=false, speech_final=false) — only sent when interim_results_frequency is set:

{
  "type": "transcription",
  "text": "Hello how",
  "confidence": 0.72,
  "language": "en",
  "is_final": false,
  "speech_final": false,
  "is_partial": true
}

Interims are live guesses for display and hints — the text is the guess for the whole turn so far, so replace the displayed line with each interim (never append). Never send interim text to your LLM or business logic — use finals only. A final always follows. Response Fields:

string

Transcribed text. Empty string = speech-end-no-result signal.

number

0.0–1.0. Telephony typically 0.35–0.75; browser 0.55–0.95.

string

Detected language code e.g. "en".

string

Uppercase language code (e.g. "EN") in the WS shape. Note: REST /stt returns the full English name ("English") here — WS preserves the legacy uppercase-code shape for client compatibility.

boolean

false = interim · true = committed final for the turn.

boolean

Same as is_final — the two flags always carry the same value.

boolean

true = this final was cut by the max_utterance_sec length cap and more of the same turn follows. Check this before speech_final.

boolean

true for interim results only.

integer

Turn number. Interims carry the id their final will have. sentence_id is the same value.

integer

Same value as utterance_id.

string

Unique id of a committed result (finals only), e.g. "aa65…:3". Use it to de-duplicate — if a network retry delivers a final twice, drop the duplicate.

string

What ended the turn: vad_silence, client_finalize, client_stop, max_utterance, language_change, or response_timeout. interim on interims.

number

Duration (seconds) of the audio segment transcribed.

number

Seconds from processing start to result ready (excludes queue time).

array

Word-level timestamps [{word, start, end, probability}], seconds from turn start. Finals only; empty on interims. For Indian languages the times are evenly-spread estimates and probability repeats confidence.When the keyword/context-terms boost replaced a word, the entry may include boosted: true and original (the pre-boost word). The segment-level text is already rebuilt from boosted words upstream — no client-side stitching required.

array

List of [{speaker, start, end}] diarization turns when config.diarize=true. Omitted or null otherwise. Raw speaker IDs look like SPEAKER_00, SPEAKER_01; clients typically re-label these as “Speaker 1”, “Speaker 2” in order of first appearance.

string

How the result was produced. On ordinary finals: sentence_complete; on a max_utterance_sec cut: max_utterance_segment; on interims: streaming_interim. A withheld final carries low_confidence_transcript (empty text, not billed). Treat any unrecognised value as “withheld”.

number

Audio signal-to-noise ratio (dB) for this utterance, when measured. Optional. Surface as a “good / fair / poor” badge: >= 15 good, 0–15 fair, < 0 poor.

`no_speech` — The turn had no transcribable audio

{
  "type": "no_speech",
  "terminal": true,
  "outcome_type": "no_speech",
  "final_id": "aa653fdf0a5c4372aee5d3af4dac2164:6",
  "is_final": true,
  "speech_final": true,
  "segment_final": false,
  "processing_mode": "vad_silence",
  "endpoint_reason": "vad_silence",
  "duration": 3.7,
  "utterance_id": 6,
  "timestamp": 1700000000.5
}

terminal: true — the turn is over with no words. Clear the interim display; if segment finals were collected for this turn, dispatch them now.
terminal: false — an interim check found no words; the turn is still open. Ignore it (or just clear the interim). There is no text field. Not billed.

`turn_complete` — Turn detector confirmed end of thought

Only with turn_detector: true. Sent right before the final it confirms.

{
  "type": "turn_complete",
  "complete": true,
  "forced": false,
  "probability": 0.983,
  "threshold": 0.5,
  "silence_ms": 1003
}

forced: true means the turn was ended by the safety limit (3 × utterance_end_ms) even though the model judged it unfinished.

`interim_delta` — Append/replace hint for interims

Only with interim_delta_streaming: true. Sent alongside each interim.

{
  "type": "interim_delta",
  "text": " hai",
  "full_text": "mera naam Kapil hai",
  "operation": "append",
  "append_only": true,
  "utterance_id": 1,
  "language": "hi",
  "confidence": 0.8,
  "timestamp": 1700000000.1
}

operation: "append" → add text to the line. operation: "replace" → the guess changed; replace the line with full_text.

`transcription_tokens` — Stable word tokens for animation

Only with token_streaming: true. Word tokens with stable ids for UIs that animate words. transcription stays the source of truth.

{
  "type": "transcription_tokens",
  "event": "append",
  "source": "interim",
  "sequence": 2,
  "utterance_id": 7,
  "tokens": [
    { "id": 1, "index": 0, "text": "Hello", "is_final": false },
    { "id": 2, "index": 1, "text": " world", "is_final": false }
  ],
  "language": "en",
  "is_partial": true,
  "append_only": true,
  "timestamp": 1700000000.345
}

event: append for interims, finalize for the final.

`language_changed` — After `config` message changes language

{
  "type": "language_changed",
  "language": "Multi-language: HI",
  "language_code": ["hi"]
}

Acknowledgements — After a `config` change

Each recognised config key is acknowledged with its own event (utterance_end_changed, turn_detector_changed, interim_frequency_changed, interim_delta_streaming_changed, token_streaming_changed, max_utterance_changed, audio_enhancement_changed, audio_format_changed, keywords_changed, context_changed). See the config table for the full mapping.

`warning` — Non-fatal notice

{
  "type": "warning",
  "code": "audio_format_assumed",
  "message": "Binary frames declare no audio format; assuming 8000 Hz mulaw.",
  "encoding": "mulaw",
  "sample_rate": 8000,
  "timestamp": 1700000000.0
}

Sent once when binary audio arrives without encoding/sample_rate declared in start. Informational — the session continues.

`session_stopped` — After `stop` is processed

{
  "type": "session_stopped",
  "billing_summary": {
    "total_duration_seconds": 12.40,
    "total_cost": 0.000620,
    "characters_transcribed": 188,
    "client_estimated_seconds": 12.42
  }
}

billing_summary fields:

Field	Type	Notes
`total_duration_seconds`	number	The audio actually billed — the sum of every committed final’s `duration`.
`total_cost`	number	USD cost for the session.
`characters_transcribed`	number	Total characters across committed transcriptions.
`client_estimated_seconds`	number	Diagnostic only — rough estimate of audio the client sent. Useful for debugging duration drift; never display as a billed number.

What gets billed: every committed final (is_final: true) that carries a non-empty text and a duration. Each final covers a non-overlapping slice of audio, so segment finals and short-pause splits sum to the real spoken time with no double-charge. Withheld finals (text: "", processing_mode: "low_confidence_transcript"), no_speech, and interims are never charged — silence, noise and unusable audio cost nothing.

total_cost here reflects only the STT usage; session_stopped also repeats the last final’s text/transcript on the upstream frame, but you should build the full call transcript from the finals you received, not from this event.

`error` — Rejected message or failed turn

There are two kinds of error frame, told apart by terminal:

Non-terminal (no terminal field) — the server rejected a client message (bad start, invalid config, unknown type, mismatched audio format). The turn and session are unaffected; fix and resend.
Terminal (terminal: true) — recognition of a turn failed (overload, timeout, internal error). That turn is closed without text but the session stays open. Handle it like a terminal no_speech: clear the interim, dispatch any finals already collected, keep the session. If it repeats, reprompt or reconnect.

{
  "type": "error",
  "terminal": true,
  "outcome_type": "error",
  "failure_type": "transcription_timeout",
  "error": "Transcription timed out.",
  "final_id": "aa653fdf0a5c4372aee5d3af4dac2164:7",
  "endpoint_reason": "vad_silence",
  "timestamp": 1700000000.1
}

`failure_type`	Meaning
`transcription_busy`	Server at capacity for this turn.
`transcription_timeout`	Recognition took too long.
`backend_unavailable`	Internal failure.
`transcription_content_incomplete`	Audio could not be fully processed.
`language_identification_uncertain`	No language from your list could be chosen.
`unsupported_asr_language`	The chosen language cannot be transcribed.

Concurrency-limit error frame

When a user has reached their per-user STT session cap (counted across REST + WS combined), the server sends an error frame and closes with code 1008:

{
  "type": "error",
  "error": "Concurrency limit reached: your Free plan allows 2 concurrent STT requests. Wait for one to finish or upgrade your plan.",
  "error_code": "STT_CONCURRENCY_LIMIT",
  "details": { "limit": 2, "plan": "free", "retry_hint": "Wait for an in-flight request to complete" }
}

Existing sessions are unaffected; the cap releases when an in-flight session ends. Do not auto-reconnect on 1008 — the limit only frees when an in-flight session completes. The frontend should distinguish concurrency-limit closes from auth/network failures so the UI message is correct.

`connection_closed` — Upstream dropped the session

Sent when the upstream STT service closes its side. Billing is finalised at this point and the client socket is closed straight after, so treat it as terminal and reconnect if you still have audio to send.

{
  "type": "connection_closed",
  "reason": "60db service disconnected",
  "code": 1006
}

`test_response` — Reply to `test` ping

{
  "type": "test_response",
  "message": "pong - qlabs-stt-proxy ready",
  "timestamp": 1700000000000,
  "processing_mode": "proxy"
}

Answered by the 60db proxy itself — it never reaches the STT server, so a reply confirms the proxy hop only, and the echoed timestamp measures round-trip to the proxy.

Complete Example

Audio Requirements

Property	Telephony (μ-law)	Browser (PCM)
Encoding	`mulaw` (8-bit)	`linear` (16-bit)
Sample Rate	8000 Hz	16000, 24000, 44100, 48000 Hz
Chunk Size	480 bytes (60ms)	960-1920 bytes (60-120ms)
Channels	Mono (1 channel)	Mono (1 channel)

Supported Languages

45 transcription languages total, including 12 Indic languages with Hinglish code-switching, Arabic (MSA), and a broad European/East-Asian set (now including ja Japanese, ko Korean, zh Mandarin, vi Vietnamese, tr Turkish, and he Hebrew). GET /v1/languages (or GET /stt/languages) is the authoritative catalog, including each language’s quality_tier. Indian (12): as Assamese, bn Bengali, gu Gujarati, hi Hindi, kn Kannada, ml Malayalam, mr Marathi, ne Nepali, or Odia, pa Punjabi, ta Tamil, te Telugu. Other (33): ar Arabic, bg Bulgarian, cs Czech, da Danish, de German, el Greek, en English, es Spanish, et Estonian, fi Finnish, fr French, he Hebrew, hr Croatian, hu Hungarian, it Italian, ja Japanese, ko Korean, lt Lithuanian, lv Latvian, mt Maltese, nb Norwegian Bokmål, nl Dutch, pl Polish, pt Portuguese, ro Romanian, ru Russian, sk Slovak, sl Slovenian, sv Swedish, tr Turkish, uk Ukrainian, vi Vietnamese, zh Mandarin. Code-switching (Indic + English): Mixed speech such as Hinglish is handled within a turn — Hindi in Devanagari, English words in Latin. en+hi is the most-tested pair. Arabic dialect tags (ar-eg, ar-lv, …) are rejected — pass ar for best-effort MSA.

Limitations and Best Practices

Handshake ordering (most common mistake) Do not send start in ws.onopen. Wait for the proxy’s connection_established message first — it marks the point at which the proxy has attached its client-message listener. Likewise, wait for session_started before sending audio frames, otherwise the upstream server returns unknown message type: audio. utterance_end_ms floor and grid The minimum is 1000 ms, in steps of 100. The upstream server rejects (does not adjust) a value below 1000 or off the grid, so the 60db proxy snaps the value onto the nearest valid step and floors it at 1000 ms before forwarding — an older client is silently corrected instead of disconnected. If you need faster turn-taking, use profile C (early_final: true) and drive barge-in from speech_started and interims, not from this value. Real-time pacing This endpoint is for live audio. Send frames in real time (20–100 ms each); bursts / faster-than-real-time audio can be rejected and may close the socket with code 1013. Use the REST /stt endpoint for pre-recorded files. Keep sending during silence — a socket with no frames for 50 s is closed. One session per caller Do not multiplex several callers on one socket. There is no echo cancellation at this layer — if bot audio leaks into the caller’s channel it may be transcribed as the caller, so cancel echo before sending audio. Language count Max 5 languages per session. Cross-backend multi-language (e.g. ["en","ar"]) runs per-utterance LID which adds ~20–50 ms per utterance. Same-backend multi-language (e.g. ["en","hi"]) collapses to a single backend’s fast path with zero LID overhead — use this whenever possible. Telephony confidence 8 kHz μ-law → 16 kHz resampling reduces backend confidence by ~0.10–0.15 compared to native wideband input. Use a client-side threshold of 0.35 for telephony vs 0.55 for browser. Buffer limits

Pre-speech ring buffer: 1.0 s (captures first word before VAD fires)
Minimum coalesce before VAD: 160 ms (μ-law 1280 bytes, linear sample_rate × 2 × 160 / 1000 bytes)
Maximum utterance duration: 30 s (anything longer force-finalizes)

VAD

Speech start: Silero probability > STT_VAD_THRESHOLD (default 0.5, server-configurable)
Silence → utterance end: utterance_end_ms of consecutive sub-threshold audio
No separate continuation threshold

Non-hallucinating architecture The 60db STT backend uses non-hallucinating architectures that do not output no_speech_prob — the CTC/RNN-T topology emits blank tokens on non-speech, surfaced as no_speech. Text is never rewritten after recognition (no LLM refinement); keywords/context.terms only correct near-miss spellings of the terms you supply. Diarization diarize=true requires HF_TOKEN on the server plus gated model approval for pyannote/speaker-diarization-3.1. Without both, the request silently falls back to a deterministic mock. Check session_started.diarize to confirm the request was accepted.

Pricing

Rate: $0.00000833 per second
Minimum: $0.01 per session
Billing: Per second of audio processed

Error Codes

Close code	`error_code` (in preceding error frame)	Description
1008	`AUTH_FAILED`	API key / JWT rejected, or missing `workspace_id` on JWT auth
1008	`INSUFFICIENT_CREDITS`	Workspace wallet has no funds
1008	`STT_CONCURRENCY_LIMIT`	Workspace plan’s STT concurrency limit reached (Free 2, Starter 10, Creator 20, Pro 50). `details.limit` carries the active cap. Do not auto-reconnect; cap releases when an in-flight session ends.
1013	—	Overloaded, or a result could not be delivered in time (also sent when audio arrives faster than real time). Reconnect with backoff.
1000	—	Normal close, including after 50 s with no frames.
1011	—	Internal server error
1006	—	Connection lost

A terminal error frame (terminal: true, with a failure_type — see error) means one turn failed but the session stays open; it is not a close code. error_code sits alongside error on the frame — for example

{"type":"error","error":"Insufficient credits to start STT session","error_code":"INSUFFICIENT_CREDITS","details":{"required":0.01,"available":0}}

. One exception: when the wallet runs out mid-session, the deduction failure is reported as {"type":"error","error":{"code":"INSUFFICIENT_CREDITS","message":"…"}} — a nested object — before the socket closes with 1008. Handle both shapes if you parse credit errors.

Testing

# Install wscat
npm install -g wscat

# Test STT
wscat -c "ws://api.60db.ai/ws/stt?apiKey=sk_live_your_key"

Then send:

{"type":"start","languages":["en"],"config":{"encoding":"mulaw","sample_rate":8000,"audio_enhancement":"on","early_final":false,"turn_detector":true,"interim_results_frequency":500}}

TTS WebSocket - Text-to-Speech endpoint
WebSocket API Reference - Complete documentation
Voices API - Get available voices

​STT WebSocket

​Endpoint

​Authentication

​Connection Details

​Session Lifecycle

​Client → Server Messages

​start — Begin session

​Turn-ending profiles

​audio — JSON audio chunk (browser mode)

​Binary frame — raw μ-law audio (telephony mode)

​config — Change settings mid-session

​finalize — End the current turn now

​stop — End session

​test — Ping / latency check

​Server → Client Messages

​connecting — Authentication in progress

​connection_established — Authentication successful

​connected — Proxy wired to upstream STT server

​session_started — start message processed, audio is now accepted

​speech_started — VAD detected voice activity

​transcription — Transcription result

​no_speech — The turn had no transcribable audio

​turn_complete — Turn detector confirmed end of thought

​interim_delta — Append/replace hint for interims

​transcription_tokens — Stable word tokens for animation

​language_changed — After config message changes language

​Acknowledgements — After a config change

​warning — Non-fatal notice

​session_stopped — After stop is processed

​error — Rejected message or failed turn

​Concurrency-limit error frame

​connection_closed — Upstream dropped the session

​test_response — Reply to test ping

​Complete Example

​Audio Requirements

​Supported Languages

​Limitations and Best Practices

​Pricing

​Error Codes

​Testing

​Related

STT WebSocket

Endpoint

Authentication

Connection Details

Session Lifecycle

Client → Server Messages

`start` — Begin session

Turn-ending profiles

`audio` — JSON audio chunk (browser mode)

Binary frame — raw μ-law audio (telephony mode)

`config` — Change settings mid-session

`finalize` — End the current turn now

`stop` — End session

`test` — Ping / latency check

Server → Client Messages

`connecting` — Authentication in progress

`connection_established` — Authentication successful

`connected` — Proxy wired to upstream STT server

`session_started` — `start` message processed, audio is now accepted

`speech_started` — VAD detected voice activity

`transcription` — Transcription result

`no_speech` — The turn had no transcribable audio

`turn_complete` — Turn detector confirmed end of thought

`interim_delta` — Append/replace hint for interims

`transcription_tokens` — Stable word tokens for animation

`language_changed` — After `config` message changes language

Acknowledgements — After a `config` change

`warning` — Non-fatal notice

`session_stopped` — After `stop` is processed

`error` — Rejected message or failed turn

Concurrency-limit error frame

`connection_closed` — Upstream dropped the session

`test_response` — Reply to `test` ping

Complete Example

Audio Requirements

Supported Languages

Limitations and Best Practices

Pricing

Error Codes

Testing

Related