Skip to main content
Every product surface shares this page. Dated entries flag the surfaces they touched (REST, WebSocket, SDK, CLI, MCP, Proxy) so you can skim by what you integrate with.
Voice limits by plan, more professional voices, marketplace KYC
Surfaces: Dashboard, REST, MCP, Docs

Changed

  • Instant voice limit — Free, Starter and Flow workspaces can keep 1 instant (cloned) voice; Creator and Pro keep 10. The limit counts live voices, so deleting one frees a slot. Over the limit, POST /voices returns 403 VOICE_LIMIT_REACHED (was 429)
  • Professional voices — available on Creator (2 voices) and Pro (5 voices); other plans get 403 PVC_NOT_AVAILABLE_ON_PLAN, and a full plan gets 403 PVC_LIMIT_REACHED. You can now create more than one professional voice, and the page lists them all
  • Marketplace listing — needs a Creator or Pro plan, a submitted KYC tax form and bank details. Submitting works while they are under review; the voice goes live only after KYC is verified and bank details are active. Otherwise submitting returns 403 MARKETPLACE_REQUIREMENTS_NOT_MET
  • Downgrades — existing voices are kept; creating and listing new ones is blocked while over the limit

Added

WebSocket STT and TTS: auto-detect fix and refreshed protocol
Surfaces: WebSocket, Proxy, Docs

Fixed

  • STT auto-detect — omitting languages, or sending null, [], "auto" or ["auto"], now expands to the full language list the STT server advertises. Before, the session started but real speech failed with No compatible route exists for this candidate set. session_started.language_mode is candidate_auto for two or more languages and selected for one
  • TTS upstream — the /ws/tts proxy connects to the TTS server through its hostname instead of a fixed IP address

Changed

  • TTS context_created now includes session_id and binary_audio (always false; audio is base64 in audio_chunk)
  • TTS warning and context_aborted are documented. warning with code synthesis_interrupted means new text arrived mid-synthesis; send flush_context to speak it
  • Realtime TTS runs without audio enhancement — the proxy sets audio_enhance to false on every create_context, whatever the client sends
POST /tts-synthesize returns the documented response
Surfaces: REST, Docs

Fixed

  • Documented response fields — POST /tts-synthesize (without stream) now returns success, message, audio_base64, sample_rate, duration_seconds, encoding and output_format, as documented. Previously it returned the engine’s raw audioContent (headerless PCM) and an internal conditioning object
  • Playable audio — for LINEAR16, audio_base64 is a complete WAV file (16-bit mono PCM with header), so it can be saved or played directly
  • Backward compatible — audioContent is still returned unchanged for existing integrations. Streaming ("stream": true) is unchanged: NDJSON lines with result.audioContent
  • No charge without audio — a response that carries no audio now returns an error and is not billed. See Text to Speech
Studio: audio projects, smart regeneration, emotion tags, MP3 download
Surfaces: Dashboard, REST

Changed

  • Audio projects — create a project, write your text and generate narration. Import and URL import were removed
  • Smart regeneration — the first generation converts every line; after that only new or edited lines are generated (and charged). Lines that need generating are marked, and “Generate all” skips finished blocks and respects your plan’s concurrency limit
  • Download — the whole project downloads as one MP3 (GET /studio/projects/:projectId/download). Missing lines are generated first; the endpoint returns 409 AUDIO_INCOMPLETE with the missing lines otherwise
  • Emotion tags — insert [happy], [laugh], [pause 1s] and the rest from the Emotion picker, or use Enhance / Remove tags, the same as Text to Speech
  • Billing — Studio generation is always charged to the workspace you are signed in to, at the TTS rate
Professional voice clones: upload only, marketplace listing optional
Surfaces: Dashboard, REST, Docs

Changed

  • Upload only — the professional clone wizard is now Intro → Upload → Verify → Review. The teleprompter recording method was removed
  • Marketplace listing is optional — choose it on the Review step (off by default). Unlisted voices are ready to use privately right after training, with no review. POST /voices/professional/jobs/:id/finalize accepts list_in_marketplace (boolean); jobs return list_in_marketplace and the new status private
  • Submit later — POST /voices/professional/jobs/:id/submit-to-marketplace sends a private voice to marketplace review (private → awaiting_approval) without retraining. See Professional Voice Clones
Plan-based concurrency limits for TTS and STT
Surfaces: REST, WebSocket, Dashboard, Docs

Changed

  • Concurrency limits now depend on your plan and apply per workspace (REST + WebSocket combined, shared by members and API keys). TTS: Free 1, Starter 5, Creator 10, Pro 20. STT: Free 2, Starter 10, Creator 20, Pro 50. Previously a fixed per-user cap (TTS 5, STT 8)
  • 429 / 1008 errors — TTS_CONCURRENCY_LIMIT / STT_CONCURRENCY_LIMIT now name your plan and its limit; details adds plan
  • API — GET /plans adds concurrency { tts, stt } per plan; GET /billing/current-plan adds concurrency for the workspace. See Concurrency limits
Annual billing — Starter, Creator and Pro can be billed yearly at 2 months free
Surfaces: REST, Docs

Added

Annual billing

  • Yearly prices — Starter $60/yr ($5/mo), Creator $220/yr ($18.33/mo), Pro $990/yr ($82.50/mo). Yearly is monthly × 10, so you get 2 months free. Credits stay a monthly allotment (120K / 400K / 2M per month) and reset every month on your subscription start day
  • Subscribe on a cycle — POST /billing/subscribe takes billing_cycle: "monthly" | "yearly" (default monthly) and returns action (subscribe, upgrade or switch_to_yearly), billing_cycle and amount. See Subscribe to a Plan
  • Plan prices — GET /plans adds pricing.yearly, yearly_monthly_equivalent and yearly_savings_pct. See List Plans
  • Current plan — GET /billing/current-plan adds data.plan.billing_cycle, next_credit_reset_at and next_payment_at. For yearly plans next_payment_usd is the yearly price. See Current Plan
  • Plan changes — monthly to yearly and upgrades take effect immediately, and unused plan credits carry over. Yearly to a higher yearly plan gets a prorated discount at checkout. Downgrades and cancellations take effect at the end of the billing cycle
  • Errors — 403 YEARLY_TO_MONTHLY_NOT_ALLOWED (cancel the yearly plan, then choose monthly), 400 DOWNGRADE_KEEPS_CYCLE, 400 CYCLE_NOT_AVAILABLE, 400 INVALID_BILLING_CYCLE

Updated

  • Plans & credits guide — new Annual billing section with the price table, plan-change rules and refund policy. Top-ups on yearly plans use the monthly plan rate
Pay-as-you-go credit top-ups for Creator & Pro — buy credits at your plan rate
Surfaces: REST, Dashboard, Docs

Added

Pay-as-you-go top-ups (Creator and Pro)

  • Buy top-ups — Creator and Pro workspaces can buy $10–$1,000 of credits (presets $10, $25, $50, $100) with POST /billing/add-funds (purpose: "credits"). Top-up credits never expire and stay usable after a downgrade
  • Plan rate — credits = floor(usd × plan monthly credits / plan price). Creator buys about 18,181.8 credits per $1 ($10 → 181,818), Pro about 20,202 per $1 ($10 → 202,020). The credits are quoted at checkout and fixed, even if you change plan before the payment completes. Coupons lower the price, not the credits
  • Quote and balance — GET /billing/topup returns eligibility, your plan rate, preset quotes and the plan / top-up credit balance. GET /billing/topup/quote?amount= previews a custom amount
  • Dashboard — Subscription → Platform has a new Top-up credits card with a Pay-as-you-go button. When credits run out, the You’re out of credits dialog offers Pay-as-you-go (Creator, Pro) or Upgrade to Creator (Free, Starter) instead of an error toast
  • Ledger — GET /billing/usage-logs lists top-ups as TOP_UP rows, and SA_TOP_UP for offline payments recorded by 60db support. AUTO_TOP_UP is reserved for automatic top-up
  • Errors — 403 TOPUP_NOT_AVAILABLE_ON_PLAN on Free and Starter, and 400 TOPUP_AMOUNT_OUT_OF_RANGE outside $10–$1,000

Updated

  • Out-of-credits 402 — the message now points to a top-up (Creator, Pro) or an upgrade (Free, Starter), and the body includes details.topup_available and details.auto_topup_enabled
  • Plans & credits guide — new Plans, Credits & Top-ups page. Usage draws plan credits first, then top-up credits
  • Pricing pages — Memory and Judge pricing no longer describe a pay-as-you-go wallet only

Coming soon

  • Automatic top-up — save a card once and credits reload automatically when your balance falls below a threshold you choose, up to a monthly limit you set. Not available yet
Emotion tags and Enhance for Text-to-Speech
Surfaces: REST, Docs

Added

  • Emotion tags — put [happy], [calm], [apologetic] and 17 other emotion tags inline in text to change how the following sentences sound, blend two with [sad, calm], and add sound tags such as [laugh], [sigh], [gasp] and [pause 500ms]. Unknown tags are dropped, never spoken. Works on POST /tts-synthesize and in the dashboard. See Emotion Tags
  • Enhance — POST /tts/enhance adds emotion and sound tags to your text (up to 5,000 characters) and changes nothing else: your words, spacing and language come back exactly as sent. No credits are charged. See Enhance Text
  • Dashboard — the Text to Speech page highlights tags as colored chips and adds Add emotion, Enhance, Undo enhance and Remove tags
  • Audio enhancement — generated audio is now polished by default. Send audio_enhance: false on POST /tts-synthesize to skip it (the older enhance still works). It is never applied to the real-time WebSocket TTS used by voice agents, so live calls keep their low latency. The dashboard has a new Audio enhancement switch. See Text to Speech
Click-to-Call — two-leg agent-then-customer calls, signed outcome webhooks, per-call recordings
Surfaces: REST, Docs

Added

Click-to-Call (Production)

  • Place a call — POST /dialer/click2call rings the agent on your own DID, then the customer, then bridges them. No SIP client, no WebRTC, no microphone. Returns 201 when a call was dialled and 200 when the reference_id was already bound to a live call and nothing new was dialled (data.placed / data.replayed)
  • Read a call — GET /dialer/click2call/{id} by call_id or your own reference_id; ?refresh=1 polls the platform for the authoritative state and repairs a call whose terminal webhook never arrived
  • History — GET /dialer/click2call with limit/offset, mine=0 for the whole workspace, and api_key_id for a per-integration statement
  • Signed webhooks — pass callback_url + callback_secret (16–256 chars, yours to choose) per call; every verified event is relayed to you with x-qcall-signature: v1=<hmac-sha256> over timestamp + "." + raw_body. Up to 6 attempts, ≥2 min apart, abandoned after 24 h
  • Delivery inspection — GET /dialer/click2call/{id}/events shows every event, whether it was applied, and the status your own endpoint returned for it
  • Recordings — record: true on placement, then POST /dialer/click2call/{id}/recording for a short-lived playback URL (audio uploads after the call ends — check recording_available)
  • Readiness — GET /dialer/click2call/config reports whether the server can dial, whether it can verify results, and the numeric limits it enforces
  • Idempotency — reference_id is optional and minted per call when omitted; it is permanent per trunk, and a failed call still consumes it. Refusals that placed no call (429, 503) are safe to retry with the same one
  • Billing — ₹0.30/min ($0.003/min) on bridged talk time only, prorated per second. Unanswered calls are free. cost_usd and billing_status are on the call itself
  • Limits — 20 calls/min per caller (C2C_RATE_LIMITED), ring timeouts 5–60 s (defaults 30 agent / 45 customer), max_duration 30–7200 s (default 3600), metadata ≤ 2 KiB
  • See the Click-to-Call guide and API reference

Updated

  • Dialer feature guide — now distinguishes the two ways to place a call: POST /dialer/calls (your SIP client dials) vs POST /dialer/click2call (60db dials both legs)
  • Introduction — Click-to-Call feature card
Music (beta) & Dialer — song generation, voice library, call management, recordings, webhooks
Surfaces: REST, SDK, MCP, Docs

Added

Music API (Beta, Free)

  • Song generation — POST /songs (simple mode: one prompt, we write lyrics; advanced: full control)
  • Song polling — GET /songs/{id} (async generation, ~1.5–2.5 min)
  • Voice management — catalog voices or save custom voices from audio samples
  • Download — GET /songs/{id}/download (MP3)
  • SDK methods — client.music.create(), list(), get(), update(), delete(), download(), waitForCompletion(), listVoices(), createVoice()
  • MCP tools — sixtydb_music_* tools for Claude Desktop
  • See Music feature guide and API reference

Dialer API (Production)

  • Number management — rent Indian numbers (E.164, +919876543210), set caller ID, release
  • Call reservation — POST /dialer/calls (SIP client places the actual call; API reserves)
  • Call history — GET /dialer/logs (CDR with billsec, disposition)
  • Recordings & transcripts — auto-record, fetch playback URL (~5 min), get transcript
  • Webhooks — subscribe to call/recording events (create, test, rotate signing key, delivery history)
  • Usage/billing — track charges per-minute and monthly subscriptions
  • Setup prerequisites — trunk + KYC done in dashboard (not API; gates are 409/403 on missing setup)
  • Pricing — ₹600/month ($6) per number; ₹0.30/min ($0.003) per call (inbound/outbound, prorated/sec)
  • SDK methods — client.dialer.status(), numbers.search()/buy()/list()/release()/setCallerId(), calls.list()/get()/reserve(), recordings.list()/playbackUrl()/transcript()/delete()/setEnabled(), billing.subscriptions()/usage(), webhooks.list()/create()/update()/delete()/rotateSecret()/test()/deliveries()
  • MCP tools — sixtydb_dialer_* tools; confirm: true required for operations that charge money or release resources
  • See Dialer feature guide and API reference

Updated

  • SDKs — Music and Dialer sections added to JavaScript and Python docs
  • Introduction — feature cards for Music and Dialer
  • MCP Server — new music and dialer tool groups
STT WebSocket — single-phase finals, turn-shaping config (early_final / turn_detector), audio_enhancement on|off
Surfaces: WebSocket, Proxy, SDK, DocsThe streaming STT protocol was simplified to a single-phase model. The server no longer rewrites text after recognition (no LLM refinement), so the two-phase speech_final flow and its accompanying fields are gone. This is the behaviour every integration (proxy, web app, teleprompter) now targets.

Changed

  • speech_final now always equals is_final. Every committed result is a single transcription with is_final: true (and speech_final: true); interims carry both false. There is no second “canonical” emit. Simple consumers append each final and clear the interim.
  • config.audio_enhancement values are now "on" | "off" (default off). The old "light" / "adaptive" (and boolean true) are accepted for compatibility and map to "on". Use "on" for noisy / telephony audio.
  • Billing now charges every committed final by its duration. Withheld finals (empty text) and no_speech are never charged. A single turn can produce several finals (short-pause splits, segment_final on long monologues) — their durations are non-overlapping, so there is no double-charge.

Added

  • Turn-shaping config: early_final (bool, default true; send false to keep whole sentences in one final) and turn_detector (bool, default false; send true to wait until the caller finished the thought — emits turn_complete). See the three latency/accuracy profiles in the STT reference.
  • More config: interim_min_voiced_ms (default 600), interim_delta_streaming, token_streaming, max_utterance_sec (3–30, default 30).
  • finalize client message ({ "type": "finalize" }) — closes the current turn now and returns its transcription (or no_speech); the session continues.
  • New server events: no_speech (turn had no words; terminal true/false), turn_complete (only with turn_detector: true), interim_delta (with interim_delta_streaming), transcription_tokens (with token_streaming), and warning (e.g. audio_format_assumed).
  • New transcription fields: segment_final (length-cap split — check before speech_final), final_id (unique per committed result — de-duplicate on it), and endpoint_reason (vad_silence | client_finalize | client_stop | max_utterance | language_change | response_timeout | interim).
  • Language coverage is now 45 (adds ja, ko, zh, vi, tr, he, and more). GET /v1/languages is authoritative.

Removed

  • config.continuous_mode — streaming is always continuous; the field is ignored (a session no longer stops after the first utterance).
  • config.remove_fillers and config.no_speech_threshold — no longer used (filler stripping was an LLM-refinement feature, which is gone).
  • The two-phase / refined protocol and its fields: the separate refined event, is_final: true, speech_final: false first emits, llm_applied / llm_reason / llm_latency_ms, tentative / tentative_reason, and session_started.llm_refinement.
  • The mode_changed event (it acknowledged continuous_mode).
  • The old skip-mode markers (speech_end_no_result, speech_end_too_short, hallucination_rejected) — withheld low-confidence finals now arrive as text: "" with processing_mode: "low_confidence_transcript".

Migration

  • Drop continuous_mode / remove_fillers / no_speech_threshold from start.config; add early_final: false and turn_detector: true for whole-sentence finals (profile A).
  • Change audio_enhancement: "adaptive" / "light" → "on".
  • Gate on any is_final: true (equivalently speech_final: true); stop waiting for a second canonical emit and remove refined-event handling.
  • words[] timings use probability, not confidence.
See the WebSocket STT reference and guide.
STT WebSocket — utterance_end_ms floor raised to 1000 ms, handshake shape corrected
Surfaces: WebSocket, Proxy, Docs

Changed

config.utterance_end_ms minimum and default are now 1000 ms (were 300 ms minimum / 500 ms default).
  • The upstream STT server rejects a start carrying a lower value instead of clamping it, which killed the session outright.
  • The 60db /ws/stt proxy now clamps any value below 1000 ms up to 1000 ms before forwarding — on start and on mid-session config — so existing clients sending 500 keep working and simply get 1000 ms behaviour.
  • Need faster turn-taking? Drive barge-in from speech_started and interim results rather than lowering this value.

Fixed (docs)

STT handshake frames were documented with the TTS shape. /ws/stt sends {"type":"connecting"} and {"type":"connection_established","service":"stt","user_id":…} — tagged by type, fields top-level. The examples previously branched on msg.connection_established, the nested /ws/tts shape, which never matches on STT. Examples now also wait for session_started before streaming audio, since connected arrives before the session exists and is emitted twice (proxy frame, then upstream capability frame).

Documented

  • session_started.llm_refinement — whether your context actually opened the refinement gate.
  • llm_applied, llm_reason, llm_latency_ms on canonical transcription emits.
  • tentative / tentative_reason — proxy-substituted text when upstream rejects an utterance as a hallucination.
  • The proxy’s word-preservation guardrail: a refinement that keeps under 40% of the first emit’s words is rolled back to the original text.
  • connection_closed, the AUTH_FAILED error code, and the nested mid-session INSUFFICIENT_CREDITS error shape.
  • Billing: only canonical finals are charged; speech_end_no_result, speech_end_too_short, hallucination_rejected and low_snr_dropped never are.
See WebSocket STT reference.
TTS REST — Inworld payload alignment (POST /tts-synthesize)

Changed

POST /tts-synthesize now accepts the full Inworld schema.
  • audio_config: { audio_encoding, sample_rate_hertz } — nested block matching the upstream schema.
  • timestamp_type (default "WORD") — enables per-word timestamps in the final NDJSON chunk.
  • Legacy flat keys (sample_rate, audio_encoding) still work; new clients should use audio_config.
Default sample_rate_hertz moved from 48000 → 24000 to match the upstream default and reduce bandwidth. Callers that need 48 kHz must send it explicitly.See Text to Speech reference.
STT WebSocket — remove_fillers config key

Added

config.remove_fillers on WebSocket start (STT).
  • New boolean field inside start.config. Default false.
  • When true and context is set (LLM refinement gate is open), the canonical transcription emit (speech_final: true) drops filler words — um, uh, like, you know, …
  • No effect when context is omitted (refinement stays off).
  • The 60db /ws/stt proxy coerces non-boolean values to false rather than rejecting the session.
See WebSocket STT reference.
Onboarding, email delivery (Cloudflare), and platform fixes
Surfaces: REST, Email

Added

Onboarding fields (POST /user/onboarding).
  • Now accepts default_voice_id (string), marketing_consent (boolean), consent_timestamp, and consent_source — all optional and backward compatible.
  • Marketing consent is server-stamped: consent_timestamp / consent_source are set on the server; any client-sent values are ignored.
  • Consent is persisted best-effort after the onboarding commit, so a consent-write failure can never fail the onboarding save.
One-click unsubscribe (/unsubscribe).
  • Public, HMAC-token-verified endpoint for lifecycle email. Sends a List-Unsubscribe (one-click) header for bulk-sender compliance.

Changed

Email delivery now uses Cloudflare Email Service.
  • Transactional and lifecycle mail send via Cloudflare (smtp.mx.cloudflare.net, implicit TLS on 465) authenticated with a Cloudflare API token; DKIM/ARC are applied automatically. Configure CLOUDFLARE_API_TOKEN (used as the SMTP password) and SMTP_EMAIL (a sender on a domain onboarded for Email Sending).
  • Signup verification email is now sent after the account is committed, so a transient email/transport failure can no longer roll back account creation.

Fixed

GET /workspaces duplicate rows.
  • A workspace with more than one active membership row no longer appears multiple times in the response; the list is de-duplicated per workspace.
Onboarding 500 on consent.
  • Submitting onboarding no longer returns 500 when the marketing-consent columns are absent from the database (consent persistence is best-effort — apply the consent migration to enable it).
Email credential safety.
  • SMTP / Cloudflare credentials are scrubbed from any thrown or logged email error.

Security

  • Cerbos authorization checks added across additional routes (STT, TTS, SLM, Studio, webhooks, sixtydb).

Migrations

  • New environments must apply the consent columns on users (marketing_consent, consent_timestamp, consent_source); plus the events / lifecycle-email tables if the lifecycle email program is enabled.
STT editor: transcribe API + audio playback
Surfaces: REST

Added

  • New STT transcribe API powering the STT editor.
  • Audio playback in the STT editor.

Fixed

  • Resolved an audio-URL leak.
  • Fixed audio not playing after a redirect.
Coupons for wallet top-ups
Surfaces: REST, Billing

Added

  • Coupon system for wallet top-ups — percentage / fixed discounts and deposit multipliers (e.g. 2× / 3×). Redemptions are recorded for audit (coupon_redemptions, referencing the top-up’s payment_transaction_id).
STT/TTS reliability + new features

Added

Longer request timeout (REST POST /stt).
  • Server-side request timeout raised from 120 s → 600 s; clients should match. The 25 MB / ~1-hour file-size cap remains.
Custom vocabulary boost.
  • New keywords form field on POST /stt. CSV with optional per-term weights, e.g. Acme:5,XYZ Pharma:8.
  • The same boost runs on the WebSocket path via the existing context.terms field — no client change required to benefit; existing terms payloads now produce boosted / original markers automatically.
  • Words replaced by the boost appear in the response with boosted: true and original: "<pre-boost word>".
Word-level timestamps and confidence.
  • return_timestamps=word adds per-word start / end to the REST response.
  • include_confidence=true adds per-word confidence (0-1).
Diarization controls.
  • min_speakers / max_speakers form fields on POST /stt.
Script correction.
  • script_correction=true for code-mixed Devanagari / Latin audio.
Audio quality indicator.
  • snr_db is now a top-level field on every STT response and a per-message field on WS transcription events. Surface as a “good / fair / poor” badge (>= 15 good, 0–15 fair, < 0 poor).

Limits

Per-user concurrency cap. Counted across REST + WS combined.Excess returns:
  • REST — 429 with error_code: STT_CONCURRENCY_LIMIT / TTS_CONCURRENCY_LIMIT and details.limit.
  • WebSocket — error frame followed by close code 1008. The cap releases when an in-flight request completes; no server-side queueing.

New error codes

499 is non-standard but used (nginx convention) to signal the client closed the connection before the response was sent.

New warning codes

The same low_snr_dropped value appears as a processing_mode on WS transcription events — treat as “skip this utterance, no useful text”.

Billing changes

  • WS double-bill fix. Sessions that previously double-billed when the client sent stop are now billed correctly. billing_summary.total_duration_seconds will show as ~50 % of historical values for affected sessions.
  • billing_summary.client_estimated_seconds (new, diagnostic) — rough estimate of audio the client sent. Useful only for debugging duration drift; never display as a billed number.
  • Low-SNR refunds. REST sessions where upstream dropped audio for low SNR no longer charge the user. WS already handled this.
  • Cancellation propagation. Client disconnects mid-request now correctly abort the upstream call. No charge for cancelled requests (499 STT_CLIENT_CANCELLED).
  • Diarization surcharge documented. When diarize=true, requests incur a documented +30 % surcharge on top of the base STT rate to cover GPU diarization (pyannote).

Plan-tier gate

context (LLM refinement) is now a paid feature. On the Free plan the field is silently stripped server-side and the response includes warning_codes: ["llm_refinement_not_in_plan"]. Surface an upgrade prompt or hide / disable the input on Free plans for better UX.

Surface summary

  • REST — POST /stt accepts new form fields (keywords, languages, min_speakers, max_speakers, return_timestamps, include_confidence, script_correction, min_split_sec); returns new fields (snr_db, language_source: "long_audio_chunked", per-word boosted/original/confidence, per-segment chunk_idx).
  • REST POST /tts, /tts-synthesize, /tts-stream — may now return 429 TTS_CONCURRENCY_LIMIT.
  • WebSocket /ws/stt — adds low_snr_dropped to processing_mode; per-word boosted/original; message-level snr_db; concurrency-limit error frame + close 1008; corrected billing_summary with new client_estimated_seconds field.
  • WebSocket /ws/tts — concurrency-limit error frame (legacy {error: {message, code, details}} shape) + close 1008.
  • Web UI — axios timeout raised to 600 s, error-code-aware toasts, warning_codes surfaced, WS 1008 distinguished from auth/network failures. The 25 MB upload cap is unchanged.
LLM context refinement for STT

Added — Context-gated LLM refinement

Speech-to-Text now accepts an optional context hint that opens a server-side LLM refinement gate. When supplied, the response transcript is polished for proper-noun accuracy, filler removal, punctuation, and script consistency in mixed-language audio.The shape differs per transport:

REST — context: string

Free-form paragraph describing the session (domain, speakers, jargon). Serialized as a multipart form field.
cURL
JavaScript
Python
CLI
MCP

WebSocket — structured {general, text, terms}

Start message

Changed — WebSocket two-phase canonical flow

When context is supplied, each utterance now produces two transcription events sharing a sentence_id:
  1. First emit — is_final: true, speech_final: false — fast dict-corrected text. Use for low-latency UI paint and voicebot barge-in / NLU.
  2. Canonical — is_final: true, speech_final: true — definitive answer. Either LLM-refined (when llm_applied: true) or the original text re-emitted (when LLM was skipped or failed).
New canonical-only fields:Simple consumers can gate exclusively on speech_final: true and ignore the rest — one canonical event per utterance regardless of whether refinement is on.See the WebSocket STT Reference for the full table and reconciliation patterns.

Added — Word-preservation guardrails (proxy + UI)

To defend against over-aggressive LLM refinement and fast-speech hallucination rejection:
  • Refined word-retention guardrail. If the canonical text drops more than ~60% of the first-emit’s token count, the proxy rolls back to the first-emit text and marks llm_applied: false, llm_reason: "dropped_too_many_words". Tuned so legitimate polish (filler removal, 10–40% compression) passes through untouched.
  • Hallucination-rejected fallback. When the upstream server’s word-rate guard emits an empty final (processing_mode: "hallucination_rejected", common on fast-paced Indic/English mixed audio), the proxy upgrades it into a tentative canonical using the cached first-emit / last-interim text plus tentative: true and tentative_reason: "hallucination_rejected" so words never silently disappear.
Both guardrails run in the /ws/stt proxy, so voicebots and web clients inherit them automatically.

Added — Backward-compat refined event handling

Pre-migration upstream builds emit LLM refinement as a separate refined event rather than a second transcription. The proxy and web client transparently accept both shapes, so existing integrations keep working through the rollout.
If you see refined events in the wire trace, upstream workers haven’t been restarted onto the two-phase build yet — it’s still fully functional; the refined event is deprecated but accepted.

Surface summary

  • REST — POST /stt, POST /v1/transcribe — context: string form field.
  • WebSocket — /v1/stream, /ws/stt — context: {general, text, terms} on start; two-phase canonical flow; llm_applied / llm_latency_ms / llm_reason on canonical.
  • JavaScript SDK — client.speechToText(audio, { context }).
  • Python SDK — client.speech_to_text(audio_file, context=...).
  • CLI — 60db stt:transcribe --context "...".
  • MCP Server — sixtydb_stt_transcribe accepts context.
  • Proxy — word-retention guardrail, hallucination fallback, first-emit caching, legacy refined shim.
  • Web UI — context input in Speech-to-Text page and realtime demo; segments rendered with tentative marker when flagged.

Environment knobs (server-side)