Legacy Native Protocol

New apps should use /v1/realtime. It speaks the OpenAI Realtime GA protocol, so OpenAI SDKs work without changes. This page covers the native /ws/realtime protocol, which existing integrations can keep using.

The native endpoint is one socket and one session. You stream microphone audio in and get spoken replies back. 60db runs speech recognition, the LLM turn and voice synthesis on the server, with server-side VAD, turn detection and barge-in.

Quick start

// npm install ws
import WebSocket from 'ws';

const API_KEY = process.env.SIXTYDB_API_KEY;

// 1. Pick a voice. Any key can list voices over REST.
const voices = await fetch('https://api.60db.ai/get-voices', {
  headers: { Authorization: `Bearer ${API_KEY}` },
}).then((r) => r.json());
const voice = voices.data.built_in_voices[0].voice_id;

// 2. Connect
const ws = new WebSocket(`wss://api.60db.ai/ws/realtime?apiKey=${API_KEY}`);

ws.on('open', () => {
  // 3. The first frame must be setup
  ws.send(JSON.stringify({
    type: 'setup',
    voice,
    instructions: 'You are a friendly voice assistant. Keep replies short.',
    input_audio: { format: 'pcm16', sample_rate: 48000 },
    output_audio: { format: 'pcm16', sample_rate: 24000 },
    turn: { mode: 'server_vad', utterance_end_ms: 1000, interim_results: true },
    languages: ['en'],
  }));
  // 4. Stream mic audio continuously, ~100 ms per frame (see Audio formats)
});

ws.on('message', (data) => {
  const msg = JSON.parse(data);
  switch (msg.type) {
    case 'input_audio_transcript.partial':   // live user transcript
      console.log('user:', msg.delta); break;
    case 'response.audio_transcript.delta': // streaming reply text
      process.stdout.write(msg.delta); break;
    case 'response.audio.delta':            // base64 PCM16 at the output rate
      /* play Buffer.from(msg.delta, 'base64') */ break;
    case 'turn.complete':                   // turn finished, with usage
      console.log('\nusage:', msg.usage); break;
    case 'error':
      console.error('error:', msg.error.code, msg.error.message); break;
  }
});

Connect

wss://api.60db.ai/ws/realtime?apiKey=sk_your_api_key

Any key with STT and TTS permissions can use the native endpoint. A token (JWT) query parameter is also accepted for the 60db web app.

Session flow

Client                                         Server
  │──── connect ?apiKey=... ────────────────────►│
  │◄─── session.created ─────────────────────────│
  │──── setup (voice, instructions, formats) ───►│
  │◄─── session.updated ─────────────────────────│  ready for audio
  │──── input_audio_buffer.append (×N) ─────────►│
  │◄─── input_audio_buffer.speech_started ───────│
  │◄─── input_audio_transcript.partial (×N) ─────│
  │◄─── input_audio_buffer.committed ────────────│
  │◄─── response.audio_transcript.delta (×N) ────│
  │◄─── response.audio.delta (×N) ───────────────│
  │◄─── response.done / turn.complete ───────────│
  │──── session.close ──────────────────────────►│
  │◄─── session.closed ──────────────────────────│

The first frame must be setup, or an OpenAI-style nested session.update. Any other event first returns a recoverable SETUP_REQUIRED error, and audio is rejected until setup succeeds. voice is required. Get one from GET /get-voices.

Setup fields

Field	Type	Default	Notes
`voice`	string or number	none	Required. Voice ID from `GET /get-voices`
`instructions`	string	`""`	System prompt, up to 32,000 characters
`input_audio.format`	`pcm16` or `mulaw`	`pcm16`	Input encoding
`input_audio.sample_rate`	number	`48000`	8000, 16000, 24000, 44100 or 48000. `mulaw` uses 8000 only
`output_audio.sample_rate`	number	`24000`	8000, 16000, 24000 or 48000
`turn.mode`	`server_vad`	`server_vad`	Server turn detection
`turn.utterance_end_ms`	number	`1000`	Silence that ends an utterance, 1000 to 5000
`turn.interim_results`	boolean	`false`	Emit live partial transcripts
`languages`	string array	auto	ISO codes, up to 6
`temperature`	number	none	0 to 2
`max_response_output_tokens`	number	none	16 to 4096
`tools`	array	none	Function definitions, up to 32. See Function tools
`tool_choice`	`auto`, `none` or `required`	`auto`
`memory.enabled`	boolean	`false`	Turns on Smart Memory. See 60db extensions
`metadata`	object	none	Tags up to 4 KB, echoed in events

The server replies session.updated with the applied config. Use session.update to change settings mid-session. Changes apply from the next turn.

Client events

Event	Purpose
`setup`	Configure the session. Must be the first frame
`session.update`	Change settings mid-session
`input_audio_buffer.append`	Send base64 audio
`input_audio_buffer.commit`	End the current utterance now
`input_audio_buffer.clear`	Discard buffered audio
`response.create`	Start a reply without waiting for speech
`response.cancel`	Stop the reply in progress
`conversation.item.create`	Add a message or `function_call_output`
`session.close`	End the session. The server finalizes billing and replies `session.closed`
`ping`	Keepalive. The server replies `pong`

Send session.close instead of dropping the socket. The server also sends protocol pings every 30 seconds, and you must answer them or the connection drops.

Server events

Event	When	Key fields
`session.created`	Handshake complete	`session_id`
`session.updated`	Config applied	Full session config
`input_audio_buffer.speech_started`	Speech detected	`audio_start_ms`
`input_audio_transcript.partial`	Live user transcript	`delta`
`input_audio_buffer.committed`	Utterance finalized	`item_id`
`response.created`	Reply begins
`response.audio_transcript.delta`	Reply text chunk	`delta`
`response.audio.delta`	Reply audio chunk	`delta` (base64 PCM16)
`response.done`	Reply finished	`status`: `completed`, `cancelled` or `failed`
`turn.complete`	Turn finished, audio drained	`usage`
`interrupted`	User barged in
`response.function_call`	The model calls a tool	`name`, `arguments`, `call_id`
`go_away`	Session nearing a limit	`reason`, `time_left_ms`
`session.closed`	Billing committed, socket closing	`usage`
`error`	Recoverable or fatal error	`error.code`, `error.message`

turn.complete example:

{
  "type": "turn.complete",
  "usage": { "stt_seconds": 4.17, "llm_tokens_in": 0, "llm_tokens_out": 46, "tts_chars": 182 }
}

Barge-in

Reply playing, user talks: the server sends interrupted, cancels the rest of the reply, and starts a new turn. Stop local playback when you get interrupted.
Reply generating, user talks: the new utterance waits behind the current reply and runs as the next turn.

Audio formats

Direction	Encoding	Sample rates
Input	`pcm16` (default)	8000, 16000, 24000, 44100, 48000
Input	`mulaw` (telephony)	8000
Output	`pcm16`	8000, 16000, 24000, 48000

Send about 100 ms per frame. At 48 kHz that is 9,600 samples. Include silence, because the VAD needs a continuous stream.

Limits and pricing

Limit	Value
Concurrent sessions	Limited per plan
Idle timeout	Close code `4000`
Maximum session length	Close code `4001`

The native endpoint uses the same billing as /v1/realtime: $0.02 per connected minute**, plus **$0.50 per 1M LLM tokens. See Realtime overview.

Get Started

Guides

Reference

Legacy Native Protocol

Quick start

Connect

Session flow

Setup fields

Client events

Server events

Barge-in

Audio formats

Limits and pricing

Next steps

​Quick start

​Connect

​Session flow

​Setup fields

​Client events

​Server events

​Barge-in

​Audio formats

​Limits and pricing

​Next steps

Quick start

Connect

Session flow

Setup fields

Client events

Server events

Barge-in

Audio formats

Limits and pricing

Next steps