Streaming

Overview

Streaming allows you to receive audio chunks in real-time as they’re generated, enabling immediate playback without waiting for the entire audio file to be created. This is essential for interactive applications like voice assistants and chatbots.

Benefits

Low Latency

Start playing audio within milliseconds

Memory Efficient

Process chunks instead of loading entire file

Better UX

Progressive audio playback feels more responsive

Long Content

Handle unlimited text length efficiently

How It Works

Send Request: Submit text to the streaming endpoint
Receive Chunks: Get audio chunks as they’re generated
Play Immediately: Start playing the first chunk
Continue Streaming: Receive and play subsequent chunks
Complete: Receive completion notification

Implementation

JavaScript
Python

import { SixtyDBClient } from '60db';

const client = new SixtyDBClient('your-api-key');

// Audio player setup
const audioContext = new AudioContext();
const audioQueue = [];

await client.textToSpeechStream(
  {
    text: 'This is a longer text that will be streamed in real-time for immediate playback.',
    voice_id: 'default-voice'
  },
  {
    onChunk: async (chunk) => {
      // Convert chunk to audio buffer
      const audioBuffer = await audioContext.decodeAudioData(chunk.buffer);
      
      // Add to queue and play
      audioQueue.push(audioBuffer);
      if (audioQueue.length === 1) {
        playNextChunk();
      }
    },
    onComplete: () => {
      console.log('Streaming complete');
    },
    onError: (error) => {
      console.error('Streaming error:', error);
    }
  }
);

function playNextChunk() {
  if (audioQueue.length === 0) return;
  
  const buffer = audioQueue[0];
  const source = audioContext.createBufferSource();
  source.buffer = buffer;
  source.connect(audioContext.destination);
  
  source.onended = () => {
    audioQueue.shift();
    playNextChunk();
  };
  
  source.start();
}

from sixtydb import SixtyDBClient
import pyaudio

client = SixtyDBClient('your-api-key')

# Audio player setup
p = pyaudio.PyAudio()
stream = p.open(
    format=pyaudio.paInt16,
    channels=1,
    rate=24000,
    output=True
)

def handle_chunk(chunk):
    # Play audio chunk immediately
    stream.write(chunk)

def handle_complete():
    print("Streaming complete")
    stream.stop_stream()
    stream.close()
    p.terminate()

def handle_error(error):
    print(f"Error: {error}")

# Stream text to speech
client.text_to_speech_stream(
    text='This is a longer text that will be streamed in real-time.',
    on_chunk=handle_chunk,
    on_complete=handle_complete,
    on_error=handle_error,
    voice_id='default-voice'
)

Advanced Usage

React Component

import { useState } from 'react';
import { SixtyDBClient } from '60db';

function StreamingTTS() {
  const [isStreaming, setIsStreaming] = useState(false);
  const [text, setText] = useState('');

  const client = new SixtyDBClient(process.env.REACT_APP_API_KEY);

  const streamAudio = async () => {
    setIsStreaming(true);

    await client.textToSpeechStream(
      { text, voice_id: 'default-voice' },
      {
        onChunk: (chunk) => {
          // Play chunk
          playAudioChunk(chunk);
        },
        onComplete: () => {
          setIsStreaming(false);
        },
        onError: (error) => {
          console.error(error);
          setIsStreaming(false);
        }
      }
    );
  };

  return (
    <div>
      <textarea
        value={text}
        onChange={(e) => setText(e.target.value)}
        placeholder="Enter text to speak..."
      />
      <button onClick={streamAudio} disabled={isStreaming}>
        {isStreaming ? 'Streaming...' : 'Speak'}
      </button>
    </div>
  );
}

Voice Assistant

class VoiceAssistant {
  constructor(apiKey) {
    this.client = new SixtyDBClient(apiKey);
    this.audioContext = new AudioContext();
  }

  async speak(text) {
    return new Promise((resolve, reject) => {
      this.client.textToSpeechStream(
        { text, voice_id: 'assistant-voice' },
        {
          onChunk: (chunk) => this.playChunk(chunk),
          onComplete: () => resolve(),
          onError: (error) => reject(error)
        }
      );
    });
  }

  async playChunk(chunk) {
    const audioBuffer = await this.audioContext.decodeAudioData(chunk.buffer);
    const source = this.audioContext.createBufferSource();
    source.buffer = audioBuffer;
    source.connect(this.audioContext.destination);
    source.start();
  }
}

// Usage
const assistant = new VoiceAssistant('your-api-key');
await assistant.speak('Hello! How can I help you today?');

Performance Optimization

Buffering Strategy

class AudioStreamer {
  constructor() {
    this.chunks = [];
    this.isPlaying = false;
    this.minBufferSize = 3; // Wait for 3 chunks before playing
  }

  addChunk(chunk) {
    this.chunks.push(chunk);
    
    if (!this.isPlaying && this.chunks.length >= this.minBufferSize) {
      this.startPlayback();
    }
  }

  async startPlayback() {
    this.isPlaying = true;
    
    while (this.chunks.length > 0) {
      const chunk = this.chunks.shift();
      await this.playChunk(chunk);
    }
    
    this.isPlaying = false;
  }

  async playChunk(chunk) {
    // Play audio chunk
    return new Promise((resolve) => {
      // Audio playback logic
      setTimeout(resolve, chunkDuration);
    });
  }
}

Best Practices

Buffer Management

Maintain a small buffer (2-3 chunks) for smooth playback
Handle network interruptions gracefully
Implement retry logic for failed chunks

Error Handling

Always implement onError callback
Provide user feedback during streaming
Have fallback for streaming failures

Performance

Reuse AudioContext instances
Clean up resources after playback
Monitor memory usage for long streams

User Experience

Show loading indicator before first chunk
Allow users to stop streaming
Provide playback controls

Use Cases

Real-time Chat

async function sendMessage(message) {
  // Display user message
  displayMessage('user', message);
  
  // Get AI response
  const response = await getAIResponse(message);
  displayMessage('assistant', response);
  
  // Stream audio response
  await client.textToSpeechStream(
    { text: response },
    {
      onChunk: (chunk) => playAudioChunk(chunk),
      onComplete: () => enableInput()
    }
  );
}

Audiobook Player

async function playChapter(chapterText) {
  let currentPosition = 0;
  
  await client.textToSpeechStream(
    { text: chapterText },
    {
      onChunk: (chunk) => {
        playChunk(chunk);
        currentPosition += chunk.duration;
        updateProgress(currentPosition);
      },
      onComplete: () => {
        moveToNextChapter();
      }
    }
  );
}

API Reference

Streaming API

View complete streaming API documentation

Get Started

SDKs

Core Features

Overview

Benefits

Low Latency

Memory Efficient

Better UX

Long Content

How It Works

Implementation

Advanced Usage

React Component

Voice Assistant

Performance Optimization

Buffering Strategy

Best Practices

Use Cases

Real-time Chat

Audiobook Player

API Reference

Streaming API

Get Started

SDKs

Core Features

Documentation Index

​Overview

​Benefits

Low Latency

Memory Efficient

Better UX

Long Content

​How It Works

​Implementation

​Advanced Usage

​React Component

​Voice Assistant

​Performance Optimization

​Buffering Strategy

​Best Practices

​Use Cases

​Real-time Chat

​Audiobook Player

​API Reference

Streaming API

Overview

Benefits

How It Works

Implementation

Advanced Usage

React Component

Voice Assistant

Performance Optimization

Buffering Strategy

Best Practices

Use Cases

Real-time Chat

Audiobook Player

API Reference