Skip to main content

​
Overview

Streaming allows you to receive audio chunks in real-time as they’re generated, enabling immediate playback without waiting for the entire audio file to be created. This is essential for interactive applications like voice assistants and chatbots.

​
Benefits

Low Latency

Start playing audio within milliseconds

Memory Efficient

Process chunks instead of loading entire file

Better UX

Progressive audio playback feels more responsive

Long Content

Handle unlimited text length efficiently

​
How It Works

  1. Send Request: Submit text to the streaming endpoint
  2. Receive Chunks: Get audio chunks as they’re generated
  3. Play Immediately: Start playing the first chunk
  4. Continue Streaming: Receive and play subsequent chunks
  5. Complete: Receive completion notification

​
Implementation

import { SixtyDBClient } from '@60db-own/60db-js';

const client = new SixtyDBClient('your-api-key');

// Audio player setup
const audioContext = new AudioContext();
const audioQueue = [];

await client.textToSpeechStream(
  {
    text: 'This is a longer text that will be streamed in real-time for immediate playback.',
    voice_id: 'default-voice'
  },
  {
    onChunk: async (chunk) => {
      // Convert chunk to audio buffer
      const audioBuffer = await audioContext.decodeAudioData(chunk.buffer);
      
      // Add to queue and play
      audioQueue.push(audioBuffer);
      if (audioQueue.length === 1) {
        playNextChunk();
      }
    },
    onComplete: () => {
      console.log('Streaming complete');
    },
    onError: (error) => {
      console.error('Streaming error:', error);
    }
  }
);

function playNextChunk() {
  if (audioQueue.length === 0) return;
  
  const buffer = audioQueue[0];
  const source = audioContext.createBufferSource();
  source.buffer = buffer;
  source.connect(audioContext.destination);
  
  source.onended = () => {
    audioQueue.shift();
    playNextChunk();
  };
  
  source.start();
}

​
Advanced Usage

​
React Component

import { useState } from 'react';
import { SixtyDBClient } from '@60db-own/60db-js';

function StreamingTTS() {
  const [isStreaming, setIsStreaming] = useState(false);
  const [text, setText] = useState('');

  const client = new SixtyDBClient(process.env.REACT_APP_API_KEY);

  const streamAudio = async () => {
    setIsStreaming(true);

    await client.textToSpeechStream(
      { text, voice_id: 'default-voice' },
      {
        onChunk: (chunk) => {
          // Play chunk
          playAudioChunk(chunk);
        },
        onComplete: () => {
          setIsStreaming(false);
        },
        onError: (error) => {
          console.error(error);
          setIsStreaming(false);
        }
      }
    );
  };

  return (
    <div>
      <textarea
        value={text}
        onChange={(e) => setText(e.target.value)}
        placeholder="Enter text to speak..."
      />
      <button onClick={streamAudio} disabled={isStreaming}>
        {isStreaming ? 'Streaming...' : 'Speak'}
      </button>
    </div>
  );
}

​
Voice Assistant

class VoiceAssistant {
  constructor(apiKey) {
    this.client = new SixtyDBClient(apiKey);
    this.audioContext = new AudioContext();
  }

  async speak(text) {
    return new Promise((resolve, reject) => {
      this.client.textToSpeechStream(
        { text, voice_id: 'assistant-voice' },
        {
          onChunk: (chunk) => this.playChunk(chunk),
          onComplete: () => resolve(),
          onError: (error) => reject(error)
        }
      );
    });
  }

  async playChunk(chunk) {
    const audioBuffer = await this.audioContext.decodeAudioData(chunk.buffer);
    const source = this.audioContext.createBufferSource();
    source.buffer = audioBuffer;
    source.connect(this.audioContext.destination);
    source.start();
  }
}

// Usage
const assistant = new VoiceAssistant('your-api-key');
await assistant.speak('Hello! How can I help you today?');

​
Performance Optimization

​
Buffering Strategy

class AudioStreamer {
  constructor() {
    this.chunks = [];
    this.isPlaying = false;
    this.minBufferSize = 3; // Wait for 3 chunks before playing
  }

  addChunk(chunk) {
    this.chunks.push(chunk);
    
    if (!this.isPlaying && this.chunks.length >= this.minBufferSize) {
      this.startPlayback();
    }
  }

  async startPlayback() {
    this.isPlaying = true;
    
    while (this.chunks.length > 0) {
      const chunk = this.chunks.shift();
      await this.playChunk(chunk);
    }
    
    this.isPlaying = false;
  }

  async playChunk(chunk) {
    // Play audio chunk
    return new Promise((resolve) => {
      // Audio playback logic
      setTimeout(resolve, chunkDuration);
    });
  }
}

​
Best Practices

  • Maintain a small buffer (2-3 chunks) for smooth playback
  • Handle network interruptions gracefully
  • Implement retry logic for failed chunks
  • Always implement onError callback
  • Provide user feedback during streaming
  • Have fallback for streaming failures
  • Reuse AudioContext instances
  • Clean up resources after playback
  • Monitor memory usage for long streams
  • Show loading indicator before first chunk
  • Allow users to stop streaming
  • Provide playback controls

​
Use Cases

​
Real-time Chat

async function sendMessage(message) {
  // Display user message
  displayMessage('user', message);
  
  // Get AI response
  const response = await getAIResponse(message);
  displayMessage('assistant', response);
  
  // Stream audio response
  await client.textToSpeechStream(
    { text: response },
    {
      onChunk: (chunk) => playAudioChunk(chunk),
      onComplete: () => enableInput()
    }
  );
}

​
Audiobook Player

async function playChapter(chapterText) {
  let currentPosition = 0;
  
  await client.textToSpeechStream(
    { text: chapterText },
    {
      onChunk: (chunk) => {
        playChunk(chunk);
        currentPosition += chunk.duration;
        updateProgress(currentPosition);
      },
      onComplete: () => {
        moveToNextChapter();
      }
    }
  );
}

​
API Reference

Streaming API

View complete streaming API documentation