Pipecat

Pipecat is an open-source Python framework for voice and multimodal bots. Pipecat ships an OpenAI Realtime service, and 60db Realtime speaks the same OpenAI GA protocol, so you point that service at 60db and keep the rest of your pipeline. One WebSocket handles speech recognition, the LLM turn and the spoken reply. Turn detection, barge-in and function calling run on the 60db server.

Speech-to-speech

One realtime connection replaces separate STT, LLM and TTS services

Server-side turns

server_vad or semantic_vad, with barge-in handled by 60db

Function calling

Pipecat tools run in your bot, the model calls them over the session

What changes

Setting	OpenAI	60db
`base_url`	`wss://api.openai.com/v1/realtime` (default)	`wss://api.60db.ai/v1/realtime`
`api_key`	OpenAI key	60db API key (`sk_...`)
Model	`gpt-realtime-2.1` (Pipecat default)	`60db-realtime-v1-mini` (`gpt-realtime*` names also map to it)
Voice	OpenAI voice names	A 60db voice ID. OpenAI voice names map to 60db stock voices

Pipecat sends the model as the ?model= query parameter and your API key as an Authorization: Bearer header. 60db reads both, so no other change is needed.

Installation

Install Pipecat

Requires Python 3.10+. The webrtc extra lets you test in the browser with Pipecat’s built-in runner.

pip install "pipecat-ai[openai,webrtc,runner]"

Set your API key

export SIXTYDB_API_KEY="sk_your_api_key"

See Authentication to create a key.

Quick Start

The bot below talks to 60db Realtime and greets the user when they connect.

bot.py

import os

from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.openai.realtime.events import (
    AudioConfiguration,
    AudioInput,
    AudioOutput,
    InputAudioTranscription,
    SessionProperties,
    TurnDetection,
)
from pipecat.services.openai.realtime.llm import OpenAIRealtimeLLMService
from pipecat.transports.base_transport import TransportParams
from pipecat.workers.runner import WorkerRunner

transport_params = {
    "webrtc": lambda: TransportParams(audio_in_enabled=True, audio_out_enabled=True),
}


async def bot(runner_args: RunnerArguments):
    transport = await create_transport(runner_args, transport_params)

    llm = OpenAIRealtimeLLMService(
        api_key=os.environ["SIXTYDB_API_KEY"],
        base_url="wss://api.60db.ai/v1/realtime",
        settings=OpenAIRealtimeLLMService.Settings(
            model="60db-realtime-v1-mini",
            system_instruction="You are a friendly voice assistant. Keep answers short.",
            session_properties=SessionProperties(
                audio=AudioConfiguration(
                    input=AudioInput(
                        transcription=InputAudioTranscription(language="en"),
                        turn_detection=TurnDetection(silence_duration_ms=700),
                    ),
                    output=AudioOutput(voice="alloy"),
                ),
            ),
        ),
    )

    context = LLMContext()
    aggregators = LLMContextAggregatorPair(context)

    pipeline = Pipeline([
        transport.input(),
        aggregators.user(),
        llm,
        transport.output(),
        aggregators.assistant(),
    ])

    worker = PipelineWorker(pipeline, params=PipelineParams(enable_metrics=True))

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        await worker.queue_frame(LLMRunFrame())  # the bot speaks first

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        await worker.cancel()

    await WorkerRunner(handle_sigint=runner_args.handle_sigint).run(worker)


if __name__ == "__main__":
    from pipecat.runner.run import main

    main()

Run it, then open the URL it prints and allow the microphone:

python bot.py

Configuration

Turn detection

60db supports both turn detection types that Pipecat exposes. Barge-in is on by default: when the user talks over the bot, 60db cancels the reply and Pipecat stops playback.

server_vad
semantic_vad

Ends the turn after a fixed silence. Good for short commands and phone calls.

from pipecat.services.openai.realtime.events import TurnDetection

turn_detection = TurnDetection(silence_duration_ms=700)

Ends the turn when the user sounds finished. eagerness sets how quickly the bot answers.

from pipecat.services.openai.realtime.events import SemanticTurnDetection

turn_detection = SemanticTurnDetection(eagerness="medium")

See Audio and turn detection for every option and how 60db filters echo and “uh-huh” during a reply.

Transcription language

Set InputAudioTranscription(language=...) to the user’s language, for example "hi" for Hindi. The model field is accepted and ignored: 60db always uses its own speech recognition.

Voice

Set AudioOutput(voice=...) to a 60db voice ID from List voices. OpenAI names such as alloy map to 60db stock voices.

Function calling

Register tools on the service and pass their schemas in the context. 60db sends the tool call, Pipecat runs your function, and the result goes back over the same session.

from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.services.llm_service import FunctionCallParams


async def get_weather(params: FunctionCallParams):
    city = params.arguments["city"]
    await params.result_callback({"city": city, "temperature_c": 24, "sky": "sunny"})


weather = FunctionSchema(
    name="get_weather",
    description="Get the current weather for a city",
    properties={"city": {"type": "string", "description": "City name"}},
    required=["city"],
)

llm.register_function("get_weather", get_weather)
context = LLMContext(tools=ToolsSchema(standard_tools=[weather]))

See Function tools for how 60db runs tool calls.

Phone calls

Use Pipecat’s Twilio, Telnyx or Plivo transport in transport_params and keep the same OpenAIRealtimeLLMService settings. For phone agents hosted entirely by 60db, with no bot server of your own, see Realtime overview.

Troubleshooting

Symptom	Fix
`401` on connect	Check `SIXTYDB_API_KEY` is a 60db key (`sk_...`), not an OpenAI key.
Connects but no audio	Make sure `base_url` ends in `/v1/realtime` and the transport has `audio_out_enabled=True`.
Bot interrupts itself	Use headphones or enable echo cancellation in the client. 60db filters most echo, but speaker playback that leaks into the mic can still sound like a user.
Replies in the wrong language	Set `InputAudioTranscription(language=...)` and say the reply language in `system_instruction`.

Get Started

Frameworks

Guides

Reference

Speech-to-speech

Server-side turns

Function calling

What changes

Installation

Quick Start

Configuration

Turn detection

Transcription language

Voice

Function calling

Phone calls

Troubleshooting

Speech-to-speech

Server-side turns

Function calling

​What changes

​Installation

​Quick Start

​Configuration

​Turn detection

​Transcription language

​Voice

​Function calling

​Phone calls

​Troubleshooting

​Related

What changes

Installation

Quick Start

Configuration

Turn detection

Transcription language

Voice

Function calling

Phone calls

Troubleshooting

Related