Text-to-Speech

The POST /v1/audio/speech endpoint returns audio data from text input. Supports multiple output formats including mp3, opus, aac, flac, wav, and pcm.

Endpoint

Speech requests accept a nonempty string or a nonempty array of content parts for input. Each part must be an object with a string type. If a part includes text, that value must be a string. Mixed parts, nontext parts, and empty text within content parts remain accepted. Unsupported values return HTTP 400 before fixture matching and do not consume a fixture sequence position.

Unit Test: Basic Speech

Using the programmatic API with vitest, register a fixture and assert on the response.

import { LLMock } from "@copilotkit/aimock";
import { describe, it, expect, beforeAll, afterAll } from "vitest";

let mock: LLMock;

beforeAll(async () => {
  mock = new LLMock();
  await mock.start();
});

afterAll(async () => {
  await mock.stop();
});

it("returns audio for text input", async () => {
  mock.onSpeech("Hello world", { audio: "SGVsbG8gd29ybGQ=" });

  const res = await fetch(`${mock.url}/v1/audio/speech`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "tts-1",
      input: "Hello world",
      voice: "alloy",
    }),
  });

  expect(res.ok).toBe(true);
  const body = await res.json();
  expect(body.audio).toBe("SGVsbG8gd29ybGQ=");
});

Format Options

The response_format field in the request controls the audio output format. Supported values:

JSON Fixture

{
  "fixtures": [
    {
      "match": { "userMessage": "Hello world" },
      "response": {
        "audio": "SGVsbG8gd29ybGQ="
      }
    }
  ]
}

Response Format

Returns audio data matching the requested format:

• audio — base64-encoded audio data in the fixture response

Record & Replay

When no fixture matches an incoming request, aimock can proxy it to the real API and record the response as a fixture for future replays. Enable recording with the --record flag or via RecordConfig in the programmatic API. Binary audio from the provider is base64-encoded in the recorded fixture, with the format derived from the response Content-Type header (e.g. audio/mpeg for mp3). Subsequent requests replay the cached audio without hitting the real API.

npx -p @copilotkit/aimock llmock --record --provider-openai https://api.openai.com

ElevenLabs Voice Design

POST /v1/text-to-voice/design returns JSON previews (each with generated_voice_id and audio_base_64). Match on voice_description the same way TTS matches on text. POST /v1/text-to-voice saves a preview as a permanent voice; unmatched saves echo a deterministic voice_id so a following /v1/text-to-speech/{voice_id} call can hit an existing TTS fixture. GET / DELETE /v1/voices/{voice_id} cover slot management (delete is idempotent).

Recorded design fixtures embed full MP3s as base64 (usually three previews per call), so they are much larger than JSON-only tapes. Proxy unmatched calls under the existing elevenlabs provider key.

mock.onElevenLabsVoiceDesign(/sea captain/, {
  previews: [
    {
      generated_voice_id: "preview_captain",
      audio_base_64: "SGVsbG8=",
      media_type: "audio/mpeg",
      duration_secs: 1.2,
      language: "en",
    },
  ],
  text: "Ahoy there.",
});

const designed = await fetch(`${mock.url}/v1/text-to-voice/design`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    voice_description: "A weathered sea captain in his sixties, gravelly, unhurried",
    model_id: "eleven_ttv_v3",
    auto_generate_text: true,
  }),
});
const previews = await designed.json();

const saved = await fetch(`${mock.url}/v1/text-to-voice`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    voice_name: "Captain",
    voice_description: "A weathered sea captain in his sixties, gravelly, unhurried",
    generated_voice_id: previews.previews[0].generated_voice_id,
  }),
});
const voice = await saved.json();
// voice.voice_id is stable; use it with onElevenLabsTTS