From c54fe1acf0d783171e02b7012484d9596b73a7dc Mon Sep 17 00:00:00 2001 From: M2Night Date: Fri, 31 Jul 2026 14:01:58 +0800 Subject: [PATCH] docs: correct TTS model header guidance across agent-facing and authored docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The model header was documented as required with only s1/s2-pro available, predating the S2.1-Pro launch. Live-verified behavior (trace IDs 5e56d2de…, 9027a9d2…, 9a40d052…, aaa50a97…): the header is optional, and omitted or unrecognized values fall back to the paid s2.1-pro registry default in edge-api. - .mintlify/skills/fish-audio-api/SKILL.md: model header optional with all four models and fallback documented; recommend s2.1-pro for production and s2.1-pro-free for the free tier (matching Models Overview); samples moved to s2.1-pro (same list price as s2-pro); multi-speaker and normalize_loudness scoped to s2-pro + S2.1-Pro family; drop the obsolete 422-on-missing-model troubleshooting entry - .mintlify/skills/fish-audio-sdk/: document the SDK typing lag — the API accepts S2.1 models and both SDKs forward them at runtime, but the type definitions still list only s1/s2-pro, so agents are told to use type overrides or the raw-API skill for S2.1 - api-reference/asyncapi.yml: ws model header optional, full enum, fallback documented; multi-speaker and normalize_loudness notes corrected - api-reference/sdk/javascript/api-reference.mdx: note that Backends lags the API and how to pass S2.1 values - features/text-to-speech.mdx: '(default)' on s2-pro qualified as the Python SDK default; raw-API section no longer claims the header is required - developer-guide/core-features/emotions.mdx: bracket emotion syntax attributed to the S2 family per Models Overview - developer-guide/resources/agent-quickstart.mdx: optional header, all four models, recommendation and fallback Capability scoping verified against models-overview.mdx, platform-web tts-models.ts, and the inference engine's model-agnostic handling. AsyncAPI validated with the official parser (0 errors). Co-Authored-By: Claude Fable 5 --- .mintlify/skills/fish-audio-api/SKILL.md | 34 ++++++++++--------- .mintlify/skills/fish-audio-sdk/SKILL.md | 4 +-- .../references/text-to-speech.md | 2 +- api-reference/asyncapi.yml | 16 ++++----- .../sdk/javascript/api-reference.mdx | 2 ++ developer-guide/core-features/emotions.mdx | 2 +- .../resources/agent-quickstart.mdx | 2 +- features/text-to-speech.mdx | 6 ++-- 8 files changed, 36 insertions(+), 32 deletions(-) diff --git a/.mintlify/skills/fish-audio-api/SKILL.md b/.mintlify/skills/fish-audio-api/SKILL.md index aefc0ac..d023927 100644 --- a/.mintlify/skills/fish-audio-api/SKILL.md +++ b/.mintlify/skills/fish-audio-api/SKILL.md @@ -44,7 +44,10 @@ Required headers: - `Authorization: Bearer ` - `Content-Type: application/json` **or** `application/msgpack` -- `model: s2-pro` (required). Values: `s1`, `s2-pro`. Default to `s2-pro` unless the user explicitly asks otherwise. + +Optional headers: + +- `model` — values: `s1`, `s2-pro`, `s2.1-pro`, `s2.1-pro-free`. If omitted or unrecognized, the server falls back to `s2.1-pro` (paid). Default to `s2.1-pro` for production; use `s2.1-pro-free` for free-tier evaluation and prototyping (same model, no TTFA/DPA guarantees). Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set by `format`. Write to a file or pipe to a player. There is **no JSON wrapper** on success. @@ -53,13 +56,13 @@ Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set | Field | Type | Default | Notes | | --- | --- | --- | --- | | `text` | string | — (required) | The text to synthesize. Use speaker tags `<\|speaker:0\|>`, `<\|speaker:1\|>` for multi-speaker. | -| `reference_id` | string \| string[] \| null | null | Voice model ID. Array = multi-speaker (S2-Pro only). | +| `reference_id` | string \| string[] \| null | null | Voice model ID. Array = multi-speaker (`s2-pro` and the S2.1-Pro family). | | `references` | ReferenceAudio[] \| ReferenceAudio[][] \| null | null | Inline zero-shot cloning samples. **Requires `application/msgpack`** because `audio` is raw bytes. 2D array for multi-speaker. | | `temperature` | number 0–1 | 0.7 | Expressiveness. | | `top_p` | number 0–1 | 0.7 | Nucleus sampling. | | `prosody.speed` | number 0.5–2 | 1 | Playback speed. | | `prosody.volume` | number (dB) | 0 | Loudness offset. | -| `prosody.normalize_loudness` | bool | true | **S2-Pro only.** | +| `prosody.normalize_loudness` | bool | true | **`s2-pro` and the S2.1-Pro family.** | | `chunk_length` | int 100–300 | 300 | Text segment size. | | `min_chunk_length` | int 0–100 | 50 | Min chars before a new chunk. | | `normalize` | bool | true | Normalize numbers/etc. for EN/ZH. | @@ -79,7 +82,7 @@ Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set 1. **Library / custom voice model** → set `reference_id` to the model `_id`. Simplest path. 2. **Zero-shot from audio** → set `references` (array of `{audio, text}`) and use **MessagePack** body. JSON cannot carry raw audio bytes. -3. **Multi-speaker dialogue (S2-Pro only)** → `reference_id: [id0, id1, ...]` and embed `<|speaker:0|>` / `<|speaker:1|>` markers inside `text`. For zero-shot multi-speaker, `references` is an array-of-arrays, one inner array per speaker. +3. **Multi-speaker dialogue (`s2-pro` and the S2.1-Pro family)** → `reference_id: [id0, id1, ...]` and embed `<|speaker:0|>` / `<|speaker:1|>` markers inside `text`. For zero-shot multi-speaker, `references` is an array-of-arrays, one inner array per speaker. ### Single-speaker curl @@ -87,7 +90,7 @@ Response: streaming audio bytes (`Transfer-Encoding: chunked`) in the format set curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ - --header "model: s2-pro" \ + --header "model: s2.1-pro" \ --data '{ "text": "Hello! Welcome to Fish Audio.", "reference_id": "", @@ -98,13 +101,13 @@ curl --request POST https://api.fish.audio/v1/tts \ --output out.mp3 ``` -### Multi-speaker curl (S2-Pro) +### Multi-speaker curl ```bash curl --request POST https://api.fish.audio/v1/tts \ --header "Authorization: Bearer $FISH_API_KEY" \ --header "Content-Type: application/json" \ - --header "model: s2-pro" \ + --header "model: s2.1-pro" \ --data '{ "text": "<|speaker:0|>Good morning!<|speaker:1|>Good morning! How are you?", "reference_id": ["", ""], @@ -128,7 +131,7 @@ payload = { headers = { "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/json", - "model": "s2-pro", + "model": "s2.1-pro", } with httpx.stream("POST", "https://api.fish.audio/v1/tts", @@ -156,7 +159,7 @@ payload = { headers = { "Authorization": f"Bearer {os.environ['FISH_API_KEY']}", "Content-Type": "application/msgpack", - "model": "s2-pro", + "model": "s2.1-pro", } body = msgpack.packb(payload, use_bin_type=True) @@ -180,7 +183,7 @@ const res = await fetch("https://api.fish.audio/v1/tts", { headers: { Authorization: `Bearer ${process.env.FISH_API_KEY}`, "Content-Type": "application/json", - model: "s2-pro", + model: "s2.1-pro", }, body: JSON.stringify({ text: "Hello from Fish Audio.", @@ -370,7 +373,7 @@ For low-latency / streaming TTS (e.g. LLM token stream → speech). All frames a ### Connection headers - `Authorization: Bearer ` -- `model: s2-pro` (or `s1`) — **required** +- `model` — optional; same values and fallback behavior as `POST /v1/tts` (falls back to `s2.1-pro` when omitted or unrecognized) ### Event sequence @@ -409,7 +412,7 @@ start = { } async def run(text_stream): - headers = {"Authorization": f"Bearer {API_KEY}", "model": "s2-pro"} + headers = {"Authorization": f"Bearer {API_KEY}", "model": "s2.1-pro"} async with websockets.connect(URL, additional_headers=headers, max_size=None) as ws: await ws.send(msgpack.packb(start, use_bin_type=True)) @@ -458,7 +461,7 @@ import { createWriteStream } from "node:fs"; const ws = new WebSocket("wss://api.fish.audio/v1/tts/live", { headers: { Authorization: `Bearer ${process.env.FISH_API_KEY}`, - model: "s2-pro", + model: "s2.1-pro", }, }); @@ -501,8 +504,7 @@ The S1 model uses `(parenthesis)` tags inside `text`, e.g. `(happy) What a day!` - 402 → out of credit. Check `/wallet/self/api-credit`. - 404 → bad `model/{id}` (voice model doesn't exist or isn't visible to you). - 422 → validation. The response is an array; each item's `loc` points at the offending field. Most common causes: - - `model` header missing on `/v1/tts` or WebSocket. - - `reference_id` is an array but model is `s1` (multi-speaker requires `s2-pro`). + - `reference_id` is an array but model is `s1` (multi-speaker requires `s2-pro` or an S2.1-Pro model). - `references` sent with `Content-Type: application/json` (must be msgpack). - Numeric param out of range (`temperature`, `top_p`, `chunk_length`, `min_chunk_length`, `prosody.speed`, `early_stop_threshold`). - `mp3_bitrate` / `opus_bitrate` set without matching `format`. @@ -512,7 +514,7 @@ The S1 model uses `(parenthesis)` tags inside `text`, e.g. `(happy) What a day!` - User just wants audio from text → `POST /v1/tts` with JSON + `reference_id`. - User has a raw voice clip and wants instant cloning → `POST /v1/tts` with MessagePack + `references`. -- User wants dialogue between multiple speakers → `POST /v1/tts` on `s2-pro` with `reference_id` array and `<|speaker:N|>` tags. +- User wants dialogue between multiple speakers → `POST /v1/tts` on `s2.1-pro` with `reference_id` array and `<|speaker:N|>` tags. - User is streaming tokens from an LLM and wants speech to play as it arrives → WebSocket `/v1/tts/live`. - User wants a persistent custom voice they can reuse → `POST /model` first, then reuse the returned `_id` as `reference_id`. - User wants a transcript → `POST /v1/asr`. diff --git a/.mintlify/skills/fish-audio-sdk/SKILL.md b/.mintlify/skills/fish-audio-sdk/SKILL.md index 605f0fe..d0123b8 100644 --- a/.mintlify/skills/fish-audio-sdk/SKILL.md +++ b/.mintlify/skills/fish-audio-sdk/SKILL.md @@ -1,6 +1,6 @@ --- name: fish-audio-sdk -description: Write code with the official Fish Audio SDKs — Python (`fishaudio`, PyPI `fish-audio-sdk`) and JavaScript/TypeScript (`fish-audio`). Use when the user wants text-to-speech, speech-to-text, voice cloning / voice-model management, or realtime WebSocket TTS through the installed SDK rather than raw HTTP. Covers install and auth, sync + async Python, the TypeScript client, exact method signatures and defaults, model selection (s1 / s2-pro), the real exception types, and the Python↔JavaScript naming differences. For raw REST/WebSocket calls without an SDK (curl, unsupported languages, edge runtimes), use the `fish-audio-api` skill instead. +description: Write code with the official Fish Audio SDKs — Python (`fishaudio`, PyPI `fish-audio-sdk`) and JavaScript/TypeScript (`fish-audio`). Use when the user wants text-to-speech, speech-to-text, voice cloning / voice-model management, or realtime WebSocket TTS through the installed SDK rather than raw HTTP. Covers install and auth, sync + async Python, the TypeScript client, exact method signatures and defaults, model selection (including the S2.1 typing caveat), the real exception types, and the Python↔JavaScript naming differences. For raw REST/WebSocket calls without an SDK (curl, unsupported languages, edge runtimes), use the `fish-audio-api` skill instead. --- # Fish Audio SDK Skill @@ -18,7 +18,7 @@ If the user wants raw `curl` / HTTP / WebSocket without installing an SDK, use t - **Auth:** both SDKs read the API key from the `FISH_API_KEY` environment variable automatically. Get keys at `https://fish.audio/app/api-keys`. Never hardcode a key — read it from the environment. - **Base URL:** `https://api.fish.audio` (override with `base_url=` in Python / `baseUrl:` in JS). -- **Models:** `s2-pro` (default — highest quality) and `s1`. `speech-1.5` / `speech-1.6` are **deprecated**. In Python pass `model="s2-pro"` (keyword); in JS pass the **positional** `backend` argument. +- **Models:** the API supports `s1`, `s2-pro`, `s2.1-pro` (recommended for production), and `s2.1-pro-free` (free tier), but the SDK type definitions currently list only `s1` and `s2-pro` (`s2-pro` = SDK default). Both SDKs forward the model value without runtime validation, so `"s2.1-pro"` works over the wire — static type checkers will flag it, so add `# type: ignore` (Python) / an `as` cast (TS), or use the `fish-audio-api` skill for raw calls. `speech-1.5` / `speech-1.6` are **deprecated**. In Python pass `model="s2-pro"` (keyword); in JS pass the **positional** `backend` argument. - **Audio formats:** `mp3` (default), `wav`, `pcm`, `opus`. - **Playback in examples:** `play()` shells out to a system audio tool — Python uses **ffmpeg/ffplay** (or `mpv`), JS uses **ffplay**. It is for local/desktop use; in a server, `save()` to a file or stream the bytes instead. See [references/installation.md](references/installation.md). diff --git a/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md b/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md index 9052b9e..58cf2e1 100644 --- a/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md +++ b/.mintlify/skills/fish-audio-sdk/references/text-to-speech.md @@ -37,7 +37,7 @@ All keyword-only: | `latency` | `"normal" \| "balanced"` | `"balanced"` | `normal` = higher quality, `balanced` = faster. (No `"low"`.) | | `speed` | `float` | — | Shortcut for prosody speed (0.5–2.0). | | `config` | `TTSConfig` | `TTSConfig()` | Reusable bundle of the settings below. | -| `model` | `"s2-pro" \| "s1"` | `"s2-pro"` | Synthesis model. `speech-1.5` / `speech-1.6` are deprecated. | +| `model` | `"s2-pro" \| "s1"` | `"s2-pro"` | Synthesis model per current SDK typing. The API also accepts `"s2.1-pro"` / `"s2.1-pro-free"` — forwarded fine at runtime, but add `# type: ignore` for static checkers. `speech-1.5` / `speech-1.6` are deprecated. | | `request_options` | `RequestOptions \| None` | `None` | Per-request timeout / headers — see [errors.md](errors.md). | Direct params (`reference_id`, `format`, `latency`, `speed`) override the matching field on `config` when set. diff --git a/api-reference/asyncapi.yml b/api-reference/asyncapi.yml index a615a01..1f4b5e9 100644 --- a/api-reference/asyncapi.yml +++ b/api-reference/asyncapi.yml @@ -56,21 +56,21 @@ channels: ws: headers: type: object - required: - - model properties: model: type: string enum: - s1 - s2-pro - description: TTS model to use for this session. Use `s2-pro` for multi-speaker dialogue synthesis. + - s2.1-pro + - s2.1-pro-free + description: TTS model to use for this session. If omitted or set to an unrecognized value, the session falls back to `s2.1-pro`. Use `s2-pro` or an S2.1-Pro model for multi-speaker dialogue synthesis. description: | Real-time TTS streaming channel. Clients send text chunks and receive audio chunks concurrently. ## Connection Headers - `Authorization: Bearer ` - Required for authentication (see security section) - - `model: ` - Required to specify which TTS model to use (see bindings) + - `model: ` - Optional; specifies which TTS model to use (see bindings). Falls back to `s2.1-pro` when omitted or unrecognized operations: receiveText: @@ -334,7 +334,7 @@ components: Request payload for text-to-speech synthesis in WebSocket StartEvent. It uses the same parameters as the HTTP [Text to Speech API](/api-reference/endpoint/openapi-v1/text-to-speech). Supports single-speaker synthesis on all compatible TTS models. - Multi-speaker dialogue synthesis is only available with `s2-pro`. + Multi-speaker dialogue synthesis is available with `s2-pro` and the S2.1-Pro family. In WebSocket mode, `text` is usually empty in StartEvent and the actual content is sent through subsequent TextEvent messages. required: @@ -401,7 +401,7 @@ components: description: | Inline voice references for zero-shot cloning. Single-speaker uses an array of ReferenceAudio. Multi-speaker uses an array of speaker - arrays. Multi-speaker dialogue is only available with `s2-pro`. + arrays. Multi-speaker dialogue is available with `s2-pro` and the S2.1-Pro family. See the HTTP Text to Speech API page for detailed examples. reference_id: @@ -417,7 +417,7 @@ components: description: | Voice model ID from Fish Audio or your custom models. Use a string for single-speaker synthesis, or an array of model IDs for - multi-speaker dialogue on `s2-pro`. When using multiple speakers, + multi-speaker dialogue on `s2-pro` or an S2.1-Pro model. When using multiple speakers, add speaker tags in `text`. See the HTTP Text to Speech API page for full examples. @@ -549,7 +549,7 @@ components: default: true description: | Normalize output loudness for more consistent perceived volume. - S2-Pro only. + Supported on `s2-pro` and the S2.1-Pro family. securitySchemes: bearerAuth: diff --git a/api-reference/sdk/javascript/api-reference.mdx b/api-reference/sdk/javascript/api-reference.mdx index 34ed8ff..7b6a2e9 100644 --- a/api-reference/sdk/javascript/api-reference.mdx +++ b/api-reference/sdk/javascript/api-reference.mdx @@ -173,6 +173,8 @@ The backend model to use. Backends = 's1' | 's2-pro'; ``` +The API also accepts `s2.1-pro` and `s2.1-pro-free`; the SDK's `Backends` type has not been updated yet, so pass them with a type assertion (for example `'s2.1-pro' as Backends`) or call the [REST API](/api-reference/endpoint/openapi-v1/text-to-speech) directly. + ## Response Classes ### STTResponse diff --git a/developer-guide/core-features/emotions.mdx b/developer-guide/core-features/emotions.mdx index 2a6ab2f..f1b21ce 100644 --- a/developer-guide/core-features/emotions.mdx +++ b/developer-guide/core-features/emotions.mdx @@ -259,7 +259,7 @@ All 13 supported languages can use emotion markers. For sentence-level control, ## S1 (legacy) syntax -The default **S2-Pro** model uses `[bracket]` cues with free-form natural language. The previous-generation **S1** model uses the same emotion names but requires `(parentheses)` and a fixed tag set: +The **S2-Pro** and **S2.1-Pro** family models use `[bracket]` cues with free-form natural language. The previous-generation **S1** model uses the same emotion names but requires `(parentheses)` and a fixed tag set: ```text (happy) What a beautiful day! diff --git a/developer-guide/resources/agent-quickstart.mdx b/developer-guide/resources/agent-quickstart.mdx index e24b4cd..f287010 100644 --- a/developer-guide/resources/agent-quickstart.mdx +++ b/developer-guide/resources/agent-quickstart.mdx @@ -107,7 +107,7 @@ Not a coding agent installing a skill — an autonomous agent, RAG pipeline, or - Base API URL: `https://api.fish.audio` - Authentication: `Authorization: Bearer ` - - TTS model selection: send a required `model` header. Recommended default: `s2-pro` + - TTS model selection: optional `model` header (`s1`, `s2-pro`, `s2.1-pro`, `s2.1-pro-free`). Recommended: `s2.1-pro` for production, `s2.1-pro-free` for the free developer tier. If omitted, the server uses `s2.1-pro` - Main REST endpoints: - `POST /v1/tts` - `POST /v1/asr` diff --git a/features/text-to-speech.mdx b/features/text-to-speech.mdx index eaa091e..b2fee42 100644 --- a/features/text-to-speech.mdx +++ b/features/text-to-speech.mdx @@ -109,10 +109,10 @@ curl --request POST https://api.fish.audio/v1/tts \ - **`s2.1-pro`** — recommended for production, with improved quality, latency, and throughput over S2-Pro. - **`s2.1-pro-free`** — the same model at $0 for testing, prototyping, development, and smaller businesses, without TTFA or DPA guarantees. -- **`s2-pro`** (default) — previous-generation S2 model with multi-speaker and natural-language expression control. +- **`s2-pro`** — previous-generation S2 model with multi-speaker and natural-language expression control. - **`s1`** — previous generation, `(parenthesis)` emotion tags. -In the API, select with the `model` request header. In Python, pass `model="s2-pro"`. See [Choosing a Model](/developer-guide/models-pricing/choosing-a-model). +In the API, select with the `model` request header; requests without it use `s2.1-pro`. The Python SDK always sends a model and defaults to `model="s2-pro"`. See [Choosing a Model](/developer-guide/models-pricing/choosing-a-model). ### Output formats @@ -265,7 +265,7 @@ with open("out.mp3", "wb") as f: f.write(resp.content) ``` -The `model` header is required on every request. JSON and MessagePack accept the same fields. +The `model` header is optional — if omitted or set to an unrecognized value, the request falls back to `s2.1-pro`. JSON and MessagePack accept the same fields. ### Advanced generation tuning