AI Tools
Audio AIAdvanced

Fish Audio S2

Dual-AR architecture, 4B parameter, high-quality text-to-speech model.

Fish Audio S2 is an open-source TTS (Text-to-Speech) model released by Fish Audio in March 2026. In a nutshell, it's a multilingual voice generator that allows you to inline specify laughter, whispering, and professional broadcast tones using 15,000 emotion tags.

Existing zero-shot TTS models either read text in a monotonous tone or required a separate prosody model to add emotion. Fish Audio S2 uses a Dual-AR architecture (4B Slow AR + 400M Fast AR) to separately predict the semantic codebook and residual voice codebook, allowing it to receive emotion and prosody tags in a natural language inline format. You can incorporate tone changes within a single sentence, such as [laugh] Hello [professional broadcast tone] This is today's news, and it achieved a Bradley-Terry score of 1st place (3.07) in a blind evaluation.

From a practical perspective, it (1) covers 80+ languages with a single model, allowing you to handle multilingual content dubbing with one model; (2) creates zero-shot voice clones with only 10-30 seconds of reference audio, generating consistent speaker voices for each character; and (3) with an RTF of 0.195 (H200 1 card) and TTFA of ~100ms, it can be directly integrated into the backend of interactive voice agents for real-time inference.

💻 System Requirements

🧠RAM

24GB recommended (for inference). CPU-only mode is possible, but real-time processing will not be available.

💾Storage

Model checkpoint size is approximately 8-15GB; the entire package, including dependencies, requires about 20GB.

Installation

4-1. Quick Start

# Create a Conda environment + install GPU support
conda create -n fish-speech python=3.12
conda activate fish-speech
pip install -e .[cu129]

4-2. Detailed Installation

# System dependencies (Linux)
apt install portaudio19-dev libsox-dev ffmpeg

# UV package manager (for faster dependency resolution)
uv sync --python 3.12 --extra cu129   # GPU
uv sync --python 3.12 --extra cpu     # CPU only

# Docker Compose (recommended deployment method)
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech
docker compose --profile webui up              # WebUI (localhost:7860)
docker compose --profile server up             # API server (localhost:8080)
COMPILE=1 docker compose --profile webui up    # Optimized build (~10x speed improvement)

4-3. Basic Usage Example

# CLI inference (clone voice using a reference audio)
# Download the model checkpoint to the checkpoints/ directory first:
# fish-speech infer --text "Hello" --reference ./ref_audio.wav

🧬 Bio Use Cases

🔬

Real-time Multilingual Text-to-Speech

Process over 80 languages with a single model, enabling global content localization. Clone native speakers with a 10-second reference audio and automatically generate multilingual dubbing. With an RTF of 0.195, it can be deployed in real-time for streaming services.

🧬

Create Emotionally Rich Audiobooks/Podcasts

Generate emotionally rich narrations beyond monotonous TTS using 15,000+ inline tags such as [whispers], [excited], and [laugh]. Automatically create podcast-style content with multi-speaker dialogue functionality.

💊

Generate Voice Data for Research

Utilize high-quality synthesized voice with Seed-TTS Eval WER of 0.54% (Chinese) / 0.99% (English) to augment ASR model training data, build multilingual voice corpora, and prototype voice interfaces.

FAQ

What is Fish Audio S2?

Fish Audio S2 is an open-source TTS (Text-to-Speech) model released by Fish Audio in March 2026. In a nutshell, it's a multilingual voice generator that allows you to inline specify laughter, whispering, and professional broadcast tones using 15,000 emotion tags. Existing zero-shot TTS models either read text in a monotonous tone or required a separate prosody model to add emotion. Fish Audio S2 uses a Dual-AR architecture (4B Slow AR + 400M Fast AR) to separately predict the semantic codebook and residual voice codebook, allowing it to receive emotion and prosody tags in a natural language inline format. You can incorporate tone changes within a single sentence, such as [laugh] Hello [professional broadcast tone] This is today's news, and it achieved a Bradley-Terry score of 1st place (3.07) in a blind evaluation. From a practical perspective, it (1) covers 80+ languages with a single model, allowing you to handle multilingual content dubbing with one model; (2) creates zero-shot voice clones with only 10-30 seconds of reference audio, generating consistent speaker voices for each character; and (3) with an RTF of 0.195 (H200 1 card) and TTFA of ~100ms, it can be directly integrated into the backend of interactive voice agents for real-time inference.

When should I use Fish Audio S2?

Dual-AR architecture, 4B parameter, high-quality text-to-speech model.

What is a biomedical use case for Fish Audio S2?

Real-time Multilingual Text-to-Speech: Process over 80 languages with a single model, enabling global content localization. Clone native speakers with a 10-second reference audio and automatically generate multilingual dubbing. With an RTF of 0.195, it can be deployed in real-time for streaming services.

📄 Official Docs🐙 GitHub

📝 Update Notes

  1. vv1.5.16/16/2026

    이번 v1.5.1 업데이트는 차세대 모델 출시를 앞두고 시스템의 안정성을 극대화한 마지막 안정화 버전이에요. 새로운 기능 추가보다는 현재 진행 중인 연구 워크플로우의 신뢰도를 높이는 데 집중했습니다. 실험 데이터의 음성 변환이나 발표 자료 제작 등, 오류 없는 안정적인 작업 환경이 필수적인 연구원분들께 이 버전을 추천드려요.

🧪 Related Code of Life

No related Code of Life posts yet.