Fish Audio S2
Dual-AR architecture, 4B parameter, high-quality text-to-speech model.
Fish Audio S2 is an open-source TTS (Text-to-Speech) model released by Fish Audio in March 2026. In a nutshell, it's a multilingual voice generator that allows you to inline specify laughter, whispering, and professional broadcast tones using 15,000 emotion tags.
Existing zero-shot TTS models either read text in a monotonous tone or required a separate prosody model to add emotion. Fish Audio S2 uses a Dual-AR architecture (4B Slow AR + 400M Fast AR) to separately predict the semantic codebook and residual voice codebook, allowing it to receive emotion and prosody tags in a natural language inline format. You can incorporate tone changes within a single sentence, such as [laugh] Hello [professional broadcast tone] This is today's news, and it achieved a Bradley-Terry score of 1st place (3.07) in a blind evaluation.
From a practical perspective, it (1) covers 80+ languages with a single model, allowing you to handle multilingual content dubbing with one model; (2) creates zero-shot voice clones with only 10-30 seconds of reference audio, generating consistent speaker voices for each character; and (3) with an RTF of 0.195 (H200 1 card) and TTFA of ~100ms, it can be directly integrated into the backend of interactive voice agents for real-time inference.
💻 System Requirements
24GB recommended (for inference). CPU-only mode is possible, but real-time processing will not be available.
Model checkpoint size is approximately 8-15GB; the entire package, including dependencies, requires about 20GB.
⚡ Installation
4-1. Quick Start
# Create a Conda environment + install GPU support
conda create -n fish-speech python=3.12
conda activate fish-speech
pip install -e .[cu129]
4-2. Detailed Installation
# System dependencies (Linux)
apt install portaudio19-dev libsox-dev ffmpeg
# UV package manager (for faster dependency resolution)
uv sync --python 3.12 --extra cu129 # GPU
uv sync --python 3.12 --extra cpu # CPU only
# Docker Compose (recommended deployment method)
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech
docker compose --profile webui up # WebUI (localhost:7860)
docker compose --profile server up # API server (localhost:8080)
COMPILE=1 docker compose --profile webui up # Optimized build (~10x speed improvement)
4-3. Basic Usage Example
# CLI inference (clone voice using a reference audio)
# Download the model checkpoint to the checkpoints/ directory first:
# fish-speech infer --text "Hello" --reference ./ref_audio.wav
🧬 Bio Use Cases
Real-time Multilingual Text-to-Speech
Process over 80 languages with a single model, enabling global content localization. Clone native speakers with a 10-second reference audio and automatically generate multilingual dubbing. With an RTF of 0.195, it can be deployed in real-time for streaming services.
Create Emotionally Rich Audiobooks/Podcasts
Generate emotionally rich narrations beyond monotonous TTS using 15,000+ inline tags such as [whispers], [excited], and [laugh]. Automatically create podcast-style content with multi-speaker dialogue functionality.
Generate Voice Data for Research
Utilize high-quality synthesized voice with Seed-TTS Eval WER of 0.54% (Chinese) / 0.99% (English) to augment ASR model training data, build multilingual voice corpora, and prototype voice interfaces.
FAQ
What is Fish Audio S2?
Fish Audio S2 is an open-source TTS (Text-to-Speech) model released by Fish Audio in March 2026. In a nutshell, it's a multilingual voice generator that allows you to inline specify laughter, whispering, and professional broadcast tones using 15,000 emotion tags. Existing zero-shot TTS models either read text in a monotonous tone or required a separate prosody model to add emotion. Fish Audio S2 uses a Dual-AR architecture (4B Slow AR + 400M Fast AR) to separately predict the semantic codebook and residual voice codebook, allowing it to receive emotion and prosody tags in a natural language inline format. You can incorporate tone changes within a single sentence, such as [laugh] Hello [professional broadcast tone] This is today's news, and it achieved a Bradley-Terry score of 1st place (3.07) in a blind evaluation. From a practical perspective, it (1) covers 80+ languages with a single model, allowing you to handle multilingual content dubbing with one model; (2) creates zero-shot voice clones with only 10-30 seconds of reference audio, generating consistent speaker voices for each character; and (3) with an RTF of 0.195 (H200 1 card) and TTFA of ~100ms, it can be directly integrated into the backend of interactive voice agents for real-time inference.
When should I use Fish Audio S2?
Dual-AR architecture, 4B parameter, high-quality text-to-speech model.
What is a biomedical use case for Fish Audio S2?
Real-time Multilingual Text-to-Speech: Process over 80 languages with a single model, enabling global content localization. Clone native speakers with a 10-second reference audio and automatically generate multilingual dubbing. With an RTF of 0.195, it can be deployed in real-time for streaming services.
📝 Update Notes
- vv1.5.16/16/2026
이번 v1.5.1 업데이트는 차세대 모델 출시를 앞두고 시스템의 안정성을 극대화한 마지막 안정화 버전이에요. 새로운 기능 추가보다는 현재 진행 중인 연구 워크플로우의 신뢰도를 높이는 데 집중했습니다. 실험 데이터의 음성 변환이나 발표 자료 제작 등, 오류 없는 안정적인 작업 환경이 필수적인 연구원분들께 이 버전을 추천드려요.
🧪 Related Code of Life
No related Code of Life posts yet.