TADA (Text-Acoustic Dual Alignment)
High-quality TTS model using a text-to-audio 1:1 token alignment method.
- Text-Acoustic Dual Alignment (1:1 token alignment): Synchronizes text tokens and audio vectors in a 1:1 manner, eliminating persistent issues in conventional TTS, such as word omission, repetition, and hallucination, at the architectural level. Achieved zero content hallucination in over 1,000 LibriTTSR tests. - 5x Faster Inference Speed: With a Real-Time Factor (RTF) of 0.09, it is approximately 5 times faster than comparable LLM-based TTS. While existing systems process 12.5 to 75 audio tokens per second, TADA operates at 2-3 frames per second while maintaining high-quality speech synthesis. With cache utilization on H100, RTF is approximately 0.12x. - 10x More Efficient Context Window: Covers approximately 700 seconds (approximately 11.7 minutes) of audio with 2,048 tokens. This is 10 times more efficient than the approximately 70 seconds of existing systems. Advantageous for long-form narration, audiobooks, and generating long conversations. - Multilingual Support (10 languages): Supports English, Arabic, Chinese, German, Spanish, French, Italian, Japanese, Polish, and Portuguese based on the 3B-ML model. Expandable with language-specific aligner modules. - Dual-Stream Generation: Generates text and audio simultaneously, and speech quality can be controlled through logit blending using Speech Free Guidance (text-only guidance). - Prompt Caching: Caches the reference speech encoding results using `EncoderOutput.save()/load()`, saving approximately 2.5GB of VRAM and improving speed during repetitive inference. - Flow Matching-Based Decoding: Restores high-fidelity audio using a flow-matching head conditioned on the LLM hidden state. By reducing the flow matching steps from 20 to 10, it achieves an additional speed improvement of approximately 1.3x without degrading quality.
💻 System Requirements
For the 3B-ML model, approximately 9GB of memory is required with bf16 precision. NVIDIA GPU with CUDA compatibility is essential. An RTX 3060 (12GB) or higher allows running the 3B model with bf16 precision. The 1B model can operate with lower VRAM.
The 3B-ML model weights require approximately 8GB, the 1B model requires approximately 2GB, and the codec (tada-codec) requires approximately 500MB. The entire package requires approximately 10-12GB.
⚡ Installation
### 4-1. Quick Start
```bash
pip install hume-tada
```
### 4-2. Build from Source
```bash
git clone https://github.com/HumeAI/tada.git
cd tada
pip install -e .
```
### 4-3. Basic Usage (Python)
```python
from tada.modules.encoder import Encoder
from tada.modules.tada import TadaForCausalLM
import torch, torchaudio
# Load the model (3B multilingual, bf16 precision)
encoder = Encoder.from_pretrained("HumeAI/tada-codec",
subfolder="encoder").to("cuda")
model = TadaForCausalLM.from_pretrained("HumeAI/tada-3b-ml",
torch_dtype=torch.bfloat16).to("cuda")
# Speaker style prompting with reference audio
audio, sr = torchaudio.load("reference.wav")
prompt = encoder(audio, text=["reference text"], sample_rate=sr)
# Generate speech from new text
output = model.generate(prompt=prompt, text="Text to synthesize")
```
> **Note**: You must pre-accept the Meta Llama 3.2 Community License on HuggingFace to download the model weights.📝 Update Notes
No update notes yet.
🧪 Related Code of Life
No related Code of Life posts yet.