CosyVoice 2, released in December 2024 by the Alibaba FunAudioLLM team, is a state-of-the-art, deep-learning-based streaming text-to-speech (TTS) and voice cloning open-source engine. This tool fuses a large language model (LLM) with a chunk-aware causal flow matching architecture, enabling it to perfectly reproduce a speaker's unique tone, emotional nuances, and breathing patterns using only a short reference audio clip of approximately 3 seconds. Users can leverage this system to generate human-quality, high-resolution audio output in real-time based on text input, with seamless support for multilingual synthesis and code-switching. Traditional speech synthesis systems required long recordings and complex fine-tuning processes to achieve natural voice cloning, and they faced limitations in establishing ultra-low-latency streaming environments suitable for real-time conversations, necessitating the implementation of separate, dual optimization models. In contrast, CosyVoice 2 utilizes finite-scalar quantization technology to encode speech signals into dense, discrete tokens, efficiently separating the semantic information of phonemes and the tonal information of the speaker. Similar to how GPT predicts text tokens to complete the next sentence, this model implements dynamic flow matching under text conditions, functioning like a "GPT for voice" that seamlessly handles high-quality batch generation and ultra-low-latency streaming output (below 150ms) within a single pipeline. In the fields of biotechnology and clinical medical research, CosyVoice 2 can be effectively used to convert academic paper summaries or patient education-related medical consultation data into automated audiobooks and multilingual dubbing files. When distributing research presentation materials at non-English-speaking seminars, providing only a 3-second recording of the presenter allows for the natural conversion and generation of audiobooks in nine languages, including English, Chinese, and Japanese, while preserving the presenter's original voice. Furthermore, when reciting complex gene structures or immune response data, the system can precisely control the intonation, speed, and emotion of the sentences, maximizing the immersive experience of the auditory material and enhancing the research convenience for visually impaired researchers.

Released in December 2024 by the Alibaba FunAudioLLM team, CosyVoice 2 is a state-of-the-art, deep learning-based streaming text-to-speech (TTS) and voice cloning open-source engine. This tool fuses a large language model (LLM) with a chunk-aware causal flow matching architecture, allowing it to perfectly reproduce a speaker's unique timbre, emotional nuances, and breathing patterns using only a short reference audio clip of approximately 3 seconds. Users can leverage this system to generate high-resolution audio that is comparable to human speech based on text input.