面壁把语音生成压进2B参数,支持30种语言加方言克隆,还附完整技术报告和Apache 2.0开源,做语音产品的可以直接拉代码跑起来了。
面壁智能 OpenBMB 发布 VoxCPM2 技术报告。该模型为最新语音生成模型,拥有 2B 参数,基于超 200 万小时多语言语音数据训练,支持 30 种语言和 9 种中文方言。具备自然语言语音设计、可控及高保真延续性语音克隆能力。技术报告涵盖架构设计、统一序列公式、AudioVAE 高保真语音重建、大规模训练评估,以及零样本和指令跟随 TTS 基准结果。采用 16kHz 语义编码 + 48kHz 波形重建,在公开 TTS 基准上达到 SOTA 或极具竞争力。模型权重、微调代码和推理工具以 Apache 2.0 开源。
🚀 VoxCPM2 Technical Report is now available on arXiv!
VoxCPM2 is the latest speech generation model in the VoxCPM family. Built with 2B parameters and trained on over 2 million hours of multilingual speech data, it supports 30 languages and 9 Chinese dialects, along with natural-language voice design, controllable voice cloning, and high-fidelity continuation-based voice cloning.
In this technical report, we provide a comprehensive overview of: 🔹 The VoxCPM2 architecture 🔹 A unified sequence formulation for speech generation and control 🔹 The design of AudioVAE for high-fidelity speech reconstruction 🔹 Large-scale multilingual training and evaluation 🔹 Benchmark results across zero-shot and instruction-following TTS tasks
With 16kHz semantic encoding and 48kHz waveform reconstruction, VoxCPM2 delivers high-quality speech generation and achieves SOTA or highly competitive performance on public TTS benchmarks.
To support open research and development, we have open-sourced the model weights, fine-tuning code, and inference toolkit under the Apache 2.0 license.
📄 Paper: https://arxiv.org/abs/2606.06928 💻 GitHub: https://github.com/OpenBMB/VoxCPM We hope VoxCPM2 helps advance the open-source multilingual speech ecosystem. Feedback, experiments, and contributions are always welcome! 🔥 #AI #OpenSource #TTS #SpeechAI #VoiceAI #GenerativeAI #MachineLearning