Text-to-Speech’s Next Leap: From Lightning-Fast G2P to Expressive, Controllable, and Multilingual Voices

Latest 14 papers on text-to-speech: Aug. 15, 2026

The world of AI-driven voice synthesis is evolving at a breakneck pace, pushing the boundaries of what’s possible in generating natural, expressive, and highly controllable speech. Once a niche research area, Text-to-Speech (TTS) is now at the forefront of human-computer interaction, promising more intuitive and personalized experiences. Recent breakthroughs, as showcased in a flurry of innovative research papers, are tackling long-standing challenges from latency and intelligibility to cross-lingual voice cloning and fine-grained prosody control. Let’s dive into the essence of these advancements.

The Big Ideas & Core Innovations

At the heart of these innovations is a drive towards more efficient, robust, and controllable speech generation. Several papers highlight the critical role of efficient grapheme-to-phoneme (G2P) conversion and the power of continuous latent representations coupled with flow matching for high-fidelity audio.

Sub-millisecond G2P for Real-time Applications: The paper, FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelines, by Charin Polpanumas from AWS, addresses the crucial need for speed in real-time voice agents. Their solution achieves sub-millisecond phonemization for Thai, primarily by combining a massive IPA dictionary with optimized regex caching. This drastically reduces latency, enabling smaller models like Kokoro-82M StyleTTS 2 to run at 4x real-time on a CPU, making voice agents viable for edge devices.

Unifying Audio Generation with LLMs: Researchers from MiLM Plus, Xiaomi Inc., and X-LANCE Lab, Shanghai Jiao Tong University introduce MiDashengLM-Gen: Unified Audio Scene Generation via LLM-Driven Autoregressive Flow Matching. This groundbreaking framework uses Large Language Models (LLMs) with per-token conditional flow matching to generate complex mixed-audio scenes (speech, music, sound effects) with unprecedented speech intelligibility. A key insight is the necessity of audio-text alignment pre-training to bridge the modality gap, drastically improving speech WER.

Bridging Discrete and Continuous Spaces for Fidelity: A common thread is the innovative use of flow matching to bridge the gap between discrete linguistic tokens and continuous acoustic generation. Phoenix TTS: High-Fidelity Synthesis and Voice Conversion via Flow-Matching-Driven Speech Tokenization by L-Lab Phoenix-Audio Team (Didichuxing Co. Ltd) and Xiamen University, jointly trains a speech tokenizer with a Flow Matching decoder. This co-optimization forces discrete tokens to align natively with the high-fidelity generation space, enabling zero-shot voice conversion as a direct byproduct. Similarly, SemBridge: Semantic Token Anchoring for Continuous-Latent Autoregressive Speech Generation from Northwestern Polytechnical University and Soul AI Lab, uses discrete semantic tokens as training-only supervision for autoregressive LM states in continuous-latent generation, improving content fidelity without impacting the continuous inference process.

Multilingual and Cross-Lingual Capabilities: Advancements in multilingual voice synthesis are particularly exciting. Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder by NetEase Youdao, demonstrates transcript-free cross-lingual voice cloning across 14 languages using a jointly trained speaker encoder leveraging self-supervised speech representations. Luna-TTS Family Technical Report by VUI Labs Research Team, unveils a diffusion-language-model-based TTS system pretrained on a massive 1 million hours of speech across Chinese, English, Japanese, and Korean, achieving state-of-the-art zero-shot voice cloning and speech editing by treating them as infilling tasks. In a novel approach, Integrating Human Linguistic Insights into AI: Theory-Driven Representation for Multilingual Text-to-Speech from Newcastle University and University of Oxford, explores using the Featurally Underspecified Lexicon (FUL) with only 20 compact features as input, enabling intelligible speech synthesis with minimal data and even for unseen languages.

Fine-Grained Control and Expressiveness: The pursuit of expressive and controllable speech is evident in CookVoice: Unified Framework for Style Controllable Multi-Modal Human Voice Generation by UNSW Sydney and Dolby Laboratories, which decomposes voice into content, prosody, and style, enabling a single model for both speech and singing voice generation with superior control. CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis from The University of Texas at Austin and Amazon, introduces a framework for iterative refinement of global speaker identity and local phone-aligned prosodic attributes (pitch, loudness, duration) using continuous speech representations.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are underpinned by sophisticated architectures, vast datasets, and rigorous evaluation methods.

• FastThaiG2P leverages a 62,112-word IPA dictionary and text normalization, demonstrating its utility with the Kokoro-82M StyleTTS 2 model and the Som-TTS dataset. Code available at github.com/aws/FastThaiG2P.
• MiDashengLM-Gen integrates pre-trained LLMs (like Qwen3 backbone) with per-token flow matching, using high-dimensional semantic-acoustic latents and essential audio-text alignment pre-training. Available on Hugging Face (https://huggingface.co/mispeech/midashenglm-gen) and GitHub (https://github.com/xiaomi-research/midashenglm-gen).
• Phoenix TTS uses W2v-BERT 2.0 for semantic feature extraction and Qwen2.5-0.5B as its LLM backbone, evaluated on SeedTTS test sets and LibriSpeech-PC-test-clean.
• Confucius4-TTS relies on jointly trained speaker encoders with self-supervised speech representations (w2v-BERT 2.0) and evaluates on CV3-Eval, X-Voice, Seed-TTS-eval, and MiniMax-MLS-Test benchmarks. Code and models are open-sourced at https://github.com/netease-youdao/Confucius4-TTS.
• Luna-TTS Family is pretrained on an unprecedented 1 million hours of speech. It uses Qwen3-0.6B as an AR text LLM backbone and employs GRPO-style RL post-training for masked speech-token diffusion. Demos and paper are available at https://vuilabs-ai.github.io/luna-tts.
• CookVoice, a lightweight model with only 43.51 million parameters, utilizes a DiT-S backbone and flow matching. Demos and paper available at https://haoweilou.github.io/CookVoice/.
• CuteTTS is a compact 0.2B-parameter streaming TTS system using patch-level autoregression over continuous latents with a bidirectional flow-matching diffusion head, enhanced by semantic alignment using WavLM and explicit speaker conditioning via a distilled ECAPA encoder.
• CtrlSpeech is built on the DiTAR architecture with continuous speech representations, using Semantic-VAE checkpoints and CosyVoice speaker embedding model. It’s pretrained on Emilia and GigaSpeech datasets and evaluated on LibriSpeech-PC and Seed-TTS test-en.
• ReLMCodec employs a preserve–control–refine principle and PAPA (Pre-quantization Anchor-Preserving Adaptation) using W2v-BERT 2.0 and WavLM-Large for predictable speech tokens. Code available at https://github.com/ggiggit/ReLMCodec.
• GROW (https://github.com/yanghaha0908/GROW) improves autoregressive diffusion TTS models with group-relative advantage-weighted on-policy reinforcement learning and a Wasserstein-2 distance penalty, showing superior training efficiency over the DiTAR-GRPO baseline.

Crucially, a paper from NetEase Cloud Music, ASR-Roundtrip Evaluation Can Mask Context- and Convention-Dependent Reading Errors in Chinese News TTS, identifies a critical flaw in common ASR-roundtrip evaluations for Chinese news TTS, revealing that it can mask context-dependent reading errors. This is complemented by Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions by ServiceNow, which introduces the first dimension-level meta-evaluation benchmark for TTS, breaking down “naturalness” into 10 linguistic dimensions. Both highlight the need for more nuanced and human-centric evaluation protocols for sophisticated TTS systems.

Impact & The Road Ahead

These collective efforts are propelling TTS into an exciting future. We’re seeing the democratization of high-quality speech synthesis, enabling real-time voice agents on resource-constrained devices, unified models for complex audio generation, and the ability to clone and control voices across multiple languages with unprecedented fidelity and expressiveness. The focus on integrating linguistic insights and designing robust evaluation metrics ensures that these systems are not just technically advanced but also linguistically sound and truly helpful.

The road ahead involves further refinement of control mechanisms, greater data efficiency for low-resource languages, and more sophisticated evaluation tools that capture the subtleties of human speech. As LLMs become increasingly central, we can expect even more integrated and intelligent audio generation systems, blurring the lines between text, speech, and complete soundscapes. The promise of fully customizable, highly natural, and globally accessible synthetic voices is rapidly becoming a reality.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.