What can users actually control right now?
The most mature control is over speaking rate. FastSpeech, a 2024 parallel TTS model, can adjust voice speed smoothly and nearly eliminates word skipping/repeating, while speeding up mel-spectrogram generation by 270x and end-to-end synthesis by 38x compared to autoregressive models [1]. That means users can get faster, more reliable speech with adjustable tempo—a big win for accessibility and on-the-fly applications.
But precise control over utterance duration—like making a sentence exactly 3 seconds long—is still hard. A 2024 study introduced a masked duration model that predicts phone durations from audio and text prompts, allowing fine-grained duration control, but it required adversarial training to keep alignment robust, and it still underperformed in naturalness compared to unconstrained synthesis [2]. So while speed control is smooth, exact duration control is a work in progress.
Can you just describe the voice you want?
Yes, and this is a hot research direction. PromptTTS (2023) lets users type a prompt like 'A lady whispers to her friend slowly' and generates matching speech, using separate style and content encoders [4]. This is more user-friendly than adjusting acoustic parameters like pitch or prosody, which require technical knowledge. However, the study had to create a new dataset because none existed, and the evaluation was limited to style control quality—not real-world robustness.
A 2025 study extended this idea to cross-lingual control: they trained a TTS model on Japanese and a description-control model on English, sharing language-agnostic timbre and style representations, so users could describe a voice in English and get Japanese speech with that style [3]. This works because the representations are based on self-supervised learning, which doesn't depend on language. But the study only tested English and Japanese, and the lack of Japanese audio-description pairs means the cross-lingual transfer is still a proof-of-concept, not a production-ready feature.
Can we edit speech after it's generated?
Yes, but with limitations. EdiTTS (2022) uses score-based diffusion models to edit pitch and content in existing speech without retraining, by applying targeted perturbations and masks to only the target region [5]. It outperformed baselines in listening tests and speech-to-text accuracy, but it's an off-the-shelf method that requires careful tuning of perturbation strength and masks—so it's not yet a plug-and-play tool for non-experts.
The gap between best-case and typical-case evidence is clear: FastSpeech shows impressive speed and robustness in controlled settings, but real-world speech is messier, and the other studies show that precise control often comes at the cost of naturalness or requires extra training data [1][2][3][4]. So while the field has made leaps, the research questions that naturally follow are about robustness, generalization, and user-friendly interfaces—not just raw capability.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2022 to 2025, 3 from 2024 or later, 1 in Q1–Q2 journals, collectively cited 405 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 36 papers retrieved from a database of over 500 million.
Sources used in this answer
FastSpeech: Fast, Robust and Controllable Text to Speech
FastSpeech, a parallel feed-forward TTS model, matches autoregressive quality while speeding up mel-spectrogram generation by 270x and end-to-end synthesis by 38x, and nearly eliminates word skipping/repeating, with smooth voice speed adjustment.
Masked Duration Model for Utterance Duration-Controllable Text-to-Speech
A masked duration model enables fine-grained utterance duration control by predicting phone durations from audio/text prompts and masks, but requires adversarial training for robust alignment; it outperformed a baseline in duration control but naturalness remains a challenge.
Description-Based Controllable Text-to-Speech With Cross-Lingual Voice Control
A description-based TTS method achieves cross-lingual voice control by combining a target-language TTS with a description-control model from another language, using SSL-based language-agnostic timbre/style representations; tested on English and Japanese, it achieves high naturalness and controllability without Japanese audio-description pairs.
Prompttts: Controllable Text-To-Speech With Text Descriptions
PromptTTS uses text descriptions (style and content) as prompts to synthesize speech, with a style encoder and content encoder; it demonstrates precise style control and high quality, but required constructing a new dataset because none existed.
EdiTTS: Score-based Editing for Controllable Text-to-Speech
EdiTTS applies score-based diffusion to edit pitch and content in existing speech without additional training, using perturbations and masks; it outperformed baselines in listening tests and speech-to-text accuracy, but requires careful tuning.