Can Kokoro-82M be distilled into an 8M-parameter single-voice text-to-speech model?

What Paradee trades away for a tenth of the parameters

Paradee is a single-voice student of Kokoro-82M, trained to speak only af_heart. It has 8.07M parameters against the teacher's 81.8M, needs 15x less compute, and stores as an 8.45MB int8 file against 32.5MB in fp32. On 200 held-out sentences it scores 4.41 on UTMOS against the teacher's 4.52, keeps the same 5.7% word error rate, and runs 25x faster than real time on one CPU thread.

The speed gain is smaller than the compute gain: 3.3x faster than the teacher in the main PyTorch comparison, because at this size PyTorch's per-operation overhead on small convolutions and LSTMs dominates. The phase-locking filter accounts for about 5% of run time. Int8 quantization cost nothing measurable in UTMOS, but quantizing to 4 bits lowered the decoder's UTMOS to 3.98 in an earlier test on 3 sentences, since a small model has no redundant channels left to absorb rounding.

Two halves trained separately against a frozen teacher

Kokoro has two halves: a text side that predicts durations, pitch, energy and phoneme features, and a decoder that turns them into audio. The paper first runs the teacher on 12,000 English sentences from WikiText-103, producing 23.9 hours of audio, and saves the phonemes, durations, F0 and energy contours, phoneme features and waveforms. Those saved outputs become the training data for both student halves.

Each half is then shrunk and trained against the frozen teacher, which only supplies targets. The student text side reads phonemes and learns to predict the teacher's saved durations, pitch, energy and phoneme features with L1 losses under teacher forcing. The student decoder is fed the teacher's saved features and learns to reproduce the teacher's waveform. Because each half trains against fixed teacher outputs, no alignment has to be learned, and the halves are joined afterward with no joint training.

Text-side size matters less than the training objective

Rendered by the frozen teacher decoder, a 4.2M text side scores UTMOS 4.36 and a 7.1M one scores 4.35, with DTW 0.91 against 0.89. Growing the text side improved every teacher-forced loss slightly but left the free-running result unchanged, so the remaining gap is set by the training objective rather than capacity.

The feature projection was a real limit. The teacher's 512 phoneme-feature channels are highly redundant: about 190 dimensions hold 90% of their variance and 430 hold 99%. A student with 192 channels and a linear projection can express at most 192 of those dimensions and came close to the best error any such projection allows (0.079 against 0.068). Replacing the linear projection with a small two-layer MLP removed this limit, cutting error to 0.066 and raising UTMOS from 4.15 to 4.36.

Decoder quality hinges on the balance of training losses

With spectral losses alone, the 3.85M decoder converged to a DTW distance of 0.50 but scored only 2.98 on UTMOS. The paper describes the full student at this point as the af_heart voice with a second, robotic voice speaking at the same time. Spectrograms show why: the student keeps crisp harmonic stripes up to 6-8kHz, whereas the teacher's harmonics dissolve into breathy noise above 3-4kHz, so the sine excitation leaks through almost unchanged. Level, frequency balance and silence were all correct.

Adversarial training fixes most of this, but only once the critics are allowed to matter. With HiFi-GAN's default spectral weight of 45, the spectral term outweighed the adversarial terms about 15 to 1 and UTMOS barely moved in 3,000 steps (2.98 to 3.02). Lowering the weight to 10 and then 3 raised the decoder to 4.37 and the full student to 4.39. Making the decoder wider or twice as large did not help.

Feature supervision transfers to a new decoder, decoder-pleasing does not

The paper also tried giving the text side an extra objective: pass its predictions through the frozen teacher decoder and compare the resulting audio with the teacher's recording using a log-mel loss. This lifted the 4.2M text side to the teacher's own score of 4.52 when rendered by the teacher decoder, but those features were specialized to that decoder. Connected to Paradee's student decoder, they scored 3.78, while the text side trained on the teacher's values directly scored 4.39 with the same decoder.

The interpretation the paper draws is that features learned by matching the teacher's features transfer to a new decoder, and features learned by pleasing one particular decoder do not. Paradee therefore uses the feature-supervised text side. The same pattern appears in the joined model: the student text side through the teacher decoder scores 4.36 and the teacher text side through the student decoder 4.37, each about 0.15 below the teacher, yet the full student scores 4.39 before the filter, so the two halves' errors do not add up.

The residual buzz sits in voiced phase between 2 and 8kHz

After adversarial training, the student's one audible flaw was a slight buzz on voiced sounds. To locate it, the paper combined the magnitude of the student's spectrogram with the teacher's phase, and the reverse, using identical excitation. The student's magnitude with the teacher's phase had no audible buzz (UTMOS 4.49, against 4.37 for the student and 4.52 for the teacher), while the teacher's magnitude with the student's phase still buzzed (4.45). Swapping in the teacher's phase only between 2 and 8kHz was enough to remove it.

That band starts just above the reach of the excitation tone, whose 9 harmonics end near 1.9kHz for this voice. Above that frequency the generator has to synthesize the harmonics itself, and the small student gets their timing slightly wrong. Changes to training did not remove the buzz: wider critics, direct waveform and phase losses, distilling the teacher's final layer, and larger decoders all left it audible.

The fix is applied after synthesis. The decoder already produces its own excitation tone with steady phase; in voiced frames and in the 2-8kHz band, the filter measures how far the output's phase drifts from this tone, smooths that drift over about a third of a second, and resynthesizes the audio with the smoothed phase and the original magnitude. The filter has no parameters and takes about 5% of run time. It adds little to UTMOS (4.39 to 4.41) but is what removes most of the audible buzz, with a slight buzz remaining on some stressed syllables where the student is 3-6dB quieter than the teacher between 2 and 8kHz.

Where Kokoro spends its parameters and compute

The paper maps the teacher's cost structure: the waveform generator is a quarter of the parameters but 89% of the compute, and the small text side is the part most sensitive to quantization. This is why the decoder, not the text side, is where the difficulty concentrates, and why the decoder's loss balance mattered more than its size.

Against three small open models on the same 200 held-out sentences, Paradee scores the highest UTMOS and has the smallest file. Kokoro-7M-Distill, the closest in design, runs about 1.4 times as fast but scores 0.23 lower on UTMOS, and KittenTTS has the lowest word error rate. Each model speaks its own voice, so UTMOS here compares naturalness, not closeness to af_heart. Paradee's speaker similarity to the teacher on the same sentences is 0.958, against 0.842 for a different Kokoro voice reading the same sentences.

What the single-voice result does and does not cover

Paradee speaks only a single English voice, and the authors did not train it to speak other voices or languages. It can only be as good as its teacher and inherits the teacher's mispronunciations. The quality evidence rests on an automatic predictor over 200 sentences and on informal listening by the author rather than a formal listening test, and speed was measured on one machine only. Besides the teacher, the comparison covers only three small open systems, each speaking a different voice.

The paper's own caution about its metric is worth carrying into any reading of the numbers: UTMOS rated one decoder 0.5 lower than another that sounded the same by ear, and it scores audibly buzzy output only 0.13 below the teacher. The most decision-relevant split in this paper is therefore not parameter count but where the compression cost lands: the text side compresses easily, while the waveform generator's phase above the excitation band is the part that needed a training-free filter to fix. For a single-voice deployment, the practical threshold the paper establishes is that int8 weights are free in UTMOS while 4-bit weights are not, and that decoder loss weighting, not decoder width, is what moves quality.

Sources used in this answer