Diversity here means narrative choices, not surface wording
The paper defines diverse short story generation as producing a set of high-quality stories for a single prompt that differ along salient narrative dimensions. It targets four of them: genre, tone, style, and named entities. This is deliberately broader than lexical or token-level variation, because two generations can differ in surface details while keeping the same underlying narrative trajectory and other high-level attributes.
That distinction matters for how the results should be read. A model that swaps synonyms or reorders sentences is not necessarily diverse in this sense. The paper's own framing is that low diversity in LLMs is largely attributed to the post-training process, where reinforcement learning from human feedback narrows the output distribution toward high-reward or typical responses, which can lead to mode collapse. The authors also note that simply raising sampling stochasticity through decoding methods such as temperature scaling does not necessarily improve diversity while coherence or quality drops.
Two phases broaden the distribution, then steer it
DivLM is a two-phase post-training framework. First, the authors perform continued pre-training (CPT) on a creative writing corpus and restore instruction-following capabilities using weight residuals. The paper reports that the proportion of the residuals added controls the trade-off between output diversity and quality. Second, they apply reinforcement learning with a custom, composite reward function that jointly maximizes diversity across the targeted narrative dimensions while maintaining response quality.
The reward is computed at the group level over multiple generations for the same prompt, which is what reinforces the set property the paper cares about rather than single-output fluency. The total diversity reward for an output is the average across four components: genre, style, tone, and named entities. The final reward combines quality and diversity with a threshold: diversity is only added once a minimum level of structural quality is reached, which the authors describe as inducing a form of curriculum learning, since early training optimizes a simpler quality-focused objective before the target expands to include diversity.
Continued pre-training alone trades quality for range
The results table separates the original instruct models (INS), the CPT-tuned models, and the full DivLM models. CPT models outperform INS models across all diversity metrics, and prompt adherence and coherence remain comparable. But completeness decreases substantially, which pulls the average quality score down: for Llama-3.1-8B the average quality falls from .922 for INS to .742 for CPT, and for Gemma-2-9B from .904 to .770.
The authors read this as evidence that CPT broadens the model's output distribution, and that the subsequent reinforcement learning phase is necessary to recover output quality. This is the paper's own interpretation of the two-phase design, and it is the reason the framework is presented as a pair of stages rather than a single intervention.
DivLM recovers quality while raising every diversity metric
After both phases, DivLM models yield quality metrics that remain close to the INS models while completeness improves substantially. For Llama-3.1-8B, average quality reaches .990 versus .922 for INS; for Gemma-2-9B, .990 versus .904. All diversity metrics improve. The RL rewards increase on average by 86.5% for Llama and 42% for Gemma compared to their INS baselines, and by 23.7% and 9.7% respectively over CPT.
The authors note that the smaller gains for Gemma may be partly because the hyperparameters were only optimized for Llama, and that the consistent improvement for Gemma nonetheless suggests the approach transfers across model families. A similar pattern appears in the coverage metrics, where CPT improves INS by 23.23% and DivLM improves over CPT by 20.65%. Cosine distance also increases, indicating greater semantic variation even though it was not directly optimized during training.
Coverage, reward components, and cosine distance answer different questions
The evaluation reports several distinct families of numbers, and they should not be substituted for one another. Coverage metrics (genre, tone, style) describe how much of each targeted dimension a model's outputs span. The reward components (genre, tone, style, named entities) are the group-level signals the training optimizes. Cosine distance is an embedding-based semantic distance that was not directly optimized. Quality metrics (coherence, prompt adherence, no meta-comments, completeness) measure separate properties, and the paper reports the no meta-commentary rate to allow meaningful averages.
The scale of the differences is visible in the table. For Llama-3.1-8B, DivLM reaches a diversity average of .672 versus .452 for INS, with style coverage rising from .542 to .801 and tone coverage from .603 to .832. For Gemma-2-9B, the diversity average is .621 versus .474, with style coverage rising from .542 to .811. These are coverage and reward-component measures, not quality measures, and the paper reports them alongside quality rather than trading one for the other.
Against DQO and DDPO, DivLM leads on diversity and quality
For Llama-3.1-8B the authors also compare against two methods from the literature, Diversity Quality Optimization (DQO) and Diversified Direct Preference Optimization (DDPO), using their datasets. DQO reaches a quality average of .983 and a diversity average of .447; DDPO reaches .534 quality and .617 diversity; DivLM reaches .990 quality and .672 diversity.
The comparison is informative because the alternatives split the two objectives. DQO preserves quality but barely moves diversity above the INS baseline of .452, while DDPO raises diversity but drops quality well below INS. DivLM is the only configuration in this table that improves both relative to the instruct baseline, which is the paper's central empirical claim.
The evidence covers short stories on models up to 9B parameters
The evaluation uses 100 test prompts with G=10 outputs per prompt, for a total of 1,000 outputs per model, across two LLM families. The authors state that due to computational and memory constraints, experiments are limited to models with at most 9B parameters, and that evaluating DivLM on larger models would provide a more complete understanding of its effectiveness and generalizability.
The study also focuses exclusively on short story generation and does not experiment with other forms of creative writing such as long-form stories and extended role-playing sessions, which introduce additional requirements such as narrative continuity and long-term memory. Finally, the paper addresses output diversity rather than creativity in its broader sense: diversity is the degree of variation across generated outputs and represents only one component of creative behavior. These boundaries qualify the headline gain rather than contradict it.
The quality threshold is what keeps diversity from being gamed
The most decision-relevant detail in the design is the quality threshold in the reward. Diversity is only rewarded once a minimum level of structural quality is achieved; below the threshold, the model is trained based only on a quality-driven signal. The authors explain that this prevents favoring low-quality but superficially diverse outputs, since incorrect outputs may otherwise artificially increase their reward by exploiting the diversity term. The threshold is calibrated on a validation set to reflect the minimum quality acceptable for the task.
For readers weighing whether to adopt this kind of post-training, the practical priority is therefore not the diversity reward alone but the pairing of a calibrated quality gate with group-level rewards over named narrative dimensions. The paper's evidence supports that pairing for short story generation on models up to 9B parameters; it does not establish that the same threshold or the same four dimensions transfer to longer or interactive writing, where the authors expect narrative continuity and long-term memory to interact with diversity in ways their current evaluation framework does not capture.
Sources used in this answer
Improving Diversity in LLM Short Story Generation
DivLM, a two-phase post-training framework combining continued pre-training on creative writing data with reinforcement learning over group-level diversity rewards, increases diversity metrics by more than 9% on average compared to alternative approaches on two LLM families while preserving instruction following, overall response quality, and similarity to human outputs.