[ArXiv 2025] FlexGuard: Beyond Binary Safety — Mastering Strictness-Adaptive LLM Moderation
FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation
FlexGuard is a continuous risk-scoring LLM moderator designed for strictness-adaptive content safety tasks. It replaces traditional binary classification with a calibrated continuous score [0, 100], allowing for flexible safety enforcement across varying platforms and strictness regimes (Strict, Moderate, Loose).
TL;DR
Content safety isn't a "one-size-fits-all" binary switch. A post that is "safe" on a liberal forum might be "unsafe" in a classroom setting. FlexGuard breaks the binary mold by outputting a calibrated continuous risk score (0-100). By introducing FlexBench, the first benchmark for strictness-adaptive evaluation, the authors demonstrate that FlexGuard maintains high accuracy where traditional models (like LlamaGuard 3) fail when enforcement policies shift.
1. The Core Tension: Static Models vs. Fluid Policies
Modern LLM moderators are typically trained on fixed binary datasets (Safe vs. Unsafe). However, in the real world:
- Context Matters: A professional surgical explanation is "Safe," while a DIY harm-tutorial is "Unsafe," yet the keywords overlap.
- Policies Evolve: Platforms frequently update their "strictness" levels based on regulatory changes or community feedback.
The authors find that when you take a SOTA model like Qwen3Guard and try to adapt it to a "Loose" setting, its F1 score plummets by 19.2%. This brittleness occurs because the model has no internal concept of severity—only a rigid boundary.
2. FlexBench: Evaluating the Moving Target
To measure this, the authors created FlexBench, mapping 4,000 instances into five severity tiers:
- Benign
- Low
- Moderate
- High
- Extreme
These tiers allow for three operational regimes: Strict (flags almost everything), Moderate, and Loose (flags only high/extreme risks).
Figure: The pipeline for constructing FlexBench and the architecture of FlexGuard.
3. Methodology: How FlexGuard Learns Severity
The magic of FlexGuard lies in how it is supervised and aligned.
A. Rubric-Guided Distillation
The team used Doubao-1.6-Pro (a high-performing judge) to annotate raw data with risk scores based on explicit rubrics. To ensure these scores didn't drift, they applied Label-Consistent Calibration, forcing the raw scores to remain within specific intervals (e.g., Safe content must stay between 0-40, Unsafe between 41-100).
B. GRPO Risk Alignment
Unlike standard Supervised Fine-Tuning (SFT), FlexGuard uses Group Relative Policy Optimization (GRPO). The reward function is particularly clever:
- Category Reward: +1 for correct category (e.g., VIO).
- Score Reward: A dense signal based on the absolute error between the model's predicted score and the distilled target score.
This forces the model to not just pick the right label, but to justify its "riskiness" level throughout its reasoning chain.
4. Key Results: Stability in the Face of Change
FlexGuard doesn't just win on average; it wins on the "Worst-Case" scenario.
| Method | Avg F1 (Prompt) | Worst-Regime F1 |
|---|---|---|
| GPT-5 | 73.26 | 70.95 |
| Qwen3Guard-8B | 75.10 | 67.06 |
| FlexGuard | 81.78 | 78.26 |
The results show that FlexGuard’s calibrated thresholding allows it to adapt its decision boundary without retraining, maintaining a stable performance profile across all three strictness levels.
Figure: Notice how FlexGuard (likely the rightmost bars in a full chart) maintains a much flatter profile across regimes compared to the sharp drops in Qwen3Guard or BingoGuard.
5. Critical Insight: Is Reasoning Enough?
An interesting finding in the ablation study was the role of the Score-Regression Reward. When the authors tried to train using GRPO with only categorical rewards (ignoring the continuous score), performance degraded significantly. This proves that nuanced severity prediction is a prerequisite for robust safety reasoning, not just an optional feature.
Interpretability Example
In a test case where a user asks for "maximum physical pain" methods:
- Binary Model: Says "Unsafe."
- FlexGuard: Reasons through the "waterboarding" and "sensory deprivation" descriptions, calculates a Score of 94, and classifies it as Extreme VIO.
6. Closing Thoughts
FlexGuard represents a move toward "Intent-Aware" and "Policy-Aware" safety. By providing two deployment strategies—Rubric Thresholding (semantic) and Calibrated Thresholding (data-driven)—it gives platform engineers a knob to turn, balancing the trade-off between safety and helpfulness in real-time.
Limitations to watch: The current work is English-only. Future SOTA work will need to prove if these continuous "severity" concepts translate across cultures and languages where the definition of "Harm" is even more fragmented.
Senior Editor's Take: FlexGuard's use of GRPO for score regression is a sophisticated application of RLHF principles to a traditionally discriminative task. It’s a blueprint for the next generation of industrial guardrails.