LiteGUI: Unlocking 2B-scale GUI Agents via SFT-Free Reinforcement Learning

TL;DR

On-device GUI automation has long been a "large model's game" due to the extreme reasoning requirements of computer use. LiteGUI flips the script by proving that 2B-3B scale models can outperform 72B giants. By ditching the traditional Supervised Fine-Tuning (SFT) in favor of Guided On-policy Distillation and Multi-solution RL, LiteGUI achieves State-of-the-Art performance while remaining efficient enough for local deployment.

Problem: The "Rigidity" of Small Models

Small-scale models (2B–3B) typically fail at GUI tasks because of Policy Rigidity:

• SFT Overfitting: Standard imitation learning makes models memorize specific click coordinates. If the UI shifts slightly, the model breaks.
• Catastrophic Forgetting: Tuning for GUI actions often destroys the model's base visual reasoning.
• Single-Path Bias: Traditional training rewards the model for matching one specific expert path, punishing it for valid alternative solutions (e.g., "Enter" vs. "Double Click").

Methodology: The Two-Stage Paradigm

The authors propose a clean break from SFT, moving directly to a two-stage exploration-based training flow.

1. Guided On-policy Distillation (Guided-OPD)

Instead of offline distillation, the student generates its own actions. A teacher model (Qwen3-VL-32B) evaluates these on-policy actions. To prevent teacher hallucinations, the authors introduce Most-Matched-GT Retrieval: the system looks at the student's intent and retrieves the most similar valid ground-truth from a pool to "guide" the teacher's feedback.

2. Multi-solution Dual-level GRPO (MD-GRPO)

This second stage uses Group Relative Policy Optimization (GRPO) with a reinforced reward structure:

• Micro Level (Action): Rewards the model if it matches any valid action in a multi-solution set, reducing false negatives.
• Macro Level (Planning): A VLM judge (Qwen3-VL-32B) scores the "Subtask Plan." It checks if the agent understands why it is doing a step and if it can recover from errors.

Figure 1: The LiteGUI training pipeline, showcasing the ATG data generation and the dual-stage distillation/RL paradigm.

Experimental Breakthroughs

The results are striking, particularly in the efficiency of the 2B model.

• ScreenSpot-Pro: LiteGUI-2B (46.86%) outperforms specialized 72B models (35.7%).
• OS-World: Performance doubled over the base model.
• Data Efficiency: LiteGUI achieved these results with only 8K samples, whereas competing models often require hundreds of thousands.

Table 4: Ablation study showing that the combination of Guided-OPD and Multi-solution RL provides the largest gain over baseline and SFT.

Critical Insight & Conclusion

The core contribution here isn't just a better model, but a better recipe. The industry has relied on SFT as a "crutch" for too long. LiteGUI demonstrates that on-policy exploration, specifically when guided by multi-solution rewards, is the key to making "small" models behave with "large" intelligence.

Limitations: The reliance on a 32B VLM judge for the planning reward and the manual effort of multi-solution annotation remain bottlenecks for scaling to every possible OS version. However, for the first time, we have a viable path for high-performance, private, on-device autonomous computer use.