The dilemma OpenPhone is built to break
Mobile GUI agents sit between two bad options. Models small enough to actually run on a phone, 4B parameters or fewer, lack sufficient GUI performance; models that are capable enough, starting around 7B, are either too large for on-device deployment or, if they are closed-source cloud models, prohibitively costly. The paper frames this as a capability-versus-cost gap with no suitable approach in the usable region.
The authors' proposed resolution is not to pick a side but to combine them: use a compact on-device model as the default executor and reserve the cloud model for subtasks that a real-time complexity assessment flags as difficult. The claim to test is whether that split preserves task completion while cutting cloud spend.
What the 3B on-device agent is trained to do
The on-device agent is Qwen2.5-VL-3B, enhanced through a two-stage SFT-then-GRPO training pipeline on synthetic GUI data. The goal is strong decision-making at a footprint small enough for a smartphone, rather than raw scale.
Two mechanisms support that. First, extended chain-of-thought reasoning at test time: the agent analyzes actionable elements on the current screen, checks whether prior actions met their goals, evaluates progress toward the user's task, and then selects a function and parameters. If history shows a prior operation failed, the model is trained to reflect and adjust rather than repeat the error. Second, efficient memory management: instead of storing raw high-resolution screenshots, which consume too many tokens, each step is compressed into a compact textual state assessment covering the current interface state, task progress, the inferred next action, expected outcome, and potential issues. The paper states this lets the agent retain roughly 10-20 steps of history as context under tight on-device resources.
How escalation to the cloud is decided
The device-cloud layer is not a fixed rule but a monitoring policy. A task complexity assessment decides when monitoring starts, how often it runs, and whether a high-risk task should go to the cloud from the beginning. A separate dynamic orchestration policy then decides, step by step, whether the cloud model should take over.
The switching criteria are concrete. Immediate triggers include detecting a critical-failure app, a complex task pattern such as financial data entry or navigation, early confusion in the first three steps, or navigation to a completely irrelevant app. Progressive triggers include the same action repeated two or more times without progress, apparent loss of direction, or misreading the current app state. The stated priority is task completion over cost optimization, which is why the policy is willing to escalate early for high-risk cases.
What the AndroidLab results actually show
Evaluation runs on the online AndroidLab benchmark, which covers nine commonly used applications and 138 evaluation tasks, plus 25 additional tasks across Gmail, Chrome, Reddit, and TikTok. All reported results use the Set-of-Mark input mode, where each clickable element gets a unique index and the model acts by index; the XML mode is for text-only models and is not the reported setting.
The headline result is that OpenPhone matches or nears larger models on this benchmark while significantly reducing cloud costs. Two boundaries matter for reading that claim. The comparison is against larger models, not a claim of outright superiority, and the cost reduction is relative to relying on cloud models, not an absolute per-task dollar figure in the extracted evidence. The 25 extra tasks are a supplement to AndroidLab, not a replacement denominator.
Why the two-stage training matters
The reward design explains what the small model is being pushed toward. Accuracy reward is task-dependent: operation tasks such as Tap(index) require strict matching between predicted and ground-truth function calls, while query tasks such as Finish(answer) use embedding-based similarity and grant reward only when similarity reaches a threshold. Format reward gives partial credit for conforming to the three-block output template and applies a multiplicative penalty for content generated outside the template, a mechanism the authors describe as targeting the irrelevant generation common in small-scale MLLM training.
GRPO is used instead of PPO because it avoids a separate value function: for each question, a group of outputs is sampled and the average group reward serves as the baseline, with advantages computed relative to that group. This matters for interpretation because it means the on-device agent's gains come from training and test-time reasoning on synthetic GUI data, not from a larger backbone.
Where the collaboration still leans on the cloud
The authors are explicit about the limit. Because current 2B-3B vision-language base models remain weak, GUI agents trained on them still struggle to independently perform complex mobile tasks, and in this system that produces a significant dependence on cloud-based models. The device-cloud framework is presented as a feasible future paradigm that should become more practical and valuable as base models improve, not as a solved problem today.
That limitation sits directly on the cost claim: the more often the orchestration policy escalates, the less of the cost advantage is realized. The paper's own risk tiers illustrate the pressure, listing apps with 0-15% on-device completion as critical-failure cases that may warrant immediate cloud use.
The real decision is the escalation threshold
Read as a whole, OpenPhone's contribution is less about the 3B model alone than about where the line between device and cloud is drawn. The paper's own framing puts task completion ahead of cost optimization, and its risk tiers, monitoring start points, and monitoring frequencies are the variables that actually set that line. A reader comparing this approach to a cloud-only or on-device-only design should look first at the escalation policy, because that is what determines whether the cost saving survives contact with hard tasks.
The most useful takeaway is therefore conditional rather than absolute: device-cloud collaboration can deliver both capability and lower cost when the on-device model handles the routine majority and escalation is reserved for genuinely hard subtasks, and its advantage narrows exactly to the extent that weak 2B-3B base models force the cloud to take over.
Sources used in this answer
OpenPhone: Mobile Agentic Foundation Models
OpenPhone trains a 3B on-device GUI agent and escalates only hard subtasks to a cloud model, matching or nearing larger models on the online AndroidLab benchmark while significantly reducing cloud costs, though weak 2B-3B base models still force significant cloud dependence.