RecGPT-Mobile: Bringing LLM Reasoning to the Edge of E-Commerce
RecGPT-Mobile: On-Device Large Language Models for User Intent Understanding in Taobao Feed Recommendation
RecGPT-Mobile is a pioneering framework for deploying lightweight Large Language Models (LLMs) on mobile devices to predict user search intent in real-time. By utilizing a quantized Qwen3-0.6B model as an "Intent Agent," it translates local user behaviors into explicit search queries, achieving significant performance gains in Taobao's recommendation feed.
TL;DR
RecGPT-Mobile marks a significant milestone in industrial AI by deploying a lightweight LLM (Qwen3-0.6B) directly onto user smartphones to predict "Next-Query" intent. By replacing distant cloud inference with local semantic reasoning, Taobao achieved a 2.5% GMV boost while maintaining stable on-device performance.
Context & Motivation: The Latency-Cost Paradox
In modern e-commerce like Taobao, user intent is a moving target. While Cloud-based systems are powerful, they face two massive hurdles:
- Communication Lag: The "round-trip" from a phone to a data center prevents the system from reacting to a user's very last click instantly.
- Inference Costs: Running billion-parameter LLMs for every single scroll for hundreds of millions of users is economically unsustainable.
The authors argue that the future lies in Edge Intelligence. If the device can "understand" the user locally, it can update recommendations in real-time without constantly pinging the server.
Methodology: The "Small but Mighty" Intent Agent
The core of RecGPT-Mobile is a quantized 0.6B parameter model. However, simply shrinking the model isn't enough; the system needs to be smart about how and when it thinks.
1. Adaptive Prompt Construction
Not all users have the same hardware. RecGPT-Mobile uses Algorithm 1, which dynamically selects the best prompt template and structural components based on a "Budget Scorer." If the device is low on memory or battery, it delivers a more compact prompt to ensure the inference doesn't crash the app.
2. The Trigger Mechanism: Avoiding "Redundant Thinking"
To save battery, the LLM shouldn't run after every single click. The authors designed an Intent Drift trigger. It calculates:
- Entropy (): Is the user's focus becoming more concentrated or more scattered?
- Jaccard Similarity: How much overlap is there between current and previous behaviors?
- JS Divergence: How much has the distribution of user interests shifted?
The LLM is only triggered when the shift exceeds a threshold ().
Experiments: Real-World Impact
The system was tested across various Taobao scenarios like the "Shopping Cart" and "Order List" pages.
Key Metrics:
- Average GMV Increase: +2.5%
- Pay Conversion: +2.7%
- Consistency: Even at P95 (tail latency), the model remains responsive, proving that model quantization (LoRA + Quant) effectively bridges the gap between accuracy and mobile efficiency.
Critical Insight & Conclusion
RecGPT-Mobile proves that we don't always need 70B parameters for specialized tasks. A refined 0.6B model with high-quality supervised fine-tuning (using LLM-augmented and human-annotated data) can outperform larger, general-purpose models in specific domains like e-commerce intent understanding.
Takeaway: The "Intent Agent" approach, combined with trigger-based inference, provides a blueprint for any mobile app looking to integrate AI without blowing their cloud budget or draining user batteries.