The Release-Waiting Farm: Mitigating Pre-release Failures in the SNS Plug-in Ecosystem
On Reducing the Pre-release Failures of Web Plug-In on Social Networking Site
This paper proposes the "Release-Waiting Farm," an experimental environment for Social Networking Sites (SNS) designed to detect pre-release failures in user-developed web plug-ins. By simulating a real-world SNS snapshot, it provides a sandboxed stage for systematic testing before final deployment.
TL;DR
In the wild west of Social Networking Site (SNS) development, end-users often release plug-ins without formal testing, leading to "post-release" disasters like DDoS attacks or privacy leaks. This paper introduces the Release-Waiting Farm, a sandboxed snapshot of the real SNS environment where developers can catch bugs before they hit the public. With a cost of only 25% of the main server, it successfully caught over 97% of potential real-world failures during testing.
Background: The End-User Development Dilemma
Web plug-ins have flourished on platforms like Facebook and Xiaonei, driven by the creativity of end-users. However, unlike professional software engineers, these developers lack rigorous QA processes. Consequently, the SNS ecosystem is plagued by pre-release failures—errors that could have been caught before launch but instead become "post-release" nightmares for users and administrators alike.
Existing solutions often involve heavy-handed restrictions or legal threats, which stifle the very creativity that makes SNS platforms successful. The authors argue that the problem isn't the users; it's the lack of a representative testing environment.
Methodology: The "Release-Waiting Farm" Insight
The core innovation is the creation of a "mini-world" that mirrors the production environment. Instead of testing in a vacuum, a developer submits their plug-in to the Release-Waiting Farm.
1. Architectural Snapshot
The farm isn't just a blank server. It contains:
- A medium-sized snapshot of the SNS database.
- A set of stable, popular plug-ins to test for compatibility.
- A functional API watcher to monitor system calls in real-time.
Figure 1: Comparison between the traditional web plug-in process and the proposed Farm-based iterative process.
2. Human-in-the-loop Testing
The paper utilizes two distinct human-centric testing strategies:
- Brainstorm Usage: General users are encouraged to "break" the plug-in for rewards.
- Invited Seniors: Experienced developers/users (Field Experts) are tasked with pinpointing architectural flaws.
Experimental Results: Proving the Sandbox
The authors conducted a five-month experiment using a custom SNS called "EXP" with 547 users.
High Fidelity of Failures
The most critical finding was the farm's predictive accuracy. Out of 772 failures found in the farm for the top 10 plug-ins, 752 (approx. 97%) were confirmed to be reproducible in the main server environment. This confirms that the "snapshot" approach effectively mirrors real-world conditions.
Table 1: Failure reproducibility between the Farm and the Main Server.
Efficiency of Testers
The data revealed a stark difference between "Brainstormers" and "Invited Seniors." While brainstormers identified more total "bugs," many were redundant. Invited Seniors had a much higher Effective Failure (EF) rate, meaning their reports were far more likely to lead to actual code changes.
Table 2: Comparison of Effective Failures (EF) between Brainstorm and Invited Senior methods.
Critical Analysis & Conclusion
The "Release-Waiting Farm" successfully balances the freedom of end-user development with the safety of professional engineering.
Values and Benefits:
- Cost-Effectiveness: The farm requires significantly lower resources (lower CPU, Memory, and Bandwidth) than the production environment, making it a viable investment for SNS providers.
- Educational Impact: Students involved in the experiment reported a shift in mindset, adopting version control (CVS) and formal testing disciplines naturally.
Future Challenges: The paper acknowledges a "timing factor" where certain time-dependent failures (e.g., voting deadlines) are missed because farm users "rush" through tests in seconds rather than days. Improving the temporal fidelity of the simulation remains a key area for future work.
Conclusion: By formalizing the "waiting period" through a structured farm environment, SNS platforms can transform a chaotic development cycle into a robust, high-quality ecosystem without alienating their creative user base.