The Power of the Crowd: A Deep Dive into Crowdsourced Software Engineering

TL;DR

Crowdsourced Software Engineering (CSE) is no longer a niche concept. Based on a monumental survey from UCL researchers, CSE has matured into a robust paradigm capable of slashing development costs by 80% and solving complex computational problems through global "mini-competitions." This blog explores how shifting from traditional teams to undefined online workforces is reshaping the SDLC.

The Evolution of a Paradigm

In 2006, Jeff Howe coined "Crowdsourcing" to describe an open call for labor. In the decade that followed, software engineering organizations realized that the "Wisdom of the Crowd" could be systematically harvested. This survey categorizes a decade of research to answer a fundamental question: How do we build complex software using people we don't know?

The Problem-Solving Model of CSE

Traditional outsourcing relies on known vendors. CSE breaks this by utilizing an undefined, potentially large group of workers. The authors adapt Simon’s problem-solving model to explain how a requester navigates this landscape:

1. Intelligence: Identifying why CSE is needed (Speed? Cost? Diversity?).
2. Design: Decomposing a massive project into "micro-tasks" (like fixing a bug) or "macro-tasks" (like architecture design).
3. Choice: Selecting the right platform (e.g., TopCoder for competition, uTest for testing).
4. Implementation: Managing the transient nature of the crowd.

Where is the Crowd Most Effective?

The paper maps CSE across the entire SDLC. While the most prominent application is Software Testing (64%), several high-impact areas show the true potential of the crowd:

• Requirements Engineering: Tools like StakeSource use social network analysis to identify and prioritize stakeholder needs automatically.
• The Oracle Problem: Using platforms like Amazon Mechanical Turk (AMT) to provide human-verified outputs for test cases that programs cannot automatically validate.
• Genetic Improvement: A fascinating hybrid approach where "Expert Crowds" generate code and "User Crowds" evaluate it through genetic programming to fix bugs or optimize energy consumption.

Industrial Heavyweights: TopCoder vs. uTest

The survey sheds light on industrial practices that outperform academic prototypes.

• TopCoder utilizes a "Competitive Methodology" where only winning solutions are accepted, driving high quality through a "min-max" game-theoretic struggle.
• uTest focuses on "on-demand matching," allowing firms to test their apps on thousands of different device-OS combinations globally—a feat impossible for in-house labs.

Future Outlook: Hybrid and Multi-Crowd Systems

The paper concludes with a vision for the future. We are moving away from "binary" choices (either crowd or in-house).

• Hybrid CSE: Tools like CrowdBlaze combine automated static analysis with human-directed exploration.
• Multi-Crowd CSE: Complex pipelines where one crowd gathers requirements, another prototypes, and a third generates test cases, all communicating via intermediate platforms.

Critical Insight: The "Overhead Issue"

Despite the benefits, the survey notes a critical limitation: Task Decomposition. Breaking software into pieces is hard. If a task isn't perfectly self-contained, the "overhead" of coordinating hundreds of developers can actually decrease productivity compared to a small, focused team.

Conclusion

Crowdsourced Software Engineering is evolving into a sophisticated ecosystem. As the distinction between traditional and crowd-based work blurs, the next generation of software will likely be "born in the crowd," utilizing a global brain to solve problems that were previously "un-testable" or too costly to pursue.

Reference: Mao, K., Capra, L., Harman, M., & Jia, Y. (2016). A survey of the use of crowdsourcing in software engineering.