Beyond the Marketplace: Harvesting Expert Knowledge from Online Communities
On the Invitation of Expert Contributors from Online Communities for Knowledge Crowdsourcing Tasks
This paper investigates effective strategies for inviting expert contributors from online social platforms (specifically Reddit) to participate in Knowledge Crowdsourcing (KC) tasks like artwork annotation. It compares two primary approaches: direct individual invitations and collective community-based invitations within the domains of botany and ornithology.
TL;DR
Knowledge Crowdsourcing (KC) requires more than just "clicks"; it requires specialized brains. This study explores how to recruit experts from Reddit for botanical and ornithological artwork annotation. The verdict? If you want people to notice you, send a DM; if you want them to actually do the work, post in their community.
The "Expertise Gap" in Crowdsourcing
Standard crowdsourcing platforms like Amazon Mechanical Turk are excellent for simple tasks (cat vs. dog classification) but fail when tasks involve specialized knowledge—such as identifying a rare 17th-century botanical illustration. The bottleneck isn't the number of workers; it's the composition of the workforce. To solve this, researchers are looking at "social platforms" like Reddit, which are organic hubs of specialized hobbyists and professionals.
Decoding the Invitation: Individual vs. Community
The study focused on two distinct ways to tap into this latent workforce:
- Individual Strategy: Targeted "cold" DMs to users identified as experts via their posting history.
- Community Strategy: Public "calls for help" posted in specific subreddits (e.g., /r/whatsthisplant or /r/birding).
Methodology & The Accurator Framework
The researchers didn't just blast links; they validated their invitation tone using a preliminary study on CrowdFlower to ensure the messages were perceived as "friendly" and "clear." They used a tool called Accurator to track the journey from "Invite Received" to "Annotation Completed."
Key Insights: Attention vs. Action
The results revealed a fascinating dichotomy in online behavior:
- The Power of the DM: Individual invitations were more effective at getting people to "look." In the botany domain, 10% of those DM'd actually visited the task page.
- The Power of the Crowd: Community posts led to much higher "Contribution Size" (CS). For birds, community invites generated 179 completed tasks compared to just 5 from individual invites.
Why Communities Outperform Individuals
The authors suggest that the social structure of Reddit—where users are accustomed to public interaction and collective problem-solving—makes community posts more persuasive. A post in a subreddit carries a "social proof" that a private message lacks. Furthermore, the "Individual" approach suffered from misclassification; many users replied just to say, "I'm actually not an expert!" (demonstrating that even errors can initialize communication).
Critical Analysis & Conclusion
Limitations
- Latency: The time between invitation and response (IRRT) is measured in hours. This is not for real-time needs.
- Platform Specificity: Reddit's culture is unique. These results might shift on platforms like Quora or StackOverflow.
Final Takeaway
If you are building a dataset for a specialized AI model, stop looking for workers in the marketplace. Instead, find where the experts hang out and engage with them as a community. The intrinsic motivation of a hobbyist often outweighs the financial motivation of a gig worker, provided the task is meaningful and the invitation is respectful.
Future Work
The next frontier is personalization—tailoring the invitation based on the exact level of an expert's knowledge and expanding this to other high-value domains like medicine or law.