Recommending Deployment Strategies: Turning Crowdsourcing into a Geometry Problem

TL;DR

Deploying a task on a crowdsourcing platform (like Amazon Mechanical Turk) is often a shot in the dark for designers. Should you hire workers sequentially or simultaneously? Hybrid or crowd-only? This paper, published at SIGMOD '19, proposes the first formal algorithmic framework to solve this. By treating cost, latency, and quality as axes in a 3D coordinate system, the authors use computational geometry to recommend the best deployment strategies or suggest realistic alternatives when your goals are impossible to meet.

Background: The Designer's Dilemma

In professional crowdsourcing, a "Plan" isn't just a single choice; it is a combination of three dimensions:

• Structure: Sequential vs. Simultaneous workflows.
• Org (Organization): Collaborative vs. Independent efforts.
• Style: Hybrid (Human + AI) vs. Crowd-only.

Until now, designers had to guess which combination would fit their budget (Cost), deadline (Latency), and standards (Quality). There was no "optimizer" to tell them, "Based on current worker availability, you cannot get 90% quality for $10 in 2 days."

The Mathematical Intuition: Deployment as Geometry

The core insight of this paper is the Geometric Interpretation.

Imagine a 3D space where the axes are $X=Cost$, $Y=Latency$, and $Z=Quality$.

• A Query $(q)$ from a designer defines a "Target Volume" (a hyper-rectangle from the origin to the query point).
• A Strategy $(s)$ is an "Available Volume" – an unbounded region representing what that specific strategy can achieve.

Consistency is defined by the intersection of these volumes. If your query volume touches a strategy's volume, that strategy is viable.

Figure 1: Visualizing how a query point and strategy region must intersect in the parameter space.

Methodology: The Optimization Engine

The authors solve two primary challenges:

1. Linking Workers to Parameters

They propose that Quality, Cost, and Latency are linear functions of Worker Availability. By analyzing historical platform data, they estimate parameters $(m,c)$ for each strategy such that: $Value=mimesAvailability+c$ This allows the system to move from abstract strategy names to concrete coordinates in the 3D space.

2. The "Empty Answer" Problem (Query Reformulation)

What if no strategies satisfy the designer’s constraints? Instead of returning an error, the system performs a Euclidean Norm Optimization. It finds a new query $q^{\prime }$ that is as close as possible to the original $q$ but is guaranteed to return at least $k$ strategies.

The optimization objective: Minimize the distance between the intended and suggested parameters subject to returning k strategies.

Critical Insight: Why This Matters

Most crowdsourcing research focuses on how workers behave. This paper shifts the focus to how designers should plan.

By utilizing space partitioning, the algorithm can quickly check all possible partitions of the 3D space to find where the "closest" valid strategy lies. This is far more efficient than brute-force testing every possible strategy combination.

Conclusion & Future Outlook

This work provides a bridge between crowdsourcing management and database query optimization. While the current model assumes a linear relationship between parameters and worker availability, the geometric framework is extensible.

Future Directions:

• Dynamic Modeling: How do these strategies change when worker availability fluctuates in real-time?
• Complex Dimensions: Incorporating worker diversity or task-specific constraints (e.g., specialized skills) into higher-dimensional geometry.

For task designers, this represents a move toward a "Search Engine for Strategies," making complex human-in-the-loop systems more predictable and cost-effective.