Contested Collective Intelligence: Navigating the "Wicked" Logic of Disagreement

TL;DR

In a world of information overload, the most valuable intelligence isn't just knowing the facts—it's understanding how those facts are contested. This paper defines Contested Collective Intelligence (CCI) and introduces a hybrid system of human-machine annotation that maps the rhetorical "moves" (claims, gaps, and contrasts) within documents to help organizations make sense of complex, ambiguous challenges.

Background: Beyond the Wisdom of Crowds

Standard Collective Intelligence (CI) research often focuses on "convergent" tasks—finding the one right answer or an optimal consensus. However, organizational life is messy. We deal with "wickedly" complex problems (as defined by Rittel and Webber) where there is no single truth, only competing narratives.

The authors argue that we need a specific infrastructure for Contested CI. Instead of smoothing over differences, we need tools that amplify them, making "rhetorical moves" like contradictions and surprises visible so we can navigate them.

The Problem: The High Cost of Sensemaking

Currently, analysts face a "foraging" problem. To understand a domain, they must read hundreds of documents, locate the points of contention, and manually build a mental map. Prior work in Information Retrieval (IR) helps find keywords but fails to show the relationships between conflicting ideas. The "Sensemaking Loop" is stalled by the sheer manual labor required to schematize these debates.

Methodology: The Mixed-Initiative Approach

The authors propose a 4-stage model that bridges the gap between raw data and coherent hypotheses:

1. Foraging: Identifying the working set of documents.
2. Machine Annotation (XIP): Using the Xerox Incremental Parser to scan text for "rhetorical markers." XIP doesn't just look for keywords; it looks for linguistic signals of "Summarizing" or "Contrasting Ideas" (e.g., phrases like "despite this," "the purpose of," or "an absence of").
3. Human Annotation (Cohere): Analysts use a web-based tool to highlight snippets and add their own interpretations.
4. Making Connections: The most critical step. Both human and machine annotations are transformed into "nodes" in a social-semantic network, connected by labeled links (e.g., "consistent with," "proves," "challenges").

Figure 1: The 4-stage model of integration between human and machine annotation for sensemaking.

A Specialized Toolkit: Cohere + XIP

• Cohere: A social-semantic web application that acts as a hybrid of a mind-map and a web annotator. It allows users to "map" the discourse beneath a document.
• XIP (Xerox Incremental Parser): An automated agent that speeds up the "foraging" loop by pre-extracting sentences that sound like they contain important claims or contradictions.

Figure 2: Cohere's sidebar allows for direct document annotation that is then linked to a global knowledge map.

Experiments: Human vs. Machine

The team tested this on a corpus of 125 reports regarding Open Educational Resources. By comparing human analysts’ manual summaries with XIP’s automated extractions, they found:

• High Fidelity: 88% of human-made summaries were literal or paraphrased extractions from the text, suggesting that machine "extractive" summaries are a valid starting point for human work.
• Shared Logic: 62% of the sentences chosen by humans contained the exact rhetorical markers XIP was programmed to find. This proves that "rhetorical moves" are a shared currency between human intuition and machine logic.

Figure 3: The visual result: a social network of annotations where machine-extracted "issues" and human "ideas" inhabit the same navigable space.

Critical Insight & Conclusion

The true power of this research lies in its Mixed-Initiative philosophy. The authors don't try to replace the analyst with AI. Instead, the AI (XIP) performs the low-level "foraging" to reduce cognitive load, while the human performs high-level "schematization"—the creative act of seeing new connections that aren't explicitly written in the text.

Takeaway: In an era of AI-generated content, the ability to map contestation remains a uniquely human-centric need. Systems that fail to account for disagreement are not truly "intelligent" in an organizational sense. CCI suggests that the future of CI is not about reaching faster consensus, but about building better maps of our collective disagreements.