CREA: Reimagining Creativity Assessment through Games and Crowdsourcing

TL;DR

Creativity has long been the "holy grail" of psychometrics—hard to define, harder to measure, and nearly impossible to scale. Janet Rafner's CREA suite breaks this deadlock by turning creativity assessment into a suite of digital games. By leveraging Generative Adversarial Networks (GANs) and crowdsourced peer review, CREA moves beyond static paper-and-pencil tests into a dynamic, "stealth" assessment environment that captures the creative process in real-time.

Contextual Positioning

While most psychological tools treat creativity as a final "product," CREA views it as a process. It sits at the intersection of HCI (Human-Computer Interaction), Cognitive Science, and Citizen Science, aiming to move creativity research from the lab to the "wild" of the digital world.

The Problem: The "Test Anxiety" of Creativity

Standardized tests like the Torrance Tests of Creative Thinking (TTCT) or the Alternate Uses Test often fail because:

1. Researcher Influence: The presence of an evaluator changes the subject's behavior.
2. Lack of Flow: Standard tests are boring. Creativity thrives in a "flow state"—where challenge meets skill—which is rarely achieved during a 10-minute written exam.
3. Scalability: Evaluating thousands of "original" ideas manually is a logistical nightmare.

Methodology: The CREA Portfolio

The CREA suite consists of four pillars designed to measure Divergent Thinking (DT)—generating many solutions—and Convergent Thinking (CT)—finding the single best solution.

1. Tiles & Blender (The Core Games)

• Tiles: Based on the Creative Foraging Game, players manipulate shapes. It tracks the "trajectory" of discovery, providing a mathematical look at how humans explore a search space.
• Blender: This is where AI meets creativity. Players use a GAN to blend images. It tests whether users can manipulate complex latent spaces to achieve a specific goal (CT) or create something entirely novel (DT).

Above: The interface for 'blender' (left) and 'tiles' (right), showing the transition from divergent exploration to convergent selection.

2. Ideas & Logic (The Benchmarks)

• Ideas: A digital version of the traditional Alternate Uses Test, used to validate the new games.
• Logic: Uses the Matrix Reasoning Item Bank (MaRs-IB) to measure abstract reasoning and ensure that creativity scores are controlled against general intelligence.

Experiments & Operationalization

One of the paper's most significant contributions is how it quantifies the subjective.

• Fluency: Automatically counted as the number of appropriate submissions.
• Flexibility & Originality: Calculated through a hybrid of semantic algorithms (to filter gibberish) and crowdsourced peer ratings.

Above: The selection phase for Convergent Thinking (CT) mode, where players must identify the correct origin of a target shape, effectively measuring "reconstructive" creativity.

Critical Insights: Why This Works

The "secret sauce" of CREA is Stealth Assessment. By embedding the measurement within the game mechanics, the "test" becomes invisible. The player isn't trying to "be creative" for a researcher; they are trying to "win the game" or "create a cool creature." This bypasses the typical biases associated with psychometric testing.

Conclusion & Future Outlook

The CREA project is a bold step toward Hybrid Intelligence. By using GANs as an interface, it doesn't just measure how creative a human is; it measures how effectively a human can collaborate with an algorithm to be creative.

Limitations: The study is currently in its early stages (PhD work), and the balance between "fun" and "scientific rigor" remains a delicate tightrope. Providing too much in-game feedback might improve engagement but contaminate the psychometric data.

Takeaway: Future creativity tools will likely look less like exams and more like the video games we play every day.