Beyond Brainstorming: Why Evaluation is the Most Creative Part of Innovation

TL;DR

Innovation isn't a lack of ideas—it's a lack of effective evaluation. This paper argues that the "filtering" stage of creativity is actually a complex, generative process involving forecasting consequences and iterative revision. By shifting from passive selection to active "situated evaluation," organizations can turn raw concepts into viable, high-impact products.

The "Generation" Trap

For decades, the standard playbook for creativity has been: Generate more ideas. We celebrate the "Aha!" moment and the whiteboard filled with sticky notes. However, as Mumford and his colleagues point out, only a tiny fraction of these ideas ever "see the light of day."

The real bottleneck isn't Generation; it's Evaluation. Prior work often treated evaluation as a cold, administrative "No" or a Darwinian "survival of the fittest." This paper challenges that view, positioning evaluation as a dynamic, constructive, and inherently creative activity.

Cognitive Architecture: How We Actually Judge Ideas

The authors propose a process model that moves evaluation from a static snapshot to a dynamic loop. Evaluation isn't about deciding if an idea is "good" or "bad"—it's about forecasting its future.

The Core Cycle: Forecasting & Revision

1. Situated Appraisal: An idea is never evaluated in a vacuum. It is appraised against the current context or envisioned future scenarios.
2. Forecasting: The brain projects the implications (both positive and negative) of implementing the idea.
3. Iteration: Instead of simply rejecting a flawed idea, the evaluator uses the "failed" forecast as a stimulus to revise and reorganize the idea.

The genius of this model is the Revision Loop. The authors argue that if an idea is original but lacks viability, we shouldn't kill it—we should apply "operative standards" to refine it. Conversely, if an idea is viable but boring, we apply "innovative standards" to inject originality.

Operative vs. Innovative Standards

A key insight of the paper is the tension between two types of mental benchmarks:

• Operative Standards: Focus on cost, efficiency, immediate ROI, and consistency with existing systems. While necessary, over-reliance on these "safe" metrics often kills the most transformative ideas.
• Innovative Standards: Focus on long-term growth, market positioning, and the potential for the idea to spawn further development.

Innovative organizations differ from stagnant ones primarily in their ability to flip between these standards at the right time.

Why Forecasting Fails

The paper highlights critical "cognitive errors" that disrupt our ability to judge new ideas:

• Oversimplification: Ignoring non-linear relationships or side effects.
• Self-Confirming Evaluation: Looking only for evidence that the idea will work.
• Overweighting Potential Loss: A biological bias toward risk aversion that stifles radical innovation.

Strategic Takeaways for Research & Industry

• Diversity is for Evaluation, not just Generation: We often use diverse groups for "brainstorming," but the paper suggests they are even more valuable during evaluation because they provide a wider range of forecasted outcomes.
• Revision as Generation: Stop treating "it won't work" as an end-point. Treat the specific reason it won't work as the prompt for the next creative session.
• Environmental Scanning: Evaluation is not a one-time event. As the context changes (new tech, market shifts), the evaluation must be reconfigured "opportunistically."

Final Thought: Evaluation is Creative

The paper concludes with a powerful reframe: Idea evaluation is not a passive filter; it is an active, constructive process. It is the art of manipulating an idea until its success is not just likely, but inevitable. To be a "Creative," one must be as skilled at forecasting and revising as they are at dreaming.

Reference: Mumford, M. D., Lonergan, D. C., & Scott, G. (2002). Evaluating Creative Ideas: Processes, Standards, and Context. Roeper Review.