Beyond the Magnitude: Why the "Evaluation" of Gaps is the Key to IS Satisfaction
Don't mind the gap: a conceptual and psychometric analysis of the individual evaluation of discrepancies in the context of is user service satisfaction
This research conducts a psychometric comparison of seven measurement approaches for assessing perceived discrepancies in Information Systems (IS) user satisfaction. By introducing the Modified Evaluative-Percept Disparity (MEPD) method, the authors achieved a state-of-the-art predictive validity (R-square up to 0.66) for user service satisfaction.
TL;DR
In the world of Information Systems, we often measure "gaps"—the distance between what a user wants and what they get. This paper argues that researchers have been focusing on the wrong thing: the size of the gap. Instead, by focusing on the evaluation (the "valence" or "goodness") of that gap through a new method called MEPD, researchers can increase the predictive power of their satisfaction models by over 300% compared to traditional additive models.
Background Positioning: This is a seminal psychometric critique and expansion of the Expectation Disconfirmation Theory (EDT). It functions as a methodological "course correction" for how we measure psychological discrepancies across IS, Marketing, and Management.
The "Size" Trap: Why Current Gap Measures Fail
For decades, researchers assumed that to measure satisfaction, you just needed to measure the distance between expectation and performance. The logic was simple: a big gap equals big dissatisfaction.
However, the authors point out a fatal flaw in this "Standard-Percept" logic:
- Asymmetry: A gap "above" expectations is qualitatively different from a gap "below," yet many scales treat them as identical "differences."
- Relevance: A huge performance gap in a feature a user doesn't care about won't impact their satisfaction.
- Evaluative Nature: Satisfaction is a summary attitude. You cannot predict an attitude using a non-evaluative physical distance (the gap size) alone.
Methodology: The Shift to Evaluative-Laden Measures
The study compares seven different ways to ask users about their experience. The breakthrough comes from the Modified Evaluative-Percept Disparity (MEPD).
While traditional methods asked: "How big was the difference?" and then "Was it good or bad?" (a two-step process), the MEPD simplifies this into a proximal antecedent:
"I am satisfied with how well the service provided has matched my expectations."
Theoretical Framework
The authors propose a temporal sequence of sense-making that places MEPD closer to the final "Satisfaction" state than simple gap measures.
Figure 1: The Expectation and Desire-based formation of Satisfaction.
Experiments & Results: A Psychometric Showdown
The authors tested these seven methods using data from 318 users of a Computing Services Center. They used Structural Equation Modeling (SEM) to see which method best-predicted overall satisfaction.
The Performance Gap
- MEPD (The Winner): Explained 61-66% of the variance.
- EPD (Valence only): Explained 56-59%.
- ADM (The Traditional Product Model): Explained a meager 10%.
Table 5: Structural Model Results across different measurement approaches.
The data is clear: the Additive Difference Model (ADM), long considered robust in marketing, failed significantly in this IS context. The "Better Than/Worse Than" (BTWT) approach, despite being simpler, actually outperformed several more "sophisticated" mathematical models because it inherently includes evaluate terms like "better" or "worse."
Critical Insight: The Temporal Sequence of Evaluation
The authors provide a powerful vision of how users process information. We don't just see a gap; we evaluate it.
Figure 2: The hypothesized sequence from Discrepancy Perception to Summary Evaluation.
By the time a user answers an MEPD question, they have already finished the "sense-making" process. This is why MEPD is such a potent predictor—it captures the result of the user's cognitive and affective hard work.
Takeaways & Future Impact
For Researchers: Stop using "difference scores" or raw gap sizes. If you are examining "Strategic Alignment" or "Task-Technology Fit," you must ask the user how they feel about the gap, not just how big it is.
For Practitioners: When surveying users, the wording is everything. Using scales that allow for exceeding expectations (e.g., "Far above my expectations") provides much more granular and valid data than simple "Agree/Disagree" or "Exactly as expected" scales.
Conclusion: "Don't Mind the Gap"—or at least, don't mind the size of it. Mind the evaluation. This paper successfully shifts the focus of IS satisfaction research from mathematical subtraction to psychological evaluation.