The Trouble with Deviation Scores: Misinterpreting Mean Shifts and Variability
Professor Dan Schley
Associate Professor of Marketing
Rotterdam School of Management
Erasmus University
ABSTRACT
Deviation-based metrics (e.g., mean absolute deviations-MAD) are commonly used to assess anchoring effects, judgment accuracy, and related phenomena. However, this manuscript demonstrates that these metrics inherently conflate meaningful shifts in mean judgment positions with incidental changes in variance, potentially misleading researchers about treatment effects. Using intuitive examples, rigorous simulations, and reanalyses of past findings, we show that deviation scores frequently produce results that contradict the true underlying effects. We illustrate this conflation of mean and variance in the context of anchoring-effect and wisdom-of-the-inner-crowd studies. Ultimately, we argue for a conceptual and analytical separation of mean position from variance.














