Ulric B. and Evelyn L. Bray Social Sciences Seminar

Survey researchers have known for over 75 years that disturbingly large proportions of people asked the same question at different times give divergent answers, even if no material changes have taken place. This "survey response instability" is at the center of important substantive debates in many scholarly fields and, more generally, is informative about the data generation process, and appropriate statistical methods, for all survey data collections. By building on developments in neuroscience, cognitive psychology, and statistical measurement, we construct an encompassing model of the survey response, narrow competing hypotheses to a single data generation process, and validate it with extensive observational and experimental data. We show that (a) human beings have stable preferences and error-prone survey responses, (b) the variability leading to response instability is mostly due to an inherent feature of human beings, not bugs in respondent attentiveness or biases in our survey instruments, and (c) the "causes of the effects" of instability can be traced backward to respondents' time on task, mind wandering, competition among brain networks, and possibly evolutionary optimization rather than accumulated mistakes.