“We Don’t Know” Means “They’re Not Sure”

The most familiar approach to handling respondent uncertainty about survey questions, the “don’t know” (DK) response, has an inconvenient feature: it records no direct information about respondents’ confidence in their answers. This article shows that DK responses contain more information than meets the eye. Using 161 survey questions from 11 surveys conducted between 1993 and 2019, the analysis demonstrates that the percentage of respondents saying DK can explain about half the variation in average confidence among respondents who provided an answer (i.e., those who did not say DK). This “we don’t know” means “they’re not sure” heuristic is especially reliable for relative comparisons between similar questions. For example, in a battery of favorability questions featuring two recent presidents and 10 contenders for the 2020 Democratic presidential nomination, the percentage DK explains more than 90 percent of the variation in average confidence among those who offered an opinion. To provide a deeper intuition, the article draws upon a threshold model of DK responding, which is consistent with a range of existing research on the properties of DK responses.

Read more