Quinnipiac University’s “worst president” poll got a lot of press. Washington Post, the New York Times, CNN, Fox, and virtually every other news outlet have carried the headline, “Obama is the Worst President since WWII.” This particular survey question is press-chum. The survey’s designers likely knew it would get the press’s attention and included it for that purpose. Mission accomplished. The problem is it tells us practically nothing.
For one, this question lacks consistent measurement. Without consistent measures it is difficult to trust the “marginals” (raw percentages of agree/disagree, approve/disapprove, etc). The reason is that any single survey is subject to a variety of errors and biases. The sample can be biased. The survey’s respondents may not represent the actual population. Question wording can bias results. For example, research by Tom Smith (1987) shows questions using the words “welfare” rather than “poor” elicit more negative responses on surveys.
Similarly, question ordering can bias results. For example, offering respondents a question about unrest in the Middle East followed later by a question asking the respondent to judge the president on foreign affairs can bias responses. In this case, the surveyor has primed respondents with hostility and violence overseas. This can lead to more negative responses than if the survey leads with questions about foreign policy successes. (I haven’t looked through the rest of the poll, but the “worst president” is question 36 in this survey.) This can make interpreting the marginals without a track-record difficult for researchers or, in this case, the public.
However, the real crux is the nature of the question itself. The question effectively asks respondents to compare current events to past events. This flies in the face of what we know about survey response. John Zaller, often in collaboration with Stanley Feldman, illustrated that respondents answer questions based on what is at the “top-of-their-head.” Recent stimuli affect the attitudes respondents use to answer a question. For example, if you ask somebody today about their thoughts on politics, you are likely to receive an opinion on issues most recently in the news cycle, like unaccompanied children crossing the border into the US. If you asked the same question a month ago, the individual likely talked about another issue entirely, one that was popular at that time. And if you ask it five months from now, these issues may not register at all.
This makes retrospective questions like the one in the Quinnipiac poll biased against sitting presidents. Obama’s rank as the “worst president since WWII” is currently plagued by lost emails at the IRS, House Republicans’ accusations of executive overreach, and a sluggish economy, among other things. Previous presidents are free from current events. Bush’s approval is no longer burdened by the Iraq War, which helped lower his approval to 25% at one point. Harry Truman’s faults in office have evidently been forgiven. He tops the list as the least-worst president despite having the lowest recorded job approval in Gallup history (22%). Reagan’s legacy is no longer burdened by the early-1980s recession, which dragged his approval into the mid-30s. Nixon’s memory has evidently escaped Watergate. He fairs better than Obama and Bush II despite leaving office with only 24% approval. Clinton’s memory has certainly escaped his improprieties in the Oval Office.
It’s unsurprising that the two times this question has been asked (in 2014 and 2006) the sitting president has been recorded as “the worst president.” This isn’t to say it will always be the case. However, our memory of previous presidents fairs much better than their actual job approval during their presidencies. The events dragging down their job approval are long over.
Ultimately, this question wasn’t necessary and it is a prime example of when not to trust the marginals. Simple job approval/disapproval ratings are far more accurate at capturing presidential disapproval. And further, they are far less misleading.
Originally posted to reviseandextend.com.