Mark Blumenthal | February 6, 2009
Topics: Measurement , Stimulus
While it is easy to plot the trends in a chart (as above), caution is in order. We have trends from only three pollsters and some have made small changes in question wording. As noted last week, small differences in wording and question format appear to make big differences in the level of support measured.
To help make that point more clearly, I put full text of the questions used in the chart above into the following table (along with more complete results and links to the source pages).
|Question Text (differences italicized)||Favor||Oppose||No Opinion|
|Gallup (2/4) - As you may know, Congress is considering a new economic stimulus package of at least 800 billion dollars. Do you favor or oppose Congress passing this legislation?||52%||38%||10%|
|Gallup (1/27) - [Same text as above]||52%||37%||11%|
|Gallup (1/6-7) - Do you favor or oppose Congress passing a new 775 billion dollar economic stimulus program as soon as possible after Barack Obama takes office?||53%||36%||11%|
|CBS News (2/2-4) - Would you approve or disapprove of the federal government passing an economic stimulus bill costing more than 800 billion dollars in order to try to help the economy?
|CBS News (1/11-15) - Would you approve or disapprove of the federal government passing a 775 billion dollar economic stimulus package in order to try to help the economy?||63%||24%||13%|
|Rasmussen (2/2-3) - Would you favor or oppose the economic recovery package proposed by Barack Obama and the Congressional Democrats?||37%||43%||20%|
|Rasmussen (1/27-28) - [Same text as above]||42%||39%||19%|
|Rasmussen (1/19-20) - Would you favor or oppose the economic recovery package proposed by Barack Obama?||45%||34%||21%|
First, notice the big difference between the Rasmussen question and those used by CBS and Gallup. In addition to Rasmussen's consistently higher "don't know" response (discussed here last week, presumably the result of a prompt for "don't know), CBS and Gallup include an approximate price tag for the stimulus plan in their question. Rasmussen includes no dollar amount. Meanwhile Rasmussen explicitly associates the stimulus plan with "Barack Obama and the Congressional Democrats," while the other two identify no specific sponsor.
As Nate Silver observed, Rasmussen consistently shows less support for the stimulus plan than other pollsters. My bet is that question text and format explain most of the difference, although variation in sampling (Rasmussen screens for "likely voters" while Gallup and CBS samples all adults) and mode (Rasmussen uses an automated methodology while Gallup and CBS use live interviewers) may also be factors.
Second, notice that both CBS and Gallup changed the dollar amounts, Gallup on their second of three surveys and CBS this week. Perhaps more important: CBS also made a subtle change in their verbiage. The old CBS question references a "775 billion dollar economic stimulus package." The new question calls it an "economic stimulus bill costing more than 800 billion dollars" (emphasis added). They needed to change the amount, but why change the sentence structure? And more important, does adding "costing" make some respondents realize that proposal is not a government giveaway but rather something they might have to pay for someday? Without a split-form experiment, it is hard to know for certain.
What we do know is that pollsters are getting different results, something that often happens when many respondents lack strongly held views. We also know that most Americans are not closely following stimulus debate.** Less than a third tell CBS (28%) and Gallup (31%) that they are following news about the stimulus debate "very closely," while almost a quarter (23% on CBS and 24% on Gallup) say they are following the issue "not too closely" or "not at all."
We have some evidence of a modest decline in support for the stimulus, although given all the potential noise around slight changes in question wording, we probably need a few more polls to know for certain.
Update: Thanks to the reader who caught something I missed. Rasmussen also changed the wording of their stimulus question. In their first test in early January, the question identified only "Barack Obama" as the sponsor. Beginning with their 1/27-28 survey, that changed to "Barack Obama and the Congressional Democrats" (emphasis added). I have corrected the table above to reflect the changed wording
That change is important: While "congressional Democrats" are earning slightly better ratings than their Republican counterparts, their numbers are nowhere near as positive as Obama's. On the CBS survey, or example, Obama 62% approve of his performance as president, but only 48% rate the "congressional Democrats" favorably. Nancy Pelosi's favorable rating, using the tougher CBS format (that encourages respondents to report when they are unfamiliar), has dropped to just 10% favorable, 30% unfavorable).
So we have yet more reason for skepticism about the apparent decline in support for the stimulus package.
**We have limited data on whether the most attentive Americans differ in their overall support. In a release earlier this week based on a different question, Gallup reported that those "most closely following news about the plan differ little from the overall national average in terms of their attitudes about the plan." However, their table showed that the attentive Americans were more likely to want to reject the stimulus package altogether (27%) than those following it "somewhat closely" (11% reject) or not closely (15% reject). Similarly, without reporting any specific numbers, CBS News tells us that "those following very closely are more likely to oppose the bill than thosfollowing just somewhat closely."