Articles and Analysis



[This Guest Pollster's Corner contribution comes from Lydia Saad, Senior Editor of The Gallup Poll, responding to criticism posted earlier today by Alan I. Abramowitz.]

Alan, I see your point about how Gallup's question explaining the difference between Bush's income threshold and the Democrats' threshold could have confused respondents. You overlook the fact that we set up the question with this introduction to the series: "As you may know, Congress is considering a bill that would increase the number of children eligible for government subsidized health insurance, but the Democrats in Congress and President Bush disagree on how much to increase the program." But your point is well-taken.

However, our question measuring concern about the Democrats' bill being a step toward socialized medicine isn't "biased" -- it was intentionally written to convey Bush's counterargument. That was the intent -- to test the strength of socialized medicine as a counterargument. And indeed we found a slim majority willing to say they are concerned. We didn't conclude from this that Americans think the bill WILL lead to socialized medicine. As you note, we merely said that Americans are sympathetic to the argument: "Americans are also generally sympathetic to Bush's concern about the program leading to socialized medicine."

It is always a challenge to write clear and balanced questions about complex policy issues. Along those lines, I would go further than your critique of Gallup, and submit that all of the public polling on SCHIP I've seen thus far can be criticized in one regard or another. Yesterday I saw this question by CNN which seems to suggest that SCHIP is a new $35 billion program for children in middle income families, and that Bush opposes the program. I see what CNN was trying to do (isolate the question to the program expansion) but their wording just doesn't succeed at accurately describing what the veto/override conflict is all about.

26A. As you may know, President Bush vetoed a bill passed by Congress that would create a program to spend 35 billion dollars to provide health insurance to some children in middle-income families. Do you think Congress should vote to create that program by overriding Bush's veto, or do you think Congress should vote to block that program by sustaining Bush's veto?


The ABC question you applaud is another reasonable attempt, but still conveys the sense that the policy choice is between supporting the Democrats' plan and not providing any insurance coverage for "millions of low-income children." And since they ask if Congress should vote to override Bush's veto, they should have a follow-up asking what should happen if that override fails: i.e. Now that Bush has vetoed the bill, should the Democrats and Republicans in Congress work together to pass a new compromise bill, or should they let the program expire?

Otherwise we are just falling into the same political traps the Democrats and Republicans are setting for each other, and not really finding out what kind of government sponsored children's health care coverage Americans want for the country.

Americans probably don't have a great command of the details of the SCHIP debate, but half say they are paying very or somewhat close attention to it. That's about the midpoint for public attention to policy-debates in Washington. Americans clearly have some opinions worth tapping, and the challenge is to probe further for a more thorough and accurate understanding of whether Americans would rather have the existing program that covers families earning up to twice the poverty level, or whether the program should be expanded to include families earning more than that. Separately, we can find out who Americans would blame if the program expires: Bush for vetoing the congressional bill, or the Democrats for not being willing to pass a compromise bill.

None of the polling I've seen thus far -- neither on a question by question basis, nor in its totality -- answers those questions for me.

Lydia Saad

Senior Editor, The Gallup Poll


Alan Abramowitz:

Lydia--Thanks for your thoughtful comments. I agree completely that it is often difficult to come up with question wording that is fair on complex public policy issues. However, your comments on the "socialized medicine" question just reinforce my own belief that this was a one-sided question. If the intent was, as you said, to test the persuasiveness of one side's arguments, in this case the president's arguments, wouldn't it have been a more meaningful test to present respondents with the arguments on the other side as well, either in the same question or in a separate question? That way, we could see a direct comparison of the relative effectiveness of the arguments on both sides. Presenting only the arguments of one side of the debate is not a recipe for fairness.

Alan Abramowitz
Emory University


C Wilson:

Gallup has always been tilted toward the Republicans. This is one more bit of evidence.



Ms. Saad from Gallup:

You used the introduction to the poll as justification, but this introduction is as empty as the other misleading questions being discussed here.

Your introduction talks about "increasing the program"; but it says nothing about the nature of this increase. You say nothing about the direction of the increase. At no point in this poll does Gallup even hint that this is a program that benefits mostly low-income people.
NPR/Kaiser/Harvard was specific, telling us the direction of the increase sought by congress:
They also say the expansion will wind up covering some children in
middle-class families).
, NPR says.

And a majority believes in that poll that those earning 60,000 belong to the middle class.

Your introduction does not say who is being benefited under the current SCHIP laws. This introduction says nothing about who the Dems. want to add to the program. The only thing you say about the Dems. in the introduction is that they "disagree with Bush".

Half the people have heard little of nothing about this SCHIP program, according to your own poll. How do you expect this group to know, based on this vague introduction, that the Democrats want this the program to go from 0-200% of the FPL to 0%-300%?

It's also worth nothig that Gallup did not mention that the taxes used to fund this program would come from cigarrete sales. WP/ABC and NPR/Kaiser/Harvard thought this was relevant enough to include it.



Yes, it's true; in general Gallup has leaned Republican, but they seem to be backtracking now, as the country moves left.



I believe there is another problem with the question about the income cutoff:

"As you may know, the Democrats want to allow a family of four earning about $62,000 to qualify for the program. President Bush wants most of the increases to go to families earning less than $41,000. Whose side do you favor?"

This wording implies that the cutoff should be considered in isolation, which many respondents could interpret as "Do you favor a $35B program with a $41,000 cutoff or a $62,000 cutoff?" It essentially adds an option that is not on the table at all - an expanded program more closely targeting lower income families - that unsurprising draws greater support that Bush's actual proposal.



So the point of this poll is to test whether biased questions can sway people's opinions? If so, I think Gallup's single question is the best way to detect this.

By comparison, the Harris poll asks about agree/disagree for both the statements "SCHIP is a bad idea because it is a form of "socialized medicine, run by big government" and the counterargument "SCHIP is a good idea because, 'it has been highly successful in decreasing the number of uninsured children in this country even as fewer people have access to employer provided healthcare coverage.'" The latter has a much higher rate of agreement. (See http://www.harrisinteractive.com/news/allnewsbydate.asp?NewsID=1251.) I would hope these questions were rotated, although it's not clear from that link.

Alternatively, one can state both the President's viewpoint (socialized medicine) and the Democratic viewpoint (cover all children) on SCHIP in the same question and see which respondants agree with more. That is what the poll at
http://www.democracycorps.com/reports/surveys/091907.fq.national.pdf does.

Compared to Gallup's one-sided question, both of these techniques give a better idea of the efficacy of the President's argument.



Lydia - Sorry, but I would have fired any analyst who produced a test instument like this. My jaw dropped when I first saw these questions.

This work had significant and unfortunate consequences for an important public policy issue.



So, Gallup, you move from being merely Republican leaning to running Republican push polls. Goodbye, ethics!



So, Gallup, you move from being merely Republican leaning to running Republican push polls. Goodbye, ethics!



Lydia Saad of Gallup:

You say you wanted to "test the strength of socialized medicine as a counterargument. And indeed we found a slim majority willing to say they are concerned."

Yet, Gallup concluded that "Americans are also generally sympathetic to Bush's concern about the program leading to socialized medicine."

Mmmhmm. Sounds like Gallup likes to draw certain conclusions....


Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.