Health Reform Opinion: In One Word
Mark Blumenthal | September 9, 2009
Topics: Barack Obama , Health Care Reform , Measurement , Open-end
In my 'outliers' update yesterday, I pointed to the very useful graphic published by the Washington Post over the weekend that included tag clouds of two "open ended" questions asked on national survey roughly two weeks ago. Jennifer Agiesta described the results in more detail in the Post's "Behind the Numbers" blog.
Both the Washington Post and the Pew Research Center have been experimenting with open-ended questions that ask respondents to answer "in a single word." In pollster lingo, an "open-ened" question suggests no answers and allows respondents to answer in their own words; a closed-ended questions prompt respondents with standard answer categories. Open-ends (as we call them) are not new, but media pollsters tend to avoid them because of the time and effort required to record verbatim answers, to probe for more details and code the answers. The one word open-end is considerably quicker to ask and record and, better yet, is easily suited to tag cloud graphics where the type size is proportional to frequency of response of each word.
On their most recent survey, the Post included two such questions that asked respondents to describe their feelings about President about and, separately, about "the proposed changes to the health care system being developed by (Congress) and (the Obama administration)" (they randomly rotated whether Congress or Obama came first). I thought the responses to the second question (reproduced below) were interesting, but the fact that the tag cloud displayed just the 22 words mentioned most often may have obscured something important.
I thought it might be useful to compare a cloud with the positive words to a cloud with the negative. I emailed Jennifer Agiesta, and she kindly provided the full list of words (unweighted) and the codes (positive, negative, neutral or none) applied to each one. Here are the graphics I created, starting with the responses that were 43% of the weighted adult sample:
And here is the cloud of the positive responses that were 31% of the adult sample (and I've made a feeble attempt to make the negative cloud roughly 1.4 times bigger than the positive to try to keep them roughly to scale -- unfortunately, Wordle, the service that creates these beautiful graphics allows little control over white space, so hopefully my sizing comes close to proportional).
The difference between the two is striking, especially if you click to view each full size. As noted by Jennfer Agiesta, the negative responses include a "broader array of words" that imply far more passion and intensity. People use many different words to say what they dislike and those words carry a lot of emotion: "scary," "terrible," "disaster/disastrous," "socialism."
The positive responses, on the other hand, tend to be less varied and convey more of a sense of ambivalence: Two words, "necessary" and "good," account for nearly half of the responses. Those two, along with "hopeful" and "okay" account for roughly 60% of the positive comments (unweighted). We see words like "great," "excited," "excellent" -- the positive analogues to the emotional words used by opponents -- far less often.
The Post question asks for "feelings" not facts, but my sense is that opponents of the Obama proposals have a greater conviction that they know what they don't like about the plan, while supporters are hopeful but more often unsure about the details. And, of course, the Post found that one adult in four (26%) is either unsure or neutral. Obama and the Democrats are hoping to reach those uncertain and hopeful Americans with his speech tonight.
That said, I wish that some of my pollster colleagues had asked Americans to describe in a single word or a sentence what they think the health reform proposals will do. For all the sturm and drang of the debate over the public option (and about poll questions that ask about the public option), I am surprised that no one has bothered to ask a question sequence like the following before asking the more standard questions in which respondents react to brief descriptions of the proposals:
- In the health care reform debate, have you heard anything about a proposed "public option" (yes/no)?
- In your own words, please tell me what you have heard about the proposed public option? (followed by a probe: can you be more specific?)
- Based on what you've heard, do you favor or oppose the public option or don't you know enough to say?
More probing along these lines would at least give us a sense of how many people think they are familiar with the proposal and some sense of what they know. It would also allow us to compare support for the public option among those who seem to know what the public option to those who do not. Instead, we are left to debate the many nuances of question language and wonder whether we are measuring current attitudes or reactions conjured up on the spot.
Pollsters, to help us understand public opinion and health reform, more open-ends please!
By Mark Blumenthal | September 9, 2009 6:27 PM | Permalink | Comments (1) | TrackBacks (0)
As always, I'd be curious as to what proportion of the negative word responses were from people who are "upset" or "disappointed" that the Democrats haven't crafted legislation that is far enough to the left.
Probably a small number, but with these word clouds that distinction gets lost.
Posted on September 9, 2009 8:41 PM