Pollster.com

Articles and Analysis

 

Does Spelling Count?

Topics: Internet Polls

"To retain respect for sausages and laws, one must not watch them in the making." Unfortunately, that quotation (widely attributed to Otto von Bismarck) may sometimes also apply to surveys. It is all too easy to put out percentages on a press release calculated to one decimal place that add to 100.0%. More often than not, the factors that distinguish excellence in survey research are invisible to the consumer. Of course, with an online survey the respondent does have the ability to save the verbatim questionnaire, and that brings us today's story about the remarkable number of spelling and grammatical errors in two recent surveys conducted online by Zogby International.

I should say up front that yours truly is in no position to cast the proverbial first stone when it comes to spelling and typographical errors. In my first week of blogging, Mickey Kaus observed that the Mystery Pollster "also seems to be a bit of a Mystery Speller." While I have tried to have others proof my copy before it goes online, I am painfully aware that typos probably remain in my archives. Alas, proofreading is not my strong suit.

However, all pollsters understand the critical importance of proofing our work product -- be it a draft questionnaire or the final results of a poll -- before it goes into the field or out the door. In a business where consumers expect precision, evidence of sloppy work takes on added significance. Just one careless, trivial mistake can undermine a client's confidence in an otherwise brilliantly executed survey.

The importance that most pollsters place on proofreading makes the email I received last week from a long time reader, someone with considerable training and experience in survey research, so remarkable:

As you may recall I'm on Zogby's Interactive panel. In the past the surveys often contained oddly worded questions, but at least they were copy-edited, spell-checked, proof-read, etc. Lately? Not so much. A few days ago I took a survey that included a number of topics, including the Israeli-Palestinian situation. I was struck by a number of misspellings. Here are the items that included simple misspellings of the type that are caught by spell-check (some are the actual question; some are response options) [Misspelled words in bold type]:

The U.S. Constitution says that a president can be impeached for high crimes and misdeameanors. Knowing this, would you favor or oppose impeaching George W. Bush?

  • Have you ever visted Bethlehem?
  • How interested are you in visting Bethlehem again or for the first time?
  • Do you support or oppose the contruction of this wall?
  • The wall seperates some Bethlehem families from one another [repeats twice]
  • The wall cuts through a historic Christian Diocese, seperating Bethlehem from Jerusalem [repeats twice]
  • The wall seperates Bethlehem and Jerusalem - two cities that have been histoircally interlinked and interdependent through community ties, trade and religious traditions [repeats twice]
  • Do you think that the land that is being confiscated by Israel in order to build the wall is primarly taken from. . .?
  • Chrisitan residents of Bethlehem
  • Of those listed, which of the following would be the greatest deterent to visiting Bethlehem?
  • Peacful Coexistence

That's bad enough, but today I received [another Zogby online] survey that was so awful it was painful. After an annoying but not terrible section on moral choice situations, they started with some civil lawsuit scenarios and then started using the word "libel" all over the place. At first I'm trying to figure out what libel has to do with anything, but then I realize that they actually mean "liable." They do this not just once, but for several scenarios, each involving several questions. It's really awful!

My reader goes on to provide examples of questions. The survey presented a series of factual "scenarios" involving two fictional physicians, Dr. Smith and Dr. Jones, then asked questions like the following:

13. On a scale of one to five, with one being not libel and five being libel, how likely are you to find Dr. Smith libel in this case?

Not libel
Probably not libel
Neutral
Probably libel
Libel
Not sure

14. On a scale of one to five, with one being not libel and five being libel, how likely are you to find Dr. Smith libel if you knew that he had never had a child suffer the damage Jane did, even though he has had several other children also complain of a stinging feeling while receiving the same vaccination?

Not libel
Probably not libel
Neutral
Probably libel
Libel
Not sure

My reader continues:

That's just the first of 21 questions like that, all of which make the same repeated mistake. All in all, the word "libel" shows up 137 times in the survey! This is indeed a train wreck.

But that's not all. With my jaw still on the floor, on the fourth or fifth scenario I noticed something else. They keep on asking "on a scale of one to five" but there's no scale, no numbers, just the response options. This error is made 30 times in the survey.

I emailed Zogby's Director of Communications for comment on Friday of last week and have not received any response. Zogby posted results of the Bethlehem survey yesterday.

Readers may ask whether these misspellings affect the results of the survey. Of course, spelling errors matter less when a survey is conducted by telephone (assuming that the interviewers pronounce misspelled words like "visted" correctly rather than phonetically). The two Zogby surveys discussed here were conducted online, yet even when online respondents notice the misspellings of words like "seperate" and "historically," they can presumably guess at the intended meaning. But the libel/liable mix-up raises more troubling questions. If respondents did not know the difference between the two words, did they understand the issues raised by the questions well enough to provide meaningful answers? If they knew the meaning of "libel" but did not guess it really meant "liable," how did they interpret the question? And if they deciphered the error, as my friend did, how many times could they tolerate its repetition before bailing out of the survey altogether?

But mostly, this sort of error should make consumers wonder. If a pollster neglects something as simple as spell checking a questionnaire before fielding it, what other mistakes is that pollster making that are not quite so obvious? Oh wait...

 

Comments
Shane Nelson:

While the spelling errors are certainly egregious, what is even more so is your Archie-Bunkerish critique of the word libel. Please look it up at M-W.com and while you're at it, if egregious is too challenging for you, check that too.

____________________

atomic dog:

"Archie-Bunkerish critique of the word libel"

huh?

What's written about the word and Zogby's confusion looks reasonable to this reader. I see no "critique" let alone something I could link to Archie B.

____________________

The criticism of the use of the word "libel" as it appears in the above scenarios is dead on. One can be "liable" for the tort of "libel", but one can't be "libel" for anything.

____________________

Shane Harter:

It looks like he did it again. This time he even misspelled his own name!

I'm not even sure how. "Shane Nelson" sounds nothing like "John Zogby"

Poor guy.

____________________

JDB:

It looks like with Mr. Nelson's comment, the nitwit at Zogby responsible for these monstrosities has outed himself. See:

http://www.zogby.com/about/detail.cfm?ID=49

Mr. Nelson, there are two separate words:

  • Libel
  • Liable
  • They are "false cognates," meaning that they sound a bit alike even though they are COMPLETELY UNRELATED. They do not mean the same thing. They are not even connected etymologically. (You can look that up at M-W.com when you're looking up "libel" and "liable." Literate people like the Mystery Pollster and Jim-PRS (and most other people who graduated high school) are aware that these are two different words with two different meanings. Where the heck did Zogby find you?

    ____________________

    Mark Lindeman:

    Zobgy Bights! [sic]

    ____________________



    Post a comment




    Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

    MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR