Charles Franklin

High Quality Local Poll Reporting

Your intrepid reporter is doing field research in south-western Washington state and stumbled across a nice example of local reporters and newspapers doing serious reporting on polling.

The setting is Washington's Third congressional district, covering most of south-western Washington from Olympia in the north to Vancouver across the river from Portland OR. An open seat rated tossup by both The Cook Political Report and the Rothenberg Political Report, Dems have held the seat easily since 1998 but with the retirement of incumbent Democrat Brian Baird the seat is clearly up for grabs. Cook's Partisan Voting Index for the district is a flat zero. Obama won the district 53-45 while Bush edged Kerry 50-48. 

Two Dems and three Reps are on the August 17th primary ballot. On the GOP side, two-term State Representative Jaime Herrera faces ex-Marine David Hedrick who cites his Tea Party origins, and ex-Bush administration official David Castillo, who claims backing from Dick Armey's FreedomWorks. Neither Hedrick nor Castillo have prior electoral experience. (The Democratic candidates are not the focus of the polling article, but include five time state Representative Denny Heck and citizen activist Cheryl Crist. Independent Norma Jean Stevens also appears on the primary ballot.) Under Washington election law, all candidates appear on a single ballot with the top two vote-getters advancing to the general, regardless of party.

In an August 10 front-page story above the fold of The Columbian, reporter Kathie Durbin, reports on new poll results cited by the Hedrick campaign which show him neck-and-neck with Herrera (and both trailing Democrat Heck.) But then the story gets to the punch line:

But the source of Hedrick's information, The Washington State Political Polls, was unknown in the state until last month and has zero credibility with polling professionals.

The story goes on to trace the vague origins of the "poll", with clear and useful quotes from Seattle pollster Stuart Elway of the Elway Poll and University of Washington political science professor Matt Barreto. In the course of the lengthy 27 paragraph article, reporter Durbin manages to cover the role of random sampling and how internet polls of dubious origins fall far short of those standards. She also includes comments from the Hedrick campaign defending its use of the "poll". My favorite:  "...it's "not my job" to vet the poll."

While the WA-03 race is an important tossup, it is not at the center of national reporting. But this article is a model of local reporting on polling, vetting claims by candidates and checking polling methods with qualified professionals. And putting the story in a prominent place in the paper.

As we enter the frenzy of the fall campaign with a bevy of new pollsters appearing, using IVR and internet methods of data collection (some well, some not so well), it is especially important that reporters check the polling methods and report to readers what is at stake between scientific polling and junk. It was a pleasure to find this job done so well.

Some Perspective on Real GDP Growth


Today's GDP numbers provide little comfort to Democrats hoping a robust "recovery summer" will aid them in November. GDP grew at an annual rate of 2.4% in the second quarter, down from 3.7% in the first quarter and 5.0% in the 4th quarter of 2009.

The GOP will point to falling growth as proof of Democratic failure, and discouraged Dems who have had a hard time selling even real gains are unlikely to successfully use this latest report to their advantage.

But as always, let's take a moment for some perspective based on the data from the four recessions since 1980.

President Reagan benefitted from very large growth rates in the four quarters following the end of his recession. Like Obama, Reagan inherited a terrible economic situation, and suffered from it throughout his first two years in office. But beginning in the fourth quarter of 1982, GDP growth rebounded very strongly. Over the next four quarters, real GDP grew an average of 5.7%. If we leave out the fourth quarter of 1982, which had a tiny +0.3% growth though technically not part of the recession quarters, then Reagan enjoyed an astonishing average GDP growth of 7.8%, providing the foundation for his "Morning again in America" reelection campaign in 1984.

The two Bush presidencies did not enjoy recoveries of anything like that of Reagan. President G. H. W. Bush suffered from just a 2.6% average growth rate in the four quarters following the 1991 recession, though the rate bumped up to over 4% more than a year after the end of the recession. That sluggish initial recovery set the narrative President Clinton used in the 1992 campaign, even through growth in 1992 was a robust 4% or more.

President George W. Bush likewise faced slow growth following the 2001 recession, with average growth of only 2.3% in the four quarters after the recession. While 9/11 undoubtedly affected the fourth quarter of 2001 (1.4% growth), the next three quarters were 3.5%, 2.1% and 2.0%, followed with a fifth quarter of just +0.1% which is not included in the average. Excluding the post 9/11 quarter and including the later quarter would lower the average growth even more.

Which brings us to President Obama. In the four quarters since the end of the recession (as defined by the end of shrinking GDP in 2009Q2), real GDP has grown an average of 3.2% each quarter. So in fact, the current recovery is a bit stronger than either of the two under Presidents Bush, though well below the extremely strong rate under President Reagan.

Even the current disappointing quarter at 2.4% is better than the average of 2.3% in 2001-2.

So a rational perspective would be that we are recovering better than in the previous two recessions, which were much less dire than the Bush-Obama recession, but well short of the energy that propelled the Reagan recovery.

For November, this rate of growth does not bode well for changing the narrative from recession to recovery, as the White House had no doubt hoped and bet on. The trend of declining growth rates over the past three quarters adds to worry that recovery will be slow or even threaten to dip back into recession. Ironically, of course, a stimulus that might enhance the macro-economy is now politically untenable as even many Democrats have accepted Republican arguments that stimulus spending didn't and doesn't work. Decades of macro-economic evidence to the contrary notwithstanding.

Research 2000, DailyKos and Transparency

Today the polling world was rocked by claims that polls by Research2000 for DailyKos are substantially flawed:

We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls

Coming on the heels of recent arguments by Nate Silver at FiveThirtyEight.com that Strategic Vision faked their polling over several years, this is a new blow to the credibility of public polling.

Mark Blumenthal here at Pollster.com has lead a concerted effort over the last two years to increase the degree of disclosure expected from polling firms, an effort that paid off in new disclosure requirements from the American Association for Public Opinion Research (AAPOR) this spring. Some three dozen firms immediately signed on to the new disclosure requirements, but there are many firms that produce widely cited polls that have not yet agreed to disclose as much as required.

I've only had time for a single quick read of the Research2000 analysis by Mark Grebner, Michael Weissman, and Jonathan Weissman. It seems to be done seriously and it raises important doubts about Research2000's practices. But with my academic's hat on, I'd like to see it receive serious review by professional statisticians with polling experience. Academic journals typically reject 80-90% of articles submitted to them because on close inspection by experts, flaws are found in the theory or the analysis. These are serious charges, and they deserve to be vetted by professionals qualified to do such an evaluation. If flaws in the analysis are discovered, they can be fixed and the conclusions corrected. If the analysis is found to be sound, then the evidence is even more compelling and worrisome for its implications for the polling industry.

There is one element of disclosure that has not been pushed, but which could significantly and easily reduce the chance of "pollsters" making up their data. Every media firm, including DailyKos, should write into their contracts the requirement that the raw data and complete questionnaire be deposited within two months with the Roper Center Polling Archive at the University of Connecticut. Two months is long enough that there is little remaining news value, but rapid enough that meaningful vetting and analysis is possible. By forcing this disclosure, by contract, the sponsors of polling would gain credibility for their polls while insisting that their pollsters live up to the standards of disclosure by AAPOR as well as making the raw data available for subsequent scrutiny.

Most major media polls already deposit their raw data with the Roper Center (including, Gallup, ABC/Post, CBS/NYT, NBC/WSJ, Pew, Time, Newsweek), though not necessarily as quickly as two months. Their example should encourage others to also deposit their data.

But most importantly, it is in the interest of the sponsors of polling to protect their reputation by requiring full disclosure and deposit of the data. Such practice would enhance the value of their polls, not diminish it.

Obey (D-WI-7th) to Retire


A bit of a parochial post but with a national figure at its center and reflecting the national midterm forces.

David Obey announced his retirement today. He has been in the House since 1969 and is currently chair of Appropriations. He was facing a challenger from Sean Duffy, a district attorney since 2002 in Ashland county (on Lake Superior). Obey has $1.4M COH while Duffy has $340K, but that is the lowest ratio of any WI Dem incumbent, at 4:1. Duffy as gotten a good bit of attention as a potential break-out challenger and recent endorsements from Palin and Pawlenty.

I'm not aware of any public polling in the 7th. We don't have any at Pollster.com and I don't know of any elsewhere either. No idea what parties or candidates may have.

Obey last faced a real challenge in 1994, the year 2010 is increasingly being compared to. He got 54% that year, 57% in 1996 and hasn't been below 60% since. Now at 71, facing the prospect of a tough reelection like that 1994 race, he must have felt he just didn't have enough to gain by fighting another battle against the prospect of leaving office by defeat. He says he is "bone tired" and that may well be. But the electoral winds are blowing hard against Dems this year, and that is making a lot of them much more "tired" than they were two years ago.

Pure speculation on my part: Obey might win a tough reelection fight (I'm sure he thinks he would win) but find himself back in the minority in the House, not as Chair of Appropriations but as ranking minority member. It wasn't any fun for him as minority member prior to 2006, and it sure would be less fun to go back to the minority after being Chair. While no one knows if the Reps will take control of the House, that had to be a consideration for Obey, and he is in a good place to read the writing on the wall. Just saying.

While the district leans Dem (Obama got 56%) in competitive races it is more of a toss up. In the 2006 Attorney General race, VanHollen (R) lost by just 7,000 votes out of 250,000 cast while winning statewide. A little worse than his statewide performance, but it shows a competitive Rep can win the district. Obey simply hasn't had competitors strong enough since the 1990s to give any sense of how far the district might go for a Rep in an open House seat race.

Dems/pundits are throwing out the obvious names: state legislators with districts in the 7th: State Senators Kreitlow (23rd), Lassa (24th), Majority Leader Decker (29th), and State Assembly members Seidel (85th) and Vruwink (70th). (Vruwink was an Obey staffer a while back.) Obviously they are doing exactly what I did: look for every elected Dem from the district and mention them! <;-) Of these, those young enough are Kreitlow (46), Lassa (40) and Vruwink (35). Decker at 57 isn't out of it, but not comparable to Duffy's 37. But Kreitlow and Decker are both up for reelection this year, making it a harder decision to go for the Congressional seat especially in a tough year for Dems.

Consider this a case study, one example of many adding to the sense that this year is moving strongly in the GOPs favor. (And I say that as one who still forecasts Dems to retain the House. But I might be wrong.)

March Unemployment in the States


March unemployment rates for the states are out today. Some down slightly, some up or steady. Here is the lede from the BLS report:

Regional and state unemployment rates were little changed in March.
Twenty-four states recorded over-the-month unemployment rate increases,
17 states and the District of Columbia registered rate decreases, and 9
states had no rate change, the U.S. Bureau of Labor Statistics reported
today. Forty-four states and the District of Columbia recorded jobless
rate increases from a year earlier, 5 states had decreases, and 1 state
had no change.

The number of jobs rose in 33 states and declined in 17, consistent with the addition of jobs nationally in March. So while the balance of results are improving there remain a significant minority of states not yet enjoying an improved jobs situation.

The chart above shows the trends by state with national comparisons.

The chart below shows the change over the past 12 months, March 2009 to March 2010. The states are mostly above the black 45 degree line, showing the general rise in unemployment among 44 states, with only 6 decreases or no change. The blue line shows the linear relationship over the year, illustrating what we might expect in 2010 given where a state was in 2009 and the overall pattern of change in those 12 months.


Finally, we can look at which states are doing better than expected and which worse, compared to the blue linear fit line. The chart below shows the residuals for that fit against current state unemployment. Below zero means a state is doing better (has lower unemployment) than expected based on where they were a year ago. Above zero are states doing worse (higher unemployment) than expected.


The most interesting take-aways here are the two states in the top right corner: Nevada and Florida have both high current unemployment AND have higher than expected rates. And to make matters worse, both Nevada and Florida lost non-farm jobs from February to March. The electoral troubles Senator Reid and Governor Crist find themselves in might be somewhat less if their states were in happier circumstances. Not the whole story of course, but it certainly doesn't help.

Generic Ballot Trends


Generic Ballot time. Mark has a piece at National Journal today that will soon be linked to here in his Monday post. Meanwhile here is a quick look at the underlying data.

Above, generic ballot trends since 1992. The contrast of 2010 to 2006 and 2008 is quite striking. No comment necessary.

Below, the linear trend over the last 200 days of the campaign, since 1946. See Mark's post for some discussion.

Off to teach! Have a good week.


Obama as Reagan, cont'd


I've been pointing out the similarities between the circumstances of Presidents Obama and Reagan for a while now. See an earlier post on this here.

The short version is both come in with inherited economic troubles that don't turn around miraculously in the first 24 months. Both replace deeply unpopular predecessors, and suffer from high expectations in comparison. And both set out to dramatically change the direction of national policy. Reagan suffered substantial losses in the House in his first midterm (26 seats lost), and Obama looks headed to similar if not larger losses in 2010.

So how is the analogy holding up? In approval terms, still quite well. The two continue to track rather well. Obama has occasionally been slightly below and recently slightly above Reagan's trend, but the parallel movement remains striking. Likewise, their relative location compared to other first term post-war presidents continues to drive home the point that these have been (so far) among the lowest approval ratings in the first 24 months.

Despite the similarity, I don't think the two presidents are metaphysically linked by fate. Both suffer from the economy and their large policy goals. At the moment, the economy is looking to have turned up sooner for Obama than it did for Reagan (who suffered until the very month of the midterm before the economy bottomed out and started to recover.) Obama has a more hopeful looking GDP trend, though his unemployment trend has not yet started down. (Political science finds that GDP is consistently a better predictor of midterm outcomes than is unemployment, despite the vastly greater emphasis on unemployment in public commentary.)

So I don't think the 2010 results are yet set in stone, nor that the track of Obama's approval is necessarily going to continue following Reagan's. Rather it has been driven by similar circumstances, and those circumstances appear to be diverging on the economy at least. Whether Obama's approval responds, and with what effect on midterm outcomes, remains to be seen. The politics is yet to finish baking.

Bonus Chart: The first term presidents through midterm but in separate charts rather than overlaid. Data are Gallup polls only to provide comparability over the decades.


Recess Appointees and Time to Confirmation


President Obama made 15 recess appointments over the weekend, the first of his presidency. The White House announced the appointments and made their case based on the long wait for Senate action in a press release here. The recess appointments have waited an "average of 214 days" according to the release (the mean in the data I analyzed is 213.6 with a median of 194 days.) The nominations and confirmations data I used is available at the White House website here. I've excluded 9 appointees who are holdovers from the Bush administration who do not require confirmation. The White House release says 217 nominees are pending though in these data I find only 207 pending, including the 15 recess appointees.

The figure above puts the delays in perspective by also showing the 550 confirmed appointees and all the wait times until final action. (At the moment, no nominee has been rejected, so the duration is days until confirmation vote or March 27 (the date of the recess appointments).

Among the confirmed nominees, the median wait time was 56 days, with 75% confirmed in 91 days and 90% confirmed in 129 days. In contrast, for the 15 recess appointees, the minimum wait was 144 days, with a median of 194 days, and a third have waited more than 249 days for a confirmation decision.

The key take-away is that 73% of nominees have been confirmed, while 27% are pending. Some of those pending are recent nominees, with 34% waiting less than the 56 day median for confirmed nominees. But there is also a considerable right tail to the distribution, with many pending nominees having waited well beyond the normal range for confirmed appointees. And the recess appointments are mostly from the right tail of the unconfirmed.


The top half of the figure shows the distribution of wait times for the confirmed and unconfirmed nominees. The red distribution for those waiting for confirmation has a much heavier right tail than the blue line for confirmed cases. The red dots show the 15 recess appointees, which are visibly to the right of the distribution of times. The lower half of the figure shows the individual wait times for the recess appointees by name and agency.

Finally, we can compare the wait times for confirmed, pending (excluding recess) and recess appointees.


The recess appointees are mostly outliers compared to those confirmed and have all waited longer than more than three-quarters of other pending nominees.

These data put the delays in confirmation in perspective. As the White House complains, the group of recess appointees have indeed waited considerably longer than is normal for eventual confirmations, and longer than a large majority of other pending appointments. But the White House also glosses over the relative speed of the 73% of nominees who have been confirmed, 90% of whom were confirmed in 129 days, or about 4 months, or less. The reason some nominees face long delays is political-- either due to the individual nominee's background or the agency to which they are nominated or holds placed by individual Senators. Those are normal political battles, and it is worth noting that the delay in decisions is not universal. That may be cold comfort to the 35 still-pending nominees who have been waiting for a vote for more than 150 days.

Addendum: University of Kansas political scientist Michael Lynch sent me and The Monkey Cage an email summarizing the results of his (and colleagues) analysis of recess appointments since 1987. The key finding is that length of delay alone is not the key determinant of who receives a recess appointment and who does not. Not too shockingly, presidents use recess appointments to further their political goals and especially so with regulatory commissions, such as NLRB and EEOC appointments among Obama's 15. The Monkey Cage has Lynch's summary of his research here. Of course, before going to a temporary recess appointment, nominees must normally have waited longer than those most easily confirmed, as my analysis above shows. Among those waiting, who gets a recess appointment and who does not certainly is a political and policy choice, as Lynch's research demonstrates. --Charles

HCR Vote Probabilities For the Undecided

The "undecided" health care votes are mostly right where you'd expect: marginal Dem districts and among the more moderate House Democrats. There are a few outliers that may be real or may be temporary hesitations, and there are a few surprises in both directions with firm yes votes in questionable districts and puzzling no votes (though the Stupak amendment and abortion account for some of these.)

Who's up in the air, and what are their probabilities of voting no? I model this based on Obama's share of the two party vote in the district in 2008 and the member's roll call record on a liberal-conservative scale as estimated by Simon Jackman of Stanford University (Thanks!).  Other variables don't add to the model: being in trouble for re-election doesn't add anything over and above the district and ideology measures. Those in trouble are in districts you'd expect to be trouble. Retirement and seeking higher office also have no measurable effect. And the Washington Post whip feature provides health industry contributions and percent uninsured in the district. Those do nothing to explain position either.  In the end, when you are down to a game of inches like this, the statistical model can only speak to the broad tendencies, not the special circumstances that may flip a member on way or another.

These positions are taken from the Washington Post and The Hill's published counts. They were updated through noon on Friday, though positions are certain to change. 

First, the members who voted No on November 7 and who are currently undecided, with their estimated probability of voting no now.

Now the much more numerous "Yes"votes in November, who haven't taken a firm position this time:

And for some perspective, how these probabilities vary by lib-con roll call records (via Simon Jackman), and then by Obama vote in the district.

Based on the probabilities alone, the Speaker and White House still have some heavy lifting to do, despite the sense that the Democratic members are shifting towards a very narrow passage.

About "True" Unemployment


Today's unemployment rate was announced as 9.7%. But some articles mention a "true" unemployment rate of 16.8%. How can both numbers coexist, and which is "really true"?

This is a lengthy, technical post, so here is the conclusion:

As a matter of compassion for people who would like to work more, U6 may be a more inclusive measure, but as a matter of statistical information, as a matter of understanding the economic (and political) implications of unemployment, there is no advantage in choosing anything other than the official unemployment rate, U3. Calling U6 the "true" or "real" unemployment is misleading and a disservice to readers and listeners. It is a claim of added value when in fact there is none. 

Now the details:

The Bureau of Labor Statistics produces an official unemployment statistic each month, known as U3. The BLS also produces five alternative measures of unemployment, U1, U2, U4, U5 and U6. U1 and U2 are narrower measures of long-term (over 15 weeks) and new job losses. U4-U6 are alternative measures including discouraged, marginally attached to the labor force and part-time workers who would prefer more hours. (See the BLS data here.)

As unemployment rises, the rates for marginal and part time workers also increases, and runs considerably above the official unemployment rate. The key difference is that discouraged and marginal workers are both not working and not looking for work but have looked in the last 12 months and say they would like to work. (Discouraged is a subset of marginal, with the addition that they cite a job-market related reason for not seeking work.) The part-time worker category includes currently employed people who want and are available for full time work but who have had to settle for part time employment.

Some news sources refer to U6, which includes both marginal and part-time workers as the "true" or "real" unemployment level. NPR has done so here and here and here and here. And the Washington Post's Frank Ahrens' Economy Watch Blog has had quite a bit to say about this including, here and here.

But in what sense is U6 more "true" than U5 or the official U3? One can certainly make a case that discouraged workers are unemployed, would like to work, but so disheartened they've given up. A slightly weaker case can be made for the marginal workers. But those two together only add modestly to the official unemployment rate, as you can see in the chart above. Where the big jump comes is among part-time employees, who are added to the mix in U6.

So is U6 a better measure? It certainly matters that 16.8% of potential workers would like more hours (or a job of any kind) compared to the 9.7% who are without jobs and actively looking. But if you are going to adopt U6 as your standard, you need to realize that even during the "full employment" of the late 1990s to 2000, when unemployment (i.e. U3) fell to just 3.8%, U6 still stood at 6.9%, 1.8 times higher. Of course a booming economy provides more full time job opportunities, but even the hottest economy of recent decades did not bring U6 below the 7-8 point range.

Since 1994 (when current unemployment measures were adopted) U6 has averaged 1.76 times the U3 rate. Today's ratio stands at 1.73, essentially the same as the historical average.

Today's Frank Ahren's Economy Watch entry points to the gap between U6 and U3, and links to an earlier post on that gap. Let's look at that gap, but also compare it to the ratio of U6 to U3:


Ahrens focuses on the difference, which has risen lately while ignoring the ratio which has fallen over the last year, and is down from pre-recession levels. U6 rises faster than U3, with U6 increasing by about 1.7 points for each 1 point increase in U3. That means the gap between them has to rise when U3 is large, as it certainly is today. But the relative rates of U6 and U3 are the ratio, and that has fallen recently.  If you want to make the case that unemployment is "really" worse than it looks (based on U3) then U6 and the gap make a good case. But if you want to know if U6 is abnormally large given the current official unemployment of 9.7%, then the ratio is a more reasonable measure, and by that it doesn't look unusually large, and in fact is slightly below historical expectations. 

That doesn't answer the "truthiness" question, but it does seem to say that U6 isn't really telling us much we wouldn't have expected from U3 and the historical relationship. And that is the key point. Does U6 add information not available in U3? If we adopted U6, we'd have a higher rate always, but would it tell us something different from U3? Not very much at all. In the top chart, I include the correlation between U3 and U6. It is a near perfect 0.992 since 1994. That means there is only the tiniest bit of independent variation between the two series. They move up and down in near-lock-step, as is apparent just from looking at the chart. The "truth" of U6 is never much different from what we'd expect based on U3. Since 1994, U6 has never been as much as 3/4 of a percentage point different from what we'd have predicted based on U3.  Of course it contains more people (by definition it is a super-set of the official unemployed) but it doesn't vary differently over time than does U3. In short, there is virtually no added information.

We can take another peak at alternative unemployment measures, this time at the state level annually since 2005.
u1tou6.pngHere you see the rise of unemployment nationally as you read the columns right-to-left. What was a small cluster of low unemployment in the lower left corner of the 2005 column has become a wide spread throughout the plot in the 2009 column. Both U3 and each of the alternative unemployment measures have increased, which is obvious. But the relationship between the measures have remained quite stable, with correlations between .86 and .99. 

For our current interest, U6 and U3 correlate each year at the state level between .95 and .96, again leaving very little room for more information to be extracted from the U6 measure.

As a matter of compassion for people who would like to work more, U6 may be a more inclusive measure, but as a matter of statistical information, as a matter of understanding the economic (and political) implications of unemployment, there is no advantage in choosing anything other than the official unemployment rate, U3. Calling U6 the "true" or "real" unemployment is misleading and a disservice to readers and listeners. It is a claim of added value when in fact there is none. 

Unemployment Holds at 9.7%


The national unemployment rate held steady in February at 9.7%, stubbornly down from the high of 10.1% in October and stubbornly refusing to decline from January's 9.7%. The White House has stressed storms but until we see the rate move down again, such talk is convincing only to economists and geeks. (State unemployment for January will be out March 10.)

Let's note the rate since President Obama took office,

Feb 09: 8.2%
Mar 09: 8.6%
Apr 09: 8.9%
May 09: 9.4%
Jun 09: 9.5%
July 09: 9.4%
Aug 09: 9.7%
Sep 09: 9.8%
Oct 09: 10.1%
Nov 09: 10.0%
Dec 09: 10.0%
Jan 10: 9.7%
Feb 10: 9.7%

12 month change: +1.5%

 6 month change: 0.0%

4 month change: -0.4%

1 month change: 0.0%

There is enough there to argue about. 

On the bright side for the administration is the first revision of 4th quarter real GDP, which was moved up from 5.7% to 5.9% growth. That is the number to watch as a better indicator of future growth and unemployment trends. If the economy continues to expand at this rate in the 1st and 2nd quarters economic optimism is likely to rise even with a lagging and gradual decline in unemployment.  Third quarter growth was, in contrast, revised downward from initial estimates to an eventual 2.2% growth. For comparison, 1st and 2nd quarters were -6.4% and -0.7% respectively. For the year, 2009 was terrible, a decline of -2.4%, compared to +0.4% in 2008 and +2.1% in 2007. 

Now we enter the 2010 campaign year with upward GDP movement for the second half of 2009 but climbing from a deep bottom of the recession. The stage is set for a narrative of recovery, but that narrative remains obscured by a stubborn unemployment rate and a preoccupation with an unpopular health care bill.

Texas Primary: Message Success, Message Failure


The Texas GOP primary for governor is a lesson in both message success and message failure. Gov. Rick Perry was badly behind a year ago. Last night he won by 20 points. His rise began in the spring, driven in large part by his embrace of strong anti-Washington and pro-Texas rhetoric. While outsiders found many of his comments "secessionist" and extreme, Perry showed a fine ear for his Texas Republican voters who are themselves quite anti-Washington and pro-Texas. If the rhetoric was at times overblown, it still resonated with his constituency, and what harm is there in a little secessionist talk if it stirs up your base. You know you don't mean it literally, regardless of how MSNBC interprets it.

Sen. Hutchinson faced the problem of how to "out-populist" an overblown but effective populist. She was never going to be able to out-do Gov. Perry on this dimension. (See the lesson's of Gov. George Wallace in Alabama in the 1960s.) So her only option was to find an effective critique of that populism. She never did.

The challenge is how to find an effective counter argument to a rhetoric that cannot be taken literally but which resonates with voters as populist calls to arms. Gov. Perry expressed a symbolic truth for over half of GOP primary voters last night. Sen. Hutchinson failed to convince more that 30% of them that those symbolic claims were in fact irresponsible and unrealistic. She could not find a way to play the grown up to Perry's teenager.

This has long been a democratic (small d) problem. When populist enthusiasms run hot, be it Joe McCarthy or George Wallace or Rick Perry, responsible grownups find it very hard to compete. Wallace and Perry, at least, were consummate politicians with fine ears for voters. That is what makes them so effective as candidates and what poses so difficult a problem for their opponents.

Same Data, Two Charts, Two Implications

This chart from Organizing for American drew a lot of comment today. On its face, it is a striking and strong contrast between the Bush and Obama records on jobs.  From a purely graphical perspective it is very effective in contrasting the rate of job loss in the past two years, and from a perspective of political rhetoric it is a strong claim that Obama has done better. And it has proven very attention getting, so it has served that political purpose as well.

But let's plot the same data in an equally relevant but strikingly different way visually. Let's look at total jobs lost over the past two years. This is simply the data above, but summed to show how many jobs the economy has shed and therefore how deep the hole is we still have to climb out of.

The OfA chart gives the impression that we have "returned" to where we were in January 2008. The sharp rise since February 2009 gives the impression that what was lost in red has now been regained in blue.  But of course, that isn't right. The rate of loss has indeed slowed tremendously in the first year of the Obama administration, something the White House has every right to crow about. But that doesn't mean we've returned to previous employment levels. In fact, we've continued to sink lower throughout the last year, just at a slower and slower rate.

This second chart makes that perspective on the data more clear. It is visually clear, if less dramatic than for OfA's chart, that the rate of job loss has slowed. But my version of the chart drives home the point that we have continued to lose jobs and now stand at over 8 million jobs lost since December of 2007. That is the other "deficit" the administration must worry about. The recovery, which GDP data show has started and at 5.7% growth in the 4th quarter is quite strong, will take a very long time to regain these lost jobs.  This fact is made clear in my chart, while it is obscured in the OfA presentation.

Interestingly, my chart is also subtly deceptive. More jobs were lost in the last Bush year than were lost in the first Obama year. But the red lines look shorter and smaller than the blue Obama lines. That makes the graph appear to show that things are worse for Obama, even though his job losses are actually about 3 million compared to Bush's 5 million. 

One can think of these two charts as data displays that reveal different aspects of data, but 
also as graphical political rhetoric. The different aspects of data are the sharp reduction in the rate of job loss shown so well in the OfA chart and the terrible cumulative loss to employment in the country that has not yet started to rebound that is shown in my chart. Both of those are "true facts" about the jobs data. They use exactly the same data, so differences are entirely matters of perspective and perception rather than "apples to oranges" comparisons. But while both are true stories, their substantive interpretations are quite different-- one is a story of an administration's success is stemming the tide of recession, the other is the high water mark of that tide, which has yet to begin receding. 

The other story is graph as rhetoric. The OfA is splendid rhetoric that seems to make an utterly persuasive point with simple yet bold graphics. But it is a rhetorical answer that conducts a slight of hand away from recovery of jobs lost to reductions in rate of loss. Credit worthy to be sure, but not so positive a result as the chart suggests. The rhetoric also succeeds because it has been so widely picked up and commented upon. Even the critics pass on the message that is sent by every viewing of the image.

My chart has its own rhetorical concerns. By focusing on the status of job losses, rather than their trajectory, mine shows the depths of job loss and the lack so far of a trend back up. Mine doesn't lie, because it too shows the reduction in rate of loss, but without a hint of even the beginning of recovery of jobs, mine clearly leaves the rhetorical impression that things are not only no better but are actually quite a bit worse than when Obama took office. The added optical illusion that the red bars are shorter than the blue, even though the opposite is the case, just adds to the false impression that most of the jobs troubles are within the Obama year.

Same data, two charts, two different impressions, both fundamentally true yet also fundamentally misleading in opposite ways.  When data and politics mix beware the power of graphs to imply their own conclusions, even with the same data. And appreciate the rhetorical success of a graph that does it's creator's bidding.

How Massachusetts Votes Shifted

The distribution of the vote shifted for Martha Coakley but not for Scott Brown last night. That was the key to Brown's win.

The top left of the chart above shows the distribution of Coakley's vote compared to Brown's. Brown's better total shifts his distribution clearly to the right. That's not interesting. But the bottom row is very interesting. The bottom left panel compares Coakley (dark blue) with Obama's light blue distribution in 2008. She's well to the left, doing worse. Of course you'd expect drop-off from a presidential to a special election. But the bottom right panel is amazing. Brown's distribution almost exactly duplicates McCain's. In a January special election, Brown's vote is a clone of McCain's in a presidential contest. That is amazing.

Here is another way to look at it. Plot last night's vote by town against their party's candidate in 2008.
Brown's votes are almost exactly on the 45-degree line showing equality between 2008 and 2010 vote totals by town. But not so the blue dots, which are all, yes every single one, well below the diagonal. Brown's total actually slightly improved upon McCain's. Coakley's total was just 56% of Obama's total.

The chart is powerful but the logarithmic scale makes the two clusters of points appear closer than they "really" are. Let's plot Coakley as a percent of Obama vote against Brown as a percent of McCain for a more compelling view.
Wow. The imbalance of performance is stark. Coakley's BEST town gave her 80% of Obama's vote. That's equal to Brown's WORST towns.  Even the towns Coakley won were places she was dramatically underperforming Obama.  And there are no pockets of strength visible here. Brown was doing over 100% except in a few blue towns but he even outperformed McCain in a number of towns that went for Coakley.

Of course this doesn't mean that Brown got exactly McCain's voters, since lots of individual switching could add up to these totals. But in the aggregate, Mass. in 2010 looks exactly like it did in 2008 on the Rep side. On the Dem side, a whole lot fewer voters.

Massachusetts Election Night Trends


We liveblogged these last night. Compulsion makes me post the final versions with 100% of precincts reported. Do note that it is common for the final certified vote to differ a bit from these 100% election night totals, so the final percentages and margin may move a tad.

Above is the trend in Brown's percent of the vote over the evening. Stable, but with a downward trend as the more Democratic areas reported relatively late in the evening after about 70% of the vote was in.

Below is the vote margin by estimated vote outstanding. It wasn't close enough to matter last night, but what you like to watch for is when the margin is getting big relative to the outstanding vote. At some point it becomes mathematically impossible to close the gap. That happened last night with 95.3% of precincts reported. Then the fat lady sings. (By the way, you read this one from right to left.)


Solid Brown Lead in Final MA-Sen Models

Republican Scott Brown holds a lead in all 18 alternative models of the Massachusetts Senate race polls, now including all polls released through 6:00 p.m. Monday. Our standard trend estimate puts the race at a 6.2 point Brown lead over Democrat Martha Coakley. The less sensitive alternative linear model puts the Brown lead at 7.3 points. Across all models, Brown leads by between 1.0 and 8.9 points. Three quarters of the estimates have Brown ahead by 4 points or more. 

Brown built this lead over the past week of polling with only some tentative sign of the trend flattening over the weekend. Of course the last available polls were completed Sunday evening so we do not know if any movement has occurred on Monday.

Here is a brief review of the polls and the various models estimated. First, the polls without any trend estimates:
One of the unusual features of the MA polls is the large number of leaks from Coakley's internal polling. No one leaks without a reason, and her leaks have been consistently better for her than other polling taken at the same times (with one exception). Past analysis has found that internal polls are typically about 5 points better on the margin for the leaker than are independent polls, but that the internal polls do track the trend rather well. This raises a question of how to treat Coakley's polls. Below, I estimate the models both with and without the leaks included, so we can see their potential impact. I have not discounted them for the historical five point bias with internal polls.

First, let's estimate the local regression models that are our standard here at Pollster. These are not identical to the dynamic charts because here I am estimating the Dem minus Rep margin while our charts estimate each candidate separately. I estimate the standard model, a more sensitive and a less sensitive version, and then repeat with the leaked polls included.
With the limited number of polls, the local its are not as smooth as in our usual trends with dozens or hundreds of polls. They may also be more sensitive to outliers. That is one reason to check the effect of sensitivity.  For the three models without leaked polls, the sensitivity matters a bit for the trajectory but hardly at all for the endpoints.  When internal polls are included, the trends end up a couple of points more Democratic, though still put Brown ahead in the end. (The more sensitive estimate touches dead even but that is due to a day with only an internal poll. The sensitive estimator chases that but then moves back down.)

Next I switch to even less sensitive linear models. The local trends show some non-linear movement, and suggest a small upturn in the last day of polling, which the linear models will miss. But with relatively few polls, much of the "bendiness" of the local trends is due to noise and overfitting the data rather than meaningful shifts. The linear fits are a hedge against the noise, at the expense of an ability to spot a reversal of trend. 
With these models we can disaggregate the data by partisan affiliation of pollsters. This gives a range of estimates from most favorable to Republicans to neutral (no picking of polls) to most favorable to the Democrats. Here I include the leaked polls in the most Democratic model, using only Dem polls plus the leaked internals, and in one model with everything we have regardless of source. The result is a range of estimates. The most Democratic model shows a 1 point Brown lead. Others range from about 5 to about 9 point Republican margins.

A final variation in the models is to fit quadratic models to allow the trends to bend according to how much the data demand a bend. If there were substantial upturns (or downturns) at the end, the quadratic model could pick that up while still maintaining less sensitivity than the local regressions we started with.
As it happens, the data don't demand much bend at all. The fit is only slightly better with the bend and the end points of the lines are only modestly changed.

Finally, let's see the garbage can of all the models at once, to see if any stand out as very different.
There is a range of estimates, but all are below zero, indicating a Republican lead. The two most Democratic models (using only Dem polls plus the internal leaks) stand out as most different from the rest. Half of the models fall between -4.3 and -7.4. The two most Republican estimates put Brown's lead at about 8.8 points.

The caveats are that turnout may yet matter, for either side. Reps enjoy an enthusiasm advantage, according to the polls, but Dems might yet mobilize their voters beyond what the polling suggests. And there is the unknown of the GOTV efforts on Tuesday.  But if Coakley wins, this will be a major surprise, and the pollsters will have a lot to rethink about their methods. A win for Brown will have huge implications for the Democratic policy agenda and will put the fear of God into Democrats running in November. 

What to believe about Massachusetts Senate Polls

There has been a wider than normal range of polling results in the last two weeks from the Massachusetts Senate special election. This has been further clouded by a number of leaked internal polls and polling by relatively unknown and unproven pollsters, some partisan but others not.  And most importantly, the rapid shifts in the race, reflected across all the polls, makes this a fast moving target. So let's take a moment to consider what we could reasonably conclude based on the data.

But no matter how you slice the data, the only reasonable conclusion is that Scott Brown has moved from well behind to a lead somewhere between 4 and 11 points.

The chart above shows all the polls we have available as of 12:36 a.m. Monday morning. That includes new PPP and Pajamas Media/CrossTarget polls released late Sunday evening. The chart also includes the leaked polls, mostly from the Coakley campaign but one from Brown as well.  These leaked polls are NOT included in most of the estimates above, though they are not out of line with the rest of the data. 

So what might you believe about these data? You could refuse to cherry pick the polls. That has long been our view here at Pollster.com. Our job is to summarize the trends as best we can, without partisan favor. If you do that, we get a 8.8 point Brown lead.

Perhaps you only trust non-partisan polls. Then the Brown lead is 6.8 points.

Maybe you are a Dem, who doesn't trust the Republican pollsters. Then Brown leads by 6.5 points.

Or you are a Dem who doesn't trust the non-partisan pollsters either and who does believe in the leaks from the Coakley campaign. Then Brown's lead is 3.8 points. (This is the only estimate that includes the leaks.)

Or you are a Rep who trusts GOP and nonpartisan polls only. Then Brown leads by 11.3. (There aren't enough Rep polls to run a Rep only estimate to parallel the Dem only, but I'd think an 11 point lead would be satisfying enough for Reps.)

There may be other ways to cut these data (IVR vs conventional phone, pollsters you've heard of vs ones you haven't) but it seems quite unlikely that any but the most selective reading of these data can find that the race remains a dead heat. Brown has a lead, as of Sunday night.

Let's back up a step to look at the data without the clutter. Here are just the polls, no trends fit.
Without the lines it is quite clear that the movement has been sharply towards Brown. Trace out what you like, ignore what you don't like, in the early polls Coakley is convincingly ahead. Then between about day -8 and -5 the polls are balanced above and below dead even. Since then no poll has shown a Coakley. 

In my models in the first chart, I use linear fits rather than our usual local regressions. The reason is there are still not very many polls,and once we subset them by party there simply aren't enough cases to get good local regression fits. That subsetting is the main point here. But it also turns out that the local regression on all the data isn't very far from the linear fits I use above. Here is the comparison:

The blue trend is our standard estimate, and it wiggles a bit due to only 12 cases. If we use a bit less sensitive local regression, we get the black line. And the red linear fit isn't very far from either of the two local fits. So I'm willing to give up some flexibility in the fit for a bit more robustness, and especially the ability to fit the models by party of pollster that was the lede above. 

Finally but significantly, we are seeing more pollster variation in this race than normal. If we look at the residuals around the trend estimates, past experience with 2004, 2006 and 2008 state and national contests has pretty consistently found that most of the polls (about 95%) fall within +/- 5 points of the trend estimate. Now that is an empirical observation, not a theoretical one. But it has been generally consistent in our data. How do these polls compare?
Only half of the current polls are inside +/-5 points of the linear trends. The number of polls is small, and this race is more dynamic than most. But one has to wonder about the problems of polling in a special election, the role of partisan and new players in the polling and the heavy use of IVR polls. This is much more variation in polls than we normally see in general elections.

Let's also recall the NY-23 special election, which was not polling's finest hour. The last three polls there had Hoffman up by 5, 5 and 17 points. Our final trend estimate based on all the polls had Hoffman up by 5,  41.8 to 36.8. 

Polling special elections is hard. Tuesday we'll see how hard, and who was good and/or lucky.

Obama as Reagan

Topics: Approval Ratings , Barack Obama , midterm , Ronald Reagan


I've been struck for some time by the similarity of circumstance between Presidents Reagan and Obama. Both replaced deeply unpopular predecessors. Both enjoyed significant gains for their party in both houses of Congress. Both faced "worst since the depression" economic circumstances. And each in his own very different ways attempted to reshape government in the early months in office.

With a bit more than 10 months of approval data on Obama, we can now make a more meaningful comparison than was possible at the first 100 days look.

The similarity of approval trajectories is striking for Reagan and Obama. Reagan started lower, but since the 3rd month of office the two have moved along quite similar paths.

Of the ten post-war presidents in the chart, Reagan and Obama currently stand as the two lowest at this point in their first term. (Clinton fell lower early, but was recovering at this point before another decline and rise.) Reagan finished as the second lowest just before his midterm in 1982, ahead of only Truman. It happens that the economy under Reagan also bottomed out in November 1982, the worst possible time for the president and his party.

(I exclude Johnson because of his entry into office after Kennedy's midterm and Ford because he took office just 3 months before the 1974 mid-term. I keep Truman because he assumed the presidency very early in Roosevelt's fourth term, effectively serving the full term.)

Whether Obama continues to look like Reagan seems to me more likely to be driven by the same force-- the economy. While health care reform and Afghanistan will surely play a role in the public's view of Obama, I think the economy remains the most crucial driver of opinion. In this the administration can hope that the upturn in GDP in the third quarter, and the small down-tick in unemployment in November, are signals that the early quarters of 2010 will see further improvements. If so, the Democrats may avoid the terrible conjunction of midterm and economic bottom that cost Republicans 26 seats in the 1982 House elections. And President Obama may not compete with Reagan to see which will be the second most unpopular president at midterm time. But there are no guarantees of this and the parallels remain quite striking.

Unemployment Trends in the States


It is of interest to look closely at which states have unemployment that is beginning to fall and which are still rising. Likewise which states are above and which below the national trend.

States are ordered from lowest unemployment at the top left to highest at the bottom right.

Click the chart for a large (1024x768) image for the details.

Obama Approval House Effects

Topics: House Effects

Obama Approval:

While many pollsters have yet to do very many presidential approval polls, we have finally built up enough to take a look at house effects. The red lines & dots are each pollster, the blue line is our standard trend estimate. I'll let the pictures have their thousand words and leave it at that for now.

Obama Disapproval:

Obama and Independents

Updated with latest Gallup data, see end of post.


A quick look at Obama trends with independents. There have been several articles in the last week about independents deserting the Dems. A good bit of that was spurred by the huge Rep margins among independents in VA (66-33) and NJ (60-30-9) governors races. There are also some indications on policy issues that independents are not supporting Democratic positions.

But support for Obama has not plummeted among independents, and that needs to be clarified before it becomes erroneous conventional wisdom. It especially makes no sense to compare independent support in January with independent support now, and conclude there has been a collapse of support. The pattern this fall, since Sept 1, has been quite stable among independents. Depending on which polls you use, a shade up or a shade down, but overall, not a huge trend either way over the past 3 months.

In the chart above I use Gallup's weekly aggregation of their tracking poll. This gives lots of cases each week, so we have good sample sizes for each of the three partisan groups. It is all within the same organization, so we also have an apples-to-apples comparison. My concern is the trend, not the level of approval. Gallup's daily runs a couple points higher than average on Obama job approval, but that's not relevant to seeing if the trend is changing. (And yes, I know approval dropped under 50 in Gallup's daily over the weekend. Give it a week or so and we'll see what affect this had on each party group but for now I care about the last three months.)

The clear message of the chart is that all three partisan groups have plateaued since the end of August, with little real change since.

What about disgruntled liberal Dems and angry conservative Reps? Let's look at those numbers:


There is no evidence that any group of Dems, especially liberal Dems are unhappy with Obama's performance. Critical is that moderate and even conservative Dems have not moved away since August. Angry conservative Reps are indeed very unhappy with Obama, at almost the same level of disgust as Dems felt for Bush, but they too have reached a plateau at a steady 10% approval. The small number of moderate Reps have also plateaued (I'd discount small moves in the last week of the aggregation.)

So the point is simple: Claims of abandonment of Obama by independents (or lib-Dems or con-Dems) are substantially exaggerated over the past three months. Significant decline from May through August, yes indeed among Inds and Reps, but that trend halted in August.

You can amuse yourself with our interactive pollster chart for independents, which includes all polls. You can remove whatever polls you don't like and see that it makes some difference to what you see recently. (including only live interviewer polls mirrors the Gallup weekly trends pretty well.) Our overall, no cherry picking allowed, estimate sees a small decline recently, that is not visible in the weekly Gallups. But it is not a large decline and until it turns more sharply, I'm not convinced we see a bit move among independents as it concerns Obama support. On other issues, yes Dems should be concerned. But not on this one. Not yet.

Technical note: The Gallup weekly plots are my local regression fits to Gallup's raw weekly numbers. They are not the raw Gallup percentages.

UPDATE: Gallup's weekly update came out this morning. As has been much discussed, Gallup's daily dipped below 50% in the last few days. So this update now reflects that week's polling and could alter the results of the old data.

There are some minor changes (as there logically must be with a dip!). But I don't think the changes we see in the last week change my conclusions. The basic stability of the last three months still show clearly. The last week sees a small dip (a point or two) among independents and among conservative Dems. Those are potentially important for the future, but my primary point remains: let's not exaggerate the changes that have occurred in the last 3 months, or confuse them with the substantial changes in May-August. If there is to be a new decline among any groups, it is a change that has not yet occurred, though this past week's Gallup results may be the first harbinger.

Also remember my lines in the chart are my local fits, not Gallup's raw numbers. The raw numbers show an overall dip of 4 percentage points, and 4 percentage points among independents. My local fits are bit resistant to change and will need to be convinced that change is real by it holding next week as well. (And by the way, I think this dip is not surprising-- I don't buy the arguments for a "floor" for this or any president. If you are close to 50%, odds are you will dip below it at some point. And you might rise too. Depends on events and how you handle them, not a fixed floor.)



Shifts in Vote and Turnout in New Jersey and Virginia


The shifts in outcomes between the 2008 presidential and 2009 gubernatorial elections in New Jersey and Virginia were driven far more by shifts in voting preferences among groups than by changes in turnout across those groups. Only age groups show consistently substantial changes in relative share of the electorate. Vote preference, in comparison, shows quite large shifts between election years. While one narrative of the 2009 election was changing turnout motivation, this turns out to be substantially false. Instead, changes in candidate preference drove the Republican wins in both New Jersey and Virginia.

The chart above shows the direction and size of change in vote preference for nine categories and 27 groups measured by exit polls in both years. The arrows start at the 2008 vote and point to the 2009 vote. The length of the arrow shows the amount of change and the arrow shows the direction of change. The colors code the shift in majority vote from 2008 to 2009. Blue indicates a Democratic majority for the group in both years. Red represents a Republican majority both years. Purple shows the groups that switched from a Democratic majority in 2008 to a Republican majority in 2009. None of the 27 groups switched from Republican to Democratic majorities.

In Virginia, large shifts in preference came among 18-29 year olds, those without a college degree, independents, rural voters and males. Smaller but still interesting changes came among lower and middle income voters, both of which shifted from majority Dem to majority Rep.

The most talked about shifts are among partisans and ideological groups. The large 16 point shift from 49 to 33 percent Dem among independents has justifiably received a lot of attention. But perhaps as interesting is the similarity of partisan loyalty among Dems and Reps. Neither shifted by enough to make the length of the arrow stand out. Virginia Democrats actually increased their Dem support by a point, while Republicans came home by a small 4 percentage points more than in 2008. Clearly the independents drove the dynamics of the outcomes.

Among conservatives, there was a modest shift of 9 points more Republican support in 2009, and a 5 point shift among moderates. Liberals moved a single point more Democratic.

By contrast, the shifts in share of the electorate were quite modest, as seen below.


By far the largest shifts are among the various age groups. The 18-29 year olds dropped 11 points, from 21 to 10 percent of the electorate. Those 30-44 also declined a bit, from 30 to 24 percent. These were matched by gains of 9 points among 45-64 year olds and of 7 points among those 65 and older. Age is one of the most potent predictors of turnout, and as this chart shows, one of the most dynamic from 2008-09.

The other two groups with interesting shifts are the rise in share of the electorate among conservatives (up from 33 to 40 percent of voters) and the similar decline in turnout among Democrats, from 39 to 33 percent.

Not only are these shifts substantial, but they also stand out against the very modest shifts in share of the electorate for all other groups. Many of the arrows have invisible lengths, indicating very small changes of one or two percent.

In New Jersey, we also see large preference shifts and even smaller turnout shifts.


The giant preference change in New Jersey is among independents, the same as Virginia. NJ independents took a massive 21 point shift from 51 percent for Obama to just 30 percent for Corzine. Also as in Virginia, Republicans came home to their party a bit, from a 14 percent defection rate for Obama to just 8 percent defection to Corzine. Democrats meanwhile barely budge, down from 89 to 86 percent Dem.

There were other substantial movements in vote preference, among 30-44 year olds, moderates, whites, hispanics and males. In short, many groups in New Jersey made substantial movements away from Democratic votes.

By contrast, the makeup of the New Jersey electorate changed a bit among age groups but hardly at all for virtually all other groups.


As with Virginia, there were declines in share of the electorate among 18-29 and 30-44 year olds and compensating increases among those 45 and above.

No other group comes close to such large changes in size. Several change by exactly zero (the open circles in the chart) and most others have lengths too small to see in the plot. The nearest exceptions are a decline of moderates of five percent and a corresponding five point rise among conservatives.

The bottom line for both states is that turnout changes were mostly about the age structure of the electorate. Younger voters are more responsive to short term stimulation, and in 2008 that translated to relatively large turnout, while in the absence of that stimulus in 2009 the more stable commitment to voting among those over 45 advantaged that group.

The shifts in preference in both states were significantly larger for many more groups. Preferences are driven by candidates and issues and those were the primary drivers of the change in outcomes from 2008 to 2009.

Below are alternative looks at the data, comparing the share and the vote for each state. These give a better look at the entire set of groups, but it is harder to compare magnitude of changes along the diagonal line in these charts than in the arrow plots above.





Election Night Recap, NJ and NY23


I'm headed to bed but here are a couple of snapshots of election night.

First, above, New Jersey then and now. Whatever else you say about the race, Corzine lost support across all regions of the state and by relatively constant amounts. This "uniform swing" shows that he didn't just lose in Rep areas, or Dem areas, or urban centers. The decline in Corzine support was very widespread and quite even. An across the board loss.

In NY23, Hoffman generally outperformed McCain's vote in 2008, but not by enough to take the race. These are based on 87% of precincts reporting, so not quite final data.


Finally, the dynamics of both NJ and NY23 were pretty stable. Despite differences in reporting times from various counties, the margins held pretty constant through the evening.


Since today wasn't exciting enough, President Obama will be in Madison tomorrow, so my day will be pretty well taken up with teaching in the morning and news coverage the rest of the day. I'll leave it to Mark and colleagues to provide the wisdom tomorrow. Happy Election Day Post.

NY23 Siena Poll Again: Some Hope for Each Side

The new Siena College poll of NY-23 has mixed news for both parties now that Scozzafava has suspended her campaign. The key question is how does her vote split between Owens and Hoffman. (And keep in mind she'll still be on the ballot, so some will vote for her anyway, and we have no idea how many that will be.)

Thanks to Siena for releasing cross tabs which allow us a much better look into the data. Here's hoping this practice becomes more widespread.

First, just as they are tied in vote, Owens and Hoffman are tied in favorable/unfavorable views. Owens registers 40/35/24 fav/unfav/dk, while Hoffman is at 41/37/22. Scozzafava has suffered from the campaign, with a poor 29/51/20 showing.

So while the campaign has been intense, Dede has been the primary victim, with Owens and Hoffman emerging at this point with essentially identical favorability profiles. 

The next question is how do Scozzafava voters feel about Owens and Hoffman? The answer is pretty much the same, and not very positive. 

NY23b.pngOwens is 19% Fav, 50% Unfav and 32% DK. For Hoffman it is 15%, 57% and 28%.  That's a tiny edge to Owens, but it is so small and flies against party identification that it is hard to see this as better than a wash. Certainly it doesn't look like Scozzafava voters will see Owens as a highly desirable second choice, but at best a poor second.

The undecided voters (9% of the sample) are also evenly split in the favs towards Owens and Hoffman, with identical 24% favs, and a 24% unfav for Owens to a 20% unfav for Hoffman. As befits undecideds, mostly they don't have an opinion: 52% DK for Owens, 56% dk for Hoffman.

Conclusion: probably remains an even split, based on these results.

What about Party Identification? Here is some bad news for Owens. He's losing 25% of the Democratic voters, versus only a 13% defection rate among Republicans.
Fully 14% of Dems say they will vote for the Conservative Hoffman. Another 11% were going for Scozzafava. Even if you think all those Scozzafava Dems come back to Owens, the party is not as unified as it needs to be.

Independents are also leaning Hoffman by 40-35, with only 15% supporting Scozzafava up for grabs.

Ironically, it may be the 29% of Republicans (45% of the Siena sample of likely voters) who could be attracted to Owens. They've by definition resisted Hoffman, but now with the party united behind him, it seems unlikely Owens can capture a disproportionate share of these Republicans who resisted the Conservative's siren song.

For Owens to make gains here, he has to see Dems coming home overwhelmingly from Scozzafava (not unreasonable) and to see a lower defection rate for Hoffman (somewhat harder to do.)  And he also has to hope that the Scozzafava Republicans are so upset with Hoffman that they defect to Owens (but we just saw above that Scozzafava voters generally split their affections evenly. We don't know how Scozzafava Reps specifically feel about Owens and Hoffman, so perhaps that group breaks more but given the power of party id, it seems unlikely to be especially fertile territory for Owens.)

It is the combination of these results that led me to say the Siena poll is bad news for Owens in my earlier "quick post". Now the logic is better laid out. It isn't that this looks like terrible news, but there isn't much good news in it either. A toss up remains a tossup, but with some partisan forces acting mostly in Hoffman's favor rather than Owens'. 

Finally, there is one interesting note that COULD be a plus for Owens. Scozzafava and undecided voters are MUCH more like Owens supporters than Hoffman on one key point: They like President Obama quite a lot:

If this race were a referendum on Obama, then the Hoffman voters look just like they should-- angry anti-Obama folks, while the Scozzafava and undecided look a lot more positive to Obama, and hence potentially attractive Democratic voters. 

Perhaps the best move Owens can make in the last three days is to drape himself in the cloak of Obama, hoping to bring home those wavering 25% of Dems, and use this favorable view of Obama among Scozzafava and undecided to bring in the margin of victory.  

Conversely, if Hoffman wants to win the Scozzafava and undecided, he should probably push Republican  loyalty more, and opposition (especially angry opposition) to the president less. He's already won over the voters with pitchforks and tea bags. He needs a strategy to close the deal with Reps and others who don't actually despise the president. (Recall the district went 52-47 for Obama.) 

So after all that, it still looks like a tossup on the two simplest most direct measures: current vote choice and favorability. When we try to parse the Scozzafava voters, they mostly look like a tossup, with at most a sliver of extra support for Owens. But at most a sliver. 

Owens must win back defecting Dems to have a chance. And some nationalization of the race even at this late date might help more than emphasizing partisanship alone. The former can both bring home Dems and win over some Scozzafava support. 

What the data don't address is how much the Republican party (and its national leadership) now united behind Hoffman can swing the majority of Scozzafava voters.

Bottom line is this will be fun on Tuesday night.

(Special thanks to Siena's Steven Greenberg and Don Levy for getting me some extra detail on what is surely a very busy day for them.)

NY-23 Siena Poll Bad News for Owens

Updated: I've now posted a less hasty look at the data. Change "bad news" to "mixed news". There is some good and some bad for both Owens and Hoffman. Still a tossup is the best characterization. The new post is here. I stand by the bad news mentioned here: Owens is not doing well among independents, and the new post shows Dems are defecting at 25%. So the headline here doesn't need to change. On the other hand, there are some weaknesses for Hoffman (and opportunities for Owens) among the Scozzafava voters, which I take up in the new post. ---charles

This is a quick note on the new Siena poll in light of Scozzafava dropping out.

Can Owens pick up from Scozzafava supporters? Not so likely given these poll results.

Owens Fav/Unfav among Scozzafava suppporters: 19/50
Hoffman Fav/Unfav among Scozzafava supporters: 15/57

Looks like a wash with many likely to skip the choice of two disliked alternatives.

And the worse news for Owens is among independents:

Ind. Fav/Unfav Owens: 39/47
Ind. Fav/Unfav Hoffman: 47/33

More after daughter's soccer game!

NY-23, RINOs and "Tea Party" Candidates

BREAKING: Scozzafava suspends campaign. Divided party now unites. Demonstrates powerful appeal of right in Rep pty. What remains to be seen is if this appeal can win the general election. Amy Walters had a good catch at Hotline: Hoffman leading among independents. Thats good news for him. But other unknown is how Scozzafava's 20% split now. Are they loyal Reps or were they looking for more progressive issues and hence not likely to back Hoffman? With all attention on Rep split, has Owens given the district reason to back the Dem over a conservative? Fun!

NY-23 is the poster child for the conservative insurgency in Republican politics. What looked a few weeks ago to be a splintered party in NY-23 is looking more and more like a consensus developing in favor of the conservative candidate. If Scozzafava continues to collapse, Democrat Owens will no longer enjoy a fractured opposition but a united one.

Mark has posted extensively on the problem of polling in NY-23, so be sure to see his comprehensive post here. There are many problems with polling in this race, especially on the turnout side.

But setting those concerns aside for the moment, Hoffman's surge shows that an insurgency can succeed in winning over (so far) well over half of Republican voters. The string of Hoffman endorsements from Palin, Pawlenty, Pataki and others shows how seriously Republican leaders are taking conservative insurgents.

Given substantial investments in the NY-23 race by both party and independent groups, this might be seen as the best case for insurgents against the party establishment. The converse is that if Owens pulls it out, NY-23 will be a symbol of party fratricide. And if Scozzafava's support collapses, it will be a message of the power of party unity around conservative nominees.

I commented on this topic in a recent Christian Science Monitor story here, and spent a pleasant hour this morning discussing this and other insurgent challenges on Wisconsin Public Radio's Joy Cardine show. The audio is available here. At the time of CSM interview, Hoffman had not yet surged, something we talk about in the audio link.

Senate 2010 Receipts, Expenditures and Cash

The Hotline gave us a summary of the 3rd quarter FEC reports for 2010 Senate candidates. Here I offer two views of the races. The plot above shows receipts against expenditures for the 3rd quarter. Above the diagonal line means more receipts than expenditures, below the line and you have a bad burn rate.

Solid dots are incumbents, open circles are non-incumbents. Blue for Dems, red for Reps (surprise!)  Open a second browser window to use the table at Hotline to see who's who in the charts. (You do have two monitors, right? Let's use them!)

The chart below shows receipts against cash on hand. Here we see the built up advantages most incumbents enjoy regardless of current receipts.
The ideal view is the chart below, combining the two above. It is formatted for a screen 1440 pixels wide and probably won't be very clear on smaller screens. But if you can view it full screen you can read across the two charts-- see the same level of receipts vertically and how that compares to expenditures and cash on hand horizontally. Click the chart to see it full size---it is too small to read in our standard window width.

 Let's take a look at the races Charlie Cook at the Cook Political Report rates as tossups

Key trick to reading the charts: Look for blue and red points close together and towards the upper right. Those are closely competitive candidates with significant resources.  Where you see open circles of the same color close together, you see potential competitive primaries. 

The Connecticut race is shaping up as a huge big money race. Three Republicans have over $1 million cash on hand and incumbent Democrat Chris Dodd has only $2 million in the bank, a small advantage for an incumbent. Ex-WWE CEO Linda McMahon and Ex Ambassador Thomas Foley both have large loan investments in their campaigns, while ex-Rep. Rob Simmons is relatively debt free and raising substantial amounts from others. Dodd raised less than McMahon or Simmons, and has a high burn rate as well. This race looks like the smallest incumbent financial advantage around, either by receipts or by cash on hand. 

Senate Majority Leader Harry Reid, whose polls have been shaky, has a competitive opponent in "Wall Street Banker" John Chachas who has pumped $1 million into his race, leaving a big debt but making his receipts competitive with Reid's $2 million. But now shift your gaze to the cash on hand chart and you see Reid's gigantic advantage in money in the bank, where he holds almost $9 million to Chachas' $1.3M. (The next strongest Republican is ex-UNLV basketball player Danny Tarkanian with just barely a quarter million raised and less than that in the bank.) Reid may be weak in the polls, but unless a large infusion of Republican donations start flowing into Nevada, Reid will be able to outspend any opponent. (Newly announced ex-NV GOP chair Sue Lowden won't file her first report until January.)  

In Illinois the open seat of Roland Burris is very competitive. Rep. Mark Kirk (R-10th) has the Republican primary all to himself in terms of money advantage, with no other Republican raising even 1% of Kirk's take in the 3rd quarter. But the Democratic race is much more balanced. The clustering of open circles in both plots shows that no Dem enjoys a big money advantage. State Treasurer Alexi Giannoulias ($1.1M raised, $2.4M cash) has a modest advantage but self-financed candidate lawyer Jacob Meister (giving himself a cool $1M) and ex-Chicago Inspector General David Hoffman (who loaned himself $500k) have pumped considerable money into their campaigns. The other serious contender, Chicago Urban League Pres. Cheryle Robinson Jackson, raised just $367k with $318k in the bank.

Kentucky's open Republican seat has real competition in both parties. Republicans Rand Paul (son of Ron) and Sec. State Trey Grayson are evenly matched, as are Democrats Atty. Gen.  Jack Conway and Lt. Gov. Dan Mongiardo. (Conway didn't report on expenditures, so is missing from the expenditures chart but not from the cash on hand chart.) This state looks like two competitive primaries as well as a competitive general. 

In Missouri's open Republican seat, each party has a clear front runner and those two are evenly matched so far in the money race. Dem Sec. State Robin Carnahan raised $1.1M with $1.8M cash while Rep. Roy Blunt (R-7th) brought in $1.3M with $2.3M cash. Carnahan has a small lead in recent polls. Close enough for another barn burner in the Show Me state.

In New Hampshire Republican ex-Atty. Gen. Kelly Ayotte outraised Democrat Rep. Paul Hodes (D-2nd) but Hodes has more in the bank. Polls show a tossup with Ayotte holding a slight edge. Should stay competitive. 

Ohio's ex-OMB Dir. and US. Rep. Rob Portman is running away with the Republican money, and has a significant advantage over Democrat Lt. Gov. Lee Fisher, by 2-1 in receipts and over 3-1 in cash. (Dem Sec. State Jennifer Brunner failed to provide data on her financial status but said she raised less than $228k brought in for the 2nd quarter.) Despite trailing in money, Fisher has a modest lead in polling as of September. Dem money needs to open up there to stay competitive. 

The loss of incumbent Republican Kay Bailey Hutchison to the governors race has so far left Republican contenders trailing in the money race.  Rep. Joe Barton (R-6th), who has not announced an official entry into the race but expressed "interest",  is best off, with half a million raised in the 3rd quarter and $1.7M in the bank. In second place for the GOP is ex-Sec. State Roger Williams, with $336k raised, and $863k cash.  But both Democrats have double that in cash and both out-raised Barton in the quarter.  Houston Mayor Bill White has an advantage on the Dem side (including 1/3 in self-financing) over Ex-Comp. John Sharp, but it could still be a competitive primary as far as money goes. (Sharp failed to supply expenditures so it missing from that chart.) Others may still enter the race and if Hutchison resigns there could be a May 2010 special election just to add to the fun.

Of the races Cook rates as "leans" rather than tossups, Colorado is looking better for Democrats worried that first term appointee Michael Bennet will have a struggle to hold the seat. So far he is well ahead in both receipts and cash over several Republican contenders, the strongest of whom, ex-LG Jane Norton raised half as much in the quarter and has only one-fifth as much in the bank. Still, Bennet has not polled well, so Rep money coming in could make a difference here.

The Pennsylvania race looks very interesting, with ex-Rep. and Club for Growth ex-Pres Pat Toomey dominating the Republican primary contest with $1.6M raised, $1.8M cash. Rep. Joe Sestak's (D-7th) intra-party challenge to newly converted Democratic incumbent Arlen Specter is so far uneven in money, with Specter both raising and banking quite a bit more. Sestak has a substantial $4.7M in cash, but Specter has $8.7. With that kind of money available for a primary it will be interesting to see how damaged the winner of the Democratic primary will be as they enter the general against a well-funded Toomey.

The Florida Republican primary between Gov. Charlie Crist and ex-FL House Speaker Marco Rubio is so far continuing to advantage Crist. Crist raised 2.5 times as much and has 6 times as much in cash. Also, Rubio's cash burn rate has been high, spending more than Crist in the quarter while taking in so much less. On the Dem side, Rep. Kendrick Meek (D-17th) is far behind Crist with $772k raised and $2.7M in cash. North Miami Mayor Kevin Burns raised just $28k in the 2nd quarter, and hasn't filed a 3rd quarter report.

Of the "likely" races in Cook's rating, North Carolina's incumbent Republican Richard Burr has a strong money advantage. And Democratic incumbent Blanche Lincoln in Arkansas has yet to see a Republican challenger closing in on money, though State Sen. Gilbert Baker stands out as the one potentially competitive candidate though trailing in cash by 4-1.

We should close noting that money alone doesn't win. Challengers need enough to compete but they don't have to match vulnerable incumbents. These data are most interesting for helping show which non-incumbents are bringing in support at levels that would give them enough to compete, in either open or incumbent seats. 

The View from the Depression, an Illustration

A bit off our usual polling focus, but Joe Nocera has a nice article in the New York Times Friday. It reviews the recently published depression diary of Benjamin Roth, giving  a contemporary account of what it was like. Nocera does a great job of using the diary as a vehicle to talk about the policy issues of that era and ties them to our current issues.

This post is an illustration for his text. The fall, then rise, of unemployment, and the rebound then second drop of the stock market in the 1937-38 period are an implicit warning for our current condition as Nocera discusses. 

And here is our current unemployment condition, in states and nationally. Click the chart for a full size version.


ADDENDUM 10/18/09: Mark Sadowski posted a great comment below on the controversy over which measures of unemployment should be used to examine the Depression and New Deal era.  This is a debate that has had a flourish of renewed interest as some have argued that the New Deal had little effect on unemployment. Mark's comment below provides a great overview of these issues with pointers to the relevant literature. 

For comparison, here is another version of the chart, with the original and revised data plotted. While the difference may matter for some purposes, as an illustration of Nocera's article it is the timing of shifts that matters, and those shifts are closely parallel.


The Afghan Dilemma


The end of the summer saw some rebounding of President Obama's domestic approval ratings, and his overall job approval.  But it is now the foreign front that poses the greatest potential to damage the president's approval ratings. Afghanistan looms large.

Domestically, things have turned in Obama's favor. After falling through much of the spring and summer, support for Obama's handling of health care reform has taken a noticeable upturn since mid-August. Approval of his handling of the economy is flat or slightly up, and his overall job rating has risen 3 points or so since it's low point at the end of August. Even his worst approval rating, on handling the deficit, has stopped declining, though it remains below 40%.

But the problem is now with foreign policy matters and most specifically with Afghanistan.  Through the spring, even as he increased troop levels in Afghanistan, Obama enjoyed an approval rating on that war of near 60%.  But those ratings have taken a sharp dive over the last four months, even as domestic issue approval rebounded. Since June, approval of his handling of Afghanistan has fallen some 17 points, from about 60% to about 43%.

During much of the summer conversation focused on the damage a falling health care approval was doing to Obama's overall job ratings. Now the attention should turn to the potential damage of  whatever decision he makes about how to wage the Afghan war. Increasing troop levels, and therefore casualties, will not be popular among progressives in his own party, and will surely not win back any Republican support. But refusing to increase commitments there is likely to result in little visible improvement while maintaining American targets, and therefore casualties. While the president may have succeeded in bringing back some support for his domestic agenda, that was an easier sell within his party than will be the case with Afghanistan. The driving mechanism with Afghanistan is going to be casualties and perceptions of progress or lack thereof plus a base of ideological opposition to the war among Democrats.

The declining approval on Afghanistan has also begun to be reflected in declines in approval of his handling of foreign policy in general. (Handling of Iraq and Iran have relatively few recent polls, so are omitted from the figure.) While foreign policy approval has been a strength, generally running higher than overall job approval, that may not be the case much longer.

If the military decisions are daunting either way, the political implications are at least as difficult. A Democrat committing more troops to a war now eight years old, with troops stressed from repeated deployments, is going to be a hard, perhaps impossible, sell to his partisans. But with Republican support now down in single digits, the president can look for support only from Democrats and some independents. Those groups are quite reticent to support an escalation.

Pollster.com/PollsAndVotes Video Review of Recent Polling

PollsAndVotes Video offers a review of the state of polling and politics as of early October. The video is an exclusive of "Office Hours with Ken Goldstein", which is a production of the University of Wisconsin and is broadcast on the Big Ten Network.

I talk about Obama, the economy, health care reform and the critical support for Blue Dog Democrats who may hold the fate of Obama's legislative agenda in their hands.  Alas, no charts this time.

Sixty-Five Views of HC Reform, One Big Trend


The trend in opinion on health care reform has been a bit tricky of late. After a long and substantial rise of opposition, and an equally long but less sizable decline in support, we came to August, the month for the strange in politics. This year was stranger than most with loud and angry town halls, fearful politicians reluctant to meet constituents, cable news of gun toting demonstrators, an extended presidential vacation and the death of an icon. And what did all the rancor produce? Apparently, the sound and fury signified, surprisingly, a flattening of the trends in opinion. Opposition slowed its rise (not accelerated), and support halted its fall and by the second week of August, began a modest rise. And that before the president spoke to the joint session of Congress.

The picture is complex in the details, but essentially unanimous in the major points. Let me lay the details out a bit.

The chart above shows all the trend lines we might estimate for our health care questions, using 65 different levels of sensitivity for the smoothing. At one end, we smooth little and get a very (overly) sensitive fit. At the opposite end we smooth a lot and get a quite insensitive trend. In between is our standard estimator. The overlapping lines above make clear that the big picture is a common pattern over all 65 different degrees of sensitivity: A rise, then a flattening of opposition, and a smaller fall then a rise in support. Off to the right, you can see "bar codes" showing all the different possible current trend estimates from each of the 65 different levels of smoothing. The range of possible current trends is small, but there is a little overlap indicating some estimates may show support slightly ahead of opposition, though most show the opposite.

If you want to pick a fight, then you can pick a level of smoothing that is a little different from the others in some details. The chart below is the most sensitive estimator, one that is quite likely to chase noise, but which also picks up short term changes.


Now you can see a post-speech spike in support, and a smaller drop in opposition. Both are sharp, but both also show the beginnings of a reversion in the latest polls, with support beginning to dip and opposition steady or slightly rising. If you believe this estimator, then the speech mattered, but wasn't a game changer because the short term gain for Obama has now begun to reverse.

Or, you could use a still sensitive but not quite so erratic estimator:


And, Aha! the speech changed everything and the trend is now declining opposition and rising support! Well, maybe. But this rests on being sensitive enough to pick up the speech effect, but not so sensitive as to turn down based on those very latest polls.

Or you could use our standard estimator, which is "sensitive enough" but which tries not to chase outliers. It needs several polls to convince it that the polling trend has really changed.


Here we see an upturn, probably driven by the speech, but the blue line hasn't caught the red line yet.

If you compare the red line in the chart above, with the red line in our interactive chart today, you'll see a bug. For reasons not yet clear, the interactive chart is producing a straight line for the opposition trend despite the fact that our standard estimator is really the red line above. Mark mentioned this mysterious bug in a previous post here. We continue to hunt it and hope for a quick fix. (For the really nerdy, we have to stabilize the line when few polls are available, and sometimes it seems the "stabilizers" kick in when they shouldn't. Probably due to the very first poll which is far from the rest. Take it out of the interactive chart with the filter option, removing HealthDay/Harris, and you get estimates much closer to the standard trend above.)

Notice that the blue line above and the corresponding black line in our interactive chart are very similar. So the interactive chart is doing the right thing with one line but not with the other. The interactive chart overshoots putting opposition at 49.9% while my estimate above is 48.5%. Meanwhile support is 45.4% on the interactive chart and 45.8% here.

Finally, what if we intentionally over-smooth-- ignoring the day-to-day noise for the long term "fundamentals" trend:


The same basic story is apparent. Opposition has grown but is now slowed to a near halt. Support reversed its decline sometime in August and has begun an upturn.

And my big point is that this is essentially the picture you see in all these different trend estimates. The details are slightly different. A bump here and a drop there, and the precise estimates of support and opposition differ by as much as 2 points up or down. But the big picture is that opposition ramped up significantly through June or July but has recently slowed or stopped. Support fell less precipitously but has been working back up for a month (despite or perhaps because of the circus coverage in August.) We could pick a chart to fight over the details, but we shouldn't. It is the big picture of public opinion that is important here. Within a couple of points, opinion is evenly divided. The White House has gained a bit of momentum, but will be challenged to lower the opposition numbers, not just raise the support numbers.

Here is a chart of how the current trend estimates depend on the degree of smoothing. The range of opposition is 47-49 and the range of support is a little wider, 45-49.


How Fast Does Approval Fall?

Topics: Approval Ratings , Barack Obama , Gallup


David Brooks commented in his column yesterday "All presidents fall from their honeymoon highs, but in the history of polling, no newly elected American president has fallen this far this fast." Mostly right, sort of. But some context would help.

First, Obama's approval has fallen 16 points from his first Gallup poll reading to his most recent, finished 8/31, from 68% to 52% approval.  That's a smidgen more than Bill Clinton, who fell 14 points from his first poll to his last prior to September 1, 58% to 44%. One might pick a nit with Brooks that while Obama has fallen two more points he started substantially higher and that 52% now is quite a bit better than the 44% Clinton had at this point in his term.

(All the polls used here are by Gallup which is the only organization with polling reaching back to FDR. I'm looking only at first term elected presidents here, so no Truman, Johnson or Ford. FDR's first term predates polling, of course. I include all polls taken prior to September 1 of the first year in office.)

A look at the chart above also shows that NOT every president "falls from their honeymoon highs", at least not by September 1. Of the 9 presidents in the chart, five actually gained in approval from late January to the end of August, and one more finished where he began. Usually the gains are modest, but the first President Bush gained 13 points over this time, and President Reagan picked up 9.  More modest gains of 6, 4 and 3 points came to Eisenhower, Kennedy and Nixon respectively. Carter held even at 0 change. So of the nine, only the second Bush (-2) and Clinton and Obama have dropped between first poll and the end of August. That, of course, aids Brook's main point which is that Obama has suffered significant loss of support compared to other presidents-- it is just not the case that everyone falls off over the first 8 months.

A more important nit is that simply comparing the first and last polls ignores all the ones in between. Polls, as we know, jump around quite a bit, so a little bump up early or late can distort the rate of approval change. It is better to use all the polls to get the best estimate we can of the rate of change.

The chart below shows the estimated monthly rate of change in approval, using all the polls from January through the end of August. I fit a simple linear trend to each president. Not all change linearly, for example Obama fell, rose and then fell more steeply. A linear fit misses the nuance, but it gets the basic story right-- how fast have approval ratings been changing when we take all the data into account. The chart below ranks the presidents according to their estimated monthly change in approval.


Based on this estimate, Obama has the second greatest rate of decline, losing 1.6 percentage points of approval each month. Clinton did worse, falling at a monthly rate of 2.3 points.  If you look back to the first figure, you see that Clinton fell faster and further, but then rebounded a bit at the end of the summer, making his net change a little smaller (-14) but his overall rate of decline a bit steeper. Obama's chart shows two phases, an initial shallow decline with a bit of a rebound and a more recent decline at a higher rate. Linear fits don't distinguish between these details. But it is pretty clear that based on the linear trend, Clinton fell faster than Obama, contra-Brooks.

What is more interesting is how many presidents managed only slight declines or even gains in the first 8 months. Carter and the second Bush both lost just over a half point per month. Kennedy, Nixon and Reagan all gained, from 0.1 to 0.2 points per month. Eisenhower gained an impressive 0.65 points per month, and the first Bush gained an astonishing 1.9 points per month. So no, not everyone falls off. But yes, Obama has fallen, and quite badly in comparison. Just not quite the absolute worst.

We might put Obama's trend in perspective of his predecessors as well. The next chart shows the trends for all these presidents, highlighting Obama.


This chart offers one important corrective to a widely repeated myth: Obama was not "wildly popular" at the start of his term. This exaggeration is repeated so often that it is becoming a universal "truth". But the fact is Obama entered office with strong support, but not so strong as that of Eisenhower or Kennedy, and after a couple of weeks his approval was above average but only by a bit. It is this myth that adds to the drama of his current fall, but the myth exaggerates how high in the pantheon Obama first stood.

The second useful perspective of this chart is that much of the decline has come after 3.5 months, or starting in May. The decline since then has been quite steady, dropping Obama from a shade above average at 3.5 months, to a next-to-last place now. This second slide should be far more worrisome to the White House than the initial polls or the net change from first to last poll.

Finally, we should remember that the first 8 months do not tell the future. Clinton ranks dead last as of September 1, yet finishes with one of the highest approval ratings, and maintained high approval for longer than anyone since Eisenhower. Nor could GW Bush's approval on 9/1 say anything about his approval following 9/11.

The best perspective is the full term, and with time marked by elections rather than months. The chart below provides this for all presidents since FDR's 3rd term. The scales are exactly comparable across presidents so your eye can show you differences in trends. And the vertical blue lines mark off mid-term and presidential election dates. Those are the moments that matter. A low approval on November 2, 2010 will push Democratic losses in the House well above 20 seats. But a rebounding approval (driven by a better economy) can hold those losses below the historic average. That is the future still in Obama's hands to write. And that is basically what David Brooks was saying to the White House.


Walter Cronkite, Most Trusted Man in America

Walter Cronkite, iconic news anchor for CBS, died July 17 at age 92. For my generation of news junkies Cronkite and the Chet Huntley-David Brinkley team at NBC defined television news. (ABC was a late bloomer-- we didn't get ABC in south Alabama in those days.) Tonight, in the shadow of the 40th anniversary of the moon landing, it is impossible not to remember our personal history through the images of Cronkite or Huntley or Brinkley on network TV telling us the great stories of the '60s and '70s. We don't have to romanticize or idealize the shortcomings of network news to appreciate the jobs they did. 

Cronkite has often been called "the most trusted man in America", and the surprisingly limited amount of public polling on this topic backs that up. The earliest poll I could find was an April 1974 Virginia Slims poll, done by Roper, that asked how much the respondent respected as list of public figures. Cronkite scored 60% "a great deal" and another 31% "somewhat" with only 4% "not at all". Secretary of State Henry Kissinger was close to Cronkite at 56/29/8. No one else was close. President Richard Nixon, 4 months from resignation, scored 24% "a great deal", and 41% "not at all". 

Cronkite was also among the most recognizable figures of the 1970s. A July 1975 Roper survey found 93% could correctly identify him as a TV journalist. 6% said they didn't know and less than 1 percent thought he was anything else. Harry Reasoner was correctly identified as a TV journalist by 82%, but 5% thought him a sports star. Actual sports stars, tennis champion Chris Evert and pitcher Nolan Ryan, were correctly identified by only 56% and 23% respectively. The one person in the survey who was the near equal of Cronkite was Johnny Carson. Like Cronkite, 93% correctly identified him as a TV entertainer, but 3% thought Carson was a TV journalist.

In the month before Cronkite's retirement in 1981, a Louis Harris & Associates poll probed Cronkite's standing with the public: "If you had to choose, who would you say is your favorite nightly network TV (television) news anchorperson--John Chancellor, Walter Cronkite, Peter Jennings, Ted Koppel, Roger Mudd, Frank Reynolds, or Max Robinson? " A full 50% picked Cronkite, with 12% for Chancellor in second place.

Asked specifically if Cronkite was "someone you could really trust" 81% said they could, and on 12% said not. (The question was a rating with excellent+pretty good as trusting, and fair+poor as not trusting. Breakdowns in the original categories are not available.) Sadly, no other news anchors were asked about on this question so we lack comparison. Even more poeple, 86%, said Cronkite presented a "balanced treatment of the news". Only 9% said not.

In April 1985, four years after he left the anchor's desk, a Roper/US News & World Report survey rated a number of TV journalists. The Excellent/Good/Don't Know ratings were
Cronkite 47/37/5
Rather 28/44/10
Jennings 22/36/28
Brokaw 18/37/28
Koppel 15/32/43
Lehrer 7/11/74

Finally, in July 1985 the Times Mirror Center for People and the Press (predecessor to the current Pew center) sponsored a poll by Gallup that was explicitly comparative. Their question was "Please rate the believability of the following people, using this same scale of 4 to 1 (where '4' means you can believe all or most of what they say and '1' means you can believe almost nothing of what they say."

These data are plotted in the figure above. Whether we look at percent in the most believable category or the mean score the order of results is essentially the same: Cronkite on top by a substantial margin. 

Most of those with low rankings are also not well known, hence a lower percent in the top category simply because few respondents know how to rate them. But this doesn't much affect the relative rankings, and doesn't touch Cronkite's dominance. If we redo the figure, calculating the percentage in the most believable category but omitting all those who can't rate the person, the basic chart is unchanged though some publicists will be pleased by improvement while others disappointed. (The correlation of raw rating and adjusted is .90, so the order doesn't shift in large ways in any case.)
So given the somewhat limited data available, the title "most trusted man in America" may actually have belonged to Walter Cronkite, at least among television journalists and a few other prominent media figures. None of these data, collected over 11 years, place Cronkite anywhere other that at the top of visibility and recognition. And the later data on quality and believability ranks him first in every measure.

That isn't a bad reputation to have created and sustained on the Evening News. 

Palin's Problem Outside the Base


Mark's must read post on Gov. Sarah Palin's support among Republicans tells one crucial side of the Palin story. Regardless of views among the entire electorate, her strength rests powerfully among those in the party who continued to support her after the election and through the spring. Go read his post now if you missed it.

But let me focus on the negative-- Palin's problem outside the base. Of all the dimensions on which Palin can be viewed, the one that is most crucial for any national ambitions she may hold is the most fundamental: is she qualified to be president? One might focus on her issue positions, her personality, her policy knowledge but the most basic question voters could and did ask in 2008 was "is she qualified to take over as president if that became necessary?" That's what all the other details boil down to. So let's take a quick look at how the 2008 campaign affected that view among voters.

From the announcement of her pick as VP through the convention and before her first national news interview with ABC's Charles Gibson, Palin had a small plurality seeing her as qualified to be president rather than not qualified. For an essentially unknown governor from a remote state, this reflected a mix of partisanship, trust in McCain's judgement, a well received convention performance and a bit of benefit of the doubt.

Confidence in Palin's qualifications declined following the ABC interview, though this also coincides with the financial crisis and the suspension and then resumption of the McCain campaign. That period also represents a shift of support which had briefly trended in McCain's direction following the GOP convention. So we should be reluctant to attribute all the change in views of Palin to her ABC interview alone and other events undoubtedly affected perceptions. Nonetheless, by mid-September a significant plurality of voters had come to see Gov. Palin as not qualified to be president.

In late September Palin appeared on the CBS Evening News with Katie Couric, first on September 24 and 25, then again the next week on the 29th and 30th. Two polls taken after the first two interviews and simultaneous with the second two, suggest some further negative impact of these appearances. And of course, reaction among political professionals was that the interviews were disasters.

These interviews were followed almost immediately on October 2 by the Vice-Presidential debate, which was generally seen as a success for Palin, certainly in comparison to the expectations set by the Gibson and Couric interviews.

The flurry of polling following the VP debate clearly shows more movement in public opinion. While the percentage seeing Palin as qualified held steady at about 40%, those seeing her as not qualified rose from just under 50% before the Couric interviews to about 53% after the debate.

But the ultimate problem Palin faced with the electorate at large is not about individual events of the campaign, but about the overall trend. From the beginning to the end of the campaign, Palin steadily lost ground with the electorate. Each week more voters perceived her as unqualified to be president. Her base of support was about 40%. Those seeing her as qualified declined from the high 40s to a stable 40% through the last 2/3rds of the campaign,  with one final poll falling a bit below that.

The "not qualified" trend rose, from the low 40s in early September, to nearly 60% by election day.

Of all the things about Palin that might be relevant to her future, this is the most important. During a campaign in which she had her best chance to present herself to voters, and in which she chose her message to a considerable extent especially after the VP debate, Palin failed to convince voters she was qualified for the presidency. In fact, she did the opposite.

The enthusiasm and size of her crowds late in the campaign is testimony to her appeal to the base that Mark highlights in his post. But to be a contender for the presidency requires her to dramatically lower that 60% that thought her unqualified by November 4.

For all the commentary about Sarah Palin, this is the fundamental perception she must change. I doubt that Friday's announcement has done anything to improve those perceptions outside her base.

Rep. McHugh to Army Secretary, Another NY Special Election

President Obama's selection of Rep. John McHugh (R-NY23) to be Secretary of the Army creates another special election for New York and another threat to Republicans' tenuous hold on just 3 House seats out of 29 from the state. In the March 31 special election for NY 20 (replacing Kirsten Gillibrand) Democrat Scott Murphy barely held the seat with 50.12% of the vote. The 23rd congressional district is similar in party registration to the 20th and past results suggest this should be a winnable seat for both sides.

The chart above shows that the 23rd and 20th are quite similar in registration- Republicans hold a 13 point edge over Democrats in the 23rd and a 15 point advantage in the 20th. On this basis, these are the two strongest Republican Congressional districts in the state.

Rep. McHugh has prospered in the district since his election in 1992, consistently winning well over 60% of the vote. But that margin reflects the advantages of incumbency a well as the partisan leanings of the district. Given the experience in the 20th, this could easily produce a close vote in the special election.

Gillibrand won the 20th in 2006, defeating incumbent Republican John Sweeney 53% to 47%. In 2008 Gillibrand cruised to reelection with 62%. And that in a district slightly more Republican by registration than is the 23rd. Gillibrand's success, however, didn't transfer easily to the special election in the 20th. Murphy prevailed after an extended recount with a 726 vote margin.

The chart shows that registration advantage alone has not  made for safe Republican seats in New York. While all three current Republican representatives come from districts with a Republican advantage, there are six seats held by Democrats despite a Republican registration advantage.

Based on recent results, incumbency and registration, a Democrat should win 48.4% of the vote in the 23rd to 51.6% for the Republican. By comparison, the same model predicts 47.5% for the Democrat in the 20th, a mark Murphy bettered by 2.6 points in the special election.  By this standard, the 23rd is just slightly more favorable for a Democrat than was the 20th. However, both districts would have been predicted to go Republican in a dead-even campaign for an open seat. The campaign, quality of the candidates and randomness in the model means the 23rd should be considered very winnable for both parties, despite McHugh's long running success in the district.

To change the focus, consider the 2008 Presidential results in the 23rd and elsewhere. The chart below shows the Democratic presidential vote by district party registration.
Obama performed slightly better in the 23rd than he did in the 20th, winning the 23rd with 52% to 47% and the 20th with 51%-48%. Obama fared less well in 8 NY congressional districts, suggesting that the 23rd is willing to deviate from it's partisan Republican leanings. Obama over-performed in the 23rd, a district predicted to go only 47% Dem for president.

Taken together the recent congressional and presidential results say the 23rd should favor a Republican for the House seat, but by a small margin of 52 or 53 percent Republican. The experience in the similar 20th district shows that a Democrat can improve on that but the razor-close margin in March (and the April recount)  indicates the need for a strong candidate and a favorable campaign. Both parties look capable of nominating high quality candidates in the 23rd so much will hang on who decides to enter and how well they play the game.

Before leaving the 23rd, let's put Rep. McHugh in perspective ideologically. Using National Journal's 2008 ratings based on a wide range of roll call votes on economic, social and foreign policy issues, McHugh is more moderate than most in his party, but noticeably more conservative than any Democratic member of the House in 2008. First, let's compare foreign policy with economic ratings. Higher scores mean more conservative.

McHugh is more moderate than most in his party on economic issues though relatively less so on foreign policy. A couple of Democrats have similar economic voting records but none come close to him on foreign policy.

On social issues McHugh is a tad further to the right though still easily in the left half of Republicans.
While moderate for a Republican, McHugh's social issue voting record is clearly to the right of the most conservative Democrats in 2008.

The Democrats McHugh most closely resembles are Jim Marshall (GA-8), Nick Lampson (TX-22), Dan Boren (OK-2) and Travis Childers (MS-1), though none of these are as conservative as McHugh across all three dimensions of economic, social and foreign policy.

By picking McHugh for Secretary of the Army, Pres. Obama has reached beyond the ideological range of Democratic House members, a hard to deny display of bipartisanship, while staying to the moderate wing of the Republican party. Despite his voting record on social issues, McHugh is said to favor a review of the "don't ask, don't tell" policy in the military, a subject that came up at Press Secretary Gibb's briefing. Exactly what McHugh's view is on the subject is not yet clear, other than calling for a review. With many on the left doubting Obama's commitment to gay rights this could become an important issue. By the same token, a Republican Army Secretary interested in reversing DADT would be well positioned to respond to conservative critics of such a change.

With this selection Obama has further advanced his claim of bipartisanship. In the process he has also transformed a safe Republican seat into a tossup or lean-Rep special election.

New Gallup has PID Tied. Yep, It's an Outlier.

The latest Gallup (5/7-10/09) poll has party identification tied at 32-32 and caused an immediate howl of "outlier!" in the comments at Pollster.com.  In this case, the howl is justified. Compared to all recent Gallup polls (so we compare apples to apples) this latest stands out quite a bit from the rest.

The chart above shows the trends since January 2001 for Gallup polls for Dems, Reps and the Rep minus Dem margin. This latest poll is circled for easy reference.  The Dem percentage is a bit below trend, while the Rep percentage is considerably higher than trend. The difference of the two is therefore quite high- a difference of zero compared to a trend estimate of -9.3.

The Dem value is just inside the 95% confidence interval while the Rep value is outside the CI, as is the difference.  (The Dem and Rep estimates are, of course, correlated, though not perfectly so since Independent is an available option as well.)

Is there anything that might explain this outlier? One intriguing possibility is that this Gallup poll is their annual "Values and Beliefs" survey, which spends most of the interview asking questions about many of the classically partisan moral issues including abortion, gay rights, stem cell research, the death penalty and other topics. (Gallup has not yet released the full questionnaire so my list here is based on the content of past surveys in this series and may be wrong about some details.)

We know how powerfully question order can sometimes affect subsequent responses. Since party id is usually asked late in the survey it is plausible that a lengthy set of questions about these social issues would affect independents who might have recently called themselves Republican. Priming these issues might be expected to provoke more pro-Republican considerations, which in turn could shift the partisan balance. That would be interesting as a finding because of the supposed durability of party id and would invert the outlier finding because the outlier would be the result of the content of the survey, rather than a mere random fluctuation in the makeup of the sample.

Happily we can test this using the five previous Values and Beliefs surveys, which are done annually in early May. (No party ID data have been released for the 2001-2003 surveys, so this analysis is based on 2004-2009 only.)  If the question order effect is real, we should see the pid distribution shift in a pro-Republican direction each year, since the content is similar and should provoke similar imbalance in considerations among independents who might lean Rep if they thought about these issues.  The chart below shows the residuals for the pid series, this time circling all the Values and Beliefs polls.

The data are quite clear: There is no evidence at all that the Beliefs and Values surveys are systematically skewed in party id. Only the new 2009 edition of this series is an outlier. It looks like idiosyncratic sampling fluctuation remains a more plausible explanation.

This "null finding" is worth appreciating for a moment. I had high hopes for the hypothesis, which fits well with research on both priming and question order effects. And yet we find no difference. This too is consistent with the literature: question order effects have proved irritatingly difficult to provoke on purpose. We find large order effects sometimes, yet when we try a different treatment which is seemingly just as likely to have an impact, we get nothing. This seeming randomness in when we can and cannot provoke order effects is a cautionary tale about being too quick with post-hoc explanations for outliers based on question order. These effects may be real, but require substantial evidence before we accept them. Theory alone is a poor guide to empirical reality in this case.

It is easier to be confident about the outlier status of this poll than to account for why it is so clearly out of line with previous Gallup results. At least we can address the outlier status empirically and with some statistical confidence. They "why" of that status must remain the always true maxim: "Outliers Happen."

Robert Groves Census Confirmation Hearing on C-SPAN

Topics: Robert Groves

Bob Groves' confirmation hearings were held today (5/15). C-SPAN has the hearings at the link below. Grove's opening statement is well worth listening to and begins at about 19 minutes into the video. He strongly speaks to the need for a non-partisan approach to the census and maintaining a focus on the scientific issues that confront the census and it's many data gathering activities.

Click here to link to the C-SPAN video.

The C-SPAN Summary:


At his confirmation hearing in front of the Senate Homeland Security & Governmental Affairs Cmte., Census Director nominee Robert Groves said that statistical sampling in the 2010 census will not take place. Mr. Groves was nominated by Pres. Obama in April.

Approval Trends and Pollsters, Part 1

Topics: Approval Ratings , Barack Obama , Charts , Rasmussen


We get a lot of questions, comments and complaints about the effect particular pollsters have on our trend estimates. This is an important question, and today I'll start a series of posts on this issue. I want to encourage your comments and feedback. Over the series of posts I'll try to answer what I can, and we'll improve our approach when you raise points we aren't doing well enough and can improve on.  The focus will be presidential approval, but many of the issues are generic.

Yesterday Mark posted on the Rasmussen daily tracker and whether IVR interview methodology was enough to explain the generally low approval readings from that poll. Here I want to extend this to address the frequent comment the Rasmussen is systematically distorting our trend estimates because his results are consistently below the trend line.

Let's start with a bit of data. Rasmussen represents 90 polls in the Obama series above, while Gallup's daily provides 87 polls and all other pollsters contribute 55 polls. 

Even a casual glance at the figure makes it clear that the Rasmussen dailies run 2-3 points below the blue trend line, while Gallup's daily runs about the same above the trend. The other pollsters scatter widely around the trend.

The most common comment we get is that Rasmussen is clearly too low and is distorting our trend estimate downward. If we removed only Rasmussen, it is certainly true the trend estimate would shift up. But the problem is how do you "know" that it is Rasmussen who is wrong?  As one commenter put it "It's so annoying to see Obama at 57% when everybody knows he's over 60%!!!"  Well, that IS annoying if you "know" the truth, but how do we know the truth?  When I talk to Republicans they are equally certain we "know" Rasmussen is right and that it is Gallup that is obviously wrong.  How can we address this difference of views in a non-partisan, data oriented, way?

The best estimates we get for our trends are when we have lots of different polling organizations represented and none of them contribute a disproportionate share of the polls. When we get in trouble is the opposite, when one poll dominates and we have few other polls to calibrate against. An extreme case would be if we only had Rasmussen right now, or only had Gallup. Happily, that isn't the case.

At the moment we have 55 polls by firms other than Rasmussen or the Gallup daily (I include 3 USAToday/Gallup and 1 Gallup only  polls which are not dailies). These 55 polls come from 23 different firms with the most from any one firm being 4 polls.  This is just what we want for a standard of comparison-- lots of pollsters, none contributing too many.

This doesn't mean there are no house effects. Every polling organization has a house effect, some larger than others. But across all the pollsters we get heterogeneity in those effects with low balancing high and the result being the best estimate of the trend we can manage with polling data alone.

The chart above estimates the trend using only these 55 polls from the 23 non-daily pollsters. That trend is plotted by the black line. The blue line is our standard trend estimate, using all the polls, including the dailies. And for comparison I've show the trends for Rasmussen only and for Gallup daily only.

Clearly both Rasmussen and Gallup are quite different from the overall trend or from the non-daily trend. Pick your poison, neither of these is in agreement with the non-dailies. You can prefer high or you can prefer low, but the dailies are about equally far off the black trend.

But the key point for us is that the black line for non-dailies is very close to the standard blue trend using all the polls. The average absolute difference is barely 1 point (1.009, in fact) and 95% of the days find less than a 2 point difference between the blue and black lines. Sometimes blue is higher and sometimes black is higher. The average difference (not absolute difference) is that blue is 0.3 points below the black line. (The black line is a bit more variable because it uses only the 55 non-daily polls rather than all 232 polls.)

There are cases where we can't do this sort of analysis because of a lack of diversity in pollsters. Approval is a happy exception. It is clear there are pollster differences, but at this point they are not drastically affecting our results. If you SELECTIVELY exclude only low polls, then of course you can drive up the trend, just as you can selectively exclude only high polls and drive the trend down.

But when we take the most diverse collection of polls, we get pretty much the same trend estimates as we do with all the polls.  (You can go to the interactive charts and pick what to include or exclude and see how big a range you can get. Selection of high or low polls is the key to making the trend move a lot.)

Now, this is only part 1 of this series. I'm not claiming our trends are infallible. Far from it! I know all too well that they can break when given too little data or various kinds of bad data.

In the next installment of the series I'll respond to your comments here, and show an example of a more problematic case.

The Irony of Tax Cuts

The Republican driven tax cuts have worked. Voters now have more faith that the federal income tax system is fair than at any other time since World War II. Moreover these changes in public opinion coincide with the Republican capture of the House in 1994 and accelerate with the Bush tax cuts. But the irony of this success is that the GOP finds it hard to claim credit for a job well done. Once the tax monster is slain, who do you fight next? Once the people are no longer grumpy about unfair taxation (tea parties notwithstanding), how do you keep the issue alive? By successfully shifting public views of the fairness and burden of federal income taxes through repeated cuts, Republicans inadvertently also reduced the salience of their best issue of the 29 years since the Reagan Revolution. The public now agrees that tax cuts are good, but they are no longer particularly angry about taxes.

The logic of the tax cut issue has been that people should keep their money rather than send it to Washington. But where do you stop that spiral short of reducing everyone's taxes to zero? If lower taxes are good for individuals and good for the economy, the theory offers no logic of where to stop.

Today Ari Fleischer, former press secretary to President Bush, offered an interesting proposal in the Wall Street Journal: raise income taxes on those who currently don't pay. That is a rather shocking proposition from the party that has spent nearly 30 years arguing that tax cuts are good for everyone. The very success of that political program has been to remove millions from the tax roles, and put nearly half within striking distance of paying no federal income taxes. So you'd think that would be cause for celebration among Republicans for a job well done and a lot of credit to claim with those voters.

Alas, those voters aren't voting Republican in overwhelming numbers. The way to not have to pay taxes is to not make a lot of money. And while these less taxed citizens appear to have been pleased with lower taxes, that hasn't translated into a majority of Republican votes among these non-taxpayers. So Mr. Fleischer has now taken on the burden of convincing nearly half of the public that it is not good for them to pay little or no income taxes. Instead, fairness demands that everyone pay taxes. That's a breathtaking argument for a Republican to make.

Fleischer goes on to argue that it is poor policy for the top 10% of earners to pay 72% of all income taxes, and that is probably a discussion worth having. But the argument for raising income taxes on the bottom 90% to provide a little more load sharing for the top 10% is an interesting electoral calculus to say the least.

Fleischer makes a pitch for a total tax system overhaul, dropping all federal taxes except income taxes and allowing no deductions or credits for anything. No Social Security taxes, no Medicare taxes. Just income taxes. He's quiet about the rate structure or about revenue neutrality. Dubbing this the "Economic Growth Code" is interesting, given that Ronald Reagan was eloquent that cutting income taxes was the way to stimulate economic growth. President Bush argued that too.

If we look at the data in the top chart, we see that patriotic support for taxes in WWII dropped sharply once the war ended. Alas, pollsters stopped asking the fairness question for nearly 30 years as well. But when we rejoin the question in the mid-1970s, the public is quite strongly of the view that the income tax is unfair. That feeling was very little affected by the Reagan administration, or the first Bush term. Perceptions began to move in the mid-1990s as Republican control of Congress gave the tax cutting policy proponents opportunities to affect legislation. The public began to respond to tax cut messages with a modest improvement in the "taxes are fair" trend. That shift accelerated in the second Bush administration, presumably driven by the joint effects of the Bush tax cuts and unified Republican anti-tax rhetoric from both White House and Congress. And so we find ourselves at the outset of the Obama administration with only 37% saying their taxes are unfair, while 60% think they are basically fair. That is nearly an exact flip of where things stood at the start of the Reagan Revolution.

Beyond fairness, Americans are also complaining less about how much they pay in taxes, though here the shift in opinion is not nearly so dramatic.
While the shifts are smaller, Republican successes are still clear. During the Reagan and Bush I administrations, those feeling they paid too much declined by close to 10 points. But that reversed with Clinton in the White House, as more complained of too great a burden, largely erasing the gains from Reagan-Bush. The second President Bush however succeeded in sharply reducing complaints about high taxes. The greater than 10 point drop in his terms was greater than that achieved by the Gipper. Ironically Obama arrives in office with a public almost evenly divided between thinking they pay too much and thinking the tax burden is about right. Obama's plan to lower taxes for more of the lower 90% (or 95%, whatever) plays to the anti-tax momentum Bush built. And it means that Republicans don't have the angry taxpayer revolt of the late 1970s that helped build the Reagan platform that transformed tax policy for a generation of Republican politicians.

And so we are left with the irony of Republican success. How do you keep tax cuts at the center of your economics when nearly half don't pay, but aren't as grateful as they might be. And if the issue doesn't have the mass appeal it did for Reagan, can it still motivate the base (remember those tea parties!) enough to continue to have legs. But I have to wonder if Mr. Fleischer's plan is really the way for the anti-tax party to go.

(Data note: The question wording varies across polling organizations. The quoted texts are typical but are not universal. Results from 12 different polling organizations are included. Gallup, with it's long history, contributed 58 of the 106 items represented in the charts. The last datapoint in each chart is the latest Gallup poll, completed 4/6-9/09.)

Turnout by Age, 2000-2008

Michael McDonald has done a great first look at the Current Population Survey voter turnout data that were just released at the end of last week. Be sure to read his discussion of the new CPS data.

Here I just wanted to update the turnout by age chart that I did last fall. What is striking is that voters under 30 increased their turnout rate over that of 2004, which in turn was up from 2000 by a lot.  But more surprisingly, voters over 35 decreased their participation rate from what it was in 2004.  These are not huge differences, but they are quite consistent for the over-35 group.

In contrast to the 2008 pattern of increase among the young and small declines among the older citizens, the 2004 pattern showed an upward shift for all age groups, though more so for the young.

There will be time to attack these data with more sophisticated tools, but one has to speculate that among older voters the appeal of Obama was less stimulating than among the young for Democratic constituencies,  and perhaps among Republican leaning constituencies there was a slightly greater willingness to stay home than in 2004. Let me label this clearly as speculation-- I've not run analysis to address this yet. The change in turnout in the CPS overall is very slightly down (by 2 tenths of a point) from 2004 to 2008, but we know the actual vote count was up a bit, by 1.6 points by McDonald's Voting Eligible Population estimate. 

So the CPS estimate may be a little at odds with the vote data, but the relative turnout among age groups tells an interesting tale.  Even with increases among the young, the classic pattern of increases in turnout with age still holds, and the recent upturn of voting by those under 30 has served to reduce that gap, but has not come close to putting an end to the age-turnout relationship.

NY-20 Vote: 50-50 with Absentee Out

Ny20Votes.pngWith 100% of precincts reporting the NY-20 special election has come down to Scott Murphy (D) 77,344 to James Tedisco (R) 77,279, a 65 vote margin. Call the Minnesota Senate Case lawyers! TPM reports some 5907 absentee votes remain outstanding so that will surely be interesting to count.

The district is made up of 10 counties or parts of counties.  The chart above shows how the vote in those counties have shifted from 2002 to 2009. The Republican seat shifted a bit Dem in 2004 and then was swept over to the Democratic column in 2006 with 53.1% of the vote. In 2008 that margin (aided by incumbency) expanded to 62.1%.  Without an incumbent it is no surprise we saw a closer election tonight, but would it be more like the 53% of 2006 or fall short of that, perhaps even back to a Republican majority? As it happens, back to 50-50.

Tedisco was helped by a strong performance in Saratoga county, which largely overlaps his state Assembly district and is also the largest turnout county in the district, about 1/3 of the district. That was a county that supported Gillibrand in her win in 2006 and landslide in 2008, in both cases at about the same rate as the district-wide vote. Such was not the case tonight. Saratoga went for Tedisco over Murphy by 30,247 to 25,837. It was almost enough.

 It is revealing to compare the 2009 and 2006 election results, since 2006 represents a close, 53.1%, Dem win. Murphy had just over 3% to give. 
Murphy lost just over 6% from Gillibrand's 2006 total in Saratoga county, Tedisco's best gain.

Other counties shifted towards the Republican candidate by 3 points, more or less, with Renssalear moving the most, almost 5 points pro Rep.  Dem gains were more limited and in smaller counties, Washington, Warren, Essex and Delaware. Net effect: Murphy lost all he had to give, save 65 votes. 

How did the polls do? The last Siena poll had the race 47-43 for Murphy (inside the margin of error.) Our trend estimate put it at 47.0-42.4. Both over estimated Murphy's margin, and had nothing to say about the 10% undecided. The simple polling average, however, had Tedisco up 45.6 to 37.6, a larger overstatement of Tedisco's margin. 

If the Minnesota Senate case is any indication, NY-20 may still be waiting for a winner on the 4th of July. Who got the absentee vote out, and who did the best job of filling out the ballot right? Stay tuned. 

Wednesday Update:

Turnout in Tuesday's special election was down but still pretty healthy.
With just over 150,000 votes case, that was down 100k from the close 2006 midterm, a 40% decline. But given typically low turnout in special elections held in the spring, not a bad effort by voters. Note that the results of the past four election don't vary in any important way with turnout. But the key question now is those 5-10 thousand absentee ballots, or 3.3-6.6% of ballots cast on election day. 

Trends and Averages in NY-20


The New York 20th congressional district special election is upon us tomorrow. Many commentators, on both sides, have considered it as a referendum of sorts, on Obama, or Michael Steele, or on Congress (either side) or on the stimulus or the budget or the war on terror. Take your pick.

In my more parochial world of polling analysis I tend to see it as a test of how we think about polling aggregation. Should we focus on estimating the current positions of the candidates based on trends or should we use simple averages of recent polls? RealClearPolitics has always preferred the simple averages, though so far as I can find they haven't posted a NY-20 average. I've always preferred trend estimates because they capture the all important dynamics of the race. The polling in NY-20 this time presents the difference in these two aggregation approaches rather vividly.

The chart shows the five polls we have for the race along with the trends and the simple averages (as diamonds on the election day line to the right.) This is a race in which the trends are both very strong and the polls surprisingly consistent with those trends. The better known political veteran, Jim Tedisco, started out at 50% support to novice Scott Murphy's 29%. That was February 3-4 polling. Since then Tedisco has slipped a little to 43% while Murphy has surged to 47% in the Siena poll done March 25-26. That's a lot of movement for any campaign. This last poll shows Murphy leading for the first time. (The Tedisco campaign claims their internal polls continue to show him in the lead. A Democratic source claims their recent polls show a 2 point Murphy lead. See that part of the story here.)

What I'm interested in is the different conclusions we reach based on simple averages versus trend estimates, and when they may differ from each other.

When a race is stable, with no trend either way, then the trend estimate and the simple average will agree with each other. A "flat" trend is just the average. But when we have sharp trends, as in this race, the two can differ a lot. The simple average of the five available NY-20 polls has Tedisco at 45.6 and Murphy at 37.6. In contrast, the current trend estimate reverses this with Murphy at 47.0 and Tedisco at 42.4. Which should we believe?

What makes the trend persuasive in this case is how consistent the polls have been in showing Murphy's gains and Tedisco's (more moderate) declines. While there are only five polls, the results for both candidates are remarkably close to the trend lines. And the trends aren't affected by dropping any single poll. The NRCC poll done by Public Opinion Strategies in early February is just as close to the trend lines as the DCCC poll done by Benenson in late February- they differ due to the trend, not due to one being out of line with the rest of the data. Likewise the three Seina polls have shown the same close match to the trend line and a steady gain for Murphy and slow decline for Tedisco. It is this consistency across polls that makes the trend compelling.

Imagine if we mixed up the order of the polls. Ignoring order we have Tedisco at 45, 50, 43, 46 and 44. An average of 45.6 +/- about 3. Likewise ignoring order we have Murphy at 41, 29, 47, 34 and 37. An average of 37.6 +/- about 9. If this random order were really what we were seeing, we'd be justified in using the simple averages, and would want to comment that while Tedisco's results are within sampling error, Murphy's are considerably more variable than we'd expect. That would be a story of noisy polls, and the best we could do is the simple average (with due note of the noise level.)

But these really aren't noisy polls. If we calculate the deviation of polls from trends, we get a surprisingly small range: only 3.4 points for Tedisco and 0.8 points for Murphy. That's another way of saying the points are all really close to the trend lines, especially in Murphy's case. That's very different from the conclusion we'd reach based on the hypothetical unordered sequence of the previous paragraph.

When we have sharp trends like these, the simple average will lag behind the current trend because the average is ignoring the order of polls, while the trend uses that order as a central element for understanding where the race stands. The trend estimate is always an estimate of where the race was when the last poll was taken, while the simple average is an estimate across all polls regardless of date. In the case here, it is clear we get very different conclusions between the two methods of aggregating polls.

With so few polls, the linear fit is more reliable than the local regression trend I normally use. For less than about 15 polls the local trend can be affected quite a bit by an outlier or two. But in this case I stretched the point and show the local fits in the chart as well. It is clear that the local trend follows the linear trend quite closely, with a small exception of the Tedisco result for the Benenson/DCCC poll. I'd make nothing of this because there isn't enough data for the local fit to be reliable on it's own. The fact that it isn't far from the linear just confirms the obvious-- the polls tend to follow a straight line trend quite well.

So this leaves us with two aggregations that predict different results tomorrow night. The trend estimates have the race tighter, and have Murphy ahead. The averages reverse that order with a wider margin for Tedisco. It is certainly a stretch to say that Murphy is a lock. There are just five polls, even if the trend is strong. And my colleague Mark Blumenthal posted here last week on the difficulties of polling in special elections. I tend to side with experienced candidates over novices, so I'd give the benefit of the doubt to Tedisco on that score. But the trends are quite consistent and point to at least a close finish and a modest advantage for Murphy. We'll see tomorrow night.

(You can also check the NY-20 race in our usual interactive chart here.)

Job Approval, Outliers and Perspective

An absurd amount of attention has been given today to a new Zogby internet poll. In a very well managed PR campaign, Zogby leaked the results last night before releasing them a bit after noon today, while also debuting a Zogby presence on Twitter. A little Googling will turn up many hits citing the not-yet-released-at-the-time results. Nate Silver at fivethirtyeight.com helped the poll get attention by attacking it before it was even released. The right is delighted-- "Obama approval drops below 50%!!" The left (with Silver's help) denounces the poll. Twitter is abuzz on both sides. And media, both old and new, couldn't contain their chatter over the results. (My favorite, the Boston Herald's Joe Dwinell's lede "The honeymoon is over, a national poll will signal today as President Obama's job approval stumbles to about 50 percent over the lack of improvement with the crippled economy." The fourth paragraph notes that "Some polls show Obama coasting with a 65 percent job approval, but not in Zogby's tally.") 

It takes five seconds to put the new poll in perspective. Take that long to look at the chart above. The Zogby poll stands out pretty clearly in the chart, no? 

To make a lot of the Zogby poll is to deliberately ignore the context in which it appears. Yes, the left is correct that it is an outlier. A huge one. The right looks desperate or ignorant by embracing this result as meaningful. 

But rather than suppress the poll or ban it from polling analysis, let's just put it in the chart and let the data speak! No stats needed to get the point. 

Poll results are far more meaningful when we look at them with a bit of perspective. The chart also makes clear how the Gallup and Rasmussen daily tracking polls differ from one another as well. Gallup's daily tracker consistently results in higher approval ratings than do non-daily national polls. Rasmussen consistently results in lower approval ratings than do non-daily national polls. But put them all together and we get a trend estimate based on all the polls that matches the trend for conventional non-daily polls quite well. If we pick only Gallup or only Rasmussen, we bias our understanding of the state of public opinion. Not by a huge amount, but by a persistent two-or-three points up or down compared to the overall trend or to the non-daily polls. This is no surprise. House effects, the tendency of polling organizations to produce modest differences from one another, are well known and much written about. By putting all the data in the chart for everyone to see we let these house effects stand out in an obvious way. 

If someone still wants to cherry pick a result to suit their partisan preferences, then fine. That's called politics, duh! I'm not against it. But let's not confuse willful distortion for partisan purposes with analysis of what the data actually show. 

The Zogby internet polls have a considerable history that I won't repeat here. The point is not to condemn them but to put them in perspective. That done, I don't think much more comment in required.

Obama Approval and the Battle of Interpretations

Topics: Approval Ratings , Barack Obama

Obama approval is at 59.7 based on my trend estimator, the blue line above. A lot or a little? High or low for other presidents? According to a Wall Street Journal Online article today by Douglas Schoen and Scott Rasmussen, "Obama's Poll Numbers are Falling to Earth".  But Nate Silver at fivethirtyeight.com counters "Yes, Obama's Approval Ratings are Declining. What did you Expect?"  The Schoen and Rasmussen piece notes a number of polling results that point to potential trouble for the new administration. Silver argues that the pace of the first 50 days is more like a full year for most presidents, so we should not think the current decline is that much and should compare other presidents at 365 days to this one at 50.  Ahhhhh... the power of predispositions.

Schoen and Rasmussen point to a number of results that should concern Democrats, were they in a mood to care about public reaction to the new president and congress and the implications those have for the next three years and 315 days. But their piece is entirely about driving a negative thesis, and picking evidence to suit. The Journal editors help that cause along with this graphic, presumably what we are supposed to think of when we read a headline like "Obama's Poll Numbers are Falling to Earth":


But that red line is just a tad off. If the scale were close to the relative change we see in approval in the chart of the actual data above (remember that blue line?) it would look more like this:

Just about as distorted as the one-sided interpretations of the data we see.

Democrats are just as inclined to find ways to deny the decline matters. One approach is to ignore results they don't like, such as Rasmussen's low daily tracker compared to Gallup' daily or other conventional polls. Silver takes that tack, but good for him for at least showing the Rasmussen data in the charts.  The problem from my point of view is that if Obama is "about average" for 50 days, then why are Dems so anxious to come up with reasons why that should not worry them? Silver is right that we should expect some decline as partisan lines become more well defined. But we should really be expecting significant decline as the economy fails to turn on a dime in the next few months. Any responsible economic analysis expects a significant lag before the stimulus has an effect, and assumes business conditions will improve after that for fundamental reasons as well. But not before the last quarter of 2009, at best. Ask Ronald Reagan how that worked out for him, as his approval sunk to 43% around midterm election day 1982.  Dems need to plan for the long term, which is likely to be pretty disappointing over the next 24 months. They have to hope for a late rebound like the one that powered Reagan to his "morning in America" landslide, just 2 years after the economic and approval bottom. 

Finally, is Obama now at or below President George W. Bush numbers at day 50? Depends on how much you want to cherry pick the polls. Here is what the real evidence is, with Clinton for good measure:

Yep. There are five polls at about this time where Bush was equal to or ahead of Obama, and the one closest to today is just about dead equal.  Problem is there are a bunch more polls lower, so that the trend estimate for Bush is 56.5 on day 50, versus Obama's 59.7 on that day.  President Clinton was at 56.

Bush and Clinton ran parallel through about day 60, then Clinton went into a true tailspin, bottoming below 40 by day 130. Bush held steady at the mid-50s through August 31. 

These are modest poll differences that mean very little for the long run success or failure of an administration. The Obama folks would do well to keep their eye on the policy ball, and work hard to survive losses in 2010 and adopt policies with long term payoffs in 2011 and 2012.  Republicans should take their shots when they can, but not count on a small decline in approval now, and cherry picked results, to make Obama seem in much worse shape than he is. The GOP needs more thoughtful analysis than that to Rebuild the Party. Both should look to their supporting coalitions for clues as to what they can do now to garner support when the next round of elections come. 

Favorable/Unfavorable Views of Rush Limbaugh

Much discussion of the influence of Rush Limbaugh in the conservative movement and the Republican party these days. This week a Public Policy Polling (PPP) poll discussed demographic differences, and there has been some buzz about the gender gap found in that poll. But let's put this in some perspective.

Polling on Limbaugh began in March 1993, and I've now found 16 national polls asking favorable or unfavorable views of him, with PPP the latest.  Gallup and Democracy Corps have also asked this question within the last 4 months. 

Point number 1 is obvious. Rush burst onto the scene in the early 1990s, and immediate enjoyed a base of favorable supporters, amounting to 25-30% of the public. Initially there was substantial lack of awareness of him as well, as we'd expect with new media figures. Those unaware or unable to rate him were over 40% initially. But as his visibility exploded, this lack of awareness shrank sharply to about 20%, where it has remained ever since 1995.  

But growth of awareness was not accompanied by a gain in favorable audience. Instead, favorable response has remained virtually unchanged since the early 1990s. Favorability has fluctuated between 25 and 30%. What has grown is unfavorable evaluations--- from 30% before awareness expanded to a stable 50% unfavorable rating that has also been largely stable since 1995. 

So the first point of perspective is that Rush's base has remained stable and consistent from the beginning. He has not gained any share of favorable evaluations since 1993. That strong base of listeners has been hugely valuable to him but his influence does not rest on gaining widespread admirers. Rather it rests on devoted listeners who have stayed with him for 15+ years, which is a remarkable achievement.  It is this ability to speak to the base (CPAC this weekend for example, as I'm hearing him say as I write this). And I assume that's how he would like it. Broad inclusiveness is not the strategy.

Second point of perspective: The PPP survey is way out of line with 16 years of polling on this. PPP finds only 10% unable to rate Rush, while other polls remain around 20% unaware. Gallup in January found 11% not heard plus 16% unable to rate. Democracy Corps in November found 13% not heard and another 13% neutral.  So PPP seems to have tapped a population more willing to rate or more aware than other national polling. This is an IVR poll with the very low response rate such polls generally have. 

As a result, while discussion of the demographic breaks in the PPP survey are interesting, the levels of favorability reported among both men and women is substantially greater than  any other poll taken over the past 16 years. PPP found 46% favorable. The highest since 2000 is 34% by Gallup in 2003, and the three polls since then have been 26%, 23% and 28%. 

Bottom line, PPP has substantially overstated the level of favorability Rush enjoys among the public. His popularity and influence don't rest on the size of his supporters as a percent of the population but rather their loyalty and their share of radio listeners, a different population than adults nationally. 

A Little Perspective on Early Approval

There have been a number of selective comparisons of President Obama's approval ratings one month into his presidency. Some stress change since earlier readings, some focus on comparison to a previous president, though seldom more than one. And some find stable high approval while others note a first reading below 60 (in their poll, not in all polling.) So let's bring some perspective to this by simply looking at all the data.

The chart above shows all newly elected presidents since 1952. The historical data are from the Gallup Poll, the only organization with such a long term history. And for Obama we have all the polls conducted so far by anyone, a total of 80 counting daily trackers by Gallup and Rasmussen. 

The previous presidents didn't "enjoy" such intensive polling, so one challenge is choosing which early polls to use in a comparison. While each previous president got at least one poll in their first thirty days, others waited until nearly 60 days for the second. This makes comparison to previous presidents "at the 30 day mark" a bit of an approximation.

President Obama has the opposite problem. With 80 polls so far the issue is which to choose. ABC/WP at 68%? CNN at 67%? CBS/NYT at 63% USAToday/Gallup at 62%, Gallup Daily at 59% ("first time below 60"), Rasmussen Daily at 60%, Greenberg/Democracy Corps at 58%? 

How about changes? CNN is a whopping 9 point drop, from 76 (an outlier) to 67. But CBS/NYT finds a 1 point rise from 62 to 63. USAToday/Gallup down 1 point, 63 to 62. Gallup Daily down 9 from the first post-inaugural 68 to 59. Rasmussen down just 1 point, 61 to 60 from first post-inaugural to now (but their overall trend is down). 

So let's try a little perspective. 

First, across all 80 polls, there has been a small downward trend in approval of Obama, from 63.7% to 60.9% based on my local trend estimate (the solid dark blue line in the figure.) 

There has also been a lot of scatter in the polling. The open blue circles in the figure show all 80 polls. We've seen lows of 52 and 54 and a high of 76, but the substantial majority of approval readings are between 60% and 67%.

That range of readings puts Obama not at the high end nor at the low end of post-war presidents.  Kennedy, Eisenhower and Carter were consistently higher early on. Nixon started slightly lower, but moved up into Obama's range. Both Bushes started a little lower but then moved up to 60% or better briefly then back to the 50s. Reagan and Clinton were consistently lower and didn't challenge the 60% range for some while. So the clear evidence is that Obama enjoys good but not extraordinary approval ratings for the first month of his presidency. 

There is some interesting variation across the polls. The chart below shows that the Gallup and Rasmussen daily tracking polls are both trending down on parallel trajectories. But the non-tracking polls, what we used to think of as the "normal" national polls, are flat or even a bit up (the early outliers pull the line down at the beginning, without them it is basically flat). Put it all together, and you get the blue line trend, with that modest 63.7% to 60.9% decline.
It is an interesting puzzle why the two daily trackers see a downward trend while the other national polls don't. Part of the answer is the role of house effects and the small number of non-daily polls. That the most recent polls are towards the middle or upper range of all polls makes the black "other polls" trend run flat and high. The advantage of the daily polls is they use the same methodology every day, so their trends can't be flukes of mixing different methods. But between the two daily polls there is a house difference of about 4 percentage points, so clearly there are methods artifacts here as well.  The "all polls" trend estimate splits these differences pretty reasonably, but the variation you see here means there are lots of opportunities to cherry pick a poll you like, whichever side you like. 

Finally, let's remember all this doesn't mean much of anything for the long-run success of a presidency. President Clinton started low but ended with the best final approval poll of all these post-war presidents. President Carter started high but ended low. The chart below tells the tale:
Statistically, there is no correlation between standing at the second poll (roughly 30 days) and final approval poll. It doesn't matter if we use any of the polls taken in the first 90 days. There is simply no relationship between early performance and the public's judgment once the presidency is at its end.  So let's be a bit slow to place too much emphasis on these early polls. At this point, we are 2% of the way through President Obama's term. Plenty of time left to learn if he succeeds or fails.

Dispatches: Are Media Polls Criminally Bad?

This post is part of Pollster.com's week-long series on Stan Greenberg's new book, Dispatches from the War Room.


Stan Greenberg's book is a "good read" from both a political and a polling perspective and I know fellow junkies will find it rewarding. But I want to start a conversation over a strong claim Greenberg makes on page 58 about the quality of polling. I'll quote the full paragraph, but interspersing my comments as we go.

The endgame in presidential campaigns brings out all sorts of irrationalities, starting with the media polls. Many are criminally bad.

Ok, that's provocative and got my attention. How so?

Some are done in one night with no time for callbacks and thus over-represent people who are easily reached by phone, often seniors.

In principle this is a problem but in practice not so much. Of 543 national polls in Pollster.com's data for 2008, only 6 have a one day field period, with another 76 having two days of interviewing. That's 15% which isn't trivial but certainly isn't representative of most polling. The most common field period in our data was 3 days, with 230 polls, and another 115 that did 4 days on interviewing. That's 64% of polls. At the state level there is more single day polling, a lot of it done by IVR (or "Robo-polls") which are still controversial in the polling profession. Of 1791 state polls in our data, 472 were done in a single day, or 26%. Another 204 were done in two days, for a combined share of 38%. So the complaint here is fairly well justified for state polls but not so much for national polling.

Greenberg continues:

They are not carefully weighted and, as a result, show wide swings in voter preference that the media interpret wrongly as voter fickleness.

I'd be curious what constitutes careless weighting. Almost all pollsters weight the data to demographic distributions derived from the Current Population Survey (a huge monthly government survey with over 90% response rate and therefore considered particularly reliable.) Pollsters might differ on some technical issues here but it is hard to believe that media polls are that different from Greenberg's own methods. None of the weighting techniques are in any way secret-- just buy a textbook on survey sampling or read the journals or attend panels at AAPOR (the pollsters conference) and the variety of options are all right there in the public domain. So there is little reason to think that variation in weighting practice is due to either secret knowledge that Greenberg and colleagues have that is unavailable to others, or that "media pollsters" systematically choose to be reckless by using poor weighting schemes. The one controversial area of weighting is whether to weight the data to some specific party identification distribution, derived either from past polling or from exit polls (which are themselves suspect on this, but I digress.) I don't know what Greenberg's position is on this, or if that is what he is referring to in this paragraph.

I would, however, strongly agree with the implication of the end of the sentence, that media interpretations of polling variation is far too quick to present random sampling noise as "real change" and to let that noise drive the narrative. I might disagree that this is a function of weighting, but I think this point is certainty right.

Continuing in the paragraph:

And they usually ask the respondent only for whom they will vote without any prior questions that build trust. With people reluctant to tell a stranger for whom they will vote without being warmed up, many of the media polls report an inflated number of undecided voters.

I think a review of the major network and newspaper polls will show that the vote question appears early in the survey, but very rarely as the first question and never the only question. This does however raise a valuable point of legitimate disagreement among pollsters: Is it better to remind a voter of a variety of issues and considerations about the candidates through your survey before asking the vote question, or is it better to get the vote response early before you have "primed" the respondent to be thinking of particular issues. For example, at Wisconsin an ongoing survey in the 1990s asked about the state of the economy as the warmup items, then asked presidential approval. We switched that to ask initial questions about interest in politics and attention to news as the warmups to avoid priming the respondents to think of the economy before they thought about presidential approval. Now some, perhaps Greenberg, would argue that asking a series of questions about current politics before the approval question (or vote in an election survey) is actually a good thing because those are the considerations voters are likely to carry with them to the polls. Others would point to evidence that the questions you prime will have greater influence over approval or vote response than questions that might matter to voters but which you happen not to bring up. I'd be interested in hearing more from Greenberg about his view of this debate.

Finally the paragraph concludes:

Worst of all, a poll that shows a result sharply different from all the others gets media attention because the difference is "news" when it is likely the result of normal sampling fluctuations or careless polling practices.

Amen! Our efforts to detect polling outliers and label them as such is an attempt to reduce the attention paid to these fluctuations. But outliers happen to every polling organization. If they don't then the pollsters are fiddling with the data in suspect ways, since sampling theory says you will produce a statistical outlier 5% of the time. If you don't then you are cheating, not being "better". The problem for journalists and public interpreters of polls is not to get rid of outliers or suppress their publication, but to recognize them as such and give an appropriate interpretation.

My bottom line is that polling techniques and methodology are "open source." Take classes in grad school and you can learn all the theory. Work in a polling firm and you'll also learn a lot of practical wisdom. Survey professionals all have access to this. Greenberg may well think that his polls are superior to those conducted by others, but I'd disagree that this has anything to do with secret knowledge or methods. I'd also question the implicit claim that campaign pollsters are better at conducting polls than are media pollsters. I am more sympathetic to a claim that campaign pollsters or advisors on policy as Greenberg's book illustrates, may be able to craft surveys that get at the ability of politicians to shape public response through how they talk about issues. Media (and academic) polls are seldom focused on this, and in that way are arguably more limited. That is a very interesting discussion that perhaps we can move on to.

The Auto Bailout and Partisanship

As Mark wrote on Friday, there is a lot of variation in question wording and in support for a loan/rescue/bailout of the automobile industry. See his discussion for much of interest. Here I just want to add a couple of points.

First, opposition is consistently higher than support, as the trend estimates above show. For all the variation in wording and results, the basic conclusion is pretty strong: the public has not been convinced to support aid to the auto industry.  That makes Republican opposition in the Senate an easy political decision. President Bush's support for the limited aid proposal cuts no mustard with the public nor with his party in the Senate which is hardly surprising given his approval rating.

Second, the partisan division over an aid package favors the Republicans at this point, despite President-elect Obama's guarded support for the aid proposal.

While Democrats show a modest balance of support for the bailout, Independents side with Republicans in their opposition. Here I average over Marist, CBS and ABC polls taken in the last 10 days or so, the polls for which I could find partisan breakdowns. 

The fundamental point is that the public relies on political leadership on complex issues such as this. Few of us have the economics background to form an independent opinion on what should be done. So we look to the President or to the President-Elect or to Congressional leaders or other sources when these fail.  Given the weakness of President Bush, and the modest engagement of President-Elect Obama, leadership has fallen to members of Congress. There the Republican and Democratic split has reinforced public opinion divisions. But the key is that Independents are more inclined to follow Republicans, at least in the absence of a more forceful (or persuasive) Democratic message.

While it looks like the Bush administration may yet offer a "bridge loan" to the industry, this is a policy area that will confront the Obama administration early. At the moment, the pro-bailout position is a minority view. Either the new president will have to persuade those Independents to change their minds, or he will find he faces an early challenge to his ability to govern from a majority position. Little will erode his strong current public support faster than pushing for policies that face majority opposition. In this case, his tepid support for the auto-industry now may have missed the opportunity to convert Independents before their opposition has hardened. Or perhaps GM and Chrysler will file for Chapter 11 and spare him the task.

Group Support and Group Shifts

ShiftChart.pngHere is another way to consider the structure and change in group support for candidates in 2008.   The horizontal axis shows the group's vote margin in 2004 (Kerry minus Bush). The vertical axis shows how much the margin CHANGED between 2004 and 2008. The general upward movement of almost all points shows how widespread Democratic gains were, with only three groups moving down (below zero on the vertical axis.) 

The colors show 2008 winners. Groups to the left of the vertical reference line at zero that are blue switched winner from 2004 to 2008. All red groups also voted Republican in 2004, but most reduced the Republican margin.

Biggest Dem gains: Income over $200k, first time voters, Latinos, high school education only and age 18-29.  The only three decreasing Dem groups were small towns and late deciders (though the late deciders still gave a majority to Obama.) And gays dropped their Dem support the most, by over 10 points though still favored Obama by a substantial margin. Fair warning that with only 4% of the national exit poll identifying themselves as gay/lesbian/bisexual/transgendered the sampling error here is pretty large.

While the labels are hard to read, a careful look replays some study. Click on the graph for a large size version.

White Vote for Obama in the States, Part 2

An update and extension on my earlier post about the white vote for Obama. Thanks for a number of helpful thoughts in the comments on the earlier post.

First, the plot above shows how Kerry did among whites in comparison to Obama's performance. Normally when a party improves from one election to another, it does so across most demographic groups. This holds true for Obama vs Kerry in general and among whites in particular, as I showed in this post on Demographic Groups and Votes. But what about in the states?

For states below 25% African American, the trend line for Obama is above that for Kerry, indicating a general improvement among whites. (Note this is the TREND, individual states may differ-- see below.) But in the deepest of Southern states, which are also the states with the highest African American percentages, Obama falls below the Kerry vote. Now this is based on just four states, GA, AL, MS and LA, but those are also the states in which Obama had his worst performance with white voters.

So in terms of the overall trend, Obama generally improved among whites, but the shift in trend towards the right of the chart is significant.

What about the shifts by individual states, rather than the overall trend? See below:
Three of the four deep south states dropped clearly below their 2004 white support for Kerry. Georgia did not, matching it's 23% white support for the Democrat in both years. Mississippi, the lowest state in 2004, shifted from 14% to 11%, while my home state of Alabama dropped from 19% to 10%, claiming the prize for lowest white support for Obama of any state in the Union. Louisiana went from 24% to 14%, the largest point drop of all. 

One other southern state registered a notable drop, Arkansas fell from 36% white support for Kerry to 30% for Obama.

Other states that declined in white support did so by small amounts and for obvious political reasons: Alaska, Arizona and ... Massachusetts. 

Two other non-southern states showed small declines: New Mexico (43% down to 42%) and West Virginia (42% down to 41%).  All these last five are inside the confidence interval for no change.

There were a number of states with considerable increases (labeled in the chart for a five point or greater gain.) The most interesting are North Carolina (up from 27% to 35%) and Virginia (up from 32% to 39%.)  Clearly Obama could not have won those states on the white vote alone, but those shifts amount to roughly a 5-6 point boost in statewide vote share, certainly enough to matter. 

Also interesting are traditional red states Indiana and Kansas, with gains from 34% to 45% and from 34% to 40% respectively. Also Montana and North Dakota are notable, with gains from 39% to 45% and from 35% to 42%. While the Democrat didn't win three of these four states, these shifts demonstrate that they are no longer as out of reach for Dems as recent past elections might have suggested.

But to also put this in perspective, most of the states shifted up by what we'd expect when a party goes from losing to winning. That means these gains are by no means now part of the "base" Democratic vote. Rather they show that most of the country found whites shifting to Obama much as they would for other Democratic candidates in a good year for Democrats. (We await further analysis to decide if the shift was as much, more or less than one might have expected with a white candidate.)  So it is now up to the coming Obama administration to do well and solidify this support, or to do poorly and lose it to an advantaged Republican candidate in 2012. The next four years will determine that legacy of the 2008 election, not what happened on November 4th alone.