Numerical oddities in polling

A little knowledge of statistics can lead to some pretty exaggerated claims. It is hardly surprising that Nate Silver is involved. In any case, these odds have to be adjusted for the number of pollsters. I don't know how many there are, but let me give you a trivial example. Suppose that there were 5,000 pollsters (obviously not true, but I just want a simple example), what are the odds that at least one pollster would have this disparity in the odds of the second digit coming up 8 versus 1. Assuming that each pollster is making an independent estimate (which isn't really close to being true), you would be certain to find one pollster with that pattern of digits. If 50 pollsters, there would be a one in 100 chance. You can see a couple of minor adjustments have changed the odds from 1 in 83 million to 1 and 100.

Nate Silver, a political-numbers analyst who blogs at www.fivethirtyeight.com, then dived in. After gathering thousands of numbers from published Strategic Vision polls, he tallied how often percentage results ended with particular digits. For example, if a Strategic Vision poll question during last year's presidential election showed Barack Obama leading John McCain by 48% to 41%, Mr. Silver recorded the second digit of each percentage, in this case 8 and 1. Similarly, if a question about a Senate contest showed one candidate leading another 57% to 36%, the digits 7 and 6 were recorded.

Mr. Silver says he found that 8 showed up as a second digit 57% more often than 1 in Strategic Vision polls, instead of a following a more natural random pattern. The odds of that kind of discrepancy happening by chance alone, Mr. Silver wrote, was "millions to one against," a figure he later refined to 83 million to 1 when comparing the results with those of another pollster, Quinnipiac University Polling Institute, which had a more uniform distribution of digits.

This week, Mr. Silver brought in a physicist and commenter on his blog to calculate the probability, which shrank to 5,000 to 1 against, when removing what he said was an unproven assumption that each digit should appear equally often. Several mathematicians said the shift in odds doesn't diminish Mr. Silver's finding that the Strategic Vision numbers were unlikely to arise by a quirk of fate. . . .