I think Bell Curve should stop for a second before emailing Nate Silver.
Firstly, there's a statistical principle here. Bell Curve is completely right that the likelihood of any particular outcome is tiny, and not a good way to measure whether the outcome was "likely" or whether the poll is "suspect". But the usual statistical comparison is what is the probability of getting the actual (sample) result or further from the mean. The right thing to look for, then, is the probability that McCain got 74% or more of the sample.
Saying "McCain's result was 74%, so let's look at the probability that the poll (sample) would come up with _70%_ or greater" is unfairly helping your own argument, since you're throwing in the chunk from 70% to 74%. So that part of the argument needs to be refined in any case.
Here's what I looked at. There's bound to be a few flaws in it, but I think it suggests Nate's numbers are closer to the mark.
First, there's the chi-squared goodness-of-fit test, as described here and here. The chi^2=41.04 for a 98-person sample producing an outcome of 73 McCain, 25 Obama, when the population is 41.25% McCain, 56.35% Obama. That's a big chi^2 value; according to the distribution calculator I just downloaded, that corresponds to a level of significance (for a one-sided tail) of about p=2.18*10^(-11). That's pretty close to Nate's odds (which give a probability of 1.83*10^(-11).) Since his numbers are so specific, and I haven't hesitated to round off, I'm thinking Nate has some exact binomial distribution data to get the precise value, and it looks to me like he's counting the tail, not just the chance of that precise outcome.
But far be it from me to just throw yet another approach out there and not produce some apples to compare to your apples. With a name like Bell Curve, you can't fault the guy for going to the Central Limit Theorem. :)
Okay, the binomial random variable Y is the number of young polled people who said they favored McCain, which has n=98 and we posit p=.425. So the expected value is np=41.65 and the variance is np(1-p)=23.94875.
The Central Limit Theorem says that Z=(Y-41.65)/sqrt(23.94875) is approximately N(0,1). The actual result was 73 for Y, or 6.406135493 for Z. The probability of Z being greater than or equal to 6.406135493 is 7.4627*10(-11), which again is more in the ballpark of Nate's result than 1 in 150.
Also, referring to Dr. S's approximations, the sigma for this case was sqrt(23.949)=4.89, pretty close to his 5% estimate, and the jump of 31.35% was 6.4 sigmas, close to his estimate of 6 sigmas or so. Which don't appear on his table, because 6 sigmas means really really unlikely. (Other than fudging p(1-p) in the sigma, which is a low-error fudge, Dr. S _is_ using the CLT, albeit with some roundoff.)
I don't have the facility to replicate Bell Curve's computation of the binomial distribution directly of the probability of getting exactly 73 McCain voters out of 98, so I couldn't say why that number came out at lower odds (=higher probability) than these tests indicate the probability of getting greater than or equal to 73 McCain voters out of 98 should be.
But I'm very suspicious of the 150 to 1 odds Bell Curve ends up with, for the reasons given above.
LTG, it may uninterest you to know that "six sigma" is a business management strategy, (some might say "fad"), that involves co-opting statistical methods for quality management of business processes, or something equally buzzwordy.
Firstly, there's a statistical principle here. Bell Curve is completely right that the likelihood of any particular outcome is tiny, and not a good way to measure whether the outcome was "likely" or whether the poll is "suspect". But the usual statistical comparison is what is the probability of getting the actual (sample) result or further from the mean. The right thing to look for, then, is the probability that McCain got 74% or more of the sample.
Saying "McCain's result was 74%, so let's look at the probability that the poll (sample) would come up with _70%_ or greater" is unfairly helping your own argument, since you're throwing in the chunk from 70% to 74%. So that part of the argument needs to be refined in any case.
Here's what I looked at. There's bound to be a few flaws in it, but I think it suggests Nate's numbers are closer to the mark.
First, there's the chi-squared goodness-of-fit test, as described here and here. The chi^2=41.04 for a 98-person sample producing an outcome of 73 McCain, 25 Obama, when the population is 41.25% McCain, 56.35% Obama. That's a big chi^2 value; according to the distribution calculator I just downloaded, that corresponds to a level of significance (for a one-sided tail) of about p=2.18*10^(-11). That's pretty close to Nate's odds (which give a probability of 1.83*10^(-11).) Since his numbers are so specific, and I haven't hesitated to round off, I'm thinking Nate has some exact binomial distribution data to get the precise value, and it looks to me like he's counting the tail, not just the chance of that precise outcome.
But far be it from me to just throw yet another approach out there and not produce some apples to compare to your apples. With a name like Bell Curve, you can't fault the guy for going to the Central Limit Theorem. :)
Okay, the binomial random variable Y is the number of young polled people who said they favored McCain, which has n=98 and we posit p=.425. So the expected value is np=41.65 and the variance is np(1-p)=23.94875.
The Central Limit Theorem says that Z=(Y-41.65)/sqrt(23.94875) is approximately N(0,1). The actual result was 73 for Y, or 6.406135493 for Z. The probability of Z being greater than or equal to 6.406135493 is 7.4627*10(-11), which again is more in the ballpark of Nate's result than 1 in 150.
Also, referring to Dr. S's approximations, the sigma for this case was sqrt(23.949)=4.89, pretty close to his 5% estimate, and the jump of 31.35% was 6.4 sigmas, close to his estimate of 6 sigmas or so. Which don't appear on his table, because 6 sigmas means really really unlikely. (Other than fudging p(1-p) in the sigma, which is a low-error fudge, Dr. S _is_ using the CLT, albeit with some roundoff.)
I don't have the facility to replicate Bell Curve's computation of the binomial distribution directly of the probability of getting exactly 73 McCain voters out of 98, so I couldn't say why that number came out at lower odds (=higher probability) than these tests indicate the probability of getting greater than or equal to 73 McCain voters out of 98 should be.
But I'm very suspicious of the 150 to 1 odds Bell Curve ends up with, for the reasons given above.
LTG, it may uninterest you to know that "six sigma" is a business management strategy, (some might say "fad"), that involves co-opting statistical methods for quality management of business processes, or something equally buzzwordy.
