Regarding probability study posted on TheCitizens

Yow. I ignore the comments for weeks, and three or four interesting topics come up.

A) you may be interested in the Iowa Gambling Task experiments, one observation of which is (I'm paraphrasing here, so read some real research if you want to see what the studies really claim) that people can develop a sense of a probability from a smallish sample, _before_ they consciously recognize that they have such a sense. (Paraphrase 2: drawing from a deck of possible rewards/penalties, someone will start reacting negatively to a "bad" deck before they consciously decide it's a "bad" deck.)

B) [not actually chronological] LTG says "I am surprised any mathematician on this blog would expect children to have an intuitive grasp of probability. Adults sure don't...It is well known...that individuals tend...to treat a 99,999 to 1 chance...differently from...a 1 in 100,000 chance..."

This, and I really do not mean to offend, is because you are not a mathematician, particularly not a mathematician who has to teach non-mathematicians.

Mathematicians (especially in education) think about what is and isn't intuitive mathematics a great deal, because what _seems_ intuitive to us is clearly not to our students. And one big idea that's come out of this is the impetus to connect the language and structure of mathematics to the structures and intuition that "normal" people already have, rather than treat math as abstract and foreign.

So we talk about "number sense" and "proportional reasoning" and "intuitive probability" and part of what we mean is a non-linguistic concept of those things. The theory is that people have a sense of probability, both in the cases of extremely rare events and relatively common events, but:
a) your intuition can be misled, and a high-level, abstract argument lacks potency (e.g., because you've heard about miraculous recoveries from coma, your intuition is that is a reasonably likely event, and a doctor telling you it's a one in a million chance doesn't dissuade you.)
b) it is difficult to connect your intuition, in the many cases when it is right, to the abstract claims of probability, if they're couched in examples not relevant to your experience or in language you don't have other intuitive connections with (most people troubled by the Monty Hall problem didn't watch episode after episode of Let's Make a Deal; most people have never thought of what "1 in 100,000" means, in any concrete way, and usually don't have a reason to.)

Thus, while mathematicians cannot help but see that most adults' sense of probability is _solely_ intuitive, and unconnected to any attempts at probabilistic abstraction or language, and (therefore?) usually lacks _nuance_, we cling to the belief (well justified, really) that there is in fact an intuition there.

(I should point out that numbers are themselves language, as I'm using the phrase. I doubt that caused any confusion, but you can never be too careful on the interweb.)

C) I took Pombat's examples to just be illustrations of the way probabilities are bandied about in misleading ways, and not a claim about DNA evidence per se. For more examples of this and other grievous misquantification, John Allen Paulos' Innumeracy (and/or A Mathematician Reads the Newspaper) are excellent reading.

D) RbR's argument (and I know I shouldn't say this, but I just can't help it, so please forgive me) assumes facts not in evidence. A claim about the sensitivity of a forensic test is just that, independent of other factors. If a jury is deciding guilt or innocence, they are (or should be) weighing all the factors RbR suggests (which surely the prosecution would present) in their "reasonable doubt" analysis. But those other factors don't affect the accuracy of the test.

RbR's description also bears some resemblance to an example in Innumeracy, about a case in Los Angeles in 1964. (I'm not saying RbR is guilty of the probabilistic fallacy in that case, just that his description is similar to the set-up for the fallacy, and the story is a fun one, so I'm going to relate it.) Brutally summarized, the argument was: a pair of lawbreakers were seen with several "distinguishing characteristics" (blonde with pony tail with black man with beard in a yellow car). The defendants matched those characteristics. In this case, there was nothing else linking the defendants to the crime, but surely the chances that a couple would match all those characteristics were so small that this must be the couple in question, right?

So it was decided. Until the appeal, in which the defense pointed out that the relevant probability was not whether a couple would have those characteristics, but rather whether, out of the ~2M couples in LA, there was _another_ couple with those characteristics. (The probability that there was one such couple was actually a certainty, since the defendants were living proof.)

It turns out (and no one would suggest that this is intuitive) the likelihood of another such couple (given the prosecution's estimate of the relevant probabilities) is about 8%, which provided reasonable enough doubt for the California supreme court
to overturn the original verdict.

E) If anyone wants to talk more about math, probability, and math education, I'm very keen on it -- you could say it's my job. Email me!

uncertainty in voting results

In a comment to this post from the Citizens about the Minnesota senate race:

I'm not a statistician, but "margin of error" refers to the likelihood of a survey not accurately reflecting the population the survey is taken from.

Since the votes are what decide the election (we don't care what the population who doesn't vote thinks), the vote is a census, not a survey, and doesn't have any margin of error in the statistical sense.

But, as Florida 2000 demonstrated, you are right in thinking there will be "a different tally every time they are tallied."  This isn't a mathematical issue, but a practical one.  Out of three million ballots, some are going to be weird, subject to interpretation, of imperfect provenance, etc.

I _think_ the error rate associated with this sort of thing varies a lot based on the exact situation.  Different voting machine types (and different voting machines) will produce ambiguous results at different rates; different jurisdictions will have different judges who consider different proportions of provisional ballots to fall into the category of "subject to legal interpretation."

If, after the error-checking of the recount, the difference is still only a couple hundred, the chances are good that there's enough "fuzziness" in some of the votes that the result will come down to luck -- who has the higher tally when the courts (or the election laws) say "enough".  But not necessarily -- it's possible that even a very small margin is demonstrably genuine in the eyes of the law, if the voting machines are so good they don't produce many ambiguous results.  (This is the point of electronic voting machines, although optical scanning offers a clearer paper trail to check results against.)

I say "in the eyes of the law" because there's no way to determine beyond the shadow of <i>any</i> doubt that every voter's intent has really been accurately captured.  This boils down to the impossibility of absolute certainty -- at some point we draw the line and say "yes, it's possible that this electronic voting machine switched the vote of every 10,000th voter and we didn't happen to catch it with comparison to exit poll data, but that possibility isn't worth pursuing."

reply to Citizens' Bell Curve commenting on 538

I think Bell Curve should stop for a second before emailing Nate Silver.

Firstly, there's a statistical principle here. Bell Curve is completely right that the likelihood of any particular outcome is tiny, and not a good way to measure whether the outcome was "likely" or whether the poll is "suspect". But the usual statistical comparison is what is the probability of getting the actual (sample) result or further from the mean. The right thing to look for, then, is the probability that McCain got 74% or more of the sample.

Saying "McCain's result was 74%, so let's look at the probability that the poll (sample) would come up with _70%_ or greater" is unfairly helping your own argument, since you're throwing in the chunk from 70% to 74%. So that part of the argument needs to be refined in any case.

Here's what I looked at. There's bound to be a few flaws in it, but I think it suggests Nate's numbers are closer to the mark.

First, there's the chi-squared goodness-of-fit test, as described here and here. The chi^2=41.04 for a 98-person sample producing an outcome of 73 McCain, 25 Obama, when the population is 41.25% McCain, 56.35% Obama. That's a big chi^2 value; according to the distribution calculator I just downloaded, that corresponds to a level of significance (for a one-sided tail) of about p=2.18*10^(-11). That's pretty close to Nate's odds (which give a probability of 1.83*10^(-11).) Since his numbers are so specific, and I haven't hesitated to round off, I'm thinking Nate has some exact binomial distribution data to get the precise value, and it looks to me like he's counting the tail, not just the chance of that precise outcome.

But far be it from me to just throw yet another approach out there and not produce some apples to compare to your apples. With a name like Bell Curve, you can't fault the guy for going to the Central Limit Theorem. :)

Okay, the binomial random variable Y is the number of young polled people who said they favored McCain, which has n=98 and we posit p=.425. So the expected value is np=41.65 and the variance is np(1-p)=23.94875.

The Central Limit Theorem says that Z=(Y-41.65)/sqrt(23.94875) is approximately N(0,1). The actual result was 73 for Y, or 6.406135493 for Z. The probability of Z being greater than or equal to 6.406135493 is 7.4627*10(-11), which again is more in the ballpark of Nate's result than 1 in 150.

Also, referring to Dr. S's approximations, the sigma for this case was sqrt(23.949)=4.89, pretty close to his 5% estimate, and the jump of 31.35% was 6.4 sigmas, close to his estimate of 6 sigmas or so. Which don't appear on his table, because 6 sigmas means really really unlikely. (Other than fudging p(1-p) in the sigma, which is a low-error fudge, Dr. S _is_ using the CLT, albeit with some roundoff.)

I don't have the facility to replicate Bell Curve's computation of the binomial distribution directly of the probability of getting exactly 73 McCain voters out of 98, so I couldn't say why that number came out at lower odds (=higher probability) than these tests indicate the probability of getting greater than or equal to 73 McCain voters out of 98 should be.

But I'm very suspicious of the 150 to 1 odds Bell Curve ends up with, for the reasons given above.

LTG, it may uninterest you to know that "six sigma" is a business management strategy, (some might say "fad"), that involves co-opting statistical methods for quality management of business processes, or something equally buzzwordy.