Wonders of Probability

Lately, I have been reading a lot about Probability, Statistics and Machine Learning. These subjects always involve a kind of awe that one can only experience on understanding the concepts. I’ve developed tremendous respect for the people who understand probability, (Bayesian) statistics, learning theory and other related subjects.

I write this post to pose some probability (pseudo) paradoxes and problems to you. Even though these paradoxes are extremely simple to understand and comprehend, but their solution is completely counter-intuitive. I’ll admit that for some of the paradoxes/problems, even after seeing the solution I wasn’t able to figure out what exactly is going on. So, without further blabber here we go with the paradoxes:

First one is called the Monty Hall Problem. Its statement goes something like this:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

Think a lot about the problem. The solution is not as simple as it seems!

While on the surface it seems that it should not matter whether you switch the door or not, switching actually turns out to be advantegous for you. In fact, with the second door you will have 66% of chance of winning while the existing door will give you 33% chance of winning. See the wikipedia article for explanation.

The second one is even more interesting. It is taken from the document titled Nuances of Probability. The paradox goes something like this:

My neighbor has two children. Assuming that the gender of a child is like a coin flip, it is most likely, a priori, that my neighbor has one boy and one girl, with probability 1/2. The other possibilities—two boys or two girls—have probabilities 1/4 and 1/4.

Suppose I ask him whether he has any boys, and he says yes. What is the probability that one child is a girl? By the above reasoning, it is twice as likely for him to have one boy and one girl than two boys, so the odds are 2:1 which means the probability is 2/3. Bayes’ rule will give the same result.

Suppose instead that I happen to see one of his children run by, and it is a boy. What is the probability that the other child is a girl?

Again, you need to really understand what is being asked in the question. To give you some guidance, here is what solution looks like (don’t worry, even after seeing the solution it would be hard to believe that it is indeed the solution).

Observing the outcome of one coin has no affect on the other, so the answer should be 1/2. In fact that is what Bayes’ rule says in this case. If you don’t believe this, draw a tree describing the possible states of the world and the possible observations, along with the probabilities of each leaf. Condition on the event observed by setting all contradictory leaf probabilities to zero and renormalizing the nonzero leaves. The two cases have two different trees and thus two different answers.

So, it is an apparent paradox! And I don’t know how it is resolved. If you get to understand this, let me know in the comments.

Another simple problem relating to Bayes’ theorem which people usually get wrong is as follows:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

What do you think the answer is? Don’t worry if you get the answer wrong. You know what, even after considering how critical will a misdiagnosis of breast cancer turn out to be, only 15% doctors get it right. That is indeed scary! This also hints at the importance of understanding and appreciating the depth of probability.

If you apply Bayes’ theorem carefully, the answer will turn out to be 7.8%. And just for your information, most doctors estimate the chance of having cancer to be between 70% and 80%.

The above were some of the interesting paradoxes cropping up due to interpretation of probability and inability of people to apply Bayes’ theorem mentally. If you are game for even more probability paradoxes and problems, head to this wikipedia article.