# Wonders of Probability

Lately, I have been reading a lot about Probability, Statistics and Machine Learning. These subjects always involve a kind of awe that one can only experience on understanding the concepts. I’ve developed tremendous respect for the people who understand probability, (Bayesian) statistics, learning theory and other related subjects.

I write this post to pose some probability (pseudo) paradoxes and problems to you. Even though these paradoxes are extremely simple to understand and comprehend, but their solution is completely counter-intuitive. I’ll admit that for some of the paradoxes/problems, even after seeing the solution I wasn’t able to figure out what exactly is going on. So, without further blabber here we go with the paradoxes:

First one is called the Monty Hall Problem. Its statement goes something like this:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

Think a lot about the problem. The solution is not as simple as it seems!

While on the surface it seems that it should not matter whether you switch the door or not, switching actually turns out to be advantegous for you. In fact, with the second door you will have 66% of chance of winning while the existing door will give you 33% chance of winning. See the wikipedia article for explanation.

The second one is even more interesting. It is taken from the document titled Nuances of Probability. The paradox goes something like this:

My neighbor has two children. Assuming that the gender of a child is like a coin flip, it is most likely, a priori, that my neighbor has one boy and one girl, with probability 1/2. The other possibilities—two boys or two girls—have probabilities 1/4 and 1/4.

Suppose I ask him whether he has any boys, and he says yes. What is the probability that one child is a girl? By the above reasoning, it is twice as likely for him to have one boy and one girl than two boys, so the odds are 2:1 which means the probability is 2/3. Bayes’ rule will give the same result.

Suppose instead that I happen to see one of his children run by, and it is a boy. What is the probability that the other child is a girl?

Again, you need to really understand what is being asked in the question. To give you some guidance, here is what solution looks like (don’t worry, even after seeing the solution it would be hard to believe that it is indeed the solution).

Observing the outcome of one coin has no affect on the other, so the answer should be 1/2. In fact that is what Bayes’ rule says in this case. If you don’t believe this, draw a tree describing the possible states of the world and the possible observations, along with the probabilities of each leaf. Condition on the event observed by setting all contradictory leaf probabilities to zero and renormalizing the nonzero leaves. The two cases have two different trees and thus two different answers.

So, it is an apparent paradox! And I don’t know how it is resolved. If you get to understand this, let me know in the comments.

Another simple problem relating to Bayes’ theorem which people usually get wrong is as follows:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

What do you think the answer is? Don’t worry if you get the answer wrong. You know what, even after considering how critical will a misdiagnosis of breast cancer turn out to be, only 15% doctors get it right. That is indeed scary! This also hints at the importance of understanding and appreciating the depth of probability.

If you apply Bayes’ theorem carefully, the answer will turn out to be 7.8%. And just for your information, most doctors estimate the chance of having cancer to be between 70% and 80%.

The above were some of the interesting paradoxes cropping up due to interpretation of probability and inability of people to apply Bayes’ theorem mentally. If you are game for even more probability paradoxes and problems, head to this wikipedia article.

## 13 comments

1. Read about gambling, poker and cards. Awesome way to learn about probability and chance.

2. Dear Paras,

your measurements contain different information:
You have seen one exactly boy;He tells you he has one or more boys.
Hence the state collapse is different.

3. Interesting post!

4. Rahul says:

Buddy why are you punishing your brain muscles so much ..at your age you should be interested in girls,fun and party and developing your body muscles and emotions more than brain muscles .
a wellwisher
Rahul

5. Ajay says:

Hi Paras,
Nice blog. Well regarding Monty hall problem there is a beautiful movie made entitled ’21’ . Do watch, lovely application of probability.

I was there with you during IAS- srf training at IISc 2006.

Keep posting wonderful articles.

6. Regarding the boys and girls problem. In the first situation, you have 3 situations possible: BB, BG, and GB. In these situations, note that there is a difference in the sibling being older and the sibling being younger, that is, there is a difference between BG and GB.

But in the second situation, the phrase “other child” makes GB = BG. Hence the difference.

7. Hi Paras,
I have visited your page after quite a while. Very interesting you write on this. Would you mind if I pick that question with the condition “I happen to see one of his children run by,” and put it on a draft that I might publish today or day-after? An equivalent problem is this: Suppose a couple have a girl child and if she has the name Florida, what is the probability that the other child is a girl? (And no it is not 1/4).

And by the way, I am happy that you experienced learning theory, only those who have can find it beautiful. And it is VERY beautiful. I would be hoping to publish the 5 pending drafts on it soon too!

– Shubhendu

8. Regarding you Girl Boy problem, there is no paradox. That elaborate description boils down to one thing – if you have two children, the probability of having at least one girl child is 50% (straight from a priori probability). The birth of the two children are mutually independent events and is actually a prime example of a case where the Bayes theorem is meaningless, because it useful in the care of dependent events.

9. In your earlier case, if your neighbour replies that he has boys, he hasn’t told you whether he has one or two boys, all he has told you is that the only possible cases now are : BB, BG and GB. You are treating the BG and GB situations as different implying an order. In the second case, there are really only two situations possible : Children of Same Sex SS or Children of different Sex SD and the fact that the sex you saw happened to be a boy makes no difference at all.

10. I enjoyed reading your blog. Keep it up.

11. RaiulBaztepo says:

Hello!
Very Interesting post! Thank you for such interesting resource!
PS: Sorry for my bad english, I’v just started to learn this language ๐
See you!
Your, Raiul Baztepo

12. Hi !!! ๐
My name is Piter Kokoniz. Just want to tell, that I like your blog very much!
And want to ask you: is this blog your hobby?
Sorry for my bad english:)
Thank you!
Your Piter Kokoniz, from Latvia

13. Theory of Probability I have learn in my school days thats is my favorite topic

Comments are closed.