Saturday, October 11, 2014

“The Monty Hall Problem”: My/Our Persistent Statistical Error

I havent posted here in a while because Ive been busy with school, but I decided to upload a paper I wrote for Intro to Statistics. Even if youve never taken statistics, you might enjoy this. 

But first, go to https://www.youtube.com/watch?v=mhlc7peGlGg and watch the short video. Then go to http://betterexplained.com/articles/understanding-the-monty-hall-problem/ and play the game itself.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~




                                               “The Monty Hall Problem”:
                                         My/Our Persistent Statistical Error

                                                           Karen Leh

                                                             10-9-14




Trying to understand “The Monty Hall Problem” presented quite a challenge, but evidently, I have good company (e.g., most of the human race). When I watched this video, I chose—incorrectly—not to switch doors, figuring that my chances were fifty-fifty. Even the explanation that was given did not fully convince me. My mind kept saying, “But, but.…”
 

What really helped was visiting a site that simulates the game show’s choices and keeps track of how well the player does. With this simulation, I did fifty trials where I opened the same door I had picked initially, then fifty trials where I switched doors. Here is how I fared: when I did not switch, I lost 72% of the time and won 28% of the time; when I switched doors, I won 50% of the time and lost the remaining 50%. I did a repeat trial of the door-switching to see if I’d do better or worse and came out farther ahead with the wins (56% in my favor, 44% losses). This activity, more than any fancy explanation, helped me to accept the realities of the game.
 

On the video, the narrator explains the game’s rules and how, when you stay with your first choice, you are really limiting yourself to that initial 33% chance of winning. (Since there are three doors and the contestant gets to choose one, the probability of winning, at the beginning, is one-third or 33%). Altering your choice, however, means a 66% chance of success. Although you might win with your initial choice, the narrator tells us, you actually increase your odds by changing position (the graphics emphasize how this is working). A simpler way of saying this is that the door you picked first has a 33% chance, and the 66% chance is still “out there.” That is why when you switch you increase the likelihood of winning. But as the video tells us, this realization is “counterintuitive” (Clarke, 2007).
 

It took a while, but I began to understand and accept how statistics was operating within the game. The author of the simulation I played notes that sticking with your first choice occasionally wins you the car, especially in a small number of trials, which is a demonstration of The Law of Large Numbers (Sullivan, 2013). The more games you play, the more the probability settles into its usual outcome. I witnessed this law in action when I repeated the switched-doors strategy and got a number closer to the predicted 66%. I’m sure that if I repeated the same-door strategy, that too would tend toward its usual 33%-66% divide. But a contestant isn’t likely to see the long view; he/she is in the present moment and not likely to see anything beyond a possible short-run windfall.
 

The other laws I could see influencing what was happening were The Conditional Probability Rule and The Complement Rule. I got a little lost in the math, but I liked how one particular writer set this up: “…let us call the curtain picked by the contestant curtain A, the curtain opened by Monty Hall curtain B, and the third curtain, C. We will define the following events: A, B, and C are the events that the prize is behind curtains A, B, and C respectively. O is the event that Monty Hall opens curtain B. The Monty Hall Problem can be restated as follows: is Pr {A|O} = Pr {C|O}?” (Zomorodian, 1998). This person went on to discuss Bayes’s Theorem, which I didn’t completely understand, but once he got me to that last understandable equation, I realized I could make use of The Complement Rule. If Pr {B} = 0, and Pr {A} = one-third, then Pr {C} = 1- Pr {A}, which leaves two-thirds. Monty has actually assisted because he’s already opened one of the doors behind which a goat awaits. As one writer put it, “[b]ecause Monty’s choice was not random...the remaining probability of two-thirds gets squeezed, as it were, into the third box” (The Economist, 1999). Another writer explains it this way: “…when Monty opens a door, he is reducing the probability that it contains a car to zero….When Monty shows…a goat behind one of those two [remaining] doors, the two-thirds chance is only for the one unopened door because the probability must be zero for the one that the host opened” (Math Images, 2010). I found these explanations especially helpful, and I then began wondering how I’d been fooled.
 

Because I was definitely fooled. And then surprised, which was followed by disbelief and confusion. I had to read a number of articles for the facts to start making sense to me. The simulation was incredibly helpful, but it took many diagrams and explanations for the situation to make sense.
 

Part of what happens is that Monty’s opening of the one door draws attention away from what’s really happening. It’s an illusion that we have a fifty-fifty chance, and the host actually “helps” us to make that assumption (Better Explained, n.d.). He’s also limited the number of doors to a total of three; if we were offered more doors, we would probably be more likely to switch (Better Explained, n.d.). But we humans usually fall for Monty’s tricks if we’re not in on the game.
 

I read one article about an experiment where simple pigeons were actually better at adjusting their behavior in these circumstances—that put in a bird-friendly experiment with feed as a reward, they would switch doors when it made sense to do so. And when the researchers changed the rules of the game so that the pigeons were rewarded for staying with the same door, the birds modified their behavior. “Replication of the procedure with human participants showed that humans failed to adopt optimal strategies, even with extensive training” (Herbranson, 2010).
 

I find that fascinating. In a way, it’s reassuring to know I’m similar to most everyone else. But now I also know better.




 

Sources:
 

Better Explained. (n.d.). Understanding the Monty Hall problem. Retrieved from betterexplained.com/articles/understanding-the-monty-hall-problem/
 

Clarke, Ron. (Jan 21, 2007). The Monty Hall problem. Video retrieved from https://www.youtube.com/watch?v=mhlc7peGlGg
 

The Economist. (Feb. 18, 1999). Getting the goat. The Economist 350. Retrieved from www.economist.com/node/187166
 

Herbranson, Walter T., and Schroeder, Julia. (2010). Are birds smarter than mathematicians? Pigeons (Columba livia) perform optimally on a version of the Monty Hall Dilemma. Journal of Comparative Psychology 124(1): 1–13. Retrieved from www.ncbi.nlm.nih.gov/pmc/articles/PMC3086893. doi:10.1037/a0017703
 

Math Images. (2010, July 19). The Monty Hall problem. Retrieved from mathforum.org/mathimages/index.php/The_Monty_Hall_Problem
 

Sullivan, Michael, III. (2013). Statistics: Informed Decisions Using Data (4th ed.). Upper Saddle River, NJ: Pearson Education, Inc.
 

Williams, Richard (2004). Appendix d: the Monty Hall controversy. Course Notes for Sociology Graduate Statistics I. Retrieved from www3.nd.edu/~rwilliam/stats1/appendices/xappxd.pdf
 

Zomorodian, Afra. (1998, Jan. 20). The Monty Hall problem. Retrieved from http://www.cs.dartmouth.edu/~afra/goodies/monty.pdf