Imagine a million monkeys, continuously typing away at keyboards. Assuming each key is equally likely to be pressed and the typing is purely random, it is a mathematical certainty that eventually one of them will produce an exact replica of some sensible text (say Daffodils by Wordsworth). Most of you may not agree (but it's true nonetheless), and even those that do, might think this point to be of little or no interest.
However, just assume for the moment that it did happen, and I brought that one in a million monkey to you (the one who wrote Daffodils). Would you bet your savings that this monkey would produce another work of Wordsworth next?? To those who answered no, what if I showed you a video recording of the monkey's typing and explained how his/her typing although seemingly random was actually logical and was bound to produce a result? I might even come up with some statistical evidence of the typing, in some fixed time interval, being strongly correlated with the letter frequency in English texts. Would that convince you? I think not.
Yet, when we read about one mutual fund amongst a sea of funds, doing well in a certain year, we feel compelled to conclude that it might be a good investment. The same fund house might have four different funds, most likely in mutually exclusive financial instruments, and possibly mutually exhaustive too!. There is a good chance that one of them will do well in a given period!! Financial markets are a typical example but the cases are numerous.
We all too often conclude that "past performance" is indicative of "future performance". In many cases it might be, but it depends on how susceptible to randomness the task is. In a million coin tosses, we may get a string of ten consecutive heads. Does that mean those ten throws were of superior skill than the others?? In the same way, although it may seem hard to believe, a lot of "successes" that we observe in the real world may also simply be the result of chance. We forget that we are often looking at that ONE monkey and not able to see the several others that have failed. We don't see the failures because they have been wiped out and are not visible to us. This is what is called the survivorship bias.
This should not be confused with an escapist view that we can do anything we want and the results are all random. Rather, it is to become aware that randomness MAY be playing a major role in the outcome and accepting it. This allows us to make more optimal decisions for the long run and not be fooled by the randomness.
I'm not sure if I have been able to drive home the point as effectively as I wanted to. But I found it to be a remarkable insight into how our thinking can sometimes be flawed and lead us to make decisions that are unlikely to produce the results we expected. Rather than treat this as some holy grail (which it is not), I just try to be a bit more skeptical when I find myself trying to "predict" the future based on past events. For further insights, I highly recommend Fooled by Randomness written by Nassim Nicholas Taleb.
However, just assume for the moment that it did happen, and I brought that one in a million monkey to you (the one who wrote Daffodils). Would you bet your savings that this monkey would produce another work of Wordsworth next?? To those who answered no, what if I showed you a video recording of the monkey's typing and explained how his/her typing although seemingly random was actually logical and was bound to produce a result? I might even come up with some statistical evidence of the typing, in some fixed time interval, being strongly correlated with the letter frequency in English texts. Would that convince you? I think not.
Yet, when we read about one mutual fund amongst a sea of funds, doing well in a certain year, we feel compelled to conclude that it might be a good investment. The same fund house might have four different funds, most likely in mutually exclusive financial instruments, and possibly mutually exhaustive too!. There is a good chance that one of them will do well in a given period!! Financial markets are a typical example but the cases are numerous.
We all too often conclude that "past performance" is indicative of "future performance". In many cases it might be, but it depends on how susceptible to randomness the task is. In a million coin tosses, we may get a string of ten consecutive heads. Does that mean those ten throws were of superior skill than the others?? In the same way, although it may seem hard to believe, a lot of "successes" that we observe in the real world may also simply be the result of chance. We forget that we are often looking at that ONE monkey and not able to see the several others that have failed. We don't see the failures because they have been wiped out and are not visible to us. This is what is called the survivorship bias.
This should not be confused with an escapist view that we can do anything we want and the results are all random. Rather, it is to become aware that randomness MAY be playing a major role in the outcome and accepting it. This allows us to make more optimal decisions for the long run and not be fooled by the randomness.
I'm not sure if I have been able to drive home the point as effectively as I wanted to. But I found it to be a remarkable insight into how our thinking can sometimes be flawed and lead us to make decisions that are unlikely to produce the results we expected. Rather than treat this as some holy grail (which it is not), I just try to be a bit more skeptical when I find myself trying to "predict" the future based on past events. For further insights, I highly recommend Fooled by Randomness written by Nassim Nicholas Taleb.
I almost feel like writing a parallel post but lets try and keep this confined to a comment. You make a fair point...In fact one that is not emphasized enough....You should further read:
ReplyDeletehttp://io9.com/how-bayes-rule-can-make-you-a-better-thinker-471233405
(Prob not the best article but I didn't wanna spend more than a few mins finding one.... I'm sure there are better articles and even more confident you'll find them)
Also as an aside you might find this podcast interesting:
http://www.radiolab.org/story/91684-stochasticity/
But I would be remiss if I did not demur with your comment on mutual funds. Right off, one can make the case that two mutual funds from the same house are not independent. And since this a testable hypothesis, I'd much rather do this than talk about all the different ways they are not:
http://www.epibiostat.ucsf.edu/biostat/sen/statgen/permutation.html
Hmmm...come to think of it, I might just do this as a project for my stats class!! Will prob need to expand the problem though, just the permutation test wont quite cross the threshold for a project.
As as aside, the prob of a run of k heads is 0.5^k. So 10 heads is fairly probable in a few thousands tosses. In fact the prob of not getting 10 heads in a million tosses is going to be abysmally low. But I don't think that is a trivial calculation. Let me think.....