Can Big Data Really Predict the Future?

Octopusby Angela Guess

Carlos Castillo of Aljazeera recently wrote, “Telling good predictions apart from bad predictions is a very difficult matter. In 2010 Paul the Octopus rose to fame by predicting the outcome of many matches during the FIFA World Cup. The octopus chose the box of food with the “correct” flag with 100 percent success rate on the seven matches that Germany played, something that has a 1 in 128 chance of happening by accident. However, most of the press overlooked the fact that Paul was not the only non-human animal making ‘prediction’ at the time in Germany, with at least a porcupine, a pigmy hippopotamus and a tamarin monkey playing the same game with less success. This is a typical pattern noted by epistemologist Nassim N. Taleb. If you start with a large group making predictions at random, for instance about the stock market, after a while most will have a mix of successes and mistakes, but inevitably, a few will have a very good record. This set of privileged people will exist independently of whether they are truly making an informed prediction, or just guessing.”

Castillo continues, “To put it succinctly: the success at predicting past events is evidence that you are not bad at making predictions, but does not prove that you are good at it. However difficult the prediction business can be, we actually use statistical truths in our daily lives. Take for instance the ‘rare side effects’ usually listed in the prospects of medicines. Aspirin may cause mild side effects such as nausea and stomach pain, but it can also cause hallucinations and seizures. Lists of side effects are almost invariably compiled not from things that may happen, but from things that actually have happened to someone during clinical trials or later on. After reading the prospect, it is the patient’s decision to think about what ‘rare’ means. Or not think about it at all.”

Read more here.

We use technologies such as cookies to understand how you use our site and to provide a better user experience. This includes personalizing content, using analytics and improving site operations. We may share your information about your use of our site with third parties in accordance with our Privacy Policy. You can change your cookie settings as described here at any time, but parts of our site may not function correctly without them. By continuing to use our site, you agree that we can save cookies on your device, unless you have disabled cookies.
I Accept