Correlation Is Not Causation – But It Can Really Help With Prediction

People use all sort of predictive technologies. Probably the most popular is meteorology. You want to have an idea about the weather, just so you know how to dress, or if an umbrella is going to be useful tomorrow. There are many other, like who is going to win the election based on polls, or how much you’ll be worth if you save every month 10% of your income.

All these predictive technologies have something in common: they are all mathematical functions. Precisely, they are all trying to minimize the error of a cost function. That’s all there is to them. They are really spectacular, as in they are surprisingly accurate, but they are all very similar: if you input a lot of parameters to such a function, it will eventually find the most plausible outcome, or the lower error – that’s your prediction. The more parameters you have, or the more data you can feed into it, the better your cost function will be.

The really interesting stuff is that the function has no idea about the meaning of the parameters. It just creates some correlations. It works pretty much the same with stock markets, as it works with weather. They are all churning a set of parameters, and the way they churn it makes them generate accurate predictions.

They have no idea about what the parameter means, they are munching numbers.

Astrology And Artificial Intelligence

Yeah, I know you didn’t see that coming. But here we are. What started as a post about predictive technologies, suddenly switched to astrology.

I have been using astrology for more than 15 years now, as a general context descriptor, just like meteorology is a context descriptor for weather. In my personal experience, astrology “worked” with more than 70% accuracy during this period. Which is more than flipping a coin.

Please note that I’m talking about astrology, as a discipline, with a very thorough study routine, not about zodiac, horoscopes or “when are you going to win the lottery” predictions. There’s a very different thing available to masses, which is referred to as being astrology, but in 99% of the cases it doesn’t have anything to do with the discipline of astrology, it’s just random wording. The discipline of astrology is way humbler and more modest than this type of “fast food divination”, as I call it.

Now, if you really think about it, astrology is very similar with AI. It’s a list of parameters (namely, the position of stars and planets) and some specific events that occurred. For instance, when Jupiter is in a specific constellation, events tend to be more spectacular, and reality is perceived as more optimistic. In “fast food divination” this is often translated as “luck”. On the other side, if Saturn is in a certain constellation, events seem a bit more restrictive and reality is perceived as pessimistic. In “fast food divination” this is often translated as “bad luck”. If Mercury is retrograde, processing of events feels slower. If Venus is Libra, we connect easier. If Mars is in Aries, we work better, but we also argue more, or even fight with each other.

Add to the list of parameters the distance between the planets in radians (the “aspects” that they make to each other) and you got yourself a pretty hefty data chunk.

People observed and noted the correlation between the position of planets in constellations, and various events, for thousands of years. In time, they corrected their predictions when new data was available. Just as a neural network minimizes the cost function, and does some back propagation, and trains a model, humans did the same, only during a few hundreds generations.

What you can achieve now by training a model in a week, astrology did by recording events and correlations during a few thousands years.

Of course, you may choose “not to believe in it”. That’s equivalent with not believing in what Facebook, or any other social media is displaying to you, because both astrology and the AI used by Facebook are the same thing. None of them attaches any meaning to the parameters they use to minimize their cost functions, but both are generating relevant results. Just because you’re not believing in what Facebook is adding to your feed, it doesn’t make it less real.

Correlation is definitely not causation. But if you start thinking in scenarios, not certainties, you’ll soon realize that correlation helps a lot with prediction.

It may not work 100% all the time, but even 70% accuracy is better than flipping a coin.




Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.