The myth of BIG data OR what makes us human
Last week I attended the international ticketing conference INTIX in Denver. It’s the biggest entertainment ticketing conference in the world. I participated in a session that was all about the use of Big Data for the purpose of better running event ticketing operations and achieving overall better business results. At some point I asked the presenter if he had done any benchmarking of his predicative analysis comparing results in the US and Canada. (One has to understand that privacy laws in Canada are much more stringent than in the US, giving a data mining company very little accuracy to work with). His answer was that there was only a marginal difference between the two countries in terms of predicative results.
Having looked at the value of BIG data for the last 10 years, I have grown very skeptical that consumers can be profiled to predict behavior. Big data works well to understand historical data. However there is no proof that big data actually works to predict the future buying behavior of a consumer. Take for example the effectiveness of online display advertising (this does NOT include search advertising). Google is reporting an average click-through-rate of 0.2% for their display network! With all the data and behavioral tracking this appears very low. Of course there is this fundamental misunderstanding that users are always looking to buy something. The fact of the matter is we don’t. In contrary the point of being bombarded with about 1,700 online ads every week has made us grow immune to any form of online ads – we just don’t see them anymore. Ask yourself how many ads do you remember over the last three days. None? Trust me you’re not alone.
Today we have access to so much data and so much cheap computing power that we should be able to predict anything that involves human behavior, if Big Data would allow us to – however the fact is we can’t. Stock markets keep on crashing, epidemics keep on forming and disappearing, city and infrastructure planning keeps on being wrong. It’s true that big data allows us to understand the past better but it does not give us anything to predict the future. Humans don’t just have physical properties, our brains don’t work like machines and we are highly influenced by our social environment. Remember the old saying that the flap of a butterfly wing in China can cause a hurricane to build in the Caribbean – and while this may not be completely true, the message is that our world is a world dominated by chaotic and therefore unpredictable reactions. Little unexpected events can have massive consequences – a terrorist attack cannot be predicted but can put the world at the brink of war and the stock markets to crash from one day to another. Falling in love with a stranger can change the path of one’s live forever. Our unpredictable nature is exactly what makes us human. Big data is not going to change that.