Big data and the bottom-up malevolence

I am kind of bored, even borderline annoyed, by the constant flux of big data gibberish and clichés coming our way - allegedly, big data is the new panacea that will help us automagically solve all the problems of every single business, not to mention mankind as a whole. Allegedly.

It is true that data today is produced at an ever accelerating pace, and that these huge datasets are being handled by algorithms and systems of unprecedented sophistication. Surprisingly enough though, in most cases, big data brings very little actual business value. This is mainly due to what I call the "bottom-up malevolence".

People solve problems in two ways: top-down or bottom-up. In the former, the problem solver constructs hypotheses, collects data-points that would confirm or invalidate these hypotheses and then reforms the hypotheses according to the evidence. In bottom-up thinking though, the problem solver attempts to construct the answer based solely on analysing the detailed evidence. Unsurprisingly, the vast majority of corporate leaders fall into the ranks of top-down (or "big picture") thinkers.

I believe that extracting value from big data is so notoriously challenging due to the mode of thinking of their operators and gatekeepers: data scientists, as well as their supporting technologists, are in most cases bottom-up thinkers. Their default mode of perceiving reality is through the analysis of details and the quest for recognising patterns within these details.

However, useful as it might be, exploratory data analysis cannot really focus on business priorities or solve the problems that are truly important to the top management and the business at large.

Therefore, whenever our clients come to us to help them "tame big data", we politely re-route the discussion to "what business problem are you trying to solve?". It is frustratingly common that, at the end of the discussion, we all realise that the data required to solve the actual business problem are not "big" at all!

So, here is how can we fight the bottom-up malevolence:

• Set actionable business targets; use the right data to serve these targets.
• Collect only data that are meaningful; establishing a humongous data warehouse with all kinds of irrelevant data sets has never helped.
• The head of analytics is primarily a business role; use a senior business executive with strong acumen and, ideally, a good understanding of technology and developed numeracy.
• When it comes to hiring external help, work with firms that understand your business and have expertise in your particular industry; the myth of the "generalist consultant" that can solve all kinds of business / data problems is, in our view, just a myth.

Big data are, indeed, the future of business - we just need the right leaders to help us figure them out.

Akis Tsekouras is the managing director of Simulen, a consulting firm that helps companies around the world succeed.