When Is “More Data!” The Wrong Solution?

By on

Click to learn more about author Karl Zimmerman.

Data-driven decision-making is vitally important to modern enterprises. Without the analysis of data, we can’t hope to make intelligent decisions about how to achieve our business goals. The data we have available to us is richer and more voluminous than ever before. Collecting data is now almost a trivial problem — many businesses have more data than they know what to do with. Cloud platforms make it easy to store our Data Lakes and use them collaboratively. Data Analytics and Machine Learning technologies are capable of mining huge amounts of data for valuable insights.

But in itself data and Data Analytics platforms don’t help us to make good decisions. Data is inert — ones and zeroes on a storage medium. Of course, all that data means something in context, but what we’re interested in is how it can provide insights within our context. And without knowing our context — understanding what it is we hope to achieve — data can be more of a liability than a benefit.

It’s cheap to collect and analyze data, and we need data to make good decisions, so it might seem that more data is always better. However, without the ability to interrogate data for answers to the questions we need to ask, it’s easy to be misled. The first stage of any Data Analytics project shouldn’t be technological. It shouldn’t focus on how to gather data or where to store it. Right at the beginning, we should be thinking about what our business goals are, the questions we need answering to move us closer to fulfilling those goals, and how we can best discover those answers in the data we can collect.

Data can provide any number of vanity metrics that sound good but don’t really help us meet our business goals. Even worse, if misleading vanity metrics are used to set targets, work within an organization can be focused in entirely the wrong direction.

In the science world, trawling data for insights before forming a hypothesis is a cardinal sin. It’s all too easy to see what you want in the data. Instead, scientists hypothesize, design experiments that will refute or confirm their hypothesis, gather the data, and perform statistical analyses to ensure that their results are significant — that they actually answer the questions they’re supposed to.

I don’t expect the average executive to become a Data Scientist — that’s one of the reasons we need better and more intuitive tooling around data analytics — but, at the very least, they should ensure that they’re clear on the specific goals that data gathering and analytics are intended to support.

Data is a powerful aid to business decision-making, but data can’t provide the answers we seek unless we put it in the context of our greater goals.


Leave a Reply