"Data Pros Sick of Big Data." To the casual observer, that might seem like an oxymoron! Yet it is a real trend I feel needs to be parsed out a bit. What do we really mean when we say that?
Lately an increasing number of people I respect have told me they are “sick of big data.” And I must admit that on a certain level, I can relate.
Yet I also have begun to sense an intellectual disconnect in this Big Data Backlash. Why would data professionals lose interest in a the first mass-media meme to feature the word “Data”…especially if you keep in mind that what we call “Big Data” today is mostly a new name for a decades-old trend of exponential growth and progress in data capture, storage, and use?
Like many of my peers, I have been making fun of “breathless big-data hype” for more than a year. Like most attendees, I had a good laugh when @datachick gave her infamous “Big Data Smackdown” at EDW12. And last spring traded jokes about Big Data hype with Neil Raden, Merv Adrian, Chuck Kelly, and Krish Krishnan.
— Jaime Fitzgerald (@jaimefitzgerald) May 9, 2012
I got a quick response…from Big Blue. I hope I didn't hurt their feelings!
— IBM big data (@IBMbigdata) May 9, 2012
I confess it was a bit of a cheap shot…but I had just seen a billboard that seemed to imply that Big Data would save lives and make babies smile more…
Recently however, I began to feel like something didn’t quite make sense when it comes to the “big data meme.” As I spent more time reflecting on the disconnect, I realized that although we are tired of the hype, we need to keep faith in the potential. The only way for serious data professionals to “keep the faith” is to filter out the hype, and focus the realistic, authentic, potential that today’s data, technology, and methodologies make possible.
I decided to deconstruct Big Data into two entirely different entities: the hype…and the real potential...just like a data steward defining “two types of customer.” And I have felt more positive about Big Data since acknowledging that the word is being used in two distinct ways:
- ”Big Data Marketing.” Big data as buzzword, as a way to get attention, and a way to sell products. Often the buzzword is used in ways that make data professionals a bit skeptical, most notably by glossing over the challenges, hard work, learning, and judgment required to unlock results from data, regardless of its size and regardless of available technology.
- “Big Data” (not for marketing) is a new name for an old trend that continued to accelerate: as Neil Raden asserted in a recent presentation, what we now call ”Big Data” is a trend that has been ongoing since the 1970s: Exponential growth in the volume and variety of data, as well as the processing power and diversity of our hardware.
I believe that “Big Data” as a long term trend is as real and as lasting as Moore’s Law . Just as Moore’s Law defined a trend that transformed the planet, I see “Big Data Reality” as something that began long ago and will inevitably continue for decades. And so will the growing opportunities to improve decisions, processes, results, and human lives.
Of course we don’t believe the marketing hype, but let’s not let the marketing hype distract us from why we are in the field to start with. Let’s unlock potential from data, and be grateful that each year, we have more data than ever, and more ways capture, store, process, and analyze that data.