Click to learn more about author Dan Kusnetzky.

Big Data has been in the media constantly recently, but its definition and use still eludes some enterprise decision-makers. Their enterprises have invested heavily in Business Intelligence (BI) processes and applications and find themselves confused if what they have been doing will live happily under the new name “Big Data.” Much of the time, unfortunately, what they’re doing today really is different.

Another Catchphrase War is Underway

Although Big Data is a relatively new subject, it has already gathered quite a number of new catchphrases that address how the data is gathered, how it is analyzed and how it is used. Let’s review a few of them.

Some Big Data Catchphrases Seen in the Wild

As suppliers build products and offer services designed to deal with Big Data as a whole or some segment of that larger field, they often come up with their own catchphrase. The hope that their phrase will come to dominate the others. This allows them to claim that they originated the concept and that all other suppliers are following them.

Under the banner of “Machine Intelligence” the industry has begun to speak about “Artificial Intelligence,” “Deep Learning,” and “Machine Learning,” These terms may be used to describe how products work with the data before the enterprise can learn from the data. It may also be used to describe how the tool finds patterns and anomalies in the data to help the enterprise’s Data Scientists.

If we focus on how the data is being used, we hear phrases such as “Predictive Analytics,” “Intelligent Risk Assessment,” and even “Big Data Analytics.” These catchphrases have been used quite heavily when Big Data techniques are deployed to improve system and application operations, network performance and data and application security.

As the industry evolves, new catchphrases appear regularly. Often this means that a supplier is trying to position their products and services in a new way rather than offering large advances in the underlying technology.

In the end, when suppliers wave the Big Data banner, they usually are talking about how enterprises can examine large to extremely large amounts of data to ferret out previously hidden patterns, the ability to leverage a wide variety of data types, and make useful correlations based upon the new understanding enabling them to take fast action. Often the distinguishing features are where and how these techniques are being deployed.

The key questions enterprise decision-makers need to ask are “What is the business or organizational impact?” and “Is Big Data something we need to know more about and start to use?”.

BI is All About Examining What is Already Known

Author Mary Pratt says that BI “leverages software and services to transform data into actionable intelligence that informs an organization’s strategic and tactical business decisions. BI tools access and analyze data sets and present analytical findings in reports, summaries, dashboards, graphs, charts and maps to provide users with detailed intelligence about the state of the business.” In other words, BI is a systematic way for enterprises to ask questions and get useful responses from their information systems.

In the end, BI is based upon enterprise knowledge that something is going on and its needs to track and understood what happened. To that end, enterprises build processes and systems to gather the needed data, analyze it and then report findings based upon that analysis. The enterprise knows what needs to be tracked, how to analyze that data and even how the analysis should be reported and who should be informed.

BI became a big moneymaker for many suppliers. They developed tools to build and then utilize “data warehouses,” and offered sophisticated tools to provide decision-makers with useful dashboards and reporting tools.

Big Data is related to BI in several important ways, but is different.

Big Data

Big Data, on the other hand, is thought of as dealing with huge amounts of data but it is broader in its scope particularly in exploring previous unknowns. Often, the goal is learning what questions to ask by sifting through the enterprise’s own operational and machine data. Once those questions are known, BI processes can be used for additional exploration and reporting, but one of the more interesting uses of Big Data is to integrate analytics into the business operations as the business events are taking place. So it is not simply a way to better explain what happened. Big Data can impact the business results directly.

The Challenges Big Data Addresses

The challenges Big Data hopes to address are:

• How to capture and store such large amounts of data efficiently
• How to analyze that data so that the enterprise can ferret out a better understanding of its own operations or what its customers want and how it is addressing those needs
• How such huge amounts of data can be collected and support the processing and analysis directly. Especially in a secure fashion that addresses a growing list of privacy regulations
• How enterprises can sift through the data, ask important questions and visualize the results.
• Reduce the delays and latency so analysis can be incorporated into the operations of the enterprise.

Another way to look at this is that the enterprise doesn’t fully understand what is going on. It has observed changes in its business operations or in customer requirements but doesn’t fully understand what is happening. It may have seen sudden, unexpected increases or decreases in revenues, customer satisfaction or change in its competitive environment. The ability to adjust to these changes in real-time provides a significant competitive advantage. Especially with respect to the Business Intelligence focus of providing business insight that lacks the integrated automation to make the changes that those insights would imply.

Unexpected Changes

When the enterprise experiences unexpected or sudden changes they often begin the process to determine the “why” and the “how” that they missed before.

Competitors, for example, may suddenly enter the market. Old competitors may disappear or be acquired by companies that are seen as being outsiders. Adjacent markets may have begun to merge with and collide with their market causing unexpected and what are perceived as undesirable changes.

Huge Store of Data May Offer Clues

Quite often, these enterprises have a huge store of data that has been it has been accumulating for a long time, but it simply didn’t know what to do with it. It might contain operational data that includes sales data, production data, research data, and weather data. It may also have huge amounts of data coming from point-of-sale devices or manufacturing process control systems. It may also have information that comes from an understanding of regulatory changes or other economic changes.

After learning about the concept of “Big Data” enterprise decision-makers become encouraged to systematically evaluate this data and look for patterns and anomalies. This valuable information can provide the appropriate context to important the recent arriving data. So as a web page is loading, the customer experience can be optimized based on the deep historical data coupled with streaming and real-time actions.

In the end, they discover new questions to ask that helps them understand what happened and drive insight. This increasingly means that they come to understand that they need to automate responses that are more intelligent, that is driven by Machine Learning, that discerns both context and meaning so that the businesses’ own practices can be improved. Their goal, of course, is to increase revenues, lower costs, or both.

Enterprises Soon Learn That New Tools and Expertise is Required

Once the enterprise embarks on its journey to utilizing Big Data, decision-makers soon learn that it requires a different set of tools and expertise. At first, this area can look like it will require that the enterprise “boil the ocean” in order to get any value from the whole process. This, of course, can be time consuming and, in the end, not lead to the value that was desired at the start of the process.

We suggest that it is best to look for something simple that is highly likely to create new value or learning. This learning should lead to new opportunities and/or changing thoughts on current lines of business, products or services rather than being the painful study of what was obvious at the beginning.

Once on this journey, the enterprise soon discovers that valuable insights that arrive too late are not as valuable. It learns that it needs to “read the tea leaves” faster. It also soon learns that doing the same thing over and over without automating the process means that any benefits might be submerged under increased time and costs that process itself creates.

Often, the enterprise also develops what might be thought of as 20/20 hindsight. That is, is realizes that it “knew” somewhere in the organization that the changes were coming and even what to do about the changes. There are times that it will realize that it took advantage of that knowledge and gained some important benefits. Other times, it learns that it didn’t take advantage of that knowledge and was “blindsided” by events.

Now is the time

Big Data tools and processes have evolved enough that enterprises can now feel safe in learning how to take advantage of them. What they will soon learn is that this field has rapidly developed new tools, new methods, new ways of thinking. Many experts believe that Data Logistics are the key (see Machine Learning Logistics by Ted Dunning and Ellen Friedman for more information.)

Don’t go it alone

Now that the concept of Big Data has had time to evolve, enterprise decision-makers no longer have to feel like they’re on their own and that there are no maps, no established roads, and no guides. Many suppliers are now offering tools, established processes and professional services that can be put to good use. Remember to start small, gather experience and obtain actual value along the way.