Real-time analytics (on data as they stream in, commonly referred to as Operational Intelligence ) provides unparalleled short term opportunities to reach customers as they are generating data and is now a reality. It is largely fueled by the burgeoning Internet of Things (IoT), e-commerce, and the brick and mortar retail industry.
Yet, its benefits can be found in a variety of vertical industries in which data helps to drive operations. Its current low adoption rates are primarily fueled by infrastructure concerns and a disconnect between business analysts and operations application developers, each of whom believes the other should be responsible for real-time analysis.
According to Chief Operations Officer of ScaleOut Software—which provisions Operational Intelligence via its ScaleOut Analytics Server and ScaleOut hServer—David Brinker, the third reason for low adoption rates might be most daunting of all:
“There’s not widespread understanding that it’s possible. It seems to be a novel idea. When we talk to certain customers and prospects there’s sort of a ‘you can really do that?’ kind of reaction. Then it’s something that people are saying ‘I want to go do that.’”
What Operational Intelligence specifically enables enterprises to do is analyze data states while they change in real time. Additionally, it is able to augment that instantaneous analysis of streaming data with historic, strategic analysis provided by conventional BI and warehousing. This article will demystify the process and explicate how enterprises can leverage this technology.
Infrastructure Needs
The principle concerns for implementing Operational Intelligence include addressing existing infrastructure needs—many of which are common to Big Data initiatives. Depending on what specific area of an industry an organization is in, those needs will involve utilizing a cluster of commodity servers and any assortment of Cloud-based data stores (such as Mongo DB), and simply accessing either of the two ScaleOut Software options through the Cloud. The scalability issues associated with Big Data can also be accounted for with Open Source Hadoop.
The Cloud-based, in-memory data grid capabilities of ScaleOut Software’s solutions can then track the state of data changes in real-time, and run specific algorithms that are relevant to an enterprise’s use of Operational Intelligence. Such relatively simple infrastructure is all that is required to enhance e-commerce and brick and mortar cross-selling and upselling opportunities via recommender engines, or to influence the analysis of hedge funds and other opportunities prevalent in the financial industry. ScaleOut Software also offers on-premise versions and those for private clouds for organizations with sensitive data needs.
“The same physical infrastructure that you could deploy for a Hadoop batch cluster and that which you would deploy for offline processing, you could also deploy for this purpose,” ScaleOut Software CEO Dr. William Bain noted. “It’s no different than deploying a cluster of commodity servers for Hadoop to get the benefits of Operational Intelligence.”
The only other additional infrastructure would be required for supplying data with domain specific, special purpose hardware (such as that in the oil and gas industries), which depends on an organization’s business needs. What is described in this section is all that is needed for real-time analytics for data supplied by conventional web servers.
BI and OI
It is fairly easy to combine real-time analytics with historic data provided by conventional BI tools and data warehouses to get business analysts to work with both data sets, and to free dev-ops personnel to work on creating apps. In much the same way that real-time changes to data states can be ingested in memory in ScaleOut Software’s technology, offline, historic data can also be ingested to enrich real-time data with static data sets.
Organizations can then run analytics on these data aggregates that effectively combine the best of BI with Operational Intelligence to provide a more thorough analysis of data that could involve, for example, customer behavior. These results can then be forwarded to a recommender engine to provision additional choices and bargains to customers in a process that would not be possible without Operational Intelligence. Bain remarked that,
“We think that there’s a beautiful co-existence between BI and Operational Intelligence. In fact, it’s almost obvious in plain sight that there’s a natural fit because the data warehouse is where you do strategic analysis for a business. You look at historic data, you really want to understand patterns and trends and you want to understand large data sets so you get valuable insight into that data, and you’re able to then make big decisions.”
Real Life Applications
Examples of use cases of Operational Intelligence abound in virtually any industry in which real-time analysis of constantly changing data can influence business processes. ScaleOut Software’s solutions effectively host in-memory models of any number of physical entities or events and provide real time feedback via parallel analytics that can be significantly enhanced with historic data.
Within the entertainment industry, a cable company customer utilizes ScaleOut Software’s technology to ingest tens of thousands of events each second from tens of millions of cable set-top boxes throughout the country and makes use of that data by offering domain specific recommendations to viewers to fuel their next selections. ScaleOut Software’s data parallel analytics enables the company to issue recommendations on an individual basis for each box, as well as to build and deploy regression models for data aggregates that might vary by region, season, or other demographics. Bain explained:
“In the cable space, there are companies that specialize in media based recommendations. Our goal here is to get them the data they need within a second or two or less. I think we can demonstrate that we can actually get them the viewer state in hundreds of milliseconds, so they can then run their recommender engines based on their domain knowledge and then generate feedback instantly as opposed to waiting for hours for this data to be put into a database, correlated, cleansed and enriched—which is what typically has to happen very slowly.”
Another ScaleOut Software client is a beauty supply wholesaler. When customers visit its site to browse and select products, the enterprise can model each one as a separate entity in ScaleOut Software’s in-memory data grids. Doing so effectively enables the wholesaler to “see” its customers’ behaviors in real time. The potential to augment that customer experience with historic data from a particular customer’s previous experience effectively provides a comprehensive set of data with which to fuel recommender engine algorithms and issue tailor made sales pitches.
IoT and More
Perhaps the most convincing examples of Operational Intelligence are found in the IoT, which is largely based on continually streaming data—typically in the form of sensor data—which companies can analyze in real time to determine maintenance and repair needs for equipment assets, levels of productivity, and basic monitoring (which can even be leveraged as a new asset that original equipment manufacturers can sell to their customers).
Regardless, Operational Intelligence represents what very well may be the final evolution of BI. It is critical to note that this evolution does not supersede the typical historic insight yielded by BI; to the contrary it complements it by utilizing it to give strategic insight to the tactical knowledge found with real-time analytics. Moreover, it is important to note that the potential for Operational Intelligence is not some far-flung, futuristic pursuit found in a science-fiction movie—the technology exists and is regularly used by most enterprises with Big Data initiatives.
“Our analytics runs on a cluster of commodity servers identical to those you would use for regular Hadoop,” Bain said. “I think the Internet of Things will be an important driver of Operational Intelligence; I don’t know if it will be a primary driver. My instinct is that web shopping will be a primary driver for the reason that it’s so easy to plug this in, and there’s so much value that’s obvious in being able to recommend with much finer granularity than most people do today.”
