You are here:  Home  >  Data Education  >  Big Data News, Articles, & Education  >  Big Data Blogs  >  Current Article

Three Big Data Predictions for a Post-Hadoop World

By   /  March 23, 2018  /  No Comments

Click to learn more about author Alex Gorelik.

Enterprise data continues to diversify in both volume and complexity, as do the roles that humans and machines play in Data Management. While many organizations in the past have relied solely on Hadoop to process Big Data, the complexity and cost has brought us into a post-Hadoop world, where the Cloud combines with Hadoop and relational databases to allow organizations to process data across multiple environments in a way that is faster, cheaper and simply more in line with common sense.

This marks a new era for Big Data now that organizations can distribute their data in ways that make sense, thanks to new cataloging innovations that allow enterprises to discover, govern and access data in a centralized fashion no matter where it resides or is originated. Based on these trends, here are my Big Data predictions for 2018:

  1. Machine Learning Will Lift the Data-Driven Enterprise

Automation is critical for the data-driven organization in 2018. With the petabytes of data that enterprises must account for, there is simply too much for humans to do this on their own. In 2018, cobbler’s children will finally get shoes and all the Data Management and Data Governance projects will get assistance from machine learning-driven automation. Automation is also important to avoid errors and bad decisions, as the supposedly same data from different sources often doesn’t agree, costing organizations time and money.

  1. The Role of the CDO Moves From Playing Defense to Playing Offense

While in sports, it is often said that “offense wins games, defense wins championships,” the opposite is true for data-driven enterprises. A defensive Data Strategy is important to comply with regulation such as GDPR, CDOs must also be on the lookout for ways to take advantage of their organizations’ data assets by taking an offensive approach to discover new ways to increase business efficiency. But in order to do this, the enterprise must have its data in order to convert governed data into actionable intelligence.

  1. Organizations Will Fail to Meet GDPR Regulations

GDPR is quite comprehensive and regulates all aspects of customer’s data handling from protection, to usage and transparency. It will affect many different companies across a variety of industries, so it’s no surprise that organizations will fail to get their data assets to comply with the new GDPR regulations coming in May, but I predict that the lack of preparedness will be worse than many of us expect. According to the recent EY Global Forensic Analytics Survey, only 33% of organizations currently have a plan for GDPR.  With only a couple of months until the deadline, the number of organizations that will not be ready by the time it rolls around is sure to be staggering.

A combination of these themes, together with regulatory compliance, Data Governance and monetization of data, will play a role in 2018 being the year that Big Data moves beyond the hype and delivers on its promise.

About the author

Alex Gorelik is the founder and CEO of Waterline Data, a startup focused on enhancing the value of Hadoop through data self-service and governance. Alex is a serial entrepreneur and innovator who has spent over 25 years inventing and bringing to market cutting-edge data-oriented technology. Prior to Waterline, Alex was an EIR at Menlo Ventures. He joined Menlo from Informatica, where he held several executive roles, including GM of Informatica’s Data Quality Business Unit—driving marketing, product management, and R&D for an $80M business—and SVP of R&D for Core Technology—driving innovation in big data and social media while managing a team of 400 engineers and product managers developing Informatica’s platform and data-integration technology. Alex joined Informatica from IBM, where he was an IBM distinguished engineer for the Information Integration team. Follow Alex and Waterline Data at: Twitter, LinkedIn, Facebook

You might also like...

Thinking Inside the Box: How to Audit an AI

Read More →