You are here:  Home  >  Data Education  >  Big Data News, Articles, & Education  >  Big Data News  >  Current Article

Beyond Hadoop: Integrating Countless Unstructured Data Streams

By   /  December 8, 2015  /  No Comments

hadby Angela Guess

Daniel Gutierrez recently wrote in InsideBigData, “Industrial environments such as those found within large electric and gas utilities are producing massive volumes of data in real-time that is overwhelming traditional ICT architectures. Additionally, the variety of operational data generated in industrial environments is orders of magnitude more complex than what is typically found in ICT environments. With the ever-increasing number of connected devices, sensors and network systems throughout industrial environments, these organizations are often struggling to integrate millions of unstructured data streams from disparate sources, trapped in siloed legacy systems, all in different formats. All this data can be a valuable resource for driving operational efficiency and coping with transformational changes, but only if it can be transformed into actionable insights and meaningful intelligence.”

Gutierrez goes on, “Many open-source architectures such as Hadoop data lakes, Storm, and even data warehousing methods have been touted as solutions to these types of ‘Big Data’ challenges. However, these approaches alone simply cannot handle the scale and complexity of industrial data, or provide real-time analysis and situational awareness. Without the ability to integrate the raw data into a common model and gain a holistic view of the big picture and potential risks in real-time, industrial enterprises are struggling to achieve operational and business benefits from the sensor technology and data integration and management architectures they have implemented. Hadoop may work in many businesses, but it simply cannot handle the massive amount of data generated in industrial environments. It simply wasn’t built for it. Solving this data challenge requires a new thought process and approach, including a new purpose-built architecture for industrial infrastructure, communications and storage.”

Read more here.

photo credit: Hadoop

You might also like...

Analytics vs. Data Discovery

Read More →