by Angela Guess
Last week Linda Briggs interviewed data evangelist David Loshin about his thoughts on Big Data and the challenges it presents. Loshin stated, “Big data is a concept that seems to have many facets, some trending towards performance and others toward flexibility. Everything centers, however, on addressing the information explosion. Not only is the amount of data growing at a tremendous rate, that growth rate continues to accelerate, encompassing both structured and unstructured data. To get a handle on extracting actionable knowledge from huge mounds of information, there is a need for a high-performance framework that enables analysis of unstructured data yet links it to our established analytic and reporting platforms.”
When asked about Big Data technology solutions, Loshin responded, “The frameworks for solutions have been around for many years; I worked on data-parallel computing 20 years ago, and it wasn’t really new then, either, but back then, the focus was on scientific programming, with the beginnings of intuition regarding high-performance business applications. Today, the barriers between performance computing and data management are really starting to crumble; the grid computing hoopla from a few years back and the popularization of programming models such as MapReduce and Hadoop are at least demonstrating some thought in moving in the right direction.”
Briggs inquired about Hadoop’s relationship with Big Data. Loshin answered, “At a high level, Hadoop is an open-source framework of software components that have been brought together to support alternative ‘big data’ programming and data management. For the most part, it incorporates a programming model that is based on Google’s MapReduce for analysis, along with a file or storage framework for managing access to large data sets.”
photo credit: Henry Swanson 420
