by Angela Guess
Wayne Applebaum, VP of Analytics & Data Science at Avalon Consulting, recently wrote in InsideBigData, “As an ever increasing variety of data gets captured and analyzed, we face challenging issues of data governance in general and metadata governance in particular. Data governance has always been a challenge. Organizations have struggled for decades to get well-defined customer or product master data. But now, in the world of data increasing in velocity, variety and volume, they are also faced with maintaining increasingly complex metadata and data lineage caused by not only the variety of ways a piece of information can be used but the increasing importance of relationships among various data elements. This is particularly true in the area of networked devices, the Internet of Things (IoT).”
He continues, “Historically, data governance has been centered on two types of data: Transactional and Master. Implementers of ERP systems have rightly focused on these two types of data. But, the game changes when instead of just wanting to process transactions (such as orders, payments, payroll) we want to consolidate or analyze transactional information to make better business decisions. Also, transactions aren’t only coming from ERP systems; in the Big Data world it is also a tweet, a post, a click or a sensor reading. The key is consolidating this disparate information so we are able to make a decision. This “Decision Data” results from the combination of information. It could be as simple as finding the mean and variance of orders of a given product or as complex as creating a scoring algorithm for the litigation potential of an email sent to customer service. In short, any time we combine data to create new information we are creating Decision Data and metadata. This metadata needs to be documented and governed.”
Photo credit: Flickr