Metadata Management makes a huge difference in finding critical data and helps enterprises not get lost within millions (or billions) of records. Metadata provides context about data, describing the information inside a data set, like a candy wrapper detailing the ingredients of the sweet stuff it encloses. This kind of data about data drives a search engine like Google to list results more precisely – or drives a person with a title and author, to find a book in the library or bookstore.
In a recent DATAVERSITY® interview, Emily Washington, Executive Vice President of Product Management at Infogix, shared five trends in Metadata Management that are aiding organizations to better understand their data and move into the future, describing how Metadata Management has transcended IT into the non-technical business world.
Historically, as Washington noted, Metadata Management originated from IT. IT created metadata to document software system design or to do Data Modeling. From this information, developers and managers could see where the data resided by viewing specifics about fields and tables. This technical metadata “informed Data Architecture and IT implementations,” she said. Now, Washington emphasized, users across an organization want to unlock additional insights from metadata, not just IT. She stated:
“Go back to the beginning of 2010 or 2015, and businesspeople start working with metadata, having a better appreciation of insights from metadata. They see it has value in providing additional context that some raw transactions or operational information cannot provide on its own.”
This additional context answers such questions as: Where does the data come from? How old is the data? And from what system did it arrive? The information fuels a wider metadata demand across an organization, impacting Metadata Management in five crucial ways.
Metadata Management Trends
As Data Management moves into the future, Washington sees Metadata Management evolving in these five ways:
1. Business-Focused Metadata Initiatives
Washington sees a norm shifting from IT to business in managing and accessing metadata. As part of that transition, she described how a business wants to build on that technical metadata, by translating the jargon (file system information, tables, fields, etc.) to the business meaning or contextual metadata. She explained:
“The business, through compliance groups and Data Governance, needs to understand what information lives out in the organization and what data fits in all of the enterprise’s systems. Companies need to trace back the available data to address these questions.”
In the meantime, other company sectors want to have access to this metadata to leverage information about where data lives, when it was last refreshed, who to go to for changes, and more clarity about the data retrieved.
This involves taking a slightly different Metadata Management approach than has been done traditionally, which includes tools and processes. Plus, as Washington elaborated, “Business-oriented teams end up partnering with IT, figuring out how to make this metadata friendly throughout the organization.” This goal is driving up the number of business-focused metadata initiatives.
2. Crucial Prioritizing of Data Quality Execution
Getting consumable contextual metadata to the rest of the organization means making metadata trustworthy, especially as Business Intelligence and other data flows down the pipeline from various sources to reports. As a result, business teams have more concern about Data Quality. Washington said that: “Traditional Data Quality tools address standardization, cleansing, enrichment, etc.”
But this leaves critical requirements around business processes and data lineage in doubt, as data moves from point to point. For example, a third party sends data through fifty different applications to a financial report. “How can this data integrity remain and its Data Quality be ensured?”
This interest in understanding Data Quality lends itself to metrics, including that of metadata. But a “financial user, marketing person, or an IT application owner defines Data Quality slightly differently,” said Washington. “While all these people share and use the same critical data elements.”
Washington understands, in response to this, Metadata Management programs coming together, ensuring availability, execution, and quality using metrics on critical data elements. “These metrics help show a return on investment (ROI) in a Metadata Management program,” she noted. This ROI drives organizations having varying Data Quality across different company teams and tools, to map this back to the Metadata Management program.
3. The Increasing Importance of Metadata in Data Privacy
Data Quality is not the only driver of Metadata Management programs. Data Privacy is as well. Washington explained that companies have to understand where sensitive data lives and what masks, encrypts, and secures it. Regulations such as the GDPR and CCPA, in addition to internal privacy rules, demand this. She said:
“Sensitive information drives a Metadata Management program because of the ‘spook factor’ to legally comply by knowing exactly where sensitive data lives across an organization.”
When it has been over decades, collecting data sources in all kinds of formats, how can a company zero in on highly sensitive and critical data sets, especially, where it has to act quickly or put specific policies in place? She asked: “How can this organization prove that it has done this and is compliant?”
The answers to these questions lie in the metadata. Organizations must use metadata as a cost-effective and easy way to prove during audits, compliance initiatives, and financial and outbound reports, that data is protected. Without that metadata piece, an organization just will not find that one customer who has asked to be forgotten, for example, and so will be unable to meet GDPR law.
4. Operationalizing Metadata Management
So, as Washington noted, if a company agrees that Metadata Management is necessary for legal compliance, how does it operationalize this after investing in all these resources and tools? Static documentation does no good. She remarked:
“Metadata information needs to be refreshed as new fields get added to systems or new inputs and outputs flow to and from them. Lots of automation helps manage metadata, keeping it up-to-date, so changes, additions, and deletions can be checked.”
Consistently updating the metadata by using some automation prevents the metadata from getting stale and becoming less useful. Maintaining Metadata Management operations along with business metadata, Data Quality, and Data Privacy initiatives give companies what they need to become data-driven and capable of monetizing metadata through machine learning and AI.
5. Applying Machine Learning and AI to Monetize Metadata
Organizations wantmachine learning and AI to automate as much as possible and to flag issues unseen by humans, providing important business insights. Washington believes this can happen through leveraging machine learning and AI in a scalable way, meaning quantifying data’s value through metadata. She asked how can an organization use metadata to quantify the value of its data, from a risk mitigation and a revenue generation standpoint? And how can companies collect hundreds and thousands of fields across all enterprise applications and get to the five percent of the critical data that means the most, the key points of interest (KPI)?
She responded to such questions with:
“Machine learning and AI can monitor historical metadata trends and usage. It can figure out, from metadata, what data has been touched most frequently, where sensitive information lives, and where redundant data exists.”
In other words, automating metadata through machine learning and AI leads a person to find high-value data, throughout the entire data ecosystem. Washington believes that this ability to use metadata to monetize data will increasingly become important to remain competitive in business.
Providing Metadata Management that is Business Friendly and Trusted
The five trends above mean businesses need accurate data, fit for use, across all the different areas, through Metadata Management. Washington described how Infogix helps companies do this by combining a portfolio of different capabilities (tools and technologies) and subject matter expertise. She said that Infogix bridges “metadata to quality initiatives.” One outgrowth of this ensures quality metadata is collected when building out a business glossary or data catalog.
She said that further capabilities, from Infogix’s Data360® product, “build-out Data Quality metrics and identify and score data value.” As a result, companies not only get Metadata Management with quality but a business-friendly self-service way to access trusted data and analyze it through BI tools. Washington thinks that many businesses, while recognizing the value of Metadata Management, just don’t know where to start. In closing she remarked that:
“It is just not possible to collect all the information across an enterprise and click an ‘easy button’ so the data is beautifully laid out for business users to access. It is a big process, program, and cultural change. In this, businesses and IT continue to struggle breaking down barriers. An organization needs support in this data journey to grow and evolve — a quick win, ROI on the first little project and then to scale up to other initiatives.”
This requires building up a Metadata Management program, brick by brick over the data journey. Infogix focuses on this, helping companies “foster successful analytical projects and expand their profits” by having confidence and trust in their data.
Image used under license from Shutterstock.com