No matter the specific data discipline, Metadata probably plays an important role for the data professional. Structural Metadata involves the description and design of data models and structures, while descriptive Metadata describes the actual data itself. This serves as more of a data dictionary role, being sometimes referred to as "metacontent."
Data industry thought leader Krish Krishnan feels that Metadata is vital for any project that involves the integration of Big Data. No matter the platform or technology, poor knowledge of Metadata can lead to project failure.
This article looks at current trends in Metadata; looking at both state of the art products and concepts about this life blood for the data professional.
Data Advantage Group's MetaCenter Platform
Data Advantage Group is known as a leading provider of enterprise level asset management software. In fact, their flagship product, MetaCenter, has won much kudos in the industry, including various readership awards from DM Review magazine (now called Information Management).
The MetaCenter Platform leverages DAG's patent-pending ActiveLinx technology to construct a catalog of an enterprise's stored Metadata across their entire IT application infrastructure -- all in real time. MetaCenter Repository provides a standards-based model to link an organization's information assets to their physical IT systems. The MetaCenter Portal allows easy access to this information for both the technology and business sides of the shop.
Data Advantage Group and the MetaCenter Platform get no better endorsement than one from data warehousing legend, Bill Inmon, "Data Advantage Group brings a fresh, modern perspective to the world of enterprise Meta Data Management." Firms with an interest in managing their corporate Metadata assets need to take a look at the MetaCenter Platform.
Meta Integration Technology
Metadata component provider, Meta Integration Technology, has a unique moniker as the "Switzerland of Metadata." The company earned this plug from Giga Information Group analyst, Lou Agosta, in an article in IdeaByte. The moniker makes sense considering any software component's ultimate value is increased by playing nicely with as many vendor platforms as possible.
The core of Meta Integration's component technology is the Meta Integration Model Bridge (MIMB) which facilitates the movement of Metadata between various database-related platforms. Users of MIMB are generally programmers and developers looking to transfer Metadata models seamlessly across methodologies and between database tools from different vendors.
MIMB works with Metadata from both relational and OLAP databases. It also handles Metadata from XML documents as well as data and object modeling tools. It operates within a host of Data Warehouse and BI environments, in addition to various Metadata repositories. Finally, nearly all of the relevant Metadata standards are supported by MIMB. A current list of supported tools and platforms is on Meta Integration's website.
Organizations looking to engage their development staff in the engineering of tools to perform Metadata management, integration, and exchange need to check out Meta Integration's MIMB product. It appears to work like a fine Swiss watch.
Metadata Professional Organization
Metadata professionals looking to further their career by joining a professional organization have an option in the Metadata Professional Organization. The MPO also sponsored the Metadata Track at this year's Enterprise Data World in Atlanta. The non-profit international organization serves professionals in all Metadata disciplines, including administrators, developers, architects, and managers.
The MPO helps with the sharing of best practices between their members, as well as providing access to external knowledge on current Metadata tool vendors and platforms. The organization's robust online community also supports professional knowledge sharing. The MPO strives to be the "premier resource for Metadata know-how."
Joining the Metadata Professional Organization is a breeze. They accept checks, money-orders, and PayPal through their website. A yearly membership is $75 for individuals and $250 for companies or groups including up to 10 memberships. Vendors can join for $750, while a student membership costs only $10.
In many cases, professional organizations are the lifeblood of any discipline, providing both a sense of community and a repository of shared knowledge. Metadata is no exception, as the MPO proves.
Demystifying Metadata at Fannie Mae
National mortgage financing company, Fannie Mae, serves American families and the housing market by securitizing mortgage loans originated by banks and other lenders. Samra Sulaiman leads the Enterprise Metadata Management and Enterprise Logical Data Modeling programs for Fannie Mae. She recently gave a talk demystifying Metadata by showing how it is able to benefit any enterprise.
Sulaiman quoted Gartner's opinion that Enterprise Metadata Management serves a role to ensure that data is usable across an entire organization. Essentially, proper management of Metadata enhances business productivity.
This increased productivity is vital, since many Metadata initiatives fail because they do not deliver increased enhanced business value. In some cases, these unsuccessful projects lack corporate level sponsorship, or they are treated more like a one-off exercise as opposed to a recurring strategic investment.
At its core, Metadata provides context, turning random data into usable business information. It enables business insight, and where relevant, assists in the regulatory compliance process.
When implementing a Metadata management program, Sulaiman feels it is important to think big as far as the scope of the initiative, but execute and deliver the work in manageable increments.
Sulaiman also covered Metadata domains, showing how they allow effective Metadata management to link corporate assets across an enterprise. Using a maturity model and metrics is also vital in measuring the impact of any Metadata management initiative. "If you can't measure it, it didn't happen!"
A Wiki Approach to Metadata
Scott Thompson is the Manager of Data Architecture for the Ontario Teachers' Pension Plan. As the largest single-profession pension plan in Canada, his organization manages over 100 billion dollars in assets. His recent EDW presentation covered the use of Semantic Web- and Wiki-related technologies as a Metadata management platform.
When beginning a Metadata management initiative, his firm focused on the findability and ownership of data, along with its overall quality. Later on, ownership of Metadata and its quality were added to the soup.
Two main benefits achieved through this Metadata initiative were a platform to publicize and share Metadata, increasing developer and analyst productivity, and the fact that Metadata documentation "transcends time," mitigating the issues of staff turnover and multiple project balancing.
Of course, the most logical structure for this Metadata platform was a Wiki. This allows for multiple people to quickly create and share content, also making it easier to reach a consensus from the entire enterprise.
Semantic technologies were also used on the project, facilitating both search functionality and automated data association, especially between business and technical Metadata. Thompson's team essentially created a Semantic MediaWiki as a Metadata management platform.
His presentation ended with a very relevant quote from wiki-management.com's Rod Collins, "the smartest organizations are not those with the smartest people but those with the quickest access to their collective knowledge."
Metadata Management at eBay
Nearly everyone is familiar with the online "garage sale" website, eBay. Their Sr. Manager of Enterprise Data Architecture, Mark Uksusman, helped to preside over the creation of Global Directory, eBay's term for their Metadata management system that helped development of a data warehouse using Agile project techniques. He covered the project at this year's EDW conference.
Scale is an important issue at eBay. With 50,000 product categories, 50 terabytes of new data each day, 100 Petabytes of data processed each day, and near 100 percent 24/7 availability, eBay knows Big Data. The company employs over 7500 business users and analysts to navigate their endless seas of data.
Before implementing Global Directory, eBay suffered from a mish-mash of multiple, unrelated Metadata repositories. Now many different sources feed Global Directory with both technical and business Metadata, allowing a common interface to be used for access, as well as the ability to create individual Data Marts.
eBay uses technology from ASG Rochade for their Metadata. For model development, the Agile Model Driven Development (AMDD) methodology is leveraged, allowing for full integration of technical and business Metadata. This allows for easy cross-domain analysis as well as the real-time generation of Virtual Data Marts.
End-users access the Data Warehouse using dimensional models and MicroStrategy cubes. Like the folks from the Ontario Teachers' Pension Plan, Wiki technology is also enabled to facilitate access to information. Global Directory is a fully developed solution providing business value to eBay.
It is obvious that successful Metadata management adds business value to any enterprise willing to undertake the initiative to implement a solution. Remember that quality metrics displaying a return on investment remain vital in earning the C-level buy-in necessary to fund this kind of project to its ultimate completion.