Click to learn more about author Tejasvi Addagada.
With the ever-increasing variety of tool stacks, managing data has become more complex. The tool-stack needs to be managed along with the data that is either stored or processed by them. As we manage this disparate data actively, self-service business intelligence is possible. Further, this ideal state of management is possible by curating the metadata from all these tools and databases and managing them in a catalog. Most organizations shy away from managing metadata, as it can be cumbersome to maintain the required coverage from all platforms and keep it current. Further metadata is required to govern data.
Data analysis becomes much easier with the basic search features and multiple views and filters on systems and object types. Finding the right data element for your analysis needs is like finding a needle in a haystack.
If an analyst has previously used a data element like date of birth from thousands of similar data elements, he can certify that data element. This will help someone to pick the same data element in the future, as someone has built trust.
Self-Service Data Quality Profiling
Creating an on-the-fly Data Quality profile helps understand the outliers and the basic trustworthiness of data from the perspective of quality. As characteristics of data are available, features like frequency distribution of values, mix-max values, length, and data-type variety can be valuable.
As people start collaborating over the catalog and chat about questions on data elements, the purposes for which a data element is being used can be curated as well. This assists in complying with data privacy and protection legislation while also helps the organization in process excellence. A monetization score can be derived by understanding the purposes as well the number of people using a business term in their operations.
Most database administrators face challenges in managing access controls associated with personnel based on the purpose. Management of entitlements for data and personnel can be organized centrally in a catalog. There can also be a workflow managed with self-service user activities like approval by the data owner to manage the access.
Data keeps on shifting in structure and meaning. It will be helpful to manage the consumption platforms automatically based on the metadata coming from sources and changing the consumption table and column structures dynamically.