Data Management has Failed! And Presented Us a Historic Opportunity

By on

Click to learn more about co-author John Ladley.

Click to learn more about co-author Thomas Redman.

In addition to Tom and John, the following folks also contributed and co-authored this piece. We all feel it is time for Data Management to move “off of the beach.”    

  • Theresa Kushner is a Consultant for AI/Analytics at NTT DATA Services in Dallas, Texas.
  • Danette McGilvray is President of Granite Falls Consulting.
  • James Price is the Managing Director of Experience Matters .
  • Len Silverston is an author, consultant, speaker, and president of Universal Data Models.
  • Also, the authors also thank Laura Sebastian-Coleman for her inputs to early drafts of this article and editing the final draft, and Loretta Mahon Smith, President of DAMA-International, for her suggestions.

Data Management has failed. As a community, we have failed to educate our leaders on the need for high-quality data. We have failed to ensure that our companies and government agencies put effective Data Strategies and Data Governance programs in place. We certainly did not cause the COVID-19 crisis.  But the explosion of misinformation, confusion, and the resulting toll in human life and well-being – all expose and amplify the overall failure of data management.

It is easy to excuse the failures of data management during this crisis as stemming from a once- in-a-lifetime event, and driven by extraordinary stresses on the health care system. But these excuses simply don’t hold up:

If Data Management was not up to the task before the crisis, it is no surprise that it is failing now.

“Wait,” you say, “We’ve worked hard, and but no one has listened. It is not our fault!” This rings hollow. We’ve had a full generation to deliver results, get over barriers, and craft messages that people will listen to.  To be clear – we are not calling anyone out.  Many individuals have done great work, often under difficult circumstances.  But overall, results have not lived up to our promises. 

We have taken first steps, securing beachheads in quality and governance. We’ve created the foundations, and specified “the who” and “the how.” We’ve proven these really work and we’ve published the case studies in places like Harvard Business Review and Sloan Management Review, on company sites, and in industry forums.  Despite this progress, too few enjoy the enormous benefits of high-quality data or the oversight that governance is intended to provide.  We’ve not yet gotten our inter-related and required disciplines into mainstream business practice. It is time to get off the beach and move inland.  

The coronavirus presents us a historic opportunity.    Government, health care, and business leaders do not have what they need to navigate the crisis and plan recovery.  People cannot get answers to basic questions like, “When will there be toilet paper at my local store?” or “When will I be able to log onto my State’s unemployment site?”  The need for high-quality data has never been clearer or more urgent!  Most people understand the needs for better data are not going away. 

We must take a hard, but short, look in the mirror, resolve to make ourselves more relevant and useful, seize the opportunity, and take the steps needed to do so.

Our diagnosis suggests three root causes. We have:

  1. Confused “management” and “technology.”
  2. Failed to build political capital, perhaps due to a lack of courage.
  3. Accepted half-hearted measures as reasonable Data Management practice.

We believe that most in the Data Management community will agree on the essential truth in this diagnosis. We also worry that some will let minor disagreements cloud the issues. This is not the time to quibble. It is the time for rational, coordinated effort on a few fronts.

First, remove technology from the conversation. Entirely. Don’t discuss tooling, platforms, and “solutions” until you have sorted out the strategy, built the right organization, and put reliable processes in place. As Deming observed two generations ago, “Automating a process that produces junk means you produce more junk faster.”

We repeat: Do not start the work with tools! Imagine asking a carpenter to build you a house. The carpenter hands you a hammer and says, “Get started – it’s called a hammer, it is all the rage.” No blueprints, no engagement on the needs of your household. Too much data management is performed this way.

Don’t misunderstand: Data Management tools are great – but only to improve the productivity of well-defined and -managed processes. They cannot help you engage people, set strategy, build organizational capability, or advance a sense of urgency. The most successful strategy one of us (John) advised on involved translating a burning platform business issue into a data program. The only “tool” implemented was a three-ring binder containing data element definitions, metrics and other metadata.

Second, become a data provocateur. Build your credibility by improving data that matters to your organization.  This is the ONLY way to earn credibility. Then build your program up, one level at a time. This requires one part evangelism and three parts entrepreneurship. Stop asking  “How do I get buy in?” because buy-in isn’t what you need.  Instead, you must earn true senior level engagement. Convince leadership that improving Data Management is more important than at least one item already on their “top five.”

Implementing Data Quality Management and Data Governance properly means new roles and responsibilities, transformed cultures, and streamlined workflows. This is hard work.  There is no disguising it. There is no secret sauce or magic phrase that will bring the CXO on board. You can tell a CEO how important data assets are all you want. In our experience, most CEOs already believe it.  But unless it helps them achieve results quickly, they are unlikely to give you any real time or resources. 

Becoming a provocateur requires skills that most data professionals do not have, but many can develop. People in the Data Management community have spent the last twenty-five years shouting, “We’ve got to get closer to the business” and then failing to take the obvious steps to do so. The fundamental issue is not skill, but courage. We get it–provoking can be scary!! Think about it this way—your chances of success as a provocateur may only be 50 percent. But your chance of success from not provoking is ZERO!

Third, focus on outcomes, not minutiae. Stop being distracted by details. Our community’s job is to improve our organizations. Not to create abstract artifacts. This shift in focus yields astonishing results. So, learn about and adopt more pro-active approaches to Data Management. We need collaboration. This means aligning the data program to business goals. NOT an architecture roll-out plan. This means doing solid work on a common language and helping the organization know its own data. NOT depending on a glossary tool. This means attacking data quality issues – finding and eliminating the root causes of error.  NOT depending on a tool to re-actively cleanse data. For governance, this means real oversight and control. NOT chatting about the benefits of a modeling or lineage tool, a glossary, or golden copies. Start by proactively educating everyone, especially senior leaders. 

Some may find this diagnosis and these admonitions harsh.  Perhaps.  But now is not the time for caution. Seizing this opportunity requires bold, powerful moves. 

Finally, to those who wonder if the time is right, we say, “Look around, the pandemic provides more examples of obviously poor Data Management every day. We need high-quality data to fight it.  People’s lives and economic well-being are at stake. If not now, when?”

Leave a Reply