by Ken Orr
“A biannual survey by the American Society of Civil Engineers, grading all categories of (U.S.) infrastructure from schools to sewers, indicates a gap of $1.6 trillion over five years between what is needed to bring national infrastructure up to reasonable standards and what is now in prospect. School buildings alone would require $125 billion to reach a minimal standard of safety and soundness.” Felix G. Rohatyn and Warren Rudman
I happen to live in a state (Kansas) which is known for quality of its roads. This is important, because in addition to agriculture and aerospace, transportation is one of the key factors in Kansas’ economy. The quality of Kansas’ roads is obvious; all you have to do is drive to any of the nearby states. When you travel from Kansas to, let’s say, Missouri or Nebraska, you are aware immediately of a number of not so subtle changes: the roads are bumpier, there are more potholes, and there are more sections under repair. As a result of these better roads, Kansas has a distinct advantage from an economic standpoint; it is a key infrastructure, even in a time of soaring oil prices.
Many of Kansas’ neighbors, especially Missouri, are now engaged in major highway programs to bring their roads up to what motorists and truckers demand, but, in the current economic environment, this is a difficult task. With the other demands of society and politics, rebuilding a degraded infrastructure can take decades. Looked at from a strictly economic viewpoint, then, Kansas’ roads give the state, in the words of Warren Buffet, a “sustainable competitive advantage”, one created by foresight and investment over the years. Indeed as I tell people, the roads that Kansas’ citizens, motor carriers and visitors ride on are the result of decisions and investments made ten or even twenty years ago.
Investing in Data Warehouses
What is true of roads is also true of information (data) systems management. Like roads, engineering and implementing a business intelligence/data infrastructure takes time and investment. Like roads, it is possible to concentrate on small road changes and fixing potholes, which are cheap and easier to implement than large major changes, but in the end, it is the major roads that provide the most benefit (and most of the traffic). Take the issue of data marts vs. data warehouses. The question of whether an enterprise business intelligence strategy should be based on central data warehouses or data marts is not a new one, indeed, this has been a hot topic since the very earliest days of the “data warehousing” movement in the early 90s. What is disturbing is that it is still being asked today nearly 20 years later.
There have always been two camps in the data warehousing debate, those who think strategically and those who think tactically, and, unfortunately, business intelligence has been largely dominated by tactical thinkers. However, I think it is important that we look at what we’ve learned in the last 20 years of data warehousing and BI infrastructure
The first thing to note is that there is a significant difference between “getting THE answer” and “getting AN answer” A marketing manager, much like a battlefield commander, faced with a critical meeting in a couple of weeks may be willing to take whatever answer he can get. However, the CEO of a multi-billion dollar business who must make serious strategic decisions in the midst of a recession or an international energy and food crisis wants to know exactly where her company is with respect, let’s say, to this quarter’s profit and loss by product and geography.
Clearly, if it is solely a question of dollars and cents, then the initial cost of building a data mart solution is nearly always cheaper than developing a more robust data warehouse solution. And building a data mart is often faster than developing a data warehouse solution as well—but not always. But, over the long haul, the cost of developing a number of data marts often exceeds the cost of developing an incremental data warehouse solution. The reason is simple—data quality.
It is easy to come to believe that business intelligence is simply a question of accessing data that is found within various (or even one) systems within the organization and presenting it quickly to the appropriate manager or professional in a slick, multi-dimensional form, but this is not the case. Much of the data that resides in various operational systems is just good enough for operational purposes. For example, data that is just used for billing doesn’t have to be concerned with customers having multiple bills based on multiple account numbers. As long as the customer pays every bill, the billing folks are happy. But the marketing folks often want to know, as close as possible, how many customers they really have and how much they purchase.
I work with trailing edge enterprises and leading edge ones. The difference is often the result of decisions that foresighted executives and professionals made years, sometimes decades ago. Recently, I have been working with different organizations working on implementing SOA. Two firms that I’ve worked with in the financial services arena were using SOA to implement customer facing applications. One organization had invested millions of dollars and a decade of time to build core data warehouses around their customer, the other had not. What was possible, in terms of moving into the next generation of customer services and cross selling, for the organization that had invested in the heavy lifting of building a data warehouse based on a robust Operational Data Store (ODS) was simply not on the table for the organization that had not. The organization without a visible customer focused data warehouse could only think about cosmetic SOA applications.
When organizations engage in data warehousing they learn a great deal about what they know and what they don’t. They begin to understand that their current operational systems only give a “blind men and the elephant” view of their data. Those who invest in integrating their data and improving the quality of that data can move on to the next level, whether that next level is SOA or Master Data Management—those who don’t invest can’t. It is simple as that—to paraphrase Bill Clinton, IT’S THE DATA STUPID. Don’t let point solution vendors and consultants talk you into taking your eye off the ball.
Reference
1. Felix G. Rohatyn and Warren Rudman, It’s time to Rebuild America, Washington Post December 13, 2005; Page A27
ABOUT THE AUTHOR
Ken Orr
Ken Orr is the Chief Scientist of The Ken Orr Institute and a Senior Consultant and Fellow of the Business/IT Trends Council of the Cutter Consortium. Mr. Orr is a leading researcher on Business Process, Business and Data Architecture, and Model Driven Development. Mr. Orr has published 3 books and dozens of research reports and technical articles.
