I have a firmly, almost righteously, held belief that you can’t do Enterprise Information Management without some form of an enterprise information model. In fact, without one I don’t think you can effectively and efficiently do Enterprise Data [fill-in-the-blank]: Architecture, Management, Integration, Warehousing, Services, etc. This seems such a truism – admittedly from the biased perspective of a data professional — that it is surprising and disappointing that so many organizations balk at the thought of even attempting to build an enterprise model, and view it as at best a luxury with little to no payback or at worst an indulgence by egg-headed data modelers playing in their impractical theoretical realm.
My self-indoctrination in the value of an enterprise model began more than two decades ago, when I was building applications for mid-sized businesses for which we were integrating several apps around a common data model. We needed a visual model that helped both developers and business sponsors understand what information was controlled by which applications (a sort of ERD meets CRUD). The next logical extension was to integrate into a conceptual model the data of the enterprise that our particular apps didn’t share. Now we were abstracting the information model from the constraints of a particular business system, but onto which we could overlay the information requirements of the individual systems in play. With this enterprise picture we could make conscious and deliberate decisions in several aspects of application and information integration. For instance, you might buy an off-the-shelf package with a proprietary data store, which held some information that was redundant with existing applications. Data redundancy was a necessary evil, owing to the COTS system’s proprietariness and need to represent the information a certain way within the app. But by knowing what applications shared the same information domains we could decide, for example, that the packaged app would “master” the data for certain entities and we would replicate it (usually, in those good ol’ days, by overnight batch interfaces) to other dependent systems, or vice versa. The business problem we were facing was how to get reports that would tie out with one another when generated from disparate applications. Having the enterprise information model greatly facilitated the technical solution.
Of course, synchronizing redundant data stores is the problem space addressed in today’s world with speed and elegance by SOA. The problem is the same; the tools and methods have gotten better with time. Well-designed enterprise data services have their XML structures based on an application-neutral canonical model — a specific implementation of an enterprise information model.
Over my years in the trade, the enterprise model has proven its value again and again: as the basis for a SOA canonical model; as the cornerstone of a managed metadata environment, in which a fully attributed enterprise model serves as what we call master metadata; as the foundation for the Third Normal Form base model of an enterprise data warehouse; as a neutral representation of data requirements to support information integration within and between enterprises.
If an enterprise model is so useful, then why are they so infrequently developed? Let me mention a few of what I’ve heard as objections/obstacles and offer some thoughts about how to overcome them:
“It would be nearly impossible; our organization is too large and complex.” Well, my own response to complexity is to simplify, simplify. If you try to model all the complexity from the get-go you will indeed probably fail. But you should at least be able to model the high-level information domains. And logical models for some of your application systems may already be extant in various IT silos and can serve as some flesh on your conceptual skeleton. This overlaps with my next point about incrementalism.
“Sure it would be valuable, but it would take too long to do.” If, as above, you begin at the conceptual high level – subject areas and major entities or information domains – you can start demonstrating some value and fostering some very useful dialog among your architects. This is the type of effort that could appeal at the CIO level and get funded as a standalone architectural project, then you can build on that foundation as time and budgets allow. Indeed, funding such work is often one of the stumbling blocks, which leads me to…
“It is hard to find budget for this sort of thing.” Try including extension and amplification of the model into project budgets, so each IT development/enhancement project is also an opportunity to make the enterprise model more complete, accurate, and useful. At my current company, this was exactly the approach that worked in developing the Enterprise Logical Data Model. A high-level model comprised the first stage of work. Then, when the need arose to document the Operational Data Store that was a mirror of our main transactional system, that presented the opportunity to build more detail at the attribute level. The ELDM today is fully realized, but continues to be a work in process.
More on this, and the benefits of abstracting data representation outside of application systems, in future posts.