Attendees at the recent Enterprise Data World 2013 Conference & Expo in San Diego spent plenty of time learning about extracting and working with Big Data. But when it comes to organizing multiple data sets, too many in the industry are still using basic office software.
George McGeachie, co-author of Metadata Made Simple with PowerDesigner, led the workshop, “Death by Spreadsheet.” McGeachie spoke of how using improper tools can hurt an organization, preventing us from seeing the relationships between the things we build.
Technology Evolution
McGeachie began his career in the 1980s, working with International Computers Limited (ICL), a large British computer company that was purchased by Fujitsu in the 1990s. At ICL, McGeachie collaborated with colleagues on a data dictionary, where all of the company’s applications were created. As McGeachie explained to attendees, data items weren’t created by developers but by a separate worker, who was responsible for both managing and creating data items.
In those early “green screen” days, Data Management was straightforward because developers had only one environment. When McGeachie moved to a company that used IBM mainframes, he found they were also using a data dictionary and developers had more of a hand in managing data. His later work with British bank Abbey National allowed McGeachie to work with Popkin System Architect, which was one of the first Windows-based computer-aided software engineering tools.
Using Popkin, which later became IBM System Architect, McGeachie noted that the industry was still using the basic concepts he learned at ICL; but, he noted a waning interest in the technology. He lamented the fact that at Abbey National, developers had no way to link the models they were building to anything that explained the “why” of their existence. Despite knowing about the existence of frameworks, professionals were not using them for anything.
Understanding Frameworks
While data professionals and developers realize frameworks exist, they are unable to fully utilize them unless they have the right tools. With the right tools, developers can manage and categorize each of their projects, while also mapping the dependencies between them.
Instead of using specialized tools, many of today’s businesses are relying on basic word processing software to manage these frameworks. Interestingly, one attendee pointed out her company was suffering from “death by Sharepoint,” where so much information is stored on the Cloud-based project management tool, it’s impossible to find anything.
During requirements analysis and development, teams have multiple issues to conceptualize at once. Decisions like which tools to use, how multiple users will authenticate at the same time, and how will multiple versions be managed all must be made at the frontend, with those decisions reflected throughout development.
To reflect these decisions, many businesses use spreadsheets and word processing documents. Businesses feel as though using spreadsheets will keep the process simple, while saving money on software licensing fees, since most businesses already have Microsoft software licenses in place.
McGeachie finds it ironic that businesses deal in so much Metadata without linking any of it together. This creates “Metadata silos,” with multiple documents, spreadsheets, data models, and more operating concurrently, developed by separate teams. Because so often these silos are being created independently of each other, it’s easy for team members to lose sight of the bigger picture without a tool to connect them all.
To craft better data organization, businesses should replace data silos, creating an environment that demonstrates cooperation between various data sets. This environment should be service-oriented and complex, and be in a format that can be easily updated, accessed, and shared.
The Solution
By leading attendees through an exercise, McGeachie demonstrated how businesses processes can become disjointed and time-consuming. By organizing these processes, he finds companies can become more efficient, avoiding duplication of effort.
In an article for The Data Administration Newsletter, McGeachie outlined the many issues with relying on software like Microsoft Excel and Sharepoint to manage frameworks. What starts as a few simple documents can quickly expand to hundreds of documents, each of which must be updated manually. Links between documents can break, leaving teams to struggle to find the one specific piece of information they need.
For his recommended resolution, McGeachie returns to his data dictionary roots, providing a central Glossary of Terms that can be used as a central resource for any changes made. Through using common data definitions and cross-referencing to use cases, data can be more easily managed.
Enterprise Data World attendees also had the privilege of attending a workshop conducted by Alec Sharpe, senior consultant for Clariteq Systems Consulting. During the workshop, titled “Business Analysis Techniques for Data Professionals—Getting Involved in Business Processes and Applications,” Sharpe discussed change management for data professionals. McGeachie embraces Sharpe’s belief that a data or concept model is a critical tool in connecting those silos. That data model is the glue that binds everything else together, Sharpe emphasizes.
Workshop attendees were treated to a reproduction of Clariteq’s business analysis framework. This framework shows how Clariteq brings multiple cases together by requiring that each new process, use case, or objective is described and named using a central terminology. That terminology is represented in Clariteq’s data model. By creating an overlying business objective that the process, use case, and service are extracted from, Clariteq’s framework .
The Right Tools
People are using fewer tools than ever, while working with larger data sets than ever. The key isn’t using more tools as much as using the correct tool for each purpose. It’s important that businesses take data mapping seriously, treating it with the same care as they would any other business process. By creating a thorough, well-thought-out glossary on the front end of a project, data professionals will find that change management is made easier, since consistency is found in every facet of operations.
As Big Data continues to evolve, more serious attention will likely be paid to organizing data sets and bringing these “silos” together. These frameworks will provide an overview of an organization’s Metadata, helping all team members to work together to create projects that meet an organization’s goals and objectives.
