There are a plethora of reasons to use Agile practices for the facilitation of Business Intelligence (BI), regardless if an organization relies on conventional warehouses, data stores, or Data Virtualization:
- Agile methodologies produce tangible results quickly and can assist in reducing the backlog of business requests for IT.
- They are ideal for implementing changes in a process that may fluctuate daily with ever-increasing sources and forms of data.
- Agile BI’s rapid iterations can provide a critical bridge between business and IT to ensure that the latter is satisfying the intelligence needs of the former.
- Agile practices provide transparency and visibility that is difficult to match with conventional BI.
Yet, all of these benefits can be negated by a couple of standard hindrances to the successful implementation of Agile BI. Frequently, organizational politics and rigid company cultures can stifle the developmental process. Misconceptions may initially abound regarding the timeframe in which goals are achieved, while insufficient planning can create needless iterations. However, the bulk of these common impediments can be greatly reduced by adhering to the subsequent guidelines for the successful implementation of Agile BI.
Product First, Not Processes
For a new project, the implementation of traditional BI methods can take upwards of a year or more to complete. IT and project consultants are bogged down in endless amounts of documentation that can transfer their focus from finishing the product to adhering to policy, which defeats the entire purpose of agility. Virtually every aspect of the product is predicted in time-consuming meetings, which may be easily outdated in a matter of months.
Agile BI yields results in a matter of weeks based on an adaptive approach which addresses all of these process-first problems:
- Documentation is significantly reduced to a description of short-term goals (referred to as Product Backlogs in one of the more popular Agile frameworks, Scrum) – with limited bureaucracy for achieving them – and a description of IT’s approaches for meeting those goals.
- Brief iterations (lasting no longer than a month) known as Sprints are employed to produce tangible, production ready features and applications for business users.
- Users in turn issue feedback to IT to influence future iterations.
Nuts and Bolts Planning
The reduction in time in which iterations can be implemented for Agile BI alludes to the fact that there is less bureaucracy involved, not less planning. Prior to compiling the description of goals for an Agile BI project, it is necessary to take stock of platforms and infrastructure. Specifically, it is essential to gauge the quality and adaptability of the various sources involved (especially for legacy systems) by examining their records. Users should also denote the various components integral to their architecture, and maximize their infrastructure by clarifying forms of software and naming conventions.
Once these fundamental aspects of Agile BI have been solidified and a description of goals is determined, organizations will want to identify dependencies for various BI components (functions that are dependent upon others for successful use). Ideally, a test iteration should precede the pursuance of any goals in order to rectify dependencies and firm up the structure of the project. In an Enterprise Data World 2013 presentation entitled “Making Agile Work with BI Projects: Tips & Best Practices,” Robert Solonynka of Data Experts, Inc. said:
“Sprint 0 is a good time to do a staggered start where you know there’s a lot of ETL work that needs to get done, and it’s dependent on data modeling. Use sprint 0 to get the data team ahead of the curve. Let’s start printing these target models so when we’re ready to produce our first tangible deliverable, ETL is already in the groove. The goal is to minimize people’s down time.”
Stakeholder Involvement: The Business
Any BI program implementation will not make it without the help of upper level management. The reality is that upper level management is motivated by the business, which is the crucial stakeholder in BI programs. There are a number of key ways in which IT – the department actually creating the program – must involve business users and C-level management for successful implementation:
- One of three definable roles in Scrum is the product owner, which can be an individual or a group. The product owner should be comprised of a combination of business and upper level management executives who define the requirements, determines the rate of release in which deliverables will be completed, and ultimately accept or reject the program itself. Except for the initial iteration, the business must be made aware of the completion of every production-ready iteration.
- With the input from the business, IT should create a release plan that denotes the projections for specific iterations so that the former knows when exactly to see results.
- Upon each release, IT should meet with the stakeholders to walk them through product developments.
- The business provides feedback regarding use-cases of the product that influences IT’s future processes in modifying the product. Other than such feedback, management and business allows IT complete freedom (based on the release plan) to implement their needs.
No “I” in Team
This final point is vital to maintaining the release schedule and meeting iterations, especially for an initial Agile BI implementation. New projects generally require a project manager to delineate tasks as an authority figure. IT members dedicated to Agile BI operate in a more republican fashion, with one team member denoted as the group’s advocate and everyone working equally to accomplish tasks for an iteration. The involvement of the project manager – like most managers during implementation – is to write reports denoting progress and scheduling.
There are several characteristics that make for an ideal Agile BI implementation team:
- Dedication: When possible, iterative teams should focus solely on working an Agile BI project, at least until it’s up and running
- Limited Membership: Iterative teams should remain small in number, to maximize individual involvement and reduce the risk of partisanship and bureaucracy.
- Multifaceted: It is not uncommon for BI technicians to specialize in various aspects such as reporting, ETL, modeling, etc. Iterative teams work well when individuals are familiar with more than one specific area, which encourages teamwork and cooperation to maintain temporal objectives.
- Communication: Team members must be frank with one another about which processes work, which doesn’t, and how to improve them. Such feedback is used to affect the processes for completing future iterations. It may help to conduct daily meetings in which individuals identify what they accomplished yesterday and what they will work on for the day.
Iterations are supposed to be extremely focused on completing one particular feature so the team can move on to others while giving the business something of value in the meantime. Points of deviation – which may appear aligned with the purpose of agility – should be minimized. Ideally, change should take place between iterations, not within them. Following is a list of commonly made errors and ways to prevent or rectify them:
- Cultural Politics: For organizations initially attempting to go Agile, seamless dedication to brief iterations may not appear substantial enough to meet expectations of all the shareholders. Grumbling about cultural inconsistency should be addressed by stakeholders (upper level management and business users) who leverage their weight to explain the program’s value.
- Outside contributions: Similarly, individuals not involved with creating iterations may not understand the value of timeliness in completing Agile projects. Reduce involvement outside of the team and plan ahead for cases in which it cannot be avoided.
- Complexity: For Agile BI, the best solution is usually the simplest one, until another iteration in which a process or tool can be refined. Simplicity also includes sticking to the goals of a particular iteration, regardless of how enticing additions may seem.
Making It Work
Agility is ultimately about teamwork as much as it is about flexibility, since the latter will not occur without a synthesis of talents and understanding between IT, the business and C-level executives. Sticking to the aforementioned guidelines can rectify situations in which cultural politics and unfamiliarity with Agile can prevent a program from taking off. With the proper planning and focus on the product, organizations can facilitate business intelligence tools tailor-made to the specific needs of its users.