Preface (Agile BI Book)
This blog post presents a preliminary version of the Preface of the collaborative Agile BI book. You can access the various sections currently available and you may register to join this collaborative effort to start contributing. I also invite you to comment this content at the bottom of the blog post.
Business intelligence (BI) applications are increasingly popular and organizations of all sizes are seeing the benefits. As a consequence, BI applications are no longer limited to senior executive but are increasingly deployed throughout departments and levels. The timely completion and successful implementation of BI projects provides a competitive advantage to the adopting organizations. Unfortunately, there are more failures and disappointments than successes.
There was a running gag in an organization we were working with that every Business Intelligence (BI) project required 3 years to complete and would cost over $3 Million. This running gag demonstrated an unfortunate situation – most BI projects are expensive, time and resource consuming and have a low rate of success. Along these lines, very few people would debate that traditional BI projects require heavy investments and result in long delays before providing value to the organization.
Much has already been said about the importance of BI applications in organizations. One of the most obvious and frequently repeated benefits of BI applications is the ability to increase the quality and timeliness of the business decisions being made. This is increasingly true as BI applications are being democratized and are now embedded in the decision making process at every level of the organization and across multiple departments.
In addition, decision makers constantly need an in-depth understanding of their business’ strengths and weaknesses and this is especially true during difficult economic times. The availability of timely and accurate information can help business leaders make the right decisions. Along these lines, the survey of over 1,500 CIOs shows that despite predicted flat IT budget growth in 2009, BI projects remain their number one technology priority.
There isn’t much need to emphasize further the value of early diagnosis and the implementation of timely solutions but in a changing business environment the business needs evolve too quickly for sequential waterfall development methodologies.
Case in point, it is estimated that 60% of BI projects end in abandonment or failure and market data confirms that out of the 3 standard project dimensions – time, resources, and scope – time lines and resources typically exceed the initial plan while scope consistently fall short of the original expectations and requirements. As such, an approach that delivers value early in the project while remaining aligned with the business priorities is recommended.
In addition, estimates show that no more than 20% of business users actually use their BI applications proactively and that a staggering 64% of systems functionalities are rarely or never used. This regrettable situation compelled us to look for an alternate and improved approach to ensure BI projects have greater success rate. More specifically that BI projects deliver the features required by business users (scope) in a timely fashion (time) while controlling expenses (budget).
Our intent is neither to rehash the unfortunate track record of BI projects nor to highlight the most dramatic failures but to propose a different approach to the development of BI projects.
This is a tall order but with the increasing popularity of AGILE iterative and incremental approaches and with repeated success in other specialities; we know using such an approach in the development of BI projects leads to much better outcomes.
The AGILE approach heavily relies on business involvement within the project team and throughout the duration of the BI project. Having direct contact with the business team, the development team will more easily understand the expectations and will increase its knowledge of the business which will make a more knowledgeable and productive team altogether.
In order to achieve our objectives to: reduce the initial investment, accelerate the availability of the information, deliver the right solution to the business users and constantly deliver value to the organization we explained that an AGILE approach is the right solution. Through iterations and increments, we use a “slicing approach” to develop from source systems to presentation layer in order to deliver on the business users’ requirements.
The benefits of an AGILE approach to a BI project are substantial. From our perspective, the most important one for the organization is that it forces the project team to focus on ROI and deliver high value by spending efforts on activities that will bring the highest value.
By reducing the duration of the development cycles the business and development team can modify their release strategy and focus on new priorities that may arise as a consequence of a changing business environment.
 Business Intelligence Summit by Gartner, 2008
 Standish Group Study Reported at XP2002 by Jim Johnson
 Within the context of this white paper and for simplicity, we combine Data Warehouse (DW) projects with Business Intelligence (BI) projects.