Skip to content

Posts tagged ‘Incremental Software Development’

Explaining the business value of incremental development

Here is the second of two posts on iterative and incremental software development. My previous post focused on explaining the business value of iterative development while the objective of this one is to explain the value of incremental development.

Incremental Software Development Applied to Business Intelligence

Incremental Software Development Applied to Business Intelligence

Starting with the decision of using an incremental development approach as opposed to a sequential approach for the development of a BI solution results in 3 immediate outcomes:

  • Reduces overhead: Once the development team determines it will be using an incremental approach, this decision allows it to focus on small tasks and build on them. As such, it is no longer required to plan upfront for every component to be built which would be the recommended approach if the team only had one attempt at building the right software. By developing the application in increments, the team can focus its efforts on the tasks at hand within the known context and not worry so much about figuring out all possibilities at the beginning of the project.
  • Allows continuous integration: Building incrementally is a good step in the right direction but if the incremental components are not continuously integrated to make a whole, the development team wouldn’t achieve all the expected benefits. As such, the team would be building many items that could fit into a bigger whole without making sure all the pieces actually fit together. By continuously integrating the components, the development team ensures that throughout the development cycle, an integrated “application” can be tested and possibly demonstrated to the business community.
  • Allows test-driven development: Using TDD as a development practice forces the development team to prepare their test cases before starting their development. In addition, every time news conditions to be tested need to be integrated, the tests are documented before the code is written and then executed once the components become available.

In turn, these 3 elements lead to 3 clear outcomes:

  • Improved software quality: Preparing test cases before starting the development, enhancing the test cases with new possibilities as they occur, and continuously executing the test cases on the incremental components as they continuously get integrated within the global build improve the overall quality of the software being developed.
  • Reduced overall cost of the project: By spending less time on overhead activities and delivering better quality software, the overall cost of the development project is greatly decreased.
  • Increased ROI of the project: Combining the impact of the various outcomes increases the overall ROI of the project.

An iterative and incremental approach for BI projects

The un-impressive track record in the development of BI applications led us to conceive an improved approach to reduce the initial investment required to launch a BI project in order to deliver key performance indicators (KPIs) much more quickly and at lower cost.

The incremental development approach we came up with is aligned with the business priorities while allowing business users to further develop the knowledge of their business and better define their expectations.

As opposed to the traditional waterfall approach to building a BI application which is: plan the project, define the requirements, prepare the architecture, develop, test, move to production in a sequential manner, we propose to use an iterative approach during which we build layers of the entire data warehouse through iterations.

For example, instead of spending time in defining in a high level requirements, work with the end users to understand their business and the indicators they use to successfully manage it. Using an iterative approach, the objective of our process is to work backward starting with the required performance indicators back to the source systems.

An Iterative and Incremental Approach for BI Projects

An Iterative and Incremental Approach for BI Projects

As can be seen in the diagram, the intent is not to divide the project into phases where data is extracted from the source systems first, then move to a centralized repository and so forth. Instead of the traditional approach, we use a layer approach where each indicator is developed from source system to the presentation layer.

Although our approach requires a substantial paradigm shift for most people having experienced a traditional waterfall approach for BI
projects, the notions we propose are simple, logical and straightforward. This approach relies on AGILE principles and
more specifically on SCRUM practices that have been demonstrated to be successful in other types of initiatives. Our objective is to present them in a very concise manner in this post.

Our proposed approach to BI development can be summarized by the following principles:

  • Segment the development of components into small iterations of 3 to 4 weeks and frequently deliver key performance indicators at the end of each iterations;
  • Repeat the iterative process and use increments to add components to the BI architecture and infrastructure;
  • Prioritize the development of the key performance indicators that will deliver the highest value to the organization first;
  • Using a top-down approach going from aggregation to granularity, develop each KPI from end-to-end or from source systems to presentation layer;
  • Avoid “analysis-paralysis” forces by using a “good enough” approach in documentation, architecture, modeling, project management
    and other overhead activities;
  • Learn from each of the iteration and adapt the process appropriately to increase efficiency.

Our approach promotes a process that encourages frequent inspection and adaptation, a set of engineering best practices that allow for
rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals. With this
approach, the project delivers in small increments with minimal planning, rather than long-term planning.