How to determine the value of KPIs?
This question is frequently asked but the answer isn’t so obvious. In many (if not most) BI projects much effort is spent in calculating the cost of the overall project and sometimes more specifically the cost associated with specific KPIs but a lot less time is spent in determining the value or benefit of the required information.
It is frequently recommended to calculate the ROI of your BI project in order to properly evaluate if it can be considered a success (or not). Unfortunately, the ROI calculation has to factor in both side of the equation – costs and benefits.
Since business requirements always exceed capacity, organizations need to implement mechanisms to properly prioritize their product backlog. Unfortunately, the approach where “the business user who screams the loudest wins” doesn’t serve the organization. Despite the fact that this approach seems to be used more often than not, such prioritization process helps promote departmental agendas as opposed to an overall corporate perspective.
As such, determining the value of the required KPIs forces the organization to get into an alignment exercise and creates a great opportunity for dialog. It is critical to keep in mind that the benefits of the assessment exercise is not on determining the right value but to agree on an acceptable quantitative value.
I suggest 2 high levels approaches to help determine the value of the KPIs.
- Determine the real value: with this approach, the team tries to determine the actual incremental revenue or cost savings the additional KPI will bring to the organization. When using this approach, it is critical to spend enough efforts to determine a good-enough estimates as opposed to determining the exact dollars and cents.
- Estimate the relative value: this approach works well in the context where people can compare various KPIs to each others as opposed to evaluating them in an absolute perspective. With this approach, the team can use pre-allocated amounts of points or virtual money to rate/vote on the various KPIs allowing the group to determine which KPIs receive the most votes. Although this approach doesn’t help with ROI calculation, it does help move the process forward by helping the team select and agree on the priorities which will in turn help the project team.
Whatever approach is used, the objective remains to agree on common (corporate) priorities as opposed to departmental preferences while determining the value of the KPIs.
PS. I had completed this post prior to attending the panel presentation (Strategies for Business Alignment) at the TDWI BI Executive Summit this morning where David Hsiao (Corporate Quality Metrics and Benchmarking at Cisco) mentioned that prioritizing requirements is much like comparing apples and oranges since most requirements do not have the same value for the organization.
Although he didn’t explain which method was being used at Cisco, he mentioned the fact that each stakeholder gets “requirements” equivalent to the percentage of total budget they fund. In my opinion, this represents a good preliminary step to slice the total pie prior to agreeing on the value of the KPIs.