Listen to the article

Continuous improvement is a vital aspect of any efficient organization. Creating a truly great product requires a team to repeat what works well, and improve what doesn’t, from various aspects of the organization.

Due to a lack of understanding of how important measurement is, many teams fail to optimize for or measure efficiency. These teams do not realize that what they do or don’t accomplish in their day-to-day tasks has a direct impact on the success of their product.

To mitigate efficiency plateau, teams should determine the agile metrics they have to measure, which are data points that demonstrate how a team is performing in their various organizational roles, such as business, product, engineering, and support. By measuring an organization’s agile metrics, teams can set expectations while producing more work with fewer defects.

Why agile metrics matter

In the era of digital transformation when companies are implementing new systems and technologies, there’s a tendency to cling tightly to the past. While tradition should be respected in many situations, holding on to inefficient and immeasurable processes can put an entire organization at risk of failing. 

Creating a system to measure agile metrics addresses this issue by giving teams a way to see a visual representation of how well they are performing, allowing them to work toward continuously improving their output. This continuous improvement not only encourages teams to practice good habits and proper communication as a whole, but it promotes an environment of open dialogue, feedback, and fresh ideas. It’s important to note that tools alone will not improve a team, and that the metrics are simply a reflection of how a team works. Every team should have an agile coach to help interpret the metric values and take action. An example of a methodology supporting this is Kanban Maturity Model.

Around a year and a half ago, we at SUMO Heavy created a continuous improvement dashboard for one of our clients that is integrated with their project management system, and now every agile team is using it to discuss their areas of improvement on a regular cadence. It started out as a small side project, and grew into a tool that is used in monthly meetings with the Chief Innovation Officer. This dashboard allows a story to be told from week to week that helps agile teams answer questions. To build on that, teams have also begun Kanban Maturity Model sessions with their assigned agile coach to improve their metrics as well as their overall process.

Agile metrics matter because they can bring an organization together. It helps move past the days of the business bypassing product to micromanage engineering teams, or design teams not having focus. Everyone needs to be present, to strike a balance across the entire company.

Know which metrics to measure

Once the need for continuous improvement has been established, teams can define the metrics they need to measure. There are industry accepted metrics to track across business, product, engineering, and support, which all affect one another. For example, if story throughput (an engineering metric) is low, then we can likely tell a two part story:

  1. The product team made stories either too big or with too little detail
  2. There was no epic refinement with product and engineering after writing the stories prior to engineering starting to work on them

A team’s agile metrics will vary depending on the business, industry, and agile methodology it employs. However, there are a few common metrics most organizations should measure, such as lead time, cycle time, and throughput. Organizations should be mindful when they implement metrics, as it’s not a race to game the system to increase the numbers – the metrics are just a representation of how a team is performing as a whole.

Show your work

When a team first begins measuring agile metrics and working to improve internally, the entire process can seem overwhelming. To overcome this dilemma, organizations should carefully plan the first few metrics that they want to measure and present on a continuous improvement dashboard. In our experience, our clients welcome the opportunity to measure their performance because it helps the business understand one of the most difficult aspects of product – estimation. For many teams, the most difficult question to answer accurately is, “How long will this feature take to build?” This is one area where an agile dashboard is incredibly effective. For example, if over the past 4 weeks, an epic took 12 days on average to complete for a given team, they can expect that another epic will take the same amount of time, with some more efficiency factored in.

Teams should also ensure that their agile project management systems will support either pushing that data to another source or have another application pull the data. The data would then have calculations run against it to allow summarized visualizations to be shown at each level of the business, and to track changes over time. Each metric should touch on these three items:

Direction. Which direction is “good”—up or down?

Goal. What threshold is each team trying to be within?

Benchmark. The way we measure our benchmark is by looking at a metric’s value for the most recent five weeks, then checking if the most recent value is inside or outside one standard deviation of that. This allows us to know whether we are normal, improving, or worsening. 

To encourage teams to communicate during the transition into measuring efficiency, organizations should hold monthly meetings that include the leadership and at least one key member from each role of the team. A representative from the business should be able to speak to high-level achievements related to the roadmap, product should be able to speak to features, and engineering / support should speak to day-to-day. Each team should only take 5 – 10 minutes to get through their slides. The important thing here is to tell a story, and not to focus on individual numbers. Together, team members can have an open discussion about how teams are performing and use the time to set expectations on how they are being led to improve their output.

Improvement is an ongoing process

It’s important to understand that change does not happen overnight. The road to continuous improvement could be months or even years long, and even then, the journey is never really over. Teams should continuously measure their agile metrics, revisit them regularly, and use them as talking points for conversations about how to achieve their goals together.