Why We Started Measuring Customer Lead Time

On the way to “done”, there are many things to consider. After all, that’s what staying agile is all about: keeping things flexible enough to continuously improve the delivery process and taking decisive action when things call for adjustments, be it little or major.

It’s far from just pushing things to the finish line at all costs.

That’s when agile metrics come in. Through metrics, both we and our partners are able to see the bigger picture of the development process. They shine a light on the projects’ progress as we go along and let us identify what’s working and what needs further improvement.

There are some key pieces to this puzzle. Customer lead time – among some other agile metrics like velocity, sprint burndown, or cumulative flow diagram – is one of the most essential.

What is customer lead time and why is it so important?

Customer lead time tracks each work item’s lifecycle: the amount of time from the moment it “starts” (enters the backlog) up to the moment it “finishes” (is released and ready to validate the outcomes).

You can probably see why it’s a crucial piece of information for everyone involved in a project: customer lead time basically measures the entire agile process. It can also be applied on a smaller scale, e.g. to track the implementation of a single feature, measure the length of code review, or even duration of a single task.

All that makes customer lead time a highly instrumental metric from a partner perspective, too (“customer” in the name is there for a reason, after all). This metric gives them more predictability in terms of estimating both the project time and budget and helps with planning team augmentation projects, which rely heavily on aligning augmented staff with internal teams and processes.

Putting it in a nutshell, measuring lead time gives us a piece of crucial data that we can use to make the development pipeline more efficient and more transparent. 

What is lead time vs delivery time?

Lead time is the total time from when a request is made until it's completed. Delivery time is the time from when the product is ready to ship until it reaches the customer.

Now that we’ve painted a pretty picture, it’s time to dive a little deeper and mess it up just a little a bit.

In search of the boost (and minimizing frustration)

We wanted to collect all the necessary data so we could then work together with our partners to optimize customer lead time for every product we’re engaged in.

But collecting raw data is one thing – using it in the right way is a whole different story. In order to do that, we first needed to ask ourselves some questions about our development process, such as:

• Do our teams break down the stories into small enough increments?
  
  
• Are code reviews picked up instantly, or is there a significant wait time?
  
  
• Are there any problems with our pipeline tooling?
  
  
• How long is our quality assurance cycle time? Are there any bottlenecks in the process? 
  
  
• What are the most frustrating roadblocks in the development process?

The last question is especially important. After all, making a process more efficient means making the work easier for the developers and designers. That’s exactly why we need to identify and deal with everything that stops them or slows them down.

The thing is, with time, teams can get used to even the most frustrating roadblocks, up to the point where they start to see them simply as part of the process. And when you want to reduce waste and allow developers and designers to be more flexible and creative, that’s not a particularly good thing – to put it mildly.

Customer is in the name

We’ve mentioned the role of customer lead time in building a transparent and predictable collaboration with our partners. We want to measure customer lead time on every project we’re currently working on and share that information openly in the future.

But when it comes to the product’s timeline, every potential partner may and probably will have different expectations. That’s only natural. 

By measuring lead time, we aim to meet those expectations more effectively. With this data, we’re able to come up with more accurate estimates and meet the set deadlines more consistently and easily. We're not the biggest fans of estimates in the first place – but, love ‘em or hate ‘em, they’re here to stay. 

We also want to decrease our partners’ carrying costs and give them more flexibility during rapid shifts in development and optimizing lead time seems like the best way to achieve it.

Getting ready to calculate & optimize customer lead time

We started with calculating lead time on internal projects: our own time tracking tool called Anthill and the development of Brainhub’s website. 

In the process, we used a selection of Jira plugins, external tools, and GitHub projects. Read on to get the full, detailed list.

After a few months, once we were done with the calculations and took some steps towards optimizing lead time on both projects, we were able to identify practices that affect it, both in a good and bad way.

Customer lead time optimization techniques:

• constant monitoring of the upstream to minimize time spent in design review;
  
  
• preventing longer breaks in the development and/or making sure that all started Product Backlog Items (PBIs) are finished before the break in order to refine the backlog (we closed PBIs that were no longer valid);
  
  
• improving the transition of different work items between design review, QA, and deployment teams and making sure that the waiting time is as short as possible; 
  
  
• frequently reviewing the product’s scope (the vision, goals, user groups, and their needs) and roadmap to ensure that backlog is efficiently refined at all times;
  
  
• identifying the project’s external dependencies, e.g. legal factors and infrastructure.

Things that increase customer lead time:

• limited availability of key stakeholders and decision-makers;
  
  
• external design approval process;
  
  
• insufficient backlog refinements;
  
  
• external QA process;
  
  
• delayed delivery of the testing infrastructure;
  
  
• release roadblocks:
  

• putting too much effort into releasing an MVP (e.g. increasing the scope, spending too much time on tweaking and polishing the MVP);
  
  
• disrupted or unspecified roles and responsibilities (ownership issues);
  
  
• other external factors, e.g. legal requirements (GDPR, audits, etc.);
  

• PBIs issues:
  

• delayed status updates and multiple PBIs in progress at the same time;
  
  
• unspecified dependencies (as a result of insufficient business or technical analysis);
  
  
• unspecified edge cases (as a result of insufficient analysis or incomplete UX designs);
  
  
• PBIs that are too large and not broken down into smaller items.

Tools to measure customer lead time

There are many useful tools you can use to measure customer lead time and start optimizing work on your projects.

We created a list of selected Jira plugins and external GitHub projects, along with prices and short descriptions, so you can choose for yourself.

You can download it anytime you like – just enter your email in the form below, and we'll send you a link right away.

Pro tips & key takeaways

1. Customer lead time is a key metric from the client’s perspective.
   
   It makes for more predictable cooperation, more precise estimations, and better planning. And when it comes to aligning different processes with external teams, these three elements are crucial.
   
   
2. Calculating customer lead time is just a first step.
   
   You need to know why you’re doing this and what parts of the development process you’re aiming to improve – otherwise, you’ll end up with raw data and nothing else.
   
   
3. Some essential parts of the development process will increase customer lead time. You shouldn’t optimize them at all costs.
   
   Don’t jump to conclusions too hastily, after seeing just a few first few metrics. Take some time to get a more realistic perspective and measure CLT at least for a few sprints. You can also calculate the median of your customer lead time.
   
   
4. There are many non-numeric factors that will impact your customer lead time.
   
   As much as we wanted it the other way, the development process is not fully predictable. Some external aspects will always affect it one way or the other. 

‍