Why Agile Velocity is the Most Dangerous Metric

Agile velocity is a useful tool for sprint planning. But when dev leaders use velocity as a productivity metric, it has dangerous consequences.

Agile velocity is arguably the most popular software development metric in the world. When used for an agile team's sprint capacity planning, it's a powerful metric. And there are two things we know about power: 1) It comes with great responsibility; and 2) It corrupts. 

When velocity is used in agile teams for anything other than capacity planning, velocity becomes the most dangerous agile metric for software development organizations. Unfortunately, every day velocity is abused by executives, engineering leaders, product leaders, and even developers. 

What is Agile Velocity?

Agile velocity measures the amount of work a single team completes during a software development iteration or sprint. It represents the number of story points completed over time and can be visualized as the slope in a classic burndown chart.

My Personal History with Story Points & Agile Velocity

When I was promoted, I was in over my head. But I was determined to do a great job. I looked for every opportunity to help my development team improve. Data. Metrics. Process. Culture.

Yet, compared to other departments like sales, marketing, and technical support, there didn’t seem to be a lot of established metrics engineering leaders used to measure team-based performance and improvement. This is ironic because we measure so many things

Then I found velocity, and we started using it for capacity planning. At some point in a management meeting, I asked, “Should we start reviewing velocity every week in this meeting?” Tim Wall, a vet on our team who I really respected, gave me the following advice, “Never use velocity to measure performance and never share velocity outside of individual teams.” 

Thankfully, I listened to Tim. 

The 3 Dangers To Avoid Around Agile Velocity

There are three main types of abuses I've seen of agile velocity. Consider this a hero's guide to help the software development community stay safe, avoid traps, and use velocity for good, not evil. 

Danger No. 1. Using Velocity to Compare Teams

Imagine a scenario where all of your teams are using agile scrum and calculating velocity on an iteration-by-iteration basis.

You have a gut intuition that some of your teams are performing better, and some of your teams are performing worse. You get an itch to see which teams are completing the most story points per iteration and which teams are completing the least story points per iteration.

Then you start thinking about it in terms of who has the best velocity? Maybe you even show and compare velocity in your management meeting or scrum of scrums. 

What Can Go Wrong?

• Story-point weights are subjective to each individual team. Therefore, story-point weights are easy to manipulate and developers have an incentive to inflate their points.

• Individual teams and developers can easily game the system, and if some team members are inflating their story points, other developers will figure this out.

From here you're only a stone's throw away from broken trust, frustration, and culture damage. Bad intentions aren't even required for this to happen. It's natural human behavior.

So, as Tim told me, agile velocity is sacred and should be used within individual scrum teams only. With that in mind, here is a tip to help navigate each situation. 

Tip: Use Alternative Metrics to Preserve Culture

There's nothing wrong with looking at comparative performance data. Without benchmarks, though, you won't be able to invest your time helping your people and teams that need it most.

Since you shouldn't use velocity, what metrics should you use? It depends on the business outcome you're trying to achieve.

• If speed to value is your main goal, consider cycle time.

• If predictability is your main goal, look at iteration churn. 

• If quality is your priority, measure change failure rate and mean time to restore.

A related question to ask yourself is, "What kind of culture do I want to create?

In fact, when I interviewed Kathryn Koehler, Director of Prod. Engineering at Netflix, recently she reminded me that comparisons lead to unhappiness.

I've seen this, too. If you want to create an environment where it's all about the team, consider throwing out metrics like individual code changes and commits -- just measure team-based metrics. 

Some of us fell into the trap of using agile velocity in the wrong way because it was there and we didn't have other metrics to use instead. That's not the case anymore. 

Once you know what kind of team culture you want to create and what business outcomes you're trying to support, there are a number of engineering metrics you can use to measure your progress without damaging trust with your developers. 

---

I created LinearB to help engineering and product leaders use metrics without damaging culture. Want to start using alternative metrics to velocity in your agile team? Get started with LinearB free today! You'll get proven DORA metrics, like cycle time, change failure rate, mean time to restore, and deployment frequency.

---

Danger No. 2. Using Velocity as a Performance Metric

It's no secret; executives love data. Especially performance-related metrics.

One of the biggest issues that we face as engineering leaders is the lack of standardized metrics describing the health and performance of our teams. Unlike all other major departments (sales, marketing, finance), we as a community lack the basics when it comes to data-driven team performance. 

Sales has revenue, pipeline, and sales-cycle length. Marketing has MQLs, SQLs, and cost of customer acquisition. HR has positive and negative employee engagement, as well as retention rates.

CEOs and business leaders get these metrics, and they know what good and bad look like. They know the leading indicators for success. They know what levers they can pull when those department leaders need help. 

Engineering is often reduced to, "Are we on track to deliver XYZ feature by the deadline?"

Their intentions are good, and they want to understand, but most CEOs don't have an engineering background. Therefore they don't have the tools they need to understand engineering in the same way. 

Then all of sudden, your CEO catches wind that you're measuring sprint velocity for your agile teams. 

You say, "Each team uses it to estimate how much work they can get done in a sprint."

They hear, "We can use it to measure dev org output over time and rank team performance." 

They say, "Let's start presenting velocity in every executive meeting and at every all-hands!"

What you should think is, "It's a trap!" 

At this point in the conversation, our spidey senses should be going crazy. Velocity at the executive table? Do not proceed. 

Check out this interview with Ben Matthews, Director of Engineering at Stack Overflow, where he emphasizes that velocity should be one of many data points.

Tip: Use Cycle Time to Demonstrate Productivity

This is your time to gracefully push back. Ask questions to understand what exactly your executive team is looking for.

Is it team-performance data? Are they worried about deadline predictability? Are they wondering about the impact of the new hires they invested in?

But start with, "Yes, we can absolutely be data-driven." A former boss told me to start with yes, then get to the outcome I want. That concept has served me well over the years, especially with CEOs.

You know that it will damage culture and encourage sandbagging. But consider keeping all of that to yourself and just explain why agile velocity is not accurate when used for anything other than an individual team's sprint capacity planning.

Present alternative metrics. Cycle time is a great one because execs already understand sales cycle time. Software development cycle time is the equivalent for engineering teams. 

Danger No. 3. Using Velocity in Agile to Predict Project Delivery Dates

Imagine a project manager who wants to estimate when a project is going to be delivered. Easy, right? We know the team's velocity over the past few iterations. All we need to do is break the epic down into user stories, assign story points, and do the math to see how long it will take the team to deliver.

Unfortunately, using velocity to estimate the delivery of a substantial long-term project spanning numerous sprints does not work. I've seen it attempted many times, and I've never seen it succeed.

At best, with good intentions, a lot of false expectations are created. At worst, it is a pressure tactic to pin a delivery date onto the engineering team. 

Why This Never Works

Capacity planning only has value/accuracy one sprint at a time. It takes many iterations for a team to really understand its sprint capacity.

Estimating the amount of work a story takes increases in accuracy as the project progresses. Mapping out stories too many iterations in advance doesn't work. Too many changes.

There is inherent pressure to hit a date, and estimates get translated into commitments through a game of telephone with business stakeholders. Teams start bending story point values to respond to the pressure. 

The outcome that I have seen most often is that it causes pressure on developers to smudge (inflate) the actual amount of story points per story. Once that starts happening, a vicious cycle begins. 

Tip: Use A Project Delivery Tracker

Predicting delivery dates are the golden problem in software engineering. It is always a combination of art and science -- rarely accurate. Using velocity and story points to map out a long-term estimation will not only come out inaccurate, but it also puts the wrong pressure on the estimating developers.

LinearB’s Project Delivery Tracker is a business alignment dashboard that helps you visualize the investment being made by issue type, the progress of projects, and sprint planning accuracy. 

It can include a board, an epic, a label, a custom field value, or any combination of these. Book a demo today to see how it works! 

This article was written for LinearB by Dan Lines