We didn’t know how long we’d have before the approval arrived so we started with a bit of training on technical practices and then started working all together (ensemble work) on reviewing and improving their existing code. We refactored the code to make it cleaner and more maintainable, and improved the automated testing. We optimized their build pipeline and got them ready for continuous delivery (CD), something they hadn’t previously considered.

When the legal approval came along a few weeks later, this codebase was already in far better shape than when we’d arrived. The code was cleaner, there were fewer bugs, and all the team members had a better shared understanding of everything.

Additionally, this new service was released on a working CD pipeline, releasing to production on every commit.

How was this all possible? The most significant reason is that for the first time, they had no delivery pressure. Nothing was splitting their attention twenty different ways. Nobody was insisting that they work on multiple things at once, or watching to make sure they were busy.

Their only goal was improving what they already had. They had time to focus on the quality of the code. They had time to optimize their pipeline. They had time to learn new skills that they didn’t already have.

And then they delivered something amazing.

The most frequent objection that I get to new improvement activities is “delivery pressure”. We’re too busy to do that now. The problem, of course, is that no matter what we might tell ourselves, that pressure is never going away. We won’t have time to improve unless we make time.

If we don’t improve then we’ll continue to deliver at the same rate that we were delivering before. If we do improve, we’re likely to deliver much faster than that. There is a cost to not improving, and it’s one we often disregard.

If that team at the beginning hadn’t taken a few weeks to fix their system then they’d still be responding to quality issues the way they always had. They’d still be manually deploying the way they always had. They’d be just as busy, and as ineffective as they always had been.

They were lucky. A legal complication had given them an amazing opportunity to improve and they did. Other teams aren’t as fortunate and don’t have that slack time handed to them.

If we want to improve, we have to make the time for that, regardless of how much delivery pressure we might feel.

See also:

• Tom DeMarco’s classic book Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency
• Playing the long game

Are you really going to look at fifty of those on a regular basis? You might for the first few days or even weeks, and then you won’t anymore.

I had an executive once ask me to email a freshly generated 250 page report every hour through the day to his cell phone, so he could keep an eye on things. It took far too long to convince him that this was not a good use of anyone’s time and that in any case, it was giving him entirely the wrong level of detail.

At the opposite end, I had one client that had automated six key metrics to be sent to all senior leaders every 45 minutes. Six simple numbers from which they could tell at a glance if everything was running smoothly or if they needed to refocus their attention.

Another client had a large wall of metrics and analytics on monitors beside the exit. Anyone leaving the building could see at a glance if everything was ok or if there was something needing their attention before they left.

These are all examples of realtime, or near realtime, metrics but not everything has to be that fast. If we’re looking at the speed at which a software development team is completing features, looking at metrics even once a week might be too often.

If we’re looking at value delivery and whether we built the right things for our customers, even quarterly might be too often.

So we need to consider not only what we’re measuring but also how frequently we review that data.

At a high level, there are three main groupings of metrics that I normally want to start with.

Flow metrics tell us if the work is moving through the system. I usually start here, partially because they’re the easiest to measure, and partially because they will uncover a huge number of the problems with little effort. If the work isn’t getting done then really nothing else matters.

Value metrics tell us if we’re spending our money wisely. Are we doing the right work? Are we doing it at the right time? Is it of the right quality?

Sustainability metrics tell us if we can continue at this rate indefinitely or if we’re just running towards a cliff and going to crash.

I tend to look at flow metrics regularly as they give me immediate feedback on the health of the system. Value and sustainability tend to be longer term views into the system. They take longer to collect and are reviewed less often.

What do I almost never care about? How busy we are. Busyness is not a proxy for either effectiveness or efficiency, and I’ve written about that before so I’ll leave it today.

So I start with “is the work moving well?”, followed by “is the right work?”, and finally “is what we’re doing sustainable?”

For each of these, I want to find a small set of metrics that give me a glimpse into the system. I don’t want too many different measurements as that becomes overwhelming and I don’t want to have too few because then they’re easily gamed.

Not sure what you should be measuring in your environment? Let’s talk.

The four flow metrics that we typically start with are Throughput, Cycle Time, WIP, and Work Item Aging, and all of them are measured relative to the start and stop points. Without clarity on those two points, none of the metrics will be helpful.

So where do we really want to measure those?

At a high level, there are two parts to the start point (Ready). Someone representing the business has agreed to spend money on this work, and the team understands what they’re being asked to do. It’s worth noting that the team does not need to understand how they’re going to do the work yet. Figuring out the how is part of doing the work and happens after the clock has started.

Our stop point is similarly simple, we aren’t coming back to this work. What does that mean? If it hasn’t been tested yet then it’s highly likely that we are coming back to it because we don’t know if it works yet. We stop the clock when we have a reasonable expectation that we are not coming back to this work.

Each team will likely have more specifics that they want to capture, to provide a checklist of the things required for each of those two points. Perhaps the code must be checked in, reviewed, and merged before we can say we’re doing. All of this is towards satisfying the high level point of we aren’t coming back to this.

A common mistake made with both of these points is that individuals think that the points are measuring them personally, when in fact the measurements are about the work. We don’t care that a developer finished coding, the work isn’t done until other people have done their work as well.

This also holds true for the start point. I’ve seen lots of teams count the work as started when a developer starts coding, yet the work usually started well before that. Has the team not already had discussions about this work during a refinement/planning meeting? The moment we committed to doing the work is when it started, not the time we touched some code.

Many people want to defer the start point as long as possible to make the numbers look good. What we really want to do is make the reality visible so that we can start improving it. If we held a refinement meeting a month ago and brought the work to ready at that time, then the work has been started for a month, whether or not a developer has touched it since.

We don’t like that because it makes the numbers look bad, but instead we should be happy that we’ve just identified problems in the workflow. Why did we refine the work so early? All we did is build up an inventory of tickets that we aren’t working on, and inventory is one of the classic wastes in Lean.

If the work has been fully refined, the clock should have started. If we’re not coming back to it, the clock should have stopped.

Always remember that we’re not collecting metrics for the sake of being busy. We’re doing it in order to make better data-informed decisions. Better decisions require a better understanding of the system, and that needs accurate measurements.

See also:

• If you’re trying to get flow metrics out of Jira, take a look at my own JiraMetrics.
• If you just want to learn more about these four metrics then check out my online course Flow Metrics Basics for Agile Teams

Frequently when I talk about that, everyone nods their heads and then immediately makes a target out of the measures they’re looking at. It’s harder than it sounds, to get this one right.

Let’s look at an example. Let’s say that’s we’ve decided we want better code coverage for our tests. We want the confidence of being able to make a change to the code and knowing that if we made a mistake, the tests will catch that. Having a safety net like that, has huge value to the organization.

So we might measure code coverage as a percentage. How much of the production code is executed during a test run?

That’s a reasonable measure, that we can then use to make some decisions. Not as many good decisions as people normally assume, but still a good measure.

The gotcha is that many people will then immediately turn it into a target. If we’re currently at 60% coverage then someone might decide that we need to increase that to 80%.

As a measure, it was great. As a target, it’s horrible as it now encourages people to add tests that increase the test coverage numbers, whether or not those tests improve the original goal of having a better safety net.

You might be thinking “nobody would game those numbers in my organization”. They will and they do. Code coverage in particular, is gamed everywhere.

Lou Gertsner, former CEO of IBM, has famously said: “People don’t do what you expect, they do what you inspect.”

What might it look like if instead of making code coverage a target, we’d left it as a measure?

We might have hypothesized that if we adopted Test-Driven Development, that might improve our safety net. So we could try that for a period of time and then looked at code coverage as an indicator of whether we were improving or not. Not a target, but an indication of the direction that we were moving in.

Side note: I want to point out that while it’s phrased as being Scrum specific, this is equally applicable to a kanban team.

My immediate answer was yes, but it requires some context to understand why.

The daily scrum is without a doubt, the simplest of all the scrum meetings, and yet it’s one that’s frequently done very poorly. The question was specifically about a “mature” scrum team though, so let’s assume that they know how to do this meeting well.

What would this meeting look like if it didn’t need facilitation?

People would show up at the appointed time, they’d look at the board and decide which was the most important item. Then someone who knew about that item would talk about it and when they were finished, the team would move on to the next most important item and talk about that. When all the items had been discussed, the meeting would disperse. Nobody would have needed to be prompted. Nobody would have needed to be reminded to share the screen. Nobody would have talked too long, or drifted off topic. Any one of those things would have required facilitation to get back on track, and we’re talking about the case where a facilitator isn’t needed.

Is this nirvana even possible? Sure, I’ve seen teams do this for a couple of days, where they stay really focused and keep each other accountable. Then they get tired or distracted and start to let things slide. The key is that although it’s possible for a short time, it doesn’t last.

That’s why we need facilitation. There needs to be someone, or a couple of someones, to keep everyone on track, to move things along when they start to stall.

Does this imply that we need someone with the title of facilitator? No. When there is no designated leader, a natural leader will emerge from the group, and that person will either start facilitating themselves or will ask someone else to do it. This is normal team dynamics.

Interestingly, even when there is a designated leader, a natural leader will often emerge anyway, and sometimes that person does a better job than the person with the official title.

So back to “does a mature team need a facilitator?”, yes, although that person doesn’t have to have the title of facilitator. If we don’t have any facilitation at all, the meeting will very quickly lose any effectiveness that it once had.

A related question is “does it always have to be the same facilitator?” and that’s a resounding no. In fact, I recommend rotating facilitators, at least for the daily scrum, through all of that meetings participants. It’s been my experience that when everyone has a turn being that facilitator, the team takes the entire meeting far more seriously, and consequently does a much better job of it.

Let’s start by defining those two terms:

• A deterministic answer is precise: “It will rain at 3:05pm today.”
• A probabilistic answer contains some imprecision, despite being more correct: “There is a 80% chance of rain after 3:00pm today.”

Or in the case of the typical projects we’re working on:

• Deterministic answer: “We’ll be done on March 1.”
• Probabilistic answer: “There is an 85% chance that we’ll be done on or before March 1.”

The deterministic answer is far more satisfying to our brains, despite being both less accurate and often misleading. Our brains are optimized for energy conservation and want to use as little energy as possible for any decision. The precision of the deterministic answer is attractive as we can just use that without doing any processing of the answer.

Our brains have many shortcuts built-in to optimize for that energy usage, and these are generically called cognitive biases. We tend to lean on these even when we’re in a calm state and then when we’re under pressure, or feeling unsafe, we use them even more.

Precision Bias is when we believe that something more precise is automatically more accurate than the less precise answer. Stating that we will deliver on March 1 appears more reliable. It feels like a better answer, even when objectively, it’s not.

An 85% chance of meeting a date means that there’s a 15% chance that you won’t. The cognitive bias of Loss Aversion tells us that we’re more likely to weigh potential losses more heavily than equivalent gains, leading to biased decisions. We want certainty and even a 15% chance of failing will be given disproportionate focus. A deterministic answer does not trigger loss aversion.

Probabilistic forecasts make variability visible, and Illusion of Control wants us to pretend that we are in full control. In an environment of low psychological safety, an uncertain answer can be interpreted as weakness and therefore dangerous.

An interesting variation on Illusion of Control is that many managers would prefer to have a deterministic answer that they KNOW is wrong because feel that they still have control over that. “I know they won’t make the first scheduled date, but they might make the second.”

Probabilistic forecasts are inputs to decision-making, not decisions themselves. When leaders expect a single “right answer”, probabilistic data can feel like a refusal to decide, and that can backfire on the person providing the forecast. It’s often safer to provide a deterministic answer that’s wrong.

So am I suggesting that you should give up on probabilistic forecasting and provide deterministic answers instead? Not at all.

What I’m suggesting is that we should understand what’s getting in the way of us making effective decisions from accurate data. If we don’t understand our own biases, then they will continue to control us. In this case, not understanding will lead to poorer decisions.

We should also strive to increase safety across our organizations, for many reasons, but in this particular case so that we can make better decisions.

In the chart below, we can see the total WIP per day grouped by whether it’s active (blue), stalled (light orange) or blocked (darker orange).

Stalled in this chart means that there has been no activity on the ticket in five days. No comments, no status changes, no subtasks moving. Most likely we haven’t had the time to work on these because we’ve been so busy with the few items in blue.

Blocked means that we can’t work on this because of some external dependency. Perhaps we’re waiting for a signoff, or an environment to become ready.

What we can see with this team is that with a WIP of roughly 15, only a few items are actually being worked on, on any given day, and this is typical in my experience. The higher the WIP is in total, the more of those items are stalled, or blocked.

The team I mentioned last week, with 227 items in progress, was adding value to almost none of those tickets. Almost everything was stalled.

The bottom line is that total WIP only tells us a part of the story. It tells us what’s started but not what’s actually being worked on at any time.

When the WIP is low, we can be fairly confident that all the started items are actually moving along. When the WIP is high, it only tells us that we’re busy, and little else.

Busy is not the same as effective. In fact, busyness and effectiveness are often inversely related. As we become better at one, we become worse at the other.

Starting work items that then just become stalled because we don’t have time to work on them, is not an improvement. That just creates waste and decreases our effectiveness.

Chart from JiraMetrics

It depends on the composition of the team, both size and skills, as well as the nature of the work that flows through the team. It depends on dependencies we might reasonably expect.

It also depends on the larger system that the team resides within, although that’s a much larger topic than I want to address right now. That gets into Systems Thinking or Theory of Constraints, and we’ll discuss them another time.

What do we know about work in progress for a single team?

We know that as a general rule, lower WIP across the team will result in work getting done faster and with higher quality, compared to higher WIP across that same team.

We know that there are very rare exceptions to this where WIP that’s too low can actually make things worse. Many teams are afraid that by lowering WIP, they’ll be affected by this, and so they’re reluctant to lower that WIP. Let me assure you that your team almost certainly won’t have that problem. Not for a long time, anyway

We know that it’s possible, despite what many people think, to consistently have a WIP of 1 or 2 across the entire team (see ensemble / mob programming, and those teams tend to be exceptionally effective.

We know that having a WIP that is measured in multiples of the number of people, is almost always a disaster (ie 20 items in progress for a team of 5). The worst I’ve seen was a team of 10 people with 227 items in progress. They spent all day either context switching or arguing over what was important. Nothing got done.

We know that having a WIP that’s too high will invalidate most of the business prioritization that you’re doing.

So what should the WIP be for your team? If you’re asking the question then almost certainly, lower than what it is now. Lower it and see what results you get.

Then lower it again. And again.

See also: the “lowering the water level” metaphor from Toyota.

Looking back, I’m sure we’re all horrified at how inefficient this was. For the price of a few more version control licenses, we could have eliminated a significant amount of process waste. The one gatekeeper would have been able to do much more of his own work, and nobody would be sitting around waiting for that person to finish.

Yet, we see similar patterns with workflow systems (ie Jira) today. Although everyone might have a license, it’s common for only one person to ever update the tickets. During our daily meeting, people will provide updates and one person will then laboriously type them into the system. Just like with that version control, we’ve chosen to have one person be the bottleneck.

Even worse, when it’s all down to one person, they often feel pressured to add some comment like “still in progress”, even when it doesn’t add any value. So not only are we slower, but we’ve added all kinds of noise to the tickets.

Consider how much more efficient we’d be if everyone updated their own tickets and had done so before the meeting even started. Then we could spend our time together talking about the things that actually need our attention, rather than watching one person type, or drag tickets around.

A class of service reflects the fact that we behave differently for each. That we give more focus and attention to a high priority item than we do to a lower priority item.

The five Jira priorities that we get out of the box are:

1. Highest
2. High
3. Medium (the default)
4. Low
5. Lowest

Let’s assume that we use all five because realistically, if the option is there then people will use it.

It’s reasonable to assume that most items will be the default (Medium) and will stay there. Given that, what’s the likelihood that we’ll ever do Low or Lowest priority items? If we’re honest with ourselves, it’s unlikely that we’ll ever run out of Medium items so the Low and Lowest items should starve for attention.

That begs the question of why we ever allowed a Low or Lowest priority items to even move onto the board in the first place. Why would we ever start it, if we know we’ll never give it any attention? Clearly we shouldn’t so those two are redundant and could be safely removed.

Yet, when we look at data from actual teams, we see that Low and Lowest items do get started, even when they haven’t run out of Medium items. What does that tell us about how we use priorities? It means we largely ignore them; we treat them as hints, not rules.

In fact, if we graph cycletimes across priorities, we see that the more priorities are in use, the less difference they seem to make. When we only have two priorities, we can clearly see that the higher priority items get done faster than the lower priority ones. When we have five, it’s very difficult to see any correlation between priority and time to complete.

So what do I recommend instead? Have only two priorities: standard and expedited. If it’s a standard priority then treat all work first-in, first-out. Older items are more important than younger items. If it’s expedited then it’s more important than any standard item so do that first. If you have multiple expedited items then do them in first-in, first-out order as well.

Having only two levels reduces all the complexity, while still giving us a way to boost an item that needs it. The vast majority of items should be normal priority.

Of course, even having two priorities doesn’t solve all the problems. We’ll still have situations where what we say doesn’t match our actions. If an item is expedited then we should be jumping on it and doing it first, yet we routinely see things marked as expedited and then everyone working on other non-expedited work instead.

That’s like an ambulance on the highway putting its lights and sirens on and then stopping to let other people pass it. Behaviours need to match labels; if the lights are on, it should be moving before anyone else. If we mark an item as expedited, we need to do it before anything else.

I’ve recorded a video on what I look for there.

Once we get to the point that the meeting is effective, we’ll often hear complaints about how long it takes. I routinely tell teams that it can be done effectively in less than five minutes, and most teams are highly skeptical of that. At least they’re skeptical until I’ve facilitated a couple and shown them what’s possible.

Note that if all we focus on is speed then we run the risk of having a ineffective meeting that was fast, and that defeats the whole point. We must retain the effectiveness of the meeting, while continuing to keep it short.

The first step is to start the meeting on time. This may seem obvious but most of these meetings don’t start on time; instead we sit around waiting for people to arrive. If the team is actively chatting with each other then this might be a good use of social time to build relationships, but I find that a significant number of teams just sit there staring at each other until it starts, and that’s complete waste.

If the meeting is booked for 9:00 then we start at 9:00 regardless of who’s there. If people show up late, we do not recap for them. The research shows that not starting on time actually reduces both the expectations and effectiveness of that meeting, and I wrote about that recently.

Many teams rotate who will facilitate this meeting, and I find that to be an excellent practice. Where it becomes a problem is if we come into the meeting and nobody is sure whose turn it is. Make sure that this is clearly known before the meeting starts, or else we’re wasting time right at the beginning.

The intention of this daily meeting is to complete the work that has already started. All the conversations we’re having are focused on that one point. How do we get work done today? Maybe we’ll work together on the tickets or maybe we’ll split the work up and work individually. It’s during this meeting that we decide on our approach.

Any conversations that are not focused on getting work complete, are off topic, and while they may still be important to address while we’re all together, they go in the parking lot and we can come back to them at the end. Quite often these conversations don’t need everyone and so the relevant people can meet after the rest of the team has been allowed to leave.

We’re going to walk through every ticket on the board in priority order from most important to least. In general, items closer to done (further right on a typical board) are more important than items further away from being completed. If we do run out of time, this ensures that we’ve talked about all of the most important ones.

The priority sequence may be affected by classes of service such as expedited work being more important than standard. The key here is that each team will have their own understanding of priority and we walk the work in reverse order. Most important through to least important.

For each item, we need to talk about how we’re going to get it finished and that conversation will change depending on what state the work is in.

Each item on the board will be in one of three states.

1. It’s active, in which case we want to know what work is remaining.
2. It’s blocked, in which case we want to know what we’re doing to get it unblocked.
3. It’s stalled, which means that we just don’t have the capacity to work on it right now because we’re so busy with other items that are either active or blocked.

For either active or blocked, we want to talk about what is being done and what is remaining so that we can look for opportunities to help each other. What we don’t care about is a status update of what we’ve already done. We don’t care that someone was stuck in meetings all day, we do care that they’ve got a lot of work left and that if someone helped them, we could get it done faster.

For stalled items, we generally just skip over them. If we have a lot of stalled items or some items remain stalled for a long time then that’s certainly a problem we should address, but one item that was stalled for a day or two isn’t a big deal. Move on.

At this point, as we’re walking across the items on the board, I see a lot of teams clicking into the ticket automatically to bring up all the details, so we’re switching from a board view to a details view and back, over and over. If someone asks the facilitator to pull up the details then by all means, do that. If we’re doing it just out of habit then we’re wasting time. There should be enough information visible on the front of the ticket to be able to talk about it in most cases.

Once we’ve walked through the board, we may have picked up some “parking lot” items. If they require the whole team then we can discuss them now. If they only require a subset then we let everyone else go.

Then we stop the meeting. While this sounds like the simplest of all the steps, it’s one that so many teams struggle with. “But we booked 15 minutes and we’ve only used six. Surely we should keep talking.” No, when we’ve satisfied the intent of the meeting then we stop. “Thanks for your time, the meeting is over.”

I’ll leave you with two last tips that both work at an unconscious level to make us move faster.

1. You may have heard this meeting described as a stand-up, and the reason for that was that if we’re standing, we’re less comfortable and therefore we’re motivated to keep the meeting moving along. If we get too comfortable then the meeting starts to drag along.
2. When I’m actively trying to speed the meeting up, I make a point of visibly timing the meeting. If we’re in person, I might write the time on a whiteboard, and keep a running tally of the recent times. If we’re remote, I’ll call it out. “Meetings over, six minutes today” At an unconscious level, this causes us to move a bit faster. When we stop timing it, we’ll see it start to drift back the other way, getting longer and longer.

When I’ve done this with teams as large as twenty people, we can get this meeting under five minutes in a relatively short time. The only exceptions are teams that are massively overburdened with WIP, and they have much more serious problems.

Try it, and see for yourself.

Interestingly, when people arrive late, our expectations for the meeting drop, as well as the actual effectiveness. This study was specifically looking at 5 and 10 minute lateness, and only in the context of American participants. Other cultures or times may vary.

So if we want the best results from our meetings, we need to start them on time. If the meeting says 10:00, then start at 10:00.

The most common objection I get to this is that it leaves no downtime between meetings. If I need a quick bio-break or need to physically move between rooms, I have no time to do that.

So why aren’t we scheduling our meetings to start at 10:05? There is nothing that says we have to start on the hour. Or establish an agreement that all meetings end with at least five minutes before the next slot. Then our previous meeting would have ended at 9:55, and this wouldn’t be a problem. I know several companies that have configured their calendaring software to do this by default.

If we wish breaks between meetings, and I do recommend that, then explicitly schedule them. When we pack everything back to back then we’re actively planning to be late, and as we can see above, that’s not good.

The other common objection I hear is “we don’t want to get right into business, we want some social context first”. There are two responses to that. The first is to open the meeting a bit earlier to allow people who want that social time to do that. The second is to make the social part an official part of the meeting and start that at the allotted time.

Either way, it’s clear that if we want effective meetings we need to start by making a schedule and keeping to it.

If that’s true, then being in-person does have advantages in that…

• someone may notice them struggling and offer to help
• they may be more likely to ask for help if they can physically see people in the space around them.

Yet, throwing them to the wolves on day one is a horrible plan, if the goal is really to get them ramped up quickly.

It would be far more effective if we paired them up with someone experienced and had them work together until that person was up to speed. Interestingly, when we do that, it doesn’t matter if we’re in-person or on a video call.

If we really care about our new hires then perhaps we should focus on those practices that actually help them. Making them work in isolation is not that.

Related:

• An example where we truly supported the new hires and ended up with two of the best teams in the company.

The facilitators job is to build the container that allows all the right conversations to happen, not to give a monologue.

Certainly, you may have to ask questions to get people to provide the right information. You may have to interject if people are going off topic, or being long-winded in their comments. You may have to provide context to help people understand what’s expected in this meeting. The key is that all of these are points where you are managing the meeting itself, not the content in the meeting.

It’s the participants job to contribute to the actual content. They work within the container that you set, and they are the ones who talk about the actual work.

If you really want to participate, then ask someone else to facilitate.

This is also true for retrospectives, and many other meetings.

Why would I sign up for something like this? Because this is basic risk management. I routinely hike in bear country (albeit mostly black bears and not grizzlies) and to effectively manage the risks, I have to know how to prepare and how to react in a dangerous situation.

I can’t just trust that everything will be ok. I need to prepare for reasonable situations so that if something does go wrong, I can still safely get through it. I don’t expect to know absolutely everything, but I do expect to know the fundamentals of hiking safely.

How can I avoid situations where I’m putting myself in unnecessary danger? How will I call for help if I do get stuck?

This is risk management. I’m not eliminating every risk, I’m managing them through education and practice.

In all the discussions about “Vibe coding” these days, I’m seeing a complete absence of basic risk management, and that’s alarming.

There is story after story of sites that were built with GenAI tools that are lacking any of the security features that we would consider basic risk management. Some of these have already been hacked and others will be in the future.

People who don’t have the education or skills around risk management have suddenly been given tools that appear to do all the thinking for them, and yet they don’t. We need to do better.

Risk management requires regular education and practice and we can’t delegate that to a tool, no matter how convenient it might seem.

First of all, “quality” is a nebulous label that means many different things for different people, so when I hear that we’re establishing an OKR for quality, it’s unclear exactly what’s meant.

Jerry Weinberg defines quality as “value to some person”, which is still nebulous but puts us on a better path. We have to consider who we are doing this for and what things they value.

I think we can all agree if we’re talking about a product and it has defects then that’s not quality. If it doesn’t do what the customer needs it to do, that’s also not quality. It it introduces too much friction and makes it more difficult for the user to do what they need, again that’s not quality.

So if we’re making an OKR then we have to come up with some specific measurements (the KR’s or Key Results), and this is where having quality as an Objective is challenging. Very few of the things that make up quality can be measured in a quantitative way - most of them are qualitative.

We can certainly measure a count of defects, and we can measure the arrival and departure rates of defects but then we’re very quickly out of things we can easily measure numerically.

For software products, people will often want to include code measurements like code coverage or “days of technical debt” as reported by SonarQube, or other health metrics, but these miss the point. Quality is value to a person and when we’re talking about OKR’s, that person is going to be a customer, not a developer. They don’t care about code metrics, or about any of the internal details. They care about how the software solves the problems that they have.

Software quality is a real thing and we should also consider that. The point is that when we’re talking about OKR’s, we’re usually talking about management objectives, not development ones.

What we need here are qualitative measurements and that requires actually talking to our customers, and not just measuring some numbers. Is the product working well, with little friction? Does it help our customers do the things they’re trying to do? Do they find that it helps them or gets in the way?

I love the idea that we set quality as an objective. We just need to recognize that this is a non-trivial thing to measure and that the KR’s are going to take real effort to collect.

• The first is prior to the work starting, when we’re deciding what we want to work on. What problem are we trying to solve?
• The second is after the work has started, when we are actually implementing whatever solution was determined to solve the problem.

Note that when we are considering HOW we’re going to do the work, in most cases that work is already started.

That start point is often called the commitment point, as it’s here that we are making a firm commitment to actually do it. Prior to that, it was merely an option.

So how is prioritization different at each step?

Prioritization before starting

Prior to work starting, we need to prioritize by some form of business value. Perhaps we expect revenue to go up or expenses to go down. Perhaps we expect to sign up new customers or reduce customers leaving. Perhaps we’re lowering risk. The key is that there should be some anticipated business value from doing this thing.

Sometimes we prioritize this work by a sense of urgency, although if that urgency isn’t backed up by importance then we need to seriously question whether this is the right thing to work on. See the Eisenhower matrix for more on that.

Sometimes we also prioritize by size, putting something in just because it’s small, and not because it’s valuable. This should be strongly discouraged. The time that we spend working on that thing, no matter how small it is, is time taken away from something that’s actually valuable.

Prioritization after starting

Once we’ve started the work, our prioritization scheme changes. Now we want to prioritize by a defined policy, not by business value.

In general, the best policy is purely by age. The item that was started first should be the one that gets the most attention, because we want it to finish first. Then the one started after that. By definition, there can be no value to any item until its finished and the oldest items are closer to finishing so they should be the priority.

Having said that, sometimes there are situations where some items really are more or less important than others and at this point we’re talking about classes of service, which are an explicit part of our policies.

The most common class of service, in my experience, is expedited. Expedited work is more important than standard work and should be done first.

Consider an ambulance with its lights and sirens on, that is driving down the street. This ambulance is expedited and all other cars are required by law to pull over the side to let it through. This is great for the ambulance as it gets to its destination faster, but is bad for every other car on the road as they are now slower and we’ve made it more difficult to forecast when they’ll arrive.

Now imagine if every third car was an ambulance. Would anyone except ambulances get to their destination? They would not, and this is the problem with having classes of service. Some work will benefit, and other work will suffer, and the overall system becomes less predictable.

So while we recognize that expedited work is occasionally required, it should be minimized as much as possible.

The effect of high WIP on prioritization

We’ve talked about two different kinds of prioritization. First purely by business value and then by policy once it’s started. Done well, this will ensure that we are maximizing the value through the system.

Except it turns out that there’s another factor that affects our ability to deliver value, and that’s how many items we have in progress at once (WIP). It turns out that if we only work on one item at a time, the order in which we deliver value is an exact match to the business priority that we laid out up front. However, if we work on everything at the same time, then all of that business priority is thrown out the window and things get done purely based on size. Small items get finished before larger items, regardless of how important they might have been.

Skeptical about this claim? See this excellent Drunk Agile Podcast that goes through the proof.

The point is that the more work we have in progress at once, the less impact our business prioritization has. If we want things to be delivered in an order that actually matters, we need to keep WIP as low as we can.

Wrap up

Development is very expensive, and we want to ensure that we’re getting the most value we can for that investment in time and money. Effective prioritization is the key to maximizing that value.

I certainly wouldn’t care about collaboration or cross training or improving any of the systems. Just get it done.

On the other hand, if I’m playing a long game and expect to be continuing to run my business in the future, then all of these things are critically important.

All of these things are part of a long term play and if we want to be better in the future, we need to make time for them.

When I suggest things like actively working together, I frequently hear “we don’t have time now because we’re so busy”. I’m sure that’s true, and yet the reality is that you’ll always be busy. If you don’t make time for improvement, it won’t happen.

Are you playing the long game or the short?

Sometimes we have teams who have no understanding of the actual product and who are just building features. Other times, we have teams who have a deep understanding of the product and the customers and the problems that their product solves for those customers.

I’m sure you can guess which of those teams are more likely to be solving the right problems.

The teams who understand the larger context are far more likely to build the right things than the teams that are just following orders. They’re both writing code but one of them is focused on solving problems for their customers, and the others are just being busy, doing what they were asked to do.

That’s not to say that the teams are necessarily unhappy just building features. There may be very interesting technical challenges in that, to keep them interested. The point to remember is that development teams are very expensive to run and if we’re not sure whether they’re delivering the right value, then we have a real problem.

Asking teams what they do, can be an enlightening experience. If they immediately dive into technical details of what they’re doing, then they’re likely disconnected from the real problems that they’re supposed to be solving, and there’s a very good chance that they’re building the wrong thing.

For example, I recently attended a demo about a web service that a team had built. They went into detail on what the service did and what information it returned, and yet couldn’t tell me who would call that service or what they would do with the information they got. What are the odds that this service will do what’s actually needed?

If you’re confident that it still would have been the right thing, because after all, we had detailed specifications, consider this other example from a huge financial company. One team had built a web service to be called by another team. Over a period of months, they’d built the service and gone through multiple rounds of governance and approval to get it deployed into production. It was only then, months later, that the team who had requested it, tried to use it. At this point, they discovered that the service required as input, a piece of information that they didn’t have. The new service was completely useless to them and months of effort had been complete waste.

So what things might we want to hear from a team, to give us confidence that they’re actually building the right thing? We’d like to hear them talk about outcomes, not outputs. What are the problems they’re trying to solve, rather than how they chose to solve those problems. Here are some positive examples:

1. One developer I talked to at a telecommunications company, said that his job was to reduce inbound calls to a call centre. He was doing all kinds of fascinating technical things in order to achieve that, but when I asked what he did, he talked first about reducing calls, not technical details. “I implemented this change which reduced calls from X to Y”
2. At a large retailer, I asked one team what they did and they took me for a walk through one of their stores. We walked up and down the aisles and they pointed out specific things in the store and explained how their software helped with the things that the store staff needed to do. They were hyper focused on what their customers needed to do and how their software helped with that.

These are both examples of individuals and teams that are really focused on delivering value. They know who their customers are, and are focused on delivering value that helps those customers.

Unfortunately, these stories are fairly rare. It’s more common to find teams that are just feature factories. Someone tells us what to build and we do it. Is that thing valuable? Maybe. It’s hard to tell.

A good place to see if the team is focused on value is in the demo to the stakeholders. Is the demo focused on the value of the changes that were made or on the technical details? Did we solve real problems for the people using the system or were we just busy?

It can be enlightening to ask the audience at a demo if they’re getting the right level of detail. You might be surprised how often they aren’t, and yet they don’t speak up unless explicitly asked.

How much does your team understand about why we’re building the products we are? We tend to assume that they all understand the big picture, and yet that’s not always true.

Update: Within hours of posting the above, I had a poor shopping experience that illustrates the concept really well, so I’m adding it here.

My hiking shoes needed to be replaced and I’d been particularly happy with the ones I’d previously had, so I went back to the website of the same store where I’d purchased the first set, with the intention of reordering. I easily find the order I’d placed and it showed me the description of the shoes I’d bought, although strangely, the image of the product that should have been shown was just a broken link.

Searching the store, I quickly discover that they don’t carry this brand of shoes anymore. Not because they explictly say that anywhere, but because they just aren’t present in the search results.

So I went to one of their competitors and bought the shoes there. I thought at this point, that I was done.

This morning, I have an email in my inbox asking if I wanted to finish my purchase. They’d noticed that I’d looked at a product but didn’t buy it. A product that they no longer sell.

Just like the order page had, the email had the correct description but a broken link where it should be showing me an image of the product.

This is a great example of a feature factory at work.

Clearly a team had been told to build an email reminder to send to people the next day but nobody had actually thought through how the customer would be using it or what edge cases might be present. Certainly nobody had considered that I might have looked through past orders at an item they don’t sell anymore.

The lesson here is that we need to be focused on solving problems for our customers, not just churning out features. We need to consider how this will improve things for them. Sending an email to suggest that I buy a product they don’t sell, is not helping the customer in any way.

For example, we talk a lot about optimizing for flow in a system and flow is certainly a problem in most places. We should almost certainly improve our flow and yet delivering the wrong thing in an optimized flow is still delivering the wrong thing, so flow by itself isn’t enough.

Building the right thing is clearly important but if we burn out our team as we’re delivering that then we’ve lost any future capacity we might have had. So building the right thing, by itself, isn’t enough.

Working at a sustainable pace is clearly important but if we can’t deliver value when the customer needs it then why were we even doing it? A sustainable pace by itself, isn’t enough.

Everything we do will have ripple effects throughout the entire system. As we improve one thing, we need to be aware that we’ve also changed other things in the system. Some of those might also now be better and some of those may be worse. The point is that they’re almost certainly different.

If you find yourself thinking “we just need to fix this one thing” then be aware that this is an illusion. As soon as we’ve fixed that one, something else will change and we’ll need to adjust again. There isn’t a single, right, answer for anything but the simplest problems.