Visualization

Paltering: Lying by telling the truth

A few days ago, someone in my LinkedIn feed shared a screenshot of a post from X. My first assumption with any screenshot from social media (rather than a direct link) is that it’s fake, so I went and found the original. It was real, and it had been viewed 7.8 million times. The post said that OpenAI’s internal benchmarks showed GPT-6.1 Sol crushing Opus 5.5, with Anthropic struggling to keep up. The chart below it backed that up nicely: three coloured lines climbing over time, OpenAI’s green line sitting comfortably on top, Anthropic’s in the middle, Grok at the bottom.

Visualizing the right things

When we’re visualizing data, the point isn’t just to make a pretty picture. It’s to help us understand what the data is saying so we can make more effective decisions.

Visualizing WIP limits

I’ve been wanting to do a good visualization around WIP limits for a while and now I finally have one that I’m happy with. This is specifically around column based WIP limits since that’s all that Jira supports, and pretty much all my clients use that.

Cumulative flow diagrams (CFD)

For years, I’ve been watching teams show off cumulative flow diagrams as an indication of their progress. Everyone nods their heads and pretends that they understand what they’re looking at, thinking that it must be obvious to everyone else. Except it isn’t.

Collecting metrics for no reason

I’ve worked in many places where an edict has come down from senior management to start collecting metrics, without any direction. While usually well intentioned, these requests are frequently poorly considered and poorly implemented. Many times I’ve asked “what decisions are going to be made from these” and nobody knows.

Stalled work

I talked recently about how many items in progress (WIP) we have and how lower is better. That’s absolutely true, and yet it’s not the whole picture. Of the items that are started, how many of those are actually being worked on?

Survivor Bias

“Survivor bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data.”
— Wikipedia”

Visualizing Flow Efficiency

I’m playing around with visualizing flow efficiency in my JiraMetrics tool. Flow efficiency is the percentage of time that we’re actually adding value to the work item divided by the total time. So if a ticket is open for 10 hours but in that time we only spend 2 hours actually working on it then the flow efficiency would be 2 / 10 or 20%.

Tracking metrics

Most companies I work with have a desire to track metrics for their development teams, and I support this. It’s hard to improve something we can’t see so metrics are a good first step as we seek to improve.

Data Accuracy

We have a tendency to believe that anything we see in a chart is 100% accurate, although that’s often not true. To understand the accuracy of the chart, we have to understand a couple of things:

  1. How accurate the initial data was.
  2. How much of the original data set was used in the chart.
  3. How good the chart is at communicating the right message.

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

Getting Kanban metrics from the Jira API

If your team is using Jira then at some point you’re going to want to look at some flow metrics to see how you can improve. You’ll very quickly discover that Jira only provides two charts out of the box and that neither one is terribly useful. The cumulative flow chart has known bugs and the control chart is both difficult to read and has problems of it’s own.