Why people don’t use Monte Carlo
A few weeks ago there was a question on Reddit, asking people who don’t use Monte Carlo forecasts, why they don’t.
Driving to the airport
Imagine we wanted to estimate how long it would take to drive to the airport. You might see that it’s 50km to the airport and that your car can drive at 100km/hour. Therefore it will take 30 minutes, right?
Monte Carlo under the covers
Monte Carlo forecasting is the most common form of probabilistic forecasting that we see. It’s compelling because it can provide a highly accurate forecast of when work will be done, with relatively little effort.
What is Probabilistic Forecasting?
Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.
Reference class forecasting
Weather predictions are probabilistic, not deterministic. That means there isn’t a single right answer that we can calculate. We can’t say it will rain at 11:05 but we can say that there’s an 80% chance of rain today. Forecasting when we’ll be done is also probabilistic, in exactly the same way. We can say based on past throughput data that we have an 85% chance of being done on or before May 12.
Per-story estimates
Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.
What is a Service Level Expectation?
A service level expectation is a probabilistic forecast of how long it will take a single item to pass through the system. For example: “85% chance of completing in four days or less”.
Steps to improving predictability
If you have a need to know when the work will be done or how much you can do in a certain period of time then predictability will be important to you. We have great tools like Monte Carlo for probabilistic forecasting but the truth is that the forecast we generate is only as good as the data we give it. Garbage in yields garbage out. So how do we improve our data to make it inherently more predictable?
One Thing vs Multiple Things
When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.