Principles of Agile Estimation

A few months ago, my wife and I were at the hospital for our first scan. "The baby's looking very happy in there," said the sonographer. My wife and I were both thrilled to see that everything was OK and that there was a strong heartbeat. "What's the due date?" asked my wife. "26th June," replied the sonographer. I asked what time on the 26th June, and got a small, slightly bored chuckle from the sonographer. Clearly I wasn't the first to think of such an obvious gag. Wind the clock back to March 2010. It was my first week in a new job, and I was doing the "getting to know you" one to one meetings with my team of project managers. We talked at length about a project he was working on to build a new payment system for our website. "It's great!", he said, "It'll allow us to set up a new payment method in around a day, rather than the months it currently takes. Every new payment method we add allows users new ways to pay, which drives up our average revenue per user." I asked him when the project was due to go live. "December 5th", he replied. I asked him what time on December 5th. He said that was a daft question. My response was that setting my expectations around a date 9 months in the future was a daft thing to do in the first place. The sonographer and the project manager both gave me a precise date on which something was going to happen. Both of them had incomplete information at or near the beginning of a complicated nine month process: one biological, one digital. Interestingly, both involved go-live dates. The sonographer knew the date was just a guideline, but the project manager believed his date was a precise forecast. Nobody in the hospital will consider the sonographer to have failed if the baby doesn't arrive on 26th June.I'm often asked by organisations to help them "improve their estimation". Underlying this request is the perception that projects are coming in "late" or if they're "on time", they're coming in with fewer features or over budget. Implicit in all of this is the idea that an estimate is a precise promise that is scientifically derived, and that teams are "failing" if they haven't delivered according to the expectations set by the estimate.

If you're already familiar with the Agile Manifesto, you'll be recognise these principles:

Individuals and interactions over Processes and tools

Working software over Comprehensive documentation

Customer collaboration over Contract negotiation

Responding to change over Following a plan

I like these pairings, they work because of the emphasis on the word "over" - there isn't an either-or here: they make you think between the extremes of both positions, and they're supported by more detailed principles. I lean heavily on these when I consult with organisations, and when I teach. So I'd like to offer a set of pairings specifically for estimation: Estimation is a necessary activity: it supports decision making relating to timings, staff allocations and investments among other things. However, we need to get away from the idea that estimates should be perfect. Software development is a very complex process of creation and learning, and there are a lot of variables at play over the lifecycle of a project. So we know the least about a project at the start, and the most by the end: precisely when it's least useful. Accept this uncertainty, and that your estimates are unlikely to be a perfect prediction, and you can spend more time delivering working software.

"It is better to be vaguely right than exactly wrong" - Carveth Read

How will senior management interpret an estimate that, nine months out, predicts a precise date like December 5th? It's likely it will be seen as a scientifically derived certainty, whereas it's more likely to be a guess. Lots of other very clever writers have written tonnes on the consequences of failing to meet expectations in product development. Favour accuracy over precision. A precise estimate is one that cites a specific date and time, like December 5th, but if instead you shipped on December 10th, your precise estimate would still be wrong. An accurate estimate would make a promise that the product will be ready "some time in December", or "some time in Q3". As you get nearer to the completion date you can refine your date with better information, derived during the development activity in your project.

"That's the good thing about science: It's true whether or not you believe in it. That's why it works." - Neil DeGrasse Tyson

Sometimes projects are estimated by senior management (or deadlines are just decided upon). That assertion made by those who won't be doing the actual work, and with the least possible information at the beginning of a project. Prepare yourself (or your management) for either a happy accident or some disappointed stakeholders. Instead, have the team that will actually do the work collaborate on producing the estimates. They'll know best how to technically deliver the business outcomes you're looking, they'll have experience of doing it in your organisation's environment, they'll know their own capabilities, and they'll be bought into the estimates they produce. People are much better at comparing one thing with another than simply deciding the size of something with no other reference. This is true for estimating the size of a piece of work, like a feature, and this is one reason why estimation techniques like planning poker, t-shirt sizing work well. Relative sizing is quicker too - if you're looking at a requirement and asking yourself if it's smaller or larger than another requirement, you're likely to estimate faster than if you were to work through the minutiae of each estimate I won't be planning any trips in the week or so leading up to the due date. I know that pregnancy is an uncertain process, and the impact of not being there is much greater than the opportunity cost of not travelling. Can your project planning accommodate similar uncertainty? Why not? We don't work with absolutes. There's no one "right way" of doing things. Every workplace is different, and every organisation needs to work out which approaches fit best for their people, processes, culture and values. But we do think that focusing on principles is a good way to empower people to decide for themselves how to work. How do these principles for estimation work for you? Join our webinar and find out how agile estimation can help you deliver digital projects better!