The statement that an estimate is not a guarantee might sound obvious but it often feels anything but. Over the years I have frequently heard managers complain that something has taken longer than was anticipated and have been told to improve the estimates.
Of course at times the estimates have been inaccurate, with features taking far more time to implement than was initially predicted. The question is how to learn from these experiences to help improve your estimates. This is especially pertinent for software engineers within financial services because this same conundrum is at the heart of pricing financial instruments.
This article looks at how the Rational Expectations Theory can help you improve your estimates in software development. It starts by looking at estimating something much easier than a software enhancement, your journey time into work, and demonstrates when that estimate should be considered good or bad. It then looks how this can be applied to software development.
Rational Expectations Theory
A few years ago, I was introduced to the Efficient Market Hypothesis whilst reading about how stocks are priced. This hypothesis is built upon the Rational Expectations Theory. Paraphrasing this theory, it states that an individual bases their estimate on:
- Their rational outlook
- All the available information
- Their previous experiences
Although both the hypothesis and the theory are out of favour with many economists, the principles of rational expectations struck a chord with me.
Rational Expectations applied to Journey Times
To demonstrate rational expectations in practice, consider the question of how long it will take you the next time you travel into work? Assuming that you don’t work at home, aren’t starting a brand new job and haven’t just moved homes, then this is something that you should be pretty good at. After all you have an abundance of experience making that journey.
My estimate for my door to door journey time would be an hour. However there is a pretty big deviation between my fastest and slowest ever times. Every now and again my interconnections align perfectly and I can make the journey in 45 minutes. On the other hand, it has taken me 3 hours when my train broke down and, to compound matters, the tube line I needed to complete my journey was closed.
Rational Expectation Theory says that estimate of 1 hour is good because it is the equilibrium of my times. An estimate of 45 minutes would be bad because it’s overly optimistic and I’d rarely expect to meet it. An estimate of 3 hours is also bad because it is too pessimistic since it has only occurred once, and the circumstances were exceptional.
The theory also states that the estimate is based on all available information. My estimate of 1 hour would no longer be reasonable if there was a weather forecast for tomorrow predicting snow. In Britain snow is renowned for crippling the travel network. It doesn’t matter whether I have heard the weather forecast or not, it is my responsibility to find out all relevant information to aid my estimate. Ignorance is no excuse.
Rational Expectations applied to Development
I like the Rational Expectation Theory because I think its three rules capture the essence of what estimating in software development is about. Prior to reading about the theory I didn’t think too much about how I came up with an estimate, it was pretty much a gut feel. Now I realise that subconsciously these rules were actually what were going through my mind. The theory helps me to break the estimate into its constituent parts.
Applying the theory to development leads to these conclusions:
- Two features on the backlog, in the same area of the code base, with a comparable depth and level of complexity should have similar estimates.
- Knowledge about how easily a new feature might be implemented within the current code base is important. If you know the code will need some refactoring in order to implement the new functionality cleanly, then this should be allowed for. Of course the law of diminishing returns applies here; familiarity with, rather than intimate knowledge of, how the feature might be implemented will be good enough in most cases.
- Allow for your past experiences with your code base. Take the case where the two similar features have been sized at 1 day (or story point, or other unit of measurement). Whilst working on the first you uncover some unexpected issues, and it ends up taking 4 days to complete. Does it still make sense for the second item to still be sized at 1 day? The answer to this depends on which of these two scenarios the overrun fell into:
- If the reason it took much longer to complete is systematic and likely to affect the estimates for all the other features, then project velocity will be able to compensate for it. The relative sizes of the estimates are still the same.
- If the reason was localised to the code surrounding that feature then it is only rational to keep the estimate at 1 day if I believe that the problems were an anomaly and won’t impact the other feature.
Well, that’s nice in Theory…
I find that the Rational Expectation Theory is a useful reminder of what to do when estimating, however in software development we are never building the same thing twice (we are DRY here aren’t we?). As a result the idea of an equilibrium point is non-existent; we are dealing with a unique piece of development with a sample set of one. However a lot of the time we are doing similar things that we’ve done before, so we do have a reasonable point of reference for comparison.
When is an Estimate bad?
In my example of a journey to work I estimated that it would take an hour. I should be pretty good at knowing how long this takes since I have made the journey over 2,000 times. If the journey takes me an hour and a half tomorrow does that make it a bad estimate? That depends. If there was planned engineering work causing a reduced service that I could have found out about in advance, then it was a bad estimate. I should have allowed for the impact of the engineering work. If the delay was caused by my train breaking down then the estimate was rational and therefore wasn’t bad. After all, it is just a estimate, not a guarantee.
The same logic applies to estimates given for development. We never have the luxury of having written exactly the same functionality many times previously, but there are often significant overlaps. Provided that your estimates are rational, then you can be confident that things should average out. Some features will take longer to write than estimated whilst others will take less time. Also don’t be afraid to re-estimate specific features if fresh evidence comes to light that shows that the estimate for a particular feature is irrational.
Afterthought: A World Free of Estimates
At Caplin we have spent the last few months trialling Kanban with several of our internal development teams, which offers a idyllic world where estimates aren’t needed.
I am still trying to grapple with the implications of this, and am looking forward to understanding it better. My gut feeling is even with Kanban estimates will still be necessary at a high level due to the inexorable fact that the priority of some features will always be based on the cost to build them, which is a multiple of number of people working on it by the time taken by them to build it.