Why oh why?

How often is an estimate correct? Why do we seem to under-estimate so often? We can’t always blame the pointy-haired boss.  Can we?

Well, no we can’t.  Really, we could have some sympathy for those pressuring us to higher certainty and lower costs.  They’re just trying to reduce risk, even if the risk is of looking bad. Maybe they’re stuck in a world of cost accounting, unaware of its impact or alternatives.  If a project has overrun, and if all you know is detailed estimation, then it’s easy to assume that you can’t have done enough detailed estimation.  So there’s an urge to do more of it!

Doing more estimation is a waste of time and energy.  Let me show you what is going on here.  It is something simple and fundamental.

Let’s Estimate Something

Here’s some work we’re estimating. Let’s consider everything that could be known about this piece of work.

First, there’s some of it about which we have a high degree of certainty.  We know what we know about this.  Let’s call this KK – Know what we Know.

Then there’s the rest, the stuff we Know we Don’t Know. Call this KDK. This must have a degree of uncertainty, by definition. We could manage the uncertainty if we buffer the estimate.  With a mature and wise customer, we could provide a range.  The risk is that we assume the high end is more realistic, and those employing us turn the low end into a deadline.

So, we’re done.  That’s our reasonable estimate.  Quick! Commit!

Easy, tiger!

Wait!  We’ve missed something.  It’s not a reasonable estimate.  As it stands, it’s pretty much a guarantee of failure.

What are we up to here? We’re developing something new.  If we were developing something that wasn’t new, we wouldn’t have to estimate it, would we?

If it’s new, there must be parts of it that we Don’t Know that we Don’t Know (DKDK), by definition.  I mean – it’s new!

Here’s my question: will DKDK lead to more  work or less work?  (Hint: it’s MORE work!  Duh!)

Well, here’s a grain of solace.  Inside DKDK is DKK: what we Don’t Know we Know.  Some stuff  turns out to be very like something else we’ve already done, and we hadn’t realized it until we started work on it.  This will account for those occasions when the estimate turns out to be high – sound of heavenly trumpets!  But since we usually remember what we know, these occasions are not the norm.

The point remains: it’s all pretend until we start work on it and start realizing – making real – what’s actually involved. Nothing is certain until– when?– until it’s actually finished!

The truth is out there

Take a look at Steve McConnell’s classic work on the Cone of Uncertainty.

Note that the broad end of the cone, at the time of producing the estimate, gives a range of accuracy of the original estimate of from 25% to 400% of the actual. 

The Standish Group’s CHAOS report shows this clearly.  Here’s the table of Time Overruns from the report.

Look at this as a histogram:

Thanks to DKDK the work more often ends up running long.  Sometimes, sure, it’s early thanks to DKK.  Only 20% of the time does the combination of KK and KDK prove roughly accurate. 

Over to you

This is why the wise thing to do is turn the iron triangle upside down and declare that the team is fixed, the time is fixed (by sprints), and the scope is variable.  This isn’t just wise, it’s because there is no such thing as a fixed scope!  The scope of anything that hasn’t yet been created is unknown, and so must be variable.  It’s all pretend!

Now, do I need to explain the genius of relative size estimation using story points, limiting work in progress, and reducing cycle time?

The only sensible thing to do is to estimate using a scale that is course grained, recognizes that the larger something is the more uncertainty there will be, and is local to those doing the work. Hence Story Points and Relative Size Estimation.

It’s only sensible to limit the amount we do to the smallest possible amount so we can check with our customers if it’s actually useful to them quickly, and adjust course accordingly. Continuous planning makes sense.

It’s only sensible to minimize up front detailed specification and estimation, as that’s the point when we know the least. Understand what is meant by Last Responsible Moment.

So give an estimate, sure.  Estimate the whole thing.  But it is unprofessional and irresponsible to then allow yourself to be held to a fixed scope in an Agile world.  Don’t do it.  You’ll only disappoint your customer when the intention was to delight them.

Comments, as ever, are more than welcome.