I am working in IT software consulting. We are doing software development projects for the unique, custom problems and requirements of our customers. These projects can be very complex, long, and very often involves large teams. When a new project is on the horizon, we have to understand the problem, plan a solution, and estimate how much time and money it will take. Well, this is the tricky part, probably the most challenging task in this industry: to give a reliable estimation for a project.

We calculate costs in man-days. This means that we have to determine how many days of work will be needed from our team to complete the project. This directly determines the cost for us and the price for our customer. It is pretty essential to get this right. Otherwise, we have a project that costs twice as much for the customer than needed if we overestimate it, or twice as much for us if we underestimated the costs.

Let’s assume we have understood the requirements as much as we could and let’s forget about their changing nature during the project (which is happening in 99% of the cases). Assume we have figured out a solution concept as well that the customer likes and accepts. So what next, how do we estimate the cost of this solution?

First, we try to decompose the project into manageable tasks or work items that we have to accomplish to build the desired system. This is a very complex and difficult challenge because you can decompose in many different ways, which will lead to very different estimation results. You need to be very smart and experienced to do it right. The more so since there is no right way to do it, just better and worse ways, but this is the topic of another post.

Now we have a list of work items, the next step is to estimate them. How should we do that?

The Traditional Estimation Process

In the traditional process (I did this for many years, and most of the industry still estimates this way), we try to estimate each individual tasks’ cost: task A takes 1 day, task B takes 5 days, task C takes 7 days, etc. There are certain recommended methods to do this as a team. More people usually result in better estimations if done the right way. The right way means that you should prevent yourselves from unintentionally influencing each others’ estimations. This is easy: everybody should come up with an estimation and write it down, then show it to each other after everybody is ready. Then you can discuss if there are significant differences, and come to an agreement.

In the end, we summarize these numbers, ruminate a little on it and usually do one of the following things:

• Multiply it by some magic number (+20%, +40%, or something like that), based on huge fucked-up projects and financial losses from the past
• Compare the relative scope of the whole project with previous projects (without decomposing it to tasks) and their actual costs, and try to correct the cost estimation based on these earlier experiences

And there we are, we have the cost of the whole project. Sounds nice and easy, but practice and experience show significant problems and flaws in this method.

Problem #1: usually there are wild differences in the estimations of different people on the same task. Someone thinks it takes 3 days, someone else thinks it takes 15 days, especially in tasks that have inherent uncertainty in them (doing a task we have never done before, trying a new technology, performing a task where we are dependent on things outside our influence, for example, an operations team we don’t know yet). In such cases, usually we agree somewhere in the middle, but the uncertainty is still massive.

Problem #2: when talking about estimated man-days, which developer do we have in mind? There is probably no other engineering profession in the world where there can be bigger performance differences between more skilled and less skilled people. It is very normal that one task takes 1 day for one person and 10 days for another. Especially if the latter one is new to the given problem or technology. How on earth can we give an estimation in man-days without thinking about who will work on the task? Nonsense!

Problem #3: if a project estimated this way actually costs between half and double our estimation, we can call ourselves lucky! I am not kidding! Very often the actual costs are at least 2x or 3x of the estimated, but double-digit multipliers are also not rare. Crazy, right? This is the world we live in, this is how this industry works. You can imagine how much stress, pain, suffering, and dissatisfaction it generates for both parties in a project like this.

The Scrum approach

The Scrum methodology recognized this problem and came up with a magic solution. The concept is simple and empirical:

• We, humans, are terrible (or rather horrible) in thinking and estimating tasks on a linear/absolute scale like days
• On the other hand, we are pretty surprisingly good at thinking and estimating on an exponential/relative scale

Sounds great, but what does this mean in practice? What should we do? When estimating, let’s forget about this linear scale of absolute integer numbers of days. Instead, create an artificial scale for the estimations, which has much fewer and more relative values, on some kind of exponential scale. This can be the powers of two (1, 2, 4, 8, 16, 32, 64, …), or the Fibonacci-numbers (1, 2, 3, 5, 8, 13, 21, 34, …), or the tweaked Fibonacci scale used in Scrum (1, 2, 3, 5, 8, 13, 20, 40, 100).

Then, do the estimations with these relative “effort” numbers in mind, and let go completely of costs or days. It is important to keep in mind that we estimate the necessary effort, whatever that means. Effort can be small or big for different reasons:

• We know exactly what we should do, we know how to do it, we have done it before, and we know it takes a significant amount of work (in this case time)
• We have done it before, we see more or less what we should do, and we know it sucks a lot (we wildly underestimated it last time)
• We only have vague ideas about what to do because we have never done that before, so it is a big effort because of this inherent uncertainty and risk
• Only think about the task and its characteristics, and forget about the person who will do it (the same task can take 1 day for one person and 5 days to another)

When I first heard about this, I was more than skeptical, but practice has proven me wrong. Actually, I was amazed and still am after doing it with several different teams and people.

Surprise #1: it takes about five task (not project!) estimations for a new team to get familiar with the method. After these clumsy few minutes, the team gets to “feel” the numbers, and start giving estimations in incredible agreement. Unlike with day estimations, it is very rare that we have wild disagreements in an estimation. And even when we have, we can discuss and rearrange our estimations in a matter of minutes. This makes the estimation process much easier, much faster, and gives us a feeling of certainty. Of course, this certainty wouldn’t worth anything without the second surprising experience.

Surprise #2: the estimations are unbelievably accurate. By this I mean once we have a team up and running, and we start working in fixed (for example 2 or 4 weeks-long) iterations, the total amount of effort numbers we deliver in an iteration is incredibly stable, almost constant. Wow, this is something! We can do something with this: we have a project, we have all the tasks and their relative effort estimations, then we have a team, we start working, after a few iterations the performance (expressed in total delivered effort points in an iteration) converges to a fix number. This number is called the velocity of the team. From here it is just basic math:

Number of iterations needed = Sum effort in the project / Velocity of the team

Cost of the project = Cost of an iteration X Number of iterations needed

What if we don’t know the velocity yet?

This sounds nice, but most often we have no fixed team and known velocity yet because we haven’t started our project. But we have to give an offer to the customer, with a price of course. How do we get actual costs from these relative estimations in this case? Well, we do the only thing we can: look at our past experiences, for statistics about previous projects done this way. Take these projects, take the sum of their relative effort points, and the sum of the man-days (or costs) needed, and check the ratio of the two. The result leads to another pleasant surprise.

Surprise #3: in the same industry, same kinds of projects, same kinds of teams, this ratio of the “man-days needed to deliver a unit of effort” is surprisingly constant. The number can vary wildly from industry to industry, based on the decomposing process of the tasks and many other factors, but it is usually the same if these conditions are the same. It is important to measure this ratio again after each project and adjust it according to newer experiences, but you’ll be surprised how stable it is in the long term.

Why and how does the Scrum estimation work?

I’d like to emphasize the fundamentals of this estimation approach:

• If we estimate on a linear/absolute scale (like man-days or costs), we are doomed to fail miserably and painfully
• If we estimate on an exponential/relative scale of effort, we can make surprisingly stable, uncertain, and reliable estimations in the long run: the velocity of our team will be pretty stable, and the cost / relative effort ratio of different projects (within the same industry) will be also incredibly constant

But (and this is a big but) this stability only surfaces in the long run, through iterations (with several tasks delivered) and projects (with several iterations). You start measuring the actual costs of individual tasks and the magic disappears, you will get the same massive variances as with the traditional estimation:

• Two tasks with the same relative estimated effort done by the same developer can take 1 day in one case and 10 days in another, and this is absolutely normal
• The same task can take 1 day for one developer and 10 days to another one, for various reasons (skills, experience, or plain luck)
• One developer in a team can constantly deliver 5x or 10x the effort points of another developer in the same team, and this is also normal

The magic is that in the long run – thanks to the decomposition of tasks and the relative estimation process – this variance flattens out:

• With the traditional linear/absolute task estimation method, you got huge variance at the task level and probably even bigger on the project level as well
• With the Scrum exponential/relative task estimation method, you keep the same huge variance at the task level but stability and reliability on the project level

How should you interpret a Scrum-based estimation?

Okay, so you are a customer, and get an offer with an estimation like this, but what is the actual meaning of it? How should you interpret it? Which questions should you ask and which not?

The estimation in the offer says something like this:

“We do all these tasks together for this price, but we don’t know how much this or that task is going to cost. We have no idea, and we know no way to find it out. But we are pretty sure that the cost of the whole project is this because we decomposed it to tasks, we classified the tasks on a relative scale, and we have reliable statistics from the past of how much the sum of these tasks are going to cost.”

Remember that the only thing the estimation never says is this:

“We can do these individual tasks for these individual prices.”

How should you negotiate?

If you are a customer, it is natural that you want to negotiate and decrease the price (or increase the scope) as much as you could. If you want to negotiate, ask the following questions:

• “Why is task A bigger relative to task B? It seems to me that they are about the same size?” This is a perfectly valid argument. You get an explanation or a correction in the estimation.
• “Why does one relative unit have the given cost? Why does 1 relative point take 2 man-days on average?” This is also a valid question. You can get some answers or maybe a discount if you are lucky.

Another common, tempting, and seemingly valid question is something like this: “Why does this particular task take so many man-days?” This is the wrong question because, in reality, nobody knows how long it is going to take, and nobody said they do, this is the whole point. If we start wondering why this or that task takes so many days, we are doomed to fail again because we castrate the very essence of the estimation process. Yes, you might be right. This task with the relative cost of 13 is too expensive. You might be right. It might take only half the estimated cost or double, nobody knows. But there are two important problems with this question.

First – this is the easy one – you never buy individual tasks, but a whole project with dozens of tasks.

Second – and this is the real problem – when you keep asking and negotiating with questions like this, arguing about the price of individual tasks (instead of their relative size), you can easily hijack the whole estimation process and push it back to old-fashioned style, even with the best of intentions. After these frequent questions, the team will unintentionally translate the relative estimations to costs and try to figure out if it will be “sellable” to the customer. If not, then they start revising and correcting themselves, the endless game of pricing tricks begins, and soon we get back to thinking and estimating in the good-old-fashioned man-days. The whole magic is broken, and we get back where we started: the 10x variance brought back up to the project level and we remember vividly how horribly painful it was. For both of us!