Estimation and Agility: a paradox?

Photo by Alex on Unsplash

The subject of estimates is at the heart of the debates when starting out in Agility. How to do ? Is it useful? are all questions to which there are just as many different answers.

One thing is certain: the Agile community is aligned on the fact that estimates, as used in the past, are no longer adapted to the current context and can even be harmful to the successful implementation of an Agile transformation.

But how would one do otherwise ? A current under the name of #NoEstimates has distinguished itself for a few years by re-questioning the place of estimation and by proposing an alternative approach.

I propose in this article to discuss estimation, Agility and to see what the #NoEstimates approach really consists of.

Good reading ! 🙂

Why estimate?

One of the main reasons for estimating, as used today in project management, is whether the project is going to be profitable and therefore worth doing.

This profitability is obtained by making the ratio between the expected income and the resources employed to obtain it (Wikipedia).

The evaluation of the cost of a project is considered to be one of the most important and delicate phases of the scoping phase because a drift in this costing can generate a risk of significant impact on the respect of the budget and the expected deadlines: this is why we spend a lot of time and effort to ensure the reliability of our results.

Estimation and Agility

It seems interesting to me to go back a little bit, to recall some key elements of the emergence of Agility.

In the 1990s, the Internet appeared and computer projects multiplied.

The way we carry out these projects is simple because as usual, we like to use good recipes from the past. We know how to build buildings so we will build software the same way! We will therefore take time to analyze the need that we believe to be constant, implement a solution and provide it all at once to the customer.

After a few years of implementation, the conclusion is as follows:

Put another way : it does not work.

Indeed, the recipe of the past does not seem to work as expected. It is therefore necessary to do otherwise. This is what these 17 experts wanted to do by promoting a light approach to software development as opposed to a heavy approach driven by documentation: in 2001, the Agile Manifesto finally saw the light of day.

Without going back to the content of the Agile Manifesto, Here are some basic ideas:

Let's see the impact on a common reference of project management, the iron triangle.

The iron triangle

Traditional Approach

In a traditional, plan-driven approach, the estimate is critical to ensure we meet the load and schedule to achieve all of the predefined content. It only works if we have everything in hand to be able to predict and that the components on which we are basing it do not move too much. We then have the conviction of being able to know, from the start, as for the construction of buildings.

The question we ask is then: “Can we do it all? »

Agile approach

If we accept uncertainty as a reality, a predictive approach is no longer suitable. We prefer an empirical approach allowing us to adapt more easily to movements and changes in the environment. By considering not knowing at the start, we implement a series of experiments to try to validate our hypotheses as quickly as possible.

The question we ask is then: “How do I optimize my production of value with what I have (time and people)? »

In this framework, we make an investment (a number of people over a period of time) and we manage to provide as much value as possible with what we have.

Thus, we can already say that the concept of estimation, as defined above, seems unsuited to the guideline offered by Agility.

Estimation: an inappropriate activity?

As in everyday life, it is important to choose the right tools for the right problem.

I find that the Cynefin model is particularly relevant in this context, describing different contexts and their associated strategies.

I will only take 2 of them here:

• A problem is defined as Complicated, if the causal link exists but is not immediately clear. The associated strategy is to analyze the problem theoretically and then to respond to it based on these results.
• A problem is defined as Complex, if the causal link does not exist and can only be defined a posteriori. The associated strategy is to set up an experiment in order to validate its hypotheses.

The paradigm shift brought about by Agility is to conceive and accept that there may be other areas than the Complicated – where everything is predictable, if you take enough time. Estimation, as defined above, belongs rather to the domain of the Complicated.

The world of software development today is so dynamic and changing that it is impossible to predict in advance what will be the next need and therefore what will be the next solution to be implemented. We would then rather place it in the domain of the Complex.

Note: Today we often say that we live in a world VUCA of which Complexity is one of the attributes.

The results of this BCG study show that in 60 years, while the external complexity index (the environment) has multiplied by 6, the solutions implemented in companies to respond to it have multiplied their complication index. internal by 35!

It's like eating soup with a fork: you can try, but you'll need a lot of fork!

The problem with estimates

If you still decide to go into the field of estimation, here are 3 things to keep in mind:

1. This always takes longer than expected, even taking into account Hofstadter's law (Hofstadter's Law)
2. The work is spread out in such a way as to occupy the time available for its completion (Parkinson's Law)
3. The cost of a feature is the cost of its Accidental Complication

I suggest that you go into a little more detail about this last notion, introduced by JB Raisenberger, which I found particularly interesting.

Indeed, he asserts that:

• Essential Complication : the difficulty of the problem itself
• Accidental Complication : the emerging complication stemming from organizational structures (like validation processes for example) and the way code is written

Knowing that the accidental complication is often greater than the essential complication, we can conclude that:

What this tells us is that estimating effort alone, as is often done in projects, is not appropriate. Indeed, the work passes through a flow and it would therefore be more appropriate to measure elements such as cycle time or Lead Time to get an idea of the true production capacity of the system. The only problem is that this system is rarely constant, dedicated and stable teams being only too few in organizations.

So to summarize, when we are asked for an estimate:

The worst part of all this is that the estimate is often requested when we know the least, and when the most important decisions are made!

Note: if the notion of accidental complication interests you, I invite you to look at this conference (7mn26).

The #NoEstimates approach

When we see #NoEstimates, we can think that the idea is to stop making estimates: This is not the case.

As well said Angel Medinilla in the preface to book :

In this direction, we can feel the Lean inspiration of the approach: the estimate in itself has no value for the final product, so we will reduce it as much as possible.

What to do without an estimate, which was the basis of classic project planning and monitoring?

The main purpose of #NoEstimates is to help with project monitoring in the simplest way possible. At the heart of the approach, the following Agile principle:

To answer this, we need to provide a functional product increment, corresponding to actionable information.

The principle of the User Story then seems quite appropriate to fulfill this role, the latter being considered as the smallest element of useful value to the customer.

Who says User Story, says INVEST, and this is particularly its component I (Independent) which will help us to better follow our project.

Indeed, the search for independence involves cutting down the needs as finely as possible, which has many benefits:

Along the same lines, Mary Poppendieck asserts that:

Thus, the Prioritization takes precedence over Estimate. We can believe that prioritization comes after the estimate, it simply comes from our habit of focusing our attention on the Return on Investment – corresponding to the relationship between the value added to the client and the cost of its realization.

This sentence is likely to make some people react, probably. However, it is today the conviction that I have: the value must be at the center of our concerns, the ROI comes second.

Indeed, if we take the different categories of the Purpose Alignment Model, an element has value not only when it makes us earn money, but also when it prevents us from losing too much. There notion of value is complex and subjective hence the importance of really studying the question in depth.

The Agile approach describes itself as being driven by value, no matter what it means to you at any given time, just make sure to deliver it as quickly as possible, incrementally, and without excuses! 😛

To #NoEstimates

Vasco Duarte offers us a path to follow, step by step, to go to #NoEstimates. Rather than seeing them as sequential steps, I invite you to think of them as a series of options depending on where you are right now.

1. Get out of the absolute estimate

a) Using Story Points

If you are still making absolute estimates in “hours” and “man days”, switching to Story points is a first step. Indeed, breaking away from the absolute estimate (subjective and individual) to move on to the relative and collective estimate is a good start and already has a lot of value!

Several practices allow you to go in this direction: the Poker schedule or theeXtreme Quotation For example.

b) By stopping estimating tasks

After estimating their User Stories in points, teams tend to break them down into tasks and then estimate them in hours to check that it remains consistent. Once again, this may appear reassuring but it does not help us to foresee the unforeseen unexpected! 🙂

It should be kept in mind that in a planning phase, what matters is to have the first tasks to be able to start and not to seek completeness. Indeed, elements will necessarily emerge during your iteration / cadence, so you might as well start as soon as possible to come up against the reality on the ground!

What matters to us is therefore the element of value (the User Story), not the way in which it is implemented.

Here is a protocol that I used in a team:

Keep in mind that theestimation is not the goal, but rather the alignment on what to do. And no, you don't need to estimate to analyze and cut! 😛

c) By removing option cards from Planning Poker 

The objective is not to try to be "precise" in the estimate.

At first, you can only keep 1, 3 and 8 for example. The idea being to reduce the number of options as much as possible in order to achieve the most coherent division possible. So the question you will ask yourself later will be more like: “Can we release this Story tomorrow if we start it today? »

In my accompaniments, I invite the teams to use only 3 cards:

In this way, each card enables decision-making leading to action rather than seeking the accuracy of the estimate.

2. Reduce the effect of Parkinson's Law

To reduce the effect of Parkinson's Law, you can reduce the time you give yourself to complete a piece of work: for example, give yourself a goal of finishing each story in 2 days. Here, it is above all the intention that counts, the reality is often quite different! The idea is to take a consistent and shared cutting habit.

It may be hard at first, but this approach will bring you concrete benefits.

3. Better understand your system

a) By accumulating data

The objective is to better understand its production system. The idea nevertheless remains to keep the measure as simple as possible so that it does not become an overload for the team.

For example, you can simply note the entry date and the exit date of your stories: we commonly speak of Lead Time.

Accumulating this data will allow you to create histograms, like the Kanban control chart:

This visual information will initially allow you to have an idea of your production capacity. This will allow you to better exchange with the profession by relying on a concrete database.

We then hope to see the questions go from " How much does it cost ? » To “When can I have this functionality? ».

b) Using Average Lead Time

The accumulation of previous data will allow you to have an idea – at a given period – of your average production capacity: this is called the average Lead Time.

You can also do the exercise by evaluating the average Lead Time of stories that you consider to be of different sizes: S, M, L for example if we use T-Shirt sizes.

You should eventually realize that these numbers are as reliable as the estimates!

c) Considering stories of the same duration

By working on a coherent and shared breakdown, on which will be based a set of concrete data, you will be able to simply count the number of stories that you are able to deliver per period of time. You will thus have a real visibility on the progress of your project and an ability to exchange with your profession in order to prioritize the work to be done as well as possible! #NoEstimates

Conclusion

Estimating is a hotly debated topic in today's project management. Indeed, as an Agilist, we have to juggle between the conviction that it is not adapted to the context of our interventions and the difficulty of companies to change the paradigm by accepting uncertainty as a reality.

Our challenge is therefore mainly to changing perceptions to reinvent the way we do things.

Movement #NoEstimates stands out as an approach that goes back to the sources of Agility: cut out the need as finely as possible, rely on empiricism by accumulating data and learn continuously in order to be able to produce value as quickly as possible! I personally really appreciated the spirit, a little less the book of Vasco Duarte !

I invite you to test this way of managing your projects by starting slowly and observing the benefits you could derive from it! 😉