The Perils of Project Parallelisation

In previous articles we have been at pains to stress the key point that a team only provides value to an organisation when it finishes work, which for a software team generally means ‘having working and tested code in the production environment’. To maximise this value we need to focus ruthlessly on getting stuff finished, and preventing unfinished work from piling up.

A common and seemingly logical response to such delivery pressures is to try to bring forward the start date on important work (and there’s never a shortage of that – sometimes almost everything seems important). After all, the sooner you start, the sooner you’ll finish – right? In simple situations this may be true, as this Gantt Chart illustrates:

Gannt Charts 1 - start earlier, finish earlier

What about more complex and realistic situations? If we have three projects, is the optimal approach to bring all the start dates forward as far as possible and run them concurrently?

Gannt Charts 2 - too much parallelisation

It turns out that in these situations, the adage “the sooner you start, the sooner you’ll finish” may NOT be true, however intuitive it seems*. In fact, starting too much too soon can lead to catastrophically non-productive teams, and a huge waste of time and money.

Let’s illustrate this with a fun example. We are building Christmas Trees, and it is a four step process. Only when all four steps are complete do we have a finished product that we can sell, generating value for our organisation – before that point nothing can be sold, and no value at all has been generated.

Xmas Tree 0 - steps to complete

Now imagine we have four projects, to build four differently coloured trees: one yellow, one red, one green and one blue. Across the four products that makes a total of 16 steps. The temptation is to start all four projects as soon as possible, working on all of them in parallel, in the hope that this will bring forward their end dates (the thing that matters) as well. Let’s see if that’s what actually happens.

First let’s visualise doing the projects in series, with a smiling customer greeting each finished tree:Xmas Tree production line 1 - working in series

Now let’s compare what happens if we get all four tree projects started as quickly as possible, and keep all four going in parallel until all are completed. Given a fixed resource base, this means people will have to continually switch between projects as we go:

Xmas Tree production line 2 - working in parallelAll we have done is swapped the order of the same 16 steps, so the total time to complete all four projects is the same. Did we achieve anything by parallelising? Well yes, but nothing good – we have in fact massively delayed the delivery points to our customers of three of the four projects (yellow, red and green), while blue remains unchanged. In other words, we have made 75% of our projects take longer by over-parallelising.

Xmas Tree production line 3 - series and parallel compared

And in fact reality would be much worse than this. Writing code is an intense intellectual activity, where true productivity comes in those precious periods where you manage to ‘get in the zone’, for some people an almost trance-like state where you are juggling multiple variables and possible paths in your short-term memory. It is perhaps comparable to playing chess, or writing a novel. It can take a long time to get into that mental state, and interruptions can be fatal, especially long ones. Post-interruption, you can’t simply hop straight back to where you were in that complex mental process; you may have to start again from scratch, and might never get that inspiration back in quite the same way.

This means that context switching has a big efficiency penalty. Let’s reflect that in our diagrams; the parallel example has a ton of context switching, further delaying all the deliverables. Compared to working in series (where there will be context switching only once a project is complete), we can see the terrible effects of excessive parallelisation on delivery timelines. Now all four projects are seriously delayed by working in parallel:

Xmas Tree production line 4 - series and parallel compared + context-switching

And we are ignoring other factors still. The biggest learning opportunity in product development occurs when you deliver a working product to your customer and get real market feedback. Working in series, we bring these learning opportunities forward, and the learning is compounded each time. In contrast, in a ‘release everything at the end’ big-batch approach, the learning opportunities are deferred, and any mistakes made in manufacturing the first tree will already have been repeated in the other three.

Let’s factor this into our example. A team working in series has, through some mix of customer feedback and introspecting on their experiences, discovered feature improvements (a plant pot, star decorations) and simplifications/savings (a tree that can be manufactured in three steps rather than four):

Xmas Tree 6 - product evolution

Including this into our diagram shows even greater benefits of working in series; not only is the speed-to-market gap bigger than before, as a process improvement is leveraged in subsequent production runs, but now there is a quality gap too thanks to the faster learning cycle. Working efficiently in series, you get stuff sooner, and it is better stuff:

Xmas Tree production line 5 - series and parallel compared + ideas

These examples illustrate the real life penalty of not focusing ruthlessly on getting things done and delivered in a sensible, priority order. All of these gains come through focus and discipline, not working extra hours; they are the epitome of “working smarter, not harder”. They are also a direct realisation of key Agile Manifesto principles: deliver working software frequently, and working software is the primary measure of progress.

Of course, none of this is meant to imply that no parallelisation whatsoever is permitted – as ever, the extremes of the argument are absurd. A measured level of parallelisation, where resources that are no longer needed for a first goal drop naturally down to a second, is often optimal:

Gannt Charts 3 - just right

But of the two extremes, it is a rare thing to find an organisation that works too much in series. It is far more common to find teams obsessed with starting but not finishing, where focus is not maintained, attention is allowed to wander, and precious resources are frittered away for little reward. Try the following cartoon out on your tech team: the chances are that the reaction will be a somewhat rueful smile…

Multi-tasking - new framework side project

In a future article we will see how the concepts discussed here apply not just at the strategic level (roadmaps, programmes, projects), but also at the tactical level (bugs, stories etc – tickets that appear on an Agile Board), where limiting WIP (Work In Progress) is critical to delivery success.

*see Essential Kanban Condensed