Why Does Agile Fail?

It’s still not fashionable to talk about Agile failure. I guess those few of us who do win few friends by it.

Never mind. My motivations stem from trying to make the world of work – of knowledge work – a better place for the millions who suffer the consequences of ineffective organisations, day in, day out. I don’t expect much thanks for it, but there it is. Take it or leave it.

I’ve given up using deontological moral and ethical arguments in favour of utilitarian ones. Not that I expect any form of argument – be that rational or emotional – to have much effect. Normative (experiential) learning seems like the best – nay, the only – game in town these days.

Definition

For the sake of clarity, let’s define what I mean by “failure”. Failure, here, is simply the failure to realise the desired or expected results from adopting an Agile approach to e.g software development. People generally know – although not always explicitly – what outcomes or results they’re looking for.

Some Reasons

So, why does Agile fail? And fail it does, in at least 75% of cases. Maybe as high as 95% of cases. Reliable numbers are hard to come by. Especially with so many vested interests in the mix.

• By design.
  As a local optimisation – because Agile was conceived as such – even when the Agile adoption “succeeds”, it only addresses some 10% of the problems with software development. The 10% the development team can fix themselves. Some 90% of the problems remain inaccessible to the team. Only when the wider organisation gets involved can those other 90% receive some attention. And without this wider attention, the rest of the organisation will assume Agile has failed. It’s a matter of expectations, really.
• By mindset.
  Bringing a classical, command-and-control, analytic mindset to a software development effort is like bringing a knife to a gunfight. Without a collective mindset bent on enabling learning, discovery, innovation, self-organisation and cognitive function, results will remain poor irrespective of practices.
• By time.
  Even when an Agile adoption succeeds, and even when the other 90% of the problems outside the development team get attended to, it’s likely that the solutions are not long for this world. Unless the question of Organisational Cognitive Dissonance is also addressed – whether by luck or intention – any short-term gains are hostage to the vicissitudes of fortune.
• By cargo-culting.
  Many Agile adoptions consist of little more than adoption of a set of practices. Agile “by the book”. With little or no inspection and adaption, or understanding of the underlying principles. This can often happen when e.g. management see Agile as just another “software development method”.
• By naivety.
  Software development is not a simple endeavour. Control and predictability are not possible, however much we might naively wish for them. Or believe we need them. Software development is about the dance between organising intent and countervailing entropy. Attempting to run a software development effort like it was easy or simple or manageable is simply asking for failure.
• By mistaking the nature of the work.
  Software development is a kind of collaborative knowledge work. Mistaking it for something else – for administrative, repetitive or manual work – is another shortcut to the failure farm.
• By bad luck.
  So many random factors can impact an Agile adoption. Let’s acknowledge that even when all our ducks are lined up, often things can just go awry. Sponsors and champions can change post or leave. Key players in the delivery teams can quit. Upturns and downturns in the organisation’s markets can distract and detract. Technologies can suddenly change. Regulatory constraints shift. New fads can overtake our plans.

Honestly, so many things can undermine an Agile adoption, it’s a surprise to me that any attempts actually succeed. And it’s not even that a successful Agile adoption means a big uplift in organisational effectiveness. Better to invest our limited time, attention, and money in something with a much bigger potential payback, if you ask me. Which you probably didn’t.

– Bob

Further Reading

When Does Agile Fail? ~ Craig Strong
Why Agile Has Failed  ~ Mike Hadlow