How organisations fail in Agile transformation
In this article, we explore these five root causes of failure and offer solutions to help your organisation gain a definitive advantage over the other 90%.
Agile methodologies are no longer niche. Under a backdrop of a volatile global economy, almost all industries are at risk of being disrupted by digital-first challengers who live and breathe customer-centricity. Established businesses in traditional marketplaces are being forced to embrace a start-up mentality, belatedly joining the digital disruption bandwagon purely to survive.
As business leaders try to keep pace with change, they increasingly view Agile ways of working as the key to meeting the needs of tech-savvy customers at a faster pace than the competition. When used correctly, the benefits of Agility are undeniable: rapidly solving complex problems to meet users' needs, whilst inspiring teams to become more autonomous and efficient.
And yet, despite the brevity of the Agile Manifesto, recent research Cross, Gardner & Crocker, 2021 shows that 90% of companies struggle to successfully deliver at an enterprise scale. Let's look at why this figure is so high, and what can be done to stack the odds in your favour and harness the benefits of true business agility.
Not embracing systemic change
It is simply not enough to mandate the use of Scrum in your technology department, and layer Agile terminology over the top of your existing governance structures. Your Agile transformation must permeate through the entire organisation, from the C-suite through to Finance and HR.
The transformation must be more than skin-deep: a simple re-branding of existing roles (from 'Project Manager' to 'Scrum Master' for example) will result in the same people driving the same processes to the same effect. The solution is a fundamental change in the process, a new set of clearly delineated roles and responsibilities, comprehensive training, and coaching to drive the new behaviour.
There will be resistance to such fundamental change, which must be met with clear communication and exceptional leadership from the highest levels of the organisation. C-Suite sponsorship is critical here to make decisive, rapid decisions to allow systemic change to happen. For instance, HR must allow a change to roles and responsibilities whilst Finance must create a model which allows flexible team structures to operate.
Not embedding Agility culture
The ubiquity of Scrum in particular means that many organisations will score highly on simple Agile maturity checklists. So why in these cases do we so often fail to see the promised benefits of Agility?
At the risk of simplifying the issue, 'working in sprints' or 'having a product backlog' is the easy part of 'becoming Agile'. The hardest part – and the one that delivers by far the greatest value – is embedding a culture of Agility. This mindset doesn't naturally appear after putting your people through a week of training, however. It must be embraced and consistently role-modeled at all levels, acting with initiative in the spirit of Agile rather than blindly following a set of iron-clad 'rules'. If your teams embrace change, create real value incrementally, and adapt their approach based on what they learn, then you have the basis for a truly Agile culture.
How then, can you tell if this is happening? Put simply, you must measure teams consistently across a diverse range of metrics to establish progress and identify poor implementation. It is the purpose of the Agile Coach, in effect, to drive these metrics and culture forward. We explore this topic further in section 5.
Attempting to scale prematurely
Now for a contentious statement: the best way to succeed in an Agile transformation is to not undertake an Agile transformation. Studies show (Rigby, Sutherland & Noble), "Agile at Scale" that top-down, big bang transformations are vastly more prone to failure than ones that evolve organically through a series of small-scale 'proofs of concept'. The latter approach has many benefits: it encourages a fail-fast mindset with smaller stakes; it gives teams the time and space to find what blend of Agility works for the unique needs of the organisation; and it creates a bow-wave of momentum via an ever-increasing number of small success stories.
It is important to remember that the process of scaling increases complexity exponentially: lines of communication become bewilderingly complex, reliance on inappropriate technology or manual processes create bottlenecks, and data overload hampers rather than drives decision-making. Therefore, it is essential to define and demonstrate success in small-scale Agile initiatives so that the enterprise can scale from a position of strength.
The lesson here is clear: take an Agile approach to your Agile transformation, scale on an incremental basis, and only progress to the next phase once you have met an objective set of success criteria.
Lack of outcome-focused strategy
Using Agile teams to deliver an incoherent strategy is an alarmingly efficient way of building the wrong thing, but quicker. Teams become feature factories that fail to address the core purpose of their existence: giving users what they want.
A laser focus on the prioritised outcomes you wish to achieve is the starting point of both your strategy and your Agile transformation. "Becoming Agile" does not generate value in and of itself; a deep understanding of your customer needs and how to meet them most certainly does. From this starting point, you are now able to consider which Agile tools and techniques can best meet the unique demands of your organisation.
A key aspect of an outcome-focused strategy is simplicity: if leadership need to micro-manage teams and business units to maintain direction, then innovation is stifled, and time-to-market increases. By aligning to value, leaders can "move authority to the information" (Marquet, L.D., 2013). Turn the Ship Around!, Portfolio and trust their people to ship what the customer truly wants.
Ineffective and inconsistent measurement
Any good Agility coach understands the concept of empiricism, defined as 'knowledge from experience and observation' (paraphrased from the Scrum Guide 2020). As the Founding Father of Total Quality Management, Prof. Deming once said, "Without data, you're just another person with an opinion". Without the ability to consistently inspect the current state of your product or organisation with total objectivity across a broad range of dimensions (performance, morale, quality, value, and maturity), it's impossible to determine if your efforts have been successful – or indeed where your focus should lie next.
Beyond shaping future strategy, demonstrating evidence of success is also an excellent means of building momentum behind any transformational change. Of course, your teams should be able to intrinsically feel the benefits of Agile ways of working, but concrete metrics are essential when trying to sell a message of success.
Implementing measurement consistently from team to enterprise is the job of your Agility Centre of Excellence, whilst driving the improvements to these metrics is the job of the Agility coaches. If these are not improving (or don't even exist), then your transformation won't improve the business either.
Conclusion
Business agility has always been a prerequisite for success in technology but is fast becoming a necessity for survival in industries ranging from energy and financial services to defence and healthcare. However, formal Agile transformations are inherently risky endeavours that often fail to realise their intended benefits. Despite this, business leaders ought to ask themselves: "Can we afford to keep working as we have been? And if we must change, can we afford to get our transformation wrong?"
Authors: Toan Nguyen (Agility Practitioner), Matthew Galante (Agile Team Lead), Will Giles (Agile Coach)
Reviewers: Dr Matthew Thomas (Head of Enterprise Agility), John Pingree (Senior Agility Coach)
References
Cross, R., Gardner, H. and Crocker, A. (2021). For an Agile Transformation, Choose the Right People. Harvard Business Review. [online] 1 Mar. Available at: https://hbr.org/2021/03/for-an-agile-transformation-choose-the-right-people.
Moore, G.A. (2014). Crossing the Chasm : Marketing and Selling Disruptive Products to Mainstream Customers. New York: Collins Business Essentials.
Rigby, D., Sutherland, J. and Takeuchi, H. (2017). Embracing Agile. [online] Harvard Business Review. Available at: https://hbr.org/2016/05/embracing-agile.
