Measuring outcomes across digital transformations

Across all industries, organisations are making a significant and promising shift towards data-driven decision-making.

Fuelled by the growing volume of data and new capabilities such as Generative AI, this shift offers a wealth of potential for meaningful insights and improved decision-making. It’s an exciting time for organisations to thrive, leading us to consider not only what and how to measure, importantly, but also how to cultivate an environment that allows this potential to flourish.

The challenge at hand

Over the past decade, organisations have embraced lean and agile transformations. Businesses have adopted new working practices, principles, cultures and tools to help their organisations thrive in VUCA (Volatile, Uncertain, Complex and Ambiguous) environments. However, transformation "success" has often been gauged by gut feelings and anecdotal evidence, offering limited insight into whether changes have significantly impacted the top and bottom lines. This reliance on intuition continues to obscure reality and hinders an organisation's ability to improve and chart a purposeful direction.

Identifying what to measure

At their core, organisations evolve with the intention to improve across several objectives. Typically, they aim to deliver more value to their customers, faster, with happier employees and fewer quality issues. Therefore, the outcomes of Value, Delivery Flow, Quality and Morale are the critical elements we should measure. Measuring these outcomes, rather than just outputs, is not only a choice – it's essential for encouraging the right behaviours and clearly articulating organisational trends.

There are several metrics to measure each of the outcomes practically. We've provided a few examples below:

• Value: Revenue, Customer Satisfaction, Objective & Key Result (OKR) achievement.
• Delivery Flow: WIP (Work In Progress), Work Item Age, Lead/Cycle Time, Throughput, Deployment Frequency.
• Morale: Listening, Communication, Camaraderie, Pace.
• Quality: Change Failure Rate, Time to Restore Service, Bug trends.

A common challenge when introducing outcome measurement is understanding and maintaining the quality of the data that drives insights. Therefore, consider data hygiene as an additional fifth outcome. Visualising data quality is essential for effective data capture and regular updates. This practice not only fosters good discipline, but also increases trust and effectiveness.

We encourage leveraging a variety of outcomes to ensure balance. For example, while efficient flow is beneficial, it shouldn’t come at the cost of product quality. Balancing these aspects is vital and ensures that all parts of your organisation evolve safely together. When considering what to measure, focus on outcomes, balance, and inspect trends.

How to measure outcomes

Although this may seem challenging, it's simpler than it appears. Many organisations we’ve partnered with already have the necessary data in their suite of tools. The last step is extracting, transforming and visualising this data centrally through various lenses.

It's natural to think of outcomes as different horizontal lenses across your organisation, but one can also leverage various levels of vertical abstraction. View horizontal outcomes at the team, domain, portfolio, and organisational levels to gain comprehensive insights and trends.

Measuring outcomes effectively

Now that we’ve explored what and how to measure, let's look at some key principles and practices to help your organisation get the most from data-driven insights.

Throughout this article, we’ve emphasised measuring outcome trends over specific targets. This approach encourages good behaviours, prevents cargo cults and reduces gamification. To measure outcomes effectively, 3 other principles are critical:

1. Contextual measurement over monitoring: Outcome measurement should be used to understand, learn and continuously improve based on your organisation's environment. Since every context and team is different, you need to consider your unique environment for outcome measures to be effective. Avoid using outcome measurement to "monitor" teams. This can create an environment of reduced psychological safety and disempowerment, which ultimately reverses transformative benefits as well as metric effectiveness.
2. Learning culture: A culture of safety, transparency, and sharing is critical to preventing gamification and destructive behaviours. If your organisation lacks this foundation, it’s worth exploring structural and evolutionary changes to embed first.
3. Aligning with narrative: With continuous improvement in mind, repeatedly check the insights from your outcome measures and compare them to your organisation's narrative. If the measures and narrative don’t align, question whether you’re measuring the right things or if your approach to measurement needs adjustment.

By embedding these simple steps, you’ll set your organisation up for success. You’ll be able to identify trends across a balanced set of outcomes and leverage data to inform decision-making, drive improvement, and foster learning. In a world where data flows fluidly through organisations, using this data to guide transformation and measure success is crucial for keeping everyone aligned.