Managing and measuring outcomes- challenges and possible solutions

By Helen Sanderson

I have been obsessing about outcomes. In the Individual Service Fund work I am doing with Martin Walker and Wigan Council, we need to be able to identify and deliver to outcomes. I asked our Wellbeing Teams Advisor in improvement, Andy Brogan, to help me explore this. What are the issues we need to consider here? Over to Andy.

Driving a focus on outcomes has become something of a hot topic, perhaps especially in health and care services but elsewhere too. Just start looking and it’s easy to find people working on how to do Outcomes Based Commissioning (OBC) and on what it means to manage for outcomes. It’s all plausible, well intentioned stuff; after all, how could we not want good outcomes?

My impression of the folk doing this work is positive too. It’s not easy and those doing it are working earnestly and intelligently against the momentum of how things have always been done. Credit is due for being prepared to work hard for a better way.

I worry though, that focusing on outcomes is at risk of becoming a case of the Emperor’s New Clothes, finding itself beaten back into a preoccupation with changing the score when the real opportunity and need is to change the game.

The Main Problems, As I See Them

Problem 1: Performance can’t be measured, only understood.

Performance is hugely complex, irreducible to our measures of it and always requiring interpretation, synthesis and sense making to be properly understood. While it’s common to refer to measures as performance indicators, it’s just as common to behave towards them as if they are substantive of performance itself; a way of keeping objective score on how well organisations are doing.

At best though, measures provide a peep-hole to a world of performance that lies beyond their description. As Michael Blastland and Andrew Dilnot put it in The Tiger That Isn’t – Seeing Through A World of Numbers:

“Numbers, pure and precise in abstract, lose precision in the real world. It is as if they are two different substances. In maths they seem hard, pristine and bright, neatly defined around the edges. In life we do better to think of something murkier and softer… Too often counting becomes an exercise in suppressing life’s imprecision.”

Using numbers to keep score on performance turns the way our organisations approach improvement into a game to be won or lost, not a process of growing understanding through relentless experimentation. If you’re not sure about that, just imagine the parallel between performance improvement and scientific progress. Are the results of scientific experiments a score to be kept or an insight to be understood?

Problem 2: Setting performance goals inclines us to behave as if problem 1 doesn’t exist.

It doesn’t matter what we are measuring (vis-a-vis outcomes). If we accept that performance exists beyond our measures of it then targeting improvement against those measures can only encourage us to make the numbers but miss the point. We have the power to resist that of course, but within the narrow frame that measures provide, we risk taking assurance from fake news, mistaking a change in our measures as an improvement in our performance. While it may be true that performance has improved we won’t know that unless we are able to look beyond – not just through – our numbers.

Problem 3: The tyranny of results

In many organisations, paying much heed to the above is heresy. The de facto focus, locked in place by norms of accountability for results and arms length governance, mean that not hitting the numbers is not really an option. Scoring against performance goals is what has been deemed to matter and so, when push comes to shove, it’s easy to find ourselves trapped managing the symptoms of performance (reported in our measures) all the while having little capacity left over to address the underlying health of the ways of working that create them. Improvement strategies can find themselves tied to “point” interventions that deal with presenting issues but which only serve to reshape rather than remove performance problems.

 So, in summary, we can think of the world of performance as being like this….

Fig. 1 – Performance Is Always In The Round

Back to me…

 Dissolving Problem 1: Performance can’t be measured, only understood.

In Wellbeing Teams we are learning how numbers – and other ways of describing our performance – can help us to build a deeper understanding of each other’s needs and strengths and how they can be a mechanism for creating shared meaning and real commitment to action. This means that numbers have a role but not the role that they may traditionally have had.

An example of this is how we are using the outcome circle; a wheel of outcomes similar to that in Fig. 1 but where the number and the nature of the domains that appear on it:

  • Are defined by the person we are supporting (often with their friends, family and community support network too).
  • Will change as a person’s context and what matters to them changes.

This means that each person we care for has a unique description of how things are going for them.

In using the outcome circle, we work with the people we support to help them articulate what matters to them and how they feel about how well their lives enable that. To do this, we ask them to rate each domain on a 10 point scale, giving each person’s outcome circle a unique and evolving pattern (Fig. 2).

Fig. 2 – An Individual’s Outcome Circle

This simple approach to helping those we support visualise what matters to them gives us a straight-forward way to build shared understanding and to talk about what actions we can take together to help them live well.

Dissolving Problem 2: Setting performance goals inclines us to behave as if problem 1 doesn’t exist.

But here comes the crunch…. In using outcome circles we don’t worry about the ratings that are given. We recognise that the purpose of rating is to help us have the right conversations. Our job is to help people live well in the round, not to act on each thing that matters to them as if it exists in isolation from the others.

When we discuss what action we can take together we therefore look at how we can help them reshape their lives so that they get all those better outcomes (and more) by default. For example, an elderly gentleman, lonely, struggling to cook for himself and suffering medical consequences from a sedentary lifestyle might not need a separate “solution” to each issue. Helping him out to have lunch with his pals a few times a week could address all those issues simultaneously, sustainably and while building greater independence.

Supporting people this way means thinking beyond a service-type response of the number of hours and visits we can offer. It makes us real partners in helping those we support to live well on terms that matter to them.

Somewhat counterintuitively, as we do this we know that it is just as possible that the ratings people put on their outcome circles may reduce as well as increase, even when they are doing well. For example, as we provide support it is our hope that people’s aims and ambitions change; that they become more aspirational about what their future live’s can offer. Success in supporting them may therefore be just as readily indicated by ratings that get “worse” only because that reflects what they consider possible in their lives to have got better. Why on earth then would we set goals to get to 10?

NB You can find out a bit more about how outcome circles are being sued by our colleagues at Here (Care Unbound) in this handy leaflet.

Dissolving Problem 3: The tyranny of results

We have lots of control over our ability to work this way but we still need to provide assurance to our commissioners that when they buy our support, they buy value. There’s no hiding the fact that this is a challenge so we’ve been developing a system of governance that works backwards from our clients’ outcome circles to make connections between the the actions we take there and the systemic impact that those may be having.

The logic of this approach is simple enough (Fig. 3).

Fig. 3 – Simple Connections For Better Governance

Working from top-right to top-left on Fig. 3:

  • Outcome

We use outcome circles to understand, discuss and frame action for the individuals we support.

  • Inputs

We use simple, explicit descriptions of what good looks like in doing our work. To do this we use the language of ‘clean flow’ (something Andy helped us develop). For example, when we make a care visit:

Clean IN means the best person to provide support arrives prepared, on time and expected.

Clean THROUGH sees us do what’s right even if it’s not what was planned and we pay attention to the tiny little things that could make our visit special.

Clean OUT means we leave a person feeling safe, supported, looking forward to their day and feeling confident in our care.

  • Patterns

We use our descriptions of clean flow to focus reflective practice and to make surfacing anything that gets in the way of doing great work a routine. To do this we use a simple template that Andy helped us develop and that he describes here.

Now working back from bottom-left to bottom-right:

  • As we take action on the patterns that we see we look for how this affects our ability to deliver clean inputs and at whether and how that has an impact on BOTH:
  • the outcomes that matter to those we are supporting (reflected through our discussions with them around their outcome circles).
  • the outcomes that matter to our commissioners at a system level (e.g. whether there has been any impact on the admission rate to institutional care that can be correlated to the cohort of people we are supporting).

In using this framework we are asking our commissioners to work with us to develop our shared understanding of how inputs relate to outcomes, what ‘clean’ looks like in operational terms and what it takes to make delivering clean flow easier.

We believe that taking this approach – which promotes a systematic focus on how things are connected over a narrow focus on what is being achieved – will help to remove the threat and fear posed by ‘the tyranny of results’. In essence, it will help us to put in place a learning system that will see us working together with our commissioners to continuously grow understanding and shared commitment to action over the long term rather than being trapped by a preoccupation with today’s results.

To put that another way, we think that it gives us a way to put an end to the performance game, not just to change how the score is kept.