Why Meaningful Measurement is important
Have you ever got to the end of a long day of work and wondered “what difference have I actually made?” Or, become so overwhelmed with trying to hit various targets set by your organisation, that it becomes harder to achieve the main goal of making a genuine difference in people’s lives, let alone understanding how much?
Meaningful measurement is how you can look at the results of all the work you or your organisation does, understand how much impact is being made, what that means for the people you’re helping, and what they need for better support.
It doesn’t just look at the numbers or stats but takes into account the story behind those numbers - the living situation, wellbeing, thoughts, emotions, and the more in-depth holistic insight of what’s happening for the individual. This is with the goal of building a bigger and more truthful picture of what’s happening in the collective. Marmalade went through our own process of trying to achieve this - you can read about it in Our Journey Into Evaluation And Storytelling.
When we consider what it really means to look at all angles of the lives of Oxford citizens, how much services help them and the real impact those services have, it becomes clear why meaningful measurement is so crucial. That said, what does it mean when that doesn’t happen? During Marmalade Week and our ongoing discussions with organisations set up to improve the quality of people’s lives, we identified some of the resulting problem areas.
Immediate challenges of conventional measurement
Various power dynamics in the conventional method mean that people far removed from the work and lived experience, end up having all the power to dictate what’s measured, and how.
In-depth measurement processes can be time-intensive and resource-heavy.
The focus on form filling and numbers often get in the way of building relationships with individuals accessing services.
It can feel extractive and irrelevant for those being ‘evaluated', and findings often aren't shared with them, creating a feeling of being a test subject rather than a human being.
What is measured can lack meaning and waste time.
Can lead service providers to overclaim.
Over-reliance on data and form filling can paint an incomplete picture of impact.
Siloed, linear and simplistic tools don't allow for the complexities of individuals and systems
Can feel like an add-on or after-thought that gets in the way of the work.
The wider impact of unhelpful approaches to measurement
When everything is very centred on the processes and requirements of organisations, it leads to people not being evaluated in a way that tells people’s stories. This perpetuates the existing power structures and imbalances on different levels. Such as between services users and providers; funders and grantees; and indeed the system at large. In part, because it dictates whose voice is listened to, who gets to determine the narrative, and who has the power to decide what’s valued, measured and how.
Those with the power to make these decisions are often far removed from the people needing support. This leads to work being designed around organisations meeting the needs of funders, which takes away the focus from communities and what is best for them. Instead, placing it on what’s prioritised by systems and methods, which tend to be scientific, and somewhat removed from the everyday reality of citizens.
Simultaneously, it also perpetuates a culture of us assuming we know more about the challenges people face and the solutions they need than those who have relevant, real lived experience.
This ultimately leads to people feeling that service providers aren’t able to identify with, empathise, or understand citizens, let alone effectively support them. It also regularly ends up in organisations coming up with short-term solutions for people that don’t take into account the complex and interlinked nature of the challenges facing them.
How organisations can start moving towards more meaningful measurement
There are various changes needed for local authorities, funders, community organisations and all service providers to build a culture of learning, adapting, and listening, that will enable them to discover better ways of helping the public.
Make learning siloed within organisations and departments/teams - rather than shared across partners/networks
Shift the focus of expectations and approaches away from reporting and funding requirements.
Actively create more trust between funders/commissioners and grantees, so that partnerships become more honest and equal.
Provide the space for grantees to challenge the status quo, and to dependence on funding
More opportunities are needed for those being evaluated to shape how the impact is defined and measured.
Place emphasis on understanding in order to learn and adapt, rather than solely on proving and validating effectiveness.
Make clear distinctions between accountability and learning.
Allow for new learnings to develop and be recognised by refraining from relying on pre-determined outcomes.
Instead of oversimplifying how change and learning happen, try to get an overview of the full context.
Give less weight solely to scientific evidence without questioning it, and more to finding the right tools for the job.
Allow greater capacity for designing, experimenting and learning, so that teams don’t become consumed with reporting.
Adopt an attitude of learning being a necessity for work and impact, instead of being considered a luxury.
Shift the emphasis from the facade of success, towards failing in order to learn and make a real, powerful impact that lasts.
These are just some of the changes that would need to happen at every level within an organisation - not just the leaders but also those on the ground who directly work with the people being supported. It’s a conversation everyone needs to be part of, which is why we’d love you to join us in our inquiry, regardless of your station, to discover how to create more meaningful, effective and human ways of measuring impact.
If you’d like to get involved with our work around meaningful measurement, drop us a line.
