1. Who benefits from your programme’s impact?
The people whom you help are an easy group of impact recipients to identify. You might already refer to them as Beneficiaries, Clients or Participants.
You can also think about other groups who might be benefiting from your programme.
Here are seven groups who are likely to benefit from your programme: Friends and Family members of participants, Professionals who support them, Local Community Members whose lives are affected by the behaviour of the participants, Society, Government and the Environment.
2. Which outcomes do your impact recipients experience?
Don’t underestimate the insight you and your team have. Based on what you have heard anecdotally, what differences does your programme make to the lives of your participants and the other impact recipients?
Think about those differences in terms of these three categories as you will find these more practical than thinking about short, medium and long-term outcomes which can often be misinterpreted because the length of term is very subjective.
What difference does your programme make to how people think?
What difference does your programme make to what people or organisations do; and how they behave?
What difference does your programme make to what people or organisations have; be it financial wealth or mental health?
There are other ways to discover your outcomes beyond looking anecdotally at the difference you make. Read our Guide to Outcome Discovery.
3. Who can confirm whether the outcomes have been achieved?
Your staff and volunteers are a good place to start. They will observe the difference that is being achieved for your participants
Your participants themselves will have a good perspective
There are additional people, organisations and digital platforms that can provide a perspective on whether an outcome has been achieved. Examples:
People: each of your Impact Recipients could potentially give their opinion on whether outcomes have been achieved for themselves or for the participants
Organisations: organisations you partner with or which also work with that participant are able to provide a perspective on the outcomes that have been achieved for that participant
Apps and Digital Services: if your programme improves people’s fitness, rather than asking your participants if they feel fitter, you could connect to their Fitbit accounts to see how many steps they have been taking on average each day.
4. Which questions tell you the extent to which each outcome has been achieved?
Your questions can take various formats in addition to simply asking open questions:
Yes or No indicators: e.g. “Did the participant find employment, yes or no?”
Scale indicators: e.g. “How much do you agree with this statement: “This programme has enabled me find employment; Strongly Disagree, Disagree, No Comment, Agree, Strongly Agree?”
Numerical indicators: e.g. “How many jobs did you apply for this week?”
When asking questions, you can apply these principles to ensure they stand up to the scrutiny of commissioners, funders and the media:
Before and After: rather than asking someone at the end of your programme if they felt that your programme contributed to them finding employment; ask someone at the start of the programme how fulfilled they were in their last job and then ask them at the end of the programme how fulfilled they are in the job you have helped them find. This will accurately show the difference you have made.
Questions should be Fair; Not Leading: Rather than asking: “Has this programme been life changing for you, yes or no?” ask someone to rate their experience of your programme on a scale.
Observable Not Subjective: while asking people how they feel about something is useful, when you are asking people how they feel about themselves, what you are actually asking is their level of self-awareness. Often times, programmes ask people how confident they feel about something and people usually say their confidence is high. Then they attend the programme and realise that they are underperforming. When they are assessed at the end of the programme they are wiser and more humble and their answers show that they are less confident than when they started. This looks like a negative outcome. Which is why it is better to ask questions about specific things people do rather than asking them about their own perception of what they do.
5. When will those questions be asked and observations made?
Programme Start: Where it is appropriate, assess your participants as early in their engagement with your programme as possible. This will give you baseline data from which you can see the difference caused by your programme.
During the Programme: The people on your team who work with your participants can record their meeting notes and observations about the participant on an ongoing basis. You can also assess how your participants are progressing towards each outcome. This will give you a truer sense of the journey that your participants go on as some will progress through the outcomes faster than others. This will give you the data to demonstrate that.
Programme End: This is the most common moment to ask questions and typically these are of the participants in the form of a survey.
6 Months After the Programme: Checking in with your participants 6 months later is a good way of proving the lasting impact of your programme.
Several times After the End of the Programme: This provides even more proof of the difference that your programme has made. The stories and data about these people for whom you do longitudinal studies, i.e. question them over a long period of time, will be valuable and is persuasive to commissioners and funders.
6. Where will the answers and observations be stored?
In your head
On paper
In spreadsheets
In online survey systems
In one place: a database. This is best for reporting, for understanding cohorts of participants over time and for complying with GDPR.
7. How, When and for Whom do you want to report on the results?
How do you want to report on the results?
Manually created reports: each time you want a report, you manually create one
Automatically generated reports: you instruct your database to create a specific report for you at a specific time each week, month, quarter, year, etc.
Interactive reporting dashboards: rather than producing reports, you give the people who want the reports access to a dashboard where they can explore your results for themselves
Balancing Effort with Trustworthiness
It will be clear to you by now that there is so much you could do when it comes to impact measurement.
It’s normal to feel a little overwhelmed by it. At this stage all you need to do is work out where on the scale you want to start.
Wherever you decide to start is fine. What’s important is that you do make a start. You can always improve your approach as time goes by.
Help is at hand
Read the next article in this series: 7 Steps Every Manager Can Take To Monitor and Evaluate Their Programme
Sign up for a free trial of Makerble Impact Analytics to start monitoring and evaluating your programme
Book a free chat to discuss ways to manage, monitor and evaluate your programme