Measuring Outcomes In The Youth Sector
This guide gives you:
A quick introduction to impact measurement in the youth sector
Common outcomes which are used in the youth sector
Data collection methods you can use to prove your impact in the youth sector
Ways you can go above and beyond to identify additional outcomes that are unique to the particular way in which your organisation works
1. A quick introduction to impact measurement in the youth sector
If you’re unfamiliar with impact measurement, you can think about it as a way of communicating the process your project or service uses in order to create change in young people’s lives.
It begins with the Activities you provide which young people attend. As a result of their participation in your activities, they will begin to change the way they think which leads to changes in what they do which ultimately leads to changes in what they have e.g. confidence, wellbeing, better relationships, etc.
You can measure each stage of your service’s Change Process:
Activities: the number of sessions you run of each type. These are basic metrics but they are a good place to start.
Participation: the number of attendees you have at each session and in total across your service. Ideally you should capture your young people’s by demographic information e.g. ethnicity, gender and age; and their referral source into your project so that you can split your reporting data by these criteria.
Changes in how people Think: your service is probably changing young people’s Attitudes, Beliefs, Opinions and Knowledge. You can demonstrate this by doing quick surveys at the end of your programme (and preferably at the start too) to see the extent to which the specific changes in attitude for example that you want to happen, have happened.
You can ask closed questions e.g. “To what extent do you agree with this statement: My aspirations have significantly improved since taking part in this programme? Choose from one of the following: Strongly disagree, Disagree, No comment, Agree, Strongly Agree.”
Or you can ask open questions e.g.: “How has this project affected you?” or “How has this project affected your aspirations?”
Changes in what people Do: at a deeper level, your service is likely influencing young people’s behaviour, be that changing their habits or enabling them to attain one-off achievements such as finding employment or enrolling in training.
You can keep it simple by having a tick-box for every achievement your young people could have so that you can tick off each time this happens for one of your young people.
Or in the case of adoption of a new habit, you can capture the frequency at which somebody does the new behaviour that your service is helping them adopt. E.g. the number of job applications they have made each week.
Changes in what people Have: ultimately your service will affect what people have, e.g. improved Wellbeing, Confidence, Relationships, Opportunities, Aspirations, Income, etc.
providing proof
Each of these “Changes” is called an Outcome. The things you observe to determine whether you are achieving those outcomes, are called Indicators and they come in three shapes and sizes. You can use as many Indicators for each Outcome as you like. Typically, the more the merrier as each additional Indicator provides additional verification that the difference you are claiming happened, really did happen. This is sometimes called “Triangulation”.
Binary Indicators are the simplest. They are Yes or No questions or tick-boxes.
E.g. “Has your ability to take the initiative improved, yes or no?”
Scale Indicators have become increasingly popular. They allow you to see how someone’s answer to a question changes over time by moving through different levels. The total change over time is called “Distance Travelled.”
E.g. “To what extent do you agree with the statement: ‘I feel able to take the initiative’: Strongly Disagree … through to … Strongly Agree?”
Numerical Indicators can be the most scientifically rigorous to use as they are based on measurable numbers and therefore less prone to subjective interpretation.
E.g. “How many times have you taken the initiative this week?”
2. Outcomes & Indicators used in the youth sector
First of all, the commonly used Outcomes.
All of these are Changes in what young people Have.
Communication
Confidence & Agency
Creativity
Managing Feelings
Planning & Problem Solving
Relationships & Leadership
Resilience & Determination
Commonly used Indicators for those Outcomes.
Once you have chosen an outcome, you need to decide how you will demonstrate the progress you are making towards it. There are plenty of ways to do this and often these ways are actually called “Assessments”. An Assessment is another word for a Survey but typically it comes with a score sheet which allows you to add up each young person’s results to the survey and based on their score, determine where they are on a scale.
The framework of outcomes for young people comes from a report by the Young Foundation who have produced this matrix of different assessments. You can find them on Google or contact us at Makerble.
3. Data collection methods you can use to prove your impact in the youth sector
SURVEYS & ASSESSMENTS
Create your own set of questions to ask young people once your programme is over
You can show the ‘D‘istance Trabelled’ if you get young people to complete the survey before and after the programme. That way you can claim that the difference in their answers was a result of your programme.
Surveys have the highest response rates when you ask people to complete them there and then rather than sending them a link to do it online later. Therefore print out your surveys or allow people to answer them there and then on their phones or using tablets, laptops or computers you provide
Case notes & evidence
Your youth workers can record notes on the calls, conversations and meetings they have with the young people you work with.
Evidence of progress made towards outcomes can be collected to, for example work from their portfolios, certificates, attendance sheets, birth certificates or any other kind of scanned document or file that is helpful
Interviews & Conversations
Rather than giving people a survey to complete by themselves, you can ask people questions in person in an interview format.
Prepare a script with the questions you want to ask or use one of the scripts available online for the outcomes you are interested in.
When you ask questions in-person you have the opportunity to ask follow-on questions, e.g. “That’s interesting! Would you mind telling me more about that?” This allows you to dig deeper and most likely uncover additional benefits and outcomes that your service is having on the young people you support.
FOCUS GROUPS
Sometimes people feel more open in the safety of their peers. This is where Focus Groups excel.
Invite 5-10 young people to a 45-90 minute discussion about the difference your service has made to them. Keep your questions as open as possible because the point of a focus group is to listen and learn insights that you might not otherwise have realised were relevant.
INSTANT FEEDBACK in-the-moment
If the young people you work with have smartphones, you can request instant feedback from them after each session of your service that they take part in.
Ask them to share something they learned today or some feedback on how the session could have been improved. That feedback could point towards a deeper insight around an issue that your service is not addressing. This is insight you can use to improve your programme and achieve more outcomes.
YOUNG PEOPLE’S STORIES
Stories will always be powerful. But rather than relying on the traditional case study format, empower your young people to share their own stories of their experiences before your programme, during and after.
4. How to demonstrate your organisation’s uniqueness through your youth outcomes
LISTEN TO YOUR YOUNG PEOPLE
Allow your young people to express the difference you have made to their lives in their own words. It is likely that you will pick out unique outcomes that are unique to the way your organisation supports young people.
LOOK BEYOND YOUR YOUNG PEOPLE
The difference you make to young people will also affect their friends & family and the local community. Why not find out what impact your service is having on them. This will help you you stand out.
OBSERVE FROM MULTIPLE PEOPLE’S PERSPECTIVES
In addition to young people’s answers to your surveys and the case notes of your colleagues; include the observations of the people and institutions around your young people. Spot the differences that their teachers, parents or other individuals have noticed.
Practical Support to Get You Started
APPROACHABLE EXPERTS
Our team will help you:
Decide which outcomes to measure
Write survey questions
Create an impact management plan and data collection plan
Analyse your survey results so that you can extract superior insights
Our network stretches across the country and we will never make you feel put out for not knowing the ins and outs of impact measurement.
FRIENDLY TECHNOLOGY
At Makerble we provide an easy to use Outcome Measurement System which you can use to do pretty much everything you can imagine to demonstrate your impact.
Produce Repeatable Reports
Create, Distribute and Analyse Surveys
Securely store Case Notes about every young person you work with
Collect young people’s Stories expressed in their own words, pictures and videos
Collect Instant Feedback from young people
Impact Measurement Planning starts at just £50. Technology starts at only £10/month.