Manage collective impact evaluations in which organisations are the Data Subject

This article outlines how to use the Makerble platform to evaluate a collective impact programme in which your evaluation focuses on organisations as the beneficiaries rather than their service users.

Each of partner in the collective impact programme would exist as a project on Makerble

Every project has a logic model composed of five stages:

• Activity outputs

• Participation outputs

• Short term outcomes

• Medium term outcomes

• Long term outcomes

You can create an Activity Output for each activity being done by the partners. 

• When progress is logged towards Activity Outputs, there is no need to tag individuals

• When progress is logged towards Participation Outputs, you can simply log the number of people who attended (a workshop for example) and if you want to, you can disaggregate that number of attendees e.g. by Age as shown in the example below.

It's fine for multiple partners to use the same Activity Outputs or for each partner to have their own set of Activity Outputs. In scenarios where they are the same, the system will aggregate the results together in an Album. You can see this on the Progress Board below. Cards 7 & 8 are the 'Album' view that aggregate the results from the three partners (called Strategic Initiative, Partner 2 and Partner 3)

For expediency, you can save a logic model as a template and apply it to multiple projects at once. (we call it a strategy)

You can engage the individuals in Participant Surveys. You can deploy these in a number of ways. The two options most suitable for an evaluation where organisations are the data subject are likely to be the Responses are Anonymous option or the Send Personalised Links via Email option.

• With Responses are anonymous a public web address is generated which anybody with the link can use to complete a survey

• With Personalised Links via Email, each individual participant receives a unique survey link that only they can use to complete the survey

Evaluating a funder's impact

Most funders approach impact reporting by communicating these three things:

• Case studies about grantees

• Amounts spent, split by cause and/or location

• Number of grants made, split by cause and/or by location

Whilst this is a good place to start, it only tells part of your story. We help funders go further to measure the difference that your grants have made.

We recognise that as a funder you have three levels of impact:

• Aggregate impact: the combined outcomes of the projects you make grants to

• Grants Plus impact: the difference that your support makes to your grantees’ projects beyond the financial benefit, for example, increased profile or improved beneficiary recruitment

• Strategic impact: the extent to which you are achieving your mission as defined in your Theory of Change

At each level of impact there is the opportunity to report on the outputs and outcomes of your work. That said, decisions about impact measurement need to be mindful of the capacity of grantees so as not to burden them with reporting requirements which are disproportionate to the funding they receive. 

At Makerble we use our Impact Measurement Methodology to identify the minimal set of metrics you need to measure and devise practical ways to measure them.

We can work with you across the full gamut of impact measurement or we can focus in on the one or two areas which are most important to you. Typically our work splits into these three phases.

1. Outcomes Framework: identifying the outcomes and metrics you need to measure

2. Data Collection: creating a data collection plan and either implementing it ourselves or supporting you to systemise it

3. Reporting: analysing your data, producing reports and giving you access to interactive dashboards that show you your data in real-time so you can use it to enhance decision-making

Contact us to talk through your current approach to impact measurement and explore how you can take your foundation’s impact measurement to the next level.