social impact measurement - case study all alien !

Don Bosco Youth-Net ivzw Naamsesteenweg 37 B-3001 Heverlee Belgium

Tel.: +32(0)16.48.78.80 Fax.: + 32(0)16.48.78.90

Email: [email protected] Website: www.donboscoyouth.net

SOCIAL IMPACT MEASUREMENT Case study: “All Alien! Step 1 Training

Course and its Social Return on Investment”

2

I. Colophon This is a publication of Don Bosco Youthnetwork of Salesian youth work offices and youth organBosco. The international secretariat of Don Bosco YouthCommission, through its “Erasmus+ projects and organisations involved in the field of nonworld.

The secretariat of Don Bosco Youththrough its 'European Youth Foundation '. organisations working on intercultural learning, antiof democracy.

This publication reflects the views only of the author, and the European Commission, nor the Council of Europe cannot be held responsible for any use which may be made of the information contained therein. This publication was written by Mshe was carrying out in Don Bosthis study, please contact Ms. Buthe project please contact info@

Social Impact Measurement

Colophon

This is a publication of Don Bosco Youth-Net ivzw. Don Bosco Youth-Net ivzw is an international network of Salesian youth work offices and youth organisations which work in the style of don

The international secretariat of Don Bosco Youth-Net ivzw is financially supported by the European “Erasmus+ - Youth in Action’-programme. This programme supports youth

rganisations involved in the field of non-formal education in Europe and the rest of the

The secretariat of Don Bosco Youth-Net is also financially supported by the Council of Europe, through its 'European Youth Foundation '. This foundation supports youth projects and youth organisations working on intercultural learning, anti-discrimination, Human Rights and the promotion

This publication reflects the views only of the author, and the European Commission, nor the f Europe cannot be held responsible for any use which may be made of the information

Margaryta Buliy in the framework of an internatiosco Youth-Net ivzw. For detailed feedback on thuliy directly through [email protected]. [email protected].


Net ivzw is an international isations which work in the style of don

Net ivzw is financially supported by the European programme. This programme supports youth formal education in Europe and the rest of the

Net is also financially supported by the Council of Europe, supports youth projects and youth

discrimination, Human Rights and the promotion

This publication reflects the views only of the author, and the European Commission, nor the f Europe cannot be held responsible for any use which may be made of the information

onal internship which he technical aspect of or general question of

3

II. Table of Content I. Colophon ..............................................................................................................................................................2

II. Table of Content .................................................................................................................................................3

III. List of figures, tables, and formulas .................................................................................................................5

IV. List of Abbreviations ..........................................................................................................................................6

1. Introduction .....................................................................................................................................................7

1.1. Methodology ........................................................................................................................................... 7

1.2. Social Impact, Social Outcome, and Social Return ........................................................................ 9

2. Basics of Impact Measurement ................................................................................................................. 11

2.1. Models and Methods for Determining Impact ............................................................................ 11

2.1.1. Historical Background .............................................................................................................. 11

2.1.2. Define a Basis for the Analysis ............................................................................................... 11

2.1.3. Using the Impact Map .............................................................................................................. 13

2.2. Impact Management ........................................................................................................................... 14

2.2.1. Five Steps Framework .............................................................................................................. 14

2.2.1.1. Identifying Scope and Setting Objectives ........................................................................ 15

2.2.1.2. Analysing Stakeholders ........................................................................................................ 16

2.2.1.3. Measuring Results ................................................................................................................. 18

2.2.1.3.1. Input and Output ........................................................................................................... 18

2.2.1.3.2. Outcome ......................................................................................................................... 19

2.2.1.3.3. Indicators ......................................................................................................................... 21

2.2.1.4. Verifying and Valuing Impact .............................................................................................. 22

2.2.1.5. Monitoring and Reporting .................................................................................................. 25

2.3. Value chain / Impact map .................................................................................................................. 26

2.4. SROI ...................................................................................................................................................... 26

2.5. SROI Categorisation .......................................................................................................................... 27

2.6. Sensitivity Analysis.............................................................................................................................. 30


4

3. Case Study: STEP 1 Training Course “All Alien!” ............................................................................... 31

3.1. Identifying scope and setting objectives ........................................................................................ 32

3.2. Stakeholder Analysis .......................................................................................................................... 36

3.3. Limitations of the Analysis ............................................................................................................... 38

3.4. Important facts of investigation (Impact) ...................................................................................... 40

3.5. Impact at a glance ............................................................................................................................... 43

3.6. SROI calculation ................................................................................................................................. 43

3.7. Sensitivity Analysis.............................................................................................................................. 44

3.8. The Impact Map of All Alien! Training Course ........................................................................... 46

4. Conclusion .................................................................................................................................................... 49

V. Annex: Exercises .............................................................................................................................................. 51

VI. Bibliography ....................................................................................................................................................... 55

5

III. List of figures, tables, and formulas Figure 1: Logic Model with Consideration of Gross and Net Effects ........................................................... 12 Figure 2: Five Steps Framework of Impact Measurement ............................................................................... 14

Figure 3: Stakeholder according to Accountability ........................................................................................... 17 Figure 4: Internal vs. External Focus: The Use of Outputs, Outcomes, or Impacts ................................. 19 Figure 5: Chain of Events: Defining distant Outcomes .................................................................................... 20 Figure 6: The Generation of Blended Value ....................................................................................................... 27

Figure 7: The Four Stages of a nef-SROI Analysis ............................................................................................. 28 Figure 8: Stakeholder Modell of the xit-SROI .................................................................................................... 30 Figure 9: Stakeholder Map All Alien! .................................................................................................................... 36

Table 1: Objects Covering the Scope of Analysis ............................................................................................. 15 Table 2: Stakeholder Matrix ................................................................................................................................... 16 Table 3: Types of Outcomes .................................................................................................................................. 20 Table 4: Types of Indicators ................................................................................................................................... 21

Table 5: Steps of Defining Blended Value ............................................................................................................ 28 Table 6: Ten Steps towards Impact Management.............................................................................................. 29 Table 7: The xit-SROI: five Types of SROI ......................................................................................................... 29 Table 8: Stakeholder Matrix All Alien! ................................................................................................................. 36

Table 9: Stakeholder All Alien! Inclusion / Exclusion ....................................................................................... 37 Table 10: Example of further Impact Generation .............................................................................................. 38 Table 11: Evaluation Online Survey "All Alien!" (Q: 7) .................................................................................... 40 Table 12: Calculation of Impact (New Volunteers) .......................................................................................... 40

Table 13: Evaluation Online Survey "All Alien!" (Q: 8) .................................................................................... 41 Table 14: Calculation of Impact (New Volunteers) considering Deadweight ............................................ 41 Table 15: Evaluation Online Survey "All Alien!" (Q: 32) .................................................................................. 41 Table 16: Calculation of Impact (Additional Income) ....................................................................................... 42

Table 17: Calculation of Impact (Potential Trainers) ........................................................................................ 42 Table 18: Summery of Cash Inflow ....................................................................................................................... 43 Table 19: Calculation of the Net Present Value ................................................................................................ 43 Table 20: Sensitivity Analysis (No New Volunteers) ........................................................................................ 44

Table 21: Sensitivity Analysis (No Additional Income) .................................................................................... 44 Table 22: Sensitivity Analysis (No Impact Generated) ..................................................................................... 45 Table 23: Annex I: Exercises: Identifying Scope and Setting Objectives ...................................................... 51 Table 24: Annex I: Identifying Stakeholders ........................................................................................................ 52

Table 25: Annex I: Impact Map .............................................................................................................................. 54 Formula 1: Net Present Value (NPV) ................................................................................................................... 26 Formula 2: (Net) SROI Ratio ................................................................................................................................. 27

Formula 3: Calculation of SROI for "All Alien!" ................................................................................................. 44


6

IV. List of Abbreviations

CoE Council of Europe

DF Discount Factor

EVPA European Venture Philanthropy Association

nef New Economics Foundation

NPO Non-profit organisation

NPV Net Present Value

SROI Social Return on Investment

TC Training course

VTR Volunteer Time Recognition

Introduction

7

1. Introduction Nowadays receiving investments for projects within the social area is not easy. Also the trend towards impact contracts complicates the situation concerning the process of applications and primarily the evidence of returns. Non-profit organisations (NPO) are almost forced to show a transparent and sustainable efficiency of their actions and thus proof that the projects and activities have an impact on the society or at least on their target groups.1 The chance to be funded is hence related to the interests of the stakeholders and the NPO. As soon as their aims resonate with each other, the possibility of getting financial sources, increases. Now it is in the interest of the NPO to show the impact or even a return on this investment and to persuade and impress the funder and also other parties involved into the project. Therefore there are plenty of methods and instruments to measure the social impact or even the social return on investment (SROI). Those are based on the pattern of existing evaluation methods but can be modified and applied according to the activity which should be evaluated.2 To be sure that both investor and investee have the same understanding of the value, it is important to tie down on a joint model for the evaluation. Moreover it has to be a method which also is understandable for all the other stakeholders within the activity. As there is usually no financial return from social projects, the SROI aligns as a method which can be used within the sector. This ratio is an instrument, which is still in development but has gained more importance over the last few years. The main purpose is to show the generation of additional value which arises after or during a social activity. Within this report you can learn about the different methods and their commonalities and especially see how to calculate the SROI. The chapters below are answering three major questions: - How does social impact measurement work? - How can the SROI used as an incentive for social investors? - How high is the social return of a STEP 1 training course accomplished by DBYN?

1.1. Methodology For a better understanding the paper first explains some meaningful terms, which have to be distinguished. It is important to know that impact management or impact in general is not necessarily a social return. The topic impact management consists of different methods including instruments which enable to calculate a return. Due to a variety of methods it is mostly difficult to choose one method. Chapter 2.2.1 ‘Five Steps Framework’ shows a framework which is recognisable in every analysis of this kind. Whereby chapter 2.5 ‘SROI Categorisation’ is representing the different types of the SROI itself, which is part of the general framework. As each of the steps is similar to each other, a summarised model was used for the analysis itself. The summed up process is then applied on a STEP 1 training course of DBYN in chapter 3 ‘Case Study: STEP 1 Training Course “All Alien!”’. For the analysis it was necessary to ask the participants

1 Cf. Smith, p. 133 2 Cf. Rauscher, p. 5


8

to take an online survey. The answers were helpful to indicate to which extent participants gained new knowledge and also which impact the training course had on them. Some of the answers could also be assigned to the member organisations, which is mainly the target group of DBYN. The entire investigation is based on the principles of the logic model, which can be found in chapter 2.1.2 ‘Define a Basis for the Analysis’. It starts with the aims and reasons of motivations for the research, followed by the analysis of stakeholders. Thereupon the input, the output, the outcomes and the impact was determined. By allocating indicators and thus validating the results, it was possible to generate financial values, which enabled us to calculate a social return. Chapter 0 ‘

Introduction

9

Limitations of the Analysis’ explains why the return might have been higher than it actually was and shows that, in case member organisations are going to follow that methodology, other facts are important to consider, depending on the situation and the social issues of the members’ country or region. Afterwards a sensitivity analysis was done, to prove how essential the different impacts are for the return itself and that the return is dependent on several factors. Moreover it shows the fluctuations of the SROI by changing impacts or even inputs. In the end all the results were summed up in an impact map and are evident at a glance.

1.2. Social Impact, Social Outcome, and Social Return Before explaining and showing the procedure of how to measure projects, it is necessary to understand the concept and the difference between social impact, social outcome and social return. All three terms refer to the measurement of social projects. - Social Impact: Social impact is usually understood as the biggest and most important

achievement out of activities which are implemented by social entrepreneurships, venture philanthropist and other social non-profit organisations. Consequently social impact is an important indicator for funders, managers but also for the service providers in order to create a better understanding of their projects, and to foster the communication and motivation within the organisation.3

- Social Outcome: To measure outcomes it is important to understand which environmental changes an organisation is going to achieve through projects and other activities. That means that outcome refers to the changes within the target population, or in general the target stakeholder. The results can be either positive or negative.4

- Social Return: Social Return on Investment (SROI) is a rate which shows the value that was created by organisations. In other terms it illustrates social objectives in form of financial and non-financial degrees. It is a comparison of the value which were achieved by the project and the investments which have been done in order to implement this project.5

3 Cf. Rauscher, Schober, Millner, p. 4 4 Cf. Mildenberger, Münscher, Schmitz, p. 290 5 Cf. Lawlor, Neitzert, Nicholls (2008), p. 4

Basics of Impact Measurement

11

2. Basics of Impact Measurement

2.1. Models and Methods for Determining Impact

2.1.1. Historical Background Since the 60’s the government decided to shift the focus of their funds toward a performance achievement. It was clear that non-profit organizations (NPO) would have to develop new strategies and methods in order to show the extent of the change they desire to reach with their activities. In the beginning the process of the projects was more important, until finally performance-contracts were introduced to the social sector in the 70’s.6 Although, both, funders and NPOs were challenged by those contracts, as they haven’t been able to track the performance outcomes in a proper way. But another decrease in using performance contracts could be identified in the mid 90’s. The government decided to improve this strategy and also NPOs were willing to involve strategies for addressing public problems.7 Overall, a lot of methods and models were developed in the last years. Some are more used at the for-profit sector, while some are more suitable at the social non-profit sector. Depending on the purpose of analysis an organisations can use different methods. E. g. benchmarking, which was established in order to compare organisations amongst different organisations, is very often used by NPOs as it mainly requires easy obtaining information. Another example is the balanced scorecard, which is “a strategic-planning tool that non-profit agency can strive to create a more efficient and effective organisation while at the same time remaining faithful to its mission.”8 Both of the above mentioned instruments won’t be used in our case, as they are not specific enough for determining outcomes and impacts. Our purpose of doing such an analysis is to see the performance of the activity. Methods such as the logic model, theory of change and social return on investment fits better into our scope, as we focus on measuring the success and thus the impact of projects and organisations.

2.1.2. Define a Basis for the Analysis The logic model, which is also well known as the impact value chain, is a process which describes the programme including the goals and the instruments for achieving them. In other terms it is a theoretical approach which guides the structuring and consequently the measurement of a social activity.9 In many countries, almost worldwide, non-profit organisations (NPO) need to include a logic model into their grant application, as the funders prefer to invest into sustainable programmes. All mapped factors can be included and summarised into a casual framework, which provides an actual working model. It has the advantage that organisations start to think about adequate commitment of resources and also about the impact and the outcomes they cause through their projects, which makes it easier to focus on their achievement. 10

6 Cf. Rathgeb Smith, p. 133 7 Cf. Rathgeb Smith, p. 134 8 Direct citation cf. Rathgeb Smith p. 135 9 Cf. Rauscher, Schober, Millner, p. 5 10 Cf. Rathgeb Smith, p.136

12

It is common that the logic model is being confused with the theory of change. In general, both have an identical framework, which consist of the factors, which Consideration of Gross and Net Effectsevaluation which won on importance strategies, which are helpful for reaching is seen more as a process or a product and can be used for the practical implementation. That meait is not only about theory, but every factor needs to be verified and valued onresearch.11 Summed up we can make the difference as follows:- Logic model: theory based approach, focuses

programme and how is it structured- Theory of Change: empirical

implementation � How to reach the goals and what are the preconditions?

Figure 1: Logic Model with

As the basis of the model is usually similar in every case,depending on the kind of project, the focus area or the degree of detail.Consideration of Gross and Net Effectsthere is the opportunity to prove assumptions and hypothesis and chosorder to reach the pre-set aims.14

Terms and conditions: Factors such as economic, political, or social relationships form terms and conditions and thus need to be considered.project is required. Afterwards we have to think about our target groups and try knowledge, values, needs, etc. Wresources (cf. chapter 3.1. ‘Identifying scope and setting objectivesexplain an organisation’s structure including information about the legal form and the structure.

11 Cf. Rauscher, Schober, Millner, p. 7 12 Source: Translated from German, Cf. Rauscher, Schober, Millner, p. 613 Cf. Rauscher, Schober, Millner, p. 5 14 Cf. Taplin, Clark, p. 1


It is common that the logic model is being confused with the theory of change. In general, both have an identical framework, which consist of the factors, which can be seen in Figure Consideration of Gross and Net Effects’. On the one hand the logic model is a theory based evaluation which won on importance since the 90s. It helps to win funders, by providing new strategies, which are helpful for reaching organisational goals. On the other hand

as a process or a product and can be used for the practical implementation. That meait is not only about theory, but every factor needs to be verified and valued on the

Summed up we can make the difference as follows: theory based approach, focuses on programmatic performance

programme and how is it structured? empirical approach which focuses on programme planning and

How to reach the goals and what are the preconditions?

: Logic Model with Consideration of Gross and Net Effects

s usually similar in every case, some adjustments are always depending on the kind of project, the focus area or the degree of detail.13 Figure Consideration of Gross and Net Effects’ shows a standard structure of such a model.

the opportunity to prove assumptions and hypothesis and chose the best procedures in 14

Factors such as economic, political, or social relationships form terms and conditions and thus need to be considered. This means a description of the entire

. Afterwards we have to think about our target groups and try knowledge, values, needs, etc. Within the section of input we mainly regard the financial and human

Identifying scope and setting objectives’). Last but not least we have to organisation’s structure including information about the legal form and the

Source: Translated from German, Cf. Rauscher, Schober, Millner, p. 6


It is common that the logic model is being confused with the theory of change. In general, both have Figure 1: Logic Model with

he logic model is a theory based since the 90s. It helps to win funders, by providing new

goals. On the other hand a theory of change as a process or a product and can be used for the practical implementation. That means

the basis of empirical

on programmatic performance � What is the

programme planning and

Consideration of Gross and Net Effects12

ome adjustments are always required, Figure 1: Logic Model with

shows a standard structure of such a model. On its basis e the best procedures in

Factors such as economic, political, or social relationships form terms and a description of the entire context of the

. Afterwards we have to think about our target groups and try to investigate their regard the financial and human

not least we have to organisation’s structure including information about the legal form and the financial


13

Plan and Implementation: The concept is giving information about the responsible for every task and thus we can define when, how, and with which activity we will achieve the desired goals. Afterwards the process description should enable us to see how tasks were implemented and which changes we have to make in order to upgrade the entire cycle. Results: The result section is the widest and also the most detailed one. - Outputs: Outputs are the tangible products and services which were produced within an activity

(e. g. new volunteer, produced campaigns, potential trainers, etc.). - Outcomes: Outcomes describe the changes, benefits, and learning process which we tried to

reach through our activities. In general it is the condition in which we can find the service user (e. g. participants of the training course) after our programme (e. g. volunteers are going to be active for the next 5 years).15

To be able to define the outcomes it is necessary to evaluate the available outputs either through logical, theoretical or empirical methods. As there are outcomes or outputs which would have been happened anyway, disregarding our projects, we have to identify those factors and deduct them from the outcomes. This enables us to see which impact was created purely from our projects or sessions. As a result we receive the impact.16 Side Effects: Side effects can be either positive or negative and arise if we reach something what initially was not planned. Chapter 2.2.1.’Five Steps Framework’ gives a closer description of the above mentioned process.

2.1.3. Using the Impact Map Once a logic model and/or the theory of change are set up, we can project it into an impact map. This tool helps to interlink our mission, our objectives and our activities with their results. It is divided into two sections. The first one should include information about the organisation and the scope of our analysis. The second section is mapping the process of impact measurement and recording everything what has been worked out and valued during the analysis. The best approach is to fill in the map step by step according to the steps of the entire process.17 A blank Impact map can be found in Annex I: Exercises.

15 Cf. Hehenberger, Harling, Scholten, p. 24 16 Cf. Rauscher, Schober, Millner, p. 6 17 Cf. Lawlor, Neitzert, Nicholls (2012), p. 30

14

2.2. Impact Management The following chapter describesmeasuring process. The steps are based on the research of the European Venture Philanthropy Association (EVPA). However framework for the calculation of social returna step of the calculation itself. Neverthelessfactors of SROI determination, which bases

2.2.1. Five Steps Framework The most methods applied to measure the social return of a social project In this case we can rely on a framework which is noticeable in every of steps will guide an organisation in measuring impacts are always important to consider.The five steps are:19

Figure

18 Cf. Hehenberger, Harling, Scholten, p. 2119 Cf. Hehenberger, Harling, Scholten, p. 27 ff.20 Source: own research based on Cf. Hehenberger, Harling, Scholten, p. 23

4. Verifying

and valueing

5. Monitoring

and reporting


Impact Management

s the methods and investigates the most important steps within measuring process. The steps are based on the research of the European Venture Philanthropy

. However those are aligned with the measurement of social impact. The framework for the calculation of social return on investment (SROI) is similar, except

calculation itself. Nevertheless, the entire structure was extended by the important factors of SROI determination, which bases itself on the method designed by the SROI Network.

ramework

st methods applied to measure the social return of a social project are similar to each other. In this case we can rely on a framework which is noticeable in every of these instruments.

tion in measuring impacts on their projects and show them which factors important to consider.18

Figure 2: Five Steps Framework of Impact Measurement20

Cf. Hehenberger, Harling, Scholten, p. 21 Cf. Hehenberger, Harling, Scholten, p. 27 ff. Source: own research based on Cf. Hehenberger, Harling, Scholten, p. 23

1. Identifying

scope and

setting

objectives

2. Managing

stakeholders

3. Measuring

results

4. Verifying

and valueing

impact

5. Monitoring

and reporting


the most important steps within the measuring process. The steps are based on the research of the European Venture Philanthropy

the measurement of social impact. The , except that it includes

the entire structure was extended by the important the SROI Network.

are similar to each other. instruments. The five

and show them which factors


15

2.2.1.1. Identifying Scope21 and Setting Objectives Within this step we have to define the scope of our measurement. One major element of setting up the scope is defining objectives. The scope itself is giving all the information which we have to consider before applying the methods on a project. In order to define the scope it is important to know the: Purpose: What is the purpose of the analysis? Audience: For whom is this analysis important? Background: Here we analyse once more the aims and objectives of our organisation and

to which extent we are able to achieve changes with our activities. Resources: Which resources do we need and how many of each? Responsible: Who will carry out the analysis? Focus area: Are the specific activities we want to measure or will we concentrate on all of

them? Time span: Is it only one project we are measuring, which lasts for a specific period of

time, or do we apply the analysis for one accounting year? Type of analysis: Either we are choosing to calculate a forecast or an evaluation.

Table 1: Objects Covering the Scope of Analysis22

In order to facilitate the first analysis, it is recommended to keep the scope as small as possible, but to describe everything clearly and detailed. After understanding the aim and gathering more experience and knowledge, you will realise that after the analysis the developed scope will need some adjustments.23 Setting objectives however should be a well-known process within organisations. The objectives which we have to establish for the impact measurement have to align to the process itself. In general we can revert to the “SMART” concept, which means that the objectives have to fit into a specific framework (Specific, Measurable, Assignable, Realistic, and Time-related). However the following questions need to be asked in order to clarify the point of view and give the right direction for objectives and aims. - What is the social problem? It must be clear which kind of social problem or obstacle we are trying to overcome. As every organisation is working in different areas and locations, the base case will vary as well. The bigger the target group, the more difficult it will be to define the social problem. In general the following information is required:

• Nature and magnitude of the social issue • Target population • Development of the problem over the years

All this information will enable us to determine the base case which later is going to depict the changes we are able to achieve within the society or our target groups.24 - How is the activity planned? An ideal case here would be a plan which includes a detailed description of the activities, their effects and the way activities are linked to each other. 21 Cf. Lawlor, Neitzer, Nicholls (2012), p. 18 f. 22 Source: own research based on cf. Lawlor, Neitzert, Nichols (2012) p. 18 f. 23 Cf. Lawlor, Neitzer, Nicholls (2012), p. 19 24 Cf. Hehenberger, Harling, Scholten, p. 32


16

- What is the input? It contains everything what was needed for planning, implementing as well as the following procedures of the activities. Therefore information such as time, technology, equipment, and other asset data is very important to track. A list of all needed resources will also provide insights in order to evaluate mismatches and overlapping between activities and their required resources and improve the processes. - What are the expected outcomes? In order to show the funder and the participants that the project will have impact, it is necessary to describe the expected outcome in advance. The easiest way to do it is to develop a plan with milestones and organise the outcomes on a scale i.e. from short-term up to long-term outcomes etc.25

2.2.1.2. Analysing Stakeholders To be able to analyse and to engage stakeholders into our analysis, first we need to clarify the term. Stakeholders are known as people or organisations that can affect or are affected by the project or the activities of the service provider.26 Stakeholders can be identified in two different ways. There is the opportunity to consider key stakeholders and excluded stakeholders by dividing them into categories (i). Another possibility is to consider the concept of accountability and follow a more deep way of mapping stakeholders (ii). (i) The Stakeholder Matrix: Direct Indirect Contributor Direct contributor, e.g.

employees Indirect contributor, e.g. families of participants

Beneficiary Direct (positive) beneficiary, e.g. participants as the focus of the training course

Indirect (negative) beneficiary, e.g. the target groups the volunteers are going to work with

Table 2: Stakeholder Matrix27

While some of our stakeholders will benefit from the project, other will contribute in different ways (in-kind or financially). All of the stakeholder can be considered as direct or indirect and as beneficiaries or contributors. Beneficiaries can positively or negatively be affected by impact and contributors can enhance or decrease impact.28 After listing all possible parties which affect or are affected by the activity it is indispensable to make a difference between key stakeholders and excluded stakeholders. The distinction between both is that key stakeholders usually have experienced material outcome out of our projects.29 By choosing key stakeholders we will have the risk that eventually we need to change and to adjust our expected outcomes. For all cases it is important to report the reduction of the scope and also to give a decent reason for excluding or including parties.

25 Cf. Hehenberger, Harling, Scholten, p. 33 26 Cf. Lawlor, Neitzert, Nicholls (2008), p. 14 27 Source: own research based on Hehenberger, Harling, Scholten, p. 40 28 Direct citation cf. Hehenberger, Harling, Scholten, p. 38 29 Cf. Lawlor, Neitzert, Nicholls (2012), p. 21

Basics of Impact Measurement (ii). Concept of Accountability:

Figure

For a more accurate mapping of stakeholder we can Depending on their extent of accountability some stakeholders may appear more important than others for the organisation. In other terms we have to decide for which outcomes we want to be obliged. E.g. are we responsible foThe different levels, shown in Figure accountability between the parties. E.g. point three, “Accountable for material but only positive outcomes on main beneficiary groups and analysing these for subgroups” means that we would consider the participants which we trained but also The fact that stakeholders can be divided in categories will facilitate the analysis. Besides it is recommended to concentrate on between five and ten stakeholders and to take care of thethroughout the entire process. In case of problems identifying key stakeholders there are two questions which help to reduce the number: - How material are the benefits and inputs provided by these stakeholders?- How relevant is the stakeholder group to my primary mission? Within this stage most of the organisations forget that the outcome of our chosen stakeholders has to be linked to the activities which are producing this outcome. This shows again that we have to choose stakeholders which also might not be that important project.32

30 Source: Cf. Hehenberger, Harling, Scholten, p.4331 Cf. Hehenberger, Harling, Scholten, p. 4232 Cf. Lawlor, Neitzert, Nicholls (2012), p. 23


Figure 3: Stakeholder according to Accountability30

For a more accurate mapping of stakeholder we can refer to their accountability.

Depending on their extent of accountability some stakeholders may appear more important than others for the organisation. In other terms we have to decide for which outcomes we want to be obliged. E.g. are we responsible for intended outcomes or for all outcomes on our stakeholders?

Figure 3: Stakeholder according to Accountabilityaccountability between the parties. E.g. point three, “Accountable for material but only positive outcomes on main beneficiary groups and analysing these for subgroups” means that we would consider the participants which we trained but also the support they got from their parents.

The fact that stakeholders can be divided in categories will facilitate the analysis. Besides it is recommended to concentrate on between five and ten stakeholders and to take care of the

In case of problems identifying key stakeholders there are two questions which help to reduce the

How material are the benefits and inputs provided by these stakeholders? How relevant is the stakeholder group to my primary mission?

Within this stage most of the organisations forget that the outcome of our chosen stakeholders has to be linked to the activities which are producing this outcome. This shows again that we have to

stakeholders which also might not be that important for the organisation but for the

g, Scholten, p.43

Cf. Hehenberger, Harling, Scholten, p. 42 Cf. Lawlor, Neitzert, Nicholls (2012), p. 23

17

refer to their accountability.

Depending on their extent of accountability some stakeholders may appear more important than others for the organisation. In other terms we have to decide for which outcomes we want to be

on our stakeholders?31 Stakeholder according to Accountability’, is illustrating the

accountability between the parties. E.g. point three, “Accountable for material but only positive outcomes on main beneficiary groups and analysing these for subgroups” means that we would

the support they got from their parents.

The fact that stakeholders can be divided in categories will facilitate the analysis. Besides it is recommended to concentrate on between five and ten stakeholders and to take care of them

In case of problems identifying key stakeholders there are two questions which help to reduce the

Within this stage most of the organisations forget that the outcome of our chosen stakeholders has to be linked to the activities which are producing this outcome. This shows again that we have to

for the organisation but for the


18

2.2.1.3. Measuring Results Before measuring the results we have to identify our inputs and outputs. Both need to be valued in a proper way, as they build the basis for the calculation of the social return.

2.2.1.3.1. Input and Output The input, as already mentioned, includes every kind of resources which were required and used for the preparation and the execution of the project. In general we distinguish between human, capital, and in-kind resources. We have to identify all of them. It means we have to know how much money we put into the project, also how many trainers or support people were needed and value their working hours. In-kind resources e.g. presents or materials can be monetised by the assignment of the market value. Important is that in the end we have the entire costs / inputs for delivering our service. Amongst the inputs we have a section of non-monetised inputs, which usually are volunteer time and contributions of goods and services.33 For valuing volunteer time, it is important to consider if any stakeholders e.g. funders have regulations of valuing it. In our case the council of Europe (CoE) expects to follow the regulation of volunteer time recognition (VTR). Before the calculation you have to be conscious about the regulations. E.g. CoE determines actual costs which are needed to execute voluntary work such as travel costs, food, and accommodation not as a payment. Calculating the “wage” of a volunteer will be different for each organisation as you have to consider the minimum wage of the country in which the activity is taking place. As important as the latter we have to adhere to the wages of the employees within the organisation and ideally calculate the costs for a replacement. Finally a reasonable, clear, and transparent calculation system should be the result of this operation.34 For contribution of items, unless otherwise agreed, you can choose your own methods. Examples would be35: - Using a fair market value - Rely on past sales of similar items - Follow an online valuation guide36, etc. In case you are not using all listed inputs for the project you have a decision to make; either you add the surpluses to the social value, which would accrue if you spend the surplus, or a reduction of the value of the inputs is needed.37 As explained before, outputs are “the tangible products and services that result from the organisation’s activities38”.

33 Cf. Lawlor, Neitzert, Nicholls (2012), p. 31f. 34 Cf. Council of Europe (2014) 35 Cf. Fishman, 07/08/2014 36 For more details see: http://www.goodwill.org/, http://www.salvationarmyusa.org/ 37 Cf. Lawlor, Neitzert, Nicholls (2012), p. 31 38 Direct citation cf. Hehenberger, Harling, Scholten, p. 46

Basics of Impact Measurement Different than with inputs, for outputs there is no risk of double accountancy as if our stakeholdersor activities have the same outputs, we need just to consider it once in the calculation. For time contribution it is possible to use the same method for calculating time as it was purposed for the input.39 After investigating those two important

2.2.1.3.2. Outcome During the entire process we have to be careful not to confuse outcomes with outputs. Outcomes are “the changes, benefits, learning or other effectsfrom the organisation’s activities As the following chart is showing inputs and outputs are direct linked to an organisation’s activities. In other terms we generate outputs internally. On the other hand we haveimpact, which both are results of a changing of a specific situation, both positive and negative changes. That means that those factors are linked to the effects we will have by executing our activities. Internal factors are easy to morganisation, other than external factors, which require closer investigations.

Figure 4: Internal vs. External Focus: The Use of Outputs, Outcomes, or Impacts

Important within this stage is that by determining outcomes they have to be related to the right stakeholder and also the change for stakeholders must be provedthing it is recommended to revert to the theory of change

39 Cf. Lawlor, Neitzert, Nicholls (2012), p. 3340 Direct citation cf. Hehenberger, Harling, Scholten, p. 4641 Cf. Hehenberger, Harling, Scholten, p42 Source: Cf. Hehenberger, Harling, Scholten, p. 4743 Cf. Lawlor, Neitzert, Nicholls (2012), p. 33


Different than with inputs, for outputs there is no risk of double accountancy as if our stakeholdersor activities have the same outputs, we need just to consider it once in the calculation. For time contribution it is possible to use the same method for calculating time as it was purposed for the

After investigating those two important factors we need to define outcomes.

During the entire process we have to be careful not to confuse outcomes with outputs.

Outcomes are “the changes, benefits, learning or other effects (both long and short term) that result from the organisation’s activities40”.

As the following chart is showing inputs and outputs are direct linked to an organisation’s activities. In other terms we generate outputs internally. On the other hand we have the outcomes and the impact, which both are results of a changing of a specific situation, both positive and negative changes. That means that those factors are linked to the effects we will have by executing our activities. Internal factors are easy to measure for us, due to we have the needed dataorganisation, other than external factors, which require closer investigations.41

: Internal vs. External Focus: The Use of Outputs, Outcomes, or Impacts

Important within this stage is that by determining outcomes they have to be related to the right the change for stakeholders must be proved. In order to measure the right

thing it is recommended to revert to the theory of change43.

Cf. Lawlor, Neitzert, Nicholls (2012), p. 33 Direct citation cf. Hehenberger, Harling, Scholten, p. 46 Cf. Hehenberger, Harling, Scholten, p. 47 Source: Cf. Hehenberger, Harling, Scholten, p. 47 Cf. Lawlor, Neitzert, Nicholls (2012), p. 33

19

Different than with inputs, for outputs there is no risk of double accountancy as if our stakeholders or activities have the same outputs, we need just to consider it once in the calculation. For time contribution it is possible to use the same method for calculating time as it was purposed for the

During the entire process we have to be careful not to confuse outcomes with outputs.

(both long and short term) that result

As the following chart is showing inputs and outputs are direct linked to an organisation’s activities. the outcomes and the

impact, which both are results of a changing of a specific situation, both positive and negative changes. That means that those factors are linked to the effects we will have by executing our

, due to we have the needed data at the

: Internal vs. External Focus: The Use of Outputs, Outcomes, or Impacts42

Important within this stage is that by determining outcomes they have to be related to the right . In order to measure the right

20

The following example will show the difference between an output and an outcome:

“..., if a training programme aims to get people into jobs then completion of the training itself is an

There are three existing types or outcomes …change: here we concentrate on the change in behaviour, skills, knowledge or the

attitudes of our focus group…targets: this outcome specifies…benchmarks: this kind of outcome

different periods of time or organisations

Not all chosen outcomes are going to happen directly after an activity. That is why it is important to consider intermediate and long term outcomes (cGross and Net Effects). This means that whenever a new outcome appears we have to find out if it is actually a new one or just a follow up from other outcomes. The following example of a ‘chain of events’ will give a better understanding of making the difference between the ups of one output.

Figure 5

Some outcomes will not occur as planned or are just not that Consequently we have to select them and to rank them by importance for us, the impact and the stakeholders. For the selection we can follow the questions below:- Which are the most important to achieve?- Which have outcome material?- Which are the most useful? - Which are the most feasible?

44 Direct citation cf. Lawlor, Neitzert, Nicholls (2012), p. 3345 Cf. Hehenberger, Harling, Scholten, p46 Source: own research based on cf. Hehenberger, Harling, Scholten, p. 5147 Source: own research based on cf. Lawlor, Neitzert, Nicholls (2012), p. 35

Activity

•Creativity

session

Output

•publication

magazine,

video report

of the training

course



“..., if a training programme aims to get people into jobs then completion of the training itself is an output, getting the job is an outcome44.”

types or outcomes, which are focused on45:

here we concentrate on the change in behaviour, skills, knowledge or the attitudes of our focus group this outcome specifies the level of our achievement this kind of outcome enables us to compare targets which are linked to different periods of time or organisations

Table 3: Types of Outcomes46

outcomes are going to happen directly after an activity. That is why it is important to diate and long term outcomes (cf. Figure 1: Logic Model with

means that whenever a new outcome appears we have to find out if it is actually a new one or just a follow up from other outcomes. The following example of a ‘chain of

e a better understanding of making the difference between the sustainability and follow

5: Chain of Events: Defining distant Outcomes47

ome outcomes will not occur as planned or are just not that important for our impact. Consequently we have to select them and to rank them by importance for us, the impact and the stakeholders. For the selection we can follow the questions below:

Which are the most important to achieve? Which have outcome material?

Which are the most feasible?48

Direct citation cf. Lawlor, Neitzert, Nicholls (2012), p. 33 Cf. Hehenberger, Harling, Scholten, p. 51 Source: own research based on cf. Hehenberger, Harling, Scholten, p. 51 Source: own research based on cf. Lawlor, Neitzert, Nicholls (2012), p. 35

publication

magazine,

video report

of the training

Outcome 1

•procviding

information

for society

and raising

awareness of

the training

course

Outcome 2

•savings for

the

organisation

as we do not

spend money

for additional

employees



“..., if a training programme aims to get people into jobs then completion of the training itself is an

here we concentrate on the change in behaviour, skills, knowledge or the

us to compare targets which are linked to

outcomes are going to happen directly after an activity. That is why it is important to : Logic Model with Consideration of

means that whenever a new outcome appears we have to find out if it is actually a new one or just a follow up from other outcomes. The following example of a ‘chain of

sustainability and follow

important for our impact. Consequently we have to select them and to rank them by importance for us, the impact and the

Outcome 3

•call for new

participants

to participate

in the next

training

course


21

If in the end some outcomes are excluded it is important to document this with a reasonable explanation within a report. It might be useful later as the outcomes vary among stakeholders. The ranking of the outcomes can be carried out by assigning indicators.

2.2.1.3.3. Indicators In order to quantify our outcomes we have to assign indicators to each of them. According to EVPA indicators are “specific and measureable actions or conditions that demonstrate progress towards or away from specified outcomes” and “...that assess progress or regression against specific operational activities49.” As well as the broad amount of outcomes there are multiple amounts of indicators which can be used to express outcomes in numbers. Therefore we distinguish different types50 of indicators: Quantitative vs. Qualitative quantitative indicators are those which result in numbers whereas

qualitative indicators result in verbal form. Direct vs. Proxy direct indicators are very difficult to measure even if they directly

result of an activity. On the other hand proxy51 indicators are easier to establish and to measure, but they are only based on the assumptions referring to their relationship to activities.

Subjective vs. Objective52 It has its benefits to mix up the subjective with the objective indicators, especially as it is better to have more than one indicator. The ideal case is also when both complement each other.

Table 4: Types of Indicators53

During the selection process of indicators we have to choose those which will produce useful data for our analysis. That means we have to prove their quality and integrity. To avoid the collection of not relevant data we have to concentrate on the important and may not necessarily easy things to measure54. Moreover indicators should be comparable with a given norm. In general a good indicator is: - Aligned with the purpose of the organization. - SMART. - Clearly defined and emphasized with evidence. - More than one indicator.55

48 Cf. Hehenberger, Harling, Scholten, p. 52f. 49 Direct citation cf. Hehenberger, Harling, Scholten, p. 47 50 Cf. Mayoux, p. 2 51 „A proxy is a value that is deemed to be close the desired indicator, for which exact data is unavailable.” (Direct citation cf. Lawlor, Neitzert, Nicholls (2008), p. 25) 52 Cf. Lawlor, Neitzert, Nicholls (2012), p. 38 53 Source: own research based on cf. Mayoux, p. 2 and Lawlor, Neitzert, Nicholls (2012), p. 38 54 Cf. Lawlor, Neitzert, Nicholls (2012), p. 39 55 Cf. Hehenberger, Harling, Scholten, p.53f.


22

2.2.1.4. Verifying and Valuing Impact Before valuing the impact and calculate the social return we have to consider five important factors, which provide the information of outcomes which happened anyway and also which clarify the extent of our contribution towards the changes. That is what is meant by the term impact.56 The following factors are important in so far, that if you deduct or add them to the outcomes it will result in impacts. Figure 1: Logic Model with Consideration of Gross and Net Effects, shows that in step four of the logic model the impact which we can attribute to our own activities is dependent on those factors. - Deadweight: It describes the facts which would have happened anyway, even without the provision of our activities. If there is a change which is not referred to our project, it automatically decreases the impact. But impact can also increase, when e.g. we would notice negative consequences, which would have happened without our activities.57 For the calculation of deadweight we can consider references which have been established before by likelihood organisations or revert to benchmarks. Deadweight is expressed as a ratio and in the calculation it is deducted or added to our outcomes.58 - Displacement: Displacements enables to see if some of our positive outcomes had a negative impact on target groups which do not count as our stakeholder, which are not inside our scope.59 Displacement does not necessarily appear in every analysis. In order to avoid double counted outcomes it makes sense to check if there is some displacement taking place within the calculation.60 - Attribution: The attribution shows how and to which extent other organisations or stakeholders were contributing to the observed changes.61 It is also calculated as a percentage and in other terms it proves partly the deadweight with better information or indicators. As a result we can refer deadweight to other parties. There are three ways to approach attribution: 1. Establish experience-based attribution 2. Ask stakeholders 3. Consultation with related organisations As attribution is difficult to define, it is challenging to reach an accurate calculation of attribution we can still use it in order to prove the relevant stakeholders.62

56 Cf. Lawlor, Neitzert, Nicholls (2012), p. 55 57 Cf. Hehenberger, Harling, Scholten, p. 55 58 Cf. Lawlor, Neitzert, Nicholls (2012), p. 56f. 59 Cf. Hehenberger, Harling, Scholten, p. 55 60 Cf. Lawlor, Neitzert, Nicholls (2012), p. 57f. 61 Cf. Hehenberger, Harling, Scholten, p. 55 62 Cf. Lawlor, Neitzert, Nicholls (2012) p. 59ff.


23

- Drop-off: Drop-off refers to the future years after a specific activity and hence the future outcomes and impacts. Within this estimation we are able to forecast the reduction of outcomes as they do not last forever. The percentage is then deducted yearly. Consequently drop-off is calculated for outcomes that last longer than one year.63 - Unintended consequences: If something what we did not consider in forehand, appears as a major effect and is caused through our activity, then it is an unintended result, which also has to be considered during the calculation. By verifying outcomes we will be able to see which outcomes we plan or actually delivering to our stakeholders. 64 EVPA is using the method of triangulation of information by verifying it against other sources and approaches in different ways.65 - Desk research - Competitive analysis - Interviews/focus group In other terms we are comparing the data, which helps to see if the impact we are planning to achieve is really doable. After finishing the first three steps introduced in this guide, we also have to follow up on our activities and prove if we are actually reaching what we planned to achieve, and to which extent we are gaining social impact. Depending on the amount of the stakeholders we defined in the second step, we have to start with the most important and move towards less important and look back to their expected outcomes. This is necessary insofar, that the results we are determining have to be on the same level as the concerned stakeholder. For this we have to prove if reached outcomes are making sense for the stakeholder and if it is relevant for them.66 The above described process is closely interlinked to the process of valuing impact. Valuing is “weighing the benefits versus the costs/sacrifices for the stakeholder67”. Summed up it means that stakeholders are incurring costs in order to create impact for other stakeholders. This can vary in every case of measurement. Valuing the outcomes enables us to define the highest level of impact and hence we can adjust the commitment of our resources and reach even a higher impact than forecasted. Valuing the results can be done in different ways. The main difference is through quantitative (i) and qualitative (ii) methods68.

63 Cf. Lawlor, Neitzert, Nicholls (2012), p. 61 64 Cf. Hehenberger, Harling, Scholten, p. 66 65 Cf. Hehenberger, Harling, Scholten, p. 67 66 Cf. Hehenberger, Harling, Scholten, p. 64 67 Direct citation cf. Hehenberger, Harling, Scholten, p. 65 68 Cf. Hehenberger, Harling, Scholten, p. 67 ff.


24

(i) Quantitative methods exist to be able to monetise outcomes and are divided into perceived value and cost-savings/cost relocation. - Perceived value methods are:

� Perceived value / revealed preference: within this method we are looking up prices for related market goods and base the value on them. Although we have to question both, the willingness to pay for the service and the willingness to accept the outcome, which is in this case a product.

� The value game: With the value game we are able to create different cards with services we can deliver and then ask our stakeholder to rate them in order to find out which one has the most value and for whom. The advantage of this game is that it combines the willingness to pay method and the impact assessment without considering the money effect of the willingness to pay. Hence we can measure the preferences of our stakeholders. In order to allocate a monetised value we can compare the services to surrounding products.69.

- Cost savings methods / stated preferences: Within these methods real finance data is used,

so that information about prevented costs, savings, and changes in financial income is visible. � Prevention cost method: considering costs which are prevented due to a activity, e.g. due to

our creativity workshop we have savings in publication costs. � Travel cost method: those are the costs which people are willing to contribute in order to

receive our service e.g. 30 % of self-contribution for travelling costs plus the participation fee.

� Hedonic pricing model: the achieved change results in a value, which is actually caused through environmental changes e.g. if our funder would provide training courses, then the value of our activity would decrease.

(ii) Qualitative70 methods however are: - Storytelling: the aim of this method is to describe the outcomes of our activities in a specific

and structured way. We are following a specific framework and interviews stakeholders in a special pattern.71 The simplicity of storytelling enables to communicate the value of the outcomes. However it should not be the only method of valuing but can be used as a supplementary instrument.

- Client satisfaction survey: as the name already says this is a method which we can apply to measure the satisfaction of our delivered service. Advantageously we can focus on a specific target group or stakeholder. By using this method we have to make sure that questions are adjusted toward the value creation.

- Participatory impact assessment (focus group): in case we are providing services to illiterate stakeholders, this method plays a big role. Due to this method stakeholders can rank preferences by ordering pictures.

69 More methods are available in the TRASI database: http://trasi.foundationcenter.org/ 70 Cf. Hehnberger, Harling, Scholten, p. 68 f. 71 A guidance on storytelling can be found here: http://www.eldrbarry.net/roos/eest.htm


25

2.2.1.5. Monitoring and Reporting This is the final step of the impact measurement process, which leads us to the monetisation of our outcomes and the reporting of our results to the stakeholders. Both steps are depending on each other, in other terms it means what monitoring is for one stakeholder is already reporting for another one. - Monitoring: This is the process of collecting specific data, which will help to track the performance against the objectives. For this it is important to prove the objectives and their indicators one more time. The collected data must refer to the indicators we set within the 3rd step, as well as the relevant information from the 4th step, which we used for verification and valuation. To be able to work with the data, it must be processed. Once we are done with monitoring we will have the opportunity to identify changes and correct our activities accordingly and hence decide which of the used strategies were useful, which need adjustments and which we shouldn’t use for specific reasons.72 Finally we will see which activities had the impact and for whom this impact is important. - Reporting: Reporting is about the presentation of our results and information. All the identified stakeholders will need their own type of report, concerning the needed information, objectives, and statutory regulations. The biggest obstacle for non-profit organisations is that they have to report in different ways to every funder they have, due to the lack of a standardised reporting system.73 To assure that result are reported in a meaningful way we have to consider that all our calculations and assumptions, as well as indicators are robust and accurate. But also we have to show enough transparency. It is also recommendable to set up a framework74 for the reporting step, as later reports and results can be compared with each other. To facilitate the measurement process for everyone who will do it in future, an inclusion of recommendations and suggestions on improving, e.g. the data collection, is necessary.75

72 Cf. Hehenberger, Harling, Scholten, p. 74 73 Cf. Hehnberger, Harling, Scholten, p. 75 74 A example framework can be found in Annex I: Exercises 75 Cf. Lawlor, Neitzert, Nicholls (2012), p. 74f.


26

2.3. Value chain / Impact map Throughout the entire measurement process we have the opportunity to apply or to develop instruments and methods in order to be able to follow what we are measuring and actually bring all the important information into one single framework. For this an impact map, or also known as the impact value chain, is a clear arranged structure, which gives an overall overview. The impact map has a simple framework: The top section provides information about the organisation and the scope which was determined in step one. Following there is space for the identified stakeholders and unintended changes. The last section provides enough space for tracking the rest such as inputs, outputs, etc.76

2.4. SROI The calculation of the social return is a separate part and is not included into the five steps, however it is a method of impact measurement, which results in quantitative values. Theoretically the calculation fits into the 4th step, as it is partly similar. Independent on the type of analysis, the first step is to map all the outcomes we planned or reached during the project and also to project them into the future. In other terms: show how long outcomes will last. Here we start with impact values which were developed in step four. Important is that the values are adjusted considering deadweight, attribution, drop-offs, etc. This impact has to be set up for every outcome which lasts at least for the coming year.77 After every outcome is valued and adjusted we need to calculate the net present value (NPV) of all the impacts. - Net Present Value The NPV is the sum of present values of all future incomes and pay-outs reduced by the amount of the entire inputs. Present value is calculated by discounting the future cash flows with a suitable discount rate78.79

�� =� ��( + �)� − ��

�

��

Formula 1: Net Present Value (NPV)80

CF cash in flow t year i interest rate I0 inputs in period 0 NPV shows how much more value we will earn by investing into this project rather than in another one. That means by calculating the NPV we have also the possibility to compare investments according to their value creation.

76 Cf. Lawlor, Neitzert, Nicholls (2012), p. 30 77 Cf. Lawlor, Neitzert, Nicholls (2012), p. 66 78 Recommended rates can be found in: HM Treasury (ed.), The green book, Appraisal and Evaluation in Central Government, Treasury Guidance, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220541/green_book_complete.pdf, 18/08/2014 79 Cf. Becker, p. 60 80 Source: based on cf. Becker, p. 60


27

Once NPV is calculated as mentioned above we can approach the social return ratio, which is calculated as follows:

(��)�� = (��)��

Formula 2: (Net) SROI Ratio81

The social return on investment gives us the amount which will return per each invested euro.

2.5. SROI Categorisation The SROI is a number or ratio, established by different companies and therefore there are different ways of analysing and interpreting it. Nevertheless it still expresses the social return generated through a social activity, in every case of calculation. The formula for the calculation is also similar or almost identical in every category of the SROI analysis. - The REDF-SROI: The REDF SROI was established in the 90s.82 Although the idea was based on the process of investment management, nowadays it is better known as impact investing. The major impulse was to analyse which investments are the most profitable. The generation of a blended value was the incentive of investigation and the related ratio was used for decision making processes concerning social investments. Blended value is a sum of the organisational value and the social value a NPO is generating.83 Consequently it is defined as “a business model that combines profitability with social benefit.”84

Figure 6: The Generation of Blended Value85

Figure 6: The Generation of Blended Value’ shows that the basis includes a double accounting system. In other terms it presents the enterprise value and the social purpose value. In case the

81 Cf. Lawlor, Neitzert, Nicholls (2012), p. 68 82 Cf. REDF-Homepage 83 Cf. Reichelt, p. 7 84 Cf. http://financial-dictionary.thefreedictionary.com/Blended+Value 85 Source: cf. Gair, p. 5


28

analyst is having issues allocating numbers to the indicators, this two values must be considered separately.86 The process of measuring a blended value is accordingly divided into two major parts which are again divided into little steps. Part 1: Indicate Value Part 2: Measure the return on investment (ROI) Step 1: Calculate enterprise value Step 2: Calculate social value Step 3: Calculate a mixed value

Step 4: Calculate ROI of enterprise value Step 5: Calculate ROI of social value Step 6: Calculate ROI of the mixed value

Table 5: Steps of Defining Blended Value87 - The nef-SROI The British ‘New Economics Foundation’ (nef) is an organisation which contributed to the development of a SROI analysis. Build upon the concept of the REDF-SROI they adjusted the different steps towards their organisational strategy and thus established a new model of this instrument.88 The difference is that nef focused their process on stakeholders. That means that not the value of the entire organisation is measured but that the analysis maintains specifically on the demanded value of the stakeholder.89 E. g. If the stakeholder is a funder then it would be interesting to calculate the impact in general, but if the report is for a member organisation it may make sense to prove the additional generated value for their target groups, etc. The nef-SROI contains four major stages of which each is divided into separate steps.

Figure 7: The Four Stages of a nef-SROI Analysis90

- Impact measurement according to Franssen and Scholten Franssen and Scholten are mainly using the term ‘impact management’ as they see the importance in not only measuring it, but also adjust the organisation’s strategy so, that the impact or return can either increase or at least stay stable over a specific period of time. But nevertheless they recommend measuring the return also with the SROI method.

86 Cf. Häubl, p. 9 87 Source: cf. Laskowski, p. 7 88 Cf. nef-Homepage 89 Cf. Lawlor, Neitzert, Nicholls (2012), p. 3 90 Source: cf. Lawlor, Neitzert, Nicholls (2008), p. 8


29

According to the authors the analysis has ten stages91 whereby every stage has its separate steps. In any case the ten stages help firstly to set up the best suitable frame for the monetisation procedure, secondly it allows to implement the SROI on different dimensions and at any time. Thirdly it enables a clear understanding and presentation of the result itself. 92

Stage Steps A. Planning 1. Aims of the analysis

2. Stakeholder analysis 3. Scope of the analysis 4. Income vs. Expenditures 5. Establishing the Impact Value Chain

B. Content 6. Defining indicators 7. Develop forecast

C. Reliability 8. Calculate SROI 9. Write the report

D. Continuity 10. Observation Table 6: Ten Steps towards Impact Management93

- The xit-SROI: According to the chapters before, which show the impact measurement as a base case, the generated impact seems to be important and of the same extent for all of the stakeholders. But actually we have more than only the society which benefits from social activities, what means that social return on investment (SROI) should be calculated out of different perspectives. Considering the different target groups we can distinguish five types of SROI. Description SROI 1 Shows the established value of the organisation

Considers incoming and outgoing financial resources from and to the government E. g. subventions vs. Payments for insurance and taxes

SROI 2 Shows the value for the direct beneficiaries Monetary values as outflows to beneficiaries E. g. the reimbursement of 70 % of the travel costs

SROI 3 Shows the value which occurs as result of cost savings Considering different alternatives E. g. savings of publication costs due to creativity workshop

SROI 4 Concentrates on value created on regional level Which impact occurs within our region E. g. the impact of every single member organisation in their region

SROI 5 Evaluates the social impact and the quality of life of beneficiaries Requires financial proxies for estimating the impact E. g. the reduced time of integration will increase the employability of refugees and migrants

Table 7: The xit-SROI: five Types of SROI94

91 Cf. Franssen, p. 152 f. 92Cf. Franssen, p. 143 ff. 93 Source: own research based on cf. Franssen, p. 143 ff. 94 Cf. Schellberg, p. 2 ff.


30

In general the classification is interlinked with the stakeholders of the project as following:

Figure 8: Stakeholder Model of the xit-SROI95

2.6. Sensitivity Analysis Sensitivity analysis is a really important tool which should be applied after every analysis. By changing factors according to assumptions, it creates several scenarios, which shows all the possible changes of the entire results.96 The most important things to prove are97: - Deadweight, attribution and drop-offs - Financial proxies - Quantity of outcomes - Value of non-financial inputs The purpose is to see how much at least we have to change in order to become a 1:1 return. The more we have to change, the less our results are sensitive and the higher our level of confidence will be. By calculating the changes we will be able to see in which ways we can change negative to positive return and vice versa.98

95 Cf. Schellberg, p. 18 96 Cf. Taylor p. 1 97 Cf. Lawlor, Neitzert, Nicholls (2012), p. 69 98 Cf. Lawlor, Neitzert, Nicholls (2012), p. 69f.

Case study: All Alien!

31

3. Case Study: STEP 1 Training Course “All Alien!” The following analysis is made from the view of DBYN, what means that the general target group are our member organisations as well as their volunteers they are sending to complete the training course. This is important to consider, as it leads to some limitations during the analysis. If member organisations are going to follow this model of measurement, it is significant to adjust the process towards their target groups and thus to consider different indicators. More information on this topic can be found in chapter 0. ‘


32

Limitations of the Analysis’.

3.1. Identifying scope and setting objectives99 The measurement of the All Alien! training course (TC) is based on a few reasons:100 - Monitoring: monitoring is an internal process which enables to evaluate the progress of our

network. For members it shows the progress of their non-profit organisation (NPO). As well it provides more transparency within the network as we can align the achievements with our milestones.

- Management tool: with the measurement of the TC we will gain more knowledge in this area and also we will have the opportunity to establish a new management tool, which we can integrate into our strategy and operations. That means it is possible to involve impact management into our policy and thus standardise the process in order to measure different activities.

- Reporting: to be able to provide more information for external stakeholders, especially funders, impact measurement can be used as a communication tool and support the marketing process within the network. At the same time it increases our transparency towards other external stakeholders but also towards our members and partners.

Besides the incentives that are driving the measurement, DBYN has the purpose to measure STEP 1 TC in order to see its impact. Some other reasons are linked to the motivation aspects, which are mentioned above. So is the purpose of communication and marketing connected to reporting which also includes the fact that DBYN wants to improve their funding opportunities.101 The analysis is mainly made for the network, in order to be able to evaluate the training structure. Additionally this example should be considered by our member organisations as they could learn how to measure their own projects, or the organisation for their own reasons and motivations. Partly this research is also made for participants, but it would fit better into the structure of a STEP 2 TC, as this could be included as a part of it. The background of this measurement is easy to define, as DBYN has clear aims and objectives for their services. Our mission statement102 is one of the most important aims and goals we want to reach. Accordingly the mission is partly fulfilled with the provision of TC. But also the visibility of the network, the involvement of young participants, and improving the intercultural involvement are the desired goals of the training structure. Finally we want to reach an international exchange which will teach the participants to evolve and practice their intercultural competences for international youth work (cf. chapter 2.1.2. ‘Define a Basis for the Analysis’ ). The analysis will be undertaken internally as a part of the internship of the current intern. The most important resource for this is time which is provided by the intern. Moreover, the analysis needs some support from participants, trainers, and member organisation which is given through questionnaires and information.

99 As assistance we partly use the question from the Guide of nef (cf. Lawlor, Neitzert, Nicholls (2008)), which are available in Annex I: Exercises 100 Cf. Hehenberger, Harling, Scholten, p. 29 101 As the distribution of grants is moving more towards impact-oriented funding, it is necessary to know the manner and the extent of changes our projects are causing. 102 “We, the partners of Don Bosco Youth-Net, wish to contribute to the total development of all young people by bringing them together through various international activities. We hereby look to broaden the opportunities for these young people who are, through various reasons, excluded. By concrete activities of co-operation, we wish to be a living example of tolerance and mutual understanding. Through our own Salesian identity, we aim to keep the heritage of Don Bosco alive and look to offer a ‘youthful voice’ in the Europe of today. By doing this, we want to make a real difference in the lives of all young people, especially for those most in need.”


33

Our focus points of measurement are the different sessions which were lead by the trainers during the project, as they are crucial during the TC and lead to the achievement of the expected outcomes and thus also to impact and social return. Furthermore the analysis should show the progress of the dissemination and exploitation process and hence we could identify the impact of the TC itself. Summing up we focus on the impact which is caused through the TC including its sessions and the progress of the participants and their impact on their sending organisations. The analysis will be done over a relatively small period of time as it is an evaluative one. Especially as the concentration is on the TC we will consider the concerned week but also the preparation and the rework. Besides the objectives which were set up by the network for the training course we consider following: - Social problem: The TC focused on the topics of migration and intercultural learning. The methods of the non-formal education were based on Human Rights education (HRE) and also applied during the entire TC. The investigation of the sending member organisations (MO) showed that the major of them are active within the area of young people with fewer opportunities with educational difficulties as well as with social or economical obstacles. Hence the our target group is very broad: young people with learning difficulties, early school-leavers, lower qualified youth in combination with young people facing discrimination because of their gender, ethnicity, religion, sexual orientation but also young people with limited social skill, anti-social or risky behaviours, etc. But concerning the topics of the TC we can emphasize the topics migration with the focus on forced migration and refugees, discrimination, and intercultural learning. Not only in Europe but also worldwide migration is a broad topic. In the year of 2010 there have been 214 million people migrating from countries to countries and 7.6 % are classified as refugees which results in 16.3 million people. Out of those, around 7.16 million are children under 18 years. If we take a look at Europe we will see that of the worldwide migration more than one third are migrating within Europe, whereas 1.6 million people of the entire wave are refugees. The issue that the number will increase is not excluded. If we have a look at the past the number of migrants worldwide increased by 42 % from 2000 until 2010. 103 Within this sector we set our target towards forced migration, which is a result of some push-factors, such as war, starvation, human rights abuse, etc. The topic migration is closely linked to discrimination, which is often one of the reasons that complicates the integration process and hence takes away the chance to enjoy human rights.

103 Cf. Brander, Witte, Ghanea, p. 537 f.


34

- Activity plan: The activities of the TC are held in a specific order and sequence, as the aim is to have a flow which interlinks sessions with each other and hence makes the TC comfortable to follow for both, participants and trainers. A good morning and a good night for every day was the responsibility of the different countries. Everyone could motivate each other in the morning and give new energy for the day and in the evening everyone had the chance to reflect on the day, either in a spiritual way or in the way they used to do it. The learning experience of the participants started with an induction game, which gave the participants a feeling of migration by experiencing rules and customs of a “new” country. As a result the exercise built a basis on which we could keep on moving deeper into the topic of migration and human rights. The session ‘prejudices and stereotypes’ was done in order to shape participants’ perception of different types of migration and expand their concept about migration in general. It was very important to clarify the difference between stereotypes, prejudices, and discrimination and to show that everyone in our society is affected by these topics. This was followed by a session on ‘intercultural communication’. By emphasising the barriers of communication, through a game, between two different cultures, we made clear that obstacles and differences in communications have a strong impact on the integration process in a multicultural setting. This session had a strong link to the one before, as the appearances, styles, languages and behaviours are partly stereotypes and not always accepted or understood. With the session on values and its system we wanted to let the participants think about their values and for what they stand in their lives. By introducing the Human Rights framework, we enabled them to refer the rights to their values and hence see that every decision or communication is challenged but also based on values. Consequently it leads to the session of conflicts, which first of all showed the participants how fast they can occur. During the session itself we approached the different methods of solving them. The participants had also the opportunity to create a space in which they practiced their conflict transformation skills, especially their listening skills, rephrasing, reframing and how to focus on the underlying interests and needs. In general we fostered the stimulation of creative thinking for solutions that transcend the conflict and meet everyone’s needs. To not be stuck in between conflicts the session ‘outdoor challenge’ provided the team several activities, which challenges the group working together as an intercultural team. This session lead to a stronger cohesion amongst the participants. They also applied practices which were discussed in former sessions. The follow up of the training course included a short debriefing on the topic of refugees. For this we invited the social worker of the Munich-based ‘Project Life’, which caters for unaccompanied minor refugees. Another part of ‘Europe and Migration’ session was a game which raised awareness about inequality of opportunity and hence connected the life of a refugee to each of the participants. In addition they could develop their imagination and critical thinking. This session was a nice transition to our visit at the concentration camp in Dachau. To visit a piece of same history, the participants were able to reflect on the learning process so far and develop a strong attitude of respect of cultural differences. The entire learning process which was build up during the whole week could be reworked in a creative workshop of their choice. Developing a tool to disseminate the outcomes after the TC was the aim of this session. Within here we also got lots of material output which can be used for publications. The project ended with an extensive evaluation session, in which the participants had the chance to reflect on what their experienced and learned during the week as well to see if their values and opinion about social issues changed and also report about the awareness, their gained during the training course. To be able to fight against social obstacles, we are providing the training course, which will enable young people to develop a better understanding of the above mentioned topics. Those sessions are executed in order to facilitate working with or being a migrant.


35

- Input The input for the project came from various funders and contributors. Mainly we earned financial income in order to be able to execute the project. But also volunteer time was a part of the input. The financial sources we received through: Stakeholder Description Calculation Input Participants (37)

Fees (70 € p.p.) 2.300,00 €

Travel costs (30 %) 30 % of total travel costs for participants

7.185,80104 € * 30 % 1.475,69 €

DBYN Solidarity fund Covering participation fees of 5 participants Covering 30 % of travel costs fort 2 participants from Montenegro

469,41 € DBYN Project fund

Covering entire travel costs for trainers and support staff

8.842,77 €

DBYN Total 9.312,18 € Trainers (4) Volunteer time contribution

valued with the minimum wage of Germany105

370 h * 8,50106 € 3.145,00 €

Chaplain Volunteer time contribution 23 h * 8,50 € 195,50 € Support staff The replacement of an

administrative worker in Belgium without any work experience

1.765,39 €

European Youth Foundation

International Activity Request Application approved 13.500,00€

Total Inputs 31.693,76 € Volunteer time calculation: � Each trainer was volunteering 11,5 h107 per day including the follow-up meetings (3

h) after the sessions. Four trainers attended the activity which last seven days. 322 h

� Also to consider is the preparation meeting where the time per day was 8 h. Only three trainers attended the preparation meeting. The meeting lasted two days.

48 h

� The chaplain, which was there as a trainer as well as for spiritual support attended the preparation meeting (16 h), the activity (only 1 session of 1 h), and he lead the good mornings and good nights (30 min / day = 3 h). In addition he spent 3 h in total, offering a volunteer mass for the participants.

23 h

104 According to the financial report of All Alien! Training course 105 Cf. Council of Europe (2014) 106 Cf. German Law for Minimum Wage, § 1 (2) MiLoG 107 Cf. training programme All Alien!


36

3.2. Stakeholder Analysis Within this step we focus on the stakeholder who had an effect or were affected by the training course. In general we could identify nine stakeholders. Some of them are divided again in smaller groups. The reason for a further is e. g. the structure of the network. The green marked stakeholders are included into the analysis whereby the red highlighted ones are excluded.

Figure 9: Stakeholder Map All Alien!108

Following the stakeholders are distributed within the stakeholder matrix, to see who contributed and which parties benefit from the project. Direct Indirect Contributor - Trainers

- Support Staff - Participants - European Youth

Foundation

- Member Organisations

Beneficiary - Participants - Council of Europe Table 8: Stakeholder Matrix All Alien!109

108 Source: own research 109 Source: own research


37

The following table gives an overview of all stakeholders, describes their role within the project and also gives information about their status of inclusion or exclusion.

Stakeholder Description Included / Excluded

Participants 37 young people from 13 different European countries

Included - contributed participant fees and 30 % of

their travel costs - direct beneficiaries as they were trained

Trainers Four trainers from four different European countries

Included - contributed in kind (volunteer time for

training the participants)

Support Staff General Manager and intern of DBYN, Chaplain for spiritual support

Included - contributed in kind (volunteer time for

management support) European Youth Foundation / Council of Europe

Funder of the training course Included - contributed financial resources - benefit in savings in publication costs

Member Organisations

12 sending European Youth organisations

Included - contributed indirect by sending

participants - benefit in savings in training costs and gain

new volunteers

Migrants, Refugees and youth with fewer opportunities

The general working groups of the member organisations and thus the focus group of the training course

Excluded - for the analysis of change, recordings must

be done over years considering the fact of better integration, savings for the government in sectors of housing, welfare benefits, insurances, etc.

Aktionszentrum Member organisation in Germany, that provided space and materials

Excluded - Due to the fact that the training course

could have taken place somewhere else (deadweight: 100%).

- Cannot be seen as contributor, as all the costs (for materials, rent) occurred during the week were covered by DBYN

Experts

- Expert on Refugees and Migration from “Project Life”

- Employee of the European Youth Foundation

Excluded - Too little extent of contribution as only

provided information about the functionality of their working tasks � no effect on the participants

Table 9: Stakeholder All Alien! Inclusion / Exclusion110

110 Source: own research


38

3.3. Limitations of the Analysis The limitation of the analysis is not really limiting the outcomes and impacts, but more describes facts which have to be measured by the member organisations themselves. As for DBYN it would be impossible to keep track of all the changes caused by the training course in every country a member is positioned. As it was visible at the stakeholder analysis, the group of migrants, refugees, and youth with fewer opportunities was excluded. The reason for not considering those stakeholders is quite simply the records of data. To investigate which savings are done within the country, every member of ours has first to look up which amount of money is really claimed for the help e. g. living support of their own target groups. What must be considered? Following there is an example which shows how a member organisation can approach further analysis. If volunteers are going to execute projects or work face to face with their target groups they could help them as follows: - Finding a job - Develop sustainability - Improve their financial situation - Giving them better orientation and hence increase the time of integration - Develop their language skills - Etc. The above mentioned impacts are transmitted to following savings for cooperative agencies: - Savings in administrative and consulting costs - Economy of basic costs (insurance, living, etc.) due to new income through a job - Higher income in taxes, due to full citizenship tax payments are required - Gaining of image and increasing publicity of helping organisations

Table 10: Example of further Impact Generation111

To find indicators for the monetisation should not be that difficult, as mostly statistics for the above mentioned facts are given. The difficulty could be to prove if there is an existing correlation between the impact of the activity and the savings. But also necessary is to prove if the savings were caused by the acting organisation (cf. 0 ‘



39

Verifying and Valuing Impac’ � Attribution). Another example could be considering the savings of our funders, e.g. savings in publication costs. For this it is recommended to have a personal interview, in order to find out if any savings will occur and to which extent.


40

3.4. Important facts of investigation (Impact) - New volunteers: The following questions were answered in the final evaluation of All Alien! If no, are you considering to be active as a volunteer after this training course in your Youth organizations?

Table 11: Evaluation Online Survey "All Alien!" (Q: 7)112

This question enabled us to calculate the following impact: Description Calculation Impact New volunteers 6 Weekly hours 6,5 h113 Yearly hours 6,5 h / week * 52 weeks 338 h Yearly hours of all volunteers 338 h * 6 2.028 h Evaluation of all hours according to VTR114 �� yearly impact in total

2.028 h * 8,50 € 17.238, 00 €

Table 12: Calculation of Impact (New Volunteers)115

Out of 37 participants 30 were already acting as volunteers in their home organisations. After the training course 6 participants showed high interest in volunteering. This means that our member organisations will have new volunteers. To prove the deadweight and to show the cause for the willingness to volunteer, participants were asked if the training course was the incentive for their decision. Two of the participants decided to volunteer because of the training course. Calculation:

• An average volunteer is active 6,5 h per week in within the 27 European countries.116 Applied to 6 volunteers it is an amount of 338 h for one volunteer within one year. Monetised according to the regulation of CoE (VTR), the financial value is 17.238,00 € (338 h * 6 * 8,50117 €)

• The duration is calculated as follows: The network is looking for volunteers who should not be older than 25 years. On the assumption that the new volunteers will be active until their 25th year the average results in 8 years of volunteering time.

As not all new volunteers were inspired by the training course to be active within their sending organisations, we need to set a deadweight of 66,7 %. The deadweight was calculated by considering how many volunteers were definitely addressed through the project. To be sure that the training course was the incentive of being active as a volunteer, the following question was asked:

112 Source: http://allalien.weebly.com/evaluation.html, p. 4 113 Cf. McCloughan, p. 17 114 Volunteer Time Recognition, cf. Council of Europe (2014) 115 Source: own research 116Cf. McCloughan, p. 17 117Cf. German Law for Minimum Wage, § 1 (2) MiLoG

Number of People Number in %

No 1 14,29%

Yes 6 85,71%


41

If yes, do you consider volunteering just because of the training course?


It enables us to recognise, that three out of six would have volunteered anyways. Finally this leads to a deadweight of 66,7 %. As only two participants were engaged by the training course, it requires a recalculation. Description Calculation Impact Yearly impact before deadweight 17.238, 00 € Deadweight 66,7 % Yearly impact after deadweight 17.238 € * (1-0,667) 5.746,00 €

Table 14: Calculation of Impact (New Volunteers) considering Deadweight119

Summed up it means that the yearly reached impact over the next 8 years is calculated with 5.746,00 €. The reason, why we see it as an impact is that instead of employing a new social worker, member organisations gain a volunteer worker, who they do not have to pay for. Consequently savings can be generated. - Additional income for DBYN through secure earnings of participation fees: In addition to new volunteers, DBYN has the possibility to generate new incomes within the next training courses, which again can be used for the training course itself, and thus have reduced expenditures out of our founds. Therefore the participants were asked the following question: Will you finish the training course structure of DBYN? Number of

People Number in %

yes, I’ll finish the other training courses (STEP 2 and STEP 3) and most possible become an international trainer for DBYN

18 49 %

I’ll only will finish STEP 2 and see after 15 41 % no, I don’t want to continue 4 11 %


Out of 37 participants 15 are willing to pay for a STEP 2 training course, while 18 are ready to take part in STEP 2 as well as in STEP 3.

118 Source: http://allalien.weebly.com/evaluation.html, p. 4 119 Source: own research 120 Source: http://allalien.weebly.com/evaluation.html, p. 59

Number of People Number in % No 3 50,00% Yes 2 33,33% No Answer 1 16,67%


42

Under the assumption that participation fee of 70 € will be asked at every training course, we can estimate the ensuing impact: Description Calculation Impact 15 participants in STEP 2 15 * 70 € 1.050,00 € 18 participants in STEP 2 and 18 participants in STEP 3

18 * 70 € 18 * 70 €

1.260,00 € 1.260,00 €

Total income 3.570,00 € Table 16: Calculation of Impact (Additional Income)121

The reasons for accounting the additional income into the SROI are: - By generating income through participation fees we will have the opportunity to include this

amount into the project and partly reduce our expenditures concerning our funds. That means that the saved money can integrated in other projects or more participants can me supported.

- On the other side the asked amount of subventions could be less than usual, as participation fees are included into input. Therefore other NPOs or even DBYN have the opportunity to apply for other funds, and thus executing further projects and social activities.

Every year all three STEP training courses are taking place. This leads to the additional income in this year, due to STEP 2 in Bratislava and to additional earnings in the year after, due to a STEP 3 training course. - Commitment of trainers, considering volunteer time input: As it is known, after completing the training structure of DBYN, volunteers become trainers and are enabled to operate in training courses. By training 18 participants it means that after the second year DBYN should gain 18 new trainers, which can contribute volunteer time into the projects of DBYN and thus teach and engage other participants to be productive and active within member organisation and find solutions to challenge the social issues. Usually DBYN provides four trainers per course, what means that the 18 new trainers could lead training courses for the next 4,5 years at least once. However this calculation is based on the assumption that new trainers will only be able to lead a STEP 1 training course due to their little experience and that every participant can be has the qualification to be a trainer. The table below shows how much impact can be generated by considering the input of volunteer time from potential trainers. Description Calculation Impact Volunteer time contributed per training course 92,5122 h Volunteer time contributed by 18 potential trainers

18 * 92,5 h 1.665 h

The amount of input evaluated with the German law for minimum wage123

1.665 h * 8,50 € 14.152,50 €

Table 17: Calculation of Impact (Potential Trainers)124

121 Source: own research 122 The contributed hours are taken from the input calculation of “All Alien!” To be more specified, every further project must be measured due to different training courses require different amount of time according to the complexity. 123 The contributed time is weighted with the German minimum wage, as the assumption of contributed time is also taken from the project “All Alien!”. If the SROI will be adjusted every year, the hourly wage must be taken from the country in which further training courses will be taken place to follow the regulations of CoE. 124 Source: own research


43

We consider the volunteer contributed time as the trainers are doing it for a charitable purpose and train and engage new volunteers who could be active within our target group the member organisations.

3.5. Impact at a glance Summed up we are able to generate the following impact: Year 1 2 3 4 5 6 7 8 New volunteers

5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 €

Additional income

2.310 € 1.260 €

Commitment of potential trainers

3.145 € 3.145 € 3.145 € 3.145 € 1.572,50 €

Total impact 8.056 € 7.006 € 8.891 € 8.891 € 8.891 € 8.891 € 7.318,50 € 5746 € Table 18: Summery of Cash Inflow125

3.6. SROI calculation As mentioned before, we have to consider a discount rate. The reason for that is that monetary values are more worth in future than at a present day. According to the ‘Green Book’ we will use the recommended discount rate of 3.5 %.126 Year 1 2 3 4 5 6 7 8 Total return

8.056 € 7.006 € 8.891 € 8.891 € 8.891 € 8.891 € 7.318,50 €

5746 €

Discount factor

1,035 1,071 1,109 1,148 1,188 1,229 1,272 1,317

Net present value

7.738,58 € 6.541,55 €

8.017,13 €

7.744,77 €

7.484,01 €

7.234,34 €

5.753,54 €

4.362,95 €

Total net value

54.876,87 €

Table 19: Calculation of the Net Present Value127

Excursus: Calculation of the discount factor In general the formula for the discount factor can be calculated as shown below:

! = 1(1 + #)$

df discount factor i discount rate t year

125 Source: own research 126 Cf. HM Treasury, p. 26 127 Source: own research


44

The discount factor enables to transform the future value of the money to a previous time period, which in our case is the year in which the training course took place.

After calculating the Net Present Value (NPV) we can use the formula which was given for the SROI calculation to see which value the training course generated per every invested Euro.

%&'( = 54.876,87€31.693,76€ = 1,73 Formula 3: Calculation of SROI for "All Alien!"128

A SROI of 1,73 was calculated. That means that per each invested euro there is social return of 1,73 €. In general such a return is good as first of all it is not negative. On the other hand we cannot compare it to other ratios, as the analysis was made for the first time within the network. Under the assumption that the analysis was limited and only the impact on our member organisations and the network itself was measured, it is clear that by considering every country separately and also research the changes there, we could have reached a higher SROI.

3.7. Sensitivity Analysis Following alternatives where proved within the sensitivity analysis: - No new volunteers - No potential trainers - No impact generated � Alternative 1: no new volunteers In this case we assume that the training course didn’t encourage any volunteers to be active within their sending organisations, which leads to the following SROI: Year 1 2 3 4 5 6 7

Σ 2.310 € 1.260 € 3.145 € 3.145 € 3.145 € 3.145 € 1.572,50 € DF 1,035 1,071 1,109 1,148 1,188 1,229 1,272 NPV 2.231,88

€ 1.176,47 € 2.835,88 € 2.739,55 € 2.647.31 € 2.558,99 € 1.236,24 € 12.686,77 €

Input 31.693,76 € SROI 0,40

Table 20: Sensitivity Analysis (No New Volunteers)129

As the SROI is below one, it means that we are losing 0,60 € per each euro we invest into the training course. Compared to the SROI which was calculated in the chapter before, we see that the encouragement of new volunteers is of high importance. New volunteers in that case are contributing about 77 % to the return.

� Alternative 2: no potential trainers after the training course Year 1 2 3 4 5 6 7 8

Σ 8.056 € 7.006 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € DF 1,035 1,071 1,109 1,148 1,188 1,229 1,272 1,317 NPV 7.783,57

€ 6.541,55

€ 5.181,24

€ 5.005,23

€ 4.836,70

€ 4.675,35

€ 4.517,30

€ 4.362,95

€ 42.903,88

€ Input 31.693,76

€ SROI 1,35

Table 21: Sensitivity Analysis (No Additional Income)130

128 Source: own research 129 Source: own research 130 Source: own research


45

Not having potential trainers after the training course would mean that the participants haven’t been satisfied and thus won’t participate in the follow up trainings. This could also lead to the decreased income which is generated through participation fees and 30 % of travel costs. Still we would gain the additional financial sources by addressing other participants. But on the other side we have to consider, that it means that the income source is not secure anymore and we would most probably need more time to encourage other volunteers to participate in STEP 2 and STEP 3 training courses of DBYN. In general the influence on the SROI by commit new trainers is about only 22 %, which is why the SROI of this alternative is still higher than 1 and means that we would still benefit from the training course. � Alternative 3: no impact generated Year 1 2 3 4 5 6 7 8

Σ 0 € 0 € 0 € 0 € 0 € 0 € 0 € 0 € DF 1,035 1,071 1,109 1,148 1,188 1,229 1,272 1,317 NPV 0 € 0 € 0 € 0 € 0 € 0 € 0 € 0 € 0 € Input 31.693,76 € SROI 0

Table 22: Sensitivity Analysis (No Impact Generated)131

If the activity would not cause any impact we would have a SROI of 0. A SROI of 0 is not a bad sign, as of course we do not have a return, but it is better than having a negative result, which would mean that the training course would have a bad influence on our target groups.



46

3.8. The Impact Map of All Alien! Training Course Impact Map

Scope

STEP 1 training course Target group Member organisations The project received a grant from the European Youth Foundation and partly by the participants

Type of analysis Forecasting analysis

Stakeholder Input Output Outcome

Description Value in € Summery in numbers Change Indicators Source Amount Duration Participants (37) Fees (70 € p.p.) 2.300,00 € 132 Own contribution (travel costs

30 %) 1.475,69 €

DBYN Solidarity Fond 30% travel costs Montenegro paid by DBYN’s solidarity fund Participation fees paid 100% from DBYN’s solidarity fund Participation fees partially paid by DBYN’s solidarity fund

469,41 € 179,41 € 280,00 € 10,00 €

18 potential trainers

Trainers can lead the training courses from the 3rd year onwards New income source for DBYN

Valuation is based on a research paper

Survey

92,5 h 5

18 participants for STEP 2 and 3

18 2

15 participants for STEP 2 15 1

Project fund Project fund - costs Salesian Chaplain Project fund - balance

8.842,77€ 1.259,17 € 7.583,60 €

Trainer (4) Volunteer contributed time valued with the German minimal wage

3.145,00 € n/a

Salesian Chaplain Volunteer contributed time 195,50 € n/a Support Staff Replacement of 2 workers without

experience 3.530,78 € n/a

Member Organisations

6 new volunteers Volunteers will contribute time for the next 8 years

Research and valuation via VTR

Survey and research

338 h 8

European Youth Foundation

Subvention for social activity 13.500,00€

132 Calculation: 32 participants paid 70 €, 1 participant paid 60 €, the rest was paid by DBYN

47

Stakeholder (copy) Outcome (copy) Outcome (cont.) Financial indicator Value in € Deadweight Attribution Drop Off Impact

DBYN 18 potential trainers

Hours and German minimal wage 8,50 €

0 % 100 % 0 % 14.152,50 €

18 participants for STEP 2 and 3

Participation fee 70 €

0 % 100 % 0 % 3.570,00 €

15 participants for STEP 2

Member Organisation 6 new volunteers Hours and German minimal wage

8,50 € 66,7 % 100 % 0 % 5.746,00 €


48

Stake-holder (copy)

Impact (copy) SROI calculation

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8

DBYN 18 participants for STEP 2 and 3

2.310 € 1.260 €

18 potential trainers

3.145 € 3.145 € 3.145 € 3.145 € 1.572,50 €

Member Organisation

6 new volunteers

5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 € 5.746 €

Sum 8.056 € 7.006 € 8.891 € 8.891 € 8.891 € 8.891 € 7.318,50 € 5746 € Discount factor 1,035 1,071 1,109 1,148 1,188 1,229 1,272 1,317 NPV 7.738,58 € 6.541,55 € 8.017,13 € 7.744,77 € 7.484,01 € 7.234,34 € 5.753,54 € 4.362,95 € 54.876,87 €

Input 31.693,76 €

SROI 1,73

Conclusion

49

4. Conclusion After such an analysis we answered the entire research questions, and most important we calculated a positive SROI which theoretically can be used as a decision making tool concerning social investments and subventions. In chapter 3 we calculated a social return of 1,73. For DBYN it means that by the execution of STEP 1 training course “All Alien!” we invested the subvention in a profitable way, even though the SROI could be higher. But to be able to verify a better return, it is up to the member organisations to do a deeper research based on this paper. Another possibility of facilitating the analysis could be the readjustment of the training course towards the measurement process. For example it would be a good idea to prepare the participants beforehand and receive better and detailed data of every country, which could be easily used within the analysis. Afterwards the SROI could be integrated into DBYN’s strategy and one day maybe be used as a performance indicator. In what way members can use the process is up to each organisation, as there are plenty of methods. It is recommended to only use one standardized method within the network, so similar projects can be compared and also the results can be summarized in one report of the entire network. During the analysis we realised that it facilitates the process, if in the beginning there is a definition of the purpose, actually meaning the theory of change, is indispensable. It creates a guide line and thus helps to stay within the scope of the analysis. But even if the organisations decided for a method, it is necessary to agree with the other stakeholders and in particular the investor. In the end all of the parties have to be satisfied with the method and should be able to understand the result. To be sure that the analysis is done right it is also recommendable to ask for expert help or discuss a few points with employees who have a financial background. Especially when it comes to the estimation of cashflows and their discounting advice is of need. The allocation of indicators is not as easy as it seems to be first. Important to consider are those which will have value for the analysis and also could be used for future investigations. Although using same indicators facilitates the analysis, by using them more than once, those should be recorded carefully due to their high importance. Consequently the track of data needs to be done according to a planned procedure. Tracking data is a time consuming process that is why help from investors or even the government should be asked. The different types of the SROI could be calculated in advance, and thus enables us to compare the organisations within the network. Just to see which ones are stable and producing a return and which ones need some support. Nevertheless the target group is also an important factor which should be included into the choice of the right SROI. Next to a social return as a number, a SROI report gives information about the quantitative achievements. This is very useful if the ratio is negative or organisations have a lack of data and cannot calculate the return ratio. If this method is suitable for every member depending on their strategies and investors, but in general the ratio cannot be influenced by the source of funds. But then still members have to decide if they want to include the instrument into their organisational strategy.


50

The sensitivity analysis towards the end of the report shows how dependent the return is on some numbers. And also gives us the possibility to react in time if aims can’t be reached. A recommendation of after every analysis can help others to improve their processes and also adapt them to calculate a proper return. In the future we can assume that the methods of measuring social return will win on importance within the social sector. Working with this methods can already give an insight into this sector and moreover show transparency and sustainability of projects. This gives us the opportunity to show the kind of change and also the amount of change which can be proceed within the society. Now it is up to DBYN to introduce the different methods in the network and to represent the process of gaining new investors. As the network is operating all over Europe, it could be a challenge to provide and implement such a measuring structure, but as the SROI can be practiced in STEP 2 training courses it is still possible. Finally we are generating a high return which consequently is attractive for further investments.

Conclusion

51

V. Annex: Exercises Purpose: What do you want to measure? A specific project, or the impact created

by the entire organisation?

What are the activities for which you want to determine an impact?

For example, if you are looking at a waste management organisation, do you want to measure the impact of a recycling programme or a computer refurbishment operation or both? Describe the activities.

Audience: Describe the intended participants, or target population.

For example, if you are looking at a skills training agency, do you want to track all of the participants, or a smaller segment; for example looking exclusively at subcategories, such as those recovering from mental ill health, or those who are aged between 16 and 24?

Resources: What resources will be required and are these available?

Consider time and funding.

Responsible: Are you independent researcher, or do you work within the project area or organisation you wish to study?

Think through how this will affect the design of the analysis, including implications for resources and time availability.

Who will be responsible for the work? Consider both who will do the research and who will have responsibility for overseeing and managing the project. Will the SROI be carried out in-house or involve an external researcher?

Time Span: What is the timeframe for the analysis? Consider how this will impact on what you are realistically able to measure.

Over what time period will the social return be measured?

For example, will you consider social returns created from the beginning of a project through to its conclusion, or just over one financial year? Often organisation will project returns over the life of the outcome being achieved, so for young offenders this might mean for a life of reduced contact with the criminal justice system.

Table 23: Annex I: Exercises: Identifying Scope and Setting Objectives133

133 Source: own research based on cf. Lawlor, Neitzert, Nicholls (2008), p. 11f. and Lawlor, Neitzert, Nicholls (2012), p. 18f.


52

Analyzing stakeholders: Contributor Who contributes to the project, either

financially or otherwise considering direct and indirect contributors?

For example, funders, staff, volunteers, partner agencies, etc.

Who else either makes the project happen or is affected by it, even if only peripherally?

Beneficiary Who are the direct beneficiaries? For example, if it is a training programme for ex-offenders, the direct beneficiaries would be the ex-offenders.

Who are the indirect beneficiaries? For example, other beneficiaries of reduced reoffending would be the families of the ex-offenders, the wider community, and the State.

Table 24: Annex I: Identifying Stakeholders134

134 Source: own research based on Lawlor, Neitzert, Nicholls (2008), p. 14f.

Conclusion

53

Example format for a report135 Executive summery

1. Scope and stakeholders

• A description of your organisation: its activities and values (if relevant), the activity under

analysis, including location, main customers and beneficiaries. • An explanation of Impact / SROI, whether it is forecasted or evaluative, the purpose and scope of

the analysis. • A discussion of stakeholders e.g. types and numbers. • A description of how stakeholders were involved and the numbers that were consulted.

2. Outcomes and evidence

• A description of the theory of change for each stakeholder e.g. how inputs lead to outputs and

outcomes. This should be presented in a table as well as in narrative form. • Description of the indicators and data sources used for each outcome. Give particular attention

to each outcome and how it is (will be) achieved. • Quantity of inputs, outputs and outcomes achieved for each stakeholder group. • Analysis of the investment required for the activity. The length of time over which the outcome

is expected to last, or against which the outcome will be attributed to the activity. • Description of the financial proxy to be used for each outcome, together with the source of the

information for each proxy and a discussion of the proxies you have chosen. 3. Impact

• Description of the other areas or groups against which deadweight is estimated. • Description of the other organisations or people to which outcomes have been attributed. • The basis for any estimates of attribution and deadweight, flagging up any data gaps and areas for

improvement. • Description of displacement, if included. • The total impact.

4. Social return calculation

• Calculation of the social return, showing sources of information, including a description of the

type or types of social return calculation used. • A description of the sensitivity analysis carried out and why. • A description of the changes to quantities as a result of the sensitivity analysis. • A comparison of the social return in the sensitivity analysis.

5. Audit trail

• Stakeholders identified but not included, and rationale for this. • Outcomes identified but not included, for each stakeholder, and the rationale. • Any financial proxies not included, and the rationale.

135 Cf. Lawlor, Neitzert, Nicholls (2012), p. 82f.


54

Blank Impact Map:

Table 25: Annex I: Impact Map136

136 Source: own research based on Lawlor, Neitzert, Nicholls (2012), section: Resources 11

Bibliography

55

VI. Bibliography Becker, H.: Investition und Finanzierung, Grundlagen der betrieblichen Finanzwirtschaft, 4th edition, Wiesbaden, 2010 Brander, P., Witte, De L., Ghanea, N.: Compass, Manual for human rights education with young people, Strasbourg, 2012 Council of Europe (ed.) (2014): Volunteer Time Recognition (VTR) in projects supported by the European Youth Foundation, Strasbourg, 2014 Fishman, S.: Nolo, Law for all, How to value Noncash Charitable Contribution, If you donate property to a nonprofit, you need to determine how much it is worth when you donate it, http://www.nolo.com/legal-encyclopedia/how-value-noncash-charitable-contributions.html, 07/08/2014 Franssen, B., Scholten, P.: Handbuch für Sozialunternehmertum, Assen, 2008 Gair, C.: A Report From the Good Ship SROI, http://www.socialreturns.org/docs/Good_Ship_SROI_Gair.pdf, 25.07.2014 Häubl, R.: Der Social Return on Investment im Kontext der Präventionsarbeit, Thesis, 2011 Hehenberger, L., Harling, A., Scholten, P.: A practical guide to measuring and managing impact, European Venture Philanthropy Association, Rotterdam / Berlin, 2013 HM Treasury (Hrsg.): The Green Book, Appraisal and Evaluation in Central Government, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220541/green_book_complete.pdf, London, 2011 Laskowski, W., Loidl-Keil, R.: SROI made in Austria, Das Konzept des Social Return on Investment für Österreich gedacht, Graz, 2007 Lawlor, E., Neitzert, E., Nicholls, J. (2008): Measuring Value: a guide to Social Return on Investment (SROI), 2nd edition, London, 2008 Lawlor, E., Neitzert, E., Nicholls, J. (2012): A Guide to Social Return on Investment, 2nd edition, 2012 Mader, M.: Implementation, iooi-Methode (Input – Output – Outcome – Impact) der Bertelsmann Stiftung, http://www.sustainicum.at/files/tmethods/12/en/iooi-Method_Implementation.pdf, Graz, 02/09/2014 Mayoux, L.: What do we want to know? Selecting indicators, 2002 McCloughan, P., Batt, W., Costine, M., Scully D.: Participation in volunteering and unpaid work, Second European Quality of Life Survey, Luxembourg, 2011 Mildenberger, G., Münscher, R., Schmitz, B.: Dimensionen der Bewertung gemeinnütziger Organisationen. In Anheier, H., Schröer, A., Then, V. (ed.): Soziale Investitionen, Interdisziplinäre Perspektiven, Wiesbaden, 2012


56

nef-Homepage: New Economics Foundation, http://www.neweconomics.org/, 25.07.2014 Rathgeb Smith, S.: Nonprofits and Public Administration: Performance Management and Citizen Engagement, The American Review of Public Administration, volume 40, number 2, p. 129-152, http://arp.sagepub.com/content/40/2/129, 2010 Rauscher O., Schober, C., Millner, R.: Social Impact Measurement und Social Return on Investment (SROI)-Analyse, Wirkungsmessung neu?, Working Paper, Wien, 2012 REDF-Homepage: REDF-Investing in Employment and Hope, http://www.redf.org/, 25.07.2014 Reichelt, D.: SROI-Social Return on Investment, Modellversuch zur Berechnung des gesellschaftlichen Mehrwertes, Hamburg, 2009 Schellberg, K.: Der Social Return on Investmen als ein Konzept zur Messung des Mehrwerts des Sozialen, 2010 Smith, S.: The American Review of Public Administration, Nonprofits and Public Administration, Reconciling Performance Management and Citizen Engagement, Volume 40, Number 2, Washington, 2010 Spreckley, F.: Result Based Monitoring and Evaluation, Toolkit, 2nd edition, Herefordshire, 2009 Taplin, D., Clark, H.: Theory of Change Basics, A primer on theory of change, New York, 2012 Taylor, M.: What is...? series, What is sensitivity analysis, http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_sens_analy.pdf, 2009

International network represented in

13 European countries

We, the partners of Don Bosco Youththem together through various international activities. We hereby look to broaden the opportunities for these young

people who are, through various reasons, excluded. By concrete activities of co

Through our own Salesian identity, we aim to keep the heritage of Don Bosco alive and look tothe Europe of today. By doing this, we want to make a real difference in the lives of all young people, especially for those

Austria

Austria

Belgium

Czech Republic

Germany

Ireland

Malta

Poland

Slovakia

Slovenia

Spain

The Netherlands

United Kingdom

International network represented in

13 European countries www.donboscoyouth.net

Mission statement DBYN

We, the partners of Don Bosco Youth-Net, wish to contribute to the total development of all young people by bringing them together through various international activities. We hereby look to broaden the opportunities for these young

people who are, through various reasons, excluded. By concrete activities of co-operation, we wish to be a living example of tolerance and mutual understanding.

Through our own Salesian identity, we aim to keep the heritage of Don Bosco alive and look to offer a ‘youthful voice’ in the Europe of today. By doing this, we want to make a real difference in the lives of all young people, especially for those

most in need.

Austria www.jugendeinewelt.at

Austria

www.donbosco.at

Belgium

www.jeugddienstdonbosco.be

Czech Republic

mladez.sdb.cz/sadba/

Germany

www.aktionszentrum.de

Italy

www.federazionescs.org

Ireland

www.salesians.ie

Malta

www.spysmalta.org

Poland

www.donbosco.pl

Slovakia

www.domka.sk

Slovenia

www.mladinski-

Spain

www.confedonbosco.org

The Netherlands

www.donbosco.nl

United Kingdom

www.salesianyouthministry.com

www.donboscoyouth.net

Net, wish to contribute to the total development of all young people by bringing them together through various international activities. We hereby look to broaden the opportunities for these young

operation, we wish to be a living example

offer a ‘youthful voice’ in the Europe of today. By doing this, we want to make a real difference in the lives of all young people, especially for those

www.jugendeinewelt.at

www.donbosco.at

www.jeugddienstdonbosco.be

mladez.sdb.cz/sadba/

www.aktionszentrum.de

www.federazionescs.org

www.salesians.ie

www.spysmalta.org

www.donbosco.pl

-ceh.si

www.confedonbosco.org

www.donbosco.nl

www.salesianyouthministry.com

social impact measurement - case study all alien !

Documents