tion manual - static1.1.sqspcdn.com

1

Evaluation manual

3

Evaluation manual

4

AcknowledgmentsTwo groups have to be thanked for their help and guidance during the development of this book: the Impact Assessment Taskforce, and the Impact Advisory committee. Our dedicated taskforce is made up of representatives of Partner organisations from four continents and representatives of international strategic organisations. The Impact Assessment Committee is composed of academics and researchers with a range of expertise. All individuals volunteered their time and effort to ensure that this becomes a comprehensive and practical set of tools. We would not have been able to have done this work without this crucial help.

Impact Assessment Committee Ray Boshara, New America FoundationJudith Bruce, Population CouncilLisa Dacanay, Asian Center for EntrepreneurshipGregory Dees, Duke UniversityJohn Elkington, SustainAbilityDean Karlan, Innovations for Poverty Action Lewis Mandell, University of BuffaloLata Narayan, MeljolSara Olsen SVT ConsultingMichael Sherraden, Washington University in St. Louis

Peter Scholten, Scholten & FranssenFred Ssewamala, Columbia University

Impact Assessment Task ForceRammani Acharya, JA NepalPatricia Formadi, WADEP, GhanaJohanna Cloete, JA,NamibiaGilberto Mendez, CCF, USA Ingrid Jones, Partnere Perfemijet, Albania

Zeina Khoury. Jordan River Foundation, Jordan

Copyright 2008. Aflatoun, Child Savings International.This work may be reproduced and redistributed, in whole or in part, with or without alteration and without prior written permission, solely by educational institutions for non-profit administrative or educational purposes providing all copies contain the following statement: Copyright 2008, Aflatoun Child Savings International. This work is reproduced and distributed with permission of Aflatoun, Child Savings International. No other use is permitted without express prior written permission of Aflatoun, Child Savings International. For permission contact: [email protected]

5

Dear Partner,Greetings from the Afl atoun team! We are pleased you will be joining us in our mission to inspire children to become agents of change.

This Afl akit explains the Afl atoun Concept, introduces the Afl atoun Network and provides guidelines for implementing the Afl atoun Programme. The Afl atoun Programme allows maximum fl exibility for the partner to adapt according to their countries’ needs. Afl atoun provides a design through which children can explore their rights and responsibilities in classes or club sessions, take part in money saving activities, and apply their entrepreneurial skills to address the social issues that are important to them.

Afl atoun enables children to do more than just acquire knowledge - the integrative and child-oriented curriculum is designed to help children develop an investigative, proactive attitude and practical skills which they can use for the rest of their lives. The Afl atoun Programme increases children’s sense of control over their own lives, builds their self-confi dence, and prepares them to participate in matters affecting them and their communities as socially and fi nancially empowered citizens.

By implementing the Afl atoun Programme in your country, you are making a very signifi cant contribution towards achieving important international development goals: the UN Convention on the Rights of the Child, which is at the heart of the program and UN’s Millennium Development Goals, through the focus on primary education, responsibility to others and use of partner networks.

Together, we can equip children with the knowledge and skills needed to become change-makers and create a more equitable global society.

Thank you for believing with us that change is also in the hands of children.

Welcome aboard!

The Afl atoun Team

7

Table OfIntroductionChapter 1 - Aflatoun’s Quality Assurance and Impact Assessment Strategy (AQIS)AQIS tools in line with the Theory of Change Principles of AQISChapter 2. Quality Assurance (QA)Partnership ProcessMultiple Partnerships in a CountryParticipatory Evaluation of SecretariatChapter 3. Impact Assessment: Surveys and Partner DatabaseSurvey Overview and MethodologyAnalysisSteps to Collecting SurveysWhen to Conduct the SurveysThe Partner DatabaseChapter 4. Impact Assessment: Other ToolsCollecting Stories and Audiovisual MaterialComplementary Research ProjectsData Collection SummaryQuality AssuranceImpact AssessmentAppendicesAppendix 1- Learning and facilitation visitAppendix 2 - Database manual

Appendix 3 - Suggestions for Training of SurveyorsAppendix 4 - Survey planning guide

910

12141616242426

2627283232333336383839404045

5153

contentscontents

9

Welcome to the Evaluation Manual, your reference manual on Aflatoun’s Quality Assurance and Impact Assessment Strategy (AQIS).

As the Aflatoun network is constantly expanding, partners have expressed a need for safeguarding the quality of the programme and understanding its impact. This is crucial to ensure efficient and effective delivery of the programme. Thus, the tools described in this manual aim to build credibility for the concept of Child Social and Financial Education by showing that the Aflatoun Network can deliver a high quality programme in a cost effective manner. We also want to be able to show the level of impact the programme has on the children who are part of the programme across the world.

The manual outlines two core key processes that are practiced within the Aflatoun Network:

- The Aflatoun Partnership Process- The Aflatoun Partner and Field Surveys

Aflatoun’s Quality assurance and Impact assessment Strategy

(from now on ‘AQIS’) combines a range of innovative and strategic approaches. It has been developed by the Aflatoun Secretariat working actively together with an advisory committee of experts and two partner taskforces – the Quality Assurance Taskforce and the Impact Assessment taskforce. Let us now take a deeper look as to how what Aflatoun’s AQIS system is all about.

introductionintroduction

10

Before we can begin discussing how Aflatoun’s AQIS system measures programme effectiveness, we first have to understand what the desired outputs and outcomes are.

On the next page you find a visualization of the change the Aflatoun programme aims to bring about – our Theory of Change. Aflatoun hopes to establish change in two ways:

1. Providing a high-quality Child Social & Financial Education (CSFE) curriculum, through the programme’s five core elements. Through the curriculum, children develop competencies and start to change related behaviour, which results in their broader social and financial empowerment.

2. Engaging in global, regional, national and local advocacy. Aflatoun advocates the inclusion of CSFE in national curricula, encourages child-friendly banking and stimulates the development of child participation systems. This is done through collaborations with different institutions such as central, commercial and microfinance banks, educational departments and multilateral institutions.

The diagram below illustrates Aflatoun’s Theory of Change by breaking it down into four steps: Curriculum Quality and Output,

Competency Outcomes, Behavioural Outcomes and Behaviour Impact. It also shows how advocacy is an ongoing process that supports the theory of change.

chapter chapter 1. Aflatoun’s Quality Assurance And Impact Assessment Strategy (AQIS)

12

Aqis Tools In Line With The Theory Of ChangeThe elements of AQIS monitor and evaluate the different expected effects of the Aflatoun educational programme and advocacy. Thus, AQIS has also been split up into four sections, complementing the theory of change:

Quality Assurance (QA): This monitors the quality and output of the curriculum and Aflatoun’s partner organisations. The main steps include the selection of strategic partners who are capable of programme implementation. We facilitate QA through several tools and processes that are aimed to guarantee certain standards for partner selection and Secretariat performance.

Output tracking: It is necessary to understand the scope, outreach and activities of the Aflatoun Programme, and this will involve measuring output – in other words, numbers; numbers of children, amount of savings, numbers of schools etc. The Secretariat facilitates all partner organisations with tools for tracking their programme outputs.

Outcome Assessment: Competency and behavioural outcomes are more difficult to quantify. Competency outcomes refer to skills, attitudes and knowledge that the children develop after having gone through the Aflatoun programme. Behavioural outcomes refer to concrete changes in children’s behaviour. To be able to measure these, we have defined concrete indicators for these and developed

a number of different research tools. Our partner database and field surveys are examples of such tools.

Impact Assessment: Aflatoun has asked leading research institutions, who use cutting-edge impact assessment methods such as randomised evaluations, to look at the impact of our programme. All these projects are complementary and optional as they require substantial time - and sometimes resource - commitment.

The diagram shows which tools are available at the different levels of AQIS.

14

The AQIS tools highlighted above are divided into two categories: - General tools for all partners.- Complementing projects that can be agreed to in cooperation

between individual partners, the Secretariat and researchers. The duration of these complementing projects can range between one or more weeks (e.g. process review) to multiple years (e.g. longitudinal studies).

We will look at the various tools in more detail in the coming chapters.

Principles Of AQISIn order to make AQIS powerful and effective, we follow five principles:1. Smart 2. Simple 3. Cost Effective4. Open5. Collaborative

1. Smart: In the network-wide field surveys we have tried to focus on the most important competencies and indicators of the programme, as per Aflatoun’s theory of change.

2. Simple: We aim to make your and our work easier and keep things simple. The tools in this guide and the partner database are meant to be easy to use.

3. Cost effective: All partners are given a free Aflatoun partner database and template of field surveys.

4. Open: There are several ways in which the learning in the different elements of AQIS are shared and used to progress our overall knowledge:

- Network sharing: The design, method, and results of the evaluation tools are shared over a global network of partner organisations and research institutions. This is done through regional and international meetings, learning & facilitation and the member’s area of our website.

- Partner database: We have developed a comprehensive database to be used by partners and in which we can collect data that will be analysed and published. The software for the database is open-sourced, which means that it can be used by other organisations. We aim to spread knowledge rather than limit it to our network!

- Publications: The key programme results of the whole network are published in our annual Children & Change publication. An important part of the quality assurance results will be also included in Aflatoun’s annual report.

- Curriculum development: Results from our AQIS tools will provide key inputs for the development of the Aflatoun

15

Curriculum and materials.

5. Collaborative: We are part of a network, and can draw on each other’s knowledge, skills and experience to constantly raise the quality of our programme in all aspects. Experts, Partner organisations, teachers and children are all involved in our processes. Specifi cally for AQIS, Afl atoun consults:

- The AQIS (Impact) Advisory Committee: An expert committee with members from a variety of research fi elds. The committee meets bi-annually and advises on AQIS and organisational strategy, as well as the methodology and implementation of specifi c research projects and evaluation tools.

- The Quality Assurance Taskforce and Impact Assessment Taskforce: During Afl atoun’s global Campaign Launch in March 2008, several groups of partner organisations were formed to deal with particular topics. The Quality Assurance taskforce and the Impact Assessment taskforce consist of representatives of the Afl atoun network and delegates of our strategic international partners. They advise and decide on AQIS strategy and implementation as well as provide an on-the-ground perspective.

- Children and teachers: To be able to test the tools, we asked a group of teachers and students to give their feedback, which has

proven to be extremely valuable.

Afl atoun’s AQIS is moving in an exciting direction. The more we understand about quality and impact of our programme, the more we can make the change that we want to see in the world!

16

The main role of Aflatoun’s Quality Assurance (QA) is to maintain high programme quality and delivery. It is composed of three elements:

1. Quality partnership selection process – setting standards for partner organisations wishing to be part of the Aflatoun Network

2. Training – ensuring that the concept of the programme is clearly communicated to partners and teachers

3. Secretariat participatory evaluation- To ensure that the Secretariat is carrying out its function effectively and efficiently as per the needs of the partners and the programme.

In this manual, the QA tools we will focus our attention on are the partnership selection process and the Secretariat participatory evaluation. More information on Aflatoun’s training model and strategy can be found in the Training Manual.

Partnership ProcessAs the Child Social and Financial Education movement grows and the Aflatoun Network expands, it is crucial to protect the integrity of the Aflatoun Concept and Programme. This begins with setting

certain quality standards for organisations joining the Aflatoun network in all the regions.The selection process happens in parallel to the six steps of programme implementation as described in the partner manual. There are three phases in the partner selection process:

Phase 1: Initial contact Phase 2: Associate partnerPhase 3: Implementing partner

The partner selection process is summarized in the graph below. This graphic shows the three phases in the partner selection process, and how they correspond to the six steps of programme implementation.

chapter chapter 2. Quality Assurance (QA)

18

Duration of selection processThe duration of the selection process differs between organisations and can vary between 12 and 24 months. This will typically depend on the amount of time which an organization needs to establish organisational and financial support for the program. The Aflatoun Network also reserves between 1 and 12 months for its visit to programs already running live in schools.

The general rules for partner selection are explained in the following section. Please refer to the “QA Questions and Answers” section of the member’s area of our website for exceptions and conditions for the signing of the partnership

agreements.

Phase 1 – Initial Contact This is an exploratory phase for both Aflatoun and the new organisation that has just been in contact with the Secretariat. It corresponds to the first two steps of the implementation process:

Step 1 – Initial PreparationStep 2 – Planning

The key elements of this phase are outlined below.

1.1 Research and Initiate ContactThe Aflatoun Secretariat has set criteria for partner selection in line with the vision of Aflatoun to reach every child with CSFE. The Secretariat reserves the right to work with countries that are strategically important, and organisations that request the

programme.

Secretariat Initial Contact

1.1 Research and Initiate Contact

Our priorities for country selectiona. Countries with a child population of above 500 thousandb. Low income or middle income countries c. Countries in conflict or high need

The selected organisation must meet some of the following criteriaFocus fields:a. Child rights based and/or a child centric organisation b. Experienced in rights based/social development educationc. Experience in financial educationd. Focus on poverty/child povertye. Experience in trainingf. Experience in working with the education system or the non formal sectorOrganisational:g. Member of a larger organisation /networkh. Leadership and commitmenti. Ethical fibrej. Scale mindsetk. In existence for 3 years minimuml. Sound financial statements of the last 2 years/ externally audited accountsm. Documented annual report last 2 years

19

Once the Secretariat team determines that the potential organisation meets most of the criteria for selection, the next phase is reached.

Contact with the potential partner organisation is usually initiated through: - A recommendation from a member of the Aflatoun network.- Desk research by the Secretariat- Referral from another network /a multinational/ a bi-lateral

organization - A member/ part of a large organization eg. INGOs- Self referrals- Potential partners met by Aflatoun staff in meetings or conferences

attended

1.2 Scoping The Secretariat provides the potential organisation with core documents to ensure understanding of the Aflatoun concept. At this stage, the organisation is requested to complete the potential partner questionnaire. If, upon completion of this phase, the Secretariat feels the contact does not meet the necessary criteria, Aflatoun can choose not to proceed with the partnership, and close dialogue. In some cases, a site visit may be conducted.

20

Secretariat Initial Contact Comments

1.2 Scoping

Provide Documentsa. Partner Manual b. Questionnaire for Potential Partners

Complete & Submit- Questionnaire for Potential Partners- National legal NGO registration form- Organisational annual report for last 2 years by independent, accredited accountant- Externally audited accounts of the organisation for the last 2 years- Two references by international Donor, from multilateral/bilateral/INGOs

* Annual Report for 2 years, must include:- Financials according to internationally defined standards- Externally Audited Financials/ Accounts- Show a variable donor base for financial sustainability

1.3 Dialogue While still in active dialogue with the Secretariat, the organisation starts its first two steps of implementation: Initial Preparation and Planning. The organisation is requested to provide some documentation:The action plan in which the organisation outlines its strategy, planning and budget for programme implementation. The plan will be discussed with the Secretariat, and will serve as the basis for further

planning and communication.

Written statement from education department (local/ national) granting permission to implement the Aflatoun programme.

Secretariat Initial Contact Comments

1.3 Dialogue

Provide Documentsa. Action Plan Templateb. Compilation of books and sample books if required

Complete & Submit- Action plan and realistic budget- Provide written statement by the local/national education department granting permission to implement the Aflatoun programme (if applicable)Implement - Step 1: Initial Preparation: & Advisory Committee Meeting- Step 2: Planning

Feedback on the position of education departments

If the organisation is working in a country where an Aflatoun partner is already present, the two organisations will be linked so they can collaborate.

1.4 ApprovalOnce all these stages have been completed, the Aflatoun Secretariat

1 This can be found in the Additional Materials booklet as part of the Aflakit

21

will prepare a summary of all relevant information in order to begin the approval process. Once approval is granted by the General Board of Afl atoun, the organisation is eligible to move to Phase 2.

The partners in the Initial Contact Phase are welcome to participate in the regional meetings, twinning and sub-regional meetings.

Secretariat Initial Contact

Comments

1.4 Approval

Review & Approval Processa. Approval by Programme Managerb. Approval by Management Teamc. Approval by Board – Regional Representatived. Ratifi cation by the Board.

Extend Invitations to a. Regional & sub-regional meetingsb.Twinnings

If the organisation is interested in attending meetings they do so at they own expense, exceptions are made on need basis.

In case Afl atoun will not proceed with the partnership, an e-mail will be sent to the organisation to communicate this.

Phase 2 – Associate Partner2.1 Signing Memorandum of Associate Partnership (MAP)In this phase, the organisation becomes an Associate partner. This is formalised with the signing of the Memorandum of Associate Partnership (MAP)1 by the

1 This can be found in the Additional Materials booklet as part of the Afl akit

Associate partner and the General Afl atoun Board.2.1 Signing Memorandum of Associate Partnership (MAP)

ProvideMAP

Review & Sign MAP

Legal sign off of Memorandum of Associate partnership (MAP)

2.2 Start of Implementation of Steps 3, 4, 5 of partnership processThe Associate partner organisation is requested to start the implementation of steps three to fi ve of programme implementation:

Step 3 - Material DevelopmentStep 4 - TrainingStep 5 - Going “Live in Schools” or within in-formal settings

The Afl atoun Secretariat will be in close touch with the associate partners to support them throughout the process.

22

Secretariat Associate Partner

Comments

2.2. Implementation

Provide Documents:a. The eight workbooks b. Trainer manualc. Games manuald. Support via phone / e-mail when required.Training of Trainersa. Provide all training materialsb. Training during sub-regional meetings, subjects demand-drivenSubject to availability of funds

Implementation - Step 3: Materials Development (contextualisation and translation)- Step 4: Training- Step 5: Live in SchoolsNotification- Notify Programme Manager when programme goes live in schools

An organisation is understood to be “live in schools” when the first children in the programme have their first Aflatoun classes (using the contextualised materials and the lessons being given by trained teachers).

2.3 Sharing & (Partner) TrainingAssociate partners will be invited to attend and participate in the Aflatoun events at least once a year. Every year Aflatoun alternates between Regional Meetings (partners from a specific region) and International Meetings (partners from all regions). Partner organisations are invited and encouraged to attend.


2.3 Sharing &Training

Invite partner organisation to:a. Sub-regional trainingsb. Regional meetingsc. International meetingsd. Twinningse. Member areaf. Email-based consultative process of the network

Attenda. Regional meeting (once every 2 years)b. Sub-regional trainings c. International meetings (once every 2 years)Subject to availability of funds

2.4 Learning & Facilitation visitOnce the organisation launches the programme and goes “live in schools” or in the non-formal settings, the Aflatoun Network commits to visiting the programme within one year. The “Guidelines for Learning & Facilitation Visits” can be found in Appendix 1 We encourage you to share these guidelines with staff and schools in your programme.

23


2.4 Learning & Facilitation Visit Report

Provide Documents:a. Guide for learning & facilitation visitsb. Actual learning & facilitation visit within one year after associate partner goes “live in schools”c. visit to the programme site by the Aflatoun Secretariatd. Briefing with partner staff on lessons learned from visit

Preparationa. Help with the preparation of the learning visitb. Recommendation report done jointly by partner and Aflatoun Secretariat

2.5 Approval The final step towards completion of Phase 2 is, once again, the approval process. The Aflatoun Secretariat will prepare a summary of the associate partner’s programme which will be presented to Aflatoun’s General Board for approval. Once this approval is gained, the organisation moves to Phase 3.

Secretariat

2.5 Approval Review & Approval Processa. Approval by Programme Manager.b. Approval by Management teamc. Approval by Board – Regional representatived. Ratification by the board.

Phase 3 – Implementing partner3.1 Signing the Implementing Partnership Agreement (IPA) At this stage, both Aflatoun’s General Board and the partner organisation sign the Implementing Partnership Agreement (IPA)2. The partner organisation then becomes an Implementing Partner.

This membership is ongoing for as long as the partner continues to implement the Aflatoun programme as specified by the IPA.


24

Secretariat Implementing Partner

Comments

3.1 SigningImplementing Partnership Agreement (IPA)

ProvideIPA

Review & Sign IPA

Legal sign off of Implementing Partnership Agreement (IPA)Upon signing this agreement, the implementing partner is entitled to:a. Participate in the decision making process of the movementb. To be actively involved (if nominated by other members) on the Aflatoun Board and task forcesc. To participate in twinning, (sub-) regional trainings/meetings, and international meetingsd. To advocate on behalf of the movement at the global and regional level

Requested documentation once an Implementing partner

There remain a set of recurring documents that we request from implementing partners:1. Each year partners are requested to provide their annual report

and their audited accounts

2. Every 2 years partners are requested to provide an updated action plan and budget

3. When there is reason for concern about a running program, the Board may request extra documentation.

Multiple Partnerships In A CountryIn the case where Aflatoun is in contact with more than one organisation in a country, the partnership selection process becomes somewhat more complex. On one hand, Aflatoun would prefer to work directly, and sign agreements with, the owners of the programme on the ground ( i.e. the partner organisations who are in charge of the programme at the local level). On the other hand, to make the Network more organised and manageable, we would like to have partners choose coordinating/ leading national organizations or alliances that can sign a partnership agreement with Aflatoun (instead of each individual partner).

We understand that the possibility of these partnerships depends on a variety of factors (e.g. the size of an organisation/country/target population, geographic position, political/NGO landscape). The rules of thumb on partnership with multiple initial contacts in a country are:

- Initial contacts in a country choose one coordinating/lead organisation from amongst themselves. This is the organisation that will sign the partnership (MAP/IPA) with the Secretariat.

-The Secretariat will only really be in direct, individual contact with

25

the coordinating/lead organization in the country. The other organisations will be included in general communication and updates and will be invited for regional meetings. We stress that all organisations, whether they are the coordinator/lead or not, are an integral and recognized part of the Afl atoun movement.

- If it is complicated or impossible to have one coordinating/lead organization for any reason, multiple partnership agreements may be signed in the country, with a maximum of 5 organisations. However, these organizations will also be requested to work on developing future alliances in order to bring down the number of partners per country.

Participatory Evaluation Of SecretariatOur role as Afl atoun’s Secretariat’s role is to facilitate the Afl atoun network and represent it with key stakeholders. It is therefore crucial that we respond to demands from partners. We need to engage in a process of self-refl ection, asking questions like:

- Do we listen enough to our partners? - Do we do what we say we will do? - Do we do things in a participatory and inclusive manner?- Is our work effective, timely and effi cient?

To help us asses our performance, we ask a third party to do an annual participatory evaluation of our work within the network and with our external stakeholders. We ask for feedback from both the partners and stakeholders, and use this to assess the quality and facilitation of our communication and how effectively we use our resources.organisations, whether they are the coordinator/lead

26

The goal of Impact Assessment is to measure the outputs and outcomes of the Aflatoun programme through the collection of relevant information from partners, schools, classes and children. The two main tools, which are the focus of this chapter, are the Surveys we have developed and the partner database, designed in collaboration with Tata Consultancy Services in India.

Survey Overview And MethodologyFive SurveysIn the past, no uniform system existed to compile information for publication or internal controls. This resulted in unreliable data that was sometimes based on guesswork and estimations. Another major point of frustration for partners was that, although they could see clear results from the Aflatoun programme, there was no way of quantifying them. Our challenge was to respond to this issue and coordinate how best to measure these outputs and outcomes.

One solution was to create five different surveys that would provide a comprehensive evaluation of our programme. These were tested and evaluated by the Impact Assessment Task Force and other partners in the Network. We have designed them so that they are simple and easy to use. They are also available in French, Spanish and Arabic. Surveys may need to be translated and contextualized for your programme.

Some of the surveys are mandatory for partners to carry out, while others are optional. Required surveys are necessary in order to ensure that we have a base level of quality assurance and comparable information from all Aflatoun partners.

Required Surveys

Partner Survey: Partners complete this once a year. It looks at basic organisational information and activities, and details about the programme delivery and costs.

School Survey: This survey is an interview with the main Aflatoun teacher or the head teacher. It captures relevant information about Aflatoun programme delivery at the school level. It asks important questions about student savings as well. In order to make sure that Aflatoun is running well, all partners are encouraged to visit their schools at least once a year. (If you have a small programme or are in your first year, you should try to have your visits correspond with the end

of the school year.) This survey can be done during those visits.

Unlike the Partner and School surveys, the Class and Child Surveys are not required. That said, we believe that they provide valuable information and we would like to see as many partners as possible using them. They are designed to be quick and resulting data entry easy.

chapter 3. chapter 3. Impact Assessment: Surveys And Partner Database

27

Optional Surveys

Class survey: This survey will provide you a better understanding of how the programme is implemented at the class level. It is both a basic class ledger book analysis and an evaluation of the teacher’s perceptions about the programme and its outcomes. It also has a section that attempts to determine children’s understanding of key Aflatoun concepts, like the motto, character and song. This does not have to be done with all classes but with a few from each school.

Young Child Survey: This optional survey looks at young children and seeks to determine if they understand the key themes and messages of the Aflatoun programme. The survey is aimed at younger children in the Aflatoun programme and questions correspond with themes from Aflatoun workbooks 1-3 (for children age 6-9).

Older Child Survey: This optional survey looks at older children in the programme and attempts to see whether they know and can apply key Aflatoun concepts. The survey is aimed at older children in the Aflatoun programme and questions correspond with themes from Aflatoun workbooks

4-8 (for children age10-14).

All surveys are to be handled by the Aflatoun partner. The Partner Survey should be filled out by the Aflatoun coordinator in collaboration with the partner. School, class, and student surveys are to be administered by surveyors sent by the partner.

AnalysisOnce all information is collected at the partner level, it is the partner’s role to send it to the Aflatoun Secretariat via the partner database.

As you see from the chart below, the information is used to become the basis for all major Aflatoun publications. These publications will be widely disseminated to raise the profile of Child Social and Financial education and the Aflatoun Network.

The results and publications are also meant to be a tool for your programme. Once we have processed the information, we will provide you with reports specific to your programme and give you access to the data to so that you can use it for your purposes. We hope that it will be both an advocacy tool for your lobbying efforts and also an administrative report that details what you are doing. You may wish to use it for programme planning and fundraising, or for communication to stakeholders like the Ministry of Education.

28

Steps To Collecting SurveysThe Afl atoun team, in consultation with the Impact Assessment Taskforce, have created some guidelines to aid you in the process of fi lling out surveys. The steps are outlined as follows:

1. Budgeting for time and costs2. Contextualisation of surveys3. Creating survey packages4. Training your Surveyors

The fi rst step is to decide which surveys you want to implement. Remember, the Partner Survey and School Survey are mandatory, whereas the Class Surveys and two Child Surveys are optional.

Tip: The information in the mandatory surveys is essential for us to be able to conduct effective Impact Assessment. We decided to make the rest of the surveys optional because we understand that they require the use of partners’ resources. Deciding which surveys to do is based on both your logistical capabilities and organisational needs. If you really want impact data, consider

doing the optional surveys.

1. Budgeting for Time and CostsTime:Time constraints can affect which surveys you do and how many of them you can fi ll out. This will mean that you will have to plan ahead.The best way to do this is by imagining a day worth of school visits.

Imagine your surveyors going to your schools. How long will it take them to reach a school, how long for their lunch, how many schools can they visit per day? From this you should be able to determine how long you expect your surveyors to be at each school.

Survey Planning

1.Total Number of Schools

2. How many schools per day will you visit?

One School � Two Schools � Three Schools �

3. Time per School Visit (reference for Q.8)

6 hours � 3 Hours � 2 Hours �

Based on this, you will be able to determine how much time you have in each school to do surveys. Based on pre test results, the average amount of time for each survey is provided below.

Number of Surveys Per School

Survey Time Per Survey/Visit

Number of Surveys

Total Time (survey time X # of surveys)

4. School (required) 60 minutes

5. Class (Optional) 30 minutes

6. Young Child Surveys (Optional)

20 minutes

7. Older Child Surveys (Optional)

20 minutes

8. Total Time (Must be equal/less than the Time per School Visit, Question 3)

29

CostsEvaluation Budget Costs-

1. Human Resources Costs- Cost of paying surveyors and partner staff time

2. Training Meetings- Cost of delivering training

3. Material Costs- Cost for creating surveys and printing and disseminating evaluation results

4. Travel Costs- Cost for transport between schools

5. Data Entry Costs- Costs for paying for data entry

TOTAL BUDGET

After deciding which surveys you are going to implement and how many schools will be visited, you can begin to budget what the costs will be for your programme.

Human Resource Costs (Surveyors) - If your own staff will be conducting the surveys, then this will not cost additional money. However, the time required for your staff should be taken into consideration. Otherwise, you can pay a surveyor to visit the schools, or find volunteers.

Finding good surveyors can be difficult. Surveyors should be comfortable speaking and interacting with children and should be reliable and honest. While they are working, you can ensure that they have visited all the schools by asking the head teacher to sign the surveyor’s sheet and you can occasionally call the head teachers to follow up on the surveyor’s performance.

Training Costs - This includes costs associated with providing training to surveyors, including space, facilitator time, and food.

Material Costs - You must also take into consideration the cost of copying materials for training and surveys for the evaluation. Also included is the cost of printing results and disseminating them to your different stakeholders or potential funders.

Travel Costs - Determining transport costs will depend on your organisation’s situation. If all schools can be accessed by motor bike or public transportation, then costs will be low. If your schools are spread out and require overnight stays or the use of your organisation’s vehicles to reach them, then these costs will be higher.

Data Entry Costs - If you plan on paying someone to enter your data, insert the salary cost here.

Total costs of the evaluation should be calculated ahead of time. This might affect some of your previous planning decisions. It might also require that you include evaluation as a future cost in outside funding proposals or when you are budgeting for the Aflatoun programme.

2. ContextualisationIt is important that the teachers and children understand the questions of the surveys very well. This means that surveys will need to be in the local language, and that they must fit the local context.

30

Some question wording may not make sense in translation or might not be culturally appropriate. You are able to change wording of questions so long as the general theme of the survey does not change.

The surveys are available in English, French, Spanish and Arabic. In the Partner Database, you will fi nd all the translated versions as well as a blank template that can be used to translate surveys into different languages.

If you are planning on translating the surveys, please use the templates provided. The important things to remember are:

All survey questions must be numbered the same

All survey answers must be numbered the same

You will notice that in the templates provided for the child surveys, we have indicated where you should insert the relevant currency.

The three questions where this is applicable are:- If you had [large local currency amount], how much would you

spend right now?- If you had [small local currency amount] every week, how much

would you save each week?- Can you tell me the name of the currency in [country]? (younger

child survey only)For the fi rst question, the number should be an amount that is a

sizable sum for a child. The second should be an amount a child might receive as an allowance.

The responses should be in quarters and we suggest that you choose easy round numbers:

Total Answer 1 Answer 2 Answer 3 Answer 4 Answer 5

Currency amount

0 ¼ ½ ¾ All

20 Dollars 0 5 10 15 20

100 Rupees

0 25 50 75 100

200 Taka 0 50 100 150 200

4 Euro 0 1 2 3 4

8 Pesos 0 2 4 6 8

Choosing or Adding Questions: You can add questions that are specifi c to your

programme, and remove you do not feel are appropriate.

3. Create Survey PackagesOnce you have chosen which surveys to use, translated and contextualised them, you will need to prepare a survey package to give to your surveyors.

All surveys can be found in the database. They are stored in the Print Survey Forms subsection, which is within the Survey section. This is presented in greater detail in Database Manual which is found

The surveys are available in English, French, Spanish

31

in Appendix 2.They are available as Microsoft Word documents and you should download the appropriate ones and save them on your computer.

You can find a standard survey package in the available languages that contains a set number of surveys. If you want your own survey package, you can print the number of surveys you require per school and make photo copies for each school. Save paper by printing on the front and back of pages and remember to staple your packages together for safekeeping!

While all this is available in the database, we suggest that you create a folder in your computer to handle all your surveys. You cannot save any changes into the documents in the database, so it is important that you have them saved for yourself.

Be sure to include the Surveyor Cover Sheet3. This sheet will provide brief instructions for your surveyors. Fill in the “Expected # of Surveys” column on the survey cover page. Bundle all the surveys with the correct number of copies of each and add the survey cover page at the top. Once you have you standard survey package, you can photocopy for the appropriate number of schools. If possible, use a photocopier that prints on two sides to save paper!


Partner Tip: If you decide to conduct the children surveys, we suggest surveying 5% of your Aflatoun children with 50 surveys as a minimum, and 1000 as a maximum. Questions should be read out to the children by a surveyor who should also fill in the forms. Remember, we are most interested in the quality

and not the quantity of these surveys.

4. Train your Surveyors:Once you have set your survey process, you can begin to train your surveyors. The Secretariat has developed a simple guideline in Appendix 3 that you can use as a basis for your training. Most important is making sure that there is a common understanding of the content of the surveyors and how the process for your programme works.The surveyors can either choose to hand out the surveys to the teachers or they may choose to fill in the teacher’s answers themselves.

Share the stories! Encourage the surveyors to note down anything interesting that they have noticed or heard during their visit to the school. At the back of the survey pack is a little form where they write notes about a special story, or enterprise or fun anecdote.

32

When To Conduct The SurveysNow that you understand how to plan for the requirements of data collection, you can determine how to go about planning when to begin your evaluation.

The Impact Assessment Taskforce and the Secretariat request that all partners do their evaluation towards the end of their country’s school year. This would allow for Afl atoun students to be in the programme for the longest period of time before they are surveyed and thus have the most accurate programme data.

You have therefore to decide when the best time for beginning your process is. This requires you to determine the following three things:

1. Preparation time: How long do you expect that it will take for you to fi nd your surveyors and train them? Also, will you need to request permission to access schools? What is the minimum time required for this process? These two tasks can be done at the same time as long as they are both done before your fi rst school visit.

2. Duration of school visits: How many weeks will it take you to complete your visits to all the schools? This can be done by a simple calculation:

3. Final date: What is the last week or month that you can enter schools? Some countries have an extended period of time for

exams or class trips in the fi nal weeks of the school year. It is important that you think about your evaluation in terms of the last week/month that you can enter a class and access all students, as opposed to the last few days of the school year.

Based on this information, you can plan ahead to insure that you have enough time to conduct your visits and that the information you get will be the most relevant.

The Partner DatabaseThe Partner database has been designed by the Afl atoun Secretariat in collaboration with Tata Consultancy Services. This was a corporate social responsibility project for Tata Consultancy Services and they worked hard to make the database comprehensive but also easy to use. Additional information about the database and its functions are available in the Database Manual in Appendix 2.

The Secretariat is also here to provide technical assistance and we are happy to help you if you run into specifi c problems.

information about the database and its functions are

33

As much as the surveys tell us about the programme, we think it is essential to gather more qualitative information as well. This Chapter outlines other tools for outcome and impact assessment that are part of AQIS.Some of the tools described are for all partners - network wide tools - while others are complementary projects that may only work with individual partners. Some rely on outside expertise or volunteers and depend on additional funding.

Network wide tools include:

- “A Child’s Story” and “Stories of Change” - Photos, drawings and audio-visual projects

Complimentary projects supplement and enrich the core Afl atoun AQIS strategy. These projects provide more in-depth impact research and quantitative research that will improve the quality of the Afl atoun Programme. They include:

- Qualitative Research - Process Review - Social Return on Investment - Randomized Evaluations - Longitudinal Studies

Collecting Stories And Audiovisual MaterialPeople can identify with stories and audiovisuals of change. Stories provide the most tangible and personal evidence of impact. Stories and audiovisuals serve multiple purposes:- They enable us all to share broadly what happens in individual

Afl atoun programmes.- They can be crucial advocacy tools and can be published in

different media, or in advocacy materials.- They serve research purposes. Stories can provide input for

qualitative research, but can identify key indicators of success if a large number of stories is collected.

Together with the Impact Assessment Taskforce, the Secretariat has drafted guidelines for collecting stories and audiovisual materials. More information on these can be found in Afl atoun’s Communication Manual. Remember, at all times, children’s rights must be protected and respected.

1. “A Child’s Story”It is Afl atoun’s mission “to inspire children to socially and fi nancially empower themselves and become agents of change in their own lives and for a more equitable world.” This tool tells the story of

chapter chapter 4. Impact Assessment: Other Tools

34

change in the life of a child, and shows the mission at work. Writing “A Child’s Story” though, requires some preparation.

Five step plan to get to meaningful, powerful children’s stories:- Step 1: Speak to teachers and stakeholders about good examples within

your Aflatoun programme. Try to get stories from both urban and rural schools, high and low income children, and both boys and girls.

- Step 2: Read the “General tips for interviews. Part A” 4 - Step 3: If needed and appropriate, arrange a tape recorder for the interview- Step 4: Interview the children, following the tips and the “A Child’s Story”

Guidelines5 - Step 5: Write the story! Keep in mind to select meaningful/powerful quotes,

and to follow the child protection guidelines

2. Stories of ChangeChange does not only happen at the individual child’s level, but can also occur in groups of children, within other groups of stakeholders in the programme, throughout the partner organisation, or in the local community. Meaningful “Stories of Change” in the programme and throughout your organisation are important tokens of impact.

Six step plan to get to meaningful, powerful “stories of change”:- Step 1: Talk to children, teachers, parents, volunteers, your staff, the local

community, other stakeholders in and around the programme- Step 2: Identify specific events, activities, or changed competencies/

behaviour- Step 3: Read the “General tips for interviews. Part B”6 - Step 4: Arrange a tape recorder to tape the interview- Step 5: Interview relevant participants or stakeholders of the programme- Step 6: Write the story! Keep in mind to select meaningful/powerful quotes,

and follow the child protection guidelines

3. DrawingsOften, children communicate their emotions and ideas better via a drawing or painting than through words. Drawings are fun, simple but effective guides to understanding children’s experiences within, and ideas about, the Aflatoun programme. They carry great value both for advocacy and research purposes.

When visiting a school – either for programme purposes or for your data collection – ask a few children whether they would like to make a drawing by asking them a general, non- leading question like: “Can you draw what Aflatoun means to you?”

When selecting pictures to be shared with the Secretariat and network, it is important to keep in mind that the value of drawings lies not primarily in whether the drawing is done artistically or is particularly beautiful, but in its underlying message or story.

4. Photos, Audio & VideosUsing photographs, videos and audio recordings is a great and fun way to show the world the Aflatoun programme in action! Just like the stories, audio and video provide anecdotal and qualitative evidence of the programme’s impact. When we have many of these, we can even use them as quantitative indicators for impact.

4-6 This can be found in the Additional Materials booklet as part of the Aflakit

35

A few basic tips for taking photos or recording video or audio: - Focus on children’s empowerment and stories of (positive) change - Make sure to have a mix of photos of boys and girls, young & older children- Make sure to follow the child protection guidelines for reporting on children- Test the equipment before you take it to the field!- Make sure the lighting is good when taking videos or photographs- Choose a quiet background or a background that is lively and represents the

location where the recording is done- Make sure only one person at a time speaks

More information can be found in the Communication Manual

Guidelines for Reporting on ChildrenAflatoun thinks it is imperative that the rights of the children we work with are protected. One of the key elements of this protection is the ethical use of the children’s images and materials.

We believe that a few key principles should be followed. It is important to remember that, should a country’s laws or norms differ, that the rules of the country be respected. Be sure to check the rules in your country to see what standards you may be expected to follow.If there are no national rules, we ask partners to follow a few key principles:

- Gain written permission from children, and their teacher, when doing Children’s Stories or highlighting a child in an audiovisual production

- Ask oral permission from teachers and also students when taking pictures in a school setting

- If you are planning on publishing photos or images for external audiences, seek the written permission of the child for the use of their image

We hope that you will share these guidelines with all your staff involved in this area.

5. Case StudiesAflatoun will be doing case studies of particular Aflatoun partner programmes. This will include details about the programme implementation and how it currently operates.

It will be used as a learning tool for partners both new and old and is meant to be used as a tool to facilitate discussions and sharing within the network.

Case studies will be done by the Aflatoun secretariat or researchers and will include field visits, interviews with programme staff and stakeholders, as well as partner input. It will be published within the network at appropriate sharing moments and may be published more broadly.

36

Complementary Research ProjectsThe remaining research tools and projects will complement our overall data collection and provide different types of information.

1. Qualitative researchQualitative research is conducted to study the quality and development of the programme, particularly Afl atoun’s core elements, workbooks and programme implementation tools (such as the piggy bank, bank account, Afl atoun clubs etc). This can be done through interviews with key informants, child interviews, participatory action research, focus groups, and focused evaluations. The resulting information will be used to refi ne the programme itself, as well as the impact assessment indicators.Research projects of this type are carried out by independent researchers with coordination from the Afl atoun programme managers and the partners.

2. Process ReviewProcess Review looks at how the programme works at the partner level. Using a mix of participatory methods and business knowledge, it attempts to look at how the different components of the programme are being implemented and delivered to determine if they are effective and effi cient. This is done by comparing the programme outcomes with Afl atoun’s strategic objectives, with a special focus on communication, participation, policy and decision-making processes

within Afl atoun programmes. It also attempts to look at the bigger picture to make sure that Partner programmes are working in a way that furthers the goals of the Afl atoun Network. This kind of study is usually done with research executives and graduate students. As such, it is a collaborative process, meaning that the partner and the person conducting the research work together for a common goal or objective. The results of this study can help us create guidelines of best practice for partners.

3. Social Return on Investment (SROI)Conventional fi nancial evaluations compare output to costs, but do not capture the social and economic benefi ts of projects and programmes. However, Social Return on Investment (SROI) evaluations aim to capture these benefi ts by giving a monetary or social value to key indicators in our programme and by comparing them to the costs of implementing the programme.The initial pilot is conducted by the Secretariat and Social Evaluator team. After that, the tools will be available to all partners.

4. Randomized EvaluationRandomized evaluation is the most rigorous way of measuring the impact of social and developmental programmes. It requires that some children get access to the Afl atoun programme and others do not. You are able to determine the effect of the programme by monitoring and comparing the resulting changes within these groups.

focused evaluations. The resulting information will be

37

5. Longitudinal StudiesLongitudinal studies attempt to determine the long-term impact of the Aflatoun programme. Some organisations follow individual student development over an extended period of time, complimented by in-depth interviews, to determine if any changes in individual behaviour can be attributed to a programme or intervention. We are considering the installation of an alumni network to keep connections with the programme & to ensure the possibility of tracking alumni.

38

Quality AssuranceBelow is a reminder of which documents are required from partners at the different stages of the partnership process:

Phase & element Documentation / information required of partners

Phase 1: Initial contact

1.2 Scoping - Questionnaire for Potential Partners - National legal NGO registration form- Annual report for last 2 years*- Externally audited accounts for last 2 years- Two references by international Donor, from multilateral/bilateral/INGOs

1.3 Dialogue - Action plan and budget **- Provide written statement by the local/national education department granting permission to implement the Aflatoun programme (if applicable)

Phase 2: Associate partner

2.1 Signing of MAP - MAP: Memorandum of Associate Partnership7

2.4 Learning & Facilitation Visit

- Joint recommendations by partner and Aflatoun Secretariat

Phase 3: Implementing partner

3.1 Signing of IPA - IPA: Implementing Partnership Agreement8

*Each year partners are requested to provide their annual report and their audited accounts

**Every two years partners are requested to update their action plan and budget

Data Collection Summary

7-8 This can be found in the Additional Materials booklet as part of the Aflakit

39

Which documents to expect from the SecretariatAll these documents can be found in the Additional Materials booklet in the Aflakit.

Questionnaire for Potential Partners During the initial contact phase, potential partner organisations are asked to fill in this questionnaire. They are asked for organisational information (e.g. mission, thematic focus, budget, accountability, Human Resources, existing programmes, references).

Action Plan The next element in the initial contact phase is the Action Plan. This crucial document outlines the potential partner organisation’s strategy and approach towards successful implementation of their Aflatoun programme. Intended planning, budget and outreach are included. All partners are asked to update their action plan and budget every two years.

Memorandum of Associate Partnership (MAP) The MAP demonstrates the shared commitment of the Associate Partner and the Secretariat to beginning work together. It formalizes the spirit of productive cooperation and recognizes the good will of both parties to work together towards fully implementing the Aflatoun Programme. It aims to help the Associate Partner develop the capacity that will see its transition to fully Implementing Partner status.Implementing Partnership Agreement (IPA) These are signed after an Associate Partner has been “live-in-

schools” and a learning & facilitation visit by a Secretariat staff member has been done within one academic year of the programme starting live in schools. The IPA recognizes that the Associate Partner has a proven ability to deliver the programme. It sets out the respective roles and responsibilities that govern the partnership.

Learning & Facilitation Visit Report with GuidelinesWithin one year of an associate partner going “live in schools,” the Aflatoun Secretariat will have a learning & facilitation visit to the programme site. Guidelines for these visits have been made to ensure shared expectations at the partners’ and Secretariat’s end.

Impact AssessmentPartners are requested to collect and submit data once a year. This can be done at any time throughout the school year, provided that all data has reached the secretariat before December 31st. This will allow the data to appear in our publication, Children & Change. We remind you that school surveys should be done towards the end of the school year.

40

AppendicesAppendix 1 - Learning & Facilitation Visit Report With GuidelinesCover SheetName staff member Afl atoun Secretariat: _______________________________Partner Organization visited: _______________________________Name staff member Partner Organization: _______________________________Country: _______________________________Places visited: _______________________________Date of visit: _______________________________

Overall observations/comments:

Overall suggestions for partner organization:

Overall suggestions for Secretariat:

Signature staff member Afl atoun Secretariat: Signature staff member partner organization:

____________________________________ ____________________________________Guidelines for Learning & Facilitation visits

41

The Aflatoun Network is all about partnership and shared learning. We (the Secretariat) greatly appreciate the opportunity to visit your programme as your programme has gone live in schools. We want to learn about your good practices, the things that have worked well for you, the innovations you have made, the challenges you have faced, the lessons learned and your thoughts on what training you would like to see us provide. To help us both get the most out of our visit we have drafted these guidelines. We hope to see you soon!

1. General Expected OutcomesBy the end of the visit, we hope to:□ have gained a better understanding of your programme and organization □ have all your documentation together for Board sign-off for Implementing Partnership□ have seen your programme “live in schools”, and interviewed a mix of participants□ have talked to your staff and understand your good practice, innovations, lessons learned, and training needs□ have assisted you in one or more stakeholder meetings, when needed by you□ have copies of class materials, some stories of change and children’s drawings/artwork

2. General visit overviewThough all different, all visits have certain essential elements in common. We suggest a general planning, which can change according to needs. Secretariat and partner staff both are asked to use the document checklist below for your preparation of the visit.

42

General Scheme What documents to bring?(Partner = P, Secretariat = S)

Desired Results/outcomes Required time

Day 1: Meeting with partner staff

For Board sign-off:□ Completed survey for potential partners (P,S)□ Completed action plan & budget (P,S)□ Partner survey (from partner database) (S)□ Annual report/externally audited accounts of last 2 years (P)□ National legal NGO regist. form (P)□ Third party references of organization (P)Other:□ Own partner’s materials (P)

- Open discussion- Review of survey for potential partners & action plan- Collection of other documents- Overview of training need & other partner expectations Overview of innovations, best practices, lessons learned

3-4 hrs with partner:- Aflatoun coordin.- Exec. dir.- Other interested/concerned

Day 1:Stakeholder meeting(s)

□ Background documents provided by partner (if applicable) (P)

- Upon the partner’s convenience- Meeting with stakeholders (e.g. educ. dep., Unicef)

3-4 hrs

Day 2-4: Visit to schools

□ “Stories of change” (P)□ “Children’s stories” (P)□ Guidelines for stories & child protection (S)□ 6 school surveys* (S)□ 6 class surveys* (S)□ 15 children’s surveys* (pre-determine ratio grade 1-3 and 4-8) (S)□ Children’s drawings

- 4-5 school visited- Mix of: - Successful/less successful - Rural/urbanInterviewed in total:- 2-3 head teachers- 5-6 teachers- 5-6 classes- 10-15 children of different ages

- 2-3 school visits p/day if possible- (3 days max.)

* These surveys are meant to guide, not to limit interviews with children, teachers, etc.

43

3. General tipsThe school, class and children’s surveys provide guidelines for field interviews. Some extra general tips for preparation and interviewing (both for Secretariat and partner):

Selecting Schools- Try varying between schools that are more & less successful at implementation- Try varying between rural/urban schools- Try varying between social/cultural/ethnic communities

Selecting classes- Try varying between classes with individual savings accounts & group accounts- Try including a range of classes with fin./soc. enterprises, and innovations

Interviewing children in class & individuallyExample questions other than in surveys:- “Did you participate in camp/culmination event?”- “Who generally decides where you spend your saved money on?- “Can you make a drawing of what Aflatoun means to you?”(you can ask children this question starting a school visit and collect them at the end)

Interviewing (head) teachersExample questions other than in surveys:- “How do you value the programme?”- “What do you think about the Aflatoun materials; any new ideas?”- “Did you see any unintended effects that we can share with other programmes?”- “What support do you need?”- “Can we go thru class ledger together?”- “Did you receive training? From whom?”- “Did you see any examples of change?”

General tips for interview & visit (IMPORTANT!)- Through language/culture words can be misinterpreted. Be clear, but always remain polite.- Make sure a translator is present who is familiar with the programme- At all times try to avoid prompting answers- If taking photos, recording videos or collecting drawings, get permission* for further use- To fuel learning, partner & Secretariat staff can share notes and agree main advice

* Please refer to Aflatoun’s guidelines for child protection

44

4. Space for key notesThis space is available for your notes on key points:

Key good practice/innovations: Follow-up needed:Item: Date & Owner:

Key lessons learned (partner & Secr. staff): Suggested improvements Secretariat docs:

45

Appendix 2 - Database ManualDatabase Manual This is the Database Manual. It will provide you an introduction to its design, its

features and how best to use it.

You will receive a user name and password from the Afl atoun Secretariat. When entering this information, remember it is case sensitive, meaning that if certain letters or words are capitalised, then it will not work.

The home section is the fi rst section that you will see once you login. The names of the sections are translated into three languages and a menu with images. To move to these sections, click on the text.

46

There are two ways to move around the database. - In the Home Section, click on the purple text. - In the navigation bar, click on the appropriate box to move to that section.

There are six core areas in the database. They will be explained in greater detail in the remainder of the manual.

The Introduction Section is where we provide an overview of the AQIS strategy, the database, the surveys, and Social Return of Investment (SROI). Click on the document name to open it up!

47

Every year there will be changes in the surveys and the database. The News section will provide updates -by year- about new things for the coming year.

There are two places where you can fi nd help. In the “?” you will be able to get help related to that page. In the Manuals-Help section, you will get other tools that provide help or describe the database.

Every year we will load all the relevant results of the annual data collections. We will provide global, national and partner reports to everyone. We are working to a more dynamic program that will allow you to do your own reports/research.

48

Once you have entered your information, you will need to send the information to the Secretariat in the Netherlands. You can do this by mail on a CD or through the internet.

The Survey section is the most important area within the database. It contains 4 parts and deals with everything to do with the surveys in the system.

Click on the School Survey and you will see it! You need to fi ll in a school survey in order to fi ll in class or child surveys. This means it is the gateway for all your survey data entry.

49

There are two key types of navigation within the surveys. If the survey is more than one screen, you need to click on next. If you want to add additional surveys, click on the appropriate button of the bottom of the screens.

Press the Partner Checklist Button and you will see the survey! This should be completed only after you have done and entered all your school, class and child surveys.

You will be able to print your survey forms from the database. When you click on the survey, it will open in Microsoft Word. All surveys will be in English, French, Spanish and Arabic.

50

You will be able to see which surveys you have entered in the view survey screen. You can also add class or child surveys by clicking on the name of the school.

Once you have clicked on the name of the school, you will be able to add a Class or Child Survey from this page.

Once you have fi nished your work in the database, you need to log out. Click on the Log Out button on either the Navigation Bar or the Home Screen.

51

Suggested Exercises and ActivitiesIn order to get the best out of your surveyors, we will need to both empower them, motivate them and inform them about what they are

supposed to do:- Introductions: Introduce yourself and give an opportunity for all your

surveyors to get to know one another!- Overview of Aflatoun and Data Collection Process: What is Aflatoun

and why are you doing survey research? It is important for them to know why they are doing this work and what the programme is about.

- Survey Logistics: It is important to be very clear with what you expect of your surveyors. Take them through exactly what they should do in each school that they go to.

- Survey Testing and Role Playing: It is important that you be able to see people administer the surveys and to let people practice. Have each surveyor ask questions and record answers with another surveyor. Encourage constructive feedback within the group and take some time at the end to determine what has been learned.

- Reading the Surveys: Every surveyor should have an identical understanding of the survey questions. This means that you should read through the school, class and student surveys and answers. If people have questions, speak about them as an entire group.

- Questions and Opinions: Give your surveyors an opportunity to talk about any questions that they might still have. Also, ask them their opinions about the surveys and the process. What they found helpful and what they think you should change in the future.

Appendix 3 - Suggestions For Training Of Surveyors Overview This document will provide some suggestions on how an Aflatoun partner could train the potential surveyors who will be delivering the surveys. These are suggestions and we hope that you use them as a starting point for your work.

RationaleIn order to have surveys that are effective, the collection of data has to be as uniform as possible. This means that all individuals who are in schools, interviewing teachers and students, should have the same understanding of how they are to go about surveying.To make sure of this, we suggest that you do training with all your surveyors to go through the data collection process, to outline your requirements, to clarify survey information and respond to any questions they might have.

Timing and LogisticsWe suggest that this process should: - Take approximately 3 hours - Be done in a group with all surveyors if possible- Be done before they go to their first school- Be participatory, within the spirit of the Aflatoun programme

52

Follow UpYou should be in regular contact with your surveyors. If one has an important question or comment once the surveys have begun, be sure to provide the answers to all your surveyors!

53

Appendix 4 - Aflatoun Survey Planning Guide

1. Survey Planning

1. Total Number of Schools

2. How many schools per day will you visit? � One School � Two Schools � Three Schools

3. Time per School Visit (reference for Q.8) � 6 hours � 3 Hours � 2 Hours

2. Number of Surveys Per School

Survey Time Per Survey/Visit Number of Surveys Total Time (survey time X # of surveys)

4. School (required) 60 minutes 1 60 minutes

5. Class (Optional) 30 minutes

6. Young Child Surveys (Optional) 20 minutes

7. Older Child Surveys (Optional) 20 minutes

8. Total Time (Must be equal/less than the Time per School Visit, Question 3)

3. Evaluation Timing

9. Last week you can visit schools

10. Number of weeks required to visit schools (Number of schools/Number of schools per day)

11. Number of Weeks to find/train surveyors and gain permission to enter schools if needed

12. Required start time for evaluation

54

4. Evaluation Budget Costs -

1. Human Resources - Pay for surveyors and partner staff time

2. Training Meetings - Cost of delivering training

3. Material Costs - Cost for creating surveys and printing and disseminating evaluation results

4. Travel Costs - Cost for transport between schools

5. Data Entry Costs - Costs for paying for data entry

TOTAL BUDGET

56

Aflatoun, Child Social & Financial EducationSarphatistraat 7 | PO Box 15991 | 1001 NL Amsterdam

ph: +31 20 626 20 25 | fx: +31 20 626 21 18