Country: Burkina Faso, Mali, Niger
Closing date: 06 Nov 2017
Terms of Reference
EXTERNAL EVALUATION OF STRØMME FOUNDATION’S SPEED SCHOOL PROGRAM IN MALI, BURKINA FASO AND NIGER
BACKGROUND AND CONTEXT OF EVALUATION
Strømme Foundation (SF) is a Norwegian NGO with a mission to eradicate poverty through a rights-based approach. Through interventions in the sectors of education and microfinance, SF works with local communities to provide primarily women and children with the knowledge, skills and tools to move out of poverty. SF has worked in West Africa since 1984 and is currently implementing programs in Mali, Burkina Faso and Niger. SF has a decentralised structure with four regional offices. In West Afrika, the Regional Office in Bamako oversees the activities in Mali as well as the Country Offices in Burkina Faso and Niger. Local partners are responsible for implementing activities on the ground. SF currently has 17 implementing partners across the West Africa region: 10 in Mali, five in Burkina Faso and two in Niger.
Designed in Mali in 2004 by West African education experts in partnership with Strømme Foundation, the Speed School model is a nine-month accelerated learning program that provides three years’ worth of primary education to children who have never been to school or who have dropped out of school and wish to reintegrate back into the formal education system. The course follows a condensed primary school curriculum, with the aim of transferring successful graduates into the fourth grade of formal school. Since the program started in 2004, around 150 000 out-of-school children (OOSC) have completed the program.
There have been a number of studies and evaluations, both internal and external, of the Speed School program over the years. Most recently, an independent impact study, in the form of a randomised controlled trial (RCT), was carried out in Mali by Innovations for Poverty Action (IPA) in 2012-14. It found that 89% of enrolled children completed the program. Most of the graduates transferred into grade 4 of primary school and adapted well to their new school environment.
Yet, there has been little systematic research on the long-term impact of the program on learners, their families and communities. Over time, with the expansion of the program, internal SF reports and evaluations have found that the transfer of Speed School graduates poses a challenge for the receiving primary schools, particularly when it comes to accommodating the large increase in children enrolled. A recent review that assessed a sample of Speed Schools, also found incidents of negative attitudes towards transferred Speed School children among teachers in formal schools.
In order to address gaps in the evidence-base and improve documentation of the Speed School program’s impact, Strømme Foundation is commissioning a comprehensive external evaluation consisting of three main components: impact study, tracer study and value for money analysis. The main purpose of this evaluation is to document the long-term impact of the program and assess the return of investment that the program offers with the aim of improving program efficiency and effectiveness. To this end, the evaluation focuses particularly on gathering evidence on the long-term impact on Speed School graduates and intervention communities, and the cost-efficiency and cost-effectiveness of the program. The evaluation further serves to develop recommendations for adjustments that will improve the program as SF enters into a new strategic period.
MAIN OBJECTIVES AND KEY QUESTIONS
The main objective of the evaluation is to assess the long-term impact of the Speed School program on its graduates, communities and local education systems and conduct a Value for Money analysis of the program. Building on existing studies of the Speed School program, the evaluation should synthesise key findings from previous evaluations and reports and document the long-term impact of the Speed School program on beneficiaries and key stakeholders. In addition to documenting results, the evaluation should include clear recommendations for improvement of the program, with a particular focus on the transition of Speed School graduates to primary schools and the role that local communities and authorities can and should play in this process.
The evaluation will be used to document the results of the program to key donors, as an input in SF’s constant strive for improving the efficiency and impact of its interventions. The findings will also feed into Strømme Foundation’s on-going process to develop a new Strategic Plan in 2018.
The evaluation should include three main components:
1.Study of the long-term impact of the Speed School program on students, households, communities, local and national education systems
The objective of this study is to examine the impact of the Speed School program on the Speed School graduates, their families and the communities that have hosted Speed School centres. The study should include a representative sample of communities from Mali, Burkina Faso and Niger and should be carried in communities where the Speed School centres closed in June 2015. Strømme Foundation and local partners will support the process of identifying the sample. The consultants are expected to take the counter-factual[1] into account. The study should look more in detail at the following issues:
I.Host primary schools and local education systems
a) To what extent are primary schools capable of absorbing the Speed School graduates upon transfer to the formal system? What, if any, are the positive and/or negative consequences for formal primary schools of receiving Speed School graduates?
b) How do primary school teachers and head teachers perceive the Speed School program and the Speed School graduates?
c) Where Speed Schools are located in close proximity to formal schools, are there negative impacts to the host formal school? Does the Speed School attract pupils and/or teachers/other staff from the host school?
d) What knowledge and understanding do primary school teachers and head teachers have of the Speed School program, in particular its curriculum and pedagogy?
e) Do local education authorities work with formal schools to prepare them for the transfer of Speed School graduates?
f) How do the local education authorities view the Speed School program in terms of quality of teaching and learning, in particular the curriculum and pedagogy? How do they perceive the quality and competence of Speed School animators? Is the training that the Speed School animators perceived to be adequate and of good quality?
g) What is the extent of ownership of the program by local education authorities?
h) Has the implementation of the Speed School program lead to any change in how the local education authorities address the issue of out-of-school children?
i) To what extent do the local education authorities recognise their responsibility as duty bearers in providing access to education for all children?
j) To what extent do local education authorities think that the Speed School supervision structure (regular monitoring visits by multiple stakeholder throughout the school year) can be applied to schools in the formal system?
k) To what extent has the Speed School program contributed to raising the primary school enrolment and completion rates in intervention communities? How does this rate differ from comparable rural areas?
l) To what extent has the Speed School program been effective in reducing the number of out-of-school children in the intervention communities?
II.Speed School graduates
a) How did Speed School graduates experience the learning environments in Speed Schools, , including in terms of child protection/use of corporal punishment, gender equality and inclusion of children with disabilities, teachers’ attitudes towards students and availability of materials etc,? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
b) What, if anything, could improve the Speed School experience for learners according to Speed School graduates?
c) How did the Speed School experience compare to the Speed School graduates’ experience once transferred to the formal sector? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
d) How did the Speed School graduates find the transition to primary school? Did they experience any barriers in transitioning and/or staying in school and what factors helped them do so? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
e) What proportion of Speed School graduates (interviewed for the evaluation) are still in school? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
f) What, if any, impact did participation in the program have on the Speed School graduates’ self-esteem and confidence level? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
g) To what extent did participation in the Speed School program change the way that the Speed School graduates were perceived and treated in their families? Is there a difference between the experience of boys and girls or children in other marginalised groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
III.Households and community levels
a) Do parents of Speed School graduates now send younger siblings to school? Is there a difference between boys and girls or children in other marginalized groups (children with disabilities, internally displaced children, orphans, child labourers etc.)?
b) To what extent has the attitude to children’s education and specifically girls’ education, changed in the intervention communities after the implementation of the Speed School program? What, if any, effect has the Speed School program had on the perception of girls in families of Speed School graduates and in the intervention communities?
c) What examples are there of School Management Committees (SMCs) successfully advocating for children’s right to education in the intervention communities, including lobbying for new classrooms to be built, new schools, more resources for education etc?
d) What, if any, role do the SMCs play in the intervention communities once the Speed School centres are closed?
IV.National education system
a) To what extent do the national ministries of education in Mali, Burkina Faso and Niger have ownership of the Speed School program? What opportunities and/or barriers exist for deepening government ownership of the program?
b) To what extent is the Speed School program reflected in national education policies and frameworks, in particularly out-of-school policies in Mali, Burkina Faso and Niger?
2.Tracer study of the 2011-2012 cohort of Speed School graduates in Mali;
The objective of this study is to measure and assess the longer-term impact of the Speed School programme on students and their immediate family. The component will focus on Mali only, and the key objective is tracing a sample of students from the 2011-2012 cohort, getting insights into their experiences before, during and after Speed School. More specifically:
a) Demographic information (income level, relationship status, disability status etc.)
b) Information on the educational level the respondent attained/is working towards
c) Information on current employment status if relevant
d) Opinions on how the knowledge acquired at Speed School is impacting on daily life and school (if applicable)
e) Identify gaps in the program and recommendations for future program implementation
Strømme Foundation and local partners will support the process of identifying the sample. The tracer study should be complemented by in-depth interviews with 20 Speed School graduates (with an even mix of boys and girls), a selection of interviews which should be filmed.
The data for the tracer study should be disaggregated and analysed by gender.
3.Assess the “value for money” of the Speed Schools program in terms of efficiency and effectiveness, looking at:
a) The cost per student enrolled in Speed Schools
b) The cost per student graduating from/completing Speed Schools
c) The cost per Speed School graduate registered in (and physically showing up to) formal school
And:
a) Comparing these costs to the cost of similar programs and/or formal school in Mali, Burkina Faso and Niger
b) Undertaking a social cost-benefit analysis, using a standard metric such as the value of an additional year of schooling
METHODOLOGY
The evaluation should utilise a mixed methods approach, combining quantitative and qualitative methodologies and use a participatory and child-friendly approach (including established child protection frameworks during consultations with children). Separate research frameworks need to be developed for the three components and should all include a gender perspective.
EVALUATION TEAM
As the evaluation is composed of three distinct components and requires field work in rural areas of Mali, Burkina Faso and Niger, SF strongly encourage prospective applicants to propose a team combined of both international and local consultants with a strong record in conducting evaluations to carry out the task. The team is expected to be able to demonstrate:
Extensive experience working in West-Africa in general, and in Burkina Faso, Mali and Niger specifically, with strong knowledge of the local context
Significant experience in producing high-quality, credible research and reports in English for clients and organisations in the development sector, including research and evaluation of education programs
Examples of previous work are required. Proven experience in conducting tracer studies and Value for Money analysis is highly desirable
A team member with strong economic background, preferably in the economics of education, to lead on the Value for Money study
Relevant educational qualifications, and strong background in education for development and assessment of learning outcomes
Professional expertise and experience in monitoring and evaluation
Relevant quantitative and qualitative research skills, demonstrable experience with participatory child-friendly methodologies
Fluency in French and English
Local consultants should have fluency in relevant local languages
Sound knowledge of and commitment to rights based approaches
A high standard of professionalism
Applicants are encouraged to include both male and female researchers at both local and international levels.
The team is expected to carry out field work in Mali, Burkina Faso and Niger. Applicants should note that the security situation in West Africa is unstable and unpredictable and the evaluation team will have to make their own risk assessment when it comes to security. Strømme Foundation will work closely with our local partners to help facilitate the field work, including logistical support. In addition, SF will make work space in country and regional offices available for the consultant(s).
OUTPUTS AND DELIVERABLES
The evaluation report should be no more than 30 pages long with an executive summary of max four pages with a focus on key findings and recommendations. The report should be available in English and French. The final report should be supplemented by a power point presentation.
Deliverables:
Inception report, including a detailed workplan for the assignment
Workshop/skype call to validate methodology and tools
Draft report
Validation workshop/skype call to discuss draft report, key findings and recommendations
Final report in English and French, including an executive summary
Donor-friendly evaluation brief of max four pages
Power point presentation with key findings and recommendations
Cleaned version of all the data used for the analysis, including data on respondent level
TIMEFRAME
The work is expected to take 10 weeks between November 2017 and April 2018. The final report should be submitted to Strømme Foundation by 23rd April 2018. Proposed timeframe:
Activities
- Deadline for submission of proposals - 6th November 2017
- Inception report due - 8th December 2017
- Meeting/skype call to validate methodology and tools - December 2017
- Finalization of tools for data collection - 15th January 2018
- Data collection the field - January/February 2018
- Data analysis and draft report - March 2018
- Deadline for draft report - 16th March 2018
- Workshop/skype call to validate draft report - Early April 2018
- Submission of final draft - 23rd April 2018
[1] Subtracting from the program’s results any changes that would likely have taken place in absence of the intervention – such as the contribution of other NGOs working in the same community, government investment in education etc.
How to apply:
APPLICATION SPECIFICATIONS
We will consider proposals from individual consultants and companies/organisations. To register interest in this consultancy, please send the following documentation to anne.breivik@stromme.org by 6th November 2017, specifying ‘Speed School evaluation’ in the subject line.
· A proposal responding to the ToR, with specific focus on addressing the Main Objectives and Key Questions, timeline and methodology to be used
· An Initial work plan based on methodology outlined, and confirming availability of the applicant
· Company or organisation profile (where applicable) and CVs of consultant(s) who would deliver the work
· A minimum of three references (organisation or individual consultant as appropriate)
· Sample of a recent education program evaluations/research produced by Consultant/Company within the last three years (if available). Examples of Tracer Studies and Value for Money analysis are highly desirable.
· Budget breakdown based on expected daily rates and initial work plan