Niger: Consultant - Final Evaluation - MADARA Program, Niger

Title:Final Program Evaluation, Niger Madara Program in the Commune of Filingué

To Apply:All prospective evaluators should provide both a technical proposal (including proposed methodology, description of past experience, and the CV or CVs of the consultant or evaluation team) and a cost proposal by April 29th, 2016.

I. Objectives and expected deliverables

1.1. Purpose of Evaluation

The main objectives of the final evaluation are to assess the program's contribution to the improvement of food security and economic recovery in the MADARA program’s area of intervention; to determine the success of the program in achieving its objectives and impact; and to assess the sustainability of its activities. This evaluation should also lead to clear and essential recommendations for a Phase II of the program, based on lessons learned from its implementation, successes, and challenges.

To this end, Mercy Corps is recruiting a dynamic consultant, group of consultants, research institution, or consulting firm experienced in conducting final evaluations with proven experience in mixed research methods. The evaluator will produce a comprehensive report based on qualitative and quantitative data to critically evaluate the program's relevance, effectiveness, efficiency, impact and sustainability in the delivery of its results. In collaboration with the program’s management team, the consultant will produce findings and recommendations that will enable Mercy Corps’ future food security and economic recovery programming to maximize its impact and to capitalize on lessons learned.

The final evaluation has the following specific objectives:

• Determine the level of achievement of program results and impact against targets (this should include reviews of baseline data, the mid-term evaluation report, and other relevant program documents that will allow an appreciation of the program’s progress);
• Determine the effectiveness of the program’s interventions and approaches;
• Evaluate the extent to which the program has taken into account gender issues;
• Examine the sustainability of program achievements and its impact on the resilience capacities of pastoralist and agro-pastoralist communities in response to shocks and stresses;
• Identify key lessons (positive and negative) learned, as well as best practices drawn from program implementation;
• Make useful recommendations for future programming, including on implementation methods, types of interventions, beneficiary targeting, and other subjects as relevant.

1.2. Expected deliverables

The consultant will provide the following during their contract:

• An evaluation plan, including:

• Evaluation methodology and sampling frame

• Qualitative and quantitative protocols for data collection and analysis

• Revised quantitative data collection tool incorporating baseline study tool

• Suggested improvements to evaluation scope

• Revised evaluation timeline

• ½-day workshop in French to present the evaluation plan to key staff

• ½ - 1 day preliminary results analysis workshop in French, including donor and other relevant stakeholders, with forward-looking section on how to change the approach or seize particular opportunities for future programming

• Final evaluation report written in English, submitted in electronic form and three hard copies, including the following components:

• Introduction

• Background

• Evaluation purpose and objectives, including research questions

• Literature review

• Theoretical framework and methodology

• Assumptions and limitations

• Sections on each program objective, incorporating results from each

• Section on program quality

• Section on cross-cutting issues including gender, resilience and institutional/stakeholders collaboration

• Section on program implementation and sustainability

• Lessons learned

• Key recommendations (esp. for future programming)

• Annexes

• Executive summary of 2 pages

• One (1) electronic file of the clean (final) qualitative and quantitative data collected.

• Background

2.1. Description of the organization

Mercy Corps is an International Humanitarian and Development Organization whose mission is to alleviate suffering, poverty and oppression by helping people build secure, productive and just communities. Mercy Corps has been present in Niger since 2005 and works in partnership with the government, private sector and the Nigerien community to contribute to a sustainable socio-economic development of the population. Mercy Corps has implemented dozens of programs in Niger and currently operates in the regions of Maradi, Zinder, and Tillabery.

2.2. Project description and implementation background

Mercy Corps is wrapping up implementation of a program called Madara (“Milk” in Hausa), which is funded by USAID/OFDA and the Vitol Foundation. The program is enhancing the food security and building the livelihoods of vulnerable households in the Filingué Department, Tillabéri region of Niger, through strengthening of the dairy value chain. It meets the early recovery needs of 23,800 beneficiaries (3,400 households), with the aim of reducing suffering while bolstering the resilience of communities facing food insecurity. The program integrates both emergency and recovery activities. Madara covers 24 villages and intervenes in three domains: agriculture and food security; entrepreneurship; and health and nutrition. With its current three-month no-cost extension, the program has a duration of 25 months (April 2014 – May 2016).

With less than two months remaining until close-out, these terms of reference have been developed for an external final evaluation of the Madara program.

2.2.1. Project objectives

The overall aim of the Madara is to, "Enhance food security, prevent malnutrition and strengthen the living conditions of vulnerable households in the Filingué Department.”

The program has three specific objectives:

• Protect and enhance productive performance of existing animal assets to improve animal nutrition and strengthen dairy production;
• Restore livelihoods of vulnerable households and improve access to affordable dairy products for communities in the Filingué department through strengthening of the dairy value chain;
• Improve uptake of key Infant and Young Child Feeding and optimal maternal nutrition practices.

2.3. Studies and evaluations conducted

The implementation of Madara has been guided by a baseline study carried out in Filingué in June 2014. The results revealed that 35% of the target population was food insecure, which was one of the chief drivers of negative household coping strategies. For example, 100% of surveyed households confirmed that their members had skipped meals for entire days or had resorted to eating the household’s seed stock. Moreover, the number of children receiving a Minimum Acceptable Diet (MAD) in Filingué is only 2.97%, virtually identical to the 3% national average. The health status of children under five health status was closely related to the food insecurity facing households, but also the lack of application of good nutrition and health practices by mothers. The baseline survey revealed that only 19% of mothers exclusively breastfed children up to the age of six months, and only 31% vaccinated their children when appropriate.

Several recommendations have been made ​​following the analysis of regular program monitoring data. These recommendations have enabled the program to better focus interventions to ensure the achievement of program objectives. Similarly, an internal midterm evaluation conducted in August 2015 helped highlight significant results that had been achieved in the implementation of the program. These include an increase in the use of health services among the target population, a decrease, from 35% at baseline to 24% at mid-term, in the proportion of the population that is food insecure, and improved hygiene and child nutrition practices.

Additional studies conducted with program funding include a Dairy Value Chain Analysis and Dairy Products Market Feasibility Study.

III. Scope of the final evaluation

3.1. Key Evaluation Questions

The key questions to be addressed by the evaluator relate to program impact, effectiveness, efficiency and sustainability. The evaluation will focus on short and medium-term changes at the beneficiary level and on the following questions:

• Did the project interventions address the primaryneeds of the target population?
• To what extent has Madara achieved its results relating to animal health and productivity, the restoration of livelihoods, improved household nutrition, and all other results linked to resilience?
• What observable changes (positive or negative) in terms of practices, knowledge, attitudes, and behavior can be identified at the beneficiary level?
• How did the main program activities reinforce the capacity of beneficiaries to cope in the face of future shocks and stresses (drought, irregular rainfall, and conflict over natural resources)? More specifically, are beneficiaries better able to manage their herds and the production of their animals during the lean seasons?
• What has been the involvement of the government technical services and other partners in the implementation of program activities, and what are the main lessons learned in this regard?
• To what extent can positive changes observed at the beneficiary level and at systems/institutional levels be expected to last after the program has ended?
• Has the Madara program effectively taken into account gender issues in its implementation and what are the effects of achievement or failure in this regard?

3.2. Evaluation methodology

A final evaluation consultant will work closely with the national M&E Manager, Madara M&E Officer, and the Madara Assistant Program Manager to finalize the evaluation methodology and to conduct the evaluation and post-evaluation workshop.

The consultant will be expected to employ quantitative, qualitative and participatory methods to conduct this evaluation in coordinating with the evaluation team, including:

• Review of basic program documents (grant agreement, baseline and mid-term reports, needs assessments, strategies, etc. as well as financial reports);
• Adaptation of the baseline study design and beneficiary-level quantitative surveys, with a statistical comparison of baseline and endline data;

• Various qualitative research methods including but not limited to:

• Focus group discussions with beneficiaries and other stakeholders

• Key informant interviews with beneficiaries and other stakeholders

• Other semi-structured interviews as relevant

The proposed methodologies and tools will be presented to Mercy Corps for validation and approval before the field work. The national M&E Manager will have final responsibility for approval of tools and methodologies as well as assuring the quality of work and deliverables.

Although the evaluation will be designed in a participatory manner with the Mercy Corps team and not directly with the beneficiaries of MADARA, note that the field work will be done through participatory methodologies to engage communities and stakeholders, particularly members of cooperatives and public service providers as well as private in the target area. The goal is to better understand their perceptions of the implementation and outputs / outcomes, and identify their beliefs about how MADARA activities have affected their lives.

During the consultation with the beneficiaries and other community members, the evaluator will be encouraged to look for and report signs of apparent impacts, intended and unintended, and consider whether these are signs of the objectives of the program : that is, if they match the logical framework and the expected results. This should involve two-way communication between all stakeholders.

3.3. Consultant Profile

The following entities are eligible to apply:

• Individual consultants or teams of consultants;
• Research institutions;
• Consulting firms or other private sector groups.

The resource persons assigned to the consultation must demonstrate:

• Proven experience in conducting evaluations, including evaluation of emergency/development projects and those funded by USAID OFDA or USAID/Food For Peace;
• Solid experience in methods of collection and analysis of qualitative and quantitative data;
• Practical experience in conducting surveys in rural and urban settings;
• Demonstrated experience in participative evaluations, assessments and appraisals approaches in intercultural context;
• Proficiency in the use of statistical software for data analysis;
• Excellent speaking, writing, analysis and synthesis skills in both French and English;
• Mastery of food security and resilience-related questions will be considered an asset.

3.4. Timeframe

The work of the final evaluation should be accomplished ​​during the month of May 2016. The final report should be made ​​available no later than May 25^Th, 2016. The consultant should propose a detailed schedule of activities to be undertaken.

All prospective evaluators should provide both a technical proposal (including proposed methodology, description of past experience, and the CV or CVs of the consultant or evaluation team) and a cost proposal.

Apply at: www.mercycorps.org/careers

Click on Consultancies, search by region