Georgia: Final Evaluation of the Project

1.Background

People in Need is a Czech non-governmental organization that provides relief aid and development assistance, while working to defend human rights and democratic freedom. It is one of the largest relief and development organizations in Central/Eastern Europe and has administered projects in 37 countries over the past 20 years. People in Need has been working in Georgia since 2005 and has offices in Tbilisi, Kazbegi and Kutaisi.

About project: The project - Support to Social Inclusion of Youth with Disabilitiesin Georgia - is focused on improving the process of social inclusion of youth with disabilities. The core activity of the project is to build capacity of the state institutions and facilitate their cooperation among themselves and with the nongovernmental sector in order to improve the support provided to youth with disabilities through education and employment services. It strives to achieve results mainly on the national level.

Overall objective of the project is to contribute to improving the quality of life and social inclusion of youth with disabilities in Georgia through inclusive education and support of employment in the labor market**.**

The project anticipated results are as follows:

1. The capacities and co-operation of both state and non-state institutions providing support and services to youth with disabilities in education and in employment is increased.

2. The involvement and inclusion of youth with disabilities in both formal and informal education is strengthened

3. The capacities of youth with disabilities to become involved in the labour market are strengthened.

Project implementation period: May 2015 – December 2017

Project Budget: 210 000 EURO

Project Donor: Czech Development Agency

Leading implementing organization of the project: People in Need (PIN)

Co-funding and implementing partner: Children and Youth Development Fund (CYDF)

Other implementing partners: Georgian Association of Social Workers (GASW)

Key Stakeholders: Ministry of Education and Science - MoES, Social Services Agency- SSA (Ministry of Labour, Health and Social Affairs - MoLHSA), Children and Youth National Center – CYNC, Ministry of Sports and Youth Affairs –MoSY.

Direct Beneficiaries:

15-20 Representatives of CYNC

30 Representatives of state institutions, NGOs and parent groups

10 Representatives of state institutions (MoSY, CYNC, CYDF, MoES, SSA)

Representatives of 6 youth initiatives, supported by PIN

Representatives of 5 local NGOs supported by CYDF

Two public schools in Tbilisi.

End Beneficiaries:

200-300 young people with disabilities, their families and parents.

2.Objective and Scope of the Evaluation

The evaluation shall provide an independent assessment of the Project's relevance, quality and impact of activities implemented by PIN and its partner organizations GASW and CYDF.

The evaluation objective: The overall goal of the evaluation is to provide an independent assessment of the project’s implementation in relation to the achievement of its overall goal and specific objectives as provided in the project proposal and logframe, including the project’s performance with respect to each of its component. The evaluator should also look into the sustainability of concrete changes in the area of implementation of the project and, based on the information gleaned from the assessment, provide recommendations (methodological or otherwise) for future work. This information and the relevant recommendations may be used as a basis for planning and implementation of further programmes and activities contributing to the enhancing quality of life of young people with disabilities in Georgia.

Below are the evaluation’s objectives, which are of particular interest to PIN.

Objective 1: Assess the extent to which the project contributed to strengthening the capacities and co-operation of both state and non-state institutions providing support and services to youth with disabilities in education and in employment.

Objective 2: Assess whether the project contributed to enhanced involvement and inclusion of youth with disabilities in both formal and informal education.

Objective 3: Assess whether the project contributed to strengthening the capabilities of youth with disabilities to become involved in the labour market.

Objective 4: Develop recommendations for PIN for future interventions focused on promoting social inclusion of young people with disabilities in Georgia. The recommendations must be in line with the mission strategy.

PIN will require specific and data-supported answers to each of the agreed evaluation questions or strong justification for why the data could not be obtained. Justification of data unavailability must be communicated by the evaluator without any delay as soon as it becomes apparent and approved by PIN. Failure to do so can result in decreasing the consultant’s remuneration.

Scope

Evaluation of project components implemented by partners and PIN to support the quality of life and social inclusion of youth with disabilities in Georgia through inclusive education and support of employment in the labour market

Geographical scope: Tbilisi, Imereti Region, Kazbegi, Gori and Borjomi

Period under review: May 2015 – December 2017

3.Methodology

The Consultant will be responsible for developing the evaluation design and methodology in order to meet the evaluation’s goal and objectives. Details about the approach and the data collection methods to undertake this task should be included in the bid. The evaluation is expected to apply qualitative methods of research - semi structured interviews, focus group discussions, desk research of related secondary data including project and the program documentation. The methodology is subject to discussion with PIN.

The work to be carried out by the Consultant will be based on the following four criteria:

• Relevance - including the extent to which various problems and needs addressed by the Project are relevant in the Georgian context;

• Effectiveness - including the extent to which the Project objectives have been met;

• Efficiency - including the extent to which the objectives of the project were achieved in a cost-effective manner;

• Impact - the extent to which projects efforts contributed to the achievement of the project’s overall goal;

• Sustainability - the extent to which outcomes of the project have been sustained or are likely to be sustained in the future.

CriteriaQuestions:

The evaluator is expected to complete this list with additional questions to better assess the relevance, efficiency, effectiveness, impact, and sustainability of the project.

Questions

Criteria for evaluation

1.

a. Was the project designed based on the needs of the target group?

b. How relevant is the project to the Georgian Context?

c. How relevant is the project to other stakeholders (MoES, MoSY, CYDF, CYNC, SSA)

d. To what extend did project activities contribute in reaching the outcomes?

e. Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?

Relevance

2.

a. How effective is the project in achieving the stated objectives and outcomes?

b. To what extent has project contributed to enhanced involvement and inclusion of youth with disabilities in both formal and informal education and labour market?

c. What is the quality and effect of the project efforts towards strengthening the capacities and co-operation of state and non-state institutions providing support and services to youth with disabilities in education and in employment?

d. Which methods/activities utilized by PIN, implementing partners or beneficiaries have been successful or less successful?

e. What were the major factors influencing the achievement or non-achievement of the objectives?

Effectiveness

3.

a. Were the resources and inputs (of PIN and project partners including financial and non-financial) converted to outputs in a timely and cost-effective manner?

b. How efficient has the capacity building of state stakeholders (CYNC, CYDF, SSA) been in improving quality of services provided to youth with disabilities?

c. Was the management, coordination and monitoring of the project efficient and appropriate?

Efficiency

4.

a. Has the project contributed or is likely to contribute to the employment or inclusion of young people with disabilities in the formal/non formal education and labour market?

b. What real difference has project made to target groups (intended or unintended, positive or negative)?

c. What can be observed as the most significant change due to the project?

d. How many people have been effected?

Impact

5.

a. To what extent are the project results (outcomes) likely to continue after the project?

b. What were the major factors that influenced the achievement or non-achievement of sustainability of the project?

c. What ownership do state institutions take and what aspects still need external support (financial, institutional, political, social aspects)?

Sustainability

Scoring the assessment criteria

For this evaluation, a common scoring system is used to assess project performance against the OECD-DAC, scoring as follows:

0 Low or no visible contribution to this criteria

1 Some evidence of contribution to this criteria but significant improvement required

2 Evidence of satisfactory contribution to this criteria but requirement for continued improvement

3 Evidence of good contribution to this criteria but with some areas for improvement remaining

4 Evidence that the contribution is strong and/or exceeding that which was expected of the intervention

Specific Tasks to be performed by the Consultant under the supervision of and in cooperation with PIN and project partners – CYDF and GASW

· Evaluation of the project

· Provision of recommendations

· Work together with a Project Officer from PIN, whose objective is to earn experience of evaluator's fieldwork, gain understanding of evaluation instruments, and assist the external evaluator (consultant) in organizing his/her activities.

The consultant is expected to conduct informal interviews and focus group discussions with following respondents:

§ CYDF representatives

· PIN representatives

· GASW representatives

· CYNC representatives

· SSA, MoES, MoSY representatives

· Grantee youth organization’s and schools representatives

· Focus group discussion with beneficiary youth

· Focus group discussion with beneficiary youth parents

· Focus group discussion with the local community representatives.

4.Specific Tasks

• Review of project documents

• Review of project logic and targets

• Meetings with project staff and relevant stakeholders

• Review of partnership relations between the implementing organizations

• Provision of recommendations for future programming

• Preparation of evaluation documents (inception report, draft report, final report)

5.Expected Deliverables

· Evaluation work plan /design–outline of methodology and data collection methods (derived from the methodology) and timeframe of evaluation

· Evaluation report in accordance to the criteria outlined in ToR;

· Final report that should include:

a) Executive summary (max.2 pages)

b) Brief description of the context and the project;

c) Description of evaluation goals and objectives;

d) Description of evaluation methodology and data collection methods;

e) Presentation of main finding and conclusions of the assessment;

f) Discussion of lessons learned;

g) Recommendations

h) Evaluation summery in PPT format.

Recommendations should:

Consultant is expected to conduct the presentation of key findings and recommendations to PIN and the project partners. All deliverables to be submitted in English via email to PIN.

All deliverables are subject to PIN review and the Consultant will be expected to respond to comments for improvement on all of these deliverables.

The evaluation report will be the sole property of PIN, which will retain the right to use it for internal and external purposes

6.Evaluation Report

The Final Report should include the following sections:

EXECUTIVE SUMMARY: Summary of the evaluation, with particular emphasis on the main findings, conclusions, lessons learned and recommendations (max. 3 pages)

INTRODUCTION: Description of the evaluated intervention, its logic, history, organization and stakeholders. Presentation of the evaluation’s purpose and questions.

METHODOLOGY: Description of the sampling strategy and methods used for data collection; description of the limitations.

FINDINGS: Factual evidence relevant to the questions asked by the evaluation and interpretations of such evidence (answered evaluation questions).

CONCLUSIONS: Assessments of intervention results and performance against given evaluation criteria.

· Problems and needs (Relevance)

· Achievement of purpose (Effectiveness)

· Sound management and value for money (Efficiency)

· Achievement of wider effects (Impact)

· Likely continuation of achieved results (Sustainability)

· Capacity building and partnerships

· M&E processes

LESSONS LEARNED: General conclusions with a potential for wider application and use.

RECOMMENDATIONS: Specific and actionable proposals regarding

improvements for PIN for future programs and interventions in the field of support to school to work transition and employment of young people with disabilities.

ANNEXES: All data collection tools, list of interviewees, etc.

The evaluation report Executive summary should not exceed the limit of 3 pages, and the remaining parts of the report should not exceed 25 pages (excluding annexes).

7.Duration

The Consultant should perform all of the tasks under this Terms of Reference during the assignment period. Expected workload is approximately 25 working days. Deadline for submission of the final evaluation report is December 10 2017.

8.Qualification requirements

Education

· Degree in social sciences, international development or other related studies

Work Experience

· At least 5 years of relevant professional experience in the civil society development, youth, and PWDs social inclusion, preferably in Georgian context.

· At least 5 years of relevant professional experience in evaluation of civil society organization’s or public projects programs

· Significant practical experience in report writing and formulation of recommendations.

Skills

· Strong analytical, organizational and communication skills

· Interest in youth and civil society development

· Ability to meet deadlines and prioritize tasks

· Fluency in English and strong writing skills required; fluency in Georgian is preferred

9.Budget

The budget of the evaluation should not exceed EUR 3 500 (GROSS) for evaluation services, including expenses for transportation to and within Georgia, accommodation, per diem, Georgian taxes (if applicable). All proposals exceeding the maximum budget will be ineligible.

Deadline for applications: September 20, 2017 at 17:00.

Interested applicants must submit the following:

· Cover Letter describing relevant professional experience and detailed description of qualifications (in English);

· Full CV (in English) with a list of previously undertaken assignments in the same field;

· Contacts of two references including the supervisor in the most recent work assignment;

· Proposed evaluation methodology and work plan ( max. 4 pages);

· Price offer and budget

The above shall be sent to the following address: georgia.admin@peopleinneed.cz and use subject heading: Ref: Evaluation Consultant/Social inclusion.

Incomplete applications will not be considered.

Requested documentation will be reviewed and only shortlisted candidates will be invited for the interview. The final decision will be made after the interview, considering candidates relevant experience, qualification and cost efficiency of the offer.

No phone calls please.