Request for Proposals:End of Project

Description :

Organization: World Vision
Country: Somalia
Closing date: 06 Mar 2017

End of Project Evaluation Terms of Reference:Primary Health Care Program Intervention**

Project Number: 202227-World Vision Somalia

Table of Contents

Evaluation Purpose

The purpose of this evaluation is to document and inform the stakeholders (donors, partners and beneficiaries) of the project’s relevance, effectiveness, sustainability, the potential impact in relation to project outcomes and the lessons learned.

Primary Methodologies

The evaluation study will adopt a mix of quantitative and qualitative techniques as summarised below:

Quantitative

§ Household surveys

Qualitative

§ Focus group discussion (FGD)

§ Key Informant Interview (KII)

§ Document Reviews

Evaluation Start and End Dates

March 15, 2017

April 30, 2017

Anticipated Evaluation Report Release Date:April 30, 2017

Description of Project Being Evaluated

World Vision Somalia (WVS) has worked with the children of Somalia, their families and communities since 1992 through a variety of emergency and rehabilitative programming to address the emergency needs of the communities while addressing some of the underlying causes of vulnerability in those same communities. During the last 25 years, the program has grown to 15 districts spread over three main operational regions of Puntland, Somaliland and South-Central with a liaison office in Mogadishu.

The strategic aim of the Somalia program is to develop and implement high quality projects that address emergency, rehabilitation and developmental needs in a demand driven and responsive manner. The overall goal of all World Vision programs is to save lives and contribute towards the economic development of the Somali people, while contributing to the child well-being outcomes. This project has been designed to fit into the strategy especially the component of saving lives through improved integrated primary health care program.

The project has been implemented for the last 2 years and will be coming to an end in May 2017. Therefore, this ToR has been prepared to hire a highly competent external Consultant to evaluate the performance of this project for the implementation period. The evaluation of this project will help the key stakeholders to measure the level of project success with reference to service delivery to the project beneficiaries.

2.1 Project Goal

The main aim of this project is improved access to quality health services for 68,400 people through Integrated Health, Protection and WASH interventions in Eyl District, Puntland State of Somalia.

Project Results Framework

Key results

Indicators

Outcome 1: Increased access to health services by target communities

# of patients accessing health services; % increase in community members accessing health services

# Of children U1 year who are fully vaccinated (Penta 3 and measles)
% increase in SAM cases accessing treatment at OTP
# of deliveries at HC

Output 1.1: Expansion of iCCM at community level

# of target communities covered by iCCM team

# of children screened for malnutrition

# of children appropriately treated for common illnesses (ARI, diarrhoea, malaria)

Output 1.2: Target men, women and children referred to adequate level of health care

# patients referred to hospital

# complicated patients admitted to hospital

# complicated deliveries conducted

Output 1.3: Provision of Primary Health Units and Health Centres

# of PHU rehabilitated

#maternity wards with surgical theatre constructed

# HC and hospital staff trained on Safe Motherhood, IMC, EPI and OTPI and Clinical Diagnosis & Treatment

Output 1.4: Improved WASH practices at health facilities

# of health facilities with clean drinking water

# health education sessions at health centres

# toilets rehabilitated

Outcome 2: Improved household hygiene and sanitation practices in target community

% of households where the mother wash hands at four critical times

% of households with appropriate faecal and waste disposal practices

Output 2.1: CHWs provide WASH information to households during quarterly visits

# Health facility staff trained on hygiene promotion

# of Households administered with WASH assessment tool by CHW
# households carrying out at least two action points from previous month’s action plan

Outcome 3: Strengthened protection mechanisms for women and children

# of GBV cases identified, referred to service providers or

% of target women that know how to access
protection services or support (check baseline)

% of mothers displaying an improvement in care techniques

Output 3.1: Establish and strengthen village GBV working groups

# community members of GBV working groups trained in GBV coordination and GBV in emergencies

# of GBV working groups with action plans and monthly meetings
Parents trained in appropriate play practices through LTP model

Output 3.2: Promote appropriate playing practices for better child development

# of parents trained on LtP.

# health providers with increased knowledge on Learning through Play

3. Evaluation Target Audiences

The project evaluation is intended to benefit multiple stakeholders that have been involved directly or indirectly in the project implementation process. In particular, the following stakeholders will be involved in the evaluation process:

• Project beneficiaries
• Community health workers
• Ministry of Health (MoH) of Puntland and the various departments
• Local government authorities in Eyl district
• Local and international organisations that are operating within Eyl District
• Donor (Government of Ireland)
• WVS staff among others.

4. Evaluation Type

This is an End of Project Evaluation that will assess the progress made by the project towards achieving the project goal of improving access to quality health services for 68,400 people through Integrated Health, Protection and WASH interventions in Eyl District, Puntland State of Somalia. The assessment of the project impact will focus on the contribution made by the project from inception.

5. Evaluation Purpose and Objectives

The main purpose of this evaluation is to assess the impact, appropriateness, relevance, effectiveness and sustainability of the primary health care project, and this should help to draw and document key lessons learnt and the best practices to the project stakeholders. In particular, the project evaluation will be shaped by the following specific evaluation objectives:

Evaluation objectives

Issues

Assess appropriateness of project design in respect to the needs of the community

Community involvement and participation in the design process, goal setting, planning and implementation. In particular, the relevancy and appropriateness of project design to the needs of the community.

Assess the scope of achievements of the project against the set objectives

What are the achievements against set objectives?

Compare actual with planned outputs and how have outputs been translated into outcomes. The evaluation shall also establish the possible deviation from planned outputs and likely outcomes.

Assess the impact of the project interventions on the community

What has been the impact of the project interventions on the community? In addition, establish the level of impact on the target beneficiaries.

To determine how primary health care project involved and benefited the community throughout the planning, design, implementation and, monitoring processes

How equitably has the project benefited the; women, men, boys and girls. Disaggregated benefits of the project by gender.

Assess the level of community participation/involvement

Are the community members knowledgeable and supportive to the project? Has there been any collaborations and networking with the different stake holders? Is there evidence of community contribution and ownership of the different project interventions?

Assess adequacy and efficiency of planning, monitoring and reporting systems, and use of both material and financial resources

How adequate were the available resources qualitatively and quantitatively? Were all the project resources utilized optimally?

Explore alternative low cost approaches that could have been used to achieve similar results? How could the efficiency of the project be improved without compromising outputs?

Assess the timeliness of implementing the project activities. How adequate were the reporting and monitoring systems of the project? Have the project outputs been achieved with a reasonable cost?

Asses the overall management and structure of the project

Are there sustainability plans, structures and skills in place to ensure there is sustainability of project benefits? How adequate are they? How is the community and local partners prepared to continue with the project outcome?

Assess the strength, areas of growth and challenges of the project

How did these affect the project implementation and how were they handled? Identify opportunities for the project if any and how they were exploited/ neglected

Provide specific and practical recommendations and document lessons to be utilized for the future projects

The best practices that have been adopted and how can they be replicated in future projects or partners for adoption.

6. Evaluation Methodology

The evaluation methodology will be designed in alignment with World Vision’s Learning through Evaluation with Accountability and Planning (LEAP) guidelines and principles to ensure the quality of evidence. The data collection process will apply both quantitative and qualitative methods. However, more detailed evaluation methodology will be designed by the external Consultant in consultation with WVS Quality Assurance and Strategy team and the Project Manager who will determine the sampling strategy and procedures. The detailed design of methodology must include the following;

§ The evaluation design

§ Data collection instruments, protocols and procedures

§ Procedures for analysing quantitative and qualitative data

§ Data presentation/dissemination methods.

§ Report writing and sharing etc.

The key data collection methods will include the following among others.

§ Document reviews including the project proposal, monthly, quarterly monitoring reports and project review reports.

§ Focus Group Discussions (FGD) involving primary project participants and

§ Key Informant Interviews with WV Somalia staff, partners, other NGOs, leaders.

§ Reflection and feedback sessions with staff and partners.

The Consultant will be expected to employ use of mobile data collection and Geographical Information System (GIS) tools in the evaluation process; ranging from data collection, analysis and presentation of results.

7. Evaluation Deliverables

The Consultant will be expected to deliver the following outputs:

§ An inception report detailing methodologies to be used and sample size calculations, a detailed execution plan, data-collection tools.

§ Draft report submitted to WVS within an agreed timeline between the WVS and the Consultant (soft-copy)

§ A presentation of the key findings and recommendations to WV Somalia and other stakeholders in Garowe (this is optional depending on if the consultant chooses to remain in country during report write up, however, consultants able to complete this deliverable will be favored. Either way, an online presentation will be required to be made to WV Somalia).

§ Collected data (raw) after analysis submitted to WV Somalia alongside the final report.

§ Final report (soft copy) submitted to QA & Strategy Manager and WVS Health and Nutrition Program Manager. However, the Consultant should note that, the final evaluation report shall follow the structure below:

i. Table of Contents

ii. Acknowledgements

iii. Affirmation

iv. Glossary/Acronyms and Abbreviations

v. Executive Summary

1. Introduction/Background
2. Methodology
3. Limitations
4. Findings
5. Conclusions and Recommendations
6. Lessons Learned from the evaluation process
7. Appendices.

8. Lessons Learned

§ Orientation and training of both internal and external data collectors, as well as the pilot testing of contextualized evaluation tools, KII and FGD tools should be given a thorough consideration; hence a considerable time should be planned for these tasks.

§ Data collection quality control mechanisms and data validation scheme should be imbedded into the evaluation design to ensure that the findings are valid and sufficient for informed decision making.

9. Time frame

The overall evaluation process is expected to take 45 days including preparation, data collection, and analysis and reporting. The Consultant should be able to undertake some of the tasks concurrently to fit within the planned time-frame, without compromising the quality expected. The assignment is expected to commence on March 15 2017, with the final evaluation report expected by April 30, 2017.

10. Authority and Responsibility

WVS will establish an evaluation team to oversee all the related tasks. The DME Manager-Northern region will be responsible for the overall coordination of all the evaluation tasks with the Consultant. In addition, the Project Manager, Regional Operations Manager and Quality Assurance & Strategy Manager will provide all the necessary technical and operational support required throughout the evaluation process.

Support from WV Somalia

World Vision Somalia will be responsible for the following:

§ Recruit the external Consultant and finalize the consultancy agreement

§ Share all necessary documents to the Consultant to finalize the evaluation methodology and data collection tools

§ Provide input for evaluation study methodology, data collection tools and report.

§ Ensure that input from WVS is circulated and shared with external Consultant and vice versa

• Flight expenses for the Consultant to Puntland
• Vehicle hire to support the evaluation exercise
• Food and accommodation for the consultant in Puntland
• Working space for the Consultant while in Puntland
• Recruitment and payment of the enumerators and translation of questionnaires, where necessary
• Stationery for data collection

§ Overall accountability of the evaluation process

§ Guidance and coordination throughout all the phases of evaluation, keeping communication with external Consultant throughout all phases

§ Provide support to the evaluation technical lead (external Consultant) for the evaluation field visits processes such as orientation and training of enumerators, FGDs and KIIs

§ Closely follow up the data collection process, ensuring quality control, daily debriefing, meeting the timelines set for interview completion;

§ Inform evaluation audience for their involvement in the study and help in setting specific dates for the evaluation field schedule.

The Consultant will be responsible for the following:

§ Review all relevant documents for evaluation study

§ Develop evaluation study design which includes survey methodology and the data collection tools (questionnaire; focus group guides, interview protocol, data entry templates, etc.) , as appropriate, including a field manual for training, in consultation with evaluation team, reflect WVS feedback on the methodology

§ Designing the data entry template, procedures and systems, and training of entry clerks in the use of the template,

§ Develop the field work schedule in consultation with evaluation team

§ Conduct training for the data collectors during field visits phase, finalize the evaluation schedule

§ Supervise the data collection process, give advice and ensure the quality of the data

§ Conduct interviews (KII) with key project staff ,

§ Data analysis and report writing, draft the first report and include WVS feedback and finalize the report for submission

§ Provide required data that is completed and labelled in English (variables and values) for both the SPSS and Microsoft file formats.

§ Provide daily field briefing to the DME and Project Managers on the progress and any challenges from the field.

11. Limitations

Time and security may be a major limitation with regard to assessment processes in fragile and versatile contexts such as Somalia and this makes it often challenging to keep up strictly with a set agenda. In addition, In Somalia; households spent a better part of the afternoon hours in prayers and it will be hard for the enumerators to administer many questionnaires per day (in an effort to complete the assessment timely). To address this issue, firstly WVS will allocate extra overflow days for field data collection. WVS team will also work closely with the security department to ensure that the evaluation field processes are conducted in the most appropriate time and secure conditions.

12. Documents

The key documents to be reviewed for the baseline study are as follows:

§ Project document (needs assessment, proposal, logframe)

§ Baseline Report

§ Monthly, quarterly, semi-annual and annual reports

§ Training reports

§ Success stories

§ Patient registers

§ Any district level secondary data and other relevant documents and reports.

13. Qualifications of the Consultant

The evaluation exercise will be undertaken by an external Consultant who will work in close collaboration with the project manager and Quality Assurance and Strategy team. Therefore, we are looking for a Consultant/team with the following skills and qualifications;

§ The team leader must possess a post graduate degree in Public health, social sciences or related discipline.

§ Strong and documented experience in conducting participatory qualitative assessments related to health and nutrition

§ Demonstrated experience in leading at least three similar project evaluation studies such as surveys and group interviews,

§ At least 10 years’ experience in conducting baseline/evaluations for complex projects such as livelihood, education, health and nutrition, water and sanitation and hygiene being implemented by non-governmental and private sector actors.

§ A solid understanding of remote learning through mobile technology, child health and child care, water and sanitation.

§ Demonstrated experience leading teams.

§ Demonstrated experience in training local staff in quantitative and qualitative data collection tools including entry template

§ Demonstrated experience in designing survey methodology, data collection tools, processing and analysis of data.

§ Ability to interact with host government, partners and/or others as requested by WVS;

§ Strong organizational, analytical and reporting skills, presentation skills, attention to detail, ability to meet deadlines, and proficiency in Microsoft Office and qualitative data analysis software/tools.

§ Previous experience in a fragile country with tight security context will be preferred.

• Capacity to use mobile data collection and GIS tools for data collection, and analysis of survey results will be an added advantage.

§ Excellent verbal and written communication in English required;

How to apply:

Qualified and interested parties are asked to submit the following;

§ Letter of interest in submission of a proposal

§ A detailed technical proposal clearly demonstrating a thorough understanding of this ToR and including but not limited to the following;

o Consultant/Company Profile

o Description of the Methodology and Sample Size Determination

o Demonstrated previous experience in similar assignments and qualifications outlined in this ToR.

o Proposed data management plan (collection, processing and analysis).

o Proposed timeframe detailing activities and a work plan.

o Team composition and level of effort of each proposed team member (include CVs of each team member).

§ A financial proposal with a detailed breakdown of costs for the study quoted in United States dollars.

 

• Email to Friend
• RSS
• Print

Email to a Friend

Friend Name :

Friend Email :

Your Name :

Your Email :

Comments :