Request For Proposals - NORAD Midterm Evaluation

About this position

RIGHT TO PLAY

Right To Play is an international humanitarian organization that uses the transformative power of sport and play to promote the holistic development of children and youth in the most disadvantaged areas of the world. Right To Play is committed to improving the lives of children and young people and to strengthening their communities by translating the best practices of sport and play into opportunities to promote learning, health and peace.

Right To Play programs foster healthy physical, social and emotional development of children and build safer and stronger communities. Right To Play’s guiding principles are inclusion – which recognizes the importance of children who may be marginalized for reasons of gender, ability/disability, religion, ethnicity, or social background; child protection and safeguarding; and sustainability – which ensures lasting impact.

As such, Right To Play is seeking a consultancy firm/group to undertake the midterm study and learning study for one of its multi-country education programs including planning and revising; data collection; review and cleaning; data analysis; reporting; and dissemination documents. This will include the writing of all accompanying reports as specified in the key deliverables. 


PROGRAM OVERVIEW
 

Project Name	The Enhancing Quality Education through the Power of Play Program (TAALOM in Jordan, Lebanon, Palestine and PAQE in Mozambique, Tanzania, Ethiopia)
Project Locations	Jordan, Lebanon, Palestine, Mozambique, Tanzania, Ethiopia
Project duration	January 2016-2019
Donor	Norwegian Agency for Development Cooperation (NORAD)
Targeted Direct Beneficiaries in each country	Children and Youth: Palestine 13,573 (school) Lebanon 24,000 (school), 2200 (community) Jordan 28,935 (school), 6,216 (community) Ethiopia 30,598 (school), 400 (community) Tanzania 72,700 (school), 5400 (community) Mozambique 60,613 (school), 4400 (community)

The Enhancing Quality Education through the Power of Play program is a four-year initiative supported by Norad which began in 2016. The initiative seeks to achieve the ultimate outcome of, “Enhanced quality of education for girls and boys in Ethiopia, Tanzania, Mozambique, Palestine (West Bank and Gaza), Lebanon and Jordan.”

While all countries have contextualized logic frameworks, the three main outcomes across all countries include:

1. Improved learning environments in participating schools through play-based learning
2. Improved life skills of participating children through sport and play-based learning
3. Strengthened organizational capacity of civil society organization partners

To improve the learning environments of participating children Right To Play is aiming to build the capacity of teachers to integrate play-based learning into the teaching process by providing in-service teacher training. Throughout the duration of the program teachers will progress through the three-year Continuum of Teacher Training (CoTT) completing sessions on play-based learning and the creation of a positive learning environment which includes orienting teachers to issues related to child safeguarding and protection, gender equality and inclusion.

Through participation in play-based learning activities, both in the classroom and in the community, children will have opportunities to build their life skills. Right To Play activities are designed to foster healthy child and individual development and strengthen life skills.

Across all countries, the program is grounded in a new organizational civil society impact strategy that emphasizes strong civil society partnerships to achieve scalable and sustainable program impact. In order to nurture strong civil society partnerships, Right To Play is focusing on developing ways of collective learning, capacity building and advocacy with partners.

In addition to these outcomes that are measured across all countries, each country has a contextualized results framework in which some countries have additional outcomes which also require measurement. These include

• Barriers to education are addressed through community-based interventions (Lebanon)
• Improved awareness of community and other stakeholders of barriers to education (Ethiopia)
• Improved capacity of education supervisors and educators to integrate play-based learning methodology within the teaching and learning process (Ethiopia)
• Improved commitment of community stakeholders to address barriers to education (Mozambique and Tanzania)
• Strengthened education systems to promote gender-sensitive child-centred play-based learning policies and practices (Mozambique and Tanzania)

 
The table below highlights the list of outcome indicators being used in each of the countries to measure the project’s performance.  
 

Global Indicators (Measured across all countries)
Improved learning environments in participating schools through play-based learning	% of trained teachers who demonstrate application of play-based methodology (disaggregated by M/F)
	% of learning environments that meet Right To Play's positive learning environment principles
Improved life skills of participating children through sport and play-based learning	% of participating children that demonstrate improved life skills  (disaggregated by M/F)
Strengthened organizational capacity of civil society organization partners	# and % of CSO partners with improved organizational capacity (disaggregated by type of capacity)
Lebanon Specific Indicators
Barriers to education are addressed through community-based interventions (	Perception of community members in relation to access and quality of education
Ethiopia Specific Indicators
Improved awareness of community and other stakeholders of barriers to education	Level of awareness of target communities of barriers to education in the community
Improved capacity of education supervisors and educators to integrate play-based learning methodology within the teaching and learning process	% of education supervisors with increased capacity in coaching and mentoring teachers for the integration of play-based learning methodologies within the teaching  & learning process
Tanzania and Mozambique Specific Indicators
Improved commitment of community stakeholders to address barriers to education	Number of community leaders/civil society organizations that initiate/ lead and/or support children's rights and girls education initiatives to address barriers to education
Strengthened education systems to promote gender sensitive child-centred play-based learning policies and practices	Perceptions of education stakeholders at the local, district and national level on the degree of implementation of play-based learning policies/approaches in education strategies, annual plans and priorities

PURPOSE OF THE EVALUATION  The purpose of the evaluation:

1. To produce results on progress towards outcomes included in the global and country results frameworks, which allow for comparison to baseline findings and program targets.
2. To provide clear, relevant recommendations for program improvement.
3. To provide information on best practices and lessons learned for future program planning and development of Right To Play’s work in education.

The midterm study for the Norad program will be conducted using a phased approach to ensure that midterm data is analyzed for all countries by March 29^th, 2019. Right To Play anticipates an overarching timeline of July 2018 to July 2019, whereby data collection will occur in October and November 2018 in the Africa region and February 2019 in the Middle East region. This timeline will allow for the review and revision of existing measurement documents, planning for data collection, and the development of both global and country specific midterm reports. The timing for data collection was set based on school schedules in each country.
 
The initial phase of the midterm study will involve a comprehensive review of all existing Norad program, monitoring, and evaluation plans including the performance measurement framework, evaluation and analysis framework and measurement tools.

A fieldwork/data collection administration and logistics plan will be developed and implemented in the six countries of implementation, ensuring collaboration with and some knowledge transfer to Right To Play country teams. Travel will be required and the consultancy team will be required to develop a robust data collection strategy for each country as well as ensure the ability to source, hire and train (as needed) enumerators, research assistants, and/or associates with the ability to collect large amounts of qualitative and quantitative data.

The final phase of the midterm study process will consist of the review, cleaning and analysis of data collected and drafting of multi-country and country-level reports. The consultancy team must also produce relevant communication and dissemination documents (presentation slides for example) that summarize findings from the evaluation at the global and country levels.

In addition to the data collection for the midterm evaluation, some additional data will be collected to inform a learning study. This learning study will focus on one country, potentially Tanzania, and focus on one key research question related to the implementation of play-based learning. The specific research question and scope of the learning study will be determined through discussions with key Right To Play offices once a consultancy group has been selected. The consultant will work with Right To Play to develop a measurement strategy for this research question as well as develop any additional data collection instruments that are required. A separate report will be required to present the results from the learning study.

For further details please reference the key deliverables section.

EVALUATION THEMES AND QUESTIONS

In addition to providing data on the specific indicators outlined in the project’s performance measurement framework, the mid-term assessment should also respond to the following specific evaluation questions:

Effectiveness: assess the achievements of the project in relation to its intended outcomes/results. This should be a systematic assessment of progress against the performance measurement framework. Other factors to consider in assessment of effectiveness include:
• Which outcomes/results have been achieved both expected and unexpected?
• Which objectives/outcomes have been most useful and successful?  What were the success factors?
• What changes occurred in the performance/behaviour of beneficiaries?
• What is the scope/magnitude of the change achieved in comparison to baseline? 
• What was not achieved (failures, missed opportunities, challenges) and why?

Impact: assess the extent to which the project is contributing to its ultimate outcome and to a long-term positive effect on the target groups/beneficiaries
• How has the project contributed to reported changes and to what extent can the changes be attributed to other external factors?
• Have there been any unintended results/outcomes at this point?

Sustainability:
• Are there indications that elements of the project will be sustainable over the coming years? What are these and what are the factors which will determine whether or not they will continue?
• Are there indications that the project contributes to lasting benefits? Which organizations could/will ensure continuity of project activities in the project areas?

LOCATION

The evaluation planning phase is expected to be carried out at the consultants’ base, with significant collaboration with Right to Play’s Monitoring, Evaluation and Learning team. Face to face meetings may be required, and as such, travel will be expected.

For the field work and data collection, extensive in-country travel is expected within all six implementing countries (Lebanon, The Palestinian Territories, Jordan, Tanzania, Mozambique and Ethiopia). The evaluation findings dissemination period may also require travel to these countries, but this is to be confirmed.


METHODOLOGY

The first phase of the consultancy will involve a document review of the baseline study evaluation framework, analysis framework, sampling framework, accompanying data collection instruments, and findings. Some revision to the data collection instruments used at baseline may be required.

The second phase of the consultancy will involve the planning and implementation of the midterm evaluation that will enable Right To Play to determine project contribution to the achievement of expected outcomes. In addition, the learning study may be planned and implemented simultaneously to inform strategic learning for Right To Play. As the fieldwork will occur in the countries of implementation, the consultancy is expected to collaborate significantly with Right To Play country teams in order to successfully conduct the field work.

The final phase of the consultancy is the analysis of the data collected and the production of global and country midterm reports, a learning study report and appropriate dissemination documents, all of which will need to incorporate feedback from relevant Right To Play staff.

Through significant, collaborative engagement with relevant staff at Right To Play, particularly the Monitoring, Evaluation and Learning team, it is expected that the consultancy will strengthen the organizations’ M&E capacity, particularly in the development and implementation of sound, rigorous research activities. As such, the consultancy will be expected to share and contribute to the Right To Play’s knowledge-base while deepening the capacities of the organization to more effectively conduct research in the future.

Data Collection Tools

At baseline, the evaluation design framework consisted of a toolkit of data collection instruments, which included:

• Classroom Observation tool
• Life Skills Assessment
• Focus Group Discussions (children, teachers and parents)
• Interview Guides (school administrators, teachers and CSO partners)

The research planning process will entail an in-depth training of enumerators. Consideration will be given to firms who have implemented data collection using mobile tools and software depending on country and financial proposal.

Data Analysis

Data will be analyzed both quantitatively and qualitatively and will cover all areas included in the performance measurement framework. An analytical framework was developed as part of the evaluation design process at baseline, which details the specific analytical methodology that was used to produce the result for each indicator measured. For the midterm study, the selected consultant will be expected to review this analytical framework as a basis for the midterm data analysis to ensure that data is comparable to baseline results.

The consultant is encouraged to utilize analytical software to analyze both the quantitative (e.g. Excel, SPSS, STATA, R), and qualitative (e.g. Nvivo) data. Outputs/datasets from data analysis will be submitted as part of the deliverables, as will the scripts (or list of commands) with clear notes/guidance, particularly for quantitative data analyzed (in SPSS, STATA, R).  The consultant should be mindful of baseline analytical methodologies and software as well (e.g. the use of Excel and descriptive statistics) whilst analyzing data for this mid-term exercise.

GENERAL CONDITIONS OF THE CONSULTANCY

Engagement with Right To Play

The consultancy firm/group will liaise with key implementation stakeholders (i.e. Right To Play Country Offices including MEL Officers and relevant field staff, Regional Offices including MEL Specialist and MEL Manager and HQ MEL Officer and Director) who will be responsible for guiding and informing the evaluation process. This group will help to inform the review of the evaluation design framework, the data collection tools and the analytical framework. They will also help to ensure that the research planning and data collection processes are sound, culturally appropriate and contextually relevant to Right To Play’s programmatic needs and to the needs of all relevant stakeholders (i.e. beneficiaries, community members and partners).

Measurement Plan

The measurement plan uses a results-based management approach for tracking the progress of outputs and outcomes against targets.

Consultancy Expectations

• The consultancy firm/group will have an orientation to Right To Play’s program delivery model;
• The consultancy firm/group will submit a detailed work plan and time frame for the completion of the mid-term evaluation research;
• The data analysis and draft reports will be shared with Right To Play with sufficient time to allow for Right To Play to review and provide feedback, which will be incorporated into subsequent work;
• After the completion of any evaluation reports, a presentation will be given to Right To Play staff and appropriate stakeholders to share both results and recommendations;
• Throughout each phase, the consultancy firm/group will be expected to maintain regular communications with Right To Play regarding progress;
• Travel and accommodations for visits to Right To Play intervention countries will be coordinated, booked and paid for by Right To Play as outlined in an agreed upon contract unless otherwise discussed with Right To Play;
• The consultancy firm/group must follow OECD-DAC and AfrEA principles for evaluation;
• All materials, processes, methodologies, reports, plans and other works provided to the consultancy firm/group or developed by the consultancy firm/group on behalf of Right To Play remain the property of Right To Play;
• All data must be stored in a safe and secure location, allowing full access to Right To Play staff during the evaluation process;
• Upon completion of the evaluation, all raw data must be submitted to Right To Play.

ROLES & RESPONSIBILITIES (SUMMARY)

The consultancy firm/group will report directly to Emily Kere, Director, Global Monitoring, Evaluation and Learning at Right To Play International, and all deliverables should be submitted according to the schedule outlined in the “Key Deliverables and Timeline” section below.

The consultancy firm/group’s roles and responsibilities include, but are not limited to, the following:
 

• Background research to familiarize themselves with the program and Right To Play’s approach (i.e. program documents provided by Right To Play);
• Document review of program, monitoring and evaluation plans, including performance measurement frameworks and tools;
• Review/development of the evaluation design/framework in collaboration with the Right To Play;
• Development and implementation of a fieldwork/data collection administration and logistics plan in the six countries of implementation;
• Recruit and manage in country data collection teams who are fluent in the local language;
• Data collection, entry & cleaning, and analysis;
• Data validation and interpretation through internal review of findings;
• Completion of final reports;
• Preparation and presentation of findings and recommendations.

KEY DELIVERABLES AND TIMELINE (NEGOTIABLE AND TO BE FINALIZED WITH RTP STAFF)
 

DELIVERABLE	DETAILS	DEADLINE
Question Period	n/a	May 17th-June 8th 2018
Submission of Proposal	Please include: Cover letter/expression of interest A complete profile of the firm/organization/group, highlighting previous experience and expertise in areas listed in the “Qualifications” section detailed in the above section; CVs of any other key team members who will be the part of the Multi-phased evaluation team Two writing samples, ideally reports the firm/organization/group has lead authorship on	June 8th 2018
Interviews	n/a	June 18th-29th 2018
Award of Contract	Contract to be signed	July 13th 2018
Right To Play Consultations and document review	Initial consultations will begin this week and extend throughout the duration of the planning period	July 16th-20th 2018
Review/develop evaluation framework, analysis framework and sampling frameworks	Consultant to review existing documents from the baseline evaluation and make any required revisions Evaluation framework, analysis framework and sampling framework	August 3rd 2018 (Midterm Evaluation) *Student lists may not be available until September, timeline for sampling framework will be adjusted accordingly
Data Collection Instruments (Review/Revise/Develop)	Consultant to review existing data collection instruments from the baseline evaluation and make any required revisions	August 24th 2018
Fieldwork (Data Collection) Plan Completed	Fieldwork logistics and schedule (i.e. in Excel) Fieldwork manual including data collectors’ training guide	September 14th 2018
Data Collectors Training Completed	Data collectors training agenda Data collectors’ training report Pilot of tools	Africa: September 24th-28th 2018 MEA: February 11th-15th 2018
Fieldwork Completed	Fieldwork completion reports	Africa: October 1st-November 30th 2018 MEA: February 18th-March 1st 2019
Data Collected	Cleaned, raw data in two formats: Excel and a stats software format (e.g. SPSS or STATA) if possible	Africa: December 14th 2018 MEA: March 15th 2018
Analyzed data	Populated performance measurement framework with analyzed data	March 29th 2019
Validation	Facilitate validation workshops with country staff and key stakeholders	Africa: January 4th 2019 MEA: March 29th 2019
Draft Mid-term evaluation Reports	Revised data analysis framework document Draft overall (multi-country) report Draft country reports (x6)	May 10th 2019 (Midterm Report) May 24th 2019 (Learning Study Report)
Final mid-term evaluation Reports	Finalized data analysis framework document Excel spreadsheets, or SPSS/Stata scripts with clear, easy to understand notes Final overall (multi-country) report Final country reports (x6) Final summary PPT	July 26th 2019

The above timeline is negotiable and subject to change based on consultation with Right To Play International, Regional and Country staff.

PROPOSED BUDGET AND PAYMENT SCHEDULE

Consultants are asked to provide a draft budget proposal along with their proposal for consideration.  Right To Play International offers competitive consultancy rates in keeping with market value and international NGO standards.
 

• First payment: After signing of the contract agreement with Right To Play (10%)
• Second payment: Submission of evaluation framework design (including draft analysis framework) and relevant instruments (15%)
• Third payment: Submission of analyzed data appropriate for PMF entry (40%)
• Fourth payment: Submission of draft reports (15%)
• Final payment: Submission of final reports, PowerPoint presentations and final analysis framework approved by Right To Play (20%)

QUALIFICATIONS

• A consultancy firm/group with 5-15 years of experience in the research and/or evaluation field, including experience in quantitative and qualitative data collection, analyzing quantitative and qualitative data, and report writing;
• Extensive experience creating measurement frameworks, refining indicators, and creating measurement tools for education-focused programs
• Extensive experience managing and designing evaluation studies in diverse contexts (e.g. within multi-country contexts, with children and young people, in remote settings, etc.);
• Extensive experience in international education at the primary levels;
• Experience in life skills measurement and behavioural change measurement;
• Experience using participatory methodologies and following OECD-DAC and AfrEA principles for evaluation and measurement;

• Applicant should have a relevant degree in social sciences, international development, statistical sciences, or another related field ;
• Excellent skills and experience with data analysis using statistical computing tools (Excel, SPSS, STATA, NVIVO);
• Excellent working proficiency in English

About us

Right To Play's mission is to protect, educate and empower children to rise above adversity using the power of play.

  Right To Play is headquartered in Toronto, Canada and has operations in North America, Europe, the Middle East, Africa and Asia.

View our Impact Profile

Application instructions

Please click here to Sign In to see how to apply.
New? Register now. It's free and fast.