The Japan International Cooperation Agency (JICA) Annual Evaluation Report is a tool that supports learning from past project evaluations and ensures organisational accountability and transparency. By promoting learning on what works and what does not work in their projects, and acting on recommendations, JICA is able to improve the quality of its operations in challenging landscapes.
Learning from evaluations to improve project quality: JICA’s Annual Evaluation Report
Abstract
Challenge
Japan’s International Cooperation Agency (JICA) aims to encourage a culture of learning within its programmes. The evaluation system is one tool that supports accountability and learning across the agency. However, time constraints make it difficult to look for and extract lessons learnt from past projects when planning a project. JICA Annual Evaluation Reports strategically analyse the results and lessons from all evaluations completed during the fiscal year. In articulating how lessons learnt from evaluations led to changes that improved the quality of similar programmes, it contributes to the objective of building a culture of learning and adaptation across the agency.
Approach
The Annual Evaluation Report provides an outline of JICA’s evaluation system and presents the results of its project evaluations. It reflects the four key features of JICA’s evaluation system:
Coherent evaluation methodologies and criteria throughout JICA’s “Plan, Do, Check, Action” (PDCA) results-based management cycle, which also incorporates good practice from Japan’s manufacturing industry.
Comprehensive evaluation system comprising: i) ex ante and ex post evaluations for projects above a JPY 200 million threshold, covering technical co‑operation, Japanese official development assistance loans and grant aid, ii) cross-sectoral evaluations focusing on specific themes and aimed to guide decision making for future projects and help develop new methodologies.
Objective and transparent evaluations through publishing evaluation findings and involving the Advisory Committee on Evaluation, which consists of third-party experts, as part of the function of its evaluation system.
Use of evaluation results for learning and steering alongside accountability. Results from evaluations are fed into improving policies, project design as well as feedback with partners. JICA’s Annual Evaluation Report features a series of case studies that show how programmes have evolved as a result of an evaluation. Following the release of the Annual Evaluation Report, seminars are held within the organisation to provide feedback on the main lessons learnt during the year.
Figure 1. JICA's evaluation and learning cycle
Source: JICA
Results
Case studies featured in the Annual Evaluation Report demonstrate how acting upon evaluation recommendations can help improve a project; they highlight how to leverage lessons from an evaluation for similar types of projects, and how an evaluation can influence future programming.
The Annual Evaluation Report, through its featured case studies, contributes to promoting a culture of peer learning across the organisation. Being open to examining failures as well as successful projects has led to better decision making at the project design stage.
Staff are becoming more aware of the importance of considering lessons learnt from previous projects in decision making.
Lessons learnt
Integrating lessons at the project planning stage is critical. The Annual Evaluation Report is crucial not only for raising awareness of lessons learnt but also for incorporating these lessons into the planning process. JICA has put in place a process to use the lessons learnt from the past during ex ante evaluations. In addition, to simplify the process of extracting and exploiting these lessons, JICA is preparing a search system and compiling the lessons learnt based on a cross-sectional analysis of evaluation reports.
Being part of an international knowledge platform allows JICA to benefit from the collective knowledge of other organisations. JICA shares its practices with the Global Delivery Initiative (GDI) operated by the World Bank. GDI combines the results of various international and bilateral organisations, think tanks, private funds and non-governmental organisations, which enables JICA to learn from other sources beyond its own evaluations.
Further information
JICA, Annual Evaluation Report 2020, https://www.jica.go.jp/english/our_work/evaluation/reports/2020/index.html.
JICA, Annual Evaluation Report 2019, https://www.jica.go.jp/english/our_work/evaluation/reports/2019/index.html.
OECD resources
OECD, Evaluation of development programmes, https://www.oecd.org/development/evaluation.
OECD, “Evaluating development co-operation”, Development Co-operation Fundamentals, [forthcoming].
OECD, “Managing for sustainable development results”, Development Co-operation Fundamentals, https://read.oecd-ilibrary.org/view/?ref=1112_1112683-sr6wf7xp34&title=Managing-for-sustainable-development-results.
OECD, Results in development co-operation, https://www.oecd.org/dac/results-development.
To learn more about Japan’s development co-operation see:
OECD (2020), OECD Development Co-operation Peer Reviews: Japan 2020, https://doi.org/10.1787/b2229106-en.
OECD, "Japan", in Development Co-operation Profiles, https://doi.org/10.1787/b8cf3944-en.
