Managing for sustainable results in development co-operation in uncertain times
Abstract
The coronavirus (COVID-19) pandemic poses the single largest short-term challenge to global development in several decades. Its short-term effects are disrupting the capacity of development co-operation to help partner countries react to this threat. But the crisis also offers a one-time opportunity for institutional learning and development. If seized, this could help strengthen the adaptability muscle of management systems, to innovate in data collection methods, and to ensure that decisions are guided by evidence in order to maximise results and impact. This policy brief is intended to help reflect on measures being considered in development co-operation that could ensure results during and after the coronavirus (COVID-19) pandemic.
The outbreak of the COVID-19 virus poses an unprecedented, major challenge to most economies and societies. Global development faces its biggest immediate danger since the 2008 financial crisis. The top priorities have been to contain the epidemic and protect people. Reinforcing health systems and supporting global medical research to ensure appropriate care is provided to everyone infected by the virus has driven the initial thrust of international co-operation in responding to the crisis; but development co-operation will also need to find fast and effective solutions to deal with the economic and social impact on developing countries – particularly on those groups of population farthest behind, who will be disproportionately affected by the crisis.
The pressure on development co-operation to show results in the response to the COVID-19 crisis will only increase with time, as donor governments face pressure in their fiscal space during the crisis aftermath. This note provides elements to reflect on how to manage for results throughout the crisis situation and in a changing environment. Scanning across short term, medium term and longer term perspectives, it presents how development co-operation providers are absorbing the COVID-19 shock and restoring their internal capacity, then suggests ways to ensure results in the next phase by adapting to the “new normal”, and finally explores opportunities for institutional learning and development.
Coping with the impacts of coronavirus (COVID-19): How development co-operation providers are absorbing the shock and restoring capacity
The transboundary nature of the COVID-19 pandemic, coupled with the lack of preparedness of countries’ health systems, and the already unprecedented social and economic costs of the crisis, calls for immediate action from the international community. Ministries and agencies in charge of development co-operation are being affected by this “wicked” three-pronged challenge: while normal organisational performance is severely affected, pressure mounts to provide a meaningful international response to the crisis in support of partner countries and keep afloat most ongoing development co-operation activities.
Many countries have taken measures to limit physical interaction and have imposed travel bans or forced quarantines. Eighty-five countries have implemented some form of national or regional lockdown. These policy changes to cross-border mobility create immediate and drastic disruptions to the normal functioning of results-based management (RBM) systems used in development co-operation, which are best visualised by the figure below (Figure 1).
Source: (OECD, 2020[1])
The vast majority of OECD Development Assistance Committee (DAC) donor countries have moved to operate in ways that limit physical interaction within their ministries, agencies and missions, as their governments introduce suppression measures to contain the pandemic. Most have introduced travel bans and switched to work from home and virtual interaction. This could potentially limit the overall organisational effectiveness of development organisations. It also creates new habits, including maximising the opportunities offered by digitalised systems supporting development work, amplifying the reach of online exchanges and events, and reducing the overall levels of CO2 emissions produced by the development co-operation sector.
In the short term, DAC members report to be at initial stages in the preparations for a development co-operation response to COVID-19 and its aftermath, and efforts are being made to ensure that it is results-oriented and able to be monitored. The emergency situation is exerting a “stress test” to existing results based management systems used to provide development co-operation effectively – in particular on the ability to adapt to the fast-evolving context, priorities and capacity shortages in partner countries.
The pandemic is exerting programming and operational pressures, some of which clash with internal policies and organisational constraints (Figure 2). To become more adaptable and agile organisations, capable of responding to uncertainty effectively, development co-operation ministries and agencies will have to reconcile and neutralise these tensions, for both the short and the medium term.
Source: Based on interviews with senior and technical officials in 20 ministries and development agencies, desk review, and data exchanges with 49 development organisations.
How long the extraordinary measures and exceptions described above will persist depends on the length and geographic prevalence of the COVID-19 pandemic. To avoid flying blind for too long and weakening the focus on development co-operation results, providers have to be prepared and adapt their results-based approaches to a prolonged “emergency” situation, while they learn and plan for its aftermath as it happens. The formal and actual demand for data and evidence-based decision making will only increase during this COVID-19 crisis.
Ensuring results in the next phase by adapting to the “new normal”
Addressing the COVID-19 crisis and its aftermath in developing countries will require a substantive transformation of the ways of working in development co-operation. While OECD countries are at the moment absorbing the impact of the crisis on their own organisations and introducing measures to fully recover their organisational effectiveness, for development co-operation managers, this is a time that also offers an unique opportunity to adapt internal processes and practices, deepen some ongoing reforms, realign priorities and incentives, and invest in improving capacities and systems, including in digitalisation. This is also a time to advance solutions for more meaningful and impact-oriented use of results information, which could garner the support of leadership in the short term and become institutionalised in the medium term.
(a) Rely on the DAC Guiding Principles for Managing for Sustainable Development Results to identify where institutional and organisational change is most needed
The DAC Guiding Principles for Managing for Sustainable Development Results1 can offer a valuable framework to explore the implications of the COVID-19 crisis for results-based management systems, and help DAC members self-assess how their various components could be better adapted for the future. This framework builds on a wealth of lessons from two decades of experience in results-based management.
Managers can use the principles to think strategically about organisational improvement during and after the crisis. To that end, Annex 1.A proposes a series of guiding questions to help DAC members observe what can be learned – with a results perspective – from current events as they unfold and affect their organisations and ways of working, as well as anticipate what may be needed for an effective development co-operation response to the crisis and its aftermath. In fact, as part of their response to the COVID-19 crisis, various members already report to be piloting new approaches to results inspired by the Guiding Principles.
(b) Rely on lessons learnt from past crises in designing the response to COVID-19
As DAC members complete the process of identifying or reallocating sources of funding to expand their programme towards COVID-19-related activities, many results and quality assurance units are being asked to provide support in the conceptualisation, design and monitoring of these emerging initiatives, within short timeframes and often in tension with normal internal approval processes and quality assurance procedures. A systematic look at lessons learnt from past crises can provide a solid foundation for well-designed development co-operation responses, as well as for alternative methods to monitoring, evaluation and learning during the crisis.2
(c) Rely on (or develop) internal strengths to allow for adaptive management and for better risk management
The OECD conducted a baseline survey on the status of 49-donor results-based management systems recently, just before the COVID-19 pandemic. The results suggest that a majority of development co-operation providers have good foundations to implement the necessary changes in due course – most donor governments and multilateral agencies meet some requirements for basic “adaptive” management in order to prepare and respond flexibly to the crisis, while others may want to use this situation to strengthen their capacity to adapt to fast-evolving development contexts.
The survey results illustrated in Figure 3 reinforce the point above in two ways:
Most providers routinely rely on quality context analysis in defining the design and results of new programmes and projects at country level. Three-quarters of 49 assessed providers consider that most of their results frameworks and approaches are well grounded in a sound understanding of local dynamics and needs. Most also report that field staff have sufficient flexibility to adapt approaches and processes to country circumstances. The design of COVID-19 responses should build on these two strengths by actively seeking inputs from country teams and field staff, and by providing them with continued autonomy so they can tailor programme and project designs to each country context (instead of applying template approaches fully designed at headquarters).
Source: “Managing for Sustainable Development Results: Baseline status” survey. Sample: 49 ministries and development agencies. (OECD, 2020[1])
A majority of providers have adopted project management practices that identify, recognise, monitor and react to risks and changing conditions. While not always done systematically, most providers have developed “good enough” risk management practices. Most also empower field staff to adjust the path of implementation of programmes and projects in light of changing conditions. These qualities should continue to be strengthened during the response to COVID-19 and its aftermath, which is plagued with uncertainties and unplanned risks. In light of this, management practices should be iterative, with frequent check-ins, and quality assurance mechanisms should place greater emphasis on encouraging adaptive designs for the implementation phase.
Some members, like Sweden, also report that a more strategic use of results frameworks, coupled with greater reliance on locally defined results and theories of change, is allowing for flexibility during this emergency period. A critical factor that enables such an approach is a history of trust-building with decentralised staff and implementing partners, which allows for increased autonomy to adapt the development interventions while relaxing other accountability-driven oversight measures.
A one-time opportunity for institutional learning and development
The current situation presents the most significant organisational challenge for development co-operation providers in decades – but it also offers a one-time opportunity for institutional learning and development. If seized, this could be an opportunity to strengthen the adaptability muscle of our results-based management systems, to learn to overcome limitations in traditional data collection methods, and to ensure that decisions are guided by evidence in order to maximise results and impact.
To that end, we would like to reflect on three policy-oriented recommendations:
First, the Sustainable Development Goals (SDGs) and their monitoring framework may enable providers to rely more on systems thinking as they approach this crisis from a results angle. The SDG framework offers a way to visualise the interconnectedness of the COVID-19 effects with other economic, social and environmental dimensions (avoiding sector silo responses) so as to help partner countries prepare holistic responses to the triple health/economic/social crisis unleashed by the coronavirus (Box 1).
As individual bilateral responses would not be sufficient to provide the scale of support that developing countries will be needing in the coming months, the SDG framework can serve as a shared set of harmonised results indicators that multiple providers use in working together in the response. Gathering “real time” results data for evidence-based policy making will be more crucial than ever in the coming months, and with limited national capacities, joined up monitoring approaches – involving bilateral and multilateral providers – may be the only cost-effective, realistic way to go. A harmonised approach such as the one offered by the SDG framework may help aligning all efforts in many countries.
As a communicable disease, COVID-19 is captured under the SDG 3.3 target. Providers may decide to step up support to the health sector in developing countries. Or they may opt to focus on the negative socio-economic impacts on the crisis. In all these cases, they may want to consider holistic results approaches to their development programme that explicitly recognise other interconnected factors – like those driving or being driven by the health crisis as well as countries’ responses to suppress its spread and mitigate its impact.
Source: (International Council for Science, 2017[2])
Where relevant and feasible, the use of SDG targets and indicators to inform these policy and programming responses to the crisis – in consultation with each partner country – can help harmonise measurement approaches to results, pool resources and innovative ideas to generate the necessary results data, and help in the overall division of labour and co-ordination amongst donors.
Source: Authors’ own elaboration based on (International Council for Science, 2017[2]). Please consult the reference for a detailed description of interactions between SDG 3.3 on communicable diseases and other targets and indicators.
Second, innovations in monitoring and evaluation [remote] methods can strengthen the positioning of these functions within DAC development ministries and agencies and lead to better use of results data for decision making and learning. Recent reflections from that community (see Box 2) anticipate a hold in their usual business which will only be overcome by innovation in remote methods. This will be followed by a period in which project designs may be less evaluable, monitoring data less abundant or of lower quality, and the learning function of evaluation will gain prominence in relation to oversight functions. In response to this, the DAC Network on Development Evaluation (EvalNet) is convening a critical mass to conduct a major, joined-up, real-time evaluation that can provide insights for decision making in a timely fashion.
The OECD/DAC Network on Development Evaluation convened 220 participants, largely from bilateral and multilateral evaluation offices, to exchange (remotely) on the impact of the COVID-19 crisis on their evaluation work.
As part of the overall results based management system, evaluation functions are affected by severe disruptions in their implementation capacity. In the context of lockdowns and travel bans, regular evaluation field work and commonly used methods (e.g. key informant interviewing, sampling, focus groups) are not possible for the foreseeable future, and remote substitutes may not be perceived as effective without the proper tools and training – prompting an intense mapping of alternative tools to complete ongoing evaluations or redesign future ones. A key concern is that most remote methods are less suited to reach those furthest behind, which will be particularly affected by the COVID-19 crisis.
At headquarters level, some countries report that evaluation functions have had to reallocate resources to advise domestic partners (as the emergency is also impacting domestically). While most are able to work remotely, for some it required front loading investments into IT systems or platforms.
Evaluation offices worry that negative impacts on monitoring capacity, data availability and project design –while expected– will affect their capacity to perform evaluations with the usual standards (the new DAC evaluation criteria may help). Reprioritising evaluation questions was seen as an option. They also anticipated a shift in demand from policy makers, less for evaluation and more for advice.
Building on the experience of past joint evaluations in crisis situations or for the Paris Declaration on Aid Effectiveness, most evaluation offices expressed an interest in joining a large, multi-donor evaluation of provider responses to COVID-19, with emphasis on quick learning and decision-making.
Source: Based on exchanges in a DAC Evalnet webinar with the participation of 280 evaluation office directors and staff (April 3, 2020).
Finally, the need for providers to report on their contributions to development and the SDGs calls for scaling up support for alternative data gathering methods and strengthening national monitoring and statistical capacities. Initial reflections from the community of national statistical offices and the donors that support them seem to indicate that lack of statistics and government data will be a generalised issue across the world – even if upper middle‑income countries and high‑income countries are already exploring alternative (non-physical) methods to conduct household surveys, censuses and other forms of data gathering (Box 3).
PARIS21, an initiative to strengthen national statistical offices (NSOs) in developing countries, convened 330 participants from NSOs and donors in early April, to discuss how to cope with the impact of COVID-19. While Colombia, the United Kingdom, Lesotho and Maldives presented different stories in terms of capacity, the bottom line was that all NSOs are on complete hold, while demand for disaggregated data and analysis for policy making has soared in parallel.
Responses to COVID-19 impacts range from (less effective) phone-based surveying for those with greater installed capacity, such as Colombia and the United Kingdom, to a temporary hold of all statistical operations in Maldives or Lesotho due to inability to telework or to roll out remote data gathering methods. All four cases will be affected by major data gaps and delays in publication of statistics.
Only innovations in data collection methods and selected donor support will allow for a much delayed publication of a subset of national statistics and development data during the next 12-18 months.
Source: Based on presentations and peer exchanges in a PARIS21 webinar with the participation of 335 staff from NSOs and development partners working on development data (8 April 2020).
At headquarters level, results-based managers may need to anticipate a significant data gap on developing country statistics in their reporting plans –at least while the pandemic remains active across the developing world– while officials in charge of country dialogue and programming may need to consider how to scale up support for alternative data gathering methods for national statistics and data – particularly in least developed countries which run the risk of flying blind in terms of household surveys, health or unemployment statistics, gross domestic product and inflation estimates, or agricultural sector data. The schedule for all those basic, regular components of national statistical systems is severely disrupted, despite their importance in responding to the health/socio-economic crisis ahead.
The COVID-19 pandemic has enormous ramifications for strategic planning, results-based management, and particularly for those in charge of programming, implementation, as well as monitoring and evaluation. To address a crisis in which policy making and action needs to be especially guided by real-time evidence, development co-operation providers are going to have to adapt their results-based approaches at fast speed to provide timely and adequate information for decision making and learning – albeit with significant trial and error in a process that is iterative by nature. Space for innovation and responsible risk taking should be encouraged.
The short-term impacts of the crisis on members’ ability to keep their results-based management systems up and running will continue to be daunting for the foreseeable future. Yet, we also need to prepare for the aftermath of the pandemic. The usual business of the results agenda will continue as a pending task – but this crisis also offers an opportunity to unlock some institutional bottlenecks, adapt norms and processes, and adopt bolder approaches to the scale of the current challenge.
This crisis also creates momentum to foster the peace-humanitarian-development nexus, by learning and exchanging with the humanitarian sector on their processes and tools, and how a combined approach could foster the collective response to current and future challenges.
In that context, real-time peer learning becomes a crucial tool for contending with this crisis. The ongoing collaborative work of the OECD/DAC Results Community – through its temporary working groups associated with the Guiding Principles for Managing for Sustainable Development Results – is initially focusing in accompanying members as they cope with the impact of the pandemic while preparing a results-oriented response for the recovery phase.
Development co-operation providers are at a critical juncture. The response to both the COVID-19 crisis and its aftermath will require enhanced capacity to generate useful, timely results information that can guide decision making, can help in internal and cross-country learning, and can be used for accountability and communication purposes, both domestically and with other international partners. Getting there in this difficult context will require innovation and greater adaptability.
References
[3] International Council for Science (2017), A Guide to SDG Interactions: From Science to Implementation, International Council for Science, https://council.science/wp-content/uploads/2017/05/SDGs-Guide-to-Interactions.pdf.
[1] OECD (2020), “A systemic resilience approach to dealing with COVID-19 and future shocks”, OECD Policy Responses to Coronavirus (COVID-19), Vol. I/1, http://www.oecd.org/coronavirus/policy-responses/a-systemic-resilience-approach-to-dealing-with-covid-19-and-future-shocks-36a5bdfb/.
[2] OECD (2020), Managing for Sustainable Development Results: Baseline Status Survey.
[5] OECD (2018), Good Jobs for All in a Changing World of Work: The OECD Jobs Strategy, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264308817-en.
[6] OECD (2014), “The crisis and its aftermath: A stress test for societies and for social policies”, in Society at a Glance 2014: OECD Social Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/soc_glance-2014-5-en.
[4] OECD (2010), OECD Employment Outlook 2010: Moving beyond the Jobs Crisis, OECD Publishing, Paris, https://dx.doi.org/10.1787/empl_outlook-2010-en.
Annex 1.A. How applying the Guiding Principles on Managing for Sustainable Development Results could help prevent, respond to, and recover from a systemic crisis?
Indicative guiding questions to strengthen a results-oriented response to the COVID-19 crisis
Guiding principle 1: Support sustainable development goals and desired change
In the short term:
How to convey the importance of development co-operation in times of crises to help achieve social, economic and environmentally sustainable development that leaves no one behind?
What is the scope for adapting the country portfolio (e.g. re-orienting or starting new programmes), delivery channels (bilateral/multilateral) and modalities?
What type of guidance should be provided to country offices regarding the scope for amending theories of change and results frameworks?
How to articulate the nexus between the humanitarian response and the development co-operation response?
How to think beyond specific interventions towards the whole system in order to understand how best to calibrate interventions?
In the longer term:
What is needed for results frameworks and indicators to help prevent, address and recover from crises effectively; e.g. being less burdensome, better aligned, and covering sectors that may have been missed (including prevention and preparedness)?
Guiding Principle 2: Adapt to context
How can managing for sustainable development results support the need for an adaptive approach to planning and delivery in a rapidly evolving situation?
How to define a set of key measures/metrics that could be used to identify triggers of changes to interventions?
What is the scope for delivery channels, modalities and procedures (e.g. procurement) to be adapted?
What is the scope for amending theories of change and results frameworks?
How to set clear processes for collecting, interpreting and acting on evidence and document these changes?
How to balance increased flexibility (e.g. deadlines, procurement procedures) and risk management?
Guiding Principle 3: Enhance country ownership, mutual accountability and transparency
How to continue promoting ownership in a crisis?
How to strengthen partner country's statistical and data collection systems to be fit for sustainable development planning and monitoring under crisis circumstances?
How to provide appropriate institutional capacity building to help partners address crises?
How to encourage and engage in locally led innovation and problem solving?
Guiding Principle 4: Maximise the use of results information for learning and decision making
Are there decisions that must be made now, which would have benefitted from greater data and evidence?
How to identify the information of most value to decision makers and other stakeholders across a range of interventions? Then using this to set out an agreed approach to how learning will inform different kinds of decision making.
How to be accountable and demonstrate the impact of the work done by development co-operation providers to address the crisis?
How to replicate and adapt examples of new/innovative decision making, information-sharing processes that have been helpful in the domestic response to the crisis?
Guiding Principle 5: Foster a culture of results and learning
How to build leadership vision and a supportive management culture across the different teams and units co-ordinating the response to the crisis? How should leadership communicate the argument for changes to interventions including on: Why we need to adapt? What we need to adapt? How we should adapt?
How to set up an objective and transparent system of governance that reviews evidence, steers and communicates the response?
How to support or prepare for real-time monitoring, evaluation and learning of the effects of the crisis on sustainable development in partner countries?
How to further the collective knowledge and ability (a) to learn about the delivery of interventions, (b) to identify whether different (or different combinations of) interventions might work better, as well as, (c) to understand the impacts of measures?
How can the evidence base be used to develop a strategic and operational research agenda that prioritises and accelerates collecting and using the most useful information?
Guiding Principle 6: Develop a results system that is manageable and reliable
How is the crisis affecting the capacity to monitor, evaluate and learn from on-going programmes? How to maintain quality monitoring and the quality standards of evaluation?
How to balance the need to collect a range of data and evidence and the need for “quick enough” and “good enough” measures, given fast changing developments and trends?
With traditional ways of data collection affected (more desk reviews and remote methods; less monitoring on the ground and less face to-face census and evaluation interviews), how to ensure accurate and comprehensive data is collected? In particular: How to ensure inclusivity/reaching out to those left behind? What is the potential for using new IT systems and sources of data?
How to re-prioritise evaluations taking a cost-benefit approach?
How to pass the message that monitoring and evaluation are all the more important in terms of crisis and need to be kept a priority?
Contacts
Alejandro GUERRERO-RUIZ - (✉ alejandro.guerrero-ruiz@oecd.org)
Chantal VERGER - (✉ chantal.verger@oecd.org)
Rahul MALHOTRA (✉ rahul.malhotra@oecd.org)
Notes
Among other recent recompilations of lessons, consult World Bank (2020). Evaluative resource and evidence to inform the COVID-19 response; OECD/UNDP (2020). Good practices during COVID-19; or USAID (2020). Monitoring, Evaluation and Learning during the COVID-19 Pandemic.
