Monitoring and Evaluation of Child and Youth Policies and Outcomes in Ireland

7. Conclusions

Comprehensive results-based M&E systems are a key public management tool to help policy makers track progress and demonstrate the impact generated by a particular policy or intervention. M&E systems can assist in formulating policy goals, identifying promising practices, detecting programme weaknesses and potential actions to correct those. They are also useful to promote transparency and accountability of policy making.

Monitoring and evaluation are distinct but complementary practices, and a rigorous M&E system should be built considering the relevance and complementary of both functions. Although there is no unique way to design and implement a results-based M&E system, there are some basic and necessary steps to be considered when building such a system, including the development of a results framework, monitoring for results, planning for evaluation, and reporting and dissemination of findings.

This report provides guidance for the development of an M&E system for Ireland’s new policy framework for children and young people. The report details the sequential steps that should be considered when developing a robust results-based M&E system, building on international good practice examples and the BOBF indicator set.

While BOBF did not plan for the design and implementation of a comprehensive M&E framework, certain elements of its indicator set could be useful cornerstones for the design of a results-based M&E system. In particular, BOBF’s indicator set has a wide range of indicators at the outcome level that could be integrated as part of the results framework of the M&E system. The extensive consultative approach through which BOBF was developed is another asset and could help to build a broad consensus. Dedicated human and financial resources were put in place for BOBF, and progress has been made in capacity building for M&E since the adoption of BOBF. Finally, Ireland has a legal framework that encourages data openness and transparency in the public sector, which resulted in regular reports on lessons learnt in the monitoring of BOBF and, in more recent years, the development of additional dissemination strategies.

The following box provides concrete recommendations for the elaboration of a results-based system to monitor and evaluate progress and performance of Ireland’s new policy framework for children and young people.

Key recommendations

• A formal results-based M&E system for the new policy framework on children and young people would allow for the systematic monitoring and tracking of progress and performance of the new policy framework.

The design and implementation of a comprehensive results-based M&E system would help policy makers to consistently track progress and demonstrate the impact generated by the government actions under the new policy framework. Furthermore, the system would allow to rigorously identify promising interventions, as well as identify any challenges or weaknesses that may arise throughout the implementation of those interventions.

• The design of the M&E system should build on an explicit and comprehensive formulation of the logic of change underpinning the policy framework.

A Theory of Change (ToC) approach could be followed in order to clearly document what the policy framework is expected to do (inputs, activities and outputs) and what it aims to achieve (outcomes and impacts). The ToC should also include a description of assumptions and risks that could have an effect in the policy’s results chain.

When articulating the policy framework’s ToC, special attention should be paid to the terminology utilised. Terms such as goals, aims, and outcomes are often used to refer to different concepts in the results chain, and should be used carefully.

• BOBF’s indicator set should be simplified and complemented with indicators for each level of the results chain.

The number of indicators at the outcome level (over 100 indicators across 70 indicator areas) should be reconsidered. Furthermore, SMART indicators should be identified and defined for the other levels of the results chain, that is, inputs, activities, and outputs.

• Sufficient and appropriate human and financial resources should be allocated for the M&E of the policy framework.

A combination of internal and external resources could be put in place for this purpose. Ideally, ownership for leading the overall development and implementation of the M&E framework should be assigned internally to an appropriately qualified person. Furthermore, if necessary, a business case could be developed for funding M&E activities that include both building inhouse capability in the reporting agencies and accessing external expertise.

• A comprehensive communication and dissemination strategy for M&E findings should be developed.

The various audiences who could be interested in M&E findings should be clearly identified to ensure that the communication strategy reaches the right people in the right way. Besides the traditional regular frequency written reports, alternative dissemination formats should be considered, such as real-time “dashboards”, or repository websites, for example.