This chapter examines approaches to enhance the sustainability of the current justice reform agenda in Portugal. It puts a particular emphasis on evaluation and monitoring, including measurement of the justice sector performance, indicators, cost-benefit analysis and stakeholder assessments. It aims to provide for a robust analytical framework based on a detailed theory of change and enhanced data collection processes.
Justice Transformation in Portugal
Building on Successes and Challenges
4. Sustaining a transformed people-centred justice sector
Abstract
Evaluating and monitoring progress
Measuring the performance of the justice sector
In consultation with the High Council of the Judiciary and the office of the General Prosecutor, the Ministry of Justice has developed a set of strategic performance goals and key performance indicators. The performance assessment framework applies to all courts of first instance and public prosecution offices at the court, process and individual judge/prosecutor levels. The indicators, which are intended to measure the efficiency, effectiveness and quality of the courts, consist of:
The clearance rate.1
The disposition time.2
Pending cases awaiting final decision.
Pending cases after final decision.
Percentage of pending cases with duration above the reference value.
Average duration of pending cases.
Average duration of completed cases.
Court performance is monitored on this basis at the local level and nationally, through quarterly meetings between representatives of the High Council of the Judiciary, the High Council of the Public Prosecution and the Ministry of Justice. This allows both the management of district clusters and the High Council to monitor performances and workloads and allocate resources accordingly.
Successive judicial reforms have increased the reliability and granularity of court-level data in Portugal but they have also introduced some discontinuities in the data that make the analysis of recent trends more challenging.3 Further changes are currently being introduced.4 Continuing efforts to strengthen the reliability of the data and to better exploit disaggregated data (based, for example, on the quality and complexity of cases and the cost of litigation) would support a more in-depth assessment of court performance (OECD, 2019).
More fundamentally, the international practice points towards the importance of extending the analysis to other aspects of court performance and other elements of the justice sector (Box 4.1). Timeliness, as measured by clearance rates and disposition times, is only one element of judicial quality, alongside others such as independence, accountability, competency, integrity, transparency and public trust. In addition, adopting a broader perspective on the performance of the justice sector – such as the use and user-friendliness of alternative dispute resolution (ADRs) mechanisms or the quality and accessibility of the legal aid system – is necessary in order to measure how well the justice needs of society are addressed.
Portugal is already developing some of the elements of a more comprehensive measurement framework. Along with Estonia, Italy and Slovenia, it actively contributed to the project “Court Quality Framework Design”, led by France and supported by the European Union, awarded in September 2016 for one year. The project focused on identifying indicators for court performance in three main phases:
1. Prior to submission of the claim (level of accessibility of judicial information, information on the proceedings, personalised legal consultations, etc.).
2. During the judicial proceedings (simplicity and reliability of information, communication between the courts and the parties, quality management tools).
3. After the court decision is issued (assistance to the parties, attention to the readability of court decisions, etc.).
Box 4.1. Examples of International assessment frameworks for judicial performance
Attempts at developing a detailed performance standard for justice institutions started in earnest in the 1980s. One of the most influential early reflections came from the Trial Court Performance Standards (TCPS) Project, launched in 1987 in the United States by the National Center for State Courts (NCSC) and the Bureau of Justice Assistance of the Department of Justice. The aim of the project was to elaborate “a common language to facilitate the description, classification and communication of court activities; a conceptual framework for understanding the work of courts; and, most important, a means for self-assessment, self-improvement and improved accountability”. In 1990, the commission in charge of elaborating the TCPS published a set of standards, which were tested by a number of courts across several states to gauge performance and updated in 1997 to account for this experience.
There are 22 TCPS under 5 performance areas: i) access to justice; ii) expedition and timeliness; iii) equality, fairness, and integrity; iv) independence and accountability; and v) public trust and confidence. They cover aspects such as the public nature of proceedings, the convenience of use and cost of access to the court, the management of caseloads and schedules, and the representativeness of juries.
Other countries also developed frameworks for assessing court performance. In the Netherlands, for instance, the Council for the Judiciary undertook in 2002 the creation of an overarching quality system for all courts known as “RechtspraaQ”, which comprises a performance measurement system applicable at the court and sectoral levels. Five areas of judicial performance are considered, each grouping several indicators: impartiality and integrity; expertise; treatment of litigants and defendants; legal unity; speed and promptness.
Initiatives were then taken to compare judicial performance across jurisdictions and to share lessons between countries. For instance, the Committee of Ministers of the Council of Europe (CoE) established the European Commission for the Efficiency of Justice (CEPEJ) in 2002 to promote the rule of law and fundamental rights in Europe, and decided to strengthen its evaluation and assistance functions in 2005 “in order to help member states to deliver justice fairly and rapidly”. The toolbox that the CEPEJ has developed to this effect includes a database of judicial statistics enabling the comparison of judicial systems on efficiency, quality and effectiveness grounds, as well as a checklist for promoting the quality of justice and the courts. The CEPEJ rests its work on seven “pillars of quality” of judicial processes and decisions derived from the European Convention on Human Rights (article 6): fairness of the proceedings; reasonable duration of the proceedings; publicity of the decision and transparency of the process; protection of minors (and other subjects for whom it is appropriate to provide a form of assistance); comprehensibility of the prosecution, the course of the procedure, and decisions; right to legal assistance and access to justice in general; legal aid (when all conditions are met).
At the international level, the International Consortium for Court Excellence, which brings together bodies from various (mainly common-law) countries as well as the World Bank and the CEPEJ, proposed a Framework for Court Excellence in 2008 for application to all courts and revised and simplified it in 2013. As a normative basis, the framework lists ten values that are deemed to “guarantee due process and equal protection of the law to all those who have business before the courts, set the court culture and provide direction for all judges and staff for a proper functioning court”: equality before the law, fairness, impartiality, independence of decision-making, competency, integrity, transparency, accessibility, timeliness and certainty. The framework proposes a questionnaire for courts to self-assess the extent to which these values are guiding their daily operations and a set of 11 Global Measures of Court Performance with the aim to “establish international standards and common definitions of court performance measurement that would, first, provide (…) good practices for successful performance measurement and performance management and, second, encourage comparative analysis and benchmarking across different jurisdictions”.
Other approaches have considered a broader set of justice institutions than simply courts. For instance, the work of the World Bank on justice sector reforms in the 1990s addressed, in addition to the judiciary, elements such as the procedural codes, alternative dispute resolution (ADR) mechanisms, legal aid, bar associations or legal awareness education for the public. In 2012, in the context of its New Directions in Justice Reform, the World Bank proposed an even broader definition of “justice systems” encompassing “the formal and informal institutions that address breaches of law and facilitate peaceful contests over rights and obligations”, including prosecutors’ offices, the police, administrative enforcement mechanisms, etc.
The OECD’s work on access to justice also pinpoints the variety of paths to justice, noting that “legal needs may be addressed through a range of legal and justice services (court and non-court-based), which could be arrayed along a service continuum from public legal education to early resolution services to full representation and resolution”. Focusing on less than the entire justice “service continuum”, by contrast, provides a partial – and possibly biased – picture of the extent to which citizens’ and businesses’ legal needs are satisfied.
Sources: Trial Court Performance Standards, www.ncsc.org; “RechtspraaQ”, Council for the Judiciary, www.rechtspraak.nl/English/The-Council-for-the-Judiciary; European Commission for the Efficiency of Justice (CEPEJ), www.coe.int/en/web/cepej/cepej-work; International Consortium for Court Excellence, “Framework for Court Excellence” www.courtexcellence.com/; “New Directions in Justice Reform”, World Bank, http://documents.worldbank.org/curated/en/928641468338516754/The-World-Bank-new-directions-in-justice-reform-a-companion-piece-to-the-updated-strategy-and-implementation-plan-on-strengthening-governance-tackling-corruption; www.oecd.org/fr/gov/access-to-justice.htm.
The Ministry of Justice is also in the process of developing a new system of indicators which would also include:
Human, technical and budgetary resources (average cost per case, availability of the system, hours of training/per person, absenteeism rate, ration of new, completed and pending cases per person).
Operational efficiency (productivity per magistrate, productivity per organic unit, productivity per official).
Simplification of procedures (percentage of electronic files, automation of acts, acts per case, average number of hearings per case).
Speed and quality (percentage of cases with more than a certain number of days, clearance rate, deviation from the statutory period).
Court management and organisation.
Satisfaction with services (percentage of postponement of trial hearings, satisfaction surveys, etc.).
Recent evolutions in court efficiency indicators
In terms of overall judicial performance, the data suggest that the Portuguese courts have made substantial efficiency gains, as recognised by the International Monetary Fund (IMF) in its 2018 assessment. In terms of specific results, the particular analysis of procedural times is one of the main indicators for individuals and enterprises in terms of judicial trust and access to justice. Portugal has considerably improved its court time proceedings in recent years, passing from 417 days in 2010 to 289 days in 2016 on average, according to CEPEJ data (Figure 4.1). Indeed, while Portugal compared unfavourably to other European countries until 2012, by 2016, the pendency rates and disposition times of commercial and civil courts had converged towards the averages of European countries. This means that courts have improved their efficiency by making procedural times shorter and justice prompter (e.g. the proportion of disposed case that take over five years has decreased).
For administrative and fiscal cases, Portugal’s courts remain within the lower range of European countries alongside Cyprus,5 Greece, Italy and Malta.
Figure 4.1. Disposition times in courts of first instance in 2010 and 2016
Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”.
Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus.
Note: Ratio between the number of pending cases and the average number of resolved cases per day during the year, i.e. the number of days needed for the court to resolve all pending cases at the current pace.
Source: CEPEJ (2018), “European judicial systems – Efficiency and quality of justice”, CEPEJ Studies, No. 26. European Commission for the Efficiency of Justice, Council of Europe.
The improvement in measured performance has been particularly marked in the area of enforcement, where clearance rates have consistently exceeded 100% since 2013, leading to a dramatic reduction in the number of pending cases (Figure 4.2).
It appears that the increase in the clearance rate has been largely caused by a fall in the number of new cases filed, which has been below 150 000 cases on average per year since 2016, against close to 350 000 in 2012. The number of disposed cases has also decreased during the period, from close to 330 000 in 2012 to slightly more than 250 000 on annual average in 2016-18.
Furthermore, it should be noted that a substantial part of the improvement observed since 2013 may stem from several changes introduced by the new Code of Civil Procedure (see Chapter 2, Box 2.1 on the Civil Procedure reform). In particular, the dispositions regarding the termination of proceedings and the closing of certain enforcement cases have likely resulted in a reduction of several hundreds of thousands in the number of pending cases.
These changes also seem to have contributed to the sudden hike in the average duration of enforcement cases and in the percentage of cases with a duration of more than five years in 2013, followed by a decline in 2014 (Figure 4.3). The steady increase in both indicators since the end of 2014 is, however, a matter of concern as it might indicate a “hard core” of cases for which efficiency gains have not taken place.
Figure 4.2. Clearance rates and pendency for enforcement cases
Source: Data provided by the Ministry of Justice, DGPJ statistics, https://estatisticas.justica.gov.pt.
Figure 4.3. The duration of enforcement cases
Source: Data provided by the Ministry of Justice, DGPJ statistics, https://estatisticas.justica.gov.pt.
Figure 4.4 provides an indication of the evolution of efficiency in other courts, as measured by the clearance rate. It appears that other courts – which were not directly impacted by the new Code of Civil Procedure dispositions on pendency – have not experienced an increase in clearance rates similar to that of courts of first instance.
Figure 4.4. Clearance rates in Portugal’s courts
Note: Ratio between the number of disposed cases and the number of new cases during the year.
Source: Data provided by the Ministry of Justice, DGPJ statistics.
In summary, while the dramatic rise in the clearance rate and drop in pendency of enforcement cases in civil and commercial courts of first instance is per se a positive development, these results would need to be interpreted with caution. First, they stem from a reduction in the filing of new cases, not an increase in the number of disposed cases. Second, they are partly related to a change in the method of accounting of pendency. Third, average durations of cases have increased in recent years. Finally, other courts (administrative, appellate, Supreme Courts, State Auditor’s office) have not experienced similar improvements.
More granular longitudinal data (e.g. broken down according to the complexity of cases) would be needed in order to fully assess the impacts of different changes introduced as part of the recent reforms and have a better understanding of the efficiency of Portugal’s courts.
Portugal has also seen an evolution of trust in justice systems over the past several years (Figure 4.5). Yet, trial length in Portugal is still seen as long when compared to other OECD countries whose average number is 221.36. Over 35% of cases still take more than 1 year to resolve (OECD, 2019). Recently, in the area of enforcement, the number of completed cases has consistently exceeded cases entering the system (whose number has been declining since the crisis years) along with the processing times. Yet, it would be important for Portugal to continue its efforts to improve the resolution of enforcement cases, including contract enforcement and insolvency proceedings, as they are essential for businesses.
Importantly, care is needed in attributing specific improvements to individual initiatives. Moreover, indicators such as trial length may also be affected by behavioural conditions and incentives of different actors, including culture, socioeconomic characteristics, business fluctuations, substantive law, as well as costs and rules, lawyer’s incentives, ADR and legal certainty, which calls for a holistic approach to justice reforms.
Figure 4.5. Evolution of trust in EU Justice system, 2014 - 2018
Sources: European Commission (2018), “Public opinion in the European Union”, Standard Eurobarometer 90, Vol. A: Countries, http://data.europa.eu/euodp/en/data/dataset/S2215_90_3_STD90_ENG; European Commission (2018), “Public opinion in the European Union”, Standard Eurobarometer 82, Vol. A: Countries, http://data.europa.eu/euodp/en/data/dataset/S2215_90_3_STD90_ENG.
Evaluation of the costs and benefits of Tribunal +
The Ministry of Justice has sought to assess the effect of the Justiça + Próxima plan, with particular emphasis on the total costs and benefits of Tribunal +. An ex ante evaluation based on the results of the Sintra pilot project was conducted at the end of 2017 (Ministry of Justice of Portugal, 2018).6 An interim and a final evaluation of Tribunal + are also planned in the course of 2019.
The ex ante study concludes that the estimated costs of Tribunal + would be close to EUR 14 million of start-up investment costs and EUR 2.4 million of operational costs per year, while its expected benefits would reach EUR 69.6 million per year. The bulk of these benefits stems from an estimated gain of 3 300 clerical positions (in full-time equivalents) in both back-office and front-office functions. Additional gains are related to the costs saved by users in their interactions with the courts, in terms of both direct costs (reduced travel) and opportunity costs (time saved). Non-monetary benefits include an estimated reduction of 320 000 in the number of pending cases at the national level.
As usual for this type of studies, the cost-benefit analysis of Tribunal + is based on data provided by the implementing entities (the Ministry of Justice for the cost figures, the consultancy in charge of the re‑engineering of back-office workflows in Sintra for data on efficiency gains in carrying out individual tasks). The study then extrapolates from this data under various modelling assumptions. For instance, productivity gains in the performance of individual clerical tasks are simply added and assumed to generate proportional cost savings and backlog reductions. Yet such an assumption may overlook the possibility of efficiency losses and bottlenecks at other points in the judicial process.
An example of possible bottlenecks could be enforcement action falling under the responsibility of enforcement agents outside the court. Indeed, preliminary estimates by the Ministry of Justice indicate that less than 15% of enforcement cases are pending on a decision or an action by judges and clerks;7 the bulk of pending case are awaiting action by the parties, third parties or an external enforcement agent, or the expiry of a period.8
More generally, when updating the evaluation of Tribunal +, it would be important to take proper account of the dampening effects that could be caused by: i) losses of efficiency in the implementation of new work processes at the court level; and ii) other capacity constraints and bottlenecks in the management of cases.
In terms of the evaluation process, potential improvement areas include submitting ex ante evaluations to an independent review and a formal ex post assessment based on monitoring data.
The ex post evaluation of Tribunal +, based on observed outcomes at the court level, will help to establish a clear picture of the improvements introduced by the project and the obstacles that it has faced.
A common method for analysing the impact of a project such as Tribunal + on court efficiency is to compare the evolution of efficiency indicators in courts that have participated in the project to that of courts that have not.9 The fact that Tribunal + was carried out as a pilot project at different times in different courts offers an opportunity for such a comparative analysis.
In application of this method, Figures 4.6 and 4.7 below compare the evolution of clearance rates and numbers of disposed cases10 in the court of Sintra, where the original pilot project was implemented from the fall of 2016 onwards, in courts of the West Lisbon Comarca, where the pilot project was launched in the fall of 2017, and in other courts in the country where the project was not implemented during the period under review (or not advanced to the point of influencing the indicators). To facilitate the comparison of evolutions since the start of the project, all values are expressed as indexes with a base value of 100 in the fourth quarter of 2016.
Figure 4.6. Clearance rates in courts implementing Tribunal +
Note: Ratio between the number of disposed cases and the number of new cases during each quarter.
Source: Data provided by the Ministry of Justice, DGPJ statistics.
Figure 4.7. Number of disposed cases in courts implementing Tribunal +
Source: Data provided by the Ministry of Justice, DGPJ statistics.
Similarly, Figure 4.8 shows the evolution of disposition times in the three groups of courts.
Figure 4.8. Disposition times in courts implementing Tribunal +
Note: Ratio between the number of pending cases and the average number of resolved cases per day during the quarter, i.e. the number of days needed for the court to resolve all pending cases at the current pace.
Source: Data provided by the Ministry of Justice, DGPJ statistics.
Several observations can be made when comparing the results of the three groups of courts:
1. Clearance rates were following an upward trend in all the reviewed courts until the end of 2016. They deteriorated during the first half of 2017 – a slump might have to do with a gradual fading of the effects of the 2013-14 reforms but have somewhat improved since. The increase since the second quarter of 2017 has been slightly stronger in Sintra than in Portuguese courts outside of the Western Lisbon Comarca but lower than in other courts of the Comarca.
2. The improvements in clearance rates between the second quarter of 2017 and the third quarter of 2018 appear to be largely due to a fall in the number of new cases. The number of disposed cases, which was following a positive trend until the end of 2016, is since on a downward path. The evolution in the number of disposed cases has been identical in the court of Sintra and in Portuguese courts outside of the Western Lisbon Comarca, which did not implement Tribunal + during the period.
3. Disposition times were on a downward trend until the end of 2016. They have since stabilised in the courts of the Western Lisbon Comarca other than Sintra, while they increased by more than 40% in the court of Sintra and in other courts of the country.
The analysis shows that at this early stage it could be difficult to fully identify a full range of effects of Tribunal + on the courts in which it has been implemented as a pilot project due to a number of factors.
First, micro-level efficiency gains might take more time to translate into an improvement in court-level indicators. More detailed indicators might help to monitor such delays in the diffusion of efficiency gains.
In terms of methodology, this points to the relevance of a full representation of case management processes – including the work of judges and prosecutors alongside that of clerks – as a basis for the evaluation and monitoring of effects. Such a representation would facilitate the identification of potential bottlenecks and efficiency losses. In terms of the evaluation process, potential improvement areas include submitting ex ante evaluations to an independent review and a formal ex post assessment based on monitoring data.
Second, the measures reviewed in the previous sections, in particular the adoption of a new Code of Civil Procedure, the creation of the district clusters and the changes it has induced in terms of court management and specialisation, can be expected to have far-reaching effects on the organisation of work, labour productivity and efficiency in handling cases. The direct impact of the new code on court pendency rates has already been mentioned. The monitoring framework for Tribunal + could seek to assess and isolate the influence of external factors on court performance.
Third, it should be emphasised that an important part of the benefits of Tribunal + resides in an improvement of outcomes other than macro-level efficiency, in particular increases in user satisfaction and in transparency. The monitoring framework for Tribunal + should also seek to identify and measure such outcomes.
These observations point to the necessity of elaborating a general analytical framework for ensuring that the evaluation and monitoring of Tribunal + are both consistent and comprehensive. The management of the project would benefit from the elaboration of a detailed logical framework (or theory of change) that would spell out the causal linkages between its key measures and their direct and indirect outcomes and ultimate impact. Figure 4.9 presents the sketch of a theory of change for the Tribunal + measures. A (more) detailed logical framework could be used to identify and assess external factors that could potentially influence outcomes, locate possible bottlenecks for efficiency gains, and orient methodological and data collection choices in future monitoring and evaluation activities.
Such a framework would help to further and complement the analysis presented above by more detailed investigations and further statistical observations.
Figure 4.9. A tentative theory of change for the Tribunal + project
Stakeholder assessments of Tribunal +
An important achievement of the reforms is to be appreciated by the judiciary staff, by lawyers and by court users. The OECD has carried out interviews with lawyers, judges, prosecutors and, in general, there is a consensus that the introduction of new information technology (IT) tools (in particular the new version of the Citius case management system), the new organisation of back offices and the new attendance model Balcão +, are positive steps that substantially improve the functioning of courts – although some improvements appear to be warranted. From the broader citizen perspective, many reported that they would still need to come to court to obtain most information.
According to the clerks who were involved in the re-engineering of work processes in the court of Sintra, the measure did not only lead to important efficiency gains; it also had a transformative effect on the type of tasks that they routinely perform. The new organisational template implemented in Sintra was based on a minute examination of the work of clerks and the selection of appropriate tools to enhance their efficiency, in combination with improved IT systems. One of the consequences was to reduce the number of purely administrative tasks in which judges have to be involved. This has increased the autonomy of the clerks and allowed the judges to focus more on adjudicatory work. The changes also allowed achieving a reduction in the pendency of intermediary decisions by judges, thus increasing the number of judicial acts with a final decision.
The measure has the potential to have similar effects in each of the 22 other court clusters, provided that:
The same degree of care is applied when implementing the template in order to take proper account of the specific conditions and constraints of each case.
The clerical staff and the magistrates have the same level of commitment to the implementation of the new scheme.
The reforms are also positively evaluated by the judges interviewed in the course of this review. The new version of the Citius platform, in which lawyers send their petitions electronically at the start of a case, enables them to better plan their work and be more efficient with the use of their time. They also perceive that digitalisation and the re-organisation of clerical work lead to a more efficient workflow. Thanks to Balcão +, hearings in court start in a smoother manner and with much fewer delays than before.
However, the judges consider that to fully reap the benefits of the new organisation, some of the needs that it has engendered will have to be addressed. These concern:
Training for the judges to fully exploit the capacity of the new information system.
Further improvement of the new information system: Citius was initially developed with the aim to simplify clerical work routines through digitalisation; it was extended to the work of judges and prosecution at a later stage, but some design flaws remain. For instance, a judge or a public prosecutor cannot open a case and modify it without the prior intervention of a clerk.
A formal instruction by the High Council of the Judiciary to judges to use clerical capacities differently.
Human resources: the judges pinpoint that the reform has revived a structural deficit of the Portuguese judiciary in paralegal staff who can assist magistrates on substantive matters (such as verifying the jurisprudence) and by making the interface with the clerks.
For the lawyers, the reform has essentially modified the conditions in which they interact with the judiciary, from their physical access to the courts to the use of Citius as a unique channel for submitting their request and documentation and receiving notifications. Their assessment of these changes is positive both on behalf of their clients and on their own, but the lawyers also indicate several areas in which adjustments and improvement are warranted:
Some tasks have been transferred from the courts to the lawyers as a consequence of the reform, e.g. digitalising documents or collecting information on the parties. The introduction of the new civil procedural code in 2013 had already caused a similar shift (e.g. on the responsibility to summon witnesses in civil cases).
Certain legal means of identification (including driver’s licences, or residence permits for foreigners) are no longer considered valid to access the courts.
The proofs of presence in the court provided to users do not list all the information required by their employers.
Lawyers have to take a separate ticket for every file on which they are working.
There is a lack of clarity concerning the lawyers’ right to access all clerical services, which should be maintained.
Businesses confirm that online services are a greatly welcomed reform that should facilitate the accessibility of justice. Greater communication on these reforms will enable better familiarity with the introduced changes. Yet they also underline the importance of addressing the complexity of laws and regulations, frequent changes in the legislation, backlogs and cost of access.
As testified by these reactions, the reforms introduced as part of the Justiça + Próxima plan and the Tribunal + project constitute an important modernisation package for the Portuguese judiciary. The fact that the key measures have been designed and implemented in dialogue with some stakeholders and that they are appreciated by the judicial staff and the users will contribute to maximising their impact and ensuring their sustainability.
However, complementary measures would be needed in order to reap the full benefits of the reforms in terms of court efficiency and – even more – of performance of the justice sector in general.
The reforms undertaken as part of Tribunal + initiative have so far focused on sectors under the direct responsibility of the Ministry of Justice; in particular, they have not directly addressed the work of judges. This has two important implications for the impact of past and ongoing reforms. First, part of the efficiency gains made in clerical tasks might be lost due to capacity constraints in judicial decision-making. Second, the emergence of new models of work organisation in the courts, which relies on strong co‑operation between the judges and the clerks, might be hampered by the limited involvement of the former. To succeed, the reforms, therefore, need to enter a new phase in which the High Council of the Judiciary and the judiciary at large play an active role in assessing and improving the adjudication process.
Further progress is also needed in the administrative and fiscal courts where, according to diverse stakeholders, efficiency is hampered by excessive formalism and less than adequate support staff (particularly in courts of first instance). Simplification and strengthening of human resources in support functions beyond the Supreme Court (i.e. legal clerks, assistant judges) appear to be the solutions to address backlogs and improve both efficiency and quality. In addition, while systematisation of the clerks’ work can allow gaining in court efficiency, it is important to maintain flexibility for clerks to respond to arising needs.
The monitoring and evaluation framework for Justiça + Próxima
The Ministry of Justice has also developed a common framework to monitor progress in the implementation of all measures included in Justiça + Próxima. The entity or person responsible for each measure reports monthly on development and/or implementation, on the achievement of target indicators and objectives, and on cost. By the end of July 2019, the monitoring indicated that 184 measures had been approved within the framework of the plan, representing a total investment of EUR 42 million. Of these, 95 are reported to be concluded and 64 under implementation (Figure 4.10).
Figure 4.10. Measures concluded vs. development of Justiça + Próxima
Source: Data and statistics of the Portuguese Justice system's official website: Portuguese Justice System (2019), Medidas concluídas vs desenvolvimento do Plano Justiça mais Próxima, https://partilha.justica.gov.pt/Transparencia/Dados-e-Estatisticas/-Medidas-concluidas-vs-execucao-do-Plano-Justica-mais-Proxima.
The assessment of the effects of other measures in the Justiça + Próxima plan usually focuses on direct savings of operational costs (e.g. savings on the cost of post stamps) and human resources. As in the case of the Tribunal + evaluation, the latter are estimated by extrapolating time savings by task to full-time equivalents, which might similarly overlook sources of efficiency loss.
The ministry has not published a global estimate of the benefits of these measures. However, the sum of its evaluations for individual effects exceeds EUR 15 million per year, to be added to the EUR 69.6 million of Tribunal +.
Indicators and objectives mainly focus on the steps of the development and implementation process, with the exception of the measures included in Tribunal +, for which certain outcome-related indicators have been defined (Table 4.1).
Table 4.1. Outcome indicators for Tribunal +
Source: Ministry of Justice of Portugal (2018), “Monitoring the consolidation process on the implementation of the pilot-scheme Tribunal + and roll out preparation at national level’”, Tribunal + project document.
While the current indicators cover the main categories of benefits identified as a result of the implementation of Tribunal +, it would be important to explicitly integrate aspects that are beyond the organisational boundaries of the courts, such as user satisfaction, trust in justice institutions and citizens’ broader access to justice.
In interviews, several stakeholders involved in the decision-making process for the Tribunal + project indicated that one of the important – albeit less tangible – goals of the project was to improve court user experiences and the perception of justice institutions in society. When monitoring the effects of the project, it would, therefore, be useful to seek to integrate these aspects.
Finally, as already noted in the case of Tribunal +, it would be important to design of a general analytical framework for ensuring that the evaluation and monitoring of measures in the Justiça + Próxima plan are both consistent and comprehensive. The Ministry of Justice would benefit from the elaboration of a detailed logical framework similar to the one sketched in Figure 4.9 above for all of the important measures – or clusters of measures – in the plan.
Towards effective scaling up of Tribunal +
Effective planning of the scaling-up process is an important component of the expansion and institutionalisation of justice innovations and modernisation efforts. One of the key challenges in scaling up is that special organisational, financial and human resources are mobilised for the pilot phase, but are not available to the same extent during the rollout phase. As a result, scaling up is often handicapped by the lack of resources, weak capacities and limited ownership. This requires careful balancing of efforts to build capacities, ensure ownership of the reform by a wide range of stakeholders and develop institutionalisation strategies (WHO, 2010).
The Ministry of Justice has identified several main risks of the rollout project to be addressed: i) the need to ensure adequate co-ordination between all entities involved in the project; ii) the need to ensure that the speeding up of registry tasks does not represent a constraint for the workload of judges; iii) disparity in terms of the availability of equipment, which necessitated a phased and gradual approach to the implementation.
To address these risks, the rollout of the pilot project is scheduled to take place in a gradual manner to accommodate local needs and capacities (both in terms of skills, resources and infrastructure). Courts have been categorised and innovations will be implemented according to the selected courts and adapted to their needs (Box 4.2).
The Ministry of Justice reports developing a number of planning and monitoring tools, such as a rollout activity planning and reporting process, use of selected indicators (on court efficiency, efficiency of human resources [e.g. the amount of time gained for court clerks and judges] and on the expansion of the project [e.g. courts that have benefitted from the intervention]), as well as risk matrix and mitigation plans.
In order to increase awareness about the project and address resistance to change, the change management plan for the project also provides for communication tools, such as: the Tribunal + newsletter; press releases and general information on the public portals (e.g. justiça.gov.pt, justicamaisproxima.mj.pt, portugal.gov.pt) issued on a regular basis; a flyer explaining the project to users, etc. Indeed, lessons learned from making reform happen in OECD countries underline the fundamental importance of regular communication.
In parallel, steps to increase the capacity of courts to implement scaling up are being taken (e.g. upgrading court infrastructure, clerks training with the involvement of the Kaizen Institute, identifying champions for rollout in 23 clusters, etc.)
Some of the ongoing challenges include increasing the resource team’s capacity to support scaling up and ongoing strategic choices to support vertical (institutionalisation) and horizontal (expansion/replication) scaling up.
Box 4.2. The current approach to rolling out Tribunal +
Back-office efficiency – through the implementation of the Kaizen methodology for the clerk work process in 23 court clusters. The implementation of the methodology involved delivering training to clerks in all clusters (how to organise daily activities, etc.) and continuous monitoring. “Train the trainers” modules have been developed and implemented as part of the training centre for clerks under the General Directorate of Justice Administration (DGAJ). The trained team leads will engage and train clerk teams in individual courts to implement Kaizen methodologies in 2019. Currently, 23 process units use the Kaizen work organisation methodology and 20 jurisdictions (Comarcas) work with the new office organisation method in 74 buildings.
Front office – the rollout of the new attendance model depends on the availability of the equipment and technology and is scheduled to be implemented in 44 courts by the end of the year. In 2019 and 2020, the attendance model will be rolled out to an additional 55 and 180 courts respectively (large and medium). The single counter for properties, project Balcão +, is now operational in 48 courts. Likewise, from 1st September 2016 to 12 February 2019, the attendance tickets and automatic presence declarations issued amounted 218 888 and 27 163 respectively.
These measures represent EUR 15 208 232 (of which EUR 5 749 000 have been already invested). The total budget from European funds such as SAMA (Sistema De Apoio À Transformação Digital Da Administração Pública), European Social Fund and the Justice Modernisation Fund (FMJ) amounts to EUR 3.5 million and the total Portuguese state budget – OE (Orçamento de Estado) represents EUR 11.7 million
Some associated projects include:
1. Online criminal record certificates (Certificados do Registo Criminal). This programme, launched in July 2016, has issued 106 026 certificates and data from 2017 and 2018 shows that online channel usage increased by 85%. The first semester of 2019 also shows that more than 20% of criminal records issued for companies were issued online. All in all, the access code was used 300 000 times.
2. Online court case consultation. Implemented in May 2017, data up to November 2018 showed that court cases were accessed 22 035 times and 60% of access occurs during out-of-office hours.
3. Alerts sent to lawyers and solicitors, including various notifications, have been operational since July 2016. The number of messages and/or emails sent for those purposes amounted to 136 534 and 70% of users already work under this system.
4. Language simplification in notifications has been applied since June 2017 and notifications sent 302 226 times. The first year of application shows that the number of injunctions where the person paid the debt when receiving notifications increased by 50%.
5. Electronic judicial certificates. Issued since July 2017, the number of online certificates amounts to 41 949, 27% of which were issued without human intervention (automatic/instant). The access code shows this programme has been used 29 000 times.
Source: Data provided by the Ministry of Justice of Portugal, 2019.
Furthermore, ensuring the sustainability of reform efforts requires paying attention to policy institutionalisation, internal regulations, budgets and other dimensions of the justice system. As such, appropriate institutions should be involved in making the transition from decision to implementation (OECD, 2010). To this end, it would be important to continue actively engaging the administration of the Ministry of Justice (i.e. the DGAJ), which is responsible for the organisation and management of courts, including the non-judicial workforce (e.g. clerks), budgets, and information technology (IT), equipment, etc. The DGAJ is also responsible for training non-judicial officers, such as clerks, through the designated training centre and organisation of court workflows. Another important institution actively involved is the Institute for Financial Management and Justice Equipment (IGFEJ), which is responsible for the IT system and equipment in the justice sector and will be overseeing the technical capacity of the courts to implement the new model. In fact, it would be important that both the DGAJ and IGFEJ have a full leading role in the management of both the rollout, implementation and monitoring of the programme.
In addition, international experience underlines the importance of ongoing political leadership and involvement in the success of reforms. As such, in line with the project evaluation, it would be beneficial if the expansion of Tribunal + could be accompanied by the establishment of a dedicated structure in charge of the management and monitoring of the programme’s various aspects. This would help facilitate and accelerate implementation by transferring knowledge, capacity and ownership to the administration while ensuring continuity and taking into account lessons learned from pilot projects.
Such a structure could involve setting up a management and monitoring committee, which could be composed of the Secretary of State for Justice (the executive sponsor of the project and the head of the committee), the Justice Assistant Secretary of State, and representatives from the DGAJ, the Secretary of State for Justice Cabinet and the IGFEJ. The co‑ordination of the team for the management of change programme, according to the usual activities that come under their jurisdictions, could also be assigned to the Secretary of State for Justice Cabinet, DGAJ and IGFEJ (Box 4.3).
Box 4.3. Proposed management model
In line with the project evaluation, the proposed management model could reflect the following roles and responsibilities (Figure 4.11):
Secretary of State for Justice – Executive sponsor of the programme, given that the administrative simplification and information system areas are under his/her responsibility. The Justice Secretary of State could chair the proposed Management and Monitoring Committee.
Office of the Justice Secretary of State – Represented by a person responsible for the overall co‑ordination of the programme and represented in the Management and Monitoring Committee. This person may be assisted by other specialists in specific technical areas.
Management and Monitoring Committee – a new unit, potentially under the office of the Justice Secretary of State – responsible for managing and monitoring the overall rollout and implementation of the programme. The Management and Monitoring Committee is also responsible and its decision is binding.
Directorate-General for Justice Administration (DGAJ) – Represented in the Monitoring Committee, in the co‑ordination of the programme management team and technical managers in areas under their responsibility (e.g. training, tenders, etc.).
Institute for Financial Management and Justice Equipment (IGFEJ) - Represented in the Monitoring Committee, in the co‑ordination of the programme management team and technical managers in areas under their responsibility (e.g. information systems, infrastructures, equipment, tenders, etc.).
Source: Based on the information provided by the Government of Portugal and adapted by the OECD.
An advisory committee, with no decision-making powers, composed of representatives from the High Council of the Judiciary, the High Council for Public Prosecution, the High Council for the Administrative and Fiscal Courts, the Council of Court Clerks, and representatives of Higher Courts, could also be set up.
The establishment of this dedicated structure could enable the participation of main stakeholder and, help ensure continuity and knowledge transfer from current pilot projects under Tribunal +.
Figure 4.11. Current roles and responsibilities
Source: Data provided by the Ministry of Justice of Portugal.
References
CEPEJ (2018), “European judicial systems – Efficiency and quality of justice”, CEPEJ Studies, No. 26. European Commission for the Efficiency of Justice, Council of Europe.
Deloitte (2016), “Tribunal+ Relatório intercalar de impacto”, Tribunal + project document, preliminary version.
European Commission (2018), “Public opinion in the European Union”, Standard Eurobarometer 82, Vol. A: Countries, http://data.europa.eu/euodp/en/data/dataset/S2215_90_3_STD90_ENG.
European Commission (2018), “Public opinion in the European Union”, Standard Eurobarometer 90, Vol. A: Countries, http://data.europa.eu/euodp/en/data/dataset/S2215_90_3_STD90_ENG.
Gertler, P. (2011), Impact Evaluation in Practice, First Edition, The World Bank.
Ministry of Justice of Portugal (2018), “Monitoring the consolidation process on the implementation of the pilot-scheme Tribunal + and roll out preparation at national level’”, Tribunal + project document.
OECD (2019), OECD Economic Surveys: Portugal 2019, OECD Publishing, Paris, https://doi.org/10.1787/eco_surveys-prt-2019-en.
OECD (2010), Making Reform Happen: Lessons from OECD Countries, OECD Publishing, Paris, https://doi.org/10.1787/9789264086296-en.
Portuguese Justice System (2019), Medidas concluídas vs desenvolvimento do Plano Justiça mais Próxima, https://partilha.justica.gov.pt/Transparencia/Dados-e-Estatisticas/-Medidas-concluidas-vs-execucao-do-Plano-Justica-mais-Proxima.
WHO (2010), Nine Steps for Developing a Scaling-up Strategy, World Health Organization, www.who.int/immunization/hpv/deliver/nine_steps_for_developing_a_scalingup_strategy_who_2010.pdf.
Notes
← 1. Ratio between the number of disposed cases and the number of new cases during a given period.
← 2. Ratio between number of pending cases and average number of resolved cases per day during the year, i.e. number of days needed for the court to resolve all pending cases at the current pace.
← 3. See the impact of the introduction of the new Code of Civil Procedure on pendency rates.
← 4. See the separation of enforcement cases that depend on actions from the court.
← 5. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”.
Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus.
← 6. The results of the Sintra pilot project were assessed in Deloitte (2016)’s interim evaluation.
← 7. Information provided by Ministry of Justice, DGPJ Statistics.
← 8. The new Code of Civil Procedure introduces a distinction between enforcement cases that are pending on a decision or action by the court and cases pending on actions by other parties or on the expiry of a period. The change, which is yet to be implemented in case statistics, will allow to distinguish pendent cases that fall within the responsibility of the court from those that do not and make the pendency rates of Portuguese courts easier to compare to those of other countries.
← 9. This impact assessment method is often called “difference in differences”. For a further description, see Gertler (2011).
← 10. Enforcement cases in civil and commercial courts of first instance.