Legal Needs Surveys and Access to Justice

4. Legal Needs Surveys and Access to Justice Indicators

Abstract

This Chapter explores how indicators derived from legal needs surveys can fit into broader measures of access to justice and inclusive development. It then describes how national and global indicators can be formulated to help drive meaningful progress.

4.1. Multidimensionality of access to justice

In 2008, the United Nations Commission on Legal Empowerment of the Poor estimated that more than half the world’s population live outside the effective protection of law. This provides profound context to United Nations Sustainable Development Goal (SDG) 16.3, to “promote the rule of law at the national and international levels and ensure equal access to justice for all.”1 Failure to ensure effective access to civil justice results in substantial social costs and undermines inclusive development. Justiciable problems commonly concern matters of basic welfare, compound disadvantage, diminish opportunities and result in considerable public expenditure.

A better and nuanced understanding of access to civil justice is crucial for developing more effective policies, models and financing. National and global policy and measurement frameworks too often fail to meaningfully account for civil justice issues. Legal needs surveys are a tool to give visibility to legal problems, drive policy responses and understand progress towards access to justice for all.

Efforts to construct measures of access to justice must start from the understanding that access to justice is a multidimensional concept, the breadth of which “heavily depends on how society receives the meaning and scope of justice” (Yuthayotin, 2015, p. 66). Even narrowly construed to refer only to the administration of the law, the concept of justice extends to many aspects of everyday life and encompasses institutional, procedural and outcome related dimensions. However, the concept is increasingly recognised to extend beyond formal process to informal dispute resolution and ultimately to social justice and the distribution of welfare, resources and opportunity.

The working definition of access to justice offered in ‎Chapter 1. centred on the ability of people to obtain just resolution of justiciable problems in compliance with human rights standards; if necessary, through impartial formal or informal institutions of justice and with appropriate legal support. The reference to conformity with human rights standards both extends the concept beyond local legal frameworks (which may conflict with accepted international norms) and indicates standards for independent adjudication.

This definition suggests at least seven distinct dimensions/sub-dimensions of access to justice:

• The substance of law

• The availability of formal or informal institutions of justice

• The quality of formal or informal institutions of justice

• The availability of legal assistance

• The quality of legal assistance

• The quality of outcomes

• Legal capability and empowerment

Moreover, the operationalisation of the constituent concepts of legal need and unmet legal need, which are detailed in ‎Chapter 2. , suggests further dimensions/sub-dimensions of access to justice:

• Timeliness of outcomes

• Legal knowledge

• Awareness of dispute resolution options

• Quality of process

• Awareness of assistance services

• Adequacy of assistance services

In addition to these, further dimensions and sub-dimensions have been set out by others. For example, the final report of HiiL’s Measuring Access to Justice in a Globalising World project set out three dimensions of the cost and quality of access to justice: costs of procedure, quality of procedure and quality of outcome. It also sets out nine sub-dimensions: monetary costs, opportunity costs, stress and emotions, procedural justice, interpersonal justice, informational justice, distributive justice, restorative justice, functionality and transparency (Barendrecht et al., 2010).

Certain dimensions also attach to specific types of justiciable problem. Yuthayotin’s framework for access to justice in business to consumer e-commerce, for example, which extends to the “protection of all aspects of justice” (Yuthayotin, 2015, p. 67), incorporated dimensions relating to the information necessary to allow informed decisions, the nature of transactions, the functioning of the marketplace, and the recognition of consumer rights and their role in the market. Linked to this, levels of access to justice will inevitably vary between justiciable problem types, meaning that being able to disaggregate broad civil justice indicators by problem type can be valuable.

Similarly, access to justice may vary considerably between population groups and vulnerable population groups may face specific barriers to access. For this reason, it is important to design and disaggregate indicators that highlight and are responsive to key demographic characteristics. Thus, the US Justice Index, which assesses laws, rules, and practices across the United States and was published in 2014 and 2016, incorporated access to justice dimensions of language access and disability access. Such an approach exposes differences among population groups. This resonates with the UN General Assembly’s broader call for SDG data to be “disaggregated by income, sex, age, race, ethnicity, migration status, disability and geographic location and other characteristics relevant in the national contexts.”2 The UN’s Inter-Agency and Expert Group (IAEG) responsible for global SDG indicators has established a working group on disaggregation that is exploring strategies to operationalise and expand opportunities for disaggregation. In general, comprehensive and well-chosen social and demographic data helps facilitate disaggregation. In addition, as discussed in Chapter 1, a number of past legal needs surveys have used oversampling and additional samples to improve understanding of marginalised groups.

Finally, dimensions of access to justice can also be dimensions of other social goals, meaning that legal needs-based indicators can further increase understanding of how policies in other sectors can be strengthened to drive progress. The Global Fund to Fight AIDS, Tuberculosis and Malaria, for example, found that “factors such as stigma, discrimination and criminalization drive [particular] populations underground and away from the health services they need.”3 Such health problems require corresponding justice-oriented approaches. Although SDG indicators in Goal 3 track health outcomes, they fail to account for the many ways injustice and health interact. Legal needs surveys could provide a complement by identifying and clarifying the relationship between health priorities and access to justice priorities. Through the introduction of civil justice indicators, policymakers will be able to better understand common legal problems and sources of advice. This will identify sectoral links in particular contexts and enable a more granular understanding of sustainable development across the SDG framework.

4.2. Principles for defining access to justice indicators

Access to justice indicators should be directed towards aspects of access to justice for which meaningful change is both a priority and feasible. Priorities should be defined through collaborative processes inclusive of government, civil society and affected communities. It is also important that indicators are sufficiently sensitive to register such progress as is reasonable to expect. This section describes principles for developing legal needs-based civil justice indicators.

4.2.1. Objectives

Indicators ultimately seek to “answer the question of how much, or whether, progress is being made toward a certain objective.”4 Access to justice indicators can be developed through operationalising defined dimensions of access to justice where operationalisation is defined as “the process of converting concepts into their empirical measurements or of quantifying variables for the purpose of measuring their occurrence, strength and frequency” (Sarantakos, 1993, p. 46).

At different levels, civil justice indicators can respond to a multitude of priorities, such as articulating “access to justice as a societal goal, and establishing the relationship between access to justice and poverty reduction; creating incentives to … expand access to justice; producing data and findings that empower reformers to expand access to justice in public institutions and in areas of law and policy in which they possess expertise … and expanding sources of funding for civil legal aid … and courts” (Columbia Law School and the National Center for Access to Justice, 2016, p. 6). They are also tools to provide project level measurements of progress. They can also be used to track impacts, cutting across institutions, organisations and populations (Vera Institute, 2003, p. 3).

“Just outcome” is at the core of the working definition of access to justice described above. Other dimensions generally support this: just law, fairness of process and appropriate support for those facing problems are guarantors and proxies of just outcomes (as well as hallmarks of constitutional integrity and legitimacy, which are the subject of SDG goals beyond SDG 16). Thus, when defining broad indicators, important consideration should be given to establishing the total number of justiciable problems and the problems considered the most important.5 The next step is to measure key proxies of just outcomes for these problems, including fairness of process and adequacy of different forms of support.

4.2.2. Form

Indicators can be simple or complex and use single or multiple sources of data. The form and nature of access to justice indicators is inevitably determined within a context of limited availability of data and resources. Such constraints were surely a dimension of the IAEG’s 2015 decision to define access to justice and rule of law through just two criminal justice themes: pre-trial detention and crime reporting rates (Columbia Law School and the National Center for Access to Justice, 2016, p. 5). These constraints also exist at a national level, where resources and data availability regularly impede contextualised understanding of access to civil justice.

Simple single data source indicators are typically easier to comprehend, define and communicate. They can also offer strong links to specific policies. Two further benefits of using a single data source are transparency and logistical simplicity. However, if goals are complex, then simple indicators can only partially address them.

“Baskets” of indicators, which bring together simple single variable indicators to provide “a combination of supplementary and complementary indicators to reflect … priorities and planning” (Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16, 2015, p. 22), can offer a more holistic and balanced assessment of progress.6 Multiple indicators can sometimes be combined in a complex single indicator (for example, through an index, such as the Colombian Index of Effective Access to Justice, described in the next section).

Multiple indicators can also be used to illuminate stages of a progression. For example, a simple pie-chart might illustrate the proportion of problems (overall, of a defined level of seriousness, of a particular type, etc.) in which people took no action for reasons that are a concern (capability, inaccessibility of support, etc.), took no action for other reasons, obtained inadequate support (overall, of a defined type, etc.) and obtained adequate support (overall, of a defined type, etc.).

As well as providing a fuller picture, the benefits of multiple or complex indicators include the reduced risk of creating “perverse incentives … [that] may be counterproductive to … broader goals” (Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16, 2015, p. 23).

However, the construction of complex indicators is often contested and involves increased logistical complexity, more complicated communication and the dilution of associations with particular policies. Consequently, a balance must be struck in determining the form of indicators.

All indicators must be defined in their scope. In the case of access to civil justice, scope must be defined with reference to, for example, problem type and population group. For example, indicators could relate to a single problem type or to a small or large set of problems, or to the general population or a smaller defined population. As noted above, indicators should reflect priorities and be directed towards aspects of access to justice for which meaningful change is feasible.

Careful consideration must also be given to the denominators used for indicators. For example, an indicator of the volume of appropriate advice that uses a denominator of population size may go down if problems decrease in number, while a similar indicator using problem number as a denominator might be expected to go up in the same circumstances. Evidently, the use of both indicators provides a balance.

4.2.3. Subject matter

Table ‎4.1 sets out the many dimensions of access to justice suggested in this and previous chapters. Each could be the subject matter of access to justice indicators. Legal needs surveys can provide indicators across a broad range of the dimensions listed. As Table ‎4.1 suggests, sometimes they can uniquely do so. In other areas their use is less suitable. Indicators from surveys have particular value in providing the perspective of individuals, communities, businesses, etc.; potentially doing so across the wide universe of justiciable problems. Simple indicators that legal needs surveys can provide include:

• the incidence of justiciable problems (of defined types),

• the level of awareness of defined support services,

• the proportion of people acting and failing to act to resolve problems (or failing to act for defined reasons),

• the proportion of people obtaining and failing to obtain defined support (or failing to obtain defined support for defined reasons),

• the proportion of people experiencing defined obstacles to defined support,

• the proportion of problems in respect of which defined support was obtained,

• the proportion of problems in which adequate support was obtained,

• the proportion of problems resolved in a manner (i.e. overall process) perceived as fair,

• the proportion of problem outcomes perceived as fair,

• the cost of problem resolution, and

• the level of perceived accessibility of justice.

Legal needs survey data is thus a valuable complement to administrative and other forms of data that can also provide access to justice indicators, particularly relating to legal support and dispute resolution processes. Where these other data sources are weak, legal needs surveys are often strong. Where legal needs surveys are weak, other data sources are often strong.

Table ‎4.1. Access to justice dimensions and relevant data sources

Access to justice dimension		Sub-dimension	Typical data sources
Substance of the law		Adherence to international norms	Expert review
		A2J related legislative provision	Expert review
Incidence of justiciable issues / problems		Volume of issues / problems	Legal needs survey data
		Seriousness of issues / problems	Administrative data, legal needs survey data
		Cost (impact) of issues / problems	Administrative data, legal needs survey data
Favourability of the environment (i.e. structural and institutional barriers beyond the justice system)		Remoteness	Geospatial data
		Broadband availability	Administrative data
		General (personal) capability	Administrative data / general survey data
Provision of public legal information		Law / dispute resolution / assistance	Administrative data
Legal capability / legal empowerment		Knowledge of the law	Survey data
		Awareness of sources of help	Survey data
		Awareness of dispute resolution options	Survey data
		Legal confidence / Subjective legal empowerment	Survey data
Legal assistance	Availability of legal assistance	Volume ● Number of providers ● Range of providers ● Level of public funding ● Form of public funding ● Sustainability of public funding	● Administrative data / Institutional survey data ● Administrative data / Institutional survey data / legal needs survey data ● Administrative data / Institutional survey data ● Administrative data / Institutional survey data ● Administrative data / Institutional survey data
		Physical access ● Geographical access ● Disability access	● Geospatial data ● Administrative data / observation data / user and institutional survey data
		Socio-economic access ● Actual cost ● Affordability ● Language ● Other	Administrative data / user and institutional survey data / legal needs survey data
		Actual use of legal assistance services ● Absolute / caseload ● Relative (per justiciable issue)	● Administrative data / user and institutional survey data / legal needs survey data ● Legal needs survey data
	Quality / appropriateness of legal assistance	Regulation	Expert review / administrative data
		Quality	File review / administrative data / user survey data
		Satisfaction	Administrative data / user survey data
		Trust / perceptions	General survey data / survey data
		Adequacy / appropriateness ● Specific service ● Overall	● Administrative data / user survey data ● Legal needs survey data
Process	Availability of formal / informal institutions of justice	Volume of provision ● Number of courts, etc. ● Rage of institutions ● Level of public funding	● Administrative data / institutional survey data ● Administrative data / institutional survey data / legal needs survey data ● Administrative data / institutional survey data
		Physical access ● Geographical access ● Disability access	● Geospatial data ● Administrative data / observation data / user and institutional survey data
		Socio-economic access ● Actual cost ● Affordability ● Language ● Other	● Administrative data / user and institutional survey data / legal needs survey data ● Administrative data / user and institutional survey data / legal needs survey data ● User survey data ● Administrative data / user and institutional survey data
		Actual use of institutions ● Absolute / caseload ● Relative (per justiciable issue)	● Administrative data / user and institutional survey data / legal needs survey data ● Legal needs survey data
	Quality of formal/informal institutions of justice	Quality of procedure ● Procedural justice / fairness ● Interpersonal justice ● Informational justice	Expert review / file review / observation data / user survey data
		Quality of decisions ● Distributive justice ● Restorative justice ● Functionality ● Transparency	Expert review / file review / user survey data
		Satisfaction	User survey data
		Trust / perceptions	Survey data
		Adequacy / appropriateness ● Specific service ● Overall	● User survey data ● Survey data
	Overall process quality	Quality of procedure ● Procedural justice / fairness ● Interpersonal justice ● Informational justice	Legal needs survey data
	Overall process cost (to individuals)	● Actual cost ● Affordability	Legal needs survey data Legal needs survey data
Overall outcome quality		Timeliness	Legal needs survey data
		Compliance / enforcement ● Specific service ● Overall	● Administrative data / user survey data ● Legal needs survey data
		Other aspects of quality ● Distributive justice ● Restorative justice ● Functionality	Case reviews / legal needs survey data
		Perceptions	Legal needs survey data
Cost of access to justice		Monetary cost ● Actual cost ● Affordability	Legal needs survey data
		Opportunity cost	Legal needs survey data
		Stress and emotions	Legal needs survey data
Inclusivity		Socio-economic disaggregation	Administrative data / legal needs survey data

4.2.4. Process

The process by which access to justice indicators are developed has a significant impact on their utility. Indicators are effective when they are developed through and support an inclusive and deliberative process.

Defining and contextualising people’s legal problems and the possible means for resolving them is a challenge. Meaningful engagement with frontline service providers, community groups and wider stakeholders in the delivery of justice equips them with a deeper understanding of the measurement process, which, in turn, is likely to generate trust and encourage those organisations to act on findings.

Access to justice, in particular, implicates a wide range of government agencies and civil society organisations in measurement. Each agency or organisation can play a role in interpreting findings and driving programmatic and policy reform. Governments, civil society organisations and others have worked together to define indicators, generate data and debate progress. Developing access to justice indicators through legal needs surveys should build on these experiences.

4.3. Using legal needs-based access to justice indicators

Increasingly legal needs surveys are used to inform national priorities and track progress. In recent years, debates around the SDG framework have provided new opportunities for policymakers to link access to justice with broader developmental themes. This section will summarise how legal needs-based indicators can contribute to a more informed and ultimately more effective policy environment for access to justice.

4.3.1. National and thematic indicators

Legal needs surveys have historically been used as a tool to provide a strategic evidence base for access to justice policy decisions. Surveys are increasingly being used to provide indicators that complement and supplement those derived from administrative or other sources of data. Given the breadth of information gathered through legal needs surveys, national policymakers are able to develop contextually specific and operational indicators for policy reform.

In addition to the three surveys which operationalised the concepts of legal need and unmet legal need (detailed in Chapter 2), two additional surveys have been used to provide access to justice indicators for development and strategic planning purposes.

The English and Welsh Civil and Social Justice Survey was originally commissioned, in 2001, to provide indicators to monitor progress against two government departmental Public Service Agreement (PSA) targets concerning access to justice (within the context of the 2000 Spending Review) (HM Treasury, 2000; Legal Services Research Centre, 2003). The first target, to increase the number of people who receive suitable assistance in priority areas of law involving fundamental rights or social exclusion7 was a simple legal need related target. The indicator was calculated by dividing the number of priority area justiciable problems that had been resolved and for which suitable assistance was received – defined as including problems that were in a priority area of law, that had concluded and about which assistance had been successfully obtained from a lawyer, solicitor, law centre, trade union, professional body, Citizen's Advice Bureau, another advice agency, local authority advice service and/or other public authority – by the total number of survey respondents. This target was later replaced by a proportional target, in which the denominator was replaced by the total number of resolved justiciable problems in priority areas. Further “explanatory” indicators also took account of “the degree of helpfulness or satisfaction with advice” (Legal Services Research Centre, 2003, p. 12).

The second target – to reduce the proportion of disputes which are resolved by resorting to the courts – was associated with a departmental objective to improve people’s knowledge and understanding of their rights and responsibilities, including how to resolve disputes that affect them. Policymakers sought to understand how resolution could be conducted in a way and at a cost proportionate to the issue at stake. This indicator was associated with an objective to make civil and family law clearer and more easily enforceable, giving priority to key government objectives in tackling social and economic issues.

The legal needs module incorporated into the 2016 Colombian Quality of Life Survey contributed 9 of the 13 exclusively survey-based indicators making up the Index of Effective Access to Justice, along with the denominator for a tenth indicator. One of the nine indicators concerned legal capability (the proportion of respondents who did nothing to resolve justiciable problems because they did not know what to do or where to obtain assistance). One concerned access to legal assistance and was similar in form to the first English and Welsh indicator (proportion of problems about which respondents obtained assistance from a legal professional). Two concerned access to justice institutions (the proportion of respondents who did nothing to resolve justiciable problems because of cost, time or procedural complexity; the proportion of respondents using justice institutions who reported problems ending through a decision or agreement). Two concerned procedural fairness (the proportion of respondents with needs still pending because of poor-quality dispute resolution provision; the proportion of respondents using justice institutions that reported they would use them again). Three concerned compliance (the proportion of problems concluded by complying with a decision or agreement following the use of a justice institution; the proportion of problems taken to justice institutions that were “resolved”; the difference in the experience of men and women as regards the previous indicator).

While the Index of Effective Access to Justice demonstrates the breadth of access to justice indicators that legal needs surveys can contribute to, their potential is greater still. The short- and longer-form legal needs surveys set out in the previous Chapter and in Annex B include questions relevant to 37 of the 54 secondary dimensions of access to justice listed in Table ‎4.1.

4.3.2. Global indicators

The SDGs provide a new forum for access to justice advocates to define critical global dimensions of access to justice. The omission of civil justice in the two official SDG 16.3 indicators spurred new debate around how to understand and track progress towards access to justice at a global level. In 2017, members of IAEG and the United Nations Statistical Commission formally acknowledged that two criminal justice indicators were insufficient to measure the breadth of SDG 16.3, which relates to the rule of law and access to justice for all. To address this limitation, the IAEG recommended the addition of a global indicator focused on “access to civil justice” into the global SDG framework.8 Over the course of 2019, the IAEG is expected to consult on a specific global indicator to measure “access to civil justice”.

The push for inclusion of “access to civil justice” builds on the knowledge that the majority of justiciable problems are not dealt with by courts or tribunals. Administrative data relating to courts and other justice institutions says little about the extent to which needs are met and justice is achieved across problems as a whole. As people obtain assistance with problems from many places, administrative data relating to assistance is fragmented and can, at best, only talk to the minority of problems for which assistance is obtained.

For all of the reasons explored in this document, legal needs surveys represent an important tool for understanding global progress towards access to justice. They present a picture of progress impossible to obtain from other data sources, and can provide indicators that uniquely extend to people’s full experience of justiciable problems. Thus, legal needs surveys can complement existing global indicators reliant on administrative and victimisation survey data, to help provide a more comprehensive global picture of access to justice.

In addition to advancing progress towards 16.3, a legal needs-based indicator focused on access to civil justice would also present a valuable complement to other SDGs. As described in Chapter 1, common legal needs across countries connect to SDG priority areas, including: land, property and housing (Goal 1: End Poverty, Goal 15: Life on Land), employment (Goal 8: Decent Work), health (Goal 3: Ensure healthy lives), family issues (Goal 5: Achieve gender equality, Goal 8: Decent Work), social protection (Goal 1: End Poverty) and economic and consumer issues (Goal 10: Reduce inequality). By including a legal needs-based indicator, policy makers will be able to better understand the ways in which justice systems interact with developmental priorities and, therefore, devise more multidimensional strategies.

The specific formulation of a single, global legal needs-based indicator will be challenging. For all the reasons discussed in this document, access to civil justice is most strategically understood and measured through multidimensional assessments. Nonetheless, in the years leading up to the adoption of the initial SDG indicator framework, a Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16 recommended a civil justice 16.3 indicator in the initial consultation process. This proposed indicator – “the proportion of those who have experienced a dispute in the past 12 months who have accessed a formal, informal, alternative or traditional dispute resolution mechanism and who feel it was just” – includes important elements for a headline global civil justice indicator. However, its reference to formal, informal, alternative or traditional dispute resolution mechanisms is unwieldy and restrictive (considering the very many paths to justice people take and the difficulties, detailed in Chapter 2, in defining dispute resolution processes in a way appropriate across all jurisdictions). Additionally, it gives insufficient attention to whether people have access to assistance, which may be an important component of challenging power asymmetries. Moreover, as suggested in Chapter 3, perceptions of process fairness may be a better choice of proxy for outcome quality than perceptions of outcomes themselves, not least as many processes can be addressed through policy.

Thus, a simple reformulation of this proposed headline civil justice indicator developed from legal needs surveys could be:

The proportion of disputes experienced in the past 24 months resolved through a process considered fair by the disputants.

This construction would provide policymakers with an overall snapshot of civil justice in a particular country. It might not, however, be particularly well suited to advance innovations in the ways in which countries deliver access to justice. General experiences of dispute resolution do not necessarily tell policymakers whether justice institutions are advancing core goals. Moreover, people’s assessment of fairness of process could depend significantly on expectations, which may vary greatly between populations and perhaps conspire against a transformative vision of access to justice.

A more meaningful and operational global indicator would endeavour to capture the adequacy of support, particularly for specific marginalised groups who need assistance in understanding, using and shaping the law. An alternative indicator focused on adequacy of support could be:

The proportion of disputes experienced in the past 24 months resolved through a fair process where the disputants received adequate support.

Drawing on the logic tree for proxy measurement of legal need and unmet legal need set out in ‎Chapter 2. , adequate support could be defined broadly (e.g. from defined sources: column 6 of Figure ‎2.1) or narrowly (e.g. from defined sources and considered adequate by recipients: columns 6 and 7), and could also refer to problem seriousness (columns 1 and 2) and legal capability (columns 3 and 4). However, the addition of each additional aspect of measurement increases the complexity and ultimately the cost of an indicator. If all the above aspects of measurement are included, the resulting indicator would be more appropriately framed as a compound and multidimensional indicator:

The proportion of non-trivial disputes experienced in the past 24 months resolved through a fair process and in respect of which the disputants received adequate support to make informed decisions and pursue a fair outcome.

The components and illustrative questionnaire questions that could be used to construct this indicator are set out in Table ‎4.2.

Table ‎4.2. Compound indicator dimensions & corresponding questionnaire numbers

Access to Justice Dimension	Sub-dimension	Illustrative questionnaire question number	Example coding
Incidence	Volume of issues / problems	Q1 – Problem identification / category Q11 – Whether problem is concluded	Problems included if in scope and concluded.
	Seriousness of issues / problems	Q3 – Problem seriousness	Problems included if seriousness exceeds defined threshold
Legal capability	Legal awareness / understanding	Q18a – Understanding of legal rights	Assistance required if legal rights not understood
	Legal confidence	Q18d – Confidence in ability to achieve fair outcome	Assistance required if no confidence
Legal assistance	Actual use of legal assistance services	Q6 – Sources of information, advice or representation	Assistance obtained if help obtained from specified sources
	Adequacy of legal assistance services (overall)	Q18c – Adequacy of help	Assistance adequate if all help needed was obtained
Process	Procedural justice / fairness	Q14 – Process fairness	Process fair if ‘fair to everybody concerned’
Outcome	Timeliness	Q22 – Problem start date Q23 – Problem end date	Timely outcome if problem duration less than defined threshold

Other factors, such as cost, could also be incorporated into such an indicator.

As described in ‎Chapter 1. , inequality in access to civil justice exacerbates disadvantage and disproportionally affects marginalised groups. Thus, as noted above and as with other SDG indicators, global access to civil justice indicators need to be capable of disaggregation in order to highlight the experience of different groups, on the basis of income, gender, age, ethnicity, geographic differences, among others. The SDG framework includes a specific target on building capacity for specific forms of disaggregation:

By 2020, enhance capacity-building support to developing countries … to increase significantly the availability of high-quality, timely and reliable data disaggregated by income, gender, age, race, ethnicity, migratory status, disability, geographic location and other characteristics relevant in national contexts9

In practice, SDG indicators have taken two approaches to meeting this requirement: For some themes, SDG indicators specifically state what level of disaggregation is required in the indicator itself. For example, SDG indicator 1.4.2 on land tenure security calls for disaggregation by gender (“the proportion of total adult population with secure tenure rights to land, with legally recognised documentation and who perceive their rights to land as secure, by sex and by type of tenure”). Indicator 2.3.2 on productivity of small holders calls for disaggregation by both gender and indigenous status (“Average income of small-scale food producers, by sex and indigenous status”).10

Other SDG indicators adopt a broad approach and do not include specific reference to particular populations in the indicator. These indicators instead rely on the SDG framework’s commitment to disaggregation across population groups.11

In constructing a global indicator for access to civil justice, policymakers should consider whether the global indicator ought to be constructed to specifically explore the experiences of marginalised groups or pursue disaggregation across those domains articulated by the IAEG. For example, a disaggregation-specific indicator could be framed as “the proportion of A, B or C disputes experienced by X, Y or Z population in the past 24 months resolved through a process considered fair by the disputants where the disputants received adequate support.” While this would ensure headline data highlighted the experiences of particular groups, there could be a risk that it ignores groups relevant in particular national or subnational contexts or fails to highlight emerging inequalities of experience. Despite these questions of scope, legal needs-based indicators nonetheless offer the potential for holistic measures of access to civil justice not capable of being generated by other sources.

Because of low levels of legal literacy, the greatest challenge to the economy of a legal needs survey is efficiency in defining the range and establishing the incidence of justiciable problems within a survey’s scope. This was discussed in Chapter 2. A “relatively straightforward” line of questioning in the context of indicators of access to health care is “direct questioning of individuals as to whether there was a time when they needed health care but did not receive it” (Allin and Masseria, 2012, p. 114). For example, the model question for “unmet need for medical examination or treatment” that appears in the EU Income and Living Conditions Survey asks, “Was there any time during the past 12 months when you really needed medical examination or treatment (excluding dental) for yourself?”12 If the response is positive, the following question is asked: “Did you have a medical examination or treatment each time you really needed?” Such an approach might be trialled in the legal domain, with people asked whether they were unable to secure needed advice or assistance concerning a dispute (with some examples of dispute types provided to indicate scope13). However, for the reasons discussed in Chapter 2, the lack of specificity is likely to be problematic, unless the question follows on from the identification of specific types of problems. Further experimental investigation into the impact of different approaches to justiciable problem identification/definition would be of great value. Beyond problem identification, further Table ‎4.1 prioritisation will also often be necessary.

4.3.3. Limits of legal needs surveys revisited

As described in Chapter 1, while legal needs surveys provide a unique insight into the experience of justiciable problems across populations, they have their limitations, as do all surveys. General population legal needs surveys are excellent for establishing levels of justiciable problem experience, levels of legal need, and the relative use of legal services and processes. However, without large sample sizes, they are limited in what they tell us about the experience of particular populations, as is important in the context of SDG indicators. As explored in Chapter 2, oversampling or additional surveys are often necessary to enable the experience of particular populations to be reliably compared to that of the general population.

General population legal needs surveys are also ill-suited to capturing details of the experience of formal processes, particular legal services and rarer problem types. Again, without large sample sizes, their utility is limited. General population surveys are inherently inefficient at capturing data concerning phenomena experienced by a relatively small proportion of the population. Thus, the great ability of general population legal needs surveys to expose the peripheral nature of justice institutions to most disputes also points to their weakness. If detailed information about particular legal institutions is required, it is usually better sourced from user surveys or, if technical or beyond the purview of users (as discussed in Chapter 2), from administrative data.

Evidently, the measurement of different dimensions of access to justice, and capture of different perspectives, requires the use of diverse data sources. As illustrated in Table ‎4.1 – which sets out the data sources most appropriate to investigating each of the broad range of dimensions of access to justice suggested in the text above – some aspects of access to justice are best measured using administrative data, geospatial data, expert review and user surveys. However, when it comes to problem incidence (a common denominator for indicators), legal capability, legal need, problem resolving behaviour, adequacy of support and overall processes fairness, it is best to use legal needs survey methodologies to get the big picture.

Moreover, the combination of measures from different data sources can yield the most valuable insights. For example, the limited knowledge of lay individuals means they are unable to provide much insight into the technical aspects of access to justice, such as the location or availability of legal assistance services. If people are unaware they live in close proximity to a legal aid office, or are unaware of or mistakenly believe they are ineligible for legal aid, then they cannot attest to its availability. In this case, geospatial data, administrative data or expert review of legal aid eligibility would be better sources of information. But the combination of measures of population awareness and availability/eligibility may be most telling for policymakers, highlighting the disconnection between provision and uptake.

An example of the above approach is the Colombian Index of Effective Access to Justice. Of the 24 indicators used, 13 rely exclusively on survey data, nine of which use legal needs data, nine administrative data and two geospatial data.14 One uses both administrative and survey data, as numerator and denominator respectively. And elsewhere, the Justice Index15 and Rule of Law Index16 utilise expert review – typically a specialised form of survey of expert stakeholders in the justice system to gauge, for example, the nature of procedural rules and the presence or absence of best practices for ensuring access to justice.

4.4. The path ahead

Many jurisdictions in which legal need surveys have been used have undergone a paradigm shift in the orientation of justice policy. Legal needs surveys have contributed to a shift away from an institutional to an individual and collective perspective, because they show that disputes occur and are resolved mostly outside the purview of both courts and lawyers. Legal needs surveys are unique in their ability to give an overview of civil justice, providing overarching, individually focussed, system-wide indicators, and exposing the reach of support services and institutions. New comparative datasets like the World Justice Project and HiiL will provide policymakers with new opportunities to experiment with different indicator formulations. However, legal needs surveys are not ideally suited to monitoring details of specific service provision and formal dispute resolution processes. Such complementary data can be gathered from other sources. In the access to justice field, there are many options to draw from when it comes to defining both headline indicators and the combination of supplementary and complementary indicators in order to complete a comprehensive and balanced picture. The inclusion of an access to civil justice indicator using legal needs survey methodologies would prove a tremendous asset to the current SDG global indicator framework. This global headline indicator should supplement and catalyse national level operationalisation of legal needs indicators.

4.6. References

Allin, S. and C. Masseria (2012) “Measuring access to health care in Europe”, in A. McGuire and J. Costa-Font (eds.), The LSE Companion to Health Policy, Edward Elgar, Cheltenham.

Barendrecht, M., M. Gramatikov, I. Giesen, M. Laxminarayan, P. Kamminga, L. Klaming, J.H. Verdonschot and C. van Zeeland (2010), Measuring Access to Justice in a Globalising World, HiiL, The Hague.

Barendrecht, M., Y.P. Kamminga and J.H. Verdonschot (2008), “Priorities for the justice system: Responding to the most urgent legal problems of individuals”, TILEC Discussion Paper No. 2008/011, TISCO Working Paper No. 001/2008), Tilburg University Faculty of Law, Tilburg.

Columbia Law School and the National Center for Access to Justice (2016), Recommended Access to Justice Indicators for Implementation of Goal 16 of the UN 2030 Sustainable Development Agenda in the United States, Columbia Law School Human Rights Institute and the National Center for Access to Justice, Fordham University, New York.

Her Majesty’s Treasury (2000), 2000 Spending Review: Public Service Agreements, HMSO, Norwich.

Legal Services Research Centre (2003), Baseline Measures for the Lord Chancellor’s Department’s PSA Targets 3 and 6, Legal Services Research Centre, London.

Pleasence, P., N.J. Balmer, T. Tam (2009), “Failure to recall,” in R.L. Sandefur (ed.), Access to Justice, Emerald, Bingley.

Sarantakos, S. (1993), Social Research, Macmillan, Basingstoke.

Vera Institute (2003), Measuring Progress Toward Safety and Justice, Vera Institute, New York.

Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16 (2015), The Indicators We Want, Federal Ministry for Economic Development and Development, Berlin and United Nations Development Programme.

World Justice Project (2018), World Justice Project Rule of Law Index 2017, World Justice Project, Washington, DC.

Yuthayotin, S. (2015), Access to Justice in Transnational B2C E-Commerce, Springer, Cham.

Notes

← 1.  Sustainable Development Goals contains target 16.3 http://www.un.org/en/development/desa/what-we-do/rule-of-law.html.

← 2.  UN Resolution 70/1, adopted by the General Assembly on 25th September 2015 paragraph 74(g).

← 3.  https://www.theglobalfund.org/en/key-populations.

← 4.  Handbook of Democracy and Governance Program Indicators (Washington, DC: U.S. Agency for International Development, 1998), referenced by Vera Institute (2003, p. 2), available at https://www.vera.org/publications/measuring-progress-toward-safety-and-justice-a-global-guide-to-the-design-of-performance-indicators-across-the-justice-sector.

← 5.  Barendrecht et al. (2008) and Pleasence et al. (2009) discuss various methods that can be used to establish the most urgent types of justiciable problem.

← 6.  The Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16 defined complementary indicators as “those that are necessary to complete measurement of a complex concept,” and supplementary indicators as those that “round out” measurement, to allow countries to “adapt the universal goals to their own contexts and identify other dimensions of the target that are important to them” (Virtual Network for the Development of Indicators on Peaceful, Just and Inclusive Societies for Goal 16 2015, p. 22).

← 7.  The original target called for a 5% increase by 2004.

← 8.  Access to civil justice is detailed as a “possible additional indicator to address” in Annex V of the Report of the Inter-agency and Expert Group on Sustainable Development Goal Indicators, 7-10 March 2017.

← 9.  Target 17.18.

← 10.  See https://unstats.un.org/sdgs/indicators/indicators-list/.

← 11.  Interagency and Expert Group Working Group on Data Disaggregation, https://unstats.un.org/sdgs/files/meetings/iaeg‑sdgs‑meeting‑06/20170607_updated%20version‑overview%20of%20standards%20of%20data%20disaggregation.pdf.

← 12.  Methodological guidelines and description of EU-SILC primary target variables in the health domain are set out at http://ec.europa.eu/eurostat/documents/1012329/8655367/PERSONAL+DATA_Health.pdf/0a942278-dd03-47f6-9af6-3f3000f678ce.

← 13.  The approach of indicating disputant types, employed in the disputes module of the 2017-2018 pilot of the dispute module of the South African Governance, Public Safety and Justice Survey, would seem to most naturally fit the form of this question.

← 14.  Ibid.

← 15.  http://justiceindex.org/methodology.

← 16.  See World Justice Project (2018).