This chapter presents the four-step framework for planning people-centred legal and justice services. It further elaborates its first step: identifying and measuring the needs for these services and good practices.
Equal Access to Justice for Inclusive Growth
Putting People at the Centre
Chapter 3. Identifying and measuring legal and justice needs
Abstract
Introducing the four-stage planning process for people-centred legal and justice services
Shifting from a system focus to a people focus
Past efforts to enhance access to justice tended to focus on the justice system perspective. In measuring access to justice, governments and international organisations historically mainly centred on measuring institutional performance, particularly as it relates to criminal justice (Dandurand, Kittayarak and MacPhail, 2015[1]). Many justice system indicators and performance measures are framed from this perspective. While important for achieving efficiency and sound functioning of the justice system, these measurement approaches are rarely based on an understanding of the types of justice problems people have, what institutions they engage, the effectiveness of dispute resolution mechanisms: what works, for whom and for what types of legal and justice needs.
One of the most important current trends in public service, as highlighted in the different OECD roundtables, is the shift towards people-centred perspective as the guiding principle. In a government-centred or institution-centred perspective, users are the passive recipients of services, whereas, in a user-, citizen-, or people-driven perspective, people voice their demands and needs, contribute to shaping the policy agenda and evaluate service content and delivery. In accordance, the OECD Serving Citizens framework highlights the governments have the responsibility to serve its people and provide public services including justice services that should be designed to meet the expectation and needs of their citizens (OECD, 2015[2]).
The people-centred approach flows directly from and is consistent with the legal needs research and inclusive growth frameworks outlined in Chapter 2. There is profound evidence that people experience service needs in ways that do not match with traditional justice pathways and formal institutional arrangements (e.g. problems that have legal and non-legal components).
This shift toward the people-centred approach can be summed up in one phrase, “people ought to be the universal denominator”, in measuring access to justice and setting up service delivery (IDLO, 2015[3]). Two comprehensive Canadian reports take as their starting point the need for substantial change in legal and judicial culture to facilitate greater access to justice (Action Committee on Access to Justice in Civil and Family Matters, 2013[4]; CBA, 2013[5]). Cultural change is seen as a precondition for the development and implementation of specific measures. The focus of this cultural shift is towards a people-centred justice system or “putting the public first” – the system focuses on people’s needs, not those of justice system professionals and institutions. This focus is based on all people, especially members of vulnerable or marginalised groups. People-centricity flows from and reinforces the concept of justice as a public service.
The value of people-centred access to justice innovation is multidimensional. People’s needs and experiences are key to identifying innovation potential in and provide the rationale for reflecting on the delivery of legal and justice services. The perspective of individuals, families and communities provides important input from the “outside in” about how this potential can best be realised, by suggesting how services can be more responsive to people’s needs and through the co-development of reforms guided both by service providers and users. A deep understanding of people’s daily realities, their diverse situations and experiences help guide policy implementation. An appreciation of people’s capabilities, situations and experiences can illuminate how these realities could serve as resources or challenges in implementation justice programmes and services. This people-focused perspective must also guide evaluation so that we can measure the extent to which legal and justice services contribute to the fair resolution of legal issues and problems, positive case outcomes, and broader outcomes such as reduction of disadvantage and increased socio-economic inclusion.
Systematic planning for people-centred services
The main challenge in delivering people-centred access to justice services is to ensure they meet justice needs, integrate user experience and that service provision extends “to the ‘right’ mix of services, to the ‘right’ clients, in the ‘right’ areas of law and in the ‘right’ locations and at the ‘right’ time” (Pleasence et al., 2014[6]).
The design and delivery of “good” legal and justice policy involve systemic planning processes1 based on four questions:
What types of legal and justice are experienced by whom?
Where and when are these needs experienced?
What works in designing services that meet these needs?
Where to deliver services and how should they be evaluated?
All of these questions are posed and answered from the perspective of those experiencing the legal and justice needs. It is important to note that the approaches to measuring needs and formulating policies and services must be tailored to specific populations in each country context (and their legal environment), in particular taking into account the situation of vulnerable individuals, families and communities and those living in conditions of disadvantage.
This report brings together research evidence, examples of country practices, tools and models that can assist OECD and partners countries to carry out each of these four stages of systematic planning. Potential measurement tools and indicators are identified and discussed with a view to assisting governments and other entities to implement necessary reform and compare changes in access to justice over time and from place to place.
Governments, researchers, service providers and others around the world increasingly recognise the need for sound and comprehensive evidence to underpin the development of good policy and cost-effective service delivery. A wide range of methodologies can be utilised to develop the information required for planning people-centred legal and justice services. These include wide-scale legal needs surveys, tailored surveys, analysis of administrative data compiled by legal, justice and other governmental institutions, and research projects using a variety of methods (e.g. content analysis, observation studies, participatory research). Other potentially useful data sources include data held by local councils and contained in planning and research reports and toolkits. On-the-ground knowledge held by local legal and human services, as well as other stakeholders, can also contribute usefully to planning legal services. Often a combination of approaches is necessary to gain a full understanding of legal needs and how, when and where to meet them, within a given country context (see below).
In some areas of human services, such as health, the tradition of evidence-based policy and decision-making is well established. While not suggesting existing evidence-based research is sufficient and comprehensive for all decisions in all areas, the intent to underpin policy decisions with sound cost-benefit, cost-effectiveness and other evidence-based approaches is widely accepted in many sectors, e.g. health, across OECD and partner countries.
In comparison to some other human service sectors, the legal sector has not historically been well grounded in co‑ordinated, robust, evidence-based research and analysis. This lack of data can be attributed in part to the fact that the information tends to be scattered across government departments and agencies as well as private service providers. Issues related to marginalised populations tend also to be data poor. Additionally, the justice portfolio tends to account for a relatively small portion of government expenditures, particularly if costs related to policing and corrections are excluded.
Effective people-centred legal and justice services that provide appropriate access to justice for all parts of society, contribute to inclusive growth and individual and community well-being and is affordable for both the state and individuals, families and small and medium-sized enterprises (SMEs), will likely only be achieved when government, policymakers, service providers and ordinary citizens have access to comprehensive, reliable and clear information to inform their decision-making. Five key audiences for such information and analysis are:
Policymakers (predominantly government).
Key service provider agencies (e.g. courts and tribunals; Alternative dispute resolution mechanisms, ADR).
Other service provider agencies (such as legal aid and other legal assistance service providers which often operate as clients of state agencies).
Individuals, families and other entities experiencing legal needs.
Social scientists working in associated fields.
The need for enhanced information and data is an issue that cuts across the four planning stages. While some of the information sought or questions to be answered may vary with the issue/legal matter type, the client and the jurisdiction, there will also be many aspects of commonality. The emerging challenge for justice sectors is to identify and implement cost-effective strategies to obtain and sustain this evidence base.
Getting started: Identifying and measuring legal need
Step 1 rationale
The first stage in systematic planning for people-centred legal and justice services is to identify and measure legal needs. The central questions are: who experiences legal needs and what legal needs do they have? Are those needs currently being met? An elaboration of these broad questions is set out in Box 3.1. Posing these questions from the perspective of the individual, family, community or SME is a prerequisite for policy and service design and planning.
Box 3.1. Identifying and measuring legal needs
When seeking to identify the legal needs of the community, the questions to be answered for policymakers and service providers might include:
What is the prevalence of legal problems across the community in a particular time period?
What are the types/areas of law of the legal problems experienced?
Are some demographic groups of individuals more likely to experience different types of legal problems?
Are some demographic groups of individuals more likely to experience multiple legal problems?
What do people do (including different demographic groups) when faced with legal problems?
For those who seek assistance, who do they seek assistance from? How do they come to that service provider? What are their motivations (e.g. rational economic motivation, proving a point, etc.)?
Why do some people not seek assistance?
How do people resolve, or seek to resolve, their legal problems?
Where are the geographic areas of high legal needs?
What are the adverse consequences on other aspects of life (health, employment, housing or other areas of social engagement) experienced by those experiencing legal need?
What are the costs of meeting the identified legal needs? What are the costs of not meeting the need?
Legal assistance services
The full range of legal services targeted at poor and disadvantaged people would include services such as: legal aid, community legal centres, legal centres targeting particular groups (such as women, indigenous people, refugees, etc.), pro bono services provided by the profession, and others.
Identifying the needs is only the first step. To provide effective legal services ensuring people have effective access to justice that allows them to enforce their rights and fully participate in economic and social life, regardless of economic or social disadvantage, the following information may be relevant for policymakers and service providers:
Where are the areas (geographic, areas of law, etc.) of greatest legal need?
What types of services are most effective and cost-effective to meet which needs for which groups and in what circumstances?
What services are available to meet the identified needs of individuals in their own geographic location?
How can the most appropriate services be made available at the right time and place across the geographic and legal breadth of jurisdictions?
How can we monitor and evaluate changing needs and the impact of service delivery to ensure the legal needs of the community continue to be met in the most effective and efficient way?
Formal dispute resolution processes (e.g. court, tribunal and ADR services)
When seeking to provide an evidence base to inform decision-making in civil justice in relation to the operation of formal civil dispute resolution organisations and processes (e.g. courts, tribunals, mediation services, etc.), questions to be answered (at any given time/period) might include:
Who is suing whom in a particular court/tribunal?
What types of civil claims are being litigated or mediated?
Of what value are the claims being pursued?
What matters are defended and by whom?
Who has legal representation and who does not?
How are matters finalised? How long does it take to reach certain outcomes in a particular court/tribunal?
What is the cost to the state of certain types of conduct? What is the cost to litigants of pursuing certain matters? (Forell and Mirrlees-Black, 2016[7]; Forell and Mirrlees-Black, 2016[8])?
For potential users of civil courts and tribunal services, other questions may be important in order to enable them to make decisions about the most efficient avenue for resolving their disputes or legal problems. Their questions might include:
How long do various stages of the likely civil litigation process take and, in particular, what is the expected timeframe to obtain resolution?
What is likely to be the personal/organisational time and effort (i.e. such as court attendance, etc.) required to obtain that resolution?
What costs are likely to be incurred?
On recorded experience of similar matters, what are the chances of success?
Sources: Coumarelos, C. et al. (2012[9]), Legal Australia-Wide Survey: Legal Need in Australia in Access to Justice and Legal Needs, Law and Justice Foundation of New South Wales; Pleasence, P. et al. (2014[6]), Reshaping Legal Assistance Services: Building on the Evidence Base, Law and Justice Foundation of New South Wales, Sydney; Forell, S. and C. Mirrlees-Black (2016[7]), Data Insights in Civil Justice: NSW Local Court, Law and Justice Foundation of NSW, Sydney.
This section considers three main people-centred approaches to identifying and measuring the legal needs of the community:
Service provider administrative data.
Legal needs surveys.
Target studies (often complementary/supplementary to the other methodologies).
The possibilities and challenges of each of these methodologies are discussed and examples of country practices are provided. To begin, a case study of the Access to Justice and Legal Needs Program of the Law and Justice Foundation of New South Wales (NSW) (Australia) imparts an introduction to these methodologies and how they can be employed together to establish a more comprehensive understanding of legal needs (Box 3.2).
Box 3.2. The Access to Justice and Legal Needs Program of the Law and Justice Foundation of New South Wales (Australia)
The Law and Justice Foundation of New South Wales’ (Australia) Access to Justice and Legal Needs research programme (A2JLN) adopted three separate but interrelated methodological streams in order to identify the legal needs of the community, with a particular emphasis on the needs of disadvantaged people. After an initial phase of public and key stakeholder consultations, during which the objectives and approaches for the research programme were refined, the programme adopted four methodological strategies:
Administrative data (service provider data) – Recognising that legal service providers were providing services to citizens daily and recording data in relation to this service delivery, the programme began by seeking to obtain access to and then harmonise this data. After an initial scan of many data sources, for reasons of manageability and to ensure a lower socio-demographic citizen focus, the programme settled on data from Legal Aid, Community Legal Centres and from LawAccess (a free, telephone and online advice, referral and information service). This approach – the first of its kind that we were aware of – revealed both the potential and the challenges of using this “administrative data” identify and measure legal needs. This potential and the challenges are discussed elsewhere in this report.
Legal needs surveys (LNS) – It became clear during the initial phase of the A2JLN programme that there was some concern across the sector in relation to how much of the existing legal need was actually reaching the formal legal service providers, despite the fact that workloads seemed to be high. In other words, the service provider data was really revealing what could be called “expressed need” or “patent need” (the needs that people actually took action through the legal service providers to resolve), but not necessarily the “unexpressed need” or “latent need” (that need that existed but did not reach formal legal service providers). The programme then adopted a strategy of legal needs surveys to identify the legal need that existed in the community – including both the need that was reaching the legal service providers but also the need that was not.
Targeted studies – It was nevertheless appreciated that there are certain priority groups that will often be missed in both these sources. Certain disadvantaged groups will be unlikely to use services and, depending on how conducted, respond to surveys. Older people (especially those in residential care), homeless people, people with mental illness or intellectual disability, and people in remote indigenous communities fall in this category. Therefore, a range of complementary studies needed to be undertaken to “fill the gap” with those groups. The A2JLN programme, therefore, included a strategy of targeted qualitative and mixed-method approaches to identify the legal needs of groups such as older people, people with mental illness, prisoners and homeless people.
Participation in law reform processes – During the establishment phase of the A2JLN programme, it was recognised that for there to be appropriate and sustainable access to justice, citizens needed to be able to realistically participate in law reform processes. As the A2JLN programme progressed, it became clear that little insight on this aspect was being revealed from the existing three strategies, and so a separate research project to examine participation in law reform processes was undertaken, resulting in a report “By the people, for the people?” (Nheu and McDonald, 2010[10]).
Sources: Law and Justice Foundation of NSW; Nheu, N. and H. McDonald (2010[10]), By the People, For the People? Community Participation in Law Reform: Summary Report, Law and Justice Foundation of NSW, Sydney.
Administrative data
Administrative data is data collected continually by a range of service providers2 and agencies as they interact with clients or users, in accordance with regulatory requirements (e.g. vehicle registration, taxation purposes) and to administer government-funded programmes (Productivity Commission, 2013[11]). In the legal and justice sector, administrative data can include data collected by courts, tribunals, ADR agencies, community justice centres and legal aid providers in relation to the demographic characteristics of clients, the matters assistance is sought for, the services provided, and the like.
Administrative data:
usually originates as a means of government and other agencies reporting to their funders for accountability purposes
is collected primarily for “administrative” and management reasons, and not generally for research purposes
is collected routinely with each provision of service, should also be distinguished from data obtained through bespoke research, using methodologies and data collection to answer specific questions
should be distinguished from what might be called “official data” such as data collected and produced by national statistics agencies, such as national census data, etc.
Systems are often designed to have some degree of flexibility, to allow for changes in the information collected, and also ad hoc data collection, i.e. capture strategically important information when needed (whether internally or externally specified).
Administrative data forms a treasure trove of information.” In Australia, for example, the Productivity Commission, the premier national economic analysis agency, long argued that “administrative datasets could be instrumental in gaining insights into whether government programmes:
Meet their stated objectives – do they work or are other influences at play?
Operate as intended – do recipients respond to (dis)incentives, are there unanticipated (good or bad) effects on recipients or the community?
Are delivered effectively – are there queuing or discouragement effects?
Deliver services in the right places – are services located near people in need?”
This information, in turn, underpins deeper assessments “about whether the particular policy mix is coherent or whether other policy initiatives work to hinder desired outcomes” (Productivity Commission, 2013[11]).
In the National Center for Access to Justice’s Justice Index, the presence and absence of policies account for a form of administrative data. Core categories of data important to analysis of access to justice include: i) data about presence and absence of policies; ii) data about degree to which policies are implemented; iii) data about whether the policies, even if implemented, are able to accomplish their intended goals; iv) data about actual impact of justice system policies as experienced by people; and v) data about the impact of justice system policies as experienced in communities. The value of data is considered at its peak when these different categories of data are all available for comparison and when different forms of data within each category are also available.
It is important to emphasise that administrative data is not in itself people-centred since the information collected is framed by institutional concerns and priorities: administrative data is institution- or system-centred. Nevertheless, when well collected and managed, administrative data can provide valuable information to monitor and evaluate service provision and answer key information requirements. In addition to information about legal need discussed below, greater access to and analysis of administrative data could, for example:
Provide greater insight into understanding disadvantage.
Assist in connecting knowledge across human service disciplines, such as to health, to allow us to better “connect the dots” among the human factors contributing to inclusive growth.
Allow analysis of the interactions between welfare and work (Productivity Commission, 2013[11]).
However, conducting research with data derived from service providers and agencies is not without problems. The use of some administrative data may raise issues related to their privileged status. Key advantages and disadvantages of administrative data are summarised in Table 3.1.
Table 3.1. Advantages and disadvantages of using administrative data for policymaking
Source: Smith, G. et al. (2004[12]), The Value of Linked Administrative Records for Longitudinal Analysis, Report to the ESRC National Longitudinal Strategy Committee.
Administrative data in the civil justice sector
The present state of administrative data in the legal and justice sectors has yet to reach its full potential and the level of other areas, e.g. social policy. In many countries, justice sector administrative data can often lack consistent terminologies, definitions and data collection protocols, within jurisdictions as well as between jurisdictions.3 This may not be surprising. The civil justice sector tends to be fragmented, diffuse and complex – partially as a consequence of the proliferation of service providers and the institutional independence of various actors and entities (CBA, 2013[5]). These conditions also exist in the criminal justice system, but it has achieved a higher degree of coherence through focused collaboration. Furthermore, legal system reforms in many countries over the last 50 years, in particular, saw major increases in legislative intervention in people’s day-to-day lives (and thus the creation of legal rights and responsibilities), many intended to improve access to justice for individuals. However, such changes generally occurred in a piecemeal fashion, responding to particular priority needs and issues arising in different jurisdictions at different times, often under the responsibility of different government portfolios, with different funding and reporting requirements.
As governments and other justice agencies moved to a more evidence-informed basis for policy development, initiatives have begun to identify the utility of presently collected administrative data for policy development/service provision purposes and to improve this data over time.4 For example, as part of Latvia’s Justice for Growth approach, Latvian court managers use administrative data as a tool for systematic planning more people-centred services including to link court management reforms with the needs of individuals and businesses and using this perspective to inform priorities. The courts are also exploring ways of merging available data to inform future reforms.
Using administrative data for insight into legal need
Administrative data on the use of legal and justice services can include information on various types of cases, use of legal assistance services, the numbers of unrepresented parties before the courts and tribunals, use of ADR mechanisms that are connected to the formal justice system, and so on. This data can be mined for insight into the extent of legal need by contributing a picture of the:
demographic groups that access particular legal services
nature of the expressed legal needs
pathways people follow to resolve the problem (or not) and the outcomes they achieve.
Communities, governments, service providers and researchers often turn first to available administrative data relating to usage of existing legal services as a means of identifying legal need. For example, by looking at the data collected daily by legal and justice service providers, it may be possible to gain insight into questions such as:
How many people are currently accessing services (such as legal aid, community legal centres, but also for courts, tribunals and ADR processes)?
Who those people are (i.e. what socio-demographic groups do they represent)?
Where do they live and where are the legal services needed?
What are the legal matters they enquire about?
Who inquires about which problem/matter types?
Are there socio-demographic groups that seem under-represented?
What pathways do people take to reach this legal service, and what subsequent pathway may be taken after the particular service to resolve the legal problem (Mirrlees-Black and Williams, 2016[13])?
As noted above, administrative data is inherently limited in its ability to contribute to a full understanding of the extent of legal need in the community because it is collected for broader purposes with different definitions. Moreover, it is important to note that, as with all data, this data is a reflection of the method of how it is obtained. Legal service delivery administrative data is shaped by many factors, which are summarised in Box 3.3.
Box 3.3. Factors shaping legal service delivery administrative data
Problems experienced – The problems recorded by service providers will usually only reflect the problems/matters that the particular service provider deals with and not the wider range of problems across the broad community, and for which no service may necessarily be available. Thus, unless the particular service handles a comprehensive range of legal problem types, what will be recorded will be the problems service providers actually deal with, not the number, range or severity of problems clients may actually experience.
Only what is recognised as “legal” – Research consistently finds that many people do not recognise problems as “legal”. If they do not, it is unlikely they will go to a legal service provider for assistance, and thus will not appear in the administrative data.
Ready, willing and able to act – Many people, particularly disadvantaged people with complex lives and multiple problems, often lack the legal literacy, capability and psychological preparedness to seek out legal assistance for legal problems they experience. Some people also may have a different cultural approach to seeking legal help.
Financial and other eligibility requirements – Public and not-for-profit legal services providers almost always have limited funding and as a result, will limit the matters/clients they can serve in accordance with a range of eligibility criteria. Clients/matters that fall outside these criteria will not appear in the administrative data.
Availability of service – geography – Physical proximity demonstrated in many situations to be a key factor in a citizen’s access to available legal services. That is, if service provision locations are too far away or otherwise inaccessible, many potential clients may not seek them out.
Availability of service/resources-budget constraints – Resource limitations will impact on service availability and hence its use.
Data classification taxonomies, quality and harmonisation – Different providers often collect data using independent frameworks and methods.
Data accessibility – While the data may be collected at an individual office level, it might not be available in a useable form for regional/jurisdictional planning and analysis.5
To address the various inconsistencies in how variables are defined, recorded, collected and classified, substantial work may be required to improve consistency and harmonisation before useful analysis is possible.
A key factor in the usefulness of administrative data on legal service delivery for any purpose is the consistency and quality of its collection, across multiple service locations and service providers, and its harmonisation across regions and jurisdictions. At present, many legal and justice service providers developed their system for the collection of administrative data outside the framework of overarching, consistent, evidence-based frameworks. In some countries, steps are taken or contemplated to improve the consistency in justice sector data (Box 3.4).
Box 3.4. Civil data improvement programme in England and Wales
“England and Wales reported plans for civil data improvement as part of their court reform programme, which aim to transform justice services and change the way they are delivered. A major emphasis is placed on IT and digitisation, with the idea of better capturing point of contact data with users. Various core data and key performance indicators (KPIs) are being developed as well as ways to understand the customer experience and satisfaction with service. For example, 48 000 claims have been issued through the reformed Online Civil Money Claim service in public test mode since Spring 2018, with user satisfaction currently at 88%. Following an exercise to specify data requirements the service will also collect enhanced Management Information in the future.”
Source: Ministry of Justice, United Kingdom.
For example, in Australia, the Law and Justice Foundation's Access to Justice and Legal Needs (A2JLN) programme’s strategy utilising legal service delivery data involved a detailed attempt to harmonise legal assistance sector data for analysis purposes without substantive changes to the manner of data collection in the different agencies. This approach was chosen in light of the magnitude of the challenge of encouraging change in data collection practices across many different agencies. A successful example of the use of administrative data as a measurement tool is the Legal Assistance Services Data Digest (LASDD) developed by the Law and Justice Foundation of New South Wales (NSW). This resource draws together data held by a number of publicly funded legal assistance providers in NSW and allows users to analyse/produce reports on:
types of legal matters for which inquiries are made
demographic characteristics of assistance seekers
pathways that service users take to resolve their problems
changes in legal inquiries over time
the rate and number of inquiries for particular population groups and geographic areas of NSW
spatial displays of the association between legal need and census-based measures of socio-economic disadvantage (LJFNSW, 2014[14]).
Given the wide variation in data collection protocols, definitions (or sometimes the lack of them) and practices likely existing across jurisdictions, short‑term progress to improve harmonisation may only be likely through the detailed consideration of unit-record data at the lowest level of data collected and then adopting concordance/conversion processes. However, longer-term results may best be achieved by the progressive adoption of common and agreed data definitions and protocols. Using this approach recent attempts in Australia were made to begin the process to facilitate the harmonisation of data collection across all legal assistance service providers, culminating in the publication of an Australian National Legal Assistance Data Standards Manual for legal assistance services6 (Box 3.5).
Box 3.5. Australian National Legal Assistance Data Standards Manual (NLADSM)
The Australian NLADSM aims to “give best practice guidance to legal assistance service providers to facilitate the collection of consistent and comparable data”. The standards are underpinned by the following rationale:
“To achieve a national, integrated system of legal assistance, service providers must work together to improve access to justice, address disadvantage and maximise service delivery within the resources available.
Consistent and comparable data collection provides the foundation for a strong, reliable evidence base that informs legal assistance policy and supports planning and resource allocation decisions to ensure that limited resources are directed to areas where services will have the greatest benefit. A reliable evidence base also provides a comprehensive overview of legal assistance services delivered and the people and organisations accessing those services and helps service providers respond to current and emerging legal need. Finally, reliable evidence demonstrates how effectively the legal assistance system, and the justice system more broadly, is functioning in Australia.”
Source: Australian Government (n.d.[15]), National Legal Assistance Data Standards Manual, Attorney-General’s Department.
The work of the Australian Attorney-General’s Department-sponsored Civil Justice System Evidence-based project, and now the NSW Department of Justice’s civil court and tribunal data project, are other manifestations of moves (although far from comprehensive) towards improved data collection and spatial displays usage practices. As part of the establishment of this latter strategy, research is currently being directed to examine in detail the administrative data collected in all New South Wales civil court and tribunals for the purpose of determining how useful it is to answer key questions, but also to inform improvement in data collection practices.7
Legal needs surveys
Legal needs surveys were introduced in Chapter 2 in the form of a high-level summary of the evidence derived from surveys carried out in over 50 countries across more than 30 jurisdictions over the past 2 decades. It is important to note that legal needs surveys are the best means of obtaining the most representative understanding of legal need from the people’s perspective. A growing trend of innovative countries is implementing legal needs surveys for service planning. Colombia employed a national legal needs survey as the starting point for its access-to-justice plan as part of its National and Regional Development Plan (Box 3.6).
Box 3.6. Colombia Legal Needs Survey
Colombia implemented one of the most comprehensive legal needs surveys to date. Building from legal-needs methodologies tested by the Colombian civil society organisation Dejustica, a national survey was carried out in collaboration between the Department of National Planning and National Statistical Office in 2016. The survey data can be disaggregated by geographical area, by gender and other identity characteristics.
Colombia developed an Effective Access to Justice Index (Índice de Acceso Efectivo a la Justicia) based on the survey results to inform their long-term justice plan. Colombia’s index explores six dimensions of access to justice: i) a favourable environment (which is concerned with structural and institutional barriers to justice that lie outside of the justice system); ii) legal capability; iii) legal assistance; iv) fair procedure; v) compliance with judicial decisions; and vi) access to institutions. The legal needs module of the Colombian National Quality of Life Survey contributes to 10 of 24 indicators, focusing on legal capability, legal assistance, access to justice institutions, procedural fairness and enforcement. Availability of legal aid is a cross-cutting factor.
In addition to the index, Colombia is using the survey data to: close the gaps between supply and demand, identifying the barriers for access to justice of vulnerable population; develop a ten-year justice plan; plan and design appropriate, timely, integrated and targeted justice services, sustainability reforms and indicators for monitoring evolution.
Source: National Planning Department, Colombia.
Given the in-depth coverage of this topic in other documents including OECD-OSJI (2019[16]), the report provides only an overview of legal needs survey methodologies and compares the differences between the identification and measurement of legal needs through surveys and administrative data.
Since the 1970s ABA/ABF national survey of legal needs, the 1994 Comprehensive Legal Needs Study in the United States and the Hazel Genn’s 1999 Paths to Justice Study in the United Kingdom (Genn, 1999[17]), a new approach to access to justice research and reform through legal needs surveys has been taken and continues to evolve. Legal needs surveys focus on the perspective of the individual by identifying their legal and justice needs and the pathways they follow to resolve them rather than merely what was generally believed to be what people wanted or needed. To achieve this aim, such research included substantial surveys of what legal problems were actually experienced by citizens and what action they took (or did not take) in response to these legal problems.
These surveys do not rely on individuals being able to identify a problem as “legal” or having a possible legal resolution, nor do they assume that everyone with a legal problem seeks to take action to defend them. Individuals do not need to have come into contact with the legal and justice sector in order to report the experience of a legal problem. Survey responses are based not on opinion but rather on the recollection of problems, impacts of the problems, action taken in response to the problems and outcomes of the problems. Legal needs surveys are regarded internationally as giving “clear indications on the trends of justiciable problems […], the ways they are solved and on the number of them that are not solved in a satisfactory manner” (Barendrecht et al., 2012[18]). The results of legal needs surveys (both civil and criminal) made an important contribution to understanding common obstacles to accessing justice experienced by both individuals and businesses and have paved the way for some reforms.
In recognition of the importance of this methodology, the OECD collaborated with Open Society Justice Initiative (OSJI) to develop a conceptual framework and toolkit to assist countries to employ person-based surveys to better shape justice policy and inclusive growth and development outcomes, and to further harmonise civil justice measures at the national and global levels (Box 3.7).
Box 3.7. OECD-OSJI Guidance on Understanding Legal Needs and Effective Access to Justice for Inclusive Growth
Understanding legal needs calls for collecting good data. Legal needs surveys are a specialised form of survey research and should adhere as far as possible to best practices in field research.
OECD-Open Society Justice Initiative (OSJI) guidance is designed to assist planners, statisticians, policymakers and advocates to develop, administer and use legal needs more effectively. Best practices build upon the lessons and experience of the past 25 years of legal needs surveys to suggest effective strategies for measuring access to justice at a local, national and global level. The toolkit discusses common pitfalls, key lessons and effective practice in the implementation of legal needs surveys. The toolkit answers common questions in developing, deploying and using legal needs surveys:
What are the essential elements of legal needs survey methodologies?
How should justiciable problems be defined and what problems should surveys include?
What period of time should surveys cover?
How can the types of legal and social support people do (or do not) seek and receive be best captured?
How can the processes and institutions involved in resolving problems be best captured?
What forms of outcome and impact associated with legal issues can be captured?
Are there elements that can be used in surveys across jurisdictions and contexts?
How can surveys contribute to indicators to measure progress on access to justice?
In seeking to answer these questions, the toolkit sets out a framework for the conceptualisation, implementation and analysis of legal needs surveys, providing guidance and tools in a modular way and allowing application into different types of surveys.
Source: Extract from OECD-OSJI (2019[16]), Legal Needs Surveys and Access to Justice, OECD, Paris.
While findings from legal needs surveys are necessarily at odds with administrative data depending on the jurisdiction, comparing them may highlight that legal needs surveys reveal a different ordering of legal problems (Table 3.2).
Table 3.2. Contrasting top 10 legal issues from needs survey and administrative data results: Example from New South Wales (Australia)
Note: The picture would be very different if a different type of service data were used for comparison.
Sources: Data from ‘Problem group - respondents with problems’ is from Coumarelos, C. et al. (2012[9]), Legal Australia-Wide Survey: Legal Need in Australia in Access to Justice and Legal Needs, Law and Justice Foundation of New South Wales; Data from ‘Problems enquired about’ is from LJF Legal Assistance Services Data Digest: LawAccess NSW, Legal Aid NSW (Advice) and NSW Community Legal Centres (2015);
In the Australian example highlighted in Table 3.2, the survey data is drawn from a representative sample of the population, through the exploration of a broad range of legal problems (although not exhaustive) and using a methodology that does not require the respondent to know whether the problem is legal or not. It is not dependent upon issues of client eligibility for service delivery nor upon the limited range of services or legal matters dealt with by a particular agency. Further, it does not rely upon the individual taking action through a legal and justice service provider to address the problem.
By contrast, the administrative data has limitations as described above. Importantly, while it provides more detailed information in some areas than legal needs surveys will (e.g. address of service user, specific legal problem type, pathways to and from the agency, etc.) and does not rely on the client’s memory to report certain facts, the administrative data is not usually obtained from a representative sample of the population.
Administrative data obtained from legal service providers sometimes better reflect the issue of differing degrees of severity of legal problems than a legal needs survey (depending on the methodology employed). While the legal and justice service provider data will be limited to those matters that were brought to (and assisted by) the service provider for assistance and resolution, and not representing a broader range of problems experienced by the community, they nevertheless potentially represent more serious matters that required assistance from a formal legal service provider. This may indeed be analogous to the difference in the contrasting pictures one obtains from health data obtained through hospital admissions as compared to data from population health surveys.
As such, while consumer, crime and housing matters were most often experienced across the population, the administrative data reflects that people seek legal advice and assistance from formal legal advisors for matters concerning the custody and residence of children post separation (as indicated by the category “Live with/spend time with”) than for any other matter.
Another general strength of legal needs surveys is the insight they provide in relation to the sources of advice used by citizens when faced with a legal problem. Surveys from a number of countries demonstrated that only a relatively small proportion of matters experienced by the community are resolved through legal processes or with the assistance of formal legal assistance services. For example, in Australia, only 3.4% of matters are resolved through courts and tribunals and a further 3.4% resolved through formal dispute resolution and complaint handling processes. These findings are also demonstrated in other similar jurisdictions: in Canada, only 7% of people use formal court or tribunal processes to resolve their legal problems (CFCJ, 2016[19]). Further, in terms of advice seeking, many people who experience a legal problem seek advice or assistance to resolve it and only a small portion of those seek assistance from a legal assistance service. In the US, a recent study found that:
“When third parties other than family and friends became involved, these seldom included lawyers or courts. Situations that were selected for detailed follow-up in the life histories provide rich information about how people handle these kinds of events. In these life histories, very few situations involved courts or tribunals of any kind: 8% of the total situations were selected for in-depth follow-up. Of the small number of situations with some kind of court involvement (n=36), people sought advice or other assistance from attorneys in just over two‑fifths (42%) of cases. In situations with no court involvement, they sought the assistance of attorneys in 5% of cases” (Sandefur, 2014[20]) (Figure 3.1).
Figure 3.1. How people handle civil justice situations: Percent handled by each means, middle city
Source: Sandefur, R. (2014[20]), Accessing Justice in the Contemporary USA: Findings from the Community Needs and Services Study, American Bar Foundation, University of Illinois at Urbana-Champaign.
These are important findings in the context of the examination of measurement and planning in the interests of access to justice. While legal service administrative data reveals the matters for which assistance is sought, and thus something about the legal needs of the people that take their problems to the service, this nevertheless represents only a small portion of the legal need in the community. Service provider data is of limited utility for robust identification of the legal needs of the community.
This data also serves other important related purposes, as discussed further in the report, including:
Identifying the number and type of services delivered to priority clients.
Matching/“mapping” the delivery of legal services against relevant measures of need as a means of determining relative greater and lesser areas of service and possibly areas of deficiency.
Monitoring the impact of reforms and changes over time.
Targeted legal needs studies: Focusing on vulnerable groups or specific sectors
Legal needs surveys and administrative datasets both assist in the identification and measurement of legal needs and contribute to the planning and evaluation of people-centred legal and justice services. Even in combination, however, these two measurement approaches may not provide a full picture of legal needs: the needs of certain groups may not be measured and even for those needs measured, the information gathered may be insufficient for planning purposes.
Both these methodologies can miss certain populations given that disadvantaged groups are often less likely to use services and respond to surveys. Older people (especially those in residential care), homeless people, people with mental illness or intellectual disability, and people in indigenous or remote communities often fall into this category. People who are vulnerable and/or living in conditions of disadvantage are more likely to have legal needs, particularly complex ones, and a restricted ability to access legal and justice services. From an inclusive growth and sustainable development perspective, these tend to be the groups that need to be prioritised.
A range of supplementary or targeted legal needs studies can be utilised to “fill the gap” with those groups and issues to ensure that a comprehensive understanding of legal needs is obtained. Targeted complementary studies, usually of particular groups or areas of legal needs, can add a greater depth to our knowledge of legal and justice needs in the community. Further, insight gained from major survey legal needs studies may need updating and enhancing more frequently than full-scale legal needs studies can be conducted.
In Australia, for example, to complement the legal needs surveys and administrative data analysis, specific legal needs studies were conducted targeting the elderly (Ellison et al., 2004[21]), prisoners (Grunseit, Forell and McCarron, 2008[22]), people who are homeless (Forell, McCarron and Schetzer, 2005[23]), people with a mental illness (Karras et al., 2006[24]), indigenous people (Cunneen and Schwartz, 2008[25]) and others. Such complementary studies can be tailored to allow in-depth investigations of the barriers experienced by each of these group in accessing justice, and methodologies can be designed to suit the needs and types of problems faced by each priority group. Similarly, in Canada, complementary and supplementary studies focused on groups such as abused women unrepresented in the family law system (Luke’s Place Support and Resource Centre for The Denise House, 2008[26]), Aboriginal peoples and their access to legal information (Zalik, 2006[27]), and migrant youth (Mah, 2011[28]) among others, while in New Zealand, studies included those targeted at people with an intellectual disability (Mirfin-Veitch et al., 2014[29]) and indigenous people (Black et al., 2013[30]), and in the United States, low-income earner (LSC, 2009[31]).
In England and Wales (United Kingdom), individual service providers and justice institutions are taking active measures to develop a targeted understanding of their clients’ or users’ justice needs and how they experience the justice system on an ongoing basis. For example, an ongoing court-based project in Sweden designed to increase public confidence and trust in the courts and contribute to the systematic quality enhancement of court functions conducted both internal and external dialogue on how their court is functioning and perceived to be functioning. This initiative included interviews with people involved in a court case directly after the case was concluded. The interview process identified a number of obstacles faced by people that affect the quality of their access to justice including, for example, shortage of legal information provided to the citizens regarding court procedures. This input from the users of court services informed a number of key policy reforms both in terms of how judges interact with parties and on common policies for reception in courthouses, information for parties and on the ways in which judgments are written. These reforms already had a positive impact and the Swedish courts are committed to continuing this people-focused approach on an ongoing basis.
Other survey data to understand people’s experience
In addition to legal needs surveys, other types of survey data can be used to assist in the planning process. For example, in Israel, legal aid providers made a shift from “gate-keepers” to “justice facilitators through a planning process that involved surveying 1 000 legal aid clients, lawyers from private practice who provide legal aid and public service legal aid lawyers. This data led to a diagnosis of the problems in legal aid delivery experienced by people trying to access the service (including difficulty providing the necessary documents, long waiting periods, misunderstanding the process). Too much lawyer time was spent on reviewing applications and the process was too bureaucratic. Reforms based on input resulted in a restructuring of the application process, a direct access pilot project and a targeted strategy to assist youth to overcome obstacles experienced by specific groups of individuals (including a specialised application for soldiers, outreach interviews and the establishment of First-Aid-Legal-Aid Stations in a number of courts). This process also resulted in the development of a specialised legal aid service for children and youth. Ongoing planning activities will include an alumni advisory board, satisfaction surveys on an ongoing basis and empirical research.
Key findings
A shift toward people-centred legal and justice services and justice as a public service is key to meeting legal needs in a way that contributes to inclusive growth and individual and community well-being.
A systematic planning based on people’s actual needs and experiences, and those of SMEs, is required to ensure that service provision extends “to the ‘right’ mix of services, to the ‘right’ clients, in the ‘right’ areas of law and in the ‘right’ locations and at the ‘right’ time”.
The design and delivery of “good” legal and justice policy involve systemic planning processes based on four questions:
Identification and measurement of legal need: who experiences legal needs and what legal needs do they have?
Mapping of legal need: where and when are these needs experienced?
Design of services: what works to meet these needs most effectively?
Delivery of services: how should these services be delivered and evaluated?
There is profound evidence that people experience legal needs in ways that do not always match up with traditional justice pathways and formal institutional arrangements (e.g. problems that have legal and non-legal components).
People’s needs and experiences should be the starting point for identifying innovation potential, provide the rationale for the design and delivery of legal and justice services, guide policy implementation and evaluation of policy and services.
A wide range of methodologies can be utilised to develop the information required for planning people-centred legal and justice services. These include: wide-scale legal needs surveys; tailored surveys; analysis of administrative data compiled by legal, justice and other governmental institutions; research projects using a variety of methods; data held by local councils and on-the-ground knowledge held by local legal and human services, as well as other stakeholders.
Targeted legal needs surveys are often required to identify and measure the needs of specific disadvantaged populations since their situation is insufficiently captured by general legal needs surveys or administrative data.
While legal needs surveys are the best single means of obtaining the most representative understanding of legal needs from the people’s perspective.
A key factor in the usefulness of administrative data on legal service delivery for any purpose will be the consistency and quality of its collection, across multiple service locations and different service providers and its harmonisation across regions and jurisdictions. The present state of administrative data in the legal and justice sectors has yet to reach its full potential.
Encouragement of flexibility in the design of data systems for key agencies would allow/promote the capture of strategically important information when needed (whether internally or externally specified).
In comparison to some other human service sectors, the legal sector has not historically been as well grounded in co‑ordinated, robust, evidence-based research and analysis. The emerging challenge for justice sectors is to identify and implement cost-effective strategies to obtain and sustain this evidence base.
References
[4] Action Committee on Access to Justice in Civil and Family Matters (2013), Access to Civil and Family Justice: A Roadmap for Change, Ottowa, Canada, http://www.cfcj-fcjc.org/sites/default/files/docs/2013/AC_Report_English_Final.pdf.
[15] Australian Government (n.d.), National Legal Assistance Data Standards Manual, Attorney-General’s Department, https://www.ag.gov.au/LegalSystem/Legalaidprogrammes/Pages/National-Legal-Assistance-Data-Standards.aspx.
[18] Barendrecht, M. et al. (2012), Towards Basic Justice Care for Everyone: Challenges and Promising Approaches, The Hague Institute for Innovation of Law (HiiL).
[30] Black, S. et al. (2013), Evaluating the Aims, Methods and Results of Indigenous Courts, Centre for Mental Health Research, University of Auckland, https://www.lawfoundation.org.nz/wp-content/uploads/2013/12/4.-Evaluating-the-aims-methods-and-results-of-indigenous-courts.pdf.
[5] CBA (2013), Reaching Equal Justice – An Invitation to Envision and Act, Canadian Bar Association, Ottawa, Canada, https://www.cba.org/CBAMediaLibrary/cba_na/images/Equal%20Justice%20-%20Microsite/PDFs/EqualJusticeFinalReport-eng.pdf.
[19] CFCJ (2016), Everyday Legal Problems and Cost of Justice in Canada: Survey Data, Canadian Forum on Civil Justice, Toronto.
[9] Coumarelos, C. et al. (2012), Legal Australia-Wide Survey: Legal Need in Australia in Access to Justice and Legal Needs, Law and Justice Foundation of New South Wales.
[25] Cunneen, C. and M. Schwartz (2008), The Family and Civil Law Needs of Aboriginal People in NSW: Final Report, University of NSW.
[1] Dandurand, Y., K. Kittayarak and A. MacPhail (2015), Justice Indicators and Criminal Justice Reform - A Reference Tool, International Centre for Criminal Law Reform and Criminal Justice Policy, Vancouver.
[21] Ellison, S. et al. (2004), The Legal Needs of Older People in NSW, Law and Justice Foundation of NSW, Sydney.
[23] Forell, S., E. McCarron and L. Schetzer (2005), No Home, No Justice? The Legal Needs of Homeless People in NSW, Law and Justice Foundation of NSW, Sydney.
[7] Forell, S. and C. Mirrlees-Black (2016), Data Insights in Civil Justice: NSW Civil and Administrative Tribunal - Overview (NCAT Part 1), Law and Justice Foundation of NSW, Sydney.
[8] Forell, S. and C. Mirrlees-Black (2016), Data Insights in Civil Justice: NSW Local Court, Law and Justice Foundation of NSW, Sydney.
[17] Genn, H. (1999), Paths to Justice: What People Do and Think about Going to Law, Hart Publishing: Oxford.
[22] Grunseit, A., S. Forell and E. McCarron (2008), Taking Justice into Custody: The Legal Needs of Prisoners, Law and Justice Foundation of NSW, Sydney.
[3] IDLO (2015), Indonesian-Dutch Comparative Justice, Policy Workshop, 10-11 December 2015, The Hague, https://www.idlo.int/sites/default/files/pdfs/events/Report%20-%20Comparative%20Justice%20Policy%20Workshop.pdf.
[24] Karras, M. et al. (2006), On the Edge of Justice: The Legal Needs of People with a Mental Illness in NSW, Law and Justice Foundation of NSW, Sydney.
[14] LJFNSW (2014), The Development of the Legal Assistance Service and Data Digest Online, Law and Justice Foundation of New South Wales.
[31] LSC (2009), Documenting the Justice Gap in America: The Current Unmet Civil Legal Needs of Low-Income Americans (An updated report).
[26] Luke’s Place Support and Resource Centre for The Denise House (2008), Needs Assessment and Gap Analysis For Abused Women Unrepresented in the Family Law System: Final Report and Recommendations, http://www.lukesplace.ca/pdf/MAG-Research-Summary-Report-Apr-13-08.pdf.
[28] Mah, S. (2011), Public Legal Education and Information (PLEI) for Immigrant Youth, A Scoping Review, Ministry of Labour, Citizens’ Services and Open Government, British Columbia, https://www2.gov.bc.ca/assets/gov/law-crime-and-justice/about-bc-justice-system/publications/plei-immigrant-youth.pdf.
[29] Mirfin-Veitch, B. et al. (2014), Developing a More Responsive Legal System for People with Intellectual Disability in New Zealand, Donald Beasley Institute, http://www.donaldbeasley.org.nz/assets/Uploads/Law-Fn-Rpt-Final-Word-Version-2.2.2015.pdf.
[13] Mirrlees-Black, C. and S. Williams (2016), Collaborative Planning Resource: Presentation to Federation of Community Legal Centres, (Victoria) Inc., Melbourne, Victoria.
[10] Nheu, N. and H. McDonald (2010), By the People, For the People? Community Participation in Law Reform: Summary Report, Law and Justice Foundation of NSW, Sydney.
[2] OECD (2015), “The OECD serving citizens’ framework”, in Government at a Glance 2015, OECD Publishing, Paris, https://doi.org/10.1787/gov_glance-2015-55-en.
[16] OECD-OSJI (2019), Legal Needs Surveys and Access to Justice, OECD, Paris.
[6] Pleasence, P. et al. (2014), Reshaping Legal Assistance Services: Building on the Evidence Base, Law and Justice Foundation of New South Wales, Sydney.
[11] Productivity Commission (2013), Annual Report 2012-13, Annual Report Series, Productivity Commission, Australian Government, https://www.pc.gov.au/about/governance/annual-reports/2012-13.
[20] Sandefur, R. (2014), Accessing Justice in the Contemporary USA: Findings from the Community Needs and Services Study, American Bar Foundation, University of Illinois at Urbana-Champaign.
[12] Smith, G. et al. (2004), The Value of Linked Administrative Records for Longitudinal Analysis, Report to the ESRC National Longitudinal Strategy Committee.
[27] Zalik, Y. (2006), Aboriginal Peoples and Access to Legal Information, Community Legal Education Ontario, http://plelearningexchange.ca/wp-content/uploads/2014/03/Aboriginal-Peoples-and-Access-to-Legal-Information-Report.pdf.
Notes
← 1. Systematic evidence-based analysis is an essential element of all good policy. It is particularly important for social services with such a major share of budget outlays (Productivity Commission, 2013[11]).
← 2. While the paper is here speaking primarily about service provider data more generally, the range of legal sector service providers might include: courts, tribunals, mediation and dispute resolution centres, legal aid offices, community legal centres, Aboriginal legal centres, telephone advice services, etc.
← 3. This perspective arises from the work across a range of legal assistance sector as well as court and tribunal data over a number of years in NSW, Australia, by the Law and Justice Foundation of NSW.
← 4. In Australia, two examples include: The Commonwealth Attorney-General Department, Civil Justice System Evidence‑based Project, https://www.ag.gov.au/LegalSystem/Pages/Anevidencebasefortheciviljusticesystem.aspx; The NSW Department of Justice/Law and Justice Foundation of NSW, Civil Courts and Tribunal Data Program, http://www.lawfoundation.net.au/ljf/app/5141D05E8AC0EF1D85258078004EC072.html.
← 5. Based on presentation by Mirrlees-Black (2016[13]).
← 6. Only focused on legal assistance services, and not necessarily comprehensive.
← 7. See: The Civil Courts and Tribunal Data Program from the NSW Department of Justice/Law and Justice Foundation of NSW, http://www.lawfoundation.net.au/ljf/app/5141D05E8AC0EF1D85258078004EC072.html.