This chapter lays out the background behind the creation of an international Technical Expert Group tasked with preparing the draft Handbook on Measuring Digital Platform Employment and Work. It then describes the content of the Handbook and summarises its key findings and statistical recommendations. Recommendations are either general and applicable to any statistical vehicle, or targeted at specific surveys.
Handbook on Measuring Digital Platform Employment and Work
Report
Table of contents
Share
Facebook
Twitter
LinkedIn
1. Overview and recommendations
Abstract
Context
Work mediated by online platforms is one of the most debated types of non-standard work. While platform employment and work activities provide workers with options for matching work with their skills and life circumstances, they also raise issues in terms of their quality, and of the legal rights and work protections available to workers engaged in them. Statisticians in all OECD countries are grappling with the challenge of adapting their statistical standards and tools to measure the number and characteristics of these jobs.
Several international organisations have addressed this issue in the context of ongoing work on the ‘future of work’, digitalisation and job quality. At the OECD, the Committee on Statistics and Statistical Policy discussed in 2018 how National Statistical Offices are addressing a growing policy demand for better statistics in this field, the need for new statistical definitions, the risk that labour force surveys may undercount the number of people involved in these jobs, and the need to leverage additional sources of information.1 Furthermore, in the context of the OECD Future of Work Initiative, the OECD Directorates on Employment, Labour and Social Affairs and on Science, Technology and Innovation assessed measurement options and provided first recommendations on these issues.2
At the European Commission, Eurostat established in 2019 a LAMAS Task Force on measuring digital platform employment (TF DPE), which provided an opportunity for collaboration in a field that is of great interest to both European and non-European countries. Eurostat’s Task Force has developed a pilot survey that was implemented in the EU Labour Force Survey in 2022, with the long-term goal to regularly produce data on digital platform employment and work – which will finally depend on the results of the pilot survey. Partial results have been presented at the last LAMAS meeting in December 2022, but at this stage Eurostat is still expecting from countries the entire dataset by the end of March 2023. Based on the final results, Eurostat will decide on the most adequate follow-up. In particular, Eurostat may organise a Conference on the topic that will address the policy demands at EU and national levels, from both other stakeholders and international organisations, for reliable statistical information on the digital platform economy, with a focus on the labor market and on how the statistical community could answer these demands. In addition, the Joint Research Centre of the European Commission (JRC) has implemented several waves of the COLLEEM survey, an instrument designed to better understand the working conditions of platform workers. JRC has published a methodological paper on the measurement of platform work3 as well as the results from the second wave of the survey.4 Last but not least, on the policy front, the European Commission released a Directive on “Improving the working conditions of people working through digital labour platforms” in December 2021, which emphasised the issues of the employment status, algorithmic management, enforcement, transparency and traceability.
Measuring work mediated by online platforms is also of great importance in the context of the Resolution concerning statistics on work relationships adopted by the 20th International Conference of Labour Statisticians (ICLS) in October 2018. The resolution includes a new international classification of status in employment (ICSE-18). The ILO serves as secretariat of the ICLS and is engaged in the follow-up and promotion of implementation of the new international statistical standards. This Resolution provides a new classification of status in employment that includes the new category of “dependent contractors”, and a broader classification of status at work. 5The Resolution included a commitment to undertake further development work on the measurement of workers whose employment is intermediated through Internet-based platforms or apps. The ILO undertook a stock-taking of recent work on Digital labour platforms and the future of work in a report published in 2018, and explored the impact of digital labour platforms on enterprises, workers and society in the World Employment and Social Outlook 2021: The role of digital labour platforms in transforming the world of work. The ILO, in collaboration with other partners, is currently preparing a report to be discussed at the upcoming 21st International Conference of Labour Statisticians (ICLS) in 2023. The report will: a) discuss, based on the work done so far by countries and agencies, both the conceptual and operational issues related to measurement of the topic; b) identify key issues requiring further work; and c) propose approaches to further develop statistics in this field.
In this context, the OECD, ILO and the European Commission (represented by EUROSTAT, DG EMPL JRC, and EUROFOUND) agreed to strengthen their collaboration in the field of measuring platform work and employment through the creation of a Technical Expert Group (TEG). The goal of the group, gathering a number of statisticians and analysts, was to prepare a set of recommendations for measuring platform employment and work, in view of producing a Handbook. A similar set-up has been used by the OECD for developing measurement recommendations in ‘new’ fields that currently lack a solid foundation within the statistical system.6 In particular, the recommendations of this Handbook are broadly aligned with those from the Handbook on Forms of Employment (UNECE, 2022) when it comes to measuring platform employment and work, and several members of the TEG also contributed to the UNECE Handbook. In the future, the work of the TEG will feed into the work envisaged by the EU LAMAS Task Force and by other (non-European) NSOs participating in the Technical Expert Group.7
Finally, the Handbook on Measuring Digital Platform Employment and Work was presented to the Committee on Statistics and Statistical Policy (CSSP), which provided feedback at the 22-23 June 2022 meetings in Geneva. Final comments from CSSP, ILO, the EU and several National Statistical Offices were inserted afterwards.
Key insights from the Handbook
The Handbook is structured into four Chapters. Chapter 2 lists the key policy issues raised by digital platform employment and work that require more reliable data on the prevalence, characteristics and working conditions of digital platform workers. Chapter 3 identifies the specific features that define digital platform employment and work among the constellation of platform activities, in view of harmonising statistical practices across surveys and countries. Chapter 4 reviews country experiences in measuring, highlighting best practices and drawing lessons from less successful attempts. Chapter 5 seeks to operationalise the lessons learned from past practices and provides statistical recommendations.
Chapter 2: Why measure digital platform employment and work?
This Chapter provides the policy motivations for building statistical guidelines on digital platform employment and work. First, it highlights the growing international demand coming from a range of different policy perspectives to measure platform employment and work, the number of workers involved, their individual and job characteristics, and their working conditions. Second, it makes the case that correctly classifying platform workers within the framework of existing labour laws requires statistical standards that can improve the enforcement of different laws and regulations impacting on digital platform work (labour law, social security and taxation). Third, it argues that the lack of data affects the quality of existing and forthcoming policies in the areas of labour market regulations, social protection and social dialogue.
Overall, Chapter 2 concludes that the paucity of information about the prevalence of platform work and the characteristics of the individuals engaged in it, risks hindering the development of adequate policies. While existing labour force surveys and household surveys provide valuable information on self-employment, fixed-term and part-time work, they have not succeeded in identifying platform workers appropriately. While other types of statistical sources (such as the COLLEEM survey) as well as data directly provided by platforms (e.g. transaction records such as those submitted by digital platforms to tax authorities) can provide much needed information, official statistics have a very important role to play in providing the evidence needed to address the broad range of policy questions described above. At the same time, policy action can improve availability of data on platform work, for example by encouraging data sharing by platform companies.
Chapter 3: Conceptual framework, concepts and definitions
Chapter 3 proposes a conceptual framework that helps understanding the nature of digital platform employment and work,8 while embracing the diversity of the online activities and of the platforms’ attributes. The key objective of this framework is to provide a general and internationally agreed terminology as well as standard definitions of digital platform work and related concepts. The Chapter describes the statistical framework of the digital economy from an economic viewpoint and provides a definition of digital platforms that is most appropriate for identifying digital platform work, as well as a general definition of digital platform work.
Based on the definition of work provided by Resolution I of the 19th International Conference of Labour Statisticians (ICLS), digital platform work is defined as:
any productive activity performed by persons to produce goods or provide services carried out through or on a digital platform, AND:
- the digital platform or a phone app controls and/or organizes essential aspects of the activities, such as the access to clients, the evaluation of the activities carried out, the tools needed for conducting the work, the facilitation of payments, distribution and prioritization of the work to be conducted; and
- the work is for at least one hour in the reference period.
First, this definition is broad and includes different forms of digital platform work. Building on the 19th ICSL Resolution, these include: i) digital platform work for own-use; ii) digital platform employment; iii) digital platform unpaid trainee work; iv) digital platform volunteer work; and v) other work activities carried out on or through a digital platform. As such, the definition recognizes that digital platform employment is only one out of many forms of work that can take place on or through a digital platform.
Second, this definition emphasises the notion of control and organisation by the platform, which is essential to disentangle digital platform work and non-DPE work taking place via a platform. For example, a customer and a service provider exchanging via Teams or Zoom does not constitute digital platform work, as these two communication platforms do not offer integral services like ratings of participants, payments and matching of the two parties. Conversely, the classification as digital platform work is straightforward with platforms offering ratings of participants, payment services and algorithmic matching, such as Uber or Upwork. In between those two examples, classification can be difficult when a platform displays some but not all of the usual attributes of a digital platform. For instance, the French platform Doctolib allows patients to make appointments with doctors (hence completes a matching based on location and availability), or to organise a video consultation for which an online payment can be made; on the other hand, this platform does not rate doctors or patients, nor does it realise payments for physical consultations. In the case of Doctolib and other ambiguous situations, the classification as DPE work depends on the objectives of the statistical analysis and on the exact specification of its scope.
Finally, Chapter 3 focuses on digital platform employment and outlines a flexible framework that lays the foundation of a comprehensive measurement of digital platform employment, and that provides the possibility to focus on one or more specific parts. This framework enables a decomposition of the broad concept of DPE along the dimensions of: i) the type of production carried out (goods or services); ii) the type of digital platforms; iii) and the type of status in employment category. This will allow countries to focus on the part of DPE that is of high policy concern while at the same time strengthening the transparency and harmonisation between countries and agencies working in this area.
Chapter 4: Critical review of existing statistical sources on digital platform employment
Chapter 4 aims to: i) review the main measurement initiatives on digital platform employment; ii) identify the lessons learnt from these initiatives; iii) understand the pros and cons of the various related statistical vehicles for answering different policy issues. The most important lessons learned from this chapter are listed below.
First, Labour Force Surveys (LFS) are best placed to give accurate and robust estimates on the overall prevalence of digital platform employment, although problems of sample size reduce their suitability for gaining insights on the characteristics of digital platform workers. Even though the sample sizes of LFS are typically large, they will nevertheless lack statistical precision when measuring the characteristics of potentially small groups in the population such as digital platform workers. This is all the more true for Information and Communication Technologies (ICT) Usage Surveys, which have a smaller sample size than LFS. Also, the nature of digital platform employment (task approach), as measured by ICT Usage Surveys, is not always compatible with the concepts underlying LFS.
Other sources (such as ad hoc surveys, household surveys covering different issues, administrative datasets or big data) provide a useful complement to LFS. For example, Time Use Surveys (TUS) have the advantage of capturing platform work done for short periods and as a secondary activity, but to date they have not included questions to investigate this topic; they also have the disadvantage of being conducted very unfrequently. Tax registers, or in general other administrative registers, can provide information from both the platforms (when it is possible to identify them as tax payers) and from the workers (when it is possible to identify them as DPE workers). Information on income is the main item that is best covered by this source. However, coverage of tax registers is limited to formal enterprises and workers covered by a regular work contract, which is an important limitation in countries where informality is common. Moreover, when low paid workers are exempted from tax declaration and/or payment, they are also excluded from the reference population of tax registers.9 More generally, the reference population of tax registers and other administrative data is affected by national legislation that usually has no statistical purpose, limiting the representativeness of the source and the cross-national comparability of measures based on them. At present, the possibilities of using administrative data are limited, although these may increase as tax authorities develop data-sharing agreements with digital platforms.
Finally, the use of online surveys can reduce costs (though possibly at the expense of reduced accuracy and higher sampling and selection bias), allowing researchers to reach out to a larger number of respondents. Such online surveys, typically undertaken by agencies that are not part of countries’ official statistical systems, can complement official surveys, which can be used to test the overall accuracy of other approaches and to calibrate their results. Overall, these various survey vehicles serve different purposes, with each of them having its own strengths and weaknesses. The choice of method depends on the research objectives, the resources available, and the trade-offs faced by statistical agencies or researchers.
Chapter 5: Measurement recommendations
Chapter 5 describes in more details some important initiatives undertaken by different institutions that have set up various surveys on digital platform employment (DPE). The Chapter takes stock of these experiences to provide statistical recommendations. Relative to Chapter 4, it is more focused on questionnaire design and adopts an operational perspective. Different members of the Technical Expert Group have contributed to this chapter, bringing their experience on generating information on DPE from different sources. In order to harmonize their contribution, and to facilitate comparisons, a common template was provided to authors, and is used in the chapter to report on these initiatives.10 Statistical recommendations, both general and pertaining to each statistical vehicle, are provided below.
General recommendations
Definitions of Digital Platform Employment and work. Measuring the same concept of digital platform employment and work across national and international surveys is key for comparisons across countries and measurement tools. The definition provided in Chapter 3 provides the benchmark for the broad concept of “digital platform work” as well as its different conceptual components. With this as a starting point, producers of data will be able to create a greater degree of transparency and understanding of what is respectively included and excluded in a given measurement initiative, as described in Chapter 5. This will contribute to a more harmonized measurement and a better understanding among users.
Identification of Digital Platform Workers through surveys. Rather than relying on a single overarching question, short questions should be asked concerning different elements of digital platform employment and work, with the interviewer or subsequent analysis then determining whether respondents should be considered as belonging to this category or not.
Use of filter questions. Filter questions should be used to determine the nature of the tasks conducted, such as whether the service was provided online or delivered in person. These filter questions should clearly identify which tasks respondents are referring to when answering subsequent questions about the nature of the work or tasks performed.
Cognitive burden. For surveys, it is key to ensure that respondents understand the meaning of digital platform employment and work. To generate consistent statistics over time, survey respondents should have a similar understanding of questions in each period, and should not be confronted with overly long introductory text, which is likely to be ignored by respondents.
Number of jobs and frequency of DPE. For each data source ideally, it would be important to document the number of jobs exercised by persons carrying out digital platform employment and/or the frequency of DPE activities over the LFS reference period, and possibly over longer reference periods given the episodic nature of DPE for many workers.
Future steps and the role of governments. This Handbook is an important first step into a statistical field that will expand in the future. Its framework, definitions and recommendations will be improved as new evidence flows in. Given the fast evolution of DPE, the authors recommend to evaluate this Handbook in a couple of years, rather than in a decade. Moreover, governments can play a role for improving high quality statistical information. For example, tax legislation providing special regimes to DPE incomes helps identify the reference population, and some countries have introduced obligations for big data producers or for labour digital platforms to provide their data to statistical authorities. Tapping from administrative surveys will be important future development. Conversely, measures of digital platform employment should be independent of legislative changes that classify digital platform workers. Failing to ensure the independence between legislation and statistics would impact on both cross-sectional and time series estimates of DPE.
Recommendations specific to different tools
Labour Force Surveys
The statistical experiences based on LFSs highlight a range of differences in terms of whether questions on DPE are asked to all respondents or to a subset thereof (e.g. those classified as “employed”, or own-account workers or dependent contractors11); whether they include or exclude activities related to renting out capital goods (where labour services only have an auxiliary function); whether different typologies of digital platform employment should be distinguished (e.g. distinguishing between tasks done on location, delivered in person, and those done entirely on line); what reference period should be considered in order to classify workers as performing a digital platform activity (e.g. one week or one year or at least one week in a particular year); whether digital platforms should be limited to those that, in addition to matching workers and clients, also manage the payment between the two; whether specific digital platforms should be mentioned to guide respondents; and more. No single answer to all these questions exists, and the best approach will partly depend on the number of questions that can be asked (i.e. few, in the case of questions included in the general or core questionnaire of LFS; potentially more, in the case of ad hoc or recurrent LFSs modules). A number of LFS-specific recommendations are made in the chapter:
The LFS should be the tool of choice when it comes to measuring the number of people involved in digital platform employment. Other measurement tool should, to the extent possible, align their definition of digital platform employment to the one used by LFS. The starting point is the LFS definition of employment, and the DPE should be a subset of it.
People’s activities on digital platforms can take various forms: i) unpaid work; ii) employment mostly involving a return on labour; iii) activities mostly involving a return on capital but still considered as employment (e.g. renting capital goods); iv) other activities involving a return on capital but no employment (e.g. using an app to trade stocks). While some NSOs may narrow their questions to DPE, those opting for a broader remit should do so in ways that allow to clearly distinguishing between the various activities.
In addition to basic breakdowns of DPE by demographic characteristics and employment status included in LFS, LFS should ask respondents on the “regular” or “occasional” nature of their DPE relation, based on either the number of hours worked or the earnings gained, and whether tasks are delivered on-line or in-person.
While some NSOs may use a longer (12-month) reference period to identify workers who engage with digital platforms only occasionally, it is recommended to also include questions about digital employment in the survey reference week within the LFS questionnaire, so as to allow measuring the incidence of DPE among all employed people by LFS.
Digital platforms should be assessed on both criteria of intermediating between clients and service providers with some degree of control over the work, and managing the payment for the services provided. Surveys should implement these two criteria in ways that allow narrowing the focus on the overlapping area. In particular, platform practices may differ as on-line services tend to be paid on-line through the platform, while on-location services have a higher probability to be paid in person.
Naming of (well-known) digital platforms should be avoided when first asking questions on DPE, to avoid the respondent focusing on the provided names only, but could be used in follow-up questions for checking and validating respondent’s answers.
The status in employment should be always investigated, as the new ICSE-18 classification provides for additional categories between employee and own-account worker that well describe the situation of workers remunerated by executed tasks but organised as a dependent working relationship. It is worth stressing that national legislations also affect workers’ status in employment: these differ a lot across countries, and workers performing the same tasks can be classified as dependent employment or own-account work in different countries.
Business surveys
Business surveys can provide information on both demand and supply side of the employment relation. They may cover the platform themselves (although in this case it will be difficult to reach all the platforms operating in a country, especially those based abroad) or the enterprises acting as clients in the market (while excluding non-business clients). Particularly important is to clearly identify the reference population of the survey, which may not always cover the entire enterprise population, and ensure that samples are representative. To the extent possible, business surveys should:
Cover the universe of business units operating in a country, focusing on business’ demands for services provided by digital platforms.
Rely on a definition of digital platform that is closely aligned to that used by LFS, i.e. platforms that mediate between clients and providers in a broad sense, also including those who intermediate the payment for the labour services provided.
Provide information on the quantitative importance of digital platforms for business turnover, as well as questions on businesses’ satisfaction with their relationship to digital platforms.
Include common breakdowns of businesses (e.g. by size of their payroll, industry, annual turnover, ownership type) allowing to compare businesses’ reliance on digital platforms across different parts of the business community.
Finally, the International Standard Industrial Classification (ISIC) needs to be updated to add industry codes that can specifically capture platform companies.
Ad hoc surveys
When the required information focuses on particular qualitative features of the work, such as working conditions or workers satisfaction, an ad hoc survey is the best choice. They can be specifically designed to focus on small or very small phenomena, ensuring a representative sample. While they require a significant planning effort and dedicated resources, they provide information that is not feasible to collect with other sources, information which is often necessary to calibrate labour market policies focusing on relevant but small (or not yet big) phenomena. While raising some selection, sampling and measurement issues, ad hoc surveys such as COLLEEM have the advantage of typically covering a broad range of countries, hence providing comparable evidence on different aspects of DPE.
Ad hoc surveys on digital platform employment and work such as COLLEEM should be routinely implemented across countries, covering both quantitative and qualitative aspects of workers’ experiences with digital platforms.
Consideration should be given to including a small set of comparable questions on digital platform employment in non-official surveys on working conditions (e.g. EWCS, ISSP, etc.), while aligning concepts and definitions to those used in LFS.
Big data
Big data sources are in an early phase of investigation. They are promising but it is clear that they can just complement other sources. By their nature it is difficult to plan in advance their information coverage, this is more an output of the study than an input, since the information extraction and its own definition proceed in parallel. Moreover private ownership of the data makes them expensive and unreliable for official statistics. Also continuity in time of the availability of this kind of data cannot be ensured. This makes this source more feasible, at the moment, for one-shot investigations of one particular aspect of digital platform employment, always in conjunction with traditional sources.
Data sharing agreements and reporting obligations for big data producers and labour digital platforms would enhance the quality of statistical information
Data from digital platforms
Data compiled by platforms themselves or ‘platforms of platforms’ can be used as stand-alone sources or, preferably, to complement results from surveys and administrative records. While the availability of timely and granular data from digital platforms provide some advantages for data analysis and policy developments on the future of work in a foreseeable context of growing incidence of platform employment, there are several issues related to the nature of data compiled by platforms on gig jobs and gig workers, on digital platform employment and work and their representativeness, potential bias in the sample or data provided that need to be carefully assessed.
Facilitating data sharing agreements with digital platforms providing labour services is one way for deepening data analysis of digital platform employment.
Compelling digital platforms to share data with public authorities as envisaged by the European Commission’s proposed Directive on “improving working conditions in platform work” may significantly increase transparency and traceability of platform work and enhance enforcement.
Notes
← 1. The discussion was based on a paper on Measuring Platform and Other New Forms of Work [SDD/CSSP(2018)10] prepared by the OECD Directorate on Employment labour and Social Affairs.
← 2. OECD (2019), "Measuring platform mediated workers", OECD Digital Economy Papers, No. 282, OECD Publishing, Paris, https://doi.org/10.1787/170a14d9-en. These recommendations are reflected in Chapter 3 of the Handbook.
← 3. Pesole, A., Fernández-Macías, E., Urzí Brancati, C., Gómez Herrera, E. (2019), “How to quantify what is not seen? Two proposals for measuring platform work, European Commission”, Seville, JRC117168. https://ec.europa.eu/jrc/sites/jrcsh/files/jrc117168.pdf
← 4. Urzí Brancati, C., Pesole, A., Fernández-Macías, E (2019). New evidence on platform workers in Europe. Results from the second COLLEEM survey, EUR 29958, , ISBN 978-92-76-12949-3, doi:10.2760/459278, JRC118570.
← 5. “Dependent contractors” share some of the features of “dependent workers”, when assessed from the perspective of the locus of authority in the employment relation, and of “workers in employment from profit”, when assessed from the perspective of economic risks. (https://www.ilo.org/wcmsp5/groups/public/---dgreports/---stat/documents/meetingdocument/wcms_648693.pdf).
← 6. This includes the Guidelines of Measuring the Quality of the Working Environment, developped as part of the CSSP programme of work for 2016-17 and released by the OECD in November 2017.
← 7. The EU-ILO-OECD Technical Expert Group (TEG) convened several meetings with participants from international organisations (ILO, EUROSTAT, JRC, EUROFOUND, OECD) and national statistical offices over the past three years. The first meeting was held physically in Paris on 12-13 September 2019, and was followed by several other virtual meetings to discuss drafts of the various chapters. Each Chapter of the Handbook was under the responsibility of a leading author and received comments and contributions from other authors. The list of contributors and the composition of the Technical Expert Group is included in the acknowledgement page of the draft Handbook. At the OECD, the work of the TEG is supported by staff from the Centre on Well-being, Inclusion, Sustainability and Equal Opportunities; the Directorate on Employment, Labour and Social Affairs; and the Directorate on Science, Technology and Innovation.
← 8. The 19th ICLS Resolution, drawing on the recommendation of the SSF Commission, introduced in 2013 a distinction between different “forms of work”. In this classification, “work” comprises any activity performed by persons to produce goods or to provide services for use by others or for own use. Within this broad category, the Resolution distinguished between: a) “own-use production work” (comprising production of goods and services for own final use); b) “employment work” (comprising work performed for others in exchange for pay or profit); c) “unpaid trainee work” (comprising work performed for others without pay to acquire workplace experience or skills); d) “volunteer work” comprising non-compulsory work performed for others without pay; and e) “other work activities” (including activities such as unpaid community service, unpaid work by prisoners, and unpaid military or alternative civilian service).
← 9. As an alternative or complement to the use of tax registers as a source of information on the income of digital platform workers, such information could be gathered by including specific questions in official household income surveys.
← 10. TEG members that contributed to these descriptions of measurement initiatives, were asked to describe the: i) original purpose of the initiative (quantitative or qualitative analysis, information needs), identifying the question the exercise aimed to answer (e.g. ‘how many persons are involved in digital platform employment’? ‘What are their characteristics and working conditions’?); ii) reference population and sampling; iii) other relevant survey features such as reference periods, data collection mode and methodological choices; iv) implied operational definition of DPE, covering the general and operational (often only implicit) definitions of the concepts analysed as well as their practical implications (e.g. which platforms are excluded? does the operational definition include goods and/or services? is it restricted to a specific employment status, for example own-account workers only or including employees or excluding volunteers?); and v) Goals and lessons learned, including a list of ‘do’ and ‘do not’ for the future.
← 11. Depending on the implementation in survey instruments of the ICSE-18 resolution adopted by the 20th International Conference of Labour Statisticians (ICLS) concerning statistics on work relationships, revising the International Classification of Status in Employment adopted in 1993 (ICSE-93).