Guidance Document on Good In Vitro Method Practices (GIVIMP)

Report

OECD Series on Testing and Assessment

10 December 2018

Chapter 9. Reporting of results

Abstract

Key message: Good reporting of in vitro methods can only be achieved when all important details are recorded in a way that allows others to reproduce the work or reconstruct fully the in vitro method study.

Key content: Guidance is given on publishing and reporting of in vitro method studies and on data reporting for regulatory purposes.

Guidance for improved practice: Examples and available sources for scientific data management are detailed to promote more transparency and openness from scientists to avoid issues related to reproducibility of data but also to stimulate electronic data sharing for a variety of research and safety assessment purposes.

Recommendations are given to not only publish or make available the in vitro method results but also all the related documents and the changes that have been introduced to improve the method and the rational for them.

In vitro methods must be fully documented following good recording and reporting practices, and must contain all pertinent details to allow subsequent and adequate analysis and reporting of results. For example, batch/lot numbers, catalogue numbers, supplier details, and expiry dates for chemicals and reagents must be listed for critical reagents, as well as temperatures and times (e.g., storage of chemicals, incubation steps in the in vitro method), specific identification of critical equipment used and, perhaps most importantly, any deviations (unintended variations) from Standard Operating Procedures (SOPs). All this information must be directly and accurately recorded, signed and dated by the person performing the activity, as these recordings are important for the correct interpretation of the results and reconstruction of the study. Where technical activities (e.g., aseptic work) preclude that person from recording the data themselves, the use of a second person to record the data may be employed. In these cases both the person performing the activity and the person recording the data must be identified in the study data.

Experimental details and results should be easily retrievable; a log page at the front of a notebook may help tracking recordings and observations. Any reference to computer files containing data should also be catalogued in the notebook. Data files should always be backed-up in case of computer failure, corruption, or deletion.

Reporting requirements depend on the different development phases of the in vitro method. For regulatory use, requirements for reporting are described in the Good Laboratory Practice (GLP) Principles. Reporting adequate information and results of all developmental phases will increase the confidence in the in vitro method and would allow for general acceptability by receiving authorities.

9.1. Publishing

There is an increasing tendency towards more transparency when publishing work which may lead to better reproducibility of published data, as described in the Guidelines for Transparency and Openness Promotion (TOP) Open Science Framework1. The EU Competitiveness Council has also announced as their target that all scientific publications resulting from publicly funded research should be publicly available by 20202. It is also good practice to publish scientific results in a timely manner. The results will be used and re-used by other scientists, competitors, modellers or validation study statisticians. Moreover, for any systemic endpoint the prediction is/will be based on the results of many different studies, using different methods performed in different facilities, e.g. studies using in vitro methods for the identification of modulators of the thyroid hormone signalling pathway3.

Sharing of data in public repositories is also being encouraged and best principles regarding the publication of scientific data have also been addressed by others, such as the FAIR (Findable, Accessible, Interoperable, and Reusable) Guiding Principles for scientific data management and stewardship, by the Nature Publishing Group4. This initiative not only promotes more transparency and openness but also promotes the use of computer readable datasets and data mining so that computers have the ability to access the data autonomously, unaided by their human operators, which is core to the FAIR Principles.

Data sharing is encouraged by default, unless there is reason for confidentiality, using public data-sharing standards and repositories such as ISA-TAB5, Dryad Digital Repository6, Figshare7, and Nature Scientific Data8. It is recommended to not only publish the results, but also the method/SOP, again using on-line repositories such as Nature Protocols9, the Journal of Visualized Experiments (JoVE)10, Testing method Exchange, Springer Protocols11, EURL ECVAM Database service on Alternative Methods (DB-ALM)12 and JRC-QSAR DB13. In the same vein, in vitro method modifications and further developments should be published. Such publications should include the changes leading to improvement, the rationale for them, and this should also entail information on which changes reduce in vitro method performance, or that do not result in an improvement.

In addition to the increasing openness and transparency, the publication of negative results is also gaining more ground e.g., the Journal of Negative Results in BioMedicine14 is an open access, peer reviewed journal that provides a platform for the publication and discussion of non-confirmatory and "negative" data.

9.2. Mutual Acceptance of Data (MAD)

To avoid costly duplication in safety testing and government assessments, the OECD developed a framework for sharing data called the MAD system. MAD is a multilateral agreement which allows participating countries (including member states and MAD-adhering economies) to share the results of various non-clinical tests acquired when applying OECD methods and principles. As such it provides governments with confidence that non-clinical in vitro method data, generated under the MAD system, can be used in regulatory assessments. The use of this data by the Receiving Authority (Section 1.9) may differ depending on the scope of the specific test guideline, i.e., some in vitro methods may be full replacement, partial replacement, part of a defined approach or only used for screening purposes/priority setting. Results derived from non-standard in vitro methods and non-testing methods may also be reported, but as supporting information. Other benefits of the MAD system include reduction in animal testing, the evaluation of more chemicals and broader availability and transparency of government-vetted, high quality information and data.

9.3. Integrated Approaches to Testing and Assessment (IATA)

The current regulatory toxicity testing and assessment approach has evolved over the past half century, however it is unlikely to efficiently meet legislative mandates that require increased numbers of chemical assessments to be undertaken without a concomitant increase in the use of animals and resources. Therefore, new approaches are necessary to close the gap between the number of chemicals in use and the number assessed to date.

IATA15 are pragmatic, science-based approaches for chemical hazard characterisation that integrates and weighs all relevant existing evidence and guides the targeted generation of new data, where required, to build up a hazard or risk assessment acceptable in regulatory decision-making. The information provided by individual in vitro methods, as well as in silico predictions, can be combined, interpreted and used for regulatory decision making by means of an IATA (OECD, 2017[1]). Ideally, an IATA should be informed by mechanistic understanding of the underlying toxicokinetics and toxicodynamics. A framework for capturing the toxicodynamic information is provided by Adverse Outcome Pathways (AOP)16. IATA and AOP knowledge, if properly captured and presented, leads to a better understanding of toxicity mechanisms, and ultimately the AOP knowledge derived from testing several chemicals may be extrapolated to predict the toxicity of all chemicals that trigger the same Molecular Initiating Event (MIE) or Key Event (KE).

Structured integration of different data types can be performed at different levels, including raw data and summarised level data (OECD, 2016[2]). Different levels of data integration can then be used including Boolean combinations of categorised results, scoring approaches, decision trees, deterministic and probabilistic approaches. As experience is gained, approaches to data integration can become standardised. Such approaches, called Defined Approaches (DAs), can thus become core elements of IATA. A DA is a formalised decision-making approach consisting of a fixed data interpretation procedure used to interpret data from a defined set of information elements (OECD, 2017[1]).

In contrast to IATA, DAs can be standardised and could therefore fall under MAD. The OECD is working on the development of a PBTG on DAs for skin sensitisation. The project, co-lead by the EC, US and Canada, aims at developing international standards that would give to DAs equal regulatory status as the current animal tests, i.e., prediction generated with valid defined approaches would fall under the OECD mutual acceptance of data program (Casati et al., 2017[3]).

It is essential to have all the results reported in a uniform manner to facilitate their use in the IATA framework, where the same dataset can be used in many different ways. The OECD GD 255 on reporting of DAs to be used within IATA provides a set of principles for reporting DAs to testing and assessment to facilitate their evaluation. Templates, for reporting individual and multiple information sources, are also available to provide consistent reporting which will ultimately facilitate the evaluation of IATA and DAs in regulatory decision-making within OECD Member Countries (OECD, 2017[1]).

9.4. Data reporting for regulatory purposes

Data and derived results from GLP studies will play an important role in increasing the relevance of in vitro data in regulatory contexts. Consideration and ultimately acceptance of in vitro GLP data can be promoted by using a standardised data format. This is facilitated by the use of IUCLID17 (International Uniform ChemicaL Information Database), a software application used to record, store, maintain and exchange data on intrinsic and hazard properties of chemical substances.

The OECD had already designed and published several OECD Harmonised Templates (OHTs)18 to report test results concerning:

1. physical/chemical properties (e.g., boiling point, density, flammability, etc.)

2. human toxicity (e.g., carcinogenicity, acute toxicity, etc.)

3. environmental toxicity (e.g., aquatic toxicity, terrestrial toxicity, etc.)

4. other properties describing degradation, accumulation etc.

These templates are geared towards results derived from classical (mostly OECD guideline) studies, focusing on apical endpoints, i.e., Adverse Outcomes (AOs).

However, reporting MIEs or KEs with such a classical OHT would tie them inseparably to the one AO the template covers, which is undesirable, as the in vitro/in silico/mechanistic information is then not easily accessible for building AOPs leading to other AOs: A Key Event can be relevant not only for one AOP, but several. Reporting the Intermediate Effect in an "AO-neutral" template makes the data available for all kinds of AOPs. A new, AO-neutral OHT was therefore needed that would allow reporting observations from mechanistic (in vitro and in silico) tests, without immediately locking into one of several AOs the Intermediate Effect could lead to.

Knowing not only about results of animal tests (classical OHTs), but being able to cross-reference these test results with the intermediate effect observations (new OHT) has the potential to lead the way towards a less animal-centred hazard assessment. The OECD therefore started an initiative to come up with a stable, stakeholder-endorsed OHT for reporting on "intermediate effects" being observed via in vitro assays and possibly other non-animal test methods (computational predictions etc.). The template, OHT 201 - Intermediate effects, was endorsed by the OECD Joint Meeting in 2015 and was finally published in August 201619. When submitting in vitro data to a receiving authority, the use of the OHT 201 is encouraged but is not yet obligatory.

The basic principle of OHT 201 is that:

1. one or several objective observation(s) (= results from non-classical test methods)

2. lead(s) to one subjective conclusion (= Intermediate Effect present, yes or no).

A properly filled in OHT 201 template therefore conveys a clear statement:

1. Based on observations O₁, O₂, …O_n

2. a certain chemical

3. triggers/does not trigger

4. a certain intermediate effect

5. on a certain biological level

6. at a certain effect concentration.

With OHT 201 being implemented in IUCLID20, a software used by industry to fulfil reporting obligations under more and more legislative programmes (e.g., REACH), the concept of Intermediate Effects (and implicitly AOPs and predictive toxicology) has started to get attention in the regulatory world. This is a first step towards the acceptance of results from alternative tests for regulatory purposes, with the ultimate goal of replacing in vivo centred AO observations with alternative-methods-centred IATA/AOP considerations as the basis for risk assessment.

In the US if a chemical is not on the TSCA Chemical Substances Control Inventory21, the substance is considered a "new chemical substance" while those already registered are considered as "existing chemical substances". Section 5 of TSCA requires anyone who plans to manufacture (including import) a new chemical substance for a non-exempt commercial purpose to notify the US EPA before initiating the activity. A pre-manufacture notice or PMN (a sample PMN form is available on the US EPA website22), must be submitted at least 90 days prior to the manufacture of the chemical.

For in vitro methods without a guideline, the Office of Pesticide Programs US EPA recommends following OECD Guidance Document 211 (OECD, 2017[4]) for describing non-guideline in vitro methods (EPA, 2016[5]).

9.5. Reporting of method validation

Validation is at the interface between in vitro method development/optimisation and regulatory acceptance/international recognition and ensures a science-based and conscientious evaluation of in vitro methods and approaches (e.g., Integrated Testing Strategies (ITS) or DAs), independent of specific interests, establishing their overall performance and fitness for a given purpose, i.e., their scientific validity23.

The approach taken by a validation body may vary according to the needs of that body, e.g., whether they will coordinate the validation study or whether a validation study may be submitted to that body for assessment. In general an independent peer review of the validation study data and in vitro method is required, usually by organisations that specialise in in vitro method evaluations, such as JaCVAM24, EURL ECVAM25 or ICCVAM26.

OECD published criteria should be met prior to seeking regulatory acceptance e.g., test method and validation study data should have been subjected to a transparent and independent peer review process, the generated data must be useful for hazard/risk assessment purposes, the submitted test method and data should adequately cover a spectrum of chemicals and products representative of those overseen by the receiving authority for which the method is proposed, the applicability and limitations of the test method should be clearly described and the test method should be time and cost effective and likely to be used in a regulatory context (OECD, 2005[6]). It is preferred that validation studies are performed and reported in accordance with the OECD Principles of GLP (OECD, 1998[7]). This will depend, however, on whether validation studies are part of the individual MA's inspection programme, as consensus has not been reached on this topic.

Submission of a new test method considered ready for proposal as an OECD Test Guideline is done via the OECD Secretariat either through a member country or through its National Co-ordinator; through the European Commission (EC) (EU only); an industry association through the Business and Industry Advisory Committee (BIAC) to the OECD; invited experts via a National Co-ordinator.

References

[3] Casati, S. et al. (2017), “Standardisation of defined approaches for skin sensitisation testing to support regulatory use and international adoption: position of the International Cooperation on Alternative Test Methods”, Archives of Toxicology, Vol. 92/2, pp. 611-617, http://dx.doi.org/10.1007/s00204-017-2097-4.

[5] EPA (2016), Process for Evaluating & Implementing Alternative Approaches To Traditional in Vivo Acute Toxicity Studies for Fifra Regulatory Use, U.S. Environmental Protection Agency.

[4] OECD (2017), Guidance Document for Describing Non-Guideline In Vitro Test Methods, OECD Series on Testing and Assessment, No. 211, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264274730-en.

[1] OECD (2017), Guidance Document on the Reporting of Defined Approaches to be Used Within Integrated Approaches to Testing and Assessment, OECD Series on Testing and Assessment, No. 255, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264274822-en.

[2] OECD (2016), Guidance Document for the Use of Adverse Outcome Pathways in Developing IATA for Skin Sensitisation, OECD Series on Testing and Assessment, No. 260, OECD Publishing, Paris.

[6] OECD (2005), Guidance Document on the Validation and International Acceptance of New or Updated Test Methods for Hazard Assessment, OECD Series on Testing and Assessment, No. 34, OECD Publishing, Paris.

[7] OECD (1998), OECD Principles on Good Laboratory Practice, OECD Series on Principles of Good Laboratory Practice and Compliance Monitoring, No. 1, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264078536-en.