Envisioning an international validation process for New Approach Methodologies in chemical hazard and risk assessment

Chia sẻ: _ _ | Ngày: | Loại File: PDF | Số trang:13

Thêm vào BST

Báo xấu

20
lượt xem 2
download

Download Vui lòng tải xuống để xem tài liệu đầy đủ

To build trust across many sites (laboratories, regulatory agencies, contract research organizations, chemical producers, and the public), stakeholders will need to agree on validation requirements for particular uses (content in relation to context) as well as how results are to be communicated (data format), and measured (metrics). There is also a need for a global orchestrator, who can exert leadership and inspire voluntary cooperation of diverse organizations to address shared validation goals, to play a key role.

Chủ đề:

Bình luận(0) Đăng nhập để gửi bình luận!

Lưu

Nội dung Text: Envisioning an international validation process for New Approach Methodologies in chemical hazard and risk assessment

Environmental Advances 4 (2021) 100061 Contents lists available at ScienceDirect Environmental Advances journal homepage: www.elsevier.com/locate/envadv Envisioning an international validation process for New Approach Methodologies in chemical hazard and risk assessment Matthieu Mondou a, Steve Maguire b,c, Guillaume Pain a,d, Doug Crump e, Markus Hecker f, Niladri Basu a, Gordon M. Hickey a,∗ a Department of Natural Resource Sciences, Faculty of Agricultural and Environmental Sciences, McGill University, Montreal, Quebec, Canada b University of Sydney Business School & University of Sydney Nano Institute (Sydney Nano), The University of Sydney, New South Wales, Australia c Department of Chemistry, Faculty of Science, McGill University, Montreal, Quebec, Canada d Faculté des sciences de l’administration, Université Laval, Sainte-Foy, Québec, Canada e National Wildlife Research Center, Environment and Climate Change Canada, Ottawa, Ontario, Canada f Toxicology Center, University of Saskatchewan, Saskatoon, Saskatchewan, Canada a r t i c l e i n f o a b s t r a c t Keywords: Despite the global toxicology community discussing New Approach Methodologies (NAMs) for chemical haz- Regulatory science ard and risk assessment, such as in vitro, in silico, and ‘omics-based approaches, for some 30 years, their for- Innovation mal adoption by regulators remains limited. Previous research suggests that insuﬃcient validation, complexity Ecotoxicology of interpretation, and lack of standardization are salient obstacles to adoption. In this paper we aim to better Pollution understand the policy challenges associated with adopting NAMs in chemical risk assessment; and to identify Governance Risk Assessment and Management actions that could facilitate and accelerate their formal adoption internationally. We conducted a Delphi study – a group communication process that solicits expert judgments through iterative questioning and feedback – with panelists from government, industry, and non-governmental organizations in Europe and North America. Expert panelists identiﬁed two key activities to facilitate and accelerate the validation of NAMs internationally: 1) the development of common data collection, reporting and sharing procedures; and 2) the improvement of knowledge about new test methods among members of the regulatory community. Both activities suggest the need for a common regulatory science infrastructure, including international regulatory dialogues, large-scale research collaborations, and coordinated innovation in technological tools, the discourse of scientiﬁc validation, and regulatory procedures. To build trust across many sites (laboratories, regulatory agencies, contract research organizations, chemical producers, and the public), stakeholders will need to agree on validation requirements for particular uses (content in relation to context) as well as how results are to be communicated (data format), and measured (metrics). There is also a need for a global orchestrator, who can exert leadership and inspire voluntary cooperation of diverse organizations to address shared validation goals, to play a key role. 1. Introduction methods proceed, there is a recognized need to accelerate chemical risk assessment (Krewski et al. 2010; Basu et al. 2019: 279). NAMs have Making greater use of NAMs in chemical risk assessment promises subsequently been identiﬁed as a potential way forward (Zeiger 1999; to overcome several of the current limitations associated with conven- Iannaccone 2001; Bradbury et al. 2004; Kavlock et al. 2018); how- tional toxicity testing (Basu et al. 2019). Social and political pressures ever, questions concerning their formal adoption in chemical risk for more eﬀective, eﬃcient and ethical regulatory risk assessment prac- assessment remain, with insuﬃcient validation, complexity of inter- tices are increasing internationally, driven in part by the signiﬁcant pretation, and lack of standardization identiﬁed as salient obstacles backlog of unassessed and “data-poor” chemicals currently in, or en- (Mondou et al. 2020; Pain et al. 2020). tering, the environment. For example, the United States Environmen- tal Protection Agency (US EPA) has estimated that using whole ani- 1.1. Validation of test methods mal testing for a single chemical can take four years and cost between $1M and $20M USD (Martin et al. 2012). Given current legislative man- The validity of a test method refers to both its reliability and rel- dates in North America and Europe and the pace at which conventional evance with regard to a speciﬁc use (Organisation for Economic Co- operation and Development (OECD) Environment Directorate 2005). ∗ Corresponding author. E-mail address: gordon.hickey@mcgill.ca (G.M. Hickey). https://doi.org/10.1016/j.envadv.2021.100061 Received 26 February 2021; Received in revised form 22 April 2021; Accepted 23 April 2021 2666-7657/© 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Here, ‘reliability’ represents an objective measure of intra- and inter- survey data from toxicology professionals suggests that limited regula- laboratory reproducibility and refers to the ability of a method to pro- tory acceptance and complex data interpretation undermine the adop- duce consistent and reproducible results that allow toxicity measures or tion and use of NAMs (Vachon et al. 2017; Zaunbrecher et al. 2017). predictions. ‘Relevance’, on the other hand, refers to the usefulness of Previous studies have highlighted that issues relating to the organiza- these results for a particular purpose based on existing scientiﬁc infor- tional and research landscape need to be addressed in order to acceler- mation (Bruner et al. 1998: 478). Validation eﬀorts in toxicology notably ate the shift towards greater use of NAMs in regulatory risk assessment involve examining the reproducibility of results across laboratories and (Environment and Climate Change Canada 2016; Busquet and Hartung at diﬀerent points in time (Eskes and Whelan 2016: 3). 2017; ICCVAM 2018; Standing Committee on Emerging Science for En- Attention to issues associated with validation in toxicology has been vironmental Health Decisions et al. 2018; Tickner et al. 2018; US EPA increasing since the early 1990s. The European Center for Validation 2018). With these considerations in mind, international initiatives to of Alternative Methods (ECVAM) was founded in 1991, followed by a rethink toxicity testing are multiplying (Carrie 2015; ECHA 2016; precursor to the US Interagency Coordinating Committee on the Valida- ICCVAM 2018; Kavlock et al. 2018). Alternative frameworks in which tion of Alternative Methods (ICCVAM) in 1994. Both organizations have NAMs could be leveraged for chemicals management are being devel- a mandate to develop validation processes and to advance the accep- oped and deployed, such as Integrated Approaches to Testing and As- tance of alternative testing by national and international stakeholders. sessment (IATAs), Adverse Outcome Pathways (AOPs), and ecological The OECD has also been active in convening international discussions Thresholds of Toxicological Concern (eco-TTCs). In addition, more ﬂex- on validation issues. For example, “in 1991, OECD decided to take a ible frameworks for validation have been proposed, such as ‘ﬁt-for- more active role in encouraging non-animal testing by bringing mem- purpose’ validation (Casey 2015) or ‘deﬁned approaches’ to validation bers together to negotiate, discuss, and arrive at a consensus.” (Herman (Hartung et al. 2004) (see Table 1 for a summary). Koëter quoted in Clay 1996: 613). These policy eﬀorts resulted in the The global regulatory ecotoxicology community is struggling to ﬁnd development of criteria and procedures with government, industry and the ‘right’ way to leverage the desirable properties of NAMs and to in- academic stakeholders to validate and facilitate the adoption of new corporate them into their daily work routines. Recognizing the multiple testing methods in Europe and the USA (Balls 1995; Balls et al. 1995; frameworks available to assist with this process, and the relatively slow NIEHS 1997). International harmonization discussions under the aus- and limited uptake of NAMs to date, there remains a need to better un- pices of the OECD have since led to broader agreement on principles derstand how the international validation process for NAMs might be and criteria for the validation and acceptance of new test methods enhanced. Building on previous survey, interview and documentary re- (OECD 1996; see also Zeiger 1999). Bruner et al. (1998) summarized search, in this paper we employ a novel futuring exercise to elicit the these eﬀorts to design a consensual validation process, stressing the im- opinions of world-leading NAM experts on how NAMs can be validated portance of predictive models for successful validation. These authors at the international level. emphasized that prediction models enable expectations concerning the Our goal in this research is to better understand the future scenario performance of a test to be ﬁxed before validation starts, thereby al- options for validating the use of NAMs. More speciﬁcally, we sought to lowing the performance standard required for validation to be set in examine the challenges associated with validating NAMs; and to identify advance.1 pragmatic actions that could facilitate and accelerate their validation Progress towards validating NAMs has continued, with numerous and adoption internationally. The overarching research questions are: national agencies established to facilitate international discussions, in- 1) What is the process through which NAMs can become standardized, cluding the Japanese Center for the Validation of Alternative Methods taken-for-granted tools in the ‘toolbox’ used to assess the health and (JaCVAM) in 2005, the South Korean Center for the Validation of Al- environmental risks of chemicals?; 2) How can new NAM-based prac- ternative Methods (KoCVAM) in 2010, the Brazilian Centre for the Val- tices have the most signiﬁcant impact on regulatory risk assessment? idation of Alternative Methods (BraCVAM) in 2011 and the Canadian Drawing on the particular regulatory experiences and perspectives of Center for Alternatives to Animal Methods (CCAAM) in 2017. The 2007 NAM experts, we aimed to generate a rich contemporary understand- report Toxicity Testing in the 21st Century: A Vision and a Strategy by ing of future possibilities to inform initiatives to establish standards and the US National Research Council pushed for the adoption of new prac- guidelines for NAMs. tices, in the context of rising interest in the potential for microarray data and ‘omics based approaches to enhance chemical risk assessment 2. Methods (Corvi et al. 2006; National Research Council 2007). Other guidance documents addressing speciﬁc NAMs have also been adopted at na- In order to forecast how the international validation process tional (see, for example, the US EPA’s New Approach Methods Work for NAMs might be enhanced, we employed the Delphi method Plan, 2020), and international levels, including the OECD Users’ Hand- (Linstone and Turoﬀ 2002; Keeney et al. 2010). The Delphi method book Supplement to the Guidance Document for Developing and As- engages experts in an iterative group communication process that so- sessing AOPs (OECD, 2018), and OECD Test Guidelines No. 442E: In licits judgments through sequential questioning combined with sum- Vitro Skin Sensitisation assays addressing the Key Event on activation maries and feedback from previous responses to facilitate reﬂexivity. of dendritic cells on the Adverse Outcome Pathway for Skin Sensiti- We adopted the Policy Delphi variant for our research using a virtual sation (OECD 2017a) and No. 236: Zebraﬁsh embryo toxicity test for and anonymous interaction environment. This approach helped inte- acute aquatic toxicity (Busquet et al. 2014). Tracking systems have also grate the knowledge, values and experiences of a select group of in- been put in place to help identify alternative testing options, as well as ternational experts on regulatory chemical risk assessment in order to their current validity and acceptance status (for example the NTP Inter- advance social and organizational learning, map consensus and dissent, agency Center for the Evaluation of Alternative Toxicological Methods while minimizing social group dynamics during deliberation (Klenk and [NICEATM], USA, and the European Union Reference Laboratory for Hickey, 2011). alternatives to animal testing [EURL ECVAM]). The Delphi method has been previously used in multiple policy Despite progress, the global toxicology community continues to puz- domains to enable forecasting initiatives designed to inform strat- zle over the best way to leverage NAMs in their daily workﬂows. Inter- egy (e.g. Klenk and Hickey 2012; Plummer and Armitage 2007; national validation eﬀorts for NAMs are progressing slowly, and recent Wentholt et al. 2009; Groves 2013; Markmann et al. 2013; Basu et al. 2015; Cazabon et al. 2017). It has also been used in environ- mental health research, often to generate indicators, models or in- 1 For further review of the history of alternative methods see Eskes and Whe- dex components, or to determine weights and estimates of variables lan 2016; Mondou 2018; Balls et al. 2019 in composite metrics (e.g. Landrigan et al. 2002; Schulz et al. 2008; 2
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Table 1 Frameworks designed to facilitate the adoption of NAMs in regulatory chemical risk assessment. Adverse Outcome Pathways framework (AOPs) According to OECD: "An AOP describes a sequence of events commencing with initial interaction(s) of a stressor with a biomolecule within an organism that causes a perturbation in its biology (i.e., molecular initiating event, MIE), which can progress through a dependent series of intermediate key events (KEs) and culminate in an adverse outcome (AO) considered relevant to risk assessment or regulatory decision-making. AOPs are typically represented sequentially, moving from one KE to another. In this respect, AOPs deﬁne a series of measurable biological changes that can be expected to occur if the perturbation is suﬃciently severe (i.e., in terms of potency, duration, frequency) to drive the pathway all the way to the AO.” (OECD. Environment Directorate 2018: 12) (Ecological) Thresholds of Toxicological As deﬁned in Belanger et al. (2015): “The TTC establishes an exposure level for chemicals below which no Concern framework (Eco-TTCs) appreciable risk to human health or the environment is expected, based on a de minimis value for toxicity identiﬁed for many chemicals.” (Belanger et al. 2015: 2864) “An extension of the human safety TTC concept for application in environmental situations, termed the ‘ecological TTC’ or ‘eco-TTC,” has been explored”, with the result that “eco-TTCs summarize the wealth of ecotoxicological information as predicted no-observed effect concentrations on diverse chemical substances in the form of statistical (probability-based) distributions.” (Lillicrap et al., 2016: 2640). Fit-for-purpose validation approach An ICCVAM discussion characterized ﬁt-for-purpose validation in the following manner: “- a process based on scientiﬁcally sound principles by which the relevance and reliability of a particular method or process are established for a speciﬁc purpose (OECD GD34)” “- Focus on purpose, as opposed to demonstrating equivalence to animal based results - Acknowledge that no assay is perfect, recognize that most will be used in weight of evidence approach.” (Casey 2015) Deﬁned approaches In a recent Strategic Plan document the EPA stated: “Deﬁned approaches are based on ﬁxed sources of information (e.g., an in vitro assay and computational model) and a ﬁxed interpretation of results from those information sources. Thus, deﬁned approaches emphasize predictions which are rule-based and separate from predictions/approaches that are based on expert judgment.” (US EPA 2018) Integrated Approaches to Testing and OECD deﬁnes an IATA as “a structured approach that strategically integrates and weighs all relevant data to Assessment framework (IATAs) inform regulatory decisions regarding potential hazard and/or risk and/or the need for further targeted testing and therefore optimizing and potentially reducing the number of tests that need to be conducted.” (OECD 2017b) OECD Extended Advisory Group for Molecular Harrill (2019) deﬁned the TRF as “a tool for documenting the details of laboratory-based toxicology studies Screening and Toxicogenomics (EAGMST) that utilize a transcriptomics technology: i.e. an assay that measures the abundance of many transcripts Transcriptomics Reporting Framework (TRF) simultaneously and that provides highly multiplexed outputs. The TRF is appropriate for use in documenting experiments involving the use of either in vivo or in vitro laboratory models” Tudisca et al. 2016), but also notably to deﬁne tobacco regulatory sci- the participation of 15 experts from this list. We proceeded in a step- ence competencies (Wipﬂi et al. 2017). The Delphi method is considered wise fashion, contacting potential participants as other invited partici- well suited to deliberative and exploratory research that requires input pants declined, seeking to maximise diversity in geographic origin, gen- from geographically and professionally diverse populations since it al- der and organizational aﬃliation (see Table 2). 15 experts subsequently lows participants to respond on their own time, and when the research agreed to participate in the study, with 13 completing the Round 1 ques- problem does not allow for deﬁned analytical techniques, but rather tionnaire, 12 completing the Round 2 questionnaire, and 9 completing can be informed by shedding light on conﬂicting points-of-view. Due to the Round 3 questionnaire. This level of attrition is typical for online its anonymous and iterative nature, the Delphi method allows partici- Delphi panels due to their requirement for repeated interaction and to pants to share their opinions without reputational risk and to contra- respect a ﬁxed deadline at each step (Khodyakov et al., 2020, citing dict others who may occupy a more powerful social position (Klenk and Elwyn et al., 2006). The Delphi was conducted between August and Hickey, 2011). We did not aim to achieve consensus amongst our panel December 2018. The limits to the Delphi method include potential lan- of experts, as is often done in the ‘classic Delphi’ method. Instead, our guage barriers to participation and the demanding nature of the Delphi goal was to reveal the range of views and opinions on the policy priori- exercise (see Keeney et al. 2010: 18–31). In our case, we sought to min- ties and potential strategies to validate NAMs, then to explore potential imize the strain on participants by limiting the number of rounds, and options for their implementation following an inquiry system approach the time requirements to 1.5 hours per round. Participants’ anonymity (Mitroﬀ & Turoﬀ, 2002). As suggested by Lindstone and Turoﬀ (2002), was preserved among their peer group of experts, with all responses we designed this Policy Delphi study as a ‘forum for ideas’ to ensure that anonymized by the moderator. The detailed research protocol was ap- all possible options are considered and to examine potential impacts and proved by the McGill University Research Ethics Board. consequences of particular options. 2.1. Participants 2.2. Procedure The goal of the Delphi method is to seek “valid expert opinion” After the recruitment phase, we sent an individualized link via email (Keeney et al. 2010: 9). The sample of experts is therefore not randomly to each participant in order for them to begin the ﬁrst round of the determined. Our goal was to maximize heterogeneity of participants Delphi study. Using this secured link, participants were able to access while ensuring a high degree of expertise in the regulatory challenges an online interface designed speciﬁcally for Delphi research.2 The ﬁrst facing the adoption of NAMs. We identiﬁed potential participants based step was for participants to sign a consent form detailing the objectives on the following broad criteria: 1) Having published on NAMs and val- of the study and their rights. Following their informed consent, par- idation in the context of toxicology; and/or 2) Having participated in ticipants were then presented with a hypothetical policy scenario (see regulatory discussions about NAMs and chemical risk assessment at the Box 1) designed to ‘set the scene’ for the following questionnaire (see international level. Supplemental material for full text). Based on documentary research and key informant interviews with NAMs experts in the toxicology community, we identiﬁed an initial 2 ‘long’ list of 58 experts that met the criteria. Our goal was to secure See https://calibrum.com/ 3
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Table 2 Proﬁle of Delphi expert panel. Stakeholder Group Professional role Region Government Senior Research Scientist North America Director North America Head of Unit Europe Head of Unit Europe Senior Toxicologist North America Senior Toxicologist North America NGO / Civil society Director North America Board Member Europe Associate Director North America Industry Senior Toxicologist North America Senior Research Scientist Europe Director of Research North America Senior Research Scientist Europe Box 1. Opening statement, hypothetical scenario (excerpts) 3. Results A series of adverse ecological outcomes owing to chemical con- In Round 1, Delphi participants expressed their initial views on vali- tamination of the environment have been uncovered in several dation in toxicology (see Fig. 1). Most of the expert panel reported that parts of the world during the last 18 months. Similar to the re- they felt the “gold standard” of conventional in vivo studies has not sponse to the 2008 ﬁnancial meltdown, a group of leading govern- actually been validated. ments and global businesses coordinated their actions to curb the In the extensive comments provided in response to whether the “gold crisis. Under pressure by public opinion, political commitments standard” in toxicity testing has actually been validated, participants have been made by government leaders in several industrialized often mentioned a contrast between ‘before’ and ‘now’ with regards to countries. They have set an ambitious goal: completely overhaul validation requirements. The common explanation was that “formal val- the current regulatory testing system in the next 3 years and tran- idation did not exist at the time” and that “the gold standard was born sition towards using New Approach Methodologies (NAMs). (…) You have been asked to be part of an elite group of scientists to out of necessity.” One participant who answered ‘no’ to the question, reﬂect on the best way to move forward within these terms. You nuanced the portrait by adding: are tasked by your organization to use your expertise to design “It was never truly validated indeed. But it still provides a broader and implement an international validation process for NAMs that level of information (than many NAMs), it includes the biology of will restore trust in the science used to regulate chemicals in the environment. the organism (many NAMs do not capture this or only partly) and leads to a level of protection that also holds in court. Having an assay that performs better in terms of validation doesn’t mean you have a replacement / better system.” Similarly, one participant who answered ‘yes’ to the question added: “Validation needs to be considered in light of the intended contexts of use. A method can be valid for one purpose but not considered The Delphi study proceeded in three rounds. Round 1 requested that valid for another.” participants initially assess a list of priority actions that have previously been identiﬁed for the validation of NAMs (see Table 3) and a list of These comments highlight the situated nature (in time, geo- frameworks that could allow for greater use of NAMs in chemical risk graphic/legal context, context of use, etc.) of validity. assessment. Participants were asked to justify their ratings of the im- portance of diﬀerent priorities as well as the beneﬁts and feasibility of diﬀerent validation frameworks. These ratings and justiﬁcations were 3.1. Priorities for an eﬀective transition towards NAMs visible to other participants in the subsequent rounds only. Participants could also add other priorities and other frameworks that were not ini- In Round 1, Delphi participants also assessed the policy priorities tially listed in the questionnaire. Round 1 ended with a series of reﬂexive for NAMs (Fig. 2). Interesting contrasts emerged in how participants’ questions about the concept of validity in toxicology. explained their priority assessments. Some felt that there were already Round 2 requested that participants begin by examining priority enough information and tools available, while others disagreed. Regard- rankings and justiﬁcations that were produced by their peers. Partici- ing the priority of “Developing new technological tools to assess hazard pants were then requested to leverage this information to create a list and exposure”, many participants felt that the current tools for haz- of action items to operationalize the priorities (e.g. ‘Developing standard ard assessment were suﬃcient, but that more could be done to develop guidelines’) and a list of requirements and consequences for using the tools that measure exposure. Similarly, some participants suggested that frameworks (e.g. IATA, AOPs). To minimize the length of time needed decision-making frameworks should be ﬂexible, while others favoured a to complete the Delphi exercise, we suggested that participants focus more structured approach with less room for subjectivity. A participant on only 2 priorities and 2 frameworks (out of a possible total of 8 pri- from industry expressed a particular viewpoint: orities and 6 frameworks). For clarity, this list of action items and of requirements and consequences that was generated by participants was “I think industry would be more comfortable with a bit more struc- summarized into a simpliﬁed list of propositions (duplicates and exam- tured approach than IATA oﬀers. So that no matter the risk assessor ples were removed). Round 3 required each participant to look back at a product will get a fair assessment. We have experienced diﬃculty their whole experience with the Delphi exercise and reﬂect on the notion in product registration with risk assessors not being familiar with of validity and how it is used in toxicology. The complete anonymized WoE (Weight of Evidence approach) and test materials that do not dataset can be made available on request. ﬁt the traditional product testing ‘mold’ ”. 4
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Table 3 The Policy Delphi exercise used in this study. Round Focus Tasks 1 Initial assessment of Priorities priorities and Given the mandate to transition towards NAMs in the next 3 years, and considering resource frameworks. limitations, please indicate your assessment of the importance for the international ecotoxicology community to invest in the following… Frameworks Keeping in mind the goal to integrate NAMs during the next 3 years, how would you rate the following frameworks? The [beneﬁts OR feasibility] of a strategy based on developing and standardizing practices tied to… Reﬂexive questions Do you think that what has up to now been referred to as the "gold standard" in toxicity testing has actually been validated? Thinking speciﬁcally about regulatory ecotoxicology, how do you respond to the following statements? A test method is ‘valid’ because… • it has been used in the past by the regulatory community. • it is the result of a development process that is considered rigorous and balanced by the regulatory community. • the most knowledgeable experts in the area approve of its use. 2 What needs to be We are asking that you choose at least 2 priorities and 2 frameworks that you believe will done in the near maximize the potential to meet the goal of overhauling the current regulatory testing future to materialize system over the next 3 years and transition towards using New Approach Methodologies concrete scenarios? (NAMs). Be forward looking Priorities: and think strategically. Examine your peer’s Round 1 comments and recommend action items Frameworks: Examine your peer’s Round 1 comments and anticipate requirements and consequences 3 Holistic point of view. After having examined your peer’s responses in Round 2, Please respond to these open-ended questions: Look back at the collective wisdom generated through • In your view, is the validity of test methods an assumption that helps to simplify and previous rounds understand risk assessment, or a template for the description of actual causal mechanisms and be reﬂexive about in risk assessment? Please explain your thinking. validity. • To what extent would criticism of the notion of validity undermine or enhance the ecotoxicology community‘s credibility with decision-makers and the public? Again, please explain your thinking. • If you had to provide one lesson that you learned from this Delphi process regarding the process of validation, what would it be? Fig. 1. Views on validation, Round 1 panel participants (n=13). A number of participants reported that knowledge levels on NAMs emergent theme calling for a common data-sharing infrastructure. One were suﬃcient in the regulatory community, while others clearly stated notable counterpoint was oﬀered: “Commercialization and cost”. This that they were not. One participant from the regulatory community sug- participant argued that both commercial availability and aﬀordability gested that knowledge about conventional approaches was lacking in (i.e. available at a reasonable cost) of NAMs are other necessary con- the academic community. ditions for uptake, in addition to superior performance as compared to Given the opportunity to suggest other priorities, Delphi participants conventional methods. oﬀered a variety of themes, including some related to data/evidence In Round 2, Delphi participants were requested to examine their integration (e.g. “Focus on integration of data, rather than single meth- peers’ Round 1 comments and to select at least two priorities they be- ods” and “Integration of evidence”), that are in line with an overarching lieved could maximize the likelihood of success in transitioning towards 5
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Fig. 2. Given the mandate to transition towards NAMs in the next 3 years, and considering resource limitations, please indicate your assessment of the importance for the international ecotoxicology community to invest in the following (Scale: 1 ‘No importance’ to 5 ‘Highest importance’). NAMs. Participants then provided up to ﬁve action items to help oper- From Table 4, we interpret two dominant themes: global commu- ationalize the priorities on which they selected to focus. Participants’ nication between stakeholders and universality. Following the need responses for the two highest ranked priorities: 1) ‘Developing common for ‘collaboration infrastructure’, and the information, learning and re- data collection, reporting and sharing procedures for new test methods source sharing beneﬁts that can ﬂow from this research asset, our par- (e.g. Quality Assurance and Quality Control)’; and 2) ‘Increasing basic ticipants further underlined the need for a global network dedicated knowledge about new test methods in the regulatory community’ were to NAMs. Here, one can imagine the beneﬁts (such as cost eﬃciencies summarized (Table 4). that could result from sharing information), the human resource require- 6
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Table 4 Suggested action items for selected priorities, simpliﬁed. Priority Action items Developing common data collection, reporting and sharing procedures for new test methods (e.g. Quality Assurance and Quality Control) • Establish a global network where information can be shared among stakeholders – a common database for all to utilize • Bring together expertise from both traditional toxicology and NAM expert communities to ensure a universal rapid validation method • Obtain the ﬁnancial resources needed to build big data approaches • Establish global criteria for assessing data • Deﬁne reporting structures through global standardization to help ensure consistency • Develop universal "output" metrics that would allow universal comparison of results across NAMs Increasing basic knowledge about new test methods in the regulatory community • Obtain knowledge about the strengths and weaknesses of NAM approaches from efforts to apply them alongside existing knowledge • Ensure collaboration/engagement with regulators to establish where NAMs would be most useful • Hold workshops/conferences on NAMs, which bring together the scientiﬁc and regulatory community • Create a database that better captures all existing toxicological information that are important for safety assessment and regulation • Ensure that research efforts are aligned with regulatory needs • Establish a site where methodologies that are standardized or in development can be easily accessed by the regulatory community • Develop training courses for regulators on NAMs data and how they could be integrated into risk assessment ments (such as ‘on loan’/secondement lateral movement of employees represents an option oﬀering incremental but possibly constant advance- between NAM stakeholders), and the harmonization challenges. On the ment, participants who positively evaluated it mentioned its ﬂexibility latter, the panel participants outlined bold and ambitious action items, and the fact that it could allow for immediate use of available NAMs. aiming for universality and global standards. Another common theme is Several participants indicated that this option enables “tailored goals” to build common understanding. or to “use methodologies based on the context of use”. One participant proposed that ﬁt-for-purpose validation needs to occur in parallel with a more ambitious long-term target of ultimate replacement, as such an 3.2. Validation Frameworks approach would enable the assessment of more complex endpoints. Re- garding ‘IATA’, participants mentioned this framework’s ﬂexibility and In Round 1, the Delphi participants assessed the beneﬁts and fea- its ‘integration of evidence aspect’, in their justiﬁcations of their high sibility of the following frameworks designed to support and facilitate assessment of its beneﬁts, which included superior ‘problem formula- validation, further deﬁned in Table 1 (the labels in brackets are used in tion’ and a ‘tiered approach’. Many participants mentioned the signif- Fig. 3): icant resources and multiple data streams\technologies that would be • Adverse Outcome Pathways (AOPs) required as obstacles to this framework’s feasibility. Regarding ‘AOPs’, • Ecological Thresholds of Toxicological Concern (eco-TTCs) most participants mentioned as a key beneﬁt this framework’s ability • Incremental ﬁt-for-purpose use of new assays within current regula- to transparently represent biological mechanisms and modes-of-action. tory decision frameworks (Fit-for-purpose) One participant summarized the argument: • Replacement of existing assays with new standardized deﬁned ap- “NAMs often make measurements at levels of biological organization proaches that are based on NAMs (Replace) that are upstream of the ‘adverse outcome of regulatory signiﬁcance’, • Integrated Approaches to Testing and Assessment (IATAs) making interpretation of the importance of their results a challenge. • The OECD EAGMST’s initiative to develop a series of guidance doc- AOPs are proposed as a means to make the regulatory signiﬁcance uments for consistent reporting of ’omics data, i.e. a Transcrip- of NAM results more clear.” tomics Reporting Framework (EAGMST) The mean value of participants’ evaluation of the feasibility of using However, when evaluating feasibility, most participants mentioned the diﬀerent frameworks to integrate NAMs in chemical risk assessment the onerous nature of developing AOPs (time and money), and the fact are all under the mid-point of the scale (‘neutral’ = 2), except for Eco- that the burden of developing AOPs is not evenly distributed or suﬃ- TTCs that is slightly over the mid-point (=2.2/4) (Fig. 3), indicating the ciently incentivized among policy actors. One participant expressed the perceived diﬃculty of the task. The option “Replacement of existing as- view that “[AOPs] have been a bit “oversold” to the regulatory commu- says with new standardized deﬁned approaches that are based on NAMs” nity and the OECD process / wiki has made things a bit overly compli- was generally viewed as producing less beneﬁts and being less feasible cated.” Another participant added, regarding AOPs: “this is a bit of a than the other options. In their comments, participants mentioned that ‘Wild West’ right now”. this option had not worked in the past. In Round 2, participants were prompted to examine their peers’ In the associated comments explaining their assessment of each Round 1 comments and to choose at least two frameworks from the framework, participants justiﬁed their lower evaluation of the ‘Replace’ list. For these frameworks, participants were then instructed to provide option by referring to past experience, mentioning “1:1 replacement” of up to three expected requirements (i.e. preconditions) and up to three methods did not work or “We saw the slow pace of progress” when 1:1 foreseeable consequences of making this framework central to a strategy replacement was tried. Regarding the ‘ﬁt-for-purpose’ framework, which for transitioning towards using NAMs in chemical risk assessment. The 7
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Fig. 3. Keeping in mind the goal to integrate NAMs during the next 3 years, how would you rate the following frameworks? (Likert scale: 0 to 4). responses are synthesized and summarized for the three highest scoring dustry and across ecological and health ﬁelds” should occur. Another frameworks in Table 5. participant interpreted that diﬀerent understandings of validation may Overall, the results presented in Table 5 suggest that many hurdles originate from “analytical or qualitative [diﬀerences] with respect to need to be overcome for the various frameworks to be eﬀectively imple- mechanisms, or even quantitative [diﬀerences] with respect to risk” and mented. Suggested ‘requirements’ indicate the current lack of conver- suggested that “the diﬀerent understandings should be discussed and gence of views with regards to IATAs (Panel A: ‘standardize/harmonize’, potentially organized with associated clear deﬁnitions”. Another par- ‘clear guidelines/structure’, ‘build consensus’, ‘prove IATA are more ef- ticipant concluded: “This exercise illustrates the tremendous diﬃculty ﬁcient’), a need for more pragmatism with regards to AOPs (Panel B: inherent in the integration of NAMs into risk assessment practice”. ‘better support’, ‘incentives’, ‘more clear mechanism’) and lack of clar- Another common lesson drawn by participants is the importance of ity with regards to ﬁt-for-purpose (Panel C: ‘a good understanding’). In thinking about validation as such, and the process through which valid- the suggested negative ‘consequences’, a common theme emerges: risks ity is conferred. One participant suggested: “signiﬁcant progress can be tied to end-user behaviour. Participants suggest that end-users might be made towards facilitating NAM uptake into the risk assessment process” unable to use, misuse, or be ‘trepid’ when faced with NAM data. Al- by reﬂecting on the “IDEA of method validation itself”. While these par- though this does not represent a consensus opinion among our panel of ticipants expressed views that appear conducive to a learning stance, NAM experts, this negative perception of end-users potentially creates a other participants clearly diﬀered, suggesting that there may be social distance between NAM experts – that are often promoters for the greater learning challenges in the expert space of NAM stakeholders: use of NAMs in chemical risk assessment – with the potential end-users. “I have been ﬁghting to implement NAMs for too many years to With regards to AOPs speciﬁcally, the participants’ ‘requirements’ and change my mind about the importance of willingness at regulatory ‘consequences’ suggest that AOPs are viewed as a long-term scientiﬁc level to accept that the often non-validated animal models are in- project, and that it cannot be reliant upon AOPs alone for NAMs to be ferior to the novel NAMs with respect to hazard identiﬁcation, and used in a regulatory context. Overall, the ﬁndings suggest no clear con- eventually human risk assessment.” sensus among our sample of NAM experts with regards to the direction of change that is necessary. 4. Discussion 3.3. Lessons identiﬁed Two priorities for an eﬀective transition towards NAMs were, on In Round 3, participants reﬂected on the entire Delphi study and average, ranked substantially higher than the other options by our in- their peers’ responses to address questions about the international val- ternational panel of experts (see Fig. 2): 1) “Developing common data idation of NAMs. To the open-ended question “If you had to pro- collection, reporting and sharing procedures for new test methods” and vide one lesson that you learned from this Delphi process regarding the 2) “Increasing basic knowledge about new test methods in the regula- process of validation, what would it be?”, 6 out of 9 participants in tory community”. We suggest that both of these priorities elevate the Round 3 independently mentioned the issue of convergence/divergence need for a common regulatory science infrastructure that could enable of views on validation. Of these 6 participants, only one expressed the the beneﬁts of collaboration and coordination to be realized amongst opinion that views on validation are converging; the 5 other partici- toxicology stakeholders. This could include international regulatory di- pants stated that views were diverging. On this topic, one participant alogues, large-scale transboundary research collaborations, and coordi- suggested that “more silo-busting across academia, government, and in- nated innovation in technological tools, the discourse of scientiﬁc vali- 8
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Table 5 Suggested requirements and consequences for selected frameworks, simpliﬁeda . Please see Panel A: IATAs; Panel B: AOPs; Panel C: Fit for Purpose. Panel A: Integrated Approaches to Testing and Assessment (IATAs) Requirements Consequences Positive • Evaluate whether IATAs can be used for hazard identiﬁcation mainly or also for • More cost eﬀective testing risk assessment • Reductions in animal use • Develop a ﬂexible standardization process for IATAs • Increased conﬁdence in decisions • Standardize/harmonize individual components of IATAs to address transparency • Increased ﬂow of chemicals through low level decisions (e.g. screening) • Develop clear guidelines/structure for chemical safety decisions when using • Superior understanding of gaps in in vivo MOAs IATAs • Increased ‘learning by doing’ in regulatory decision making • Build consensus on the requirements for using IATAs in diﬀerent decision contexts Negative • Acquire information on suitability/limitations of NAMs within most developed IATAs. • Develop case studies of the applications of NAMs as part of IATAs • Challenges of scope of use, especially as concerns higher order decisions (e.g., full • Make changes to regulatory testing programs and/or regulatory acceptance assessment of highly visible chemicals) processes to allow for IATAs to be used • Human and other resource requirements (IATAs will be more time consuming than • Actions that “prove“ IATA approaches are more eﬃcient, cost eﬀective and still conventional methods.) health protective. • Challenges of interpretation, as end-users may not be able to interpret the broad • Use lessons from previous IATAs to develop new IATAs spectrum of NAM information used in IATAs • Development of IATAs for every MOA and trophic level/taxa combination • Challenges of integration, especially as concerns the OECD Mutual Acceptance of Data Agreements (MAD). Neither positive or negative • Heterogeneous outcomes, since there will still be diﬀerent ways of coming to a regulatory conclusion, and potentially diﬀerent conclusions. • Lack of ﬁt with current hazard classiﬁcation schemes (e.g. GHS -Globally Harmonized System) which are based on apical endpoints (e.g. LC50). • The need to “crosswalk” [cross-reference] IATA results with numerical toxicity test results (e.g. NOAELs or BMDs) from lab animal studies. Panel B: Adverse Outcome Pathways (AOPs) Requirements Consequences Positive • Crowd sourced participation is needed to realize the full potential of AOPs • Would focus scientiﬁc investigation on what we critically need to know • Prioritize key endpoints in developing AOPs • Better use of existing information, leading to better safety decisions • AOPs developed for major chemical classes (e.g. non-polar narcosis) and all major trophic levels Negative • A “small army” of data managers to gather, organize and evaluate relevant data • Better support and guidance for using AOPs • Regulatory incentive to develop AOPs • Collection of AOP knowledge will take a lot of time • Institutionalize AOPs as formal way of collecting information (e.g. by making • Creates a risk that end users focus on trivial/unimportant pathways for assessing AOP entry a stipulation of funding and/or publication of results) overall risk of a chemical • Allow for AOP response to be linked back to standard methods and standard hazard classiﬁcation schemes Neither positive or negative • A more clear mechanism to incorporate complex data into something that is easily interpretable • Will result in a more precautionary approach to chemical management • Inform chemical safety assessors about the applications of AOPs in • Will provide assessors with the “list“ of “pathways of toxicological relevance“ that decision-making were envisioned in the Toxicology in the 21st century document Panel C: Incremental ﬁt-for-purpose use of new assays within current regulatory decision frameworks (Fit for purpose) Requirements Consequences Positive • Consensus on how to assess ﬁt-for-purpose is required to increase conﬁdence • Enhance NAM uptake, particularly in the short term and in contexts where a higher • Law of parsimony/ measured application of NAM data sources when applying degree of uncertainty is acceptable NAMs in existing regulatory decision frameworks • May increase/pave the way for stronger overall regulatory acceptance of NAMs in • Utilize case examples with diﬀerent NAMs to better deﬁne what ﬁt-for-purpose the long term means • Makes safety assessment and regulation more application relevant compared to • Deﬁne/provide information on what diﬀerent NAMs can and cannot provide for OECD TGs any given decision context and chemical applicability domain (purpose) • Will introduce more ﬂexibility in safety assessment and regulation, as compared to • A good understanding – and ability to communicate – uncertainties associated OECD TGs with NAMs • Better dialog between data submitter and regulatory authority, improving mutual • Transparency in describing how NAMs are used and why they are ﬁt-for purpose understanding of needs, applications, and uncertainties • Mechanistic assays that can provide relevant information in a weight of evidence approach within IATAs Negative • Systematic review principles are needed for the integration of extant data into decision foci intended for regulatory applications • Discuss areas for which validation of a NAM with in vivo animal data is not • Fitness is judged at the method level for mechanistic relevance, but the individual reasonable methods still need to be integrated to fulﬁll existing data requirements • Will lack united global focus on one solution because of the diﬃculty to work, share and harmonize across jurisdiction in the ﬁt-for-purpose approach • Limited applicability domains based upon trepidation of end-user community to alter or modify existent decision frameworks Neither positive or negative • Regulatory frameworks evolve from a ﬁxed set of data requirements to a ﬁxed set of information requirements, therefore avoiding the need to predict and replace animal data • Problem formulation will become a crucial ﬁrst step that determine targeted applications of NAMs a For clarity, authors organized participants’ responses re “consequences” into “positive”, “negative” or “neither positive or negative”. 9
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 dation, and regulatory procedures. This ﬁnding, which suggests the need derway for at least 30 years. Validation centers have been founded in for a ‘collaboration infrastructure’ agenda for the validation of NAMs, Europe, US, Canada, Japan, Korea and Brazil. In depth international converges with a key ﬁnding from the science and technology studies discussions and workshops have been conducted. Guidelines and work- literature (Sismondo 2010), a social science area of research that is rel- plans have been adopted. As many panel members noted, a wealth of evant to the validation of emergent regulatory science, such as NAMs. information has been generated and many alternative tests have been Studying a variety of natural science disciplines, science and technology designed. However, these data have not yet translated into a “common studies have shown how science relies on the functioning of a stable in- language”, which limits the potential beneﬁts of collaboration. There frastructure of data gathering and reporting (Star and Ruhleder 1996; remains a need for stakeholders to rally around a common collabora- Slota and Bowker 2017) and that the trustworthiness and credibility of tion infrastructure agenda. For this ambition to materialize, a network ‘evidence’ is produced by a network of organizations, institutions and of trust and cooperation needs to be woven across many sites (labo- actors (Callon 1986; Latour 1987). Such an infrastructure of organiza- ratories, national regulatory agencies, contract research organizations, tions, rules, norms and practices remains to be built in order for the chemical producers, chemical users and the public) creating links where larger toxicology community to better communicate, share and learn validity and measurement are communicated and transferred. This will about NAMs. require that stakeholders come to a level of agreement on what matters For the toxicology community to beneﬁt from greater collaboration for particular uses (content in relation to context), how it is commu- infrastructure, the adoption of common language is necessary: nicated (data format) and measured (metrics). Such agreement would need to be credible to all stakeholders acting in the governance network “The extension of scientiﬁc practice from a laboratory to any other to facilitate trust. Cooperation between elements of the network is key. place in society […] requires the construction of long and consis- Furthermore, the content, data format and metrics would likely need to tent networks along which precision and reliability can circulate.” have the desirable properties of standards; that is, 1) represent the phe- (Mallard 1998: 573). nomena that they observe, 2) create a manageable workload, and 3) be Panel participants identiﬁed this imperative, suggesting action items stable across time and locations. that help create the many connections that are needed for the network Important practical questions follow from the above: what is the role to operate, in particular the link between NAM researchers and regu- of existing international collaborations, e.g., the OECD Mutual Accep- lators (‘engagement’ or ‘alignment’ with regulators, workshops, science tance of Data (MAD) agreements, therein? Should the information pro- training for regulators, database, easily accessible website; see Table 4). vided by NAMs be translated into commonly used regulatory concepts With no clear organization that has global authority to make binding and conventional, numerical toxicity test results from laboratory animal decisions that entail a change in Member State’s internal regulations, a studies? Or should they be expressed in a new baseline language that can high degree of uncertainty is likely to endure for some time. Interna- accommodate both? Perhaps a ﬁrst step working towards convergence tional toxicology reﬂects a complex governance situation, where deci- that could be accomplished before agreement on these larger questions sions and mandates operate in pluralistic, overlapping, and fragmented are settled, may be, as several of our Delphi study participants have regimes of rules (Pattberg and Stripple 2008; Boyd 2010; Keohane and mentioned, to better document the strengths and weaknesses of NAM- Victor 2011; Zelli et al. 2013). In this setting, where no global orga- based assays so that the toxicity information that they provide can be nization has overarching direct governing authority, collective policy better integrated in the practice of risk assessment. goals are achieved through delegation, trusteeship, cooptation, or or- Putting in place a global infrastructure that enables valid measure- chestration (Abbott et al. 2019). The latter, orchestration, is perhaps ment to be communicated is one prerequisite. One way forward is to the governance arrangement that ﬁts best with the needs identiﬁed by follow a global orchestrator that would inspire stakeholders to “play in our Delphi panel. Abbott et al. (2015: 4) describe orchestration in the the same key”, despite there being no global authority directing them following manner: to do so. This puzzle of regulatory science coordination at a global scale will require that stakeholders learn science diplomacy skills that are cru- “An international organisation (IGO) enlists and supports intermedi- cial in designing systems of risk measurement and monitoring. Such a ary actors to address target actors in pursuit of IGO governance goals. skillset is very diﬀerent from the skillset used to work within an ex- The key to orchestration is that the IGO brings third parties into the isting and stable system. On this subject, Demortain (2011) showed governance arrangement to act as intermediaries between itself and how a particular class of entrepreneurial elite scientists that travel the targets [private entities], rather than trying to govern the targets across jurisdictional borders, across private and public organizations directly. […] An orchestrator is therefore not a “governor” in the and that navigate diﬀerent professional identities can be particu- hierarchical sense. Rather, the orchestrator uses ideational and/or larly inﬂuential in shaping international regulatory standards in their material inducements to create, integrate and maintain a multi-actor area. system of soft and indirect governance, geared toward shared goals that neither orchestrator nor intermediaries could achieve on their 4.2. Limitations and future directions own.” An orchestrator therefore exerts leadership to inspire voluntary co- The Delphi method collates and integrates expert opinion and there- operation of enlisted organizations to address shared goals. This may fore relies on the quality and relevance of the panel participants and well be an important component of the path forward, if the toxicology the questions asked. While it is not the purpose of the expert panel in a community wishes to adopt NAMs. Finding an organization that could Delphi study to represent a larger population –their senior professional accept such a role is a challenge in itself, but in order for this new global roles and degree of expertise aﬀords panelists in this study the capacity toxicology orchestrator to be eﬀective, NAM stakeholders will need to to oﬀer informed and contemporary insights into the international adop- calibrate their instruments using the same tuning system, lest they will tion of NAMs in regulatory chemical risk assessment. Our participants not play in key. produced abundant inputs to each round of the Delphi, diligently ad- dressing the issues being explored and leveraging their peers’ responses 4.1. Accelerating the international validation of NAMs where possible. Participants also expressed strong motivation to partic- ipate in interactions with the moderator (the lead author) during re- What can be done to accelerate the validation process for NAMs in- cruitment and throughout each round. However, the demanding nature ternationally? Our study has done an initial mapping of expert views of the Delphi exercise created barriers to participation, with one notable on the core issue of validation. Important progress has already been absence in our sample being academics. Despite our best recruitment ef- made, and an international conversation on ways forward has been un- forts, all of the academics that we approached either declined or did not 10
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 respond to our invitation. The language barrier may also have been an Author Statement obstacle to participation, limiting the geographic diversity of this study to Europe and North America. This occurred despite our eﬀorts to ap- Matthieu Mondou: Conceptualization; Data curation; Formal anal- proach NAMs experts in Asia and Latin America. These barriers may ysis; Investigation; Methodology; Project administration; Roles/Writing have had an impact on our results, with viewpoints from academics and - original draft. Steven Maguire: Conceptualization; Funding acquisi- non-Western experts on NAMs potentially not represented. By design, tion; Project administration; Supervision; Writing - review & editing. our study has another limitation: it did not ask participants to reﬂect on Guillaume Pain: Writing - review & editing. Doug Crump: Funding ac- the sequence or coherence of actions needed to validate NAMs. Future quisition; Writing - review & editing. Markus Hecker: Funding acqui- action-oriented Delphi research could perhaps isolate a smaller problem sition; Writing - review & editing. Niladri Basu: Funding acquisition; identiﬁed through our Delphi study and ask panel participants to reach Writing - review & editing. Gordon Hickey: Conceptualization; Formal consensus on a coherent set and sequence of actions. Focussing on a nar- analysis; Funding acquisition; Methodology; Project administration; Re- rower range of strategies could help to craft a clearer path forward for sources; Supervision; Writing - review & editing. the international validation of NAMs in toxicology. A more quantitative Delphi study designed to enlist a larger sample of policy actors could Declaration of Competing Interest also be useful in showing the pathways recommended by the greatest number of experts internationally. The authors declare that they have no known competing ﬁnancial interests or personal relationships that could have appeared to inﬂuence the work reported in this paper. 5. Conclusion Acknowledgments Over the last century, the global toxicology community has ﬁne- We would like to thank our expert participants for donating their tuned its system of regulatory science to the temperament of whole valuable time and knowledge to our study. This study informs the GE3LS animal in vivo studies. This adoption of globally standardized prac- (Genomics and its Ethical, Environmental, Economic, Legal, and So- tices has generated immense coordination beneﬁts for industry and cial aspects) component of the project “EcoToxChip: A toxicogenomics government stakeholders, while ensuring that toxicology profession- tool for chemical prioritization and environmental management”. The als around the world can engage in dialogue to assess and manage authors thank the ﬁnancial sponsors of this project (Genome Canada; risk and to make evidence-based determinations that protect the pub- Génome Québec; Genome Prairie; the Government of Canada; Envi- lic interest. However, just as the adoption of previous tuning systems ronment and Climate Change Canada; Ministère de l’Économie, de la in the history of music generated areas of dissonance (the ‘wolf inter- Science et de l’Innovation du Québec; the University of Saskatchewan; val’ in the ‘circle of ﬁfths’) in an otherwise harmonious musical space, and McGill University). We gratefully acknowledge Tuviere Onookome- the adoption of what is considered today ‘conventional’ in vivo testing Okome for her excellent work as research assistant. has generated areas of dissidence in an otherwise functional regulatory space. References In the previous three decades, the global toxicology community has discovered new ways of producing regulatory science with NAMs – but Abbott, K.W., Genschel, P., Snidal, D., Zangl, B., 2015. International Organizations as Orchestrators. Cambridge University Press, Cambridge. they need an alternative tuning system to be leveraged. These New Ap- Abbott, K.W., Genschel, P., Snidal, D., Zangl, B., 2019. Competence versus control: the proach Methodologies have put in brighter contrast the potential knowl- governor’s dilemma. Regul. Gov. Early View doi:10.1111/rego.12234. [accessed 2019 edge gaps in the current system of conventional testing, but as many of Apr 4] https://onlinelibrary.wiley.com/doi/abs/10.1111/rego.12234 . Balls, M., 1995. Deﬁning the role of ECVAM in the development, validation and ac- the Delphi panel participants have suggested, they too have their limi- ceptance of alternative tests and testing strategies. Toxicol. in Vitro 9 (6), 863–869. tations. For the global toxicology community to leverage the strengths doi:10.1016/0887-2333(95)00082-8. of both conventional testing and NAMs, there likely needs to be a global Balls, M., Blaauboer, B.J., Fentem, J.H., Bruner, L., Combes, R.D., Ekwall, B., Fielder, R.J., Guillouzo, A., Lewis, R.W., Lovell, D.P., et al., 1995. Practical aspects of the validation shift towards a regulatory science tuning system that can accommodate of toxicity test procedures. The report and recommendations of ECVAM workshop 5. a diversity of testing strategies. If this shift occurs, it will transform cur- ATLA Altern. Lab. Anim. 23 (1), 129–147. rent practices, creating unease and new uncertainties in the transition. It Balls M., Combes R., Worth A.P. 2019. The history of alter- native test methods in toxicology. [accessed 2019 Mar 27]. would also provoke a diﬀerent distribution of test limitations within the http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db= practice of risk assessment – akin to new areas of dissonance – that will nlabk&AN=1724645. require professionals to interpret a diﬀerent set of uncertainties. It is this Basu, A., Phipps, S., Long, R., Basu, N., 2015. Identiﬁcation of Response Options to Arti- new set of uncertainties that now needs to be embraced and discussed sanal and Small-Scale Gold Mining (ASGM) in Ghana via the Delphi Process. Int. J. Environ. Res. Public Health. 12, 11345–11363. globally in order for the toxicology community to adopt a new regula- Basu, N., Crump, D., Head, J., Hickey, G., Hogan, N., Maguire, S., Xia, J., Hecker, M., tory science tuning system that minimizes limitations and uncertainties 2019. EcoToxChip: a next-generation toxicogenomics tool for chemical prioritiza- (acknowledging that there will always be limitations), while maximiz- tion and environmental management. Environ. Toxicol. Chem 38 (2), 279–288. doi:10.1002/etc.4309. ing the beneﬁts of a diversity of test methods. If this global discussion Belanger, S.E., Sanderson, H., Embry, M.R., Coady, K., DeZwart, D., Farr, B.A., Gutsell, S., takes example from the history of music, which saw the large-scale adop- Halder, M., Sternberg, R., Wilson, P., 2015. It is time to develop ecological thresholds tion of the equal temperament (a tuning system in which no interval is of toxicological concern to assist environmental hazard assessment. Environ. Toxicol. Chem. 34 (12), 2864–2869. doi:10.1002/etc.3132. perfectly in tune, but all intervals are very close to natural harmony), Boyd, W., 2010. Climate change, fragmentation, and the challenges of global environ- then toxicology stakeholders around the world would aim to distribute mental law: elements of a post-Copenhagen assemblage. Univ. Pa J. Int. Law 32 (2), the dissonance and the dissidence that the new temperament of regula- 457. Bradbury, S.P., Feijtel, T.C.J., Leeuwen, C.J.V., 2004. Meeting the scientiﬁc needs of eco- tory science will provoke equally across stakeholders, geography, and logical risk assessment in a regulatory context. Environ. Sci. Technol. 38 (23), 463A– decision-points in regulatory chemical risk assessment. 470A. doi:10.1021/es040675s. Bruner, L.H., Carr, G.J., Curren, R.D., Chamberlain, M., 1998. Validation of alternative methods for toxicity testing. Environ. Health Perspect. 106, 8. Busquet, F., Hartung, T., 2017. The need for strategic development of safety sciences. Funding Altern. Anim. Exp. ALTEX 34 (1), 3–21. Busquet, F., Strecker, R., Rawlings, J.M., Belanger, S.E., Braunbeck, T., Carr, G.J., Genome Canada and Genome Québec. Grant number: 10501. Cenijn, P., Fochtman, P., Gourmelon, A., Hübler, N., et al., 2014. OECD validation study to assess intra- and inter-laboratory reproducibility of the zebraﬁsh embryo toxi- Social Sciences and Humanities Research Council of Canada city test for acute aquatic toxicity testing. Regul. Toxicol. Pharmacol. 69 (3), 496–511. (SSHRC). Grant number: 756-2017-0131 doi:10.1016/j.yrtph.2014.05.018. 11
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Callon, M., 1986. Éléments pour une sociologie de la traduction: la domestication des the context of a tiered testing program. Syst. Biol. Reprod. Med. 58 (1), 3–9. coquilles Saint-Jacques et des marins-pêcheurs dans la baie de Saint-Brieuc. Année doi:10.3109/19396368.2011.652288. Sociol. 19401948-, 169–208. Mitroﬀ, I., Turoﬀ, M., 2002. Philosophical and methodological foundations of Delphi. Carrie, A., 2015. ToxCastTM wants you: recommendations for engaging the broader scien- In: Linstone, H., Turoﬀ, M. (Eds.), The Delphi Method: Techniques and Applications. tiﬁc community. Environ. Health Perspect. 123 (1), A20. doi:10.1289/ehp.123-A20. Addison-Wesley Pub. Co., Reading, Mass, pp. 17–34 Advanced Book Program. Casey W. 2015. Fit-for-purpose validation. Mondou, M., 2018. A timeline of the development of new approach methodologies. Eco- Cazabon, D., Fobil, J.N., Essegbey, G., Basu, N., 2017. Structured identiﬁcation of response ToxChip Proj [accessed 2019 Mar 29] http://www.ecotoxchip.ca/project-tools.html. options to address environmental health risks at the Agbogbloshie electronic waste Mondou, M., Hickey, G.M., Rahman, H.T., Maguire, S., Pain, G., Crump, D., Hecker, M., site. IEAM 13 (6), 980–991. Basu, N., 2020. Factors aﬀecting the perception of New Approach Methodologies Clay, R., 1996. Global agreement on alternative testing. Environ. Health Perspect. 104 (6), (NAMs) in the ecotoxicology community. Integr. Environ. Assess. Manag. 16 (2), 612–614. 269–281. Corvi, R., Ahr, H.-J., Albertini, S., Blakey, D.H., Clerici, L., Coecke, S., Douglas, G.R., National Research Council, 2007. Toxicity Testing in the 21st Century: A Vision and Gribaldo, L., Groten, J.P., Haase, B., et al., 2006. Meeting report: validation of a Strategy. National Academies Press, Washington, D.C. [accessed 2017 Jun 19] toxicogenomics-based test systems: ECVAM–ICCVAM/NICEATM considerations for http://www.nap.edu/catalog/11970. regulatory use. Environ. Health Perspect. 114 (3), 420–429. doi:10.1289/ehp.8247. NIEHS, 1997. Validation and Regulatory Acceptance of Toxicological Test Meth- Demortain, D., 2011. Scientists and the Regulation of Risk: Standardising Control. Edward ods: A Report of the Ad Hoc Interagency Coordinating Committee on Elgar Publishing. the Validation of Alternative Methods. Research Triangle Park NC: NIEHS Elwyn, G., O’Connor, A., Stacey, D., Volk, R., Edwards, A., Coulter, A., Butow, P., 2006. Report No.: NIH PUBLICATION NO: 97-3981. [accessed 2017 Sep 25] Developing a quality criteria framework for patient decision aids: online international https://ntp.niehs.nih.gov/iccvam/docs/about_docs/validate.pdf. Delphi consensus process. BMJ 333 (7565), 417. OECD, 1996. Final Report of the OECD Workshop on Harmonization of Validation and ECHA, 2016. New Approach Methodologies in Regulatory Science - Proceedings of a Sci- Acceptance Criteria for Alternative Toxicological Test Methods. Organisation for Eco- entiﬁc Workshop. ECHA, Helsinki, Finland. [accessed 2017 Sep 25]. nomic Co-operation and Development, Paris Report No.: ENV/MC/CHEM/TG(96)9. Environment and Climate Change Canada, 2016. Integrating New Approach Methodolo- OECD. 2017a. Test No. 442E: In Vitro Skin Sensitisation assays address- gies within the CMP: Identifying Priorities for Risk Assessment, Existing Substances ing the Key Event on activation of dendritic cells on the Adverse Out- Risk Assessment Program. Government of Canada, Ottawa [accessed 2017 Aug 15] come Pathway for Skin Sensitisation. Paris. [accessed 2017 Nov 2]. http://www.ec.gc.ca/ese-ees/default.asp?lang=En&n=172614CE-1. http://www.oecd-ilibrary.org/docserver/download/9716121e.pdf?expires= Eskes, C, Whelan, M (Eds.), 2016. Validation of Alternative Methods for Toxicity Testing. 1509596123&id=id&accname=guest&checksum=DC05B3E2DAA281135E544071 Springer, Cham, Switzerland (Advances in experimental medicine and biology). 61B5264B. EURL ECVAM. TSAR | Tracking system for alternative methods towards Regulatory ac- OECD. 2017b. Adverse outcome pathways, molecular screening and toxicoge- ceptance (TSAR). [accessed 2019 Mar 29]. https://tsar.jrc.ec.europa.eu/. nomics. [accessed 2017 Sep 11]. http://www.oecd.org/chemicalsafety/testing/ Groves, C., 2013. Four scenarios for nanotechnologies in the UK, 2011-2020. adverse-outcome-pathways-molecular-screening-and-toxicogenomics.htm. Technol. Anal. Strateg. Manag. 25 (5), 507–526. doi:10.1080/09537325.2013. OECD. 2018. Users’ Handbook Supplement to the Guidance Document 785510. for Developing and Assessing AOPs. Paris. [accessed 2021 Feb 25]. Harrill, J., 2019. Update on Development of Guidance Document(s) for Consistent Report- http://www.oecd.org/oﬃcialdocuments/publicdisplaydocumentpdf/?cote=env/ ing of ‘Omics Data From Various Sources (OECD WPHA meeting). Proceedings of the jm/mono(2016)12&doclanguage=en Meeting Presented at OECD Working Party on Hazard Assessment (WPHA) Summer Environment Directorate, 2005. Guidance Document on the Valida- Meeting, Paris, France June 17–21 doi:10.23645/epacomptox.8855936. tion and International Acceptance of New or Updated Test Meth- Hartung, T., Bremer, S., Casati, S., Coecke, S., Corvi, R., Fortaner, S., Gribaldo, L., ods for Hazard Assessment. OECD, Paris [accessed 2017 Sep 25] Halder, M., Hoﬀmann, S., Roi, A.J., et al., 2004. A modular approach to the ECVAM http://www.oecd.org/oﬃcialdocuments/publicdisplaydocumentpdf/?doclanguage= principles on test validity. Altern. Lab. Anim. ATLA 32 (5), 467–472. en&cote=env/jm/mono(2005)14 . Iannaccone, P.M., 2001. Toxicogenomics:" the call of the wild chip". Environ. Health Per- OECD. Environment Directorate. 2018. Users’ Handbook Supplement to the Guidance Doc- spect. 109 (1), A8. ument for Developing and Assessing AOPs. Paris: OECD Series on Testing & Assess- ICCVAM. 2018. A strategic roadmap for establishing new approaches to evaluate the ment No. 233 Report No.: ENV/JM/MONO(2016)12. safety of chemicals and medical products in the United States. [accessed 2018 Pain, G., Hickey, G., Mondou, M., Crump, D., Hecker, M., Basu, N., Maguire, S., 2020. Jan 31]. https://ntp.niehs.nih.gov/iccvam/docs/roadmap/iccvam_strategicroadmap_ Drivers of and obstacles to the adoption of toxicogenomics for chemical risk assess- january2018_document_508.pdf. ment: insights from social science perspectives. Environ. Health Perspect. 128 (10), Kavlock, R.J., Bahadori, T., Barton-Maclaren, T.S., Gwinn, M.R., Rasenberg, M., 105002. Thomas, R.S., 2018. Accelerating the pace of chemical risk assessment. Chem. Res. Pattberg, P., Stripple, J., 2008. Beyond the public and private divide: remapping transna- Toxicol. 31 (5), 287–290. doi:10.1021/acs.chemrestox.7b00339. tional climate governance in the 21st century. Int. Environ. Agreem. Polit. Law Econ. Keeney, S., McKenna, H., Hasson, F., 2010. The Delphi Technique in Nursing and Health 8 (4), 367–388. doi:10.1007/s10784-008-9085-3. Research. John Wiley & Sons. Plummer, R., Armitage, D.R., 2007. Charting the new territory of adaptive co-manage- Keohane, R.O., Victor, D.G., 2011. The regime complex for climate change. Perspoli Per- ment: a Delphi study. Ecol. Soc. 12 (2), 10. spect. Polit. 9 (1), 7–23. Schulz, V., Hagen, N., Kozell, K., Martins, L., Tonkin, K., Biondo, P., Stiles, C., 2008. Khodyakov, D., Grant, S., Denger, B., et al., 2020. Practical considerations in using on- Malignant wound assessment tool (MWAT): the Delphi process validation study. J. line modiﬁed-Delphi approaches to engage patients and other stakeholders in clinical Palliat. Care 24 (3), 207. practice guideline development. Patient 13, 11–21. Sismondo, S., 2010. An Introduction to Science and Technology Studies. Wiley-Blackwell, Klenk, N.L., Hickey, G.M., 2011. A virtual and anonymous, deliberative and analytic par- Chichester, U.K.. ticipation process for public policy planning and decision-making: the concept map- Slota, S.C., Bowker, G.C., 2017. How Infrastructure Matters. In: Felt, U, Fouché, R, ping policy Delphi. Int. J. Forecast. 27 (1), 152–165. Miller, CA, Smith-doerr, L, Law, J (Eds.), The Handbook of Science and Technology Klenk, N.L., Hickey, G.M., 2012. Improving the social robustness of research networks for Studies. MIT Press, Cambridge, Massachusetts, pp. 529–554. sustainable natural resource management: results of a Delphi study in Canada. Sci. Proceedings of a Workshop in Brief Standing Committee on Emerging Science for Envi- Public Policy 39 (3), 357–372. doi:10.1093/scipol/scs024. ronmental Health Decisions, Board on Life Sciences, Board on Environmental Studies Krewski, D., Jr, D.A., Andersen, M., Anderson, H., JCB, I.I.I., Boekelheide, K., Brent, R., and Toxicology, Division on Earth and Life Studies, National Academies of Sciences, Charnley, G., Cheung, V.G., Jr, S.G., et al., 2010. Toxicity testing in the 21st cen- Engineering, and Medicine, 2018. In: Fitzpatrick, J (Ed.), Understanding Pathways to tury: a vision and a strategy. J. Toxicol. Environ. Health Part B 13 (2–4), 51–138. a Paradigm Shift in Toxicity Testing and Decision-Making. National Academies Press, doi:10.1080/10937404.2010.483176. Washington, D.C.. Proceedings of a Workshop in Briefeditor[accessed 2018 Jun 1]. Landrigan, P.J., Schechter, C.B., Lipton, J.M., Fahs, M.C., Schwartz, J., 2002. Environ- Star, S.L., Ruhleder, K., 1996. Steps toward an ecology of infrastructure: design and access mental pollutants and disease in American children: estimates of morbidity, mortality, for large information spaces. Inf. Syst. Res. 7 (1), 111–134. doi:10.1287/isre.7.1.111. and costs for lead poisoning, asthma, cancer, and developmental disabilities. Environ. Tickner, J., Jacobs, M., Malloy, T., Buck, T., Stone, A., Blake, A., Ed- Health Perspect. 110 (7), 721–728. doi:10.1289/ehp.02110721. wards, S., 2018. Advancing alternatives assessment for safer chemi- Latour, B., 1987. Science in Action: How to Follow Scientists and Engineers Through So- cal substitution: a research and practice agenda. Integr. Environ. As- ciety. Harvard University Press, Cambridge, Mass. sess. Manag. doi:10.1002/ieam.4094, pre-print[accessed 2018 Aug 23] Lillicrap, A., Belanger, S., Burden, N., Du Pasquier, D., Embry, M.R., Halder, M., https://setac.onlinelibrary.wiley.com/doi/abs/10.1002/ieam.4094. Lampi, M.A., Lee, L., Norberg-King, T., Rattner, B.A., Schirmer, K., 2016. Alternative Tudisca, V., Castellani, T., Aro, A.R., Stahl, T., Van de Goor, I., Radl-Karimi, C., Spitters, H., approaches to vertebrate ecotoxicity tests in the 21st century: a review of develop- Syed, A.M., Rus, D., Roelofs, S., et al., 2016. REPOPA indicators for evidence-informed ments over the last 2 decades and current status. Environ. Toxicol. Chem. 35 (11), policy making validated by an international Delphi study. Eur. J. Public Health 26, 2637–2646. 115. Linstone, H.A., Turoﬀ, M., 2002. The Delphi Method: Techniques and Applications. Ad- US EPA, 2018. Strategic Plan to Promote the Development and Implementation of Alter- dison-Wesley Pub. Co., Reading, Mass Advanced Book Program. native Test Methods within the TSCA Program. US EPA. Oﬃce of Chemical Safety and Mallard, A., 1998. Compare, standardize and settle agreement on some usual metrological Pollution Prevention Report No.: EPA-740-R1-8004, Washington, D.C.. problems. Soc. Stud. Sci. 28 (4), 571–601. US EPA. 2020. New Approach Methods Work Plan: Reducing Markmann, C., Darkow, I.-L., von der Gracht, H., 2013. A Delphi-based risk analy- use of animals in chemical testing. [Accessed 2021 Feb 25]. sis - Identifying and assessing future challenges for supply chain security in a https://www.epa.gov/sites/production/ﬁles/2020-06/documents/epa_nam_work_ multi-stakeholder environment. Technol. Forecast Soc. Change 80 (9), 1815–1833. plan.pdf. doi:10.1016/j.techfore.2012.10.019. Vachon, J., Campagna, C., Rodriguez, M.J., Sirard, M.-A., Levallois, P., 2017. Bar- Martin, M.T., Knudsen, T.B., Judson, R.S., Kavlock, R.J., Dix, D.J., 2012. Eco- riers to the use of toxicogenomics data in human health risk assessment: a nomic beneﬁts of using adaptive predictive models of reproductive toxicity in survey of Canadian risk assessors. Regul. Toxicol. Pharmacol. 85, 119–123. doi:10.1016/j.yrtph.2017.01.008. 12
M. Mondou, S. Maguire, G. Pain et al. Environmental Advances 4 (2021) 100061 Wentholt, M.T.A., Rowe, G., Koenig, A., Marvin, H.J.P., Frewer, L.J., 2009. The views Zeiger, E., 1999. Federal interagency activities toward validation and regulatory accep- of key stakeholders on an evolving food risk governance framework: results from a tance of alternative tests. In: Toxicity Assessment Alternatives: Methods, Issues, Op- Delphi study. Food Policy 34 (6), 539–548. doi:10.1016/j.foodpol.2009.06.002. portunities. Springer, pp. 247–256 [accessed 2017 Jun 19]. Wipﬂi, H.L., Berman, M., Hanson, K., Kelder, S., Solis, A., Villanti, A.C., Ribeiro, C.M.P., Zelli, F., Gupta, A., van Asselt, H., 2013. Institutional interactions at the crossroads of Meissner, H.I., Anderson, R., 2017. Deﬁning tobacco regulatory science competencies. trade and environment: the dominance of liberal environmentalism? Glob. Gov. Rev. Nicotine Tob. Res. 19 (2), 222–230. doi:10.1093/ntr/ntw178. Multilateralism Int. Organ 19 (1), 105–118. doi:10.5555/1075-2846-19.1.105. Zaunbrecher V., Beryt E., Telesca D., Doherty J., Malloy T., Allard P. 2017. Probing the Barriers and Drivers behind Toxicity Testing into the 21st Century: A Survey of Pro- fessionals in the Field of Toxicology. 13