Search

Acceptance of new approach methodologies (NAMs) for use in characterizing chemical hazard and risk requires informed expectations regarding the minimum precision and maximum accuracy of their results. Uncertainty in NAMs derived from variability in the traditional reference data used to train or validate performance of the NAM, and uncertainty in the modeling procedures themselves, limits NAM performance. Herein, we review current approaches to characterizing uncertainty in NAMs. We discuss variability in in vivo data used as a reference for NAM development and validation; the quantitative uncertainty in concentration–response modeling for high-throughput in vitro bioactivity screening; the uncertainties associated with in vitro to in vivo extrapolation using toxicokinetic information; and the quantitative uncertainty in the experimental inputs and modeled outputs from quantitative structure activity relationship models for prediction of point of departure doses. Communication of the amount of uncertainty, both in the input and output for NAMs, often involves a confidence or prediction interval around a given potency estimate, derived from an understanding of the variability in the data modeled. Tuning expectations of NAM performance to an understanding of the reproducibility and variability, both of traditional approaches and NAM approaches, provides a path for the adoption of NAMs as alternatives in screening chemicals for risk.

Regulatory agencies world-wide face the challenge of performing risk-based prioritization of thousands of substances in commerce. In this study, a major effort was undertaken to compile a large genotoxicity dataset (54,805 records for 9299 substances) from several public sources (e.g., TOXNET, COSMOS, eChemPortal). The names and outcomes of the different assays were harmonized, and assays were annotated by type: gene mutation in Salmonella bacteria (Ames assay) and chromosome mutation (clastogenicity) in vitro or in vivo (chromosome aberration, micronucleus, and mouse lymphoma Tk+/− assays). This dataset was then evaluated to assess genotoxic potential using a categorization scheme, whereby a substance was considered genotoxic if it was positive in at least one Ames or clastogen study. The categorization dataset comprised 8442 chemicals, of which 2728 chemicals were genotoxic, 5585 were not and 129 were inconclusive. QSAR models (TEST and VEGA) and selected OECD Toolbox structural alerts/profilers (e.g., OASIS DNA alerts for Ames and chromosomal aberrations) were used to make in silico predictions of genotoxicity potential. The performance of the individual QSAR tools and structural alerts resulted in balanced accuracies of 57–73%. A Naive Bayes consensus model was developed using combinations of QSAR models and structural alert predictions. The ‘best’ consensus model selected had a balanced accuracy of 81.2%, a sensitivity of 87.24% and a specificity of 75.20%. This in silico scheme offers promise as a first step in ranking thousands of substances as part of a prioritization approach for genotoxicity.

BACKGROUND: Humans are exposed to tens of thousands of chemical substances that need to be assessed for their potential toxicity. Acute systemic toxicity testing serves as the basis for regulatory hazard classification, labeling, and risk management. However, it is cost- and time-prohibitive to evaluate all new and existing chemicals using traditional rodent acute toxicity tests. In silico models built using existing data facilitate rapid acute toxicity predictions without using animals. OBJECTIVES: The U.S. Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) Acute Toxicity Workgroup organized an international collaboration to develop in silico models for predicting acute oral toxicity based on five different end points: Lethal Dose 50 (LD50 value, U.S. Environmental Protection Agency hazard (four) categories, Globally Harmonized System for Classification and Labeling hazard (five) categories, very toxic chemicals [LD50 (LD50≤50mg/kg)], and nontoxic chemicals (LD50>2,000mg/kg). METHODS: An acute oral toxicity data inventory for 11,992 chemicals was compiled, split into training and evaluation sets, and made available to 35 participating international research groups that submitted a total of 139 predictive models. Predictions that fell within the applicability domains of the submitted models were evaluated using external validation sets. These were then combined into consensus models to leverage strengths of individual approaches. RESULTS: The resulting consensus predictions, which leverage the collective strengths of each individual model, form the Collaborative Acute Toxicity Modeling Suite (CATMoS). CATMoS demonstrated high performance in terms of accuracy and robustness when compared with in vivo results. DISCUSSION: CATMoS is being evaluated by regulatory agencies for its utility and applicability as a potential replacement for in vivo rat acute oral toxicity studies. CATMoS predictions for more than 800,000 chemicals have been made available via the National Toxicology Program's Integrated Chemical Environment tools and data sets (ice.ntp.niehs.nih.gov). The models are also implemented in a free, standalone, open-source tool, OPERA, which allows predictions of new and untested chemicals to be made. https://doi.org/10.1289/EHP8495.

New approach methodologies (NAMs) for chemical hazard assessment are often evaluated via comparison to animal studies; however, variability in animal study data limits NAM accuracy. The US EPA Toxicity Reference Database (ToxRefDB) enables consideration of variability in effect levels, including the lowest effect level (LEL) for a treatment-related effect and the lowest observable adverse effect level (LOAEL) defined by expert review, from subacute, subchronic, chronic, multi-generation reproductive, and developmental toxicity studies. The objectives of this work were to quantify the variance within systemic LEL and LOAEL values, defined as potency values for effects in adult or parental animals only, and to estimate the upper limit of NAM prediction accuracy. Multiple linear regression (MLR) and augmented cell means (ACM) models were used to quantify the total variance, and the fraction of variance in systemic LEL and LOAEL values explained by available study descriptors (e.g., administration route, study type). The MLR approach considered each study descriptor as an independent contributor to variance, whereas the ACM approach combined categorical descriptors into cells to define replicates. Using these approaches, total variance in systemic LEL and LOAEL values (in log(10)-mg/kg/day units) ranged from 0.74 to 0.92. Unexplained variance in LEL and LOAEL values, approximated by the residual mean square error (MSE), ranged from 0.20–0.39. Considering subchronic, chronic, or developmental study designs separately resulted in similar values. Based on the relationship between MSE and R-squared for goodness-of-fit, the maximal R-squared may approach 55 to 73% for a NAM-based predictive model of systemic toxicity using these data as reference. The root mean square error (RMSE) ranged from 0.47 to 0.63 log(10)-mg/kg/day, depending on dataset and regression approach, suggesting that a two-sided minimum prediction interval for systemic effect levels may have a width of 58 to 284-fold. These findings suggest quantitative considerations for building scientific confidence in NAM-based systemic toxicity predictions.

Poster for the Society of Toxicology annual meeting March 2021, March 13-26 2021

The toxicokinetic (TK) parameters fraction of the chemical unbound to plasma proteins and metabolic clearance are critical for relating exposure and internal dose when building in vitro-based risk assessment models. However, experimental toxicokinetic studies have only been carried out on limited chemicals of environmental interest (~1000 chemicals with TK data relative to tens of thousands of chemicals of interest). This work evaluated the utility of chemical structure information to predict TK parameters in silico; development of cluster-based read-across and quantitative structure-activity relationship models of fraction unbound or fub (regression) and intrinsic clearance or Cl(int) (classification and regression) using a dataset of 1487 chemicals; utilization of predicted TK parameters to estimate uncertainty in steady-state plasma concentration (C(ss)); and subsequent in vitro–in vivo extrapolation analyses to derive bioactivity-exposure ratio (BER) plot to compare human oral equivalent doses and exposure predictions using androgen and estrogen receptor activity data for 233 chemicals as an example dataset. The results demonstrate that fub is structurally more predictable than Cl(int). The model with the highest observed performance for fub had an external test set RMSE/σ=0.62 and R(2)=0.61, for Cl(int) classification had an external test set accuracy = 65.9%, and for intrinsic clearance regression had an external test set RMSE/σ=0.90 and R(2)=0.20. This relatively low performance is in part due to the large uncertainty in the underlying Cl(int) data. We show that C(ss) is relatively insensitive to uncertainty in Cl(int). The models were benchmarked against the ADMET Predictor software. Finally, the BER analysis allowed identification of 14 out of 136 chemicals for further risk assessment demonstrating the utility of these models in aiding risk-based chemical prioritization.

Read-across is an important data gap filling technique used within category and analog approaches for regulatory hazard identification and risk assessment. Although much technical guidance is available that describes how to develop category/analog approaches, practical principles to evaluate and substantiate analog validity (suitability) are still lacking. This case study uses hindered phenols as an example chemical class to determine: (1) the capability of three structure fingerprint/descriptor methods (PubChem, ToxPrints and MoSS MCSS) to identify analogs for read-across to predict Estrogen Receptor (ER) binding activity and, (2) the utility of data confidence measures, physicochemical properties, and chemical R-group properties as filters to improve ER binding predictions. The training dataset comprised 462 hindered phenols and 257 non- hindered phenols. For each chemical of interest (target), source analogs were identified from two datasets (hindered and non-hindered phenols) that had been characterized by a fingerprint/descriptor method and by two cut-offs: (1) minimum similarity distance (range: 0.1 – 0.9) and, (2) N closest analogs (range: 1 – 10). Analogs were then filtered using: (1) physicochemical properties of the phenol (termed global filtering) and, (2) physicochemical properties of the R-groups neighboring the active hydroxyl group (termed local filtering). A read-across prediction was made for each target chemical on the basis of a majority vote of the N closest analogs. The results demonstrate that: (1) concordance in ER activity increases with structural similarity, regardless of the structure fingerprint/descriptor method, (2) increased data confidence significantly improves read-across predictions, and (3) filtering analogs using global and local properties can help identify more suitable analogs. This case study illustrates that the quality of the underlying experimental data and use of endpoint relevant chemical descriptors to evaluate source analogs are critical to achieving robust read-across predictions.

Silver nanoparticles (AgNP) are incorporated into medical devices for their anti-microbial characteristics. The potential exposure and toxicity of AgNPs is unknown due to varying physicochemical particle properties and lack of toxicological data. The aim of this safety assessment is to derive a provisional tolerable intake (pTI) value for AgNPs released from blood-contacting medical devices. A literature review of in vivo studies investigating critical health effects induced from intravenous (i. v.) exposure to AgNPs was evaluated by the Annapolis Accords principles and Toxicological Data Reliability Assessment Tool (ToxRTool). The point of departure (POD) was based on an i. v. 28-day repeated AgNP (20 nm) dose toxicity study reporting an increase in relative spleen weight in rats with a 5% lower confidence bound of the benchmark dose (BMDL05) of 0.14 mg/kg bw/day. The POD was extrapolated to humans by a modifying factor of 1,000 to account for intraspecies variability, interspecies differences and lack of long-term toxicity data. The pTI for long-term i. v. exposure to 20 nm AgNPs released from blood-contacting medical devices was 0.14 μg/kg bw/day. This pTI may not be appropriate for nanoparticles of other physicochemical properties or routes of administration. The methodology is appropriate for deriving pTIs for nanoparticles in general.

Poster presented to the International Society of Exposure Scientists annual meeting September 2020

Poster presented at the American Society for Cellular and Computational Toxicology (ASCCT) annual meeting in September 2019.

Poster submitted to the Society of Toxicology annual meeting March 2020

Traditional approaches for chemical risk assessment cannot keep pace with the number of substances requiring assessment. Thus, in a global effort to expedite and modernize chemical risk assessment, New Approach Methodologies (NAMs) are being explored and developed. Included in this effort is the OECD Integrated Approaches for Testing and Assessment (IATA) program, which provides a forum for OECD member countries to develop and present case studies illustrating the application of NAM in various risk assessment contexts. Here, we present an IATA case study for the prediction of estrogenic potential of three target phenols: 4-tert-butylphenol, 2,4-di-tert-butylphenol and octabenzone. Key features of this IATA include the use of two computational approaches for analogue selection for read-across, data collected from traditional and NAM sources, and a workflow to generate predictions regarding the targets' ability to bind the estrogen receptor (ER). Endocrine disruption can occur when a chemical substance mimics the activity of natural estrogen by binding to the ER and, if potency and exposure are sufficient, alters the function of the endocrine system to cause adverse effects. The data indicated that of the three target substances that were considered herein, 4-tert-butylphenol is a potential endocrine disruptor. Further, this IATA illustrates that the NAM approach explored is health protective when compared to in vivo endpoints traditionally used for human health risk assessment.

Endocrine-disrupting chemicals have the ability to interfere with and alter functions of the hormone system, leading to adverse effects on reproduction, growth and development. Despite growing concerns over their now ubiquitous presence in the environment, endocrine-related human health effects remain largely outside of comparative human toxicity characterization frameworks as applied for example in life cycle impact assessments. In this paper, we propose a new methodological framework to consistently integrate endocrine-related health effects into comparative human toxicity characterization. We present two quantitative and operational approaches for extrapolating towards a common point of departure from both in vivo and dosimetry-adjusted in vitro endocrine-related effect data and deriving effect factors as well as corresponding characterization factors for endocrine-active/endocrine-disrupting chemicals. Following the proposed approaches, we calculated effect factors for 323 chemicals, reflecting their endocrine potency, and related characterization factors for 157 chemicals, expressing their relative endocrine-related human toxicity potential. Developed effect and characterization factors are ready for use in the context of chemical prioritization and substitution as well as life cycle impact assessment and other comparative assessment frameworks. Endocrine-related effect factors were found comparable to existing effect factors for cancer and non-cancer effects, indicating that (1) the chemicals' endocrine potency is not necessarily higher or lower than other effect potencies and (2) using dosimetry-adjusted effect data to derive effect factors does not consistently overestimate the effect of potential endocrine disruptors. Calculated characterization factors span over 8-11 orders of magnitude for different substances and emission compartments and are dominated by the range in endocrine potencies.

Human health risk assessment for environmental chemical exposure is limited by a vast majority of chemicals with little or no experimental in vivo toxicity data. Data gap filling techniques, such as quantitative structure activity relationship (QSAR) models based on chemical structure information, can predict hazard in the absence of experimental data. Risk assessment requires identification of a quantitative point-of-departure (POD) value, the point on the dose-response curve that marks the beginning of a low-dose extrapolation. This study presents two sets of QSAR models to predict POD values (POD(QSAR)) for repeat dose toxicity. For training and validation, a publicly available in vivo toxicity dataset for 3592 chemicals was compiled using the U.S. Environmental Protection Agency’s Toxicity Value database (ToxValDB). The first set of QSAR models predict point-estimates of POD values (POD(QSAR)) using structural and physicochemical descriptors for repeat dose study types and species combinations. A random forest QSAR model using study type and species as descriptors showed the best performance, with an external test set root mean square error (RMSE) of 0.71 log(10)-mg/kg/day and coefficient of determination (R(2)) of 0.53. The second set of QSAR models predict the 95% confidence intervals for POD(QSAR) using a constructed POD distribution with a mean equal to the median POD value and a standard deviation of 0.5 log(10)-mg/kg/day, based on previously published typical study-to-study variability that may lead to uncertainty in model predictions. Bootstrap resampling of the pre-generated POD distribution was used to derive point-estimates and 95% confidence intervals for each POD prediction. Enrichment analysis to evaluate the accuracy of POD(QSAR) showed that 80% of the 5% most potent chemicals were found in the top 20% of the most potent chemical predictions, suggesting that the repeat dose POD QSAR models presented here may help inform screening level human health risk assessments in the absence of other data.

New approach methodologies (NAMs) for chemical hazard assessment are often evaluated via comparison to animal studies; however, variability in animal study data limits NAM accuracy. The US EPA Toxicity Reference Database (ToxRefDB) enables consideration of variability in effect levels, including the lowest effect level (LEL) for a treatment-related effect and the lowest observable adverse effect level (LOAEL) defined by expert review, from subacute, subchronic, chronic, multi-generation reproductive, and developmental toxicity studies. The objectives of this work were to quantify the variance within systemic LEL and LOAEL values, defined as potency values for effects in adult or parental animals only, and to estimate the upper limit of NAM prediction accuracy. Multiple linear regression (MLR) and augmented cell means (ACM) models were used to quantify the total variance, and the fraction of variance in systemic LEL and LOAEL values explained by available study descriptors (e.g., administration route, study type). The MLR approach considered each study descriptor as an independent contributor to variance, whereas the ACM approach combined categorical descriptors into cells to define replicates. Using these approaches, total variance in systemic LEL and LOAEL values (in log10-mg/kg/day units) ranged from 0.74 to 0.92. Unexplained variance in LEL and LOAEL values, approximated by the residual mean square error (MSE), ranged from 0.20-0.39. Considering subchronic, chronic, or developmental study designs separately resulted in similar values. Based on the relationship between MSE and R-squared for goodness-of-fit, the maximal R-squared may approach 55 to 73% for a NAM-based predictive model of systemic toxicity using these data as reference. The root mean square error (RMSE) ranged from 0.47 to 0.63 log10-mg/kg/day, depending on dataset and regression approach, suggesting that a two-sided minimum prediction interval for systemic effect levels may have a width of 58 to 284-fold. These findings suggest quantitative considerations for building scientific confidence in NAM-based systemic toxicity predictions.

Poster presentation at Society of Toxicology annual meeting March 2019

Poster presented at the American Society for Cellular and Computational Toxicology (ASCCT) annual meeting in September 2019

Presentation at the 2019 Society of Toxicology Continuing Education Session

The application of toxic equivalency factors (TEFs) or toxic units to estimate toxic potencies for mixtures of chemicals which contribute to a biological effect through a common mechanism is one approach for filling data gaps. Toxic Equivalents (TEQ) have been used to express the toxicity of dioxin-like compounds (i.e., dioxins, furans, and dioxin-like polychlorinated biphenyls (PCBs)) in terms of the most toxic form of dioxin: 2,3,7,8-tetrachlorodibenzo-p-dioxin (2,3,7,8-TCDD). This study sought to integrate two data gap filling techniques, quantitative structure–activity relationships (QSARs) and TEFs, to predict neurotoxicity TEQs for PCBs. Simon et al. (2007) previously derived neurotoxic equivalent (NEQ) values for a dataset of 87 PCB congeners, of which 83 congeners had experimental data. These data were taken from a set of four different studies measuring different effects related to neurotoxicity, each of which tested overlapping subsets of the 83 PCB congeners. The goals of the current study were to: (i) evaluate an alternative neurotoxic equivalent factor (NEF) derivations from an expanded dataset, relative to those derived by Simon et al., and (ii) develop QSAR models to provide NEF estimates for the large number of untested PCB congeners. The models used multiple linear regression, support vector regression, k-nearest neighbor and random forest algorithms within a 5-fold cross validation scheme. and position-specific chlorine substitution patterns on the biphenyl scaffold as descriptors. Alternative NEF values were derived but the resulting QSAR models had relatively low predictivity (RMSE ~0.24). This was mostly driven by the large uncertainties in the underlying data and NEF values. The derived NEFs and the QSAR predicted NEFs to fill data gaps should be applied with caution.

Multi-city population-based epidemiological studies of short-term fine particulate matter (PM(2.5)) exposures and mortality have observed heterogeneity in risk estimates between cities. Factors affecting exposures, such as pollutant infiltration, which are not captured by central-site monitoring data, can differ between communities potentially explaining some of this heterogeneity. This analysis evaluates exposure factors as potential determinants of the heterogeneity in 312 Core-Based Statistical Areas (CBSA) -specific associations between PM(2.5) and mortality using inverse variance weighted linear regression. Exposure factor variables were created based on data on housing characteristics, commuting patterns, heating fuel usage, and climatic factors from national surveys. When survey data were not available, air conditioning (AC) prevalence was predicted utilizing machine learning techniques. Across all CBSAs, there was a 0.95% (Interquartile range (IQR) of 2.25) increase in non-accidental mortality per 10 μg/m(3) increase in PM(2.5) and significant heterogeneity between CBSAs. CBSAs with larger homes, more heating degree days, a higher percentage of home heating with oil had significantly (p < 0.05) higher health effect estimates, while cities with more gas heating had significantly lower health effect estimates. While univariate models did not explain much of heterogeneity in health effect estimates (R(2) < 1%), multivariate models began to explain some of the observed heterogeneity (R(2) = 13%).

This study introduces a novel technique for advanced image watermarking utilizing recently created nature motivated Firefly Algorithm (FA). The principal focus of this paper is to insert the imperceptible watermark (logo) image into the host image with the end goal that the strength of the image will be extended. In the proposed system, we used two strategies to implant the watermark. First is Discrete Wavelet Transform (DWT) with Singular Value Decomposition (SVD) and another is Discrete Cosine Transform (DCT) with SVD. FA was discovered by Yang which is spurred by flickering behavior of fireflies. Thought behind the FA is firefly with less brightness will be pulled in towards the firefly with more brightness. Structural Similarity Index Measure (SSIM) is utilized to demonstrate the measure of robustness and invisibility of the watermarking scheme.

Molecular docking is a computational technique which predicts the binding energy and the preferred binding mode of a ligand to a protein target. Virtual screening is a tool which uses docking to investigate large chemical libraries to identify ligands that bind favorably to a protein target. We have developed a novel scoring based distributed protein docking application to improve enrichment in virtual screening. The application addresses the issue of time and cost of screening in contrast to conventional systematic parallel virtual screening methods in two ways. Firstly, it automates the process of creating and launching multiple independent dockings on a high performance computing cluster. Secondly, it uses a Nȧive Bayes scoring function to calculate binding energy of un-docked ligands to identify and preferentially dock (Autodock predicted) better binders. The application was tested on four proteins using a library of 10,573 ligands. In all the experiments, (i). 200 of the 1,000 best binders are identified after docking only ${\sim} 14$ percent of the chemical library, (ii). 9 or 10 best-binders are identified after docking only ${\sim} 19$ percent of the chemical library, and (iii). no significant enrichment is observed after docking ${\sim} 70$ percent of the chemical library. The results show significant increase in enrichment of potential drug leads in early rounds of virtual screening.

Many gene expression data are based on two experiments where the gene expressions of the targeted genes under both experiments are correlated. We consider problems in which objectives are to find genes that are simultaneously upregulated/downregulated under both experiments. A Bayesian methodology is proposed based on directional multiple hypotheses testing. We propose a false discovery rate specific to the problem under consideration, and construct a Bayes rule satisfying a false discovery rate criterion. The proposed method is compared with a traditional rule through simulation studies. We apply our methodology to two real examples involving microRNAs; where in one example the targeted genes are simultaneously downregulated under both experiments, and in the other the targeted genes are downregulated in one experiment and upregulated in the other experiment. We also discuss how the proposed methodology can be extended to more than two experiments.

Poster presentation at the North Carolina Society of Toxicology annual meeting Oct 2017

Presentation at the OpenTox USA meeting in Durham, NC July 2017

Poster presentation at the OpenTox meeting July 2017

Presentation at an SOT specialty section webinar on QSAR Nov 2017

Poster presentation at the NC SOT annual meeting Oct 2017

Poster presented at Society of Toxicology annual meeting March 2018

Poster presented at OpenTox USA meeting July 2018

Poster for American Society for Cellular and Computational Toxicology (ASCCT) Meeting Sep 2017

Presentation on IATA case study for phenols which is joint project with Health Canada and NCCT, presented at the OECD IATA Case Study Meeting No. 2 in Paris, France November 2016

The Toxic Substances Control Act (TSCA) mandates the US EPA perform risk-based prioritisation of chemicals in commerce and then, for high-priority substances, develop risk evaluations that integrate toxicity data with exposure information. One approach being considered for data poor chemicals is the Threshold of Toxicological Concern (TTC). Here, TTC values derived using oral (sub)chronic No Observable (Adverse) Effect Level (NO(A)EL) data from the EPA's Toxicity Values database (ToxValDB) were compared with published TTC values from Munro et al. (1996). A total of 4554 chemicals with structures present in ToxValDB were assigned into their respective TTC categories using the Toxtree software tool, of which toxicity data was available for 1304 substances. The TTC values derived from ToxValDB were similar, but not identical to the Munro TTC values: Cramer I ((ToxValDB) 37.3 c. f. (Munro) 30 μg/kg-day), Cramer II (34.6 c. f. 9.1 μg/kg-day) and Cramer III (3.9 c. f. 1.5 μg/kg-day). Cramer III 5th percentile values were found to be statistically different. Chemical features of the two Cramer III datasets were evaluated to account for the differences. TTC values derived from this expanded dataset substantiated the original TTC values, reaffirming the utility of TTC as a promising tool in a risk-based prioritisation approach.

Presented at BoSC

Presented at ASCCT

Presented at the Annual Society of Toxicology meeting

Presented at BoSC

A Systematic Evaluation of Analogs for Read-across Prediction of Estrogenicity

Integration of Different Data-gap Filling Techniques to Facilitate Assessment of Polychlorinated Biphenyls: A Proof of Principle Case Study

Additional file 2. Matlab generated code file and is self-explanatory.