Search

The interpretability of machine learning models, even though with an excellent prediction performance, remains a challenge in practical applications. The model interpretability and variable importance for well-performed supervised machine learning models are investigated in this study. With the commonly accepted concept of odds ratio (OR), we propose a novel and computationally efficient Variable Importance evaluation framework based on the Personalized Odds Ratio (VIPOR). It is a model-agnostic interpretation method that can be used to evaluate variable importance both locally and globally. Locally, the variable importance is quantified by the personalized odds ratio (POR), which can account for subject heterogeneity in machine learning. Globally, we utilize a hierarchical tree to group the predictors into five groups: completely positive, completely negative, positive dominated, negative dominated, and neutral groups. The relative importance of predictors within each group is ranked based on different statistics of PORs across subjects for different application purposes. For illustration, we apply the proposed VIPOR method to interpreting a multilayer perceptron (MLP) model, which aims to predict the mortality of subarachnoid hemorrhage (SAH) patients using real-world electronic health records (EHR) data. We compare the important variables derived from MLP with other machine learning models, including tree-based models and the L1-regularized logistic regression model. The top importance variables are consistently identified by VIPOR across different prediction models. Comparisons with existing interpretation methods are also conducted and discussed based on publicly available data sets.

Currently, methods for conducting multiple treatment propensity scoring in the presence of high-dimensional covariate spaces that result from ‘big data’ are lacking – the most prominent method relies on inverse probability treatment weighting (IPTW). However, IPTW only utilizes one element of the generalized propensity score (GPS) vector, which can lead to a loss of information and inadequate covariate balance in the presence of multiple treatments. This limitation motivates the development of a novel propensity score method that uses the entire GPS vector to establish a scalar balancing score that, when adjusted for, achieves covariate balance in the presence of potentially high-dimensional covariates. Specifically, the generalized propensity score cumulative distribution function (GPS-CDF) method is introduced. A one-parameter power function fits the CDF of the GPS vector and a resulting scalar balancing score is used for matching and/or stratification. Simulation results show superior performance of the new method compared to IPTW both in achieving covariate balance and estimating average treatment effects in the presence of multiple treatments. The proposed approach is applied to a study derived from electronic medical records to determine the causal relationship between three different vasopressors and mortality in patients with non-traumatic aneurysmal subarachnoid hemorrhage. Results suggest that the GPS-CDF method performs well when applied to large observational studies with multiple treatments that have large covariate spaces.

Differential equations are extensively used for modeling dynamics of physical processes in many scientific fields such as engineering, physics, and biomedical sciences. Parameter estimation of differential equation models is a challenging problem because of high computational cost and high-dimensional parameter space. In this paper, we propose a novel class of methods for estimating parameters in ordinary differential equation (ODE) models, which is motivated by HIV dynamics modeling. The new methods exploit the form of numerical discretization algorithms for an ODE solver to formulate estimating equations. First a penalized-spline approach is employed to estimate the state variables and the estimated state variables are then plugged in a discretization formula of an ODE solver to obtain the ODE parameter estimates via a regression approach. We consider three different order of discretization methods, Euler’s method, trapezoidal rule and Runge-Kutta method. A higher order numerical algorithm reduces numerical error in the approximation of the derivative, which produces a more accurate estimate, but its computational cost is higher. To balance the computational cost and estimation accuracy, we demonstrate, via simulation studies, that the trapezoidal discretization-based estimate is the best and is recommended for practical use. The asymptotic properties for the proposed numerical discretization-based estimators (DBE) are established. Comparisons between the proposed methods and existing methods show a clear benefit of the proposed methods in regards to the trade-off between computational cost and estimation accuracy. We apply the proposed methods to an HIV study to further illustrate the usefulness of the proposed approaches.

Studies on HIV dynamics in AIDS research are very important in understanding the pathogenesis of HIV-1 infection and also in assessing the effectiveness of antiretroviral (ARV) treatment. Viral dynamic models can be formulated through a system of nonlinear ordinary differential equations (ODE), but there has been only limited development of statistical methodologies for inference. This article, motivated by an AIDS clinical study, discusses a hierarchical Bayesian nonlinear mixed-effects modeling approach to dynamic ODE models without a closed-form solution. In this model, we fully integrate viral load, medication adherence, drug resistance, pharmacokinetics, baseline covariates and time-dependent drug efficacy into the data analysis for characterizing long-term virologic responses. Our method is implemented by a data set from an AIDS clinical study. The results suggest that modeling HIV dynamics and virologic responses with consideration of time-varying clinical factors as well as baseline characteristics may be important for HIV/AIDS studies in providing quantitative guidance to better understand the virologic responses to ARV treatment and to help the evaluation of clinical trial design in response to existing therapies.

In this paper, the mechanism-based ordinary differential equation (ODE) model and the flexible semiparametric regression model are employed to identify the significant covariates for antiretroviral response in AIDS clinical trials. We consider the treatment effect as a function of three factors (or covariates) including pharmacokinetics, drug adherence and susceptibility. Both clinical and simulated data examples are given to illustrate these two different kinds of modeling approaches. We found that the ODE model is more powerful to model the mechanism-based nonlinear relationship between treatment effects and virological response biomarkers. The ODE model is also better in identifying the significant factors for virological response, although it is a little bit liberal and there is a trend to include more factors (or covariates) in the model. The semiparametric mixed-effects regression model is very flexible to fit the virological response data, but it is too liberal to identify correct factors for virological response; sometimes it may miss the correct factors. The ODE model is also biologically justifiable and good for predictions and simulations for various biological scenarios. The limitations of the ODE models include the high cost of computation and the requirement of biological assumptions that sometimes may not be easy to validate. The methodologies reviewed in this paper are also generally applicable to studies of other viruses such as hepatitis B virus (HBV) or hepatitis C virus (HCV).

HIV dynamics studies have significantly contributed to the understanding of HIV infection and antiviral treatment strategies. But most studies are limited to short-term viral dynamics due to the difficulty of establishing a relationship of antiviral response with multiple treatment factors such as drug exposure and drug susceptibility during long-term treatment. In this article, a mechanism-based dynamic model is proposed for characterizing long-term viral dynamics with antiretroviral therapy, described by a set of nonlinear differential equations without closed-form solutions. In this model we directly incorporate drug concentration, adherence, and drug susceptibility into a function of treatment efficacy, defined as an inhibition rate of virus replication. We investigate a Bayesian approach under the framework of hierarchical Bayesian (mixed-effects) models for estimating unknown dynamic parameters. In particular, interest focuses on estimating individual dynamic parameters. The proposed methods not only help to alleviate the difficulty in parameter identifiability, but also flexibly deal with sparse and unbalanced longitudinal data from individual subjects. For illustration purposes, we present one simulation example to implement the proposed approach and apply the methodology to a data set from an AIDS clinical trial. The basic concept of the longitudinal HIV dynamic systems and the proposed methodologies are generally applicable to any other biomedical dynamic systems.

In this paper, we consider a semiparametric time-varying coefficients regression model where the influences of some covariates vary non-parametrically with time while the effects of the remaining covariates follow certain parametric functions of time. The weighted least squares type estimators for the unknown parameters of the parametric coefficient functions as well as the esti- mators for the non-parametric coefficient functions are developed. We show that the kernel smoothing that avoids modelling of the sampling times is asymptotically more efficient than a single nearest neighbour smoothing that depends on the estimation of the sampling model. The asymptotic optimal bandwidth is also derived. A hypothesis testing procedure is proposed to test whether some covariate effects follow certain parametric forms. Simulation studies are conducted to compare the finite sample performances of the kernel neighbourhood smoothing and the single nearest neighbour smoothing and to check the empirical sizes and powers of the proposed testing procedures. An application to a data set from an AIDS clinical trial study is provided for illustration.

Mixed censoring is useful extension of Type I and Type II censoring and combines some advantages of both types of censoring. This paper proposes a general Bayesian framework for designing a variable acceptance sampling scheme with mixed censoring. A general loss function which includes the sampling cost, the time-consuming cost, the salvage value, and the decision loss is employed to determine the Bayes risk and the corresponding optimal sampling plan. An explicit expression of the Bayes risk is derived. The new model can easily be adapted to create life testing models for different distributions. Specifically, two commonly used distributions including the exponential distribution and the Weibull distribution are considered with a special decision loss function. We demonstrate that the proposed model is superior to models with Type I or Type II censoring. Numerical examples are reported to illustrate the effectiveness of the method proposed. © 2004 Wiley Periodicals, Inc. Naval Research Logistics, 2004

The potency of antiretroviral agents in AIDS clinical trials can be assessed on the basis of an early viral response such as viral decay rate or change in viral load (number of copies of HIV RNA) of the plasma. Linear, parametric nonlinear, and semiparametric nonlinear mixed-effects models have been proposed to estimate viral decay rates in viral dynamic models. However, before applying these models to clinical data, a critical question that remains to be addressed is whether these models produce coherent estimates of viral decay rates, and if not, which model is appropriate and should be used in practice. In this paper, we applied these models to data from an AIDS clinical trial of potent antiviral treatments and found significant incongruity in the estimated rates of reduction in viral load. Simulation studies indicated that reliable estimates of viral decay rate were obtained by using the parametric and semiparametric nonlinear mixed-effects models. Our analysis also indicated that the decay rates estimated by using linear mixed-effects models should be interpreted differently from those estimated by using nonlinear mixed-effects models. The semiparametric nonlinear mixed-effects model is preferred to other models because arbitrary data truncation is not needed. Based on real data analysis and simulation studies, we provide guidelines for estimating viral decay rates from clinical data. (© 2004 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)

In this paper, we propose a random varying-coefficient model for longitudinal data. This model is different from the standard varying-coefficient model in the sense that the time- varying coefficients are assumed to be subject-specific, and can be considered as realizations of stochastic processes. This modelling strategy allows us to employ powerful mixed-effects modelling techniques to efficiently incorporate the within-subject and between-subject variations in the esti- mators of time-varying coefficients. Thus, the subject-specific feature of longitudinal data is effectively considered in the proposed model. A backfitting algorithm is proposed to estimate the coefficient functions. Simulation studies show that the proposed estimation methods are more efficient in finite-sample performance compared with the standard local least squares method. An application to an AIDS clinical study is presented to illustrate the proposed methodologies.

The study of HIV dynamics is one of the most important developments in recent AIDS research. It greatly improves our understanding of the pathogenesis of HIV infection. Recently it has been proposed to use HIV dynamics to evaluate the efficacy of antiviral treatments. Currently a large number of AIDS clinical trials on HIV dynamics are in development worldwide. However, many design issues that arise from HIV dynamic studies have not been addressed. In this paper, we study these problems using intensive Monte Carlo simulations and analytic methods. We evaluate a finite number of feasible candidate designs, which are currently used and proposed in AIDS clinical trials from different perspectives. We compare the viral dynamic marker and classical viral load change markers in terms of power for identifying treatment difference, asymptotic relative efficiency, and sensitivity. Finally we propose some useful suggestions for practitioners based on our results.

Non-linear mixed-effects models are powerful tools for modelling HIV viral dynamics. In AIDS clinical trials, the viral load measurements for each subject are often sparse. In such cases, linearization procedures are usually used for inferences. Under such linearization procedures, however, standard covariate selection methods based on the approximate likelihood, such as the likelihood ratio test, may not be reliable. In order to identify significant host factors for HIV dynamics, in this paper we consider two alternative approaches for covariate selection: one is based on individual non-linear least square estimates and the other is based on individual empirical Bayes estimates. Our simulation study shows that, if the within-individual data are sparse and the between-individual variation is large, the two alternative covariate selection methods are more reliable than the likelihood ratio test, and the more powerful method based on individual empirical Bayes estimates is especially preferable. We also consider the missing data in covariates. The commonly used missing data methods may lead to misleading results. We recommend a multiple imputation method to handle missing covariates. A real data set from an AIDS clinical trial is analysed based on various covariate selection methods and missing data methods.

We propose a three-step multiple imputation method, implemented by Gibbs sampler, for estimating parameters in non-linear mixed-effects models with missing covariates. Estimates obtained by the proposed multiple imputation method are compared to those obtained by the mean-value imputation method and the complete-case method through simulations. We find that the proposed multiple imputation method offers smaller biases and smaller mean-squared errors for the estimates of covariate coefficients compared to other two methods. We apply the three missing data methods to modelling HIV viral dynamics from an AIDS clinical trial. We believe that the results from the proposed multiple imputation method are more reliable than that from the other two commonly used methods.

Summary. The study of HIV dynamics is one of the most important developments in recent AIDS research. It has led to a new understanding of the pathogenesis of HIV infection. But, although important findings in HIV dynamics have been published in prestigious scientific journals in the last 5 years, the model-fitting procedures used in these publications have not been studied in any detail. In this paper, we evaluate the performance of four model-fitting procedures proposed and used in biphasic HIV dynamic data analysis via extensive Monte Carlo simulations. We propose some guidelines for practitioners to select an appropriate method for their own data analysis. Real data examples from an AIDS clinical trial are provided as illustrations.

In this paper, we introduce a novel application of hierarchical nonlinear mixed-effect models to HIV dynamics. We show that a simple model with a sum of exponentials can give a good fit to the observed clinical data of HIV-1 dynamics (HIV-1 RNA copies) after initiation of potent antiviral treatments and can also be justified by a biological compartment model for the interaction between HIV and its host cells. This kind of model enjoys both biological interpretability and mathematical simplicity after reparameterization and simplification. A model simplification procedure is proposed and illustrated through examples. We interpret and justify various simplified models based on clinical data taken during different phases of viral dynamics during antiviral treatments. We suggest the hierarchical nonlinear mixed-effect model approach for parameter estimation and other statistical inferences. In the context of an AIDS clinical trial involving patients treated with a combination of potent antiviral agents, we show how the models may be used to draw biologically relevant interpretations from repeated HIV-1 RNA measurements and demonstrate the potential use of the models in clinical decision-making.

Investigation of HIV viral dynamics is important for understanding the HIV pathogenesis and for development of treatment strategies. Perelson et al. demonstrated that simple viral dynamic models fit to data on viral load as measured by plasma HIV-RNA could produce estimates of rates of clearance of virus and of infected CD4+ T-lymphocytes. In this paper we extend the work of Perelson et al. by proposing models with less restrictive assumptions about drug activity. Our models take into account the fact that infectious and non-infectious virions are produced by infected T-cells both before and after the treatment. We also show that direct measurement of infectious virus load provides sufficient information for estimation of antiretroviral drug efficacy parameter. For characterizing viral dynamics of populations and estimation of dynamic parameters, we propose a hierarchical non-linear model. Compared to other methods such as the non-linear least square method used by Perelson et al., we show that the proposed approach has the following advantages: (i) it is more appropriate for modelling within-patient and between-patient variation and to characterize the population dynamics; (ii) it is flexible enough to deal with both rich and sparse individual data; (iii) it has more power to detect model misspecification; (iv) it allows incorporation of covariates for viral dynamic parameters; (v) it makes more efficient use of between-subject information to get better parameter estimates. We give two simulation examples to illustrate the proposed approach and its advantages. Finally, we discuss practical issues regarding the clinical trial design for viral dynamic studies.