Search

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

Background and purpose: The study’s purpose was to develop and internally validate a prognostic survival model exploring baseline variables for adolescent idiopathic scoliosis curve progression. Methods: A longitudinal prognostic cohort analysis was performed on trial data (n = 135) including girls and boys, Cobb angle 25–40°, aged 9–17 years, remaining growth > 1 year, and previously untreated. Prognostic outcome was defined as curve progression of Cobb angle of > 6° prior to skeletal maturity. 34 candidate prognostic variables were tested. Time-to-event was measured with 6-month intervals. Cox proportional hazards regression survival model (CoxPH) was used for model development and validation in comparison with machine learning models (66.6/33.3 train/test data set). The models were adjusted for treatment exposure. Results: The final primary prognostic model included 127 patients, predicting progress with acceptable discriminative ability (concordance = 0.79, 95% confidence interval [CI] 0.72–0.86). Significant prognostic risk factors were Risser stage of 0 (HR 4.6, CI 2.1–10.1, P < 0.001), larger major curve Cobb angle (HRstandardized 1.5, CI 1.1–2.0, P = 0.005), and higher score on patient-reported pictorial Spinal Appearance Questionnaire (pSAQ) (HRstandardized 1.4, CI 1.0–1.9, P = 0.04). Treatment exposure, entered as a covariate adjustment, contributed significantly to the final model (HR 3.1, CI 1.5–6.0, P = 0.001). Sensitivity analysis displayed that CoxPH maintained acceptable discriminative ability (AUC 0.79, CI 0.65–0.93) in comparison with machine learning algorithms. Conclusion: The prognostic model (Risser stage, Cobb angle, pSAQ, and menarche) predicted curve progression of > 6° Cobb angle with acceptable discriminative ability. Adding patient report of the pSAQ may be of clinical importance for the prognosis of curve progression.

Background and purpose: The study’s purpose was to develop and internally validate a prognostic survival model exploring baseline variables for adolescent idiopathic scoliosis curve progression. Methods: A longitudinal prognostic cohort analysis was performed on trial data (n = 135) including girls and boys, Cobb angle 25–40°, aged 9–17 years, remaining growth > 1 year, and previously untreated. Prognostic outcome was defined as curve progression of Cobb angle of > 6° prior to skeletal maturity. 34 candidate prognostic variables were tested. Time-to-event was measured with 6-month intervals. Cox proportional hazards regression survival model (CoxPH) was used for model development and validation in comparison with machine learning models (66.6/33.3 train/test data set). The models were adjusted for treatment exposure. Results: The final primary prognostic model included 127 patients, predicting progress with acceptable discriminative ability (concordance = 0.79, 95% confidence interval [CI] 0.72–0.86). Significant prognostic risk factors were Risser stage of 0 (HR 4.6, CI 2.1–10.1, P < 0.001), larger major curve Cobb angle (HRstandardized 1.5, CI 1.1–2.0, P = 0.005), and higher score on patient-reported pictorial Spinal Appearance Questionnaire (pSAQ) (HRstandardized 1.4, CI 1.0–1.9, P = 0.04). Treatment exposure, entered as a covariate adjustment, contributed significantly to the final model (HR 3.1, CI 1.5–6.0, P = 0.001). Sensitivity analysis displayed that CoxPH maintained acceptable discriminative ability (AUC 0.79, CI 0.65–0.93) in comparison with machine learning algorithms. Conclusion: The prognostic model (Risser stage, Cobb angle, pSAQ, and menarche) predicted curve progression of > 6° Cobb angle with acceptable discriminative ability. Adding patient report of the pSAQ may be of clinical importance for the prognosis of curve progression.

Background and purpose: The study’s purpose was to develop and internally validate a prognostic survival model exploring baseline variables for adolescent idiopathic scoliosis curve progression. Methods: A longitudinal prognostic cohort analysis was performed on trial data (n = 135) including girls and boys, Cobb angle 25–40°, aged 9–17 years, remaining growth > 1 year, and previously untreated. Prognostic outcome was defined as curve progression of Cobb angle of > 6° prior to skeletal maturity. 34 candidate prognostic variables were tested. Time-to-event was measured with 6-month intervals. Cox proportional hazards regression survival model (CoxPH) was used for model development and validation in comparison with machine learning models (66.6/33.3 train/test data set). The models were adjusted for treatment exposure. Results: The final primary prognostic model included 127 patients, predicting progress with acceptable discriminative ability (concordance = 0.79, 95% confidence interval [CI] 0.72–0.86). Significant prognostic risk factors were Risser stage of 0 (HR 4.6, CI 2.1–10.1, P < 0.001), larger major curve Cobb angle (HRstandardized 1.5, CI 1.1–2.0, P = 0.005), and higher score on patient-reported pictorial Spinal Appearance Questionnaire (pSAQ) (HRstandardized 1.4, CI 1.0–1.9, P = 0.04). Treatment exposure, entered as a covariate adjustment, contributed significantly to the final model (HR 3.1, CI 1.5–6.0, P = 0.001). Sensitivity analysis displayed that CoxPH maintained acceptable discriminative ability (AUC 0.79, CI 0.65–0.93) in comparison with machine learning algorithms. Conclusion: The prognostic model (Risser stage, Cobb angle, pSAQ, and menarche) predicted curve progression of > 6° Cobb angle with acceptable discriminative ability. Adding patient report of the pSAQ may be of clinical importance for the prognosis of curve progression.

Background and purpose: The study’s purpose was to develop and internally validate a prognostic survival model exploring baseline variables for adolescent idiopathic scoliosis curve progression. Methods: A longitudinal prognostic cohort analysis was performed on trial data (n = 135) including girls and boys, Cobb angle 25–40°, aged 9–17 years, remaining growth > 1 year, and previously untreated. Prognostic outcome was defined as curve progression of Cobb angle of > 6° prior to skeletal maturity. 34 candidate prognostic variables were tested. Time-to-event was measured with 6-month intervals. Cox proportional hazards regression survival model (CoxPH) was used for model development and validation in comparison with machine learning models (66.6/33.3 train/test data set). The models were adjusted for treatment exposure. Results: The final primary prognostic model included 127 patients, predicting progress with acceptable discriminative ability (concordance = 0.79, 95% confidence interval [CI] 0.72–0.86). Significant prognostic risk factors were Risser stage of 0 (HR 4.6, CI 2.1–10.1, P < 0.001), larger major curve Cobb angle (HRstandardized 1.5, CI 1.1–2.0, P = 0.005), and higher score on patient-reported pictorial Spinal Appearance Questionnaire (pSAQ) (HRstandardized 1.4, CI 1.0–1.9, P = 0.04). Treatment exposure, entered as a covariate adjustment, contributed significantly to the final model (HR 3.1, CI 1.5–6.0, P = 0.001). Sensitivity analysis displayed that CoxPH maintained acceptable discriminative ability (AUC 0.79, CI 0.65–0.93) in comparison with machine learning algorithms. Conclusion: The prognostic model (Risser stage, Cobb angle, pSAQ, and menarche) predicted curve progression of > 6° Cobb angle with acceptable discriminative ability. Adding patient report of the pSAQ may be of clinical importance for the prognosis of curve progression.

Short Tandem Repeats polymorphisms on the Y-chromosome (Y-STRs) play a significant role in history of human hereditary. The recently introduced 6-dyes Qiagen investigator Argus Y-28 Kit which include 28Y-STR loci (DYS389I, DYS391, DYS389II, DYS533, DYS390, DYS627, DYS458, DYS393, DYS19, DYS437, DYS449, DYS460, DYS576, YGATAH4, DYS481, DYS448, DYS518, DYS439, DYS549, DYS438, DYS456, DYS643, QS1, DYS570, DYS635, DYS385, DYS392, QS2) has been used for internal validation study used in the forensic laboratories which followed sensitivity study, mixture studies, PCR parameter limits, stability study, and performance on mock casework samples. The results were reported by statistical analysis using ANOVA program. The results validate the kit’s robustness, reliability, and suitability as an assay for human identification with casework DNA samples. This kit is expected to improve male relative differentiation in future and human hereditary applications.

Semi-distributed hydrological and water quality models are increasingly used as innovative and scientific-based management tools. However, their application is usually restricted to the gauging stations where they are originally calibrated, limiting their spatial capability. In this study, the semi-distributed hydrological water quality model HYPE (HYdrological Predictions for the Environment) was tested spatially to represent nitrate-N (NO3-N) and total phosphorus (TP) concentrations and loads of the nested and heterogeneous Selke catchment (463 km2) in central Germany. First, an automatic calibration procedure and uncertainty analysis were conducted using the DiffeRential Evolution Adaptive Metropolis (DREAM) tool to simulate discharge, NO3-N and TP concentrations. A multi-site and multi-objective calibration approach was applied using three main gauging stations, covering the most important hydro-meteorological and physiographical characteristics of the whole catchment. Second, the model’s capability was tested to represent further internal stations, which were not initially considered for calibration. Results showed that discharge was well represented by the model at all three main stations during both calibration (1994-1998) and validation (1999-2014) periods with lowest Nash-Sutcliffe Efficiency (NSE) of 0.71 and maximum Percentage BIAS (PBIAS) of 18.0%. The model was able to reproduce the seasonal dynamics of NO3-N and TP concentrations with low predictive uncertainty at the three main stations, reflected by PBIAS values in the ranges from -16.1% to 6.4% and from -20.0% to 11.5% for NO3-N and TP load simulations, respectively. At internal stations, the model could represent reasonably well the seasonal variation of nutrient concentrations with PBIAS values in the ranges from -9.0% to 14.2% for NO3-N and from -25.3% to 34.3% for TP concentration simulations. Overall, results suggested that the spatial validation of a nutrient transport model can be better ensu

Background Many colorectal cancer (CRC) survivors experience persisting health problems post-treatment that compromise their health-related quality of life (HRQoL). Prediction models are useful tools for identifying survivors at risk of low HRQoL in the future and for taking preventive action. Therefore, we developed prediction models for CRC survivors to estimate the 1-year risk of low HRQoL in multiple domains. Methods In 1458 CRC survivors, seven HRQoL domains (EORTC QLQ-C30: global QoL; cognitive, emotional, physical, role, social functioning; fatigue) were measured prospectively at study baseline and 1 year later. For each HRQoL domain, scores at 1-year follow-up were dichotomized into low versus normal/high. Separate multivariable logistic prediction models including biopsychosocial predictors measured at baseline were developed for the seven HRQoL domains, and internally validated using bootstrapping. Results Average time since diagnosis was 5 years at study baseline. Prediction models included both non-modifiable predictors (age, sex, socio-economic status, time since diagnosis, tumor stage, chemotherapy, radiotherapy, stoma, micturition, chemotherapy-related, stoma-related and gastrointestinal complaints, comorbidities, social inhibition/negative affectivity, and working status) and modifiable predictors (body mass index, physical activity, smoking, meat consumption, anxiety/depression, pain, and baseline fatigue and HRQoL scores). Internally validated models showed good calibration and discrimination (AUCs: 0.83–0.93). Conclusions The prediction models performed well for estimating 1-year risk of low HRQoL in seven domains. External validation is needed before models can be applied in practice.

Background: Many colorectal cancer (CRC) survivors experience persisting health problems post-treatment that compromise their health-related quality of life (HRQoL). Prediction models are useful tools for identifying survivors at risk of low HRQoL in the future and for taking preventive action. Therefore, we developed prediction models for CRC survivors to estimate the 1-year risk of low HRQoL in multiple domains. Methods: In 1458 CRC survivors, seven HRQoL domains (EORTC QLQ-C30: Global QoL; cognitive, emotional, physical, role, social functioning; fatigue) were measured prospectively at study baseline and 1 year later. For each HRQoL domain, scores at 1-year follow-up were dichotomized into low versus normal/high. Separate multivariable logistic prediction models including biopsychosocial predictors measured at baseline were developed for the seven HRQoL domains, and internally validated using bootstrapping. Results: Average time since diagnosis was 5 years at study baseline. Prediction models included both non-modifiable predictors (age, sex, socio-economic status, time since diagnosis, tumor stage, chemotherapy, radiotherapy, stoma, micturition, chemotherapy-related, stoma-related and gastrointestinal complaints, comorbidities, social inhibition/negative affectivity, and working status) and modifiable predictors (body mass index, physical activity, smoking, meat consumption, anxiety/depression, pain, and baseline fatigue and HRQoL scores). Internally validated models showed good calibration and discrimination (AUCs: 0.83-0.93). Conclusions: The prediction models performed well for estimating 1-year risk of low HRQoL in seven domains. External validation is needed before models can be applied in practice.

Background It is important to find ways to minimize errors when children self-report food consumption. Objective The objective of this study was to investigate whether assistance given to children completing a self-administered 24-hour dietary recall instrument called SACANA (Self-Administered Child, Adolescent and Adult Nutrition Assessment) increased the repeatability and plausibility of energy intake (EI) estimates. Participants/setting The study was conducted between October 2013 and March 2016 in a convenience sample of 395 children, aged 8 to 17 years, from eight European countries participating in the I.Family study. Design SACANA was used to recall the previous day's food intake, twice in a day, once with and once without assistance. Main outcome measures The difference in EI between the first and second recalls was the main repeatability measure; the ratio of EI to basal metabolic rate was the plausibility measure. Statistical methods Generalized linear mixed models, adjusted for sex, age, and body mass index z-score, were used to assess whether assistance during the first vs second recall influenced repeatability and plausibility. Results The difference in estimated EI (EI from second recall minus EI from first recall) was significantly lower (P<0.001) in those assisted at first (median=-76 kcal) than those assisted at second recall (median=282 kcal). Modeling showed that EI at assisted first recall was 19% higher (95% CI 1.13 to 1.24) than in assisted second recall. Overall, 60% of recalls had a plausible EI. Modeling to estimate the simultaneous effects of second vs first recall and assistance vs no assistance on plausibility showed that those assisted at first recall had significantly higher odds of a plausible recall than those unassisted (odds ratio 3.64, 95% CI 2.20 to 6.01), with no significant difference in plausibility of second recall compared to the first (odds ratio 1.48, 95% CI 0.92 to 2.35). Conclusions When children are assisted at first, Authors on behalf of the I.Family consortium© 2018 by the Academy of Nutrition and Dietetics. The RightsLink Digital Licensing and Rights Management Service (including RightsLink for Open Access) is available (A) to users of copyrighted works found at the websites of participating publishers who are seeking permissions or licenses to use those works, and (B) to authors of articles and other manuscripts who are seeking to pay author publication charges in connection with the submission of their works to publishers.

Objectives We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. Study Design and Setting We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). Results At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FP1, or RCS3 showed better performance (differences 0.01 and 0.002). Conclusion Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity.

Quantitative pharmacokinetic-pharmacodynamic and disease progression models are the core of the science of pharmacometrics which has been identified as one of the strategies that can make drug development more effective. To adequately develop and utilize these models one needs to carefully consider the nature of the data, choice of appropriate estimation methods, model evaluation strategies, and, most importantly, the intended use of the model. The general aim of this thesis was to investigate how the use of pharmacometric models can improve the design and analysis of clinical trials within drug development. The development of pharmacometric models for clinical assessment scales in stroke and graded severity events, in this thesis, show the benefit of describing data as close to its true nature as possible, as it increases the predictive abilities and allows for mechanistic interpretations of the models. Performance of three estimation methods implemented in the mixed-effects modeling software NONMEM; 1) Laplace, 2) SAEM, and 3) Importance sampling, applied when modeling repeated time-to-event data, was investigated. The two latter methods are to be preferred if less than approximately half of the individuals experience events. In addition, predictive performance of two validation procedures, internal and external validation, was explored, with internal validation being preferred in most cases. Model-based analysis was compared to conventional methods by the use of clinical trial simulations and the power to detect a drug effect was improved with a pharmacometric design and analysis. Throughout this thesis several examples have shown the possibility of significantly reducing sample sizes in clinical trials with a pharmacometric model-based analysis. This approach will reduce time and costs spent in the development of new drug therapies, but foremost reduce the number of healthy volunteers and patients exposed to experimental drugs.

Quantitative pharmacokinetic-pharmacodynamic and disease progression models are the core of the science of pharmacometrics which has been identified as one of the strategies that can make drug development more effective. To adequately develop and utilize these models one needs to carefully consider the nature of the data, choice of appropriate estimation methods, model evaluation strategies, and, most importantly, the intended use of the model. The general aim of this thesis was to investigate how the use of pharmacometric models can improve the design and analysis of clinical trials within drug development. The development of pharmacometric models for clinical assessment scales in stroke and graded severity events, in this thesis, show the benefit of describing data as close to its true nature as possible, as it increases the predictive abilities and allows for mechanistic interpretations of the models. Performance of three estimation methods implemented in the mixed-effects modeling software NONMEM; 1) Laplace, 2) SAEM, and 3) Importance sampling, applied when modeling repeated time-to-event data, was investigated. The two latter methods are to be preferred if less than approximately half of the individuals experience events. In addition, predictive performance of two validation procedures, internal and external validation, was explored, with internal validation being preferred in most cases. Model-based analysis was compared to conventional methods by the use of clinical trial simulations and the power to detect a drug effect was improved with a pharmacometric design and analysis. Throughout this thesis several examples have shown the possibility of significantly reducing sample sizes in clinical trials with a pharmacometric model-based analysis. This approach will reduce time and costs spent in the development of new drug therapies, but foremost reduce the number of healthy volunteers and patients exposed to experimental drugs.

Quantitative pharmacokinetic-pharmacodynamic and disease progression models are the core of the science of pharmacometrics which has been identified as one of the strategies that can make drug development more effective. To adequately develop and utilize these models one needs to carefully consider the nature of the data, choice of appropriate estimation methods, model evaluation strategies, and, most importantly, the intended use of the model. The general aim of this thesis was to investigate how the use of pharmacometric models can improve the design and analysis of clinical trials within drug development. The development of pharmacometric models for clinical assessment scales in stroke and graded severity events, in this thesis, show the benefit of describing data as close to its true nature as possible, as it increases the predictive abilities and allows for mechanistic interpretations of the models. Performance of three estimation methods implemented in the mixed-effects modeling software NONMEM; 1) Laplace, 2) SAEM, and 3) Importance sampling, applied when modeling repeated time-to-event data, was investigated. The two latter methods are to be preferred if less than approximately half of the individuals experience events. In addition, predictive performance of two validation procedures, internal and external validation, was explored, with internal validation being preferred in most cases. Model-based analysis was compared to conventional methods by the use of clinical trial simulations and the power to detect a drug effect was improved with a pharmacometric design and analysis. Throughout this thesis several examples have shown the possibility of significantly reducing sample sizes in clinical trials with a pharmacometric model-based analysis. This approach will reduce time and costs spent in the development of new drug therapies, but foremost reduce the number of healthy volunteers and patients exposed to experimental drugs.

Quantitative pharmacokinetic-pharmacodynamic and disease progression models are the core of the science of pharmacometrics which has been identified as one of the strategies that can make drug development more effective. To adequately develop and utilize these models one needs to carefully consider the nature of the data, choice of appropriate estimation methods, model evaluation strategies, and, most importantly, the intended use of the model. The general aim of this thesis was to investigate how the use of pharmacometric models can improve the design and analysis of clinical trials within drug development. The development of pharmacometric models for clinical assessment scales in stroke and graded severity events, in this thesis, show the benefit of describing data as close to its true nature as possible, as it increases the predictive abilities and allows for mechanistic interpretations of the models. Performance of three estimation methods implemented in the mixed-effects modeling software NONMEM; 1) Laplace, 2) SAEM, and 3) Importance sampling, applied when modeling repeated time-to-event data, was investigated. The two latter methods are to be preferred if less than approximately half of the individuals experience events. In addition, predictive performance of two validation procedures, internal and external validation, was explored, with internal validation being preferred in most cases. Model-based analysis was compared to conventional methods by the use of clinical trial simulations and the power to detect a drug effect was improved with a pharmacometric design and analysis. Throughout this thesis several examples have shown the possibility of significantly reducing sample sizes in clinical trials with a pharmacometric model-based analysis. This approach will reduce time and costs spent in the development of new drug therapies, but foremost reduce the number of healthy volunteers and patients exposed to experimental drugs.

Quantitative pharmacokinetic-pharmacodynamic and disease progression models are the core of the science of pharmacometrics which has been identified as one of the strategies that can make drug development more effective. To adequately develop and utilize these models one needs to carefully consider the nature of the data, choice of appropriate estimation methods, model evaluation strategies, and, most importantly, the intended use of the model. The general aim of this thesis was to investigate how the use of pharmacometric models can improve the design and analysis of clinical trials within drug development. The development of pharmacometric models for clinical assessment scales in stroke and graded severity events, in this thesis, show the benefit of describing data as close to its true nature as possible, as it increases the predictive abilities and allows for mechanistic interpretations of the models. Performance of three estimation methods implemented in the mixed-effects modeling software NONMEM; 1) Laplace, 2) SAEM, and 3) Importance sampling, applied when modeling repeated time-to-event data, was investigated. The two latter methods are to be preferred if less than approximately half of the individuals experience events. In addition, predictive performance of two validation procedures, internal and external validation, was explored, with internal validation being preferred in most cases. Model-based analysis was compared to conventional methods by the use of clinical trial simulations and the power to detect a drug effect was improved with a pharmacometric design and analysis. Throughout this thesis several examples have shown the possibility of significantly reducing sample sizes in clinical trials with a pharmacometric model-based analysis. This approach will reduce time and costs spent in the development of new drug therapies, but foremost reduce the number of healthy volunteers and patients exposed to experimental drugs.

The estimation of benzo(a)pyrene (BaP) concentrations in ambient air is very important from an environmental point of view especially with the introduction of the Directive 2004/107/EC and due to the carcinogenic character of this pollutant. A sampling campaign of particulate matter less or equal than 10 microns (PM10) carried out during 2008-2009 in four locations of Spain was collected to determine experimentally BaP concentrations by gas chromatography-mass spectrometry-mass spectrometry (GC/MS/MS). Multivariate linear regression models (MLRM) were used to predict BaP air concentrations in two sampling places, taking PM10 and meteorological variables as possible predictors. The model obtained with data from two sampling sites (all sites model) (R2=0.817, PRESS/SSY=0.183) included the significant variables like PM10, temperature, solar radiation and wind speed and was internally and externally validated. The first validation was performed by cross validation and the last one by BaP concentrations from previous campaigns carried out in Zaragoza from 2001-2004. The proposed model constitutes a first approximation to estimate BaP concentrations in urban atmospheres with very good internal prediction (Q2CV=0.813, PRESS/SSY=0.187) and with the maximal external prediction for the 2001-2002 campaign (Q2ext=0.679 and PRESS/SSY=0.321) versus the 2001-2004 campaign (Q2ext=0.551, PRESS/SSY=0.449).