Search

In low-dimensional data and within the framework of a classical linear regression model, we intend to compare variable selection methods and investigate the role of shrinkage of regression estimates in a simulation study. Our primary aim is to build descriptive models that capture the data structure parsimoniously, while our secondary aim is to derive a prediction model. Simulation studies are an important tool in statistical methodology research if they are well designed, executed, and reported. However, bias in favor of an "own" preferred method is prevalent in most simulation studies in which a new method is proposed and compared with existing methods. To overcome such bias, neutral comparison studies, which disregard the superiority or inferiority of a particular method, have been proposed. In this paper, we designed a simulation study with key principles of neutral comparison studies in mind, though certain unintentional biases cannot be ruled out. To improve the design and reporting of a simulation study, we followed the recently proposed ADEMP structure, which entails defining the aims (A), data-generating mechanisms (D), estimand/target of analysis (E), methods (M), and performance measures (P). To ensure the reproducibility of results, we published the protocol before conducting the study. In addition, we presented earlier versions of the design to several experts whose feedback influenced certain aspects of the design. We will compare popular penalized regression methods (lasso, adaptive lasso, relaxed lasso, and nonnegative garrote) that combine variable selection and shrinkage with classical variable selection methods (best subset selection and backward elimination) with and without post-estimation shrinkage of parameter estimates. [ABSTRACT FROM AUTHOR]

Background: Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted. Methods: A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted. Results: Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance. Conclusions: A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research. [ABSTRACT FROM AUTHOR]

Background: Factors contributing to the lack of understanding of research studies include poor reporting practices, such as selective reporting of statistically significant findings or insufficient methodological details. Systematic reviews have shown that prognostic factor studies continue to be poorly reported, even for important aspects, such as the effective sample size. The REMARK reporting guidelines support researchers in reporting key aspects of tumor marker prognostic studies. The REMARK profile was proposed to augment these guidelines to aid in structured reporting with an emphasis on including all aspects of analyses conducted.Methods: A systematic search of prognostic factor studies was conducted, and fifteen studies published in 2015 were selected, three from each of five oncology journals. A paper was eligible for selection if it included survival outcomes and multivariable models were used in the statistical analyses. For each study, we summarized the key information in a REMARK profile consisting of details about the patient population with available variables and follow-up data, and a list of all analyses conducted.Results: Structured profiles allow an easy assessment if reporting of a study only has weaknesses or if it is poor because many relevant details are missing. Studies had incomplete reporting of exclusion of patients, missing information about the number of events, or lacked details about statistical analyses, e.g., subgroup analyses in small populations without any information about the number of events. Profiles exhibit severe weaknesses in the reporting of more than 50% of the studies. The quality of analyses was not assessed, but some profiles exhibit several deficits at a glance.Conclusions: A substantial part of prognostic factor studies is poorly reported and analyzed, with severe consequences for related systematic reviews and meta-analyses. We consider inadequate reporting of single studies as one of the most important reasons that the clinical relevance of most markers is still unclear after years of research and dozens of publications. We conclude that structured reporting is an important step to improve the quality of prognostic marker research and discuss its role in the context of selective reporting, meta-analysis, study registration, predefined statistical analysis plans, and improvement of marker research. [ABSTRACT FROM AUTHOR]

Background: In clinical trials, there is considerable interest in investigating whether a treatment effect is similar in all patients, or that one or more prognostic variables indicate a differential response to treatment. To examine this, a continuous predictor is usually categorised into groups according to one or more cutpoints. Several weaknesses of categorization are well known. To avoid the disadvantages of cutpoints and to retain full information, it is preferable to keep continuous variables continuous in the analysis. To handle this issue, the Subpopulation Treatment Effect Pattern Plot (STEPP) was proposed about two decades ago, followed by the multivariable fractional polynomial interaction (MFPI) approach. Provided individual patient data (IPD) from several studies are available, it is possible to investigate for treatment heterogeneity with meta-analysis techniques. Meta-STEPP was recently proposed and in patients with primary breast cancer an interaction of estrogen receptors with chemotherapy was investigated in eight randomized controlled trials (RCTs).Methods: We use data from eight randomized controlled trials in breast cancer to illustrate issues from two main tasks. The first task is to derive a treatment effect function (TEF), that is, a measure of the treatment effect on the continuous scale of the covariate in the individual studies. The second is to conduct a meta-analysis of the continuous TEFs from the eight studies by applying pointwise averaging to obtain a mean function. We denote the method metaTEF. To improve reporting of available data and all steps of the analysis we introduce a three-part profile called MethProf-MA.Results: Although there are considerable differences between the studies (populations with large differences in prognosis, sample size, effective sample size, length of follow up, proportion of patients with very low estrogen receptor values) our results provide clear evidence of an interaction, irrespective of the choice of the FP function and random or fixed effect models.Conclusions: In contrast to cutpoint-based analyses, metaTEF retains the full information from continuous covariates and avoids several critical issues when performing IPD meta-analyses of continuous effect modifiers in randomised trials. Early experience suggests it is a promising approach.Trial Registration: Not applicable. [ABSTRACT FROM AUTHOR]

In 2005, several experts in tumor biomarker research publishe the REporting recommendations for Tumor MARKer prognostic studies (REMARK) criteria. Coupled with the subsequent Biospecimen Reporting for Improved Study Quality (BRISQ) criteria, these initiatives provide a framework for transparently reporting of the methods of study conduct and analyses. [ABSTRACT FROM AUTHOR]

The Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) were developed to address widespread deficiencies in the reporting of such studies. The REMARK checklist consists of 20 items to report for published tumor marker prognostic studies. A detailed paper was published explaining the rationale behind checklist items, providing positive examples and giving empirical evidence of the quality of reporting. REMARK provides a comprehensive overview to educate on good reporting and provide a valuable reference for the many issues to consider when designing, conducting, and analyzing tumor marker studies and prognostic studies in medicine in general. Despite support for REMARK from major cancer journals, prognostic factor research studies remain poorly reported. To encourage dissemination and uptake of REMARK, we have produced this considerably abridged version of the detailed explanatory manuscript, which may also serve as a brief guide to key issues for investigators planning tumor marker prognostic studies. To summarize the current situation, more recent papers investigating the quality of reporting and related reporting guidelines are cited, but otherwise the literature is not updated. Another important impetus for this paper is that it serves as a basis for literal translations into other languages. Translations will help to bring key information to a larger audience world-wide. Many more details can be found in the original paper. [ABSTRACT FROM AUTHOR]

If a number of candidate variables are available, variable selection is a key task aiming to identify those candidates which influence the outcome of interest. Methods as backward elimination, forward selection, etc. are often implemented, despite their drawbacks. One of these drawbacks is the instability of their results with respect to small perturbations in the data. To handle this issue, resampling-based procedures have been introduced; using a resampling technique, e.g. bootstrap, these procedures generate several pseudo-samples that are used to compute the inclusion frequency of each variable, i.e. the proportion of pseudo-samples in which the variable is selected. Based on the inclusion frequencies, it is possible to discriminate between relevant and irrelevant variables. These procedures may fail in case of correlated variables. To deal with this issue, two procedures based on 2×2 tables of inclusion frequencies have been developed in the literature. In this paper we analyse the behaviours of these two procedures and the role of their tuning parameters in an extensive simulation study. [ABSTRACT FROM PUBLISHER]

The article provides an overview of the STRengthening Analytical Thinking for Observational Studies (STRATOS) initiative as an international collaboration which was formed to offer guidance to help bridge the gap between methodological application and innovation. STRATOS is focused on distinguishing issues and promising approaches for analysing and planning observational studies.

In a recent article, Royston (2015, Stata Journal 15: 275-291) introduced the approximate cumulative distribution (ACD) transformation of a continuous covariate x as a route toward modeling a sigmoid relationship between x and an outcome variable. In this article, we extend the approach to multivariable modeling by modifying the standard Stata program mfp. The result is a new program, mfpa, that has all the features of mfp plus the ability to fit a new model for userselected covariates that we call FP1(p1, p2). The FP1(p1, p2) model comprises the best-fitting combination of a dimension-one fractional polynomial (FP1) function of x and an FP1 function of ACD(x). We describe a new model-selection algorithm called function-selection procedure with ACD transformation, which uses significance testing to attempt to simplify an FP1(p1, p2) model to a submodel, an FP1 or linear model in x or in ACD(x). The function-selection procedure with ACD transformation is related in concept to the FSP (FP function-selection procedure), which is an integral part of mfp and which is used to simplify a dimension-two (FP2) function. We describe the mfpa command and give univariable and multivariable examples with real data to demonstrate its use. [ABSTRACT FROM AUTHOR]

In biomedical literature, numerous prediction models for clinical outcomes have been developed based either on clinical data or, more recently, on high-throughput molecular data (omics data). Prediction models based on both types of data, however, are less common, although some recent studies suggest that a suitable combination of clinical and molecular information may lead to models with better predictive abilities. This is probably due to the fact that it is not straightforward to combine data with different characteristics and dimensions (poorly characterized high-dimensional omics data, well-investigated low-dimensional clinical data). In this paper, we analyze two publicly available datasets related to breast cancer and neuroblastoma, respectively, in order to show some possible ways to combine clinical and omics data into a prediction model of time-to-event outcome. Different strategies and statistical methods are exploited. The results are compared and discussed according to different criteria, including the discriminative ability of the models, computed on a validation dataset. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The validity and practical utility of observational medical research depends critically on good study design, excellent data quality, appropriate statistical methods and accurate interpretation of results. Statistical methodology has seen substantial development in recent times. Unfortunately, many of these methodological developments are ignored in practice. Consequently, design and analysis of observational studies often exhibit serious weaknesses. The lack of guidance on vital practical issues discourages many applied researchers from using more sophisticated and possibly more appropriate methods when analyzing observational studies. Furthermore, many analyses are conducted by researchers with a relatively weak statistical background and limited experience in using statistical methodology and software. Consequently, even 'standard' analyses reported in the medical literature are often flawed, casting doubt on their results and conclusions. An efficient way to help researchers to keep up with recent methodological developments is to develop guidance documents that are spread to the research community at large. These observations led to the initiation of the strengthening analytical thinking for observational studies (STRATOS) initiative, a large collaboration of experts in many different areas of biostatistical research. The objective of STRATOS is to provide accessible and accurate guidance in the design and analysis of observational studies. The guidance is intended for applied statisticians and other data analysts with varying levels of statistical education, experience and interests. In this article, we introduce the STRATOS initiative and its main aims, present the need for guidance documents and outline the planned approach and progress so far. We encourage other biostatisticians to become involved. © 2014 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In a large simulation study reported in a companion paper, we investigated the significance levels of 21 methods for investigating interactions between binary treatment and a continuous covariate in a randomised controlled trial. Several of the methods were shown to have inflated type 1 errors. In the present paper, we report the second part of the simulation study in which we investigated the power of the interaction procedures for two sample sizes and with two distributions of the covariate (well and badly behaved). We studied several methods involving categorisation and others in which the covariate was kept continuous, including fractional polynomials and splines. We believe that the results provide sufficient evidence to recommend the multivariable fractional polynomial interaction procedure as a suitable approach to investigate interactions of treatment with a continuous variable. If subject-matter knowledge gives good arguments for a non-monotone treatment effect function, we propose to use a second-degree fractional polynomial approach, but otherwise a first-degree fractional polynomial (FP1) function with added flexibility (FLEX3) is the method of choice. The FP1 class includes the linear function, and the selected functions are simple, understandable, and transferable. Furthermore, software is available. We caution that investigation of interactions in one dataset can only be interpreted in a hypothesis-generating sense and needs validation in new data. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Interactions between treatments and covariates in RCTs are a key topic. Standard methods for modelling treatment-covariate interactions with continuous covariates are categorisation or linear functions. Both approaches are easily criticised, but for different reasons. Multivariable fractional polynomial interactions, an approach based on fractional polynomials with the linear interaction model as the simplest special case, was proposed. Four variants of multivariable fractional polynomial interaction (FLEX1-FLEX4), allowing varying flexibility in functional form, were suggested. However, their properties are unknown, and comparisons with other procedures are unavailable. Additionally, we consider various methods based on categorisation and on cubic regression splines. We present the results of a simulation study to determine the significance level (probability of a type 1 error) of various tests for interaction between a binary covariate ('treatment effect') and a continuous covariate in univariate analysis. We consider a simplified setting in which the response variable is conditionally normally distributed, given the continuous covariate. We consider two main cases with the covariate distribution well behaved (approximately symmetric) or badly behaved (positively skewed). We construct nine scenarios with different functional forms for the main effect. In the well-behaved case, significance levels are in general acceptably close to nominal and are slightly better for the larger sample size ( n = 250 and 500 were investigated). In the badly behaved case, departures from nominal are more pronounced for several approaches. For a final assessment of these results and recommendations for practice, a study of power is needed. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In observational studies, many continuous or categorical covariates may be related to an outcome. Various spline-based procedures or the multivariable fractional polynomial (MFP) procedure can be used to identify important variables and functional forms for continuous covariates. This is the main aim of an explanatory model, as opposed to a model only for prediction. The type of analysis often guides the complexity of the final model. Spline-based procedures and MFP have tuning parameters for choosing the required complexity. To compare model selection approaches, we perform a simulation study in the linear regression context based on a data structure intended to reflect realistic biomedical data. We vary the sample size, variance explained and complexity parameters for model selection. We consider 15 variables. A sample size of 200 (1000) and R2 = 0.2 (0.8) is the scenario with the smallest (largest) amount of information. For assessing performance, we consider prediction error, correct and incorrect inclusion of covariates, qualitative measures for judging selected functional forms and further novel criteria. From limited information, a suitable explanatory model cannot be obtained. Prediction performance from all types of models is similar. With a medium amount of information, MFP performs better than splines on several criteria. MFP better recovers simpler functions, whereas splines better recover more complex functions. For a large amount of information and no local structure, MFP and the spline procedures often select similar explanatory models. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

When several studies are available, a meta-analytic assessment of the effect of a risk or prognostic factor on an outcome is often required. We propose a new strategy, requiring individual participant data, to provide a summary estimate of the functional relationship between a continuous covariate and the outcome in a regression model, adjusting for confounding factors. Our procedure comprises three steps. First, we determine a confounder model. Ideally, the latter should include the same variables across studies, but this may be impossible. Next, we estimate the functional form for the continuous variable of interest in each study, adjusted for the confounder model. Finally, we combine the individual functions by weighted averaging to obtain a summary estimate of the function. Fractional polynomial methodology and pointwise weighted averaging of functions are the key components. In contrast to a pooled analysis, our approach can reflect more variability between functions from different studies and more flexibility with respect to confounders. We illustrate the procedure by using data from breast cancer patients in the Surveillance, Epidemiology, and End Results Program database, where we consider data from nine individual registries as separate studies. We estimate the functional forms for the number of positive lymph nodes and age. The former is an example where a strong prognostic effect has long been recognized, whereas the prognostic effect of the latter is weak or even controversial. We further discuss some general issues that are found in meta-analyses of observational studies. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Hundreds of ‘molecular signatures’ have been proposed in the literature to predict patient outcome in clinical settings from high-dimensional data, many of which eventually failed to get validated. Validation of such molecular research findings is thus becoming an increasingly important branch of clinical bioinformatics. Moreover, in practice well-known clinical predictors are often already available. From a statistical and bioinformatics point of view, poor attention has been given to the evaluation of the added predictive value of a molecular signature given that clinical predictors or an established index are available. This article reviews procedures that assess and validate the added predictive value of high-dimensional molecular data. It critically surveys various approaches for the construction of combined prediction models using both clinical and molecular data, for validating added predictive value based on independent data, and for assessing added predictive value using a single data set. [ABSTRACT FROM PUBLISHER]

A common task in epidemiology is to estimate the dose-response function for a continuous exposure. Often a proportion of subjects is unexposed. Typical examples are cigarette consumption, alcohol intake, or occupational exposures. The question arises as to how to model such variables statistically. The fractional polynomial method of modelling continuous exposure variables is extended to allow for a proportion unexposed. A binary variable for the unexposed fraction is added to the model. In a two-stage procedure, we assess whether the binary variable and/or the continuous function for the exposed individuals is required for a good fit to the data. Extension to the multivariable situation is described. Three data sets with different characteristics are used as illustrations. The analyses of the three studies using the proposed procedure give differing results. In one example, only the binary variable seems to be required. In the other two examples, the binary variable and fractional polynomial functions of the exposure variable are needed. One function is monotonic and the other has a minimum. In the third example, adjusting for confounders has almost no effect on the function selected. In conclusion, the new procedure offers a worthwhile extension of dose-response modelling with an unexposed fraction. It is simple to carry out with standard software. Copyright © 2010 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In developing regression models, data analysts are often faced with many predictor variables that may influence an outcome variable. After more than half a century of research, the 'best' way of selecting a multivariable model is still unresolved. It is generally agreed that subject matter knowledge, when available, should guide model building. However, such knowledge is often limited, and data-dependent model building is required. We limit the scope of the modelling exercise to selecting important predictors and choosing interpretable and transportable functions for continuous predictors. Assuming linear functions, stepwise selection and all-subset strategies are discussed; the key tuning parameters are the nominal P-value for testing a variable for inclusion and the penalty for model complexity, respectively. We argue that stepwise procedures perform better than a literature-based assessment would suggest. Concerning selection of functional form for continuous predictors, the principal competitors are fractional polynomial functions and various types of spline techniques. We note that a rigorous selection strategy known as multivariable fractional polynomials (MFP) has been developed. No spline-based procedure for simultaneously selecting variables and functional forms has found wide acceptance. Results of FP and spline modelling are compared in two data sets. It is shown that spline modelling, while extremely flexible, can generate fitted curves with uninterpretable 'wiggles', particularly when automatic methods for choosing the smoothness are employed. We give general recommendations to practitioners for carrying out variable and function selection. While acknowledging that further research is needed, we argue why MFP is our preferred approach for multivariable model building with continuous covariates. [ABSTRACT FROM AUTHOR]

Despite its importance in the theoretical literature, the bootstrap appears to play a negligible role in pharmaceutical research, as will be demonstrated by a brief literature review. As will be shown by examples, the bootstrap is a useful tool in the planning and analysis of clinical trials. The first example shows that some important information required in the design of a study can best be gained by using the bootstrap. It is argued from two further examples that more information can be extracted from large clinical trials by data-dependent modelling. This is shown by identifying a prognostic factor that may play a role as an inclusion criterion of a new study and by an interaction of a continuous predictor with treatment. To protect against erroneous conclusions from data-dependent modelling in a multivariable context, detailed checks of the results and stability analyses should be performed by the bootstrap. To conclude, a discussion of the future of modelling and the future of the bootstrap is given. [ABSTRACT FROM AUTHOR]

The identification and assessment of prognostic markers constitutes one of the major tasks in clinical research. Despite huge research effort, the prognostic value of most traditional factors under discussion is uncertain and the usefulness of many specific markers, prognostic indices, and classification schemes is still unproven. Results from different studies are often contradictory, and a general assessment of the usefulness of a specific marker is very difficult. One reason is that systematic reviews of prognostic marker studies have received rather little attention in the literature. It is obvious that a clinically useful and sensible systematic review of a prognostic marker is only possible if the published studies reflect the true nature of the marker and if sufficient details are given in each report. An important goal of a systematic review is to produce a quantitative summary of an effect of interest by a meta-analysis, a statistical approach that combines the results of individual primary studies by a weighted average. For observational studies, an estimate from a univariate model is only of limited interest; a multivariable approach is absolutely essential to derive an estimate that is adjusted for other factors. However, even when “adjusted estimates” are presented, it is common for different studies to use different variables for adjustment, and specific “adjustment” variables may be measured in different ways or may be used with different scales. These difficulties are partly caused by the large variety of available statistical methods of analyzing prognostic marker studies. In three related papers published in a proceedings volume, Holländer and Sauerbrei (2006), Riley et al. (2006), and Altman et al. (2006) discuss statistical approaches for multivariable analysis, issues of reporting of primary studies, and the feasibility of obtaining individual patient data from multiple studies on prognosis. Holländer and Sauerbrei (2006) show that the specific statistical method can have a strong influence on the final multivariable model and on the interpretation of the effect of a specific factor. Possible approaches to help improve reporting standards are discussed in the paper by Riley et al. (2006), which also considers other important issues such as how to improve the design and clinical relevance of primary prognostic studies. For a sensible summary assessment, individual patient data (IPD) and a close collaboration between different study groups seem to be essential. However, Altman et al. (2006) discuss in their paper practical problems in using the IPD approach to evaluate evidence relating to a prognostic marker. Here the three papers are summarized with the aim of demonstrating difficulties and making some recommendations to improve future research in evidence-based assessment of prognostic markers. For many more details we refer to the original papers. [ABSTRACT FROM AUTHOR]

Predictions of disease outcome in prognostic factor models are usually based on one selected model. However, often several models fit the data equally well, but these models might differ substantially in terms of included explanatory variables and might lead to different predictions for individual patients. For survival data, we discuss two approaches to account for model selection uncertainty in two data examples, with the main emphasis on variable selection in a proportional hazard Cox model. The main aim of our investigation is to establish the ways in which either of the two approaches is useful in such prognostic models. The first approach is Bayesian model averaging (BMA) adapted for the proportional hazard model, termed 'approx. BMA' here. As a new approach, we propose a method which averages over a set of possible models using weights estimated from bootstrap resampling as proposed by Buckland et al., but in addition, we perform an initial screening of variables based on the inclusion frequency of each variable to reduce the set of variables and corresponding models. For some necessary parameters of the procedure, investigations concerning sensible choices are still required. The main objective of prognostic models is prediction, but the interpretation of single effects is also important and models should be general enough to ensure transportability to other clinical centres. In the data examples, we compare predictions of our new approach with approx. BMA, with 'conventional' predictions from one selected model and with predictions from the full model. Confidence intervals are compared in one example. Comparisons are based on the partial predictive score and the Brier score. We conclude that the two model averaging methods yield similar results and are especially useful when there is a high number of potential prognostic factors, most likely some of them without influence in a multivariable context. Although the method based on bootstrap resampling lacks formal justification and requires some ad hoc decisions, it has the additional positive effect of achieving model parsimony by reducing the number of explanatory variables and dealing with correlated variables in an automatic fashion. [ABSTRACT FROM AUTHOR]

We consider modelling interaction between a categoric covariate T and a continuous covariate Z in a regression model. Here T represents the two treatment arms in a parallel-group clinical trial and Z is a prognostic factor which may influence response to treatment (known as a predictive factor). Generalization to more than two treatments is straightforward. The usual approach to analysis is to categorize Z into groups according to cutpoint(s) and to analyse the interaction in a model with main effects and multiplicative terms. The cutpoint approach raises several well-known and difficult issues for the analyst. We propose an alternative approach based on fractional polynomial (FP) modelling of Z in all patients and at each level of T. Other prognostic variables can also be incorporated by first constructing a multivariable adjustment model which may contain binary covariates and FP transformations of continuous covariates other than Z. The main step involves FP modelling of Z and testing equality of regression coefficients between treatment groups in an interaction model adjusted for other covariates. Extensive experience suggests that a two-term fractional polynomial (FP2) function may describe the effect of a prognostic factor on a survival outcome quite well. In a controlled trial, this FP2 function describes the prognostic effect averaged over the treatment groups. We refit this function in each treatment group to see if there are substantial differences between groups. Allowing different parameter values for the chosen FP2 function is flexible enough to detect such differences. Within the same algorithm we can also deal with the conceptually different cases of a predefined hypothesis of interaction or searching for interactions. We demonstrate the ability of the approach to detect and display treatment/covariate interactions in two examples from controlled trials in cancer. Copyright © 2004 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

When investigating the effects of potential prognostic or risk factors that have been measured on a quantitative scale, values of these factors are often categorized into two groups. Sometimes an 'optimal' cutpoint is chosen that gives the best separation in terms of a two-sample test statistic. It is well known that this approach leads to a serious inflation of the type I error and to an overestimation of the effect of the prognostic or risk factor in absolute terms. In this paper, we illustrate that the resulting confidence intervals are similarly affected. We show that the application of a shrinkage procedure to correct for bias, together with bootstrap resampling for estimating the variance, yields confidence intervals for the effect of a potential prognostic or risk factor with the desired coverage. Copyright © 2004 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Multivariable prognostic models are widely used in cancer and other disease areas, and have a range of applications in clinical medicine, clinical trials and allocation of health services resources. A well-founded and reliable measure of the prognostic ability of a model would be valuable to help define the separation between patients or prognostic groups that the model could provide, and to act as a benchmark of model performance in a validation setting. We propose such a measure for models of survival data. Its motivation derives originally from the idea of separation between Kaplan-Meier curves. We define the criteria for a successful measure and discuss them with respect to our approach. Adjustments for 'optimism', the tendency for a model to predict better on the data on which it was derived than on new data, are suggested. We study the properties of the measure by simulation and by example in three substantial data sets. We believe that our new measure will prove useful as a tool to evaluate the separation available-with a prognostic model. Copyright © 2004 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Statistical regression models are used for predicting outcomes based on the values of some predictor variables or for describing the association of an outcome with predictors. With a data set at hand, a regression model can be easily fit with standard software packages. This bears the risk that data analysts may rush to perform sophisticated analyses without sufficient knowledge of basic properties, associations in and errors of their data, leading to wrong interpretation and presentation of the modeling results that lacks clarity. Ignorance about special features of the data such as redundancies or particular distributions may even invalidate the chosen analysis strategy. Initial data analysis (IDA) is prerequisite to regression analyses as it provides knowledge about the data needed to confirm the appropriateness of or to refine a chosen model building strategy, to interpret the modeling results correctly, and to guide the presentation of modeling results. In order to facilitate reproducibility, IDA needs to be preplanned, an IDA plan should be included in the general statistical analysis plan of a research project, and results should be well documented. Biased statistical inference of the final regression model can be minimized if IDA abstains from evaluating associations of outcome and predictors, a key principle of IDA. We give advice on which aspects to consider in an IDA plan for data screening in the context of regression modeling to supplement the statistical analysis plan. We illustrate this IDA plan for data screening in an example of a typical diagnostic modeling project and give recommendations for data visualizations. [ABSTRACT FROM AUTHOR]

A letter to the editor is presented in response to the article "Performance of Using Multiple Stepwise Algorithms for Variable Selection," by R. E. Wiegard in the 2010 issue.

Background: With progress on both the theoretical and the computational fronts the use of spline modelling has become an established tool in statistical regression analysis. An important issue in spline modelling is the availability of user friendly, well documented software packages. Following the idea of the STRengthening Analytical Thinking for Observational Studies initiative to provide users with guidance documents on the application of statistical methods in observational research, the aim of this article is to provide an overview of the most widely used spline-based techniques and their implementation in R.Methods: In this work, we focus on the R Language for Statistical Computing which has become a hugely popular statistics software. We identified a set of packages that include functions for spline modelling within a regression framework. Using simulated and real data we provide an introduction to spline modelling and an overview of the most popular spline functions.Results: We present a series of simple scenarios of univariate data, where different basis functions are used to identify the correct functional form of an independent variable. Even in simple data, using routines from different packages would lead to different results.Conclusions: This work illustrate challenges that an analyst faces when working with data. Most differences can be attributed to the choice of hyper-parameters rather than the basis used. In fact an experienced user will know how to obtain a reasonable outcome, regardless of the type of spline used. However, many analysts do not have sufficient knowledge to use these powerful tools adequately and will need more guidance. [ABSTRACT FROM AUTHOR]

Risk and prognostic factors in epidemiological and clinical research are often semicontinuous such that a proportion of individuals have exposure zero, and a continuous distribution among those exposed. We call this a spike at zero (SAZ). Typical examples are consumption of alcohol and tobacco, or hormone receptor levels. To additionally model non-linear functional relationships for SAZ variables, an extension of the fractional polynomial (FP) approach was proposed. To indicate whether or not a value is zero, a binary variable is added to the model. In a two-stage procedure, called FP-spike, it is assessed whether the binary variable and/or the continuous FP function for the positive part is required for a suitable fit. In this paper, we compared the performance of two approaches – standard FP and FP-spike – in the Cox model in a motivating example on breast cancer prognosis and a simulation study. The comparisons lead to the suggestion to generally using FP-spike rather than standard FP when the SAZ effect is considerably large because the method performed better in real data applications and simulation in terms of deviance and functional form. Abbreviations: CI: confidence interval; FP: fractional polynomial; FP1: first degree fractional polynomial; FP2: second degree fractional polynomial; FSP: function selection procedure; HT: hormone therapy; OR: odds ratio; SAZ: spike at zero [ABSTRACT FROM AUTHOR]

Background: In high-dimensional data (HDD) settings, the number of variables associated with each observation is very large. Prominent examples of HDD in biomedical research include omics data with a large number of variables such as many measurements across the genome, proteome, or metabolome, as well as electronic health records data that have large numbers of variables recorded for each patient. The statistical analysis of such data requires knowledge and experience, sometimes of complex methods adapted to the respective research questions. Methods: Advances in statistical methodology and machine learning methods offer new opportunities for innovative analyses of HDD, but at the same time require a deeper understanding of some fundamental statistical concepts. Topic group TG9 "High-dimensional data" of the STRATOS (STRengthening Analytical Thinking for Observational Studies) initiative provides guidance for the analysis of observational studies, addressing particular statistical challenges and opportunities for the analysis of studies involving HDD. In this overview, we discuss key aspects of HDD analysis to provide a gentle introduction for non-statisticians and for classically trained statisticians with little experience specific to HDD. Results: The paper is organized with respect to subtopics that are most relevant for the analysis of HDD, in particular initial data analysis, exploratory data analysis, multiple testing, and prediction. For each subtopic, main analytical goals in HDD settings are outlined. For each of these goals, basic explanations for some commonly used analysis methods are provided. Situations are identified where traditional statistical methods cannot, or should not, be used in the HDD setting, or where adequate analytic tools are still lacking. Many key references are provided. Conclusions: This review aims to provide a solid statistical foundation for researchers, including statisticians and non-statisticians, who are new to research with HDD or simply want to better evaluate and understand the results of HDD analyses. [ABSTRACT FROM AUTHOR]

Patient-reported outcomes (PROs) are increasingly used in single-arm cancer studies. We reviewed 60 papers published between 2018 and 2021 of single-arm studies of cancer treatment with PRO data for current practice on design, analysis, reporting, and interpretation. We further examined the studies' handling of potential bias and how they informed decision making. Most studies (58; 97%) analysed PROs without stating a predefined research hypothesis. 13 (22%) of the 60 studies used a PRO as a primary or co-primary endpoint. Definitions of PRO objectives, study population, endpoints, and missing data strategies varied widely. 23 studies (38%) compared the PRO data with external information, most often by using a clinically important difference value; one study used a historical control group. Appropriateness of methods to handle missing data and intercurrent events (including death) were seldom discussed. Most studies (51; 85%) concluded that PRO results supported treatment. Conducting and reporting of PROs in cancer single-arm studies need standards and a critical discussion of statistical methods and possible biases. These findings will guide the Setting International Standards in Analysing Patient-Reported Outcomes and Quality of Life Data in Cancer Clinical Trials–Innovative Medicines Initiative (SISAQOL-IMI) in developing recommendations for the use of PRO-measures in single-arm studies. [ABSTRACT FROM AUTHOR]

In most epidemiologic studies and in clinical research generally, there are variables with a spike at zero, namely variables for which a proportion of individuals have zero exposure (e.g., never smokers) and among those exposed the variable has a continuous distribution. Different options exist for modeling such variables, such as categorization where the nonexposed form the reference group, or ignoring the spike by including the variable in the regression model with or without some transformation or modeling procedures. It has been shown that such situations can be analyzed by adding a binary indicator (exposed/nonexposed) to the regression model, and a method based on fractional polynomials with which to estimate a suitable functional form for the positive portion of the spike-at-zero variable distribution has been developed. In this paper, we compare different approaches using data from 3 case-control studies carried out in Germany: the Mammary Carcinoma Risk Factor Investigation (MARIE), a breast cancer study conducted in 2002-2005 (Flesch-Janys et al., IntJ Cancer. 2008;123(4):933-941); the Rhein-Neckar Larynx Study, a study of laryngeal cancer conducted in 1998-2000 (Dietz et al., Int J Cancer. 2004;108(6):907-911); and a lung cancer study conducted in 1988-1993 (Jöckel et al., Int J Epidemiol. 1998; 27(4):549-560). Strengths and limitations of different procedures are demonstrated, and some recommendations for practical use are given. [ABSTRACT FROM AUTHOR]

In recent years, increasing attention has been devoted to the problem of the stability of multivariable regression models, understood as the resistance of the model to small changes in the data on which it has been fitted. Resampling techniques, mainly based on the bootstrap, have been developed to address this issue. In particular, the approaches based on the idea of "inclusion frequency" consider the repeated implementation of a variable selection procedure, for example backward elimination, on several bootstrap samples. The analysis of the variables selected in each iteration provides useful information on the model stability and on the variables' importance. Recent findings, nevertheless, show possible pitfalls in the use of the bootstrap, and alternatives such as subsampling have begun to be taken into consideration in the literature. Using model selection frequencies and variable inclusion frequencies, we empirically compare these two different resampling techniques, investigating the effect of their use in selected classical model selection procedures for multivariable regression. We conduct our investigations by analyzing two real data examples and by performing a simulation study. Our results reveal some advantages in using a subsampling technique rather than the bootstrap in this context. [ABSTRACT FROM AUTHOR]

Although regression models play a central role in the analysis of medical research projects, there still exist many misconceptions on various aspects of modeling leading to faulty analyses. Indeed, the rapidly developing statistical methodology and its recent advances in regression modeling do not seem to be adequately reflected in many medical publications. This problem of knowledge transfer from statistical research to application was identified by some medical journals, which have published series of statistical tutorials and (shorter) papers mainly addressing medical researchers. The aim of this review was to assess the current level of knowledge with regard to regression modeling contained in such statistical papers. We searched for target series by a request to international statistical experts. We identified 23 series including 57 topic-relevant articles. Within each article, two independent raters analyzed the content by investigating 44 predefined aspects on regression modeling. We assessed to what extent the aspects were explained and if examples, software advices, and recommendations for or against specific methods were given. Most series (21/23) included at least one article on multivariable regression. Logistic regression was the most frequently described regression type (19/23), followed by linear regression (18/23), Cox regression and survival models (12/23) and Poisson regression (3/23). Most general aspects on regression modeling, e.g. model assumptions, reporting and interpretation of regression results, were covered. We did not find many misconceptions or misleading recommendations, but we identified relevant gaps, in particular with respect to addressing nonlinear effects of continuous predictors, model specification and variable selection. Specific recommendations on software were rarely given. Statistical guidance should be developed for nonlinear effects, model specification and variable selection to better support medical researchers who perform or interpret regression analyses. [ABSTRACT FROM AUTHOR]

The article presents a study which provides detailed explanations on the checklist items in the Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK). It is inferred that the recommendations were developed by the European Organization for Research and Treatment of Cancer and the National Cancer Institute. An overview on the proper method of reporting prognostic marker research is provided.

In medical research, continuous variables are often converted into categorical variables by grouping values into two or more categories. We consider in detail issues pertaining to creating just two groups, a common approach in clinical research. We argue that the simplicity achieved is gained at a cost; dichotomization may create rather than avoid problems, notably a considerable loss of power and residual confounding. In addition, the use of a data-derived 'optimal' cutpoint leads to serious bias. We illustrate the impact of dichotomization of continuous predictor variables using as a detailed case study a randomized trial in primary biliary cirrhosis. Dichotomization of continuous data is unnecessary for statistical analysis and in particular should not be applied to explanatory variables in regression models. [ABSTRACT FROM AUTHOR]

Despite years of research and hundreds of reports on tumor markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often, initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodologic problems have been cited to explain these discrepancies. Unfortunately, many tumor marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalizability of study results. The development of guidelines for the reporting of tumor marker studies was a major recommendation of the National Cancer Institute-European Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. As for the successful CONSORT initiative for randomized trials and for the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply. [ABSTRACT FROM AUTHOR]

Abstract: Despite years of research and hundreds of reports on tumour markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodologic problems have been cited to explain these discrepancies. Unfortunately, many tumour marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalisability of study results. The development of guidelines for the reporting of tumour marker studies was a major recommendation of the National Cancer Institute-European Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. As for the successful CONSORT initiative for randomised trials and for the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, pre-planned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply. [Copyright &y& Elsevier]

Background: No standards exist for the handling and reporting of data quality in health research. This work introduces a data quality framework for observational health research data collections with supporting software implementations to facilitate harmonized data quality assessments.Methods: Developments were guided by the evaluation of an existing data quality framework and literature reviews. Functions for the computation of data quality indicators were written in R. The concept and implementations are illustrated based on data from the population-based Study of Health in Pomerania (SHIP).Results: The data quality framework comprises 34 data quality indicators. These target four aspects of data quality: compliance with pre-specified structural and technical requirements (integrity); presence of data values (completeness); inadmissible or uncertain data values and contradictions (consistency); unexpected distributions and associations (accuracy). R functions calculate data quality metrics based on the provided study data and metadata and R Markdown reports are generated. Guidance on the concept and tools is available through a dedicated website.Conclusions: The presented data quality framework is the first of its kind for observational health research data collections that links a formal concept to implementations in R. The framework and tools facilitate harmonized data quality assessments in pursue of transparent and reproducible research. Application scenarios comprise data quality monitoring while a study is carried out as well as performing an initial data analysis before starting substantive scientific analyses but the developments are also of relevance beyond research. [ABSTRACT FROM AUTHOR]

In the last decades, statistical methodology has developed rapidly, in particular in the field of regression modeling. Multivariable regression models are applied in almost all medical research projects. Therefore, the potential impact of statistical misconceptions within this field can be enormous Indeed, the current theoretical statistical knowledge is not always adequately transferred to the current practice in medical statistics. Some medical journals have identified this problem and published isolated statistical articles and even whole series thereof. In this systematic review, we aim to assess the current level of education on regression modeling that is provided to medical researchers via series of statistical articles published in medical journals. The present manuscript is a protocol for a systematic review that aims to assess which aspects of regression modeling are covered by statistical series published in medical journals that intend to train and guide applied medical researchers with limited statistical knowledge. Statistical paper series cannot easily be summarized and identified by common keywords in an electronic search engine like Scopus. We therefore identified series by a systematic request to statistical experts who are part or related to the STRATOS Initiative (STRengthening Analytical Thinking for Observational Studies). Within each identified article, two raters will independently check the content of the articles with respect to a predefined list of key aspects related to regression modeling. The content analysis of the topic-relevant articles will be performed using a predefined report form to assess the content as objectively as possible. Any disputes will be resolved by a third reviewer. Summary analyses will identify potential methodological gaps and misconceptions that may have an important impact on the quality of analyses in medical research. This review will thus provide a basis for future guidance papers and tutorials in the field of regression modeling which will enable medical researchers 1) to interpret publications in a correct way, 2) to perform basic statistical analyses in a correct way and 3) to identify situations when the help of a statistical expert is required. [ABSTRACT FROM AUTHOR]

Data integration, i.e. the use of different sources of information for data analysis, is becoming one of the most important topics in modern statistics. Especially in, but not limited to, biomedical applications, a relevant issue is the combination of low-dimensional (e.g. clinical data) and high-dimensional (e.g. molecular data such as gene expressions) data sources in a prediction model. Not only the different characteristics of the data, but also the complex correlation structure within and between the two data sources, pose challenging issues. In this paper, we investigate these issues via simulations, providing some useful insight into strategies to combine low- and high-dimensional data in a regression prediction model. In particular, we focus on the effect of the correlation structure on the results, while accounting for the influence of our specific choices in the design of the simulation study. [ABSTRACT FROM AUTHOR]

Comments on a meta-analysis of the tumor suppressor protein TP53 which is used as a prognostic marker for head and neck cancer. View that selective reporting biases are a major barrier to conducting meaningful meta-analyses of prognostic marker studies; Association between TP53 status and mortality; Randomized controlled trial.

Background: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN).Methods: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification.Results: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument.Interpretation: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others. [ABSTRACT FROM AUTHOR]

Background: Omics data can be very informative in survival analysis and may improve the prognostic ability of classical models based on clinical risk factors for various diseases, for example breast cancer. Recent research has focused on integrating omics and clinical data, yet has often ignored the need for appropriate model building for clinical variables. Medical literature on classical prognostic scores, as well as biostatistical literature on appropriate model selection strategies for low dimensional (clinical) data, are often ignored in the context of omics research. The goal of this paper is to fill this methodological gap by investigating the added predictive value of gene expression data for models using varying amounts of clinical information.Methods: We analyze two data sets from the field of survival prognosis of breast cancer patients. First, we construct several proportional hazards prediction models using varying amounts of clinical information based on established medical knowledge. These models are then used as a starting point (i.e. included as a clinical offset) for identifying informative gene expression variables using resampling procedures and penalized regression approaches (model based boosting and the LASSO). In order to assess the added predictive value of the gene signatures, measures of prediction accuracy and separation are examined on a validation data set for the clinical models and the models that combine the two sources of information.Results: For one data set, we do not find any substantial added predictive value of the omics data when compared to clinical models. On the second data set, we identify a noticeable added predictive value, however only for scenarios where little or no clinical information is included in the modeling process. We find that including more clinical information can lead to a smaller number of selected omics predictors.Conclusions: New research using omics data should include all available established medical knowledge in order to allow an adequate evaluation of the added predictive value of omics data. Including all relevant clinical information in the analysis might also lead to more parsimonious models. The developed procedure to assess the predictive value of the omics data can be readily applied to other scenarios. [ABSTRACT FROM AUTHOR]