Search

Dealing with missing data problems for skewed data is a difficult task especially since most of the imputation and data augmentation methodologies assume multivariate normality. The performance of imputation and hence the accuracy of inference on parameters become questionable when the violation of the above assumption occurs. One approach to solve the normality violation is to apply normalizing transformation prior to the imputation phase. However, this approach may introduce new problems such as altering the dependence structure among random variables. We present a general purpose multiple imputation approach based on Copula transformation. The approach is used to effectively transform any continuous multivariate non-normal data to multivariate normal, thereby allowing the imputation using standard normality-based techniques. The method then allows to conveniently back transform the data into original space. Real data sets are used to illustrate the techniques. We then compare the performance of our Copula-based method with other traditional normality-based multiple imputation approaches through extensive simulated and real non-normal multivariate datasets. We demonstrate that this method significantly mitigates the problem and hence the practice of making the blind assumption of multivariate normality for non-normal multivariate data under the assumption that data are missing at different mechanisms. [ABSTRACT FROM AUTHOR]

Study objectivesTo identify actigraphy sleep health profiles in older men (Osteoporotic Fractures in Men Study; N = 2640) and women (Study of Osteoporotic Fractures; N = 2430), and to determine whether profile predicts mortality.MethodsWe applied a novel and flexible clustering approach (Multiple Coalesced Generalized Hyperbolic mixture modeling) to identify sleep health profiles based on actigraphy midpoint timing, midpoint variability, sleep interval length, maintenance, and napping/inactivity. Adjusted Cox models were used to determine whether profile predicts time to all-cause mortality.ResultsWe identified similar profiles in men and women: High Sleep Propensity [HSP] (20% of women; 39% of men; high napping and high maintenance); Adequate Sleep [AS] (74% of women; 31% of men; typical actigraphy levels); and Inadequate Sleep [IS] (6% of women; 30% of men; low maintenance and late/variable midpoint). In women, IS was associated with increased mortality risk (Hazard Ratio [HR] = 1.59 for IS vs. AS; 1.75 for IS vs. HSP). In men, AS and IS were associated with increased mortality risk (1.19 for IS vs. HSP; 1.22 for AS vs. HSP).ConclusionsThese findings suggest several considerations for sleep-related interventions in older adults. Low maintenance with late/variable midpoint is associated with increased mortality risk and may constitute a specific target for sleep health interventions. High napping/inactivity co-occurs with high sleep maintenance in some older adults. Although high napping/inactivity is typically considered a risk factor for deleterious health outcomes, our findings suggest that it may not increase risk when it occurs in combination with high sleep maintenance.

The Birnbaum–Saunders (BS) distribution, well-known as the fatigue-life distribution, has been used in numerous disciplines ranging from engineering to medical sciences. In this article, we develop a test for analysis of medians for BS distributed response to assess the impact of two interacting factors on the median, where no test is presently available. The proposed integrated likelihood ratio test (ILRT) eliminates the nuisance shape parameters by integrating them out. The second-order accurate asymptotic chi-square distribution of ILRT is derived. An in-depth simulation study strongly supports its excellent performance even under small group sizes. Furthermore, ILRT developed under the one-way model is found uniformly superior over its peers, is straightway extendable under general multiway setup, and has potential to be extended to other non-normal response variables. Its genuine need in industry, where non-normal responses are commonly encountered, is highlighted through analysis of three real data sets: ILRT strongly picked out the deposition time as influential factor in epitaxial layer experiment, revealed significant impact of spools on fiber life for the failure times of Kevlar 49 fiber data, and gave more accurate parameter estimates in delivery time data experiment, as assessed by various model adequacy tools, where its competitors failed to deliver desired results. [ABSTRACT FROM AUTHOR]

Abstract Background Inference using standard linear regression models (LMs) relies on assumptions that are rarely satisfied in practice. Substantial departures, if not addressed, have serious impacts on any inference and conclusions; potentially rendering them invalid and misleading. Count, bounded and skewed outcomes, common in physical activity research, can substantially violate LM assumptions. A common approach to handle these is to transform the outcome and apply a LM. However, a transformation may not suffice. Methods In this paper, we introduce the generalized linear model (GLM), a generalization of the LM, as an approach for the appropriate modelling of count and non-normally distributed (i.e., bounded and skewed) outcomes. Using data from a study of physical activity among older adults, we demonstrate appropriate methods to analyse count, bounded and skewed outcomes. Results We show how fitting an LM when inappropriate, especially for the type of outcomes commonly encountered in physical activity research, substantially impacts the analysis, inference, and conclusions compared to a GLM. Conclusions GLMs which more appropriately model non-normally distributed response variables should be considered as more suitable approaches for managing count, bounded and skewed outcomes rather than simply relying on transformations. We recommend that physical activity researchers add the GLM to their statistical toolboxes and become aware of situations when GLMs are a better method than traditional approaches for modeling count, bounded and skewed outcomes.

Background: Inference using standard linear regression models (LMs) relies on assumptions that are rarely satisfied in practice. Substantial departures, if not addressed, have serious impacts on any inference and conclusions; potentially rendering them invalid and misleading. Count, bounded and skewed outcomes, common in physical activity research, can substantially violate LM assumptions. A common approach to handle these is to transform the outcome and apply a LM. However, a transformation may not suffice. Methods: In this paper, we introduce the generalized linear model (GLM), a generalization of the LM, as an approach for the appropriate modelling of count and non-normally distributed (i.e., bounded and skewed) outcomes. Using data from a study of physical activity among older adults, we demonstrate appropriate methods to analyse count, bounded and skewed outcomes. Results: We show how fitting an LM when inappropriate, especially for the type of outcomes commonly encountered in physical activity research, substantially impacts the analysis, inference, and conclusions compared to a GLM. Conclusions: GLMs which more appropriately model non-normally distributed response variables should be considered as more suitable approaches for managing count, bounded and skewed outcomes rather than simply relying on transformations. We recommend that physical activity researchers add the GLM to their statistical toolboxes and become aware of situations when GLMs are a better method than traditional approaches for modeling count, bounded and skewed outcomes. [ABSTRACT FROM AUTHOR]

We consider the setting of an aggregate data meta-analysis of a continuous outcome of interest. When the distribution of the outcome is skewed, it is often the case that some primary studies report the sample mean and standard deviation of the outcome and other studies report the sample median along with the first and third quartiles and/or minimum and maximum values. To perform meta-analysis in this context, a number of approaches have recently been developed to impute the sample mean and standard deviation from studies reporting medians. Then, standard meta-analytic approaches with inverse-variance weighting are applied based on the (imputed) study-specific sample means and standard deviations. In this article, we illustrate how this common practice can severely underestimate the within-study standard errors, which results in poor coverage for the pooled mean in common effect meta-analyses and overestimation of between-study heterogeneity in random effects meta-analyses. We propose a straightforward bootstrap approach to estimate the standard errors of the imputed sample means. Our simulation study illustrates how the proposed approach can improve the estimation of the within-study standard errors and consequently improve coverage for the pooled mean in common effect meta-analyses and estimation of between-study heterogeneity in random effects meta-analyses. Moreover, we apply the proposed approach in a meta-analysis to identify risk factors of a severe course of COVID-19. [ABSTRACT FROM AUTHOR]

The usefulness of (probability) distributions in the field of biomedical science cannot be underestimated. Hence, several distributions have been used in this field to perform statistical analyses and make inferences. In this study, we develop the arctan power (AP) distribution and illustrate its application using biomedical data. The distribution is flexible in the sense that its probability density function exhibits characteristics such as left-skewedness, right-skewedness, and J and reversed-J shapes. The characteristic of the corresponding hazard rate function also suggests that the distribution is capable of modeling data with monotonic and non-monotonic failure rates. A bivariate extension of the AP distribution is also created to model the interdependence of two random variables or pairs of data. The application reveals that the AP distribution provides a better fit to the biomedical data than other existing distributions. The parameters of the distribution can also be fairly accurately estimated using a Bayesian approach, which is also elaborated. To end the study, the quantile and modal regression models based on the AP distribution provided better fits to the biomedical data than other existing regression models. [ABSTRACT FROM AUTHOR]

Abstract Background Baseline imbalances, skewed costs, the correlation between costs and effects, and missing data are statistical challenges that are often not adequately accounted for in the analysis of cost-effectiveness data. This study aims to illustrate the impact of accounting for these statistical challenges in trial-based economic evaluations. Methods Data from two trial-based economic evaluations, the REALISE and HypoAware studies, were used. In total, 14 full cost-effectiveness analyses were performed per study, in which the four statistical challenges in trial-based economic evaluations were taken into account step-by-step. Statistical approaches were compared in terms of the resulting cost and effect differences, ICERs, and probabilities of cost-effectiveness. Results In the REALISE study and HypoAware study, the ICER ranged from 636,744€/QALY and 90,989€/QALY when ignoring all statistical challenges to − 7502€/QALY and 46,592€/QALY when accounting for all statistical challenges, respectively. The probabilities of the intervention being cost-effective at 0€/ QALY gained were 0.67 and 0.59 when ignoring all statistical challenges, and 0.54 and 0.27 when all of the statistical challenges were taken into account for the REALISE study and HypoAware study, respectively. Conclusions Not accounting for baseline imbalances, skewed costs, correlated costs and effects, and missing data in trial-based economic evaluations may notably impact results. Therefore, when conducting trial-based economic evaluations, it is important to align the statistical approach with the identified statistical challenges in cost-effectiveness data. To facilitate researchers in handling statistical challenges in trial-based economic evaluations, software code is provided.

Background Dependent variables in health psychology are often counts, for example, of a behaviour or number of engagements with an intervention. These counts can be very strongly skewed, and/or contain large numbers of zeros as well as extreme outliers. For example, ‘How many cigarettes do you smoke on an average day?’ The modal answer may be zero but may range from 0 to 40+. The same can be true for minutes of moderate-to-vigorous physical activity. For some people, this may be near zero, but take on extreme values for someone training for a marathon. Typical analytical strategies for this data involve explicit (or implied) transformations (smoker v. non-smoker, log transformations). However, these data types are ‘counts’ (i.e. non-negative whole numbers) or quasi-counts (time is ratio but discrete minutes of activity could be analysed as a count), and can be modelled using count distributions – including the Poisson and negative binomial distribution (and their zero-inflated and hurdle extensions, which alloweven more zeros). Methods In this tutorial paper I demonstrate (in R, Jamovi, and SPSS) the easy application of these models to health psychology data, and their advantages over alternative ways of analysing this type of data using two datasets – one highly dispersed dependent variable (number of views on YouTube, and another with a large number of zeros (number of days on which symptoms were reported over a month). Results The negative binomial distribution had the best fit for the overdispersed number of views on YouTube. Negative binomial, and zero-inflated negative binomial were both good fits for the symptom data with over-abundant zeros. Conclusions In both cases, count distributions provided not just a better fit but would lead to different conclusions compared to the poorly fitting traditional regression/linear models.

Dependent variables in health psychology are often counts, for example, of a behaviour or number of engagements with an intervention. These counts can be very strongly skewed, and/or contain large numbers of zeros as well as extreme outliers. For example, 'How many cigarettes do you smoke on an average day?' The modal answer may be zero but may range from 0 to 40+. The same can be true for minutes of moderate-to-vigorous physical activity. For some people, this may be near zero, but take on extreme values for someone training for a marathon. Typical analytical strategies for this data involve explicit (or implied) transformations (smoker v. non-smoker, log transformations). However, these data types are 'counts' (i.e. non-negative whole numbers) or quasi-counts (time is ratio but discrete minutes of activity could be analysed as a count), and can be modelled using count distributions – including the Poisson and negative binomial distribution (and their zero-inflated and hurdle extensions, which alloweven more zeros). In this tutorial paper I demonstrate (in R, Jamovi, and SPSS) the easy application of these models to health psychology data, and their advantages over alternative ways of analysing this type of data using two datasets – one highly dispersed dependent variable (number of views on YouTube, and another with a large number of zeros (number of days on which symptoms were reported over a month). The negative binomial distribution had the best fit for the overdispersed number of views on YouTube. Negative binomial, and zero-inflated negative binomial were both good fits for the symptom data with over-abundant zeros. In both cases, count distributions provided not just a better fit but would lead to different conclusions compared to the poorly fitting traditional regression/linear models. [ABSTRACT FROM AUTHOR]

The usefulness of (probability) distributions in the field of biomedical science cannot be underestimated. Hence, several distributions have been used in this field to perform statistical analyses and make inferences. In this study, we develop the arctan power (AP) distribution and illustrate its application using biomedical data. The distribution is flexible in the sense that its probability density function exhibits characteristics such as left-skewedness, right-skewedness, and J and reversed-J shapes. The characteristic of the corresponding hazard rate function also suggests that the distribution is capable of modeling data with monotonic and non-monotonic failure rates. A bivariate extension of the AP distribution is also created to model the interdependence of two random variables or pairs of data. The application reveals that the AP distribution provides a better fit to the biomedical data than other existing distributions. The parameters of the distribution can also be fairly accurately estimated using a Bayesian approach, which is also elaborated. To end the study, the quantile and modal regression models based on the AP distribution provided better fits to the biomedical data than other existing regression models.

A major challenge when trying to detect fraud is that the fraudulent activities form a minority class which make up a very small proportion of the data set. In most data sets, fraud occurs in typically less than 0.5 % of the cases. Detecting fraud in such a highly imbalanced data set typically leads to predictions that favor the majority group, causing fraud to remain undetected. We discuss some popular oversampling techniques that solve the problem of imbalanced data by creating synthetic samples that mimic the minority class. A frequent problem when analyzing real data is the presence of anomalies or outliers. When such atypical observations are present in the data, most oversampling techniques are prone to create synthetic samples that distort the detection algorithm and spoil the resulting analysis. A useful tool for anomaly detection is robust statistics, which aims to find the outliers by first fitting the majority of the data and then flagging data observations that deviate from it. In this paper, we present a robust version of ROSE, called robROSE, which combines several promising approaches to cope simultaneously with the problem of imbalanced data and the presence of outliers. The proposed method achieves to enhance the presence of the fraud cases while ignoring anomalies. The good performance of our new sampling technique is illustrated on simulated and real data sets and it is shown that robROSE can provide better insight in the structure of the data. The source code of the robROSE algorithm is made freely available. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a new three-parameter lifetime model called the Topp–Leone odd log-logistic exponential distribution is proposed. Its density function can be expressed as a linear mixture of exponentiated exponential densities and can be reversed-J shaped, skewed to the left and to the right. Further, the hazard rate function of the new model can be monotone, unimodal, constant, J-shaped, constant-increasing-decreasing and decreasing-increasing-decreasing and bathtub-shaped. Our main focus is on estimation from a frequentist point of view, yet, some statistical and reliability characteristics for the proposed model are derived. We briefly describe different estimators namely, the maximum likelihood estimators, ordinary least-squares estimators, weighted least-squares estimators, percentile estimators, maximum product of spacings estimators, Cramér-von-Mises minimum distance estimators, Anderson-Darling estimators and right-tail Anderson-Darling estimators. Monte Carlo simulations are performed to compare the performance of the proposed methods of estimation for both small and large samples. We illustrate the performance of the proposed distribution by means of two real data sets and both the data sets show the new distribution is more appropriate as compared to some other well-known distributions.

Background: Baseline imbalances, skewed costs, the correlation between costs and effects, and missing data are statistical challenges that are often not adequately accounted for in the analysis of cost-effectiveness data. This study aims to illustrate the impact of accounting for these statistical challenges in trial-based economic evaluations.Methods: Data from two trial-based economic evaluations, the REALISE and HypoAware studies, were used. In total, 14 full cost-effectiveness analyses were performed per study, in which the four statistical challenges in trial-based economic evaluations were taken into account step-by-step. Statistical approaches were compared in terms of the resulting cost and effect differences, ICERs, and probabilities of cost-effectiveness.Results: In the REALISE study and HypoAware study, the ICER ranged from 636,744€/QALY and 90,989€/QALY when ignoring all statistical challenges to - 7502€/QALY and 46,592€/QALY when accounting for all statistical challenges, respectively. The probabilities of the intervention being cost-effective at 0€/ QALY gained were 0.67 and 0.59 when ignoring all statistical challenges, and 0.54 and 0.27 when all of the statistical challenges were taken into account for the REALISE study and HypoAware study, respectively.Conclusions: Not accounting for baseline imbalances, skewed costs, correlated costs and effects, and missing data in trial-based economic evaluations may notably impact results. Therefore, when conducting trial-based economic evaluations, it is important to align the statistical approach with the identified statistical challenges in cost-effectiveness data. To facilitate researchers in handling statistical challenges in trial-based economic evaluations, software code is provided. [ABSTRACT FROM AUTHOR]

Scientists in biomedical and psychosocial research need to deal with skewed data all the time. In the case of comparing means from two groups, the log transformation is commonly used as a traditional technique to normalize skewed data before utilizing the two-group t-test. An alternative method that does not assume normality is the generalized linear model (GLM) combined with an appropriate link function. In this work, the two techniques are compared using Monte Carlo simulations; each consists of many iterations that simulate two groups of skewed data for three different sampling distributions: gamma, exponential, and beta. Afterward, both methods are compared regarding Type I error rates, power rates and the estimates of the mean differences. We conclude that the t-test with log transformation had superior performance over the GLM method for any data that are not normal and follow beta or gamma distributions. Alternatively, for exponentially distributed data, the GLM method had superior performance over the t-test with log transformation. [ABSTRACT FROM AUTHOR]

The problem of efficiently classifying imbalanced data has become one of the most challenging tasks in machine learning. Some real world examples include medical image analysis, fraud detection, fault diagnosis, and anomaly detection. Although several data-level algorithms have been developed to address imbalance, they are typically subject to some restrictions. We propose a novel variant of the k -NN family of classifiers, and name this as Density-based Adaptive-distance k NN (DA k NN). It can effectively handle data with skewed distributions and varying class-densities using the concept of adaptive distance. Comparative superiority is experimentally established over related data-level algorithms (SMOTE, ADASYN), using ten sets of two-class data, in terms of geometric mean (of the true positive and negative rates) and accuracy. Additionally, five sets of multi-class data are considered and compared with different variants of k -NN, which are currently very popular. Finally, DA k NN is successfully applied on the highly imbalanced Lower Grade Glioma (LGG) MR images, with an Average-Dice score of 0.9082 for delineating the tumor regions. The results demonstrate clear superiority over state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

We consider the problem of meta‐analyzing two‐group studies that report the median of the outcome. Often, these studies are excluded from meta‐analysis because there are no well‐established statistical methods to pool the difference of medians. To include these studies in meta‐analysis, several authors have recently proposed methods to estimate the sample mean and standard deviation from the median, sample size, and several commonly reported measures of spread. Researchers frequently apply these methods to estimate the difference of means and its variance for each primary study and pool the difference of means using inverse variance weighting. In this work, we develop several methods to directly meta‐analyze the difference of medians. We conduct a simulation study evaluating the performance of the proposed median‐based methods and the competing transformation‐based methods. The simulation results show that the median‐based methods outperform the transformation‐based methods when meta‐analyzing studies that report the median of the outcome, especially when the outcome is skewed. Moreover, we illustrate the various methods on a real‐life data set. [ABSTRACT FROM AUTHOR]

This study investigates the level of agreement between problem gamblers and their concerned significant others (CSOs) regarding the amount of money lost when gambling. Reported losses were analyzed from 266 participants (133 dyads) seeking treatment, which included different types of CSO-gambler dyads. The intraclass correlation coefficients (ICCs) concerning the money lost when gambling during the last 30 days were calculated based on the timeline followback. In order to model reports that were highly skewed and included zeros, a two-part generalized linear mixed-effects model was used. The results were compared from models assuming either a Gaussian, two-part gamma, or two-part lognormal response distribution. Overall, the results indicated a fair level of agreement, ICC = .57, 95% CI (.48, .64), between the gamblers and their CSOs. The partner CSOs tended to exhibit better agreement than the parent CSOs with regard to the amount of money lost, ICCdiff = .20, 95% CI (.03, .39). The difference became smaller and inconclusive when reports of no losses (zeros) were included, ICCdiff = .16, 95% CI (- .05, .36). A small simulation investigation indicated that the two-part model worked well under assumptions related to this study, and further, that calculating the ICCs under normal assumptions led to incorrect conclusions regarding the level of agreement for skewed reports (such as gambling losses). For gambling losses, the normal assumption is unlikely to hold and ICCs based on this assumption are likely to be highly unreliable. [ABSTRACT FROM AUTHOR]

Abstract Background In a random effects meta-analysis model, true treatment effects for each study are routinely assumed to follow a normal distribution. However, normality is a restrictive assumption and the misspecification of the random effects distribution may result in a misleading estimate of overall mean for the treatment effect, an inappropriate quantification of heterogeneity across studies and a wrongly symmetric prediction interval. Methods We focus on problems caused by an inappropriate normality assumption of the random effects distribution, and propose a novel random effects meta-analysis model where a Box-Cox transformation is applied to the observed treatment effect estimates. The proposed model aims to normalise an overall distribution of observed treatment effect estimates, which is sum of the within-study sampling distributions and the random effects distribution. When sampling distributions are approximately normal, non-normality in the overall distribution will be mainly due to the random effects distribution, especially when the between-study variation is large relative to the within-study variation. The Box-Cox transformation addresses this flexibly according to the observed departure from normality. We use a Bayesian approach for estimating parameters in the proposed model, and suggest summarising the meta-analysis results by an overall median, an interquartile range and a prediction interval. The model can be applied for any kind of variables once the treatment effect estimate is defined from the variable. Results A simulation study suggested that when the overall distribution of treatment effect estimates are skewed, the overall mean and conventional I 2 from the normal random effects model could be inappropriate summaries, and the proposed model helped reduce this issue. We illustrated the proposed model using two examples, which revealed some important differences on summary results, heterogeneity measures and prediction intervals from the normal random effects model. Conclusions The random effects meta-analysis with the Box-Cox transformation may be an important tool for examining robustness of traditional meta-analysis results against skewness on the observed treatment effect estimates. Further critical evaluation of the method is needed.

The distribution of progression scores in rheumatoid arthritis is highly skewed, requiring advanced statistical analysis techniques, with different techniques resulting in different outcomes. Three databases were searched to identify rheumatoid arthritis clinical trials and observational studies that described radiographic analysis techniques, comparing at least two groups. Of 5980 identified papers, 225 were eligible for data extraction. Parametric techniques (t-tests, ANOVA or linear regression) were used in 39 studies, of which 18% took the skewed distribution into account. In 53 studies, continuous data was categorized and analyzed with binomial or ordinal methods (chi-square tests or logistic regression). Two studies treated the outcome as a 'count' outcome variable (applying a Poisson). There is large heterogeneity in the analysis strategy of radiographic progression in recent rheumatoid arthritis clinical trials and observational studies, with the majority of studies applying simple, suboptimal or inappropriate methods. [ABSTRACT FROM AUTHOR]

Twin Support Vector Machines (TWSVMs) have emerged as an efficient alternative to Support Vector Machines (SVM) for learning from imbalanced datasets. The TWSVM learns two non-parallel classifying hyperplanes by solving a couple of smaller sized problems. However, it is unsuitable for large datasets, as it involves matrix operations. In this paper, we discuss a Twin Neural Network (Twin NN) architecture for learning from large unbalanced datasets. The objective functions of the networks in the Twin NN are designed to realize the idea of the Twin SVM with non-parallel decision boudaries for the respective classes, while also being able to reduce model complexity. The Twin NN optimizes the feature map, allowing for better discrimination between classes. The paper also discusses an extension of the Twin NN for multiclass datasets. This architecture trains as many neural networks as the number of classes, and has the additional advantage that it does not have any hyper-parameter which requires tuning. Results presented in the paper demonstrate that the Twin NN generalizes well and scales well on large unbalanced datasets. [ABSTRACT FROM AUTHOR]

An aggregate data meta‐analysis is a statistical method that pools the summary statistics of several selected studies to estimate the outcome of interest. When considering a continuous outcome, typically each study must report the same measure of the outcome variable and its spread (eg, the sample mean and its standard error). However, some studies may instead report the median along with various measures of spread. Recently, the task of incorporating medians in meta‐analysis has been achieved by estimating the sample mean and its standard error from each study that reports a median in order to meta‐analyze the means. In this paper, we propose two alternative approaches to meta‐analyze data that instead rely on medians. We systematically compare these approaches via simulation study to each other and to methods that transform the study‐specific medians and spread into sample means and their standard errors. We demonstrate that the proposed median‐based approaches perform better than the transformation‐based approaches, especially when applied to skewed data and data with high inter‐study variance. Finally, we illustrate these approaches in a meta‐analysis of patient delay in tuberculosis diagnosis. [ABSTRACT FROM AUTHOR]

Scientists in biomedical and psychosocial research need to deal with skewed data all the time. In the case of comparing means from two groups, the log transformation is commonly used as a traditional technique to normalize skewed data before utilizing the two-group t-test. An alternative method that does not assume normality is the generalized linear model (GLM) combined with an appropriate link function. In this work, the two techniques are compared using Monte Carlo simulations; each consists of many iterations that simulate two groups of skewed data for three different sampling distributions: gamma, exponential, and beta. Afterward, both methods are compared regarding Type I error rates, power rates and the estimates of the mean differences. We conclude that the t-test with log transformation had superior performance over the GLM method for any data that are not normal and follow beta or gamma distributions. Alternatively, for exponentially distributed data, the GLM method had superior performance over the t-test with log transformation.

In recent years, several extensions of the Hadoop system have been proposed for dealing with spatial data. SpatialHadoop belongs to this group of projects and includes some MapReduce implementations of spatial operators, like range queries and spatial join. the MapReduce paradigm is based on the fundamental principle that a task can be parallelized by partitioning data into chunks and performing the same operation on them, (map phase), eventually combining the partial results at the end (reduce phase). Thus, the applied partitioning technique can tremendously affect the performance of a parallel execution, since it is the key point for obtaining balanced map tasks and exploiting the parallelism as much as possible. When uniformly distributed datasets are considered, this goal can be easily obtained by using a regular grid covering the whole reference space for partitioning the geometries of the input dataset; conversely, with skewed distributed datasets, this might not be the right choice and other techniques have to be applied. for instance, SpatialHadoop can produce a global index also by means of a Quadtree-based grid or an Rtree-based grid, which in turn are more expensive index structures to build. This paper proposes a technique based on both a box counting function and a heuristic, rooted on theoretical properties and experimental observations, for detecting the degree of skewness of an input spatial dataset and then deciding which partitioning technique to apply in order to improve as much as possible the performance of subsequent operations. Experiments on both synthetic and real datasets are presented to confirm the effectiveness of the proposed approach.

In sleep research, applying finite mixture models to sleep characteristics captured through multiple data types, including self-reported sleep diary, a wrist monitor capturing movement (actigraphy), and brain waves (polysomnography), may suggest new phenotypes that reflect underlying disease mechanisms. However, a direct mixture model application is challenging because there are many sleep variables from which to choose, and sleep variables are often highly skewed even in homogenous samples. Moreover, previous sleep research findings indicate that some of the most clinically interesting solutions will be those that incorporate all three data types. Thus, we present two novel skewed variable selection algorithms based on the multivariate skew normal (MSN) distribution: one that selects the best set of variables ignoring data type and another that embraces the exploratory nature of clustering and suggests multiple statistically plausible sets of variables that each incorporate all data types. Through a simulation study, we empirically compare our approach with other asymmetric and normal dimension reduction strategies for clustering. Finally, we demonstrate our methods using a sample of older adults with and without insomnia. The proposed MSN-based variable selection algorithm appears to be suitable for both MSN and multivariate normal cluster distributions, especially with moderate to large-sample sizes. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Background: Some respondents may respond at random to self-report surveys, rather than responding conscientiously (Meade and Craig, 2012), and this has only recently come to the attention of researchers in the addictions field (Godinho et al., 2016). Almost no research in the published addictions literature has reported screening for random responses. We illustrate how random responses can bias statistical estimates using simulated and real data, and how this is especially problematic in skewed data, as is common with substance use outcomes.Method: We first tested the effects of varying amounts and types of random responses on covariance-based statistical estimates in distributions with varying amounts of skew. We replicated these findings in correlations from a real dataset (Add Health) by replacing varying amounts of real data with simulated random responses.Results: Skew and the proportion of random responses influenced the amount and direction of bias. When the data were not skewed, uniformly random responses deflated estimates, while long-string random responses inflated estimates. As the distributions became more skewed, all types of random responses began to inflate estimates, even at very small proportions. We observed similar effects in the Add Health data.Conclusions: Failing to screen for random responses in survey data produces biased statistical estimates, and data with only 2.5% random responses can inflate covariance-based estimates (i.e., correlations, Cronbach's alpha, regression coefficients, factor loadings, etc.) when data are heavily skewed. Screening for random responses can substantially improve data quality, reliability and validity. [ABSTRACT FROM AUTHOR]

Purpose: The aim of this study is to compare the methods used for the estimation of the area under the ROC curve (AUC) with skewed data. Materials and Methods: The area under the ROC curve can be estimated by using parametric or nonparametric methods. In the parametric method, the distributions of biomarker in both patient and non-patient groups are assumed to be normal distribution or transformed to the normal distribution. In non-parametric methods, there are two approaches: using the rank statistics or using the kernel smoothing methods. In this study, these methods are compared by using simulated skewed data and real data. Results: According to the simulation results, the estimates of the investigated methods are affected by the real AUC value, the sample size and the degree of the skewness. For the scenarios with the skewed normal distribution, Mann Whitney method gets the smallest bias value and fort he scenario with the gamma distribution, Binormal model with a Box-Cox transformation gets the smallest bias value. Conclusion: For skewed data, after applying transformation techniques, the Binormal model can be used to estimate the AUC. After Box-Cox transformation, the estimate of the Binormal model is very close to the real AUC value for highly skewed data. It is recommended to use this method for this situation. [ABSTRACT FROM AUTHOR]