Search

The impact of missing data on statistical inference varies depending on several factors such as the proportion of missingness, missing-data mechanism, and method employed to handle missing values. While these topics have been extensively studied, most recommendations have been made assuming that all missing values are from the same missing-data mechanism. In reality, it is very likely that a mixture of missing-data mechanisms is responsible for missing values in a dataset and even within the same pattern of missingness. Although a mixture of missing-data mechanisms and causes within a dataset is a likely scenario, the performance of popular missing-data methods under these circumstances is unknown. This study provides a realistic evaluation of methods for handling missing data in this setting using Monte Carlo simulation in the context of regression. This study also seeks to identify acceptable proportions of missing values that violate the missing-data mechanism assumed by the method used to handle missing values. Results indicate that multiple imputation (MI) performs better than other principled or ad-hoc methods. Different missing-data methods are also compared via the analysis of a real dataset in which mixtures of missingness mechanisms are created. Recommendations are provided for the use of different methods in practice.

Chi-square type test statistics are widely used in assessing the goodness-of-fit of a theoretical model. The exact distributions of such statistics can be quite different from the nominal chi-squar...

With single-level data, Yuan, Cheng and Maxwell developed a two-level regression model for more accurate moderation analysis. This article extends the two-level regression model to a two-level moderated latent variable (2MLV) model, and uses a Bayesian approach to estimate and test the moderation effects. Monte Carlo results indicate that: 1) the new method yields more accurate estimate of the interaction effect than those via the product-indicator (PI) approach and latent variable interaction (LVI) with single-level model, both are also estimated via Bayesian method; 2) the coverage rates of the credibility interval following the 2MLV model are closer to the nominal 95% than those following the other methods; 3) the test for the existence of the moderation effect is more reliable in controlling Type I errors than both PI and LVI, especially under heteroscedasticity conditions. Moreover, a more interpretable measure of effect size is developed based on the 2MLV model, which directly answers the question as to what extent a moderator can account for the change of the coefficient between the predictor and the outcome variable. A real data example illustrates the application of the new method.

Data in social sciences are typically non-normally distributed and characterized by heavy tails. However, most widely used methods in social sciences are still based on the analyses of sample means and sample covariances. While these conventional methods continue to be used to address new substantive issues, conclusions reached can be inaccurate or misleading. Although there is no 'best method' in practice, robust methods that consider the distribution of the data can perform substantially better than the conventional methods. This article gives an overview of robust procedures, emphasizing a few that have been repeatedly shown to work well for models that are widely used in social and behavioural sciences. Real data examples show how to use the robust methods for latent variable models and for moderated mediation analysis when a regression model contains categorical covariates and product terms. Results and logical analyses indicate that robust methods yield more efficient parameter estimates, more reliable model evaluation, more reliable model/data diagnostics, and more trustworthy conclusions when conducting replication studies. R and SAS programs are provided for routine applications of the recommended robust method.

Moderation analysis is useful for addressing interesting research questions in social sciences and behavioural research. In practice, moderated multiple regression (MMR) models have been most widely used. However, missing data pose a challenge, mainly because the interaction term is a product of two or more variables and thus is a non-linear function of the involved variables. Normal-distribution-based maximum likelihood (NML) has been proposed and applied for estimating MMR models with incomplete data. When data are missing completely at random, moderation effect estimates are consistent. However, simulation results have found that when data in the predictor are missing at random (MAR), NML can yield inaccurate estimates of moderation effects when the moderation effects are non-null. Simulation studies are subject to the limitation of confounding systematic bias with sampling errors. Thus, the purpose of this paper is to analytically derive asymptotic bias of NML estimates of moderation effects with MAR data. Results show that when the moderation effect is zero, there is no asymptotic bias in moderation effect estimates with either normal or non-normal data. When the moderation effect is non-zero, however, asymptotic bias may exist and is determined by factors such as the moderation effect size, missing-data proportion, and type of missingness dependence. Our analytical results suggest that researchers should apply NML to MMR models with caution when missing data exist. Suggestions are given regarding moderation analysis with missing data.

Survey data often contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. However, conventional SEM methods are not crafted to handle data with a large number of variables (

Moderation analysis has many applications in social sciences. Most widely used estimation methods for moderation analysis assume that errors are normally distributed and homoscedastic. When these assumptions are not met, the results from a classical moderation analysis can be misleading. For more reliable moderation analysis, this article proposes two robust methods with a two-level regression model when the predictors do not contain measurement error. One method is based on maximum likelihood with Student's t distribution and the other is based on M-estimators with Huber-type weights. An algorithm for obtaining the robust estimators is developed. Consistent estimates of standard errors of the robust estimators are provided. The robust approaches are compared against normal-distribution-based maximum likelihood (NML) with respect to power and accuracy of parameter estimates through a simulation study. Results show that the robust approaches outperform NML under various distributional conditions. Application of the robust methods is illustrated through a real data example. An R program is developed and documented to facilitate the application of the robust methods.

In structural equation modeling (SEM), parameter estimates are typically computed by the Fisher-scoring algorithm, which often has difficulty in obtaining converged solutions. Even for simulated data with a correctly specified model, non-converged replications have been repeatedly reported in the literature. In particular, in Monte Carlo studies it has been found that larger factor loadings or smaller error variances in a confirmatory factor model correspond to a higher rate of convergence. However, studies of a ridge method in SEM indicate that adding a diagonal matrix to the sample covariance matrix also increases the rate of convergence for the Fisher-scoring algorithm. This article addresses these two seemingly contradictory phenomena. Using statistical and numerical analyses, the article clarifies why both approaches increase the rate of convergence in SEM. Monte Carlo results confirm the analytical results. Recommendations are provided on how to increase both the speed and rate of convergence in parameter estimation.

Survey data often contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. With typical nonnormally distributed data in practice, a rescaled statistic Trml proposed by Satorra and Bentler was recommended in the literature of SEM. However, Trml has been shown to be problematic when the sample size N is small and/or the number of variables p is large. There does not exist a reliable test statistic for SEM with small N or large p, especially with nonnormally distributed data. Following the principle of Bartlett correction, this article develops empirical corrections to Trml so that the mean of the empirically corrected statistics approximately equals the degrees of freedom of the nominal chi-square distribution. Results show that empirically corrected statistics control type I errors reasonably well even when N is smaller than 2p, where Trml may reject the correct model 100% even for normally distributed data. The application of the empirically corrected sta...

Means and covariance/dispersion matrix are the building blocks for many statistical analyses. By naturally extending the score functions based on a multivariate $$t$$ -distribution to estimating equations, this article defines a class of M-estimators of means and dispersion matrix for samples with missing data. An expectation-robust (ER) algorithm solving the estimating equations is obtained. The obtained relationship between the ER algorithm and the corresponding estimating equations allows us to obtain consistent standard errors when robust means and dispersion matrix are further analyzed. Estimating equations corresponding to existing ER algorithms for computing M- and S-estimators are also identified. Monte Carlo results show that robust methods outperform the normal-distribution-based maximum likelihood when the population distribution has heavy tails or when data are contaminated. Applications of the results to robust analysis of linear regression and growth curve models are discussed.

Normal-distribution-based maximum likelihood (NML) is most widely used for missing data analysis although real data seldom follow a normal distribution. When missing values are missing at random (MAR), recent results indicate that NML estimates (NMLEs) are still consistent for nonnormally distributed populations as long as the variables are linearly related. However, NMLEs are generally not consistent when the variables are nonlinearly related in the population. Similarly, NMLEs are generally not consistent when data are missing not at random (MNAR). It is well-known that including proper auxiliary variables mitigates the bias in MLEs caused by MNAR mechanism. With nonlinear relationships underlying the manifest variables and under MAR mechanism, the article contains a theoretical result showing that NMLEs are still consistent when proper nonlinear functions of the observed variables are included as auxiliary variables. Empirical results indicate that including auxiliary variables reduces bias in the estimates, but may also increase their standard errors substantially when sample size is small and the proportion of missing data is not trivial. Empirical results also imply that bias in NMLEs due to a nonnormally distributed population and MAR mechanism can be considerably greater when compared to bias caused by MNAR mechanism with a normally distributed population. How to select auxiliary variables in practice is also discussed.

This paper reviews various methods of identifying missing data mechanisms. The three well-known mechanisms of missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) are considered. A number of tests deem rejection of homogeneity of means and/or covariances (HMC) among observed data patterns as a means to reject MCAR. Utility of these tests as well as their shortcomings are discussed. In particular, examples of MAR and MNAR data with homogeneous means and covariances between their observed data patterns are provided for which tests of HMC fail to reject MCAR. More generally, tests of homogeneity of parameter estimates between various subsets of data are reviewed and their utility as tests of MCAR and MAR (in special cases) is pointed out. Since many tests of MCAR assume multinormality, methods to assess this assumption in the context of incomplete data are reviewed. Tests of homogeneity of distributions among observed data patterns for MCAR are also considered. A new nonparametric test of this type is proposed on the basis of pairwise comparison of marginal distributions. Finally, methods of examining missing data mechanism based on sensitivity analysis including methods that model missing data mechanism based on logistic, probit, and latent variable regression models, as well as methods that do not require modeling of missing data mechanism are reviewed. The paper concludes with some practical comments about the validity and utility of tests of missing data mechanism. © 2013 Wiley Periodicals, Inc. How to cite this article

Missing data are a common problem in almost all areas of empirical research. Ignoring the missing data mechanism, especially when data are missing not at random (MNAR), can result in biased and/or inefficient inference. Because MNAR mechanism is not verifiable based on the observed data, sensitivity analysis is often used to assess it. Current sensitivity analysis methods primarily assume a model for the response mechanism in conjunction with a measurement model and examine sensitivity to missing data mechanism via the parameters of the response model. Recently, Jamshidian and Mata (Post-modelling sensitivity analysis to detect the effect of missing data mechanism, Multivariate Behav. Res. 43 (2008), pp. 432–452) introduced a new method of sensitivity analysis that does not require the difficult task of modelling the missing data mechanism. In this method, a single measurement model is fitted to all of the data and to a sub-sample of the data. Discrepancy in the parameter estimates obtained from the the t...

Data in psychology are often collected using Likert-type scales, and it has been shown that factor analysis of Likert-type data is better performed on the polychoric correlation matrix than on the product-moment covariance matrix, especially when the distributions of the observed variables are skewed. In theory, factor analysis of the polychoric correlation matrix is best conducted using generalized least squares with an asymptotically correct weight matrix (AGLS). However, simulation studies showed that both least squares (LS) and diagonally weighted least squares (DWLS) perform better than AGLS, and thus LS or DWLS is routinely used in practice. In either LS or DWLS, the associations among the polychoric correlation coefficients are completely ignored. To mend such a gap between statistical theory and empirical work, this paper proposes new methods, called ridge GLS, for factor analysis of ordinal data. Monte Carlo results show that, for a wide range of sample sizes, ridge GLS methods yield uniformly more accurate parameter estimates than existing methods (LS, DWLS, AGLS). A real-data example indicates that estimates by ridge GLS are 9-20% more efficient than those by existing methods. Rescaled and adjusted test statistics as well as sandwich-type standard errors following the ridge GLS methods also perform reasonably well.

This paper develops a ridge procedure for structural equation modelling (SEM) with ordinal and continuous data by modelling the polychoric/polyserial/product-moment correlation matrix R. Rather than directly fitting R, the procedure fits a structural model to R(a) =R+aI by minimizing the normal distribution-based discrepancy function, where a > 0. Statistical properties of the parameter estimates are obtained. Four statistics for overall model evaluation are proposed. Empirical results indicate that the ridge procedure for SEM with ordinal data has better convergence rate, smaller bias, smaller mean square error, and better overall model evaluation than the widely used maximum likelihood procedure.

This article shows that, when variables with missing values are linearly related to observed variables, the normal-distribution-based pseudo MLEs are still consistent. The population distribution may be unknown while the missing data process can follow an arbitrary missing at random mechanism. Enough details are provided for the bivariate case so that readers having taken a course in statistics/probability can fully understand the development. Sufficient conditions for the consistency of the MLEs in higher dimensions are also stated, while the details are omitted.

Existing studies of mediation models have been limited to normal-theory maximum likelihood (ML). Because real data in the social and behavioral sciences are seldom normally distributed and often contain outliers, classical methods generally lead to inefficient or biased parameter estimates. Consequently, the conclusions from a mediation analysis can be misleading. In this article, we propose 2 approaches to alleviate these problems. One is to identify cases that strongly affect testing of mediation using local influence methods or robust methods. The other is to use robust methods for parameter estimation and subsequently test the mediated effect based on the robust estimates. The application of these 2 approaches is illustrated using 1 simulated and 2 real data examples. The interest in 1 real data set is the relationship among marital conflict, children's emotional insecurity, and children's internalizing problems. The other example is concerned with whether ethnic identity mediates the effect of family cohesion on Korean language fluency. Results show that local influence and robust methods rank the influence of cases similarly, but robust methods are more objective. Moreover, when the normality assumption is violated, robust methods give estimates with smaller standard errors and more reliable tests of the mediated effect compared with normal-theory ML. An R program that implements the local influence and robust procedures for mediation analysis is also provided.

Evaluating the fit of a structural equation model via bootstrap requires a transformation of the data so that the null hypothesis holds exactly in the sample. For complete data, such a transformation was proposed by Beran and Srivastava (1985) for general covariance structure models and applied to structural equation modeling by Bollen and Stine (1992) . An extension of this transformation to missing data was presented by Enders (2002) , but it is an approximate and not an exact solution, with the degree of approximation unknown. In this article, we provide several approaches to obtaining an exact solution. First, an explicit solution for the special case when the sample covariance matrix within each missing data pattern is invertible is given. Second, 2 iterative algorithms are described for obtaining an exact solution in the general case. We evaluate the rejection rates of the bootstrapped likelihood ratio statistic obtained via the new procedures in a Monte Carlo study. Our main finding is that model-based bootstrap with incomplete data performs quite well across a variety of distributional conditions, missing data mechanisms, and proportions of missing data. We illustrate our new procedures using empirical data on 26 cognitive ability measures in junior high students, published in Holzinger and Swineford (1939) .

When missing data are either missing completely at random (MCAR) or missing at random (MAR), the maximum likelihood (ML) estimation procedure preserves many of its properties. However, in any statistical modeling, the distribution specification for the likelihood function is at best only an approximation to the real world. In particular, since the normal-distribution-based ML is typically applied to data with heterogeneous marginal skewness and kurtosis, it is necessary to know whether such a practice still generates consistent parameter estimates. When the manifest variables are linear combinations of independent random components and missing data are MAR, this paper shows that the normal-distribution-based MLE is consistent regardless of the distribution of the sample. Examples also show that the consistency of the MLE is not guaranteed for all nonnormally distributed samples. When the population follows a confirmatory factor model, and data are missing due to the magnitude of the factors, the MLE may not be consistent even when data are normally distributed. When data are missing due to the magnitude of measurement errors/uniqueness, MLEs for many of the covariance parameters related to the missing variables are still consistent. This paper also identifies and discusses the factors that affect the asymptotic biases of the MLE when data are not missing at random. In addition, the paper also shows that, under certain data models and MAR mechanism, the MLE is asymptotically normally distributed and the asymptotic covariance matrix is consistently estimated by the commonly used sandwich-type covariance matrix. The results indicate that certain formulas and/or conclusions in the existing literature may not be entirely correct.

This article provides the theory and application of the 2-stage maximum likelihood (ML) procedure for structural equation modeling (SEM) with missing data. The validity of this procedure does not require the assumption of a normally distributed population. When the population is normally distributed and all missing data are missing at random (MAR), the direct ML procedure is nearly optimal for SEM with missing data. When missing data mechanisms are unknown, including auxiliary variables in the analysis will make the missing data mechanism more likely to be MAR. It is much easier to include auxiliary variables in the 2-stage ML than in the direct ML. Based on most recent developments for missing data with an unknown population distribution, the article first provides the least technical material on why the normal distribution-based ML generates consistent parameter estimates when the missing data mechanism is MAR. The article also provides sufficient conditions for the 2-stage ML to be a valid statistical procedure in the general case. For the application of the 2-stage ML, an SAS IML program is given to perform the first-stage analysis and EQS codes are provided to perform the second-stage analysis. An example with open- and closed-book examination data is used to illustrate the application of the provided programs. One aim is for quantitative graduate students/applied psychometricians to understand the technical details for missing data analysis. Another aim is for applied researchers to use the method properly.

Conventional structural equation modeling involves fitting a structural model to the sample covariance matrix S. Due to collinearity or small samples with practical data, nonconvergences often occur in the estimation process. For a small constant a, this paper proposes to fit the structural model to the covariance matrix S"a=S+aI. When treating S"a as the sample covariance matrix in the maximum likelihood (ML) procedure, consistent parameter estimates are still obtained. The asymptotic distributions of the parameter estimates and the corresponding likelihood ratio statistic are studied and compared to those by the conventional ML. Two rescaled statistics for the overall model evaluation with modeling S"a are constructed. Empirical results imply that the estimates from modeling S"a are more efficient than those of fitting the structural model to S even when data are normally distributed. Simulations and real data examples indicate that modeling S"a allows us to evaluate the overall model structure even when S is literally singular. Implications of modeling S"a in a broader context are discussed.

In the literature of mean and covariance structure analysis, noncentral chi-square distribution is commonly used to describe the behavior of the likelihood ratio (LR) statistic under alternative hypothesis. Due to the inaccessibility of the rather technical literature for the distribution of the LR statistic, it is widely believed that the noncentral chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the noncentral chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Using the one-dimensional case, this article provides the details showing that the LR statistic asymptotically follows a normal distribution, which also leads to an asymptotically correct confidence interval for the discrepancy between the null hypothesis/model and the population. For each one-dimensional result, the corresponding results in the higher dimensional case are pointed out and references are provided. Examples with real data illustrate the difference between the noncentral chi-square distribution and the normal distribution. Monte Carlo results compare the strength of the normal distribution against that of the noncentral chi-square distribution. The implication to data analysis is discussed whenever relevant. The development is built upon the concepts of basic calculous, linear algebra, and introductory probability and statistics. The aim is to provide the least technical material for quantitative graduate students in social science to understand the condition and limitation of the noncentral chi-square distribution.

Model evaluation is one of the most important aspects of structural equation modeling (SEM). Many model fit indices have been developed. It is not an exaggeration to say that nearly every publication using the SEM methodology has reported at least one fit index. Most fit indices are defined through test statistics. Studies and interpretation of fit indices commonly assume that the test statistics follow either a central chi-square distribution or a noncentral chi-square distribution. Because few statistics in practice follow a chi-square distribution, we study properties of the commonly used fit indices when dropping the chi-square distribution assumptions. The study identifies two sensible statistics for evaluating fit indices involving degrees of freedom. We also propose linearly approximating the distribution of a fit index/statistic by a known distribution or the distribution of the same fit index/statistic under a set of different conditions. The conditions include the sample size, the distribution of the data as well as the base-statistic. Results indicate that, for commonly used fit indices evaluated at sensible statistics, both the slope and the intercept in the linear relationship change substantially when conditions change. A fit index that changes the least might be due to an artificial factor. Thus, the value of a fit index is not just a measure of model fit but also of other uncontrollable factors. A discussion with conclusions is given on how to properly use fit indices.

Mediation analysis investigates how certain variables mediate the effect of predictors on outcome variables. Existing studies of mediation models have been limited to normal theory maximum likelihood (ML) or least squares with normally distributed data. Because real data in the social and behavioral sciences are seldom normally distributed and often contain outliers, classical methods can result in biased and inefficient estimates, which lead to inaccurate or unreliable test of the meditated effect. The authors propose two approaches for better mediation analysis. One is to identify cases that strongly affect test results of mediation using local influence methods and robust methods. The other is to use robust methods for parameter estimation, and then test the mediated effect based on the robust estimates. Analytic details of both local influence and robust methods particular for mediation models were provided and one real data example was given. We first used local influence and robust methods to identify influential cases. Then, for the original data and the data with the identified influential cases removed, the mediated effect was tested using two estimation methods: normal theory ML and the robust method, crossing two tests of mediation: the Sobel (1982) test using information-based standard error (z I ) and sandwich-type standard error (z SW ). Results show that local influence and robust methods rank the influence of cases similarly, while the robust method is more objective. The widely used z I statistic is inflated when the distribution is heavy-tailed. Compared to normal theory ML, the robust method provides estimates with smaller standard errors and more reliable test.

Model evaluation in covariance structure analysis is critical before the results can be trusted. Due to finite sample sizes and unknown distributions of real data, existing conclusions regarding a particular statistic may not be applicable in practice. The bootstrap procedure automatically takes care of the unknown distribution and, for a given sample size, also provides more accurate results than those based on standard asymptotics. But the procedure needs a matrix to play the role of the population covariance matrix. The closer the matrix is to the true population covariance matrix, the more valid the bootstrap inference is. The current paper proposes a class of covariance matrices by combining theory and data. Thus, a proper matrix from this class is closer to the true population covariance matrix than those constructed by any existing methods. Each of the covariance matrices is easy to generate and also satisfies several desired properties. An example with nine cognitive variables and a confirmatory factor model illustrates the details for creating population covariance matrices with different misspecifications. When evaluating the substantive model, bootstrap or simulation procedures based on these matrices will lead to more accurate conclusion than that based on artificial covariance matrices.

In the structural equation modeling literature, the normal-distribution-based maximum likelihood (ML) method is most widely used, partly because the resulting estimator is claimed to be asymptotically unbiased and most efficient. However, this may not hold when data deviate from normal distribution. Outlying cases or nonnormally distributed data, in practice, can make the ML estimator (MLE) biased and inefficient. In addition to ML, robust methods have also been developed, which are designed to minimize the effects of outlying cases. But the properties of robust estimates and their standard errors (SEs) have never been systematically studied. This article studies two robust methods and compares them against the ML method with respect to bias and efficiency using a confirmatory factor model. Simulation results show that robust methods lead to results comparable with ML when data are normally distributed. When data have heavy tails or outlying cases, robust methods lead to less biased and more efficient estimators than MLEs. A formula to obtain consistent SEs for one of the robust methods is also developed. The formula-based SEs for both robust estimators match the empirical SEs very well with medium-size samples. A sample of the Cross Racial Identity Scale with a 6-factor model is used for illustration. Results also confirm conclusions of the simulation study.

The normal distribution based likelihood ratio (LR) statistic is widely used in structural equation modeling. Under a sequence of local alternative hypotheses, this statistic has been shown to asymptotically follow a noncentral chi-square distribution. In practice, the population mean vector and covariance matrix as well as the model and sample size are always fixed. It is hard to justify the validity of the noncentral chi-square distribution for the resulting LR statistic even when data are normally distributed and sample size is large. By extending results in the literature, this paper develops normal distributions to describe the behavior of the LR statistic for mean and covariance structure analysis. A sequence of local alternative hypotheses is not necessary for the proposed distributions to be asymptotically valid. When the effect size is medium and above or when the model is not trivially misspecified, empirical results indicate that a refined normal distribution describes the behavior of the LR statistic better than the commonly used noncentral chi-square distribution, as measured by the Kolmogorov–Smirnov distance. Quantile–quantile plots are also provided to better understand the different distributions.

Data in social and behavioral sciences are often hierarchically organized. Multilevel statistical methodology was developed to analyze such data. Most of the procedures for analyzing multilevel data are derived from maximum likelihood based on the normal distribution assumption. Standard errors for parameter estimates in these procedures are obtained from the corresponding information matrix. Because practical data typically contain heterogeneous marginal skewnesses and kurtoses, this paper studies how nonnormally distributed data affect the standard errors of parameter estimates in a two-level structural equation model. Specifically, we study how skewness and kurtosis in one level affect standard errors of parameter estimates within its level and outside its level. We also show that, parallel to asymptotic robustness theory in conventional factor analysis, conditions exist for asymptotic robustness of standard errors in a multilevel factor analysis model.

This paper is concerned with removing the influence of non-normality in the classical t-statistic for contrasting means. Using higher-order expansion to quantify the effect of non-normality, four corrected statistics are provided. Two aim to correct the mean bias and two to correct the overall distribution. The classical t-statistic is also robust against non-normality when the observed variables satisfy certain structures. A special case is when the marginal distributions of the contrast are independent and identically distributed.

This article provides three approximate solutions to the multivariate Behrens–Fisher problem: the F statistic, the Bartlett, as well as the modified Bartlett corrected statistics. Empirical results indicate that the F statistic outperforms the other two and five existing procedures. The modified Bartlett corrected statistic is also very competitive.

Mardia's measure of multivariate kurtosis has been implemented in many statistical packages commonly used by social scientists. It provides important information on whether a commonly used multivariate procedure is appropriate for inference. Many statistical packages also have options for missing data. However, there is no procedure for applying Mardia's kurtosis to a data set with missing values. In this article Mardia's kurtosis is extended to a data set with missing values. Based on the complete data counterpart, the behavior of a standardized version of the extended sample kurtosis can be described by the standard normal distribution. Analytical and Monte-Carlo studies imply that the proposed distribution description is as good as its complete data counterpart when missing variables are either missing completely at random or missing at random when observed marginals do not sit in a cluster with a restricted range. Application of this procedure is illustrated with a real data set.

Unidimensionality is the hallmark psychometric feature of a well-constructed measurement scale. However, in determining the degree to which a set of items form a unidimensional scale, aberrant item response patterns may distort our investigations. For example, aberrant response patterns may adversely impact interitem covariances which, in turn, can distort estimates of a scale's dimensionality and reliability. In this study, we investigate and compare the utility of three Mahalanobis distance (M-distance) measures in identifying and downweighting aberrant item response patterns. Our findings indicated that a residual-based M-distance measure had the best properties. Specifically, response patterns having greater residual-based M-distances were responsible for observed violations of unidimensionality. When these response patterns were properly downweighted according to this M-distance, the data fitted a one-factor model better and scale reliability increased. The procedures are illustrated using three real data sets.

Data in social and behavioral sciences are often hierarchically organized though seldom normal. They typically contain heterogeneous marginal skewnesses and kurtoses. With such data, the normal theory based likelihood ratio statistic is not reliable when evaluating a multilevel structural equation model. Statistics that are not sensitive to sampling distributions are desirable. Six statistics for evaluating a structural equation model are extended from the conventional context to the multilevel context. These statistics are asymptotically distribution free, that is, their distributions do not depend on the sampling distribution when sample size at the highest level is large enough. The performance of these statistics in practical data analysis is evaluated with a Monte Carlo study simulating conditions encountered with real data. Results indicate that each of the statistics is very insensitive to the underlying sampling distributions even with finite sample sizes. However, the six statistics perform quite differently at smaller sample sizes; some over-reject the correct model and some under-reject the correct model. Comparing the six statistics with two existing ones in the multilevel context, two of the six new statistics are recommended for model evaluation in practice.

A small proportion of outliers can distort the results based on classical procedures in covariance structure analysis. We look at the quantitative effect of outliers on estimators and test statistics based on normal theory maximum likelihood and the asymptotically distribution-free procedures. Even if a proposed structure is correct for the majority of the data in a sample, a small proportion of outliers leads to biased estimators and significant test statistics. An especially unfortunate consequence is that the power to reject a model can be made arbitrarily--but misleadingly--large by inclusion of outliers in an analysis.

In a variety of statistical problems the estimate Θn of a parameter Θ is defined as the root of a generalized estimating equation Gn(Θnγn)=0 where γn is an estimate of a nuisance parameter γ. We give sufficient conditions for the asymptotic normality of #x0398;n defined in this way and derive their asymptotic distribution. A circumstance under which the asymptotic distribution of #x0398;n will not be influenced by that of γn) is noted. As an example, we consider a covariance structure analysis in which both the population mean and the population fourth-order moment are nuisance parameters. Applications to pseudo maximum likelihood, generalized least squares with estimated weights, and M-estimation with an estimated scale parameter are discussed briefly.

Data sets in social and behavioural sciences are seldom normal. Influential cases or outliers can lead to inappropriate solutions and problematic conclusions in structural equation modelling. By giving a proper weight to each case, the influence of outliers on a robust procedure can be minimized. We propose using a robust procedure as a transformation technique, generating a new data matrix that can be analysed by a variety of multivariate methods. Mardia's multivariate skewness and kurtosis statistics are used to measure the effect of the transformation in achieving approximate normality. Since the transformation makes the data approximately normal, applying a classical normal theory based procedure to the transformed data gives more efficient parameter estimates. Three procedures for parameter evaluation and model testing are discussed. Six examples illustrate the various aspects with the robust transformation.

Within some classes of nonnormal distributions, we study the asymptotic distribution of the MLE in a covariance structure model based on an incorrect assumption of normality. The asymptotic covariance matrix of the MLE has a similar form as found when the sampling distribution is elliptical, though the true sampling distribution can have arbitrary marginal skewnesses and kurtoses. The asymptotic covariance of some subset of the parameter estimators can be obtained by rescaling its normal theory counterpart. Specific models are considered as examples.

Structural equation modeling is a well-known technique for studying relationships among multivariate data. In practice, high dimensional nonnormal data with small to medium sample sizes are very common, and large sample theory, on which almost all modeling statistics are based, cannot be invoked for model evaluation with test statistics. The most natural method for nonnormal data, the asymptotically distribution free procedure, is not defined when the sample size is less than the number of nonduplicated elements in the sample covariance. Since normal theory maximum likelihood estimation remains defined for intermediate to small sample size, it may be invoked but with the probable consequence of distorted performance in model evaluation. This article studies the small sample behavior of several test statistics that are based on maximum likelihood estimator, but are designed to perform better with nonnormal data. We aim to identify statistics that work reasonably well for a range of small sample sizes and distribution conditions. Monte Carlo results indicate that Yuan and Bentler's recently proposed F-statistic performs satisfactorily.

In a variety of statistical problems one needs to manipulate a sequence of stochastic functions involving some unknown parameters. The asymptotic behavior of the estimated parameters often depends on the asymptotic properties of such functions. Especially, the consistency of the estimated parameters follows from the uniform convergence of the sequence of stochastic functions. A theorem on uniform convergence of a sequence of vector valued random functions is presented. The forms of these functions are very general and the assumptions are rather natural. If the sequence of random functions is generated by a sequence of random vectors, these random vectors are only required to be independently distributed and can be of different dimensions. As applications, we consider the consistency of the estimated regression parameters in logistic regression and in M-estimation in a linear model.

The most widely used multivariate statistical models in the social and behavioral sciences involve linear structural relations among observed and latent variables. In practice, these variables are generally nonnormally distributed; hence classical multivariate analysis, based on multinomial error-free variables having no simultaneous interrelations, is not adequate to deal with such data. A promising alternative, based on asymptotically distribution-free (ADF) covariance structure analysis, has been found to be virtually useless in practical model evaluation at finite sample sizes with nonnormal data. We take a new look at the basic statistical theory of structural models under arbitrary distributions, using the methodology of nonlinear regression and generalized least squares estimation. For example, we adopt the use of residual weight matrices from regression theory. We develop a series of estimators and tests based on arbitrary distribution theory. We obtain a type of probabilistic Bartlett co...

A test for linear trend among a set of eigenvalues of k covariance matrices is developed. A special case of this test is Flury's (1986) test for the equality of eigenvalues. The linear trend hypothesis appears to be more relevant to data analysis than the equality hypothesis. Examples show how the linear trend hypothesis can be acceptable while the equality hypothesis is rejected.

Many test statistics are asymptotically equivalent to quadratic forms of normal variables, which are further equivalent to T = sigma(d)(i=1) lambda(i)z(i)(2) with z(i) being independent and following N(0,1). Two approximations to the distribution of T have been implemented in popular software and are widely used in evaluating various models. It is important to know how accurate these approximations are when compared to each other and to the exact distribution of T. The paper systematically studies the quality of the two approximations and examines the effect of the lambda(i) and the degrees of freedom d by analysis and Monte Carlo. The results imply that the adjusted distribution for T can be as good as knowing its exact distribution. When the coefficient of variation of the lambda(i) is small, the rescaled statistic T(R) = dT/(sigma(d)(i=1) lambda(i)) is also adequate for practical model inference. But comparing T(R) against chi2(d) will inflate type I errors when substantial differences exist among the lambda(i), especially, when d is also large.

Commonly used formulae for standard error (SE) estimates in covariance structure analysis are derived under the assumption of a correctly specified model. In practice, a model is at best only an approximation to the real world. It is important to know whether the estimates of SEs as provided by standard software are consistent when a model is misspecified, and to understand why if not. Bootstrap procedures provide nonparametric estimates of SEs that automatically account for distribution violation. It is also necessary to know whether bootstrap estimates of SEs are consistent. This paper studies the relationship between the bootstrap estimates of SEs and those based on asymptotics. Examples are used to illustrate various versions of asymptotic variance-covariance matrices and their validity. Conditions for the consistency of the bootstrap estimates of SEs are identified and discussed. Numerical examples are provided to illustrate the relationship of different estimates of SEs and covariance matrices.

We study several aspects of bootstrap inference for covariance structure models based on three test statistics, including Type I error, power and sample-size determination. Specifically, we discuss conditions for a test statistic to achieve a more accurate level of Type I error, both in theory and in practice. Details on power analysis and sample-size determination are given. For data sets with heavy tails, we propose applying a bootstrap methodology to a transformed sample by a downweighting procedure. One of the key conditions for safe bootstrap inference is generally satisfied by the transformed sample but may not be satisfied by the original sample with heavy tails. Several data sets illustrate that, by combining downweighting and bootstrapping, a researcher may find a nearly optimal procedure for evaluating various aspects of covariance structure models. A rule for handling non-convergence problems in bootstrap replications is proposed.

In the social and behavioural sciences, structural equation modelling has been widely used to test a substantive theory or causal relationship among latent constructs. Cross-validation (CV) is a valuable tool for selecting the best model among competing structural models. Influential cases or outliers are often present in practical data. Therefore, even the correct model for the majority of the data may not cross-validate well. This paper discusses various drawbacks of CV based on sample covariance matrices, and develops a procedure for using robust covariance matrices in the model calibration and validation stages. Examples illustrate that the CV index based on sample covariance matrices is very sensitive to influential cases, and even a single outlier can cause the CV index to support a wrong model. The CV index based on robust covariance matrices is much less sensitive to influential cases and thus leads to a more valid conclusion about the practical value of a model structure.

Problems such as improper solution, non-convergence, subsets of variables having different distribution, and latent variables with single indicators are common in the practice of structural equation modelling. In such cases, it may be feasible to fix some model parameters at prespecified values while concentrating on estimating some other parameters. This paper formulates such a model fitting process through a model segregation approach. The statistical properties of this procedure are studied using the theory of estimating equations and optimal estimating functions. The dependency of the new parameter estimates on those of the prespecified parameter estimates is characterized for several commonly used estimating equations. A rescaled model fit statistic is proposed. Examples illustrate various applications of this procedure.

Even though data sets in psychology are seldom normal, the statistics used to evaluate covariance structure models are typically based on the assumption of multivariate normality. Consequently, many conclusions based on normal theory methods are suspect. In this paper, we develop test statistics that can be correctly applied to the normal theory maximum likelihood estimator. We propose three new asymptotically distribution-free (ADF) test statistics that technically must yield improved behaviour in samples of realistic size, and use Monte Carlo methods to study their actual finite sample behaviour. Results indicate that there exists an ADF test statistic that also performs quite well in finite sample situations. Our analysis shows that various forms of ADF test statistics are sensitive to model degrees of freedom rather than to model complexity. A new index is proposed for evaluating whether a rescaled statistic will be robust. Recommendations are given regarding the application of each test statistic.

Covariance structure analysis is used to evaluate hypothesized influences among unmeasured latent and observed variables. As implemented, it is not robust to outliers and bad data. Several robust methods in model fitting and testing are proposed. These include direct estimation of M-estimators of structured parameters and a two-stage procedure based on robust M- and S-estimators of population covariances. The large sample properties of these estimators are obtained. The equivalence between a direct M-estimator and a two-stage estimator based on an M-estimator of population covariance is established when sampling from an elliptical distribution. Two test statistics are presented in judging the adequacy of a hypothesized model; both are asymptotically distribution free if using distribution free weight matrices. So these test statistics possess both finite sample and large sample robustness. The two-stage procedures can be easily adapted into standard software packages by modifying existing asymptotically distribution free procedures. To demonstrate the two-stage procedure, S-estimator and M-estimators under different weight functions are calculated for some real data sets.

Principal component analysis and factor analysis are the most widely used tools for dimension reduction in data analysis. Both methods require some good criterion to judge the number of dimensions to be kept. The classical method focuses on testing the equality of eigenvalues. As real data hardly have this property, practitioners turn to some ad hoc criterion in judging the dimensionality of their data. One such popular method, the ‘scree test’ or ‘scree plot’ as described in many texts and statistical programs, is based on the trend in eigenvalues of sample covariance (correlation) matrix. The principal components or common factors corresponding to eigenvalues which exhibit a slow linear decrease arc discarded in further data analysis. This paper develops a formal statistical test for the ‘scree plot’. A special case of this test is the classical test for equality of eigenvalues which has been suggested in several texts as the criterion to decide the number of principal components to retain. Comparisons between equality of eigenvalues and the slow linear decrease in eigenvalues on some classical examples support the hypothesis of slow linear decrease. A physical background to such a phenomenon is also suggested.