Search

Missing data is a common problem in virtually all surveys. This study focuses on variance estimation and its consequences for analysis of survey data from the National Center for Education Statistics (NCES). Methods suggested by C. Sarndal (1992), S. Kaufman (1996), and S. Shao and R. Sitter (1996) are reviewed in detail. In section 3, the bootstrap method of Shao and Sitter is applied to the Schools and Staffing Survey (SASS) 1993-94 Public School Teacher Survey component to assess the magnitude of imputation variance. This method is appealing, but requires repeated imputations, so for large scale surveys, the data files become too large. The empirical study shows, however, that using the hot deck imputation method in the 1993-94 SASS can affect the standard error seriously. However, the majority of items have very low stage 2 (hot deck) imputation rates. When the imputation rate is low, the inflation in variance is not severe. It appears feasible for NCES to compute imputation rates and document the problem with the next user's manual. (Contains 8 tables and 11 references.) (SLD)

Much of the literature on random-digit-dial (RDD) response rates focuses on the benefits and drawbacks of various screening and sampling procedures. The common assumption is that enumeration is more invasive, leading to lower response rates, but there are concerns about population coverage and self-selection with other methods. When the target population is found in a subset of households, a screen-out question may be used to eliminate ineligible households prior to employing sampling methods. Using approximately 10,000 telephone numbers from the National Household Education Survey (NHES) sampling frame, this experiment examined the impact on screening response rates of: full enumeration of all households (no screen-out) versus a screen-out question, and mailing an advance letter. An RDD sample was divided into four quarter samples: (1) screen-out and letter; (2) screen-out and no letter; (3) no screen-out and letter; and (4) no screen-out and no letter. Response rates were significantly higher in the screen-out condition. The advance mailing increased cooperation in the no screen-out condition, but not in the screen-out condition. The no screen-out condition consumed substantially more resources than a screen-out sample of the same size, but also provided more data for each completed case. Implications of this experiment for the design of future surveys are discussed. Two appendixes present the NHES field test letter and the advance letter. (Contains 7 tables, 1 exhibit, and 24 references.) (Author/SLD)

The National Household Education Survey (NHES) is a telephone survey of the noninstitutionalized civilian population of the United States that collects data on educational issues that are best explored through contact with households rather than with institutions. The NHES has been conducted in 1991, 1993, 1995, and 1996. In the 1996 NHES (NHES:96), the topical components were parent/family involvement in education and civic involvement. This working paper presents information on the unit response rates, item response rates, weighting procedures, and imputation procedures in the NHES:96. The description of the response and completion rates in the first section, "Unit Response Rates," includes data on the rates for the Screener interview, the Parent/Family Involvement/Civic Involvement interview, the Youth Civic Involvement interview, and the Adult Civic Involvement interview. The next section, "Item Response and Imputation Procedures for the NHES:96," discusses reasons for nonresponse and ways to handle missing data. Imputation of missing data was done to develop complete variables used in developing sampling weights and to enable users to compute estimates more easily. The final section, "Weighting and Standard Error Calculation Procedures for the NHES:96," describes the procedures used to produce the weights for use in estimating characteristics from the NHES:96 sample and for estimating the sampling errors of those estimates. (Contains 39 tables and 7 references.) (SLD)

The National Household Education Survey (NHES) is a data collection system of the National Center for Education Statistics that is designed to provide information on educational issues that can best be studied through contacting households rather than educational institutions. This report compares the information collected through telephone interviews by the NHES on adult education in 1991 and 1995 with that collected through the Current Population Survey (CPS) of the Bureau of the Census at 3-year intervals between 1969 and 1984 and in 1992. Included in the study were examinations of population coverage and the selection of adults for interviews, survey nonresponse, the use of proxy reports of adult education participation, and survey context. In the NHES in 1991, 60,314 households were screened, and the total number of adult interviews completed was 12,568, a figure that included 9,774 adults who participated in adult education within the preceding 12 months. In 1992, approximately 57,000 households were interviewed for the CPS, a monthly survey of households. Estimates of adult education participation provided by the CPS are substantially lower than those from the NHES. The 1984 participation estimate from the CPS was 14% and the 1992 estimate was 21%. Differences in CPS and NHES estimates range from 13 to 20%. Examination of methodological, operational, and definitional issues that might account for the differences suggest that NHES figures are more accurate, although even these figures may underestimate participation. Reasons for the discrepancies are discussed. An appendix presents adult education participation items from the surveys. (Contains 9 tables and 37 references.) (SLD)

The National Household Education Survey (NHES) is a data collection system of the National Center for Education Statistics (NCES) designed to address a wide range of education-related issues. It is a telephone survey of the noninstitutionalized civilian population. This working paper presents information on the potential for undercoverage bias in estimates from the 1995 NHES. Estimates from the 1995 NHES are subject to bias because only households with telephones were sampled. Data from the 1992 October supplement to the Current Population Survey of the U.S. Census are used to evaluate the potential size of the bias of the estimates. The focus in the report is on the statistics for two separate populations: 0- to 2-year-olds who were sampled as part of the Early Childhood Program Participation (ECPP) component and civilian adults who were sampled for the Adult Education (AE) component. The analysis of undercoverage bias shows that the coverage biases for estimates of adult characteristics are not very large, but biases for 0- to 2-year-olds are somewhat larger, yet still relatively small. The coverage bias is small even though some of the differences in characteristics for persons living in telephone and nontelephone households are large. The undercoverage bias for subgroups in the NHES may be more of a problem. In this study, the undercoverage biases for characteristics of Black and Hispanic American 0- to 2-year-olds were generally larger than for all children. For adults, these undercoverage biases were not as large as for children. Overall, however, results support the use of the telephone survey methodology. An appendix presents three tables of study data. (Contains nine references.) (SLD)

The National Household Education Survey (NHES) is a data collection system of the National Center for Education Statistics (NCES) designed to address a wide range of education-related issues. It is a telephone survey of the noninstitutionalized civilian population. In 1995, the NHES included a survey of adult education (AE). This report examines errors arising from the interviewing process in the AE components of the NHES:95. The estimates from this component, and from every survey, are subject to both sampling error and nonsampling error. In this analysis, measurement errors are estimated by reinterviewing a sample of respondents and asking them a subset of the same questions included in the original interview. Statistic produced by comparing the original and reinterviews estimate the consistency of reporting. In the NHES:95 reinterviews, discrepancies between some of the original interview questions and the reinterview questions were reconciled. Reinterviews were completed with 1,109 cases. For most survey questions, the reporting was consistent, and only a few modifications need to be considered for future surveys. However, for questions associated with barriers to AE participation, more problems were identified. The greatest discrepancies were for questions about work-related activities and personal development courses. The reconciliation of the responses to the original interview and reinterview did not give much information about response bias for the questions, but the results did generally support the use of the reinterview as an independent replication of the original interview. (Contains 9 tables, 1 exhibit, and 10 references.) (SLD)

The 1991 Survey of Recent College Graduates (RCG:91) is the sixth study in a series begun in 1976. The series provides data on the occupational and educational outcomes of recent bachelor's and master's graduates one year after graduation. The survey was conducted by Westat, Inc. in a two-stage sample involving 400 institutions of higher education and 18,000 graduates contacted by telephone. Along with estimates, reports on the RCG typically include standard errors of the estimates, indicating the nature and size of sampling error. Errors due to nonsampling error are often not included in estimated standard errors, but this report examines nonsampling errors and their impact on the estimates from the RCG:91. The major sources of nonsampling errors are nonresponse, random measurement errors, and systematic errors due to interviewers. Each source is discussed, and ways to estimate the potential consequences of nonsampling errors are explored. Nine figures, 19 tables, and 3 exhibits present statistical information. Eight appendixes contain supplemental and detailed information about the conduct of the survey. (SLD)

In the spring of 1991 the first full-scale National Household Education Survey (NHES:91) was conducted for the National Center for Education Statistics (NCES). The NHES:91 was a national random digit dial telephone survey of about 14,000 parents of 3- to 8-year-old children concerning the educational experiences of young children. A reinterview program was included to examine the impact of measurement errors on estimates of the characteristics of early educational experience. For the whole subject population, there were between under 10 days and 41 days between completion of the original interview and the reinterview. The methodology and results of this computer-assisted reinterview program are discussed, comparing responses from the original and reconciled interviews. A sample of 604 cases was selected for reinterviews. Reinterviews were completed with 534 households (a response rate of 88 percent). Reinterview results are encouraging in that most items included had small to moderate measurement errors. For the specific items with high measurement errors, potential associated problems included vague or ambiguous classifications and lack of parental knowledge about the item. One weakness was the limited scope and size of the reinterview program. Four tables present analysis data. (Contains 3 references.) (SLD)

The National Household Education Survey (NHES) was conducted for the first time in 1991 as a way to collect data on the early childhood education experiences of young children and participation in adult education. Because the NHES methodology is relatively new, field tests were necessary. A large field test of approximately 15,000 households was conducted during the fall of 1989 to examine several methodological issues. This report examines a technique that was used to increase the coverage of 14- to 21-year olds and to capture more dropouts in the sample. The effectiveness of the multiplicity sample in achieving these goals is described. The multiplicity sample results in more older youths and status dropouts, but does little to add to the sample size of younger youths and the event dropouts. The multiplicity sample is effective in reducing undercoverage bias for some statistics, but not others. Ways to improve estimation are discussed. Since the cost of screening households to find those with 14- to 21-year-old members (less than 1 in 4 households) is large relative to the cost of conducting the Household Respondent Interview, it is recommended that the multiplicity sampling approach be implemented in any future survey on dropouts. One table and six figures supplement the discussion. An appendix contains five detailed tables of field test findings. (SLD)

The National Household Education Survey (NHES) was conducted for the first time in 1991 as a way to collect data on the early childhood education experiences of young children and participation in adult education. Because the NHES methodology is relatively new, field tests were necessary. A large field test of approximately 15,000 households was conducted during the fall of 1989 to examine several methodological issues. This report describes the design of the field test and the outcomes of its collection activities. Unit and item response rates and the burden associated with survey participation are described. The telephone methodology of the NHES survey offers some important advantages for the program; it is less expensive than a comparable personal interview survey, and it can also be instituted relatively quickly, address several subject matter areas at the same time, and provide results in a very timely fashion. The NHES offers the potential for examining education issues that have been difficult to address through the more traditional National Center for Education Statistics surveys that are based on institutional samples. The field test has shown that many of the concerns can be handled adequately. Fifteen tables and six figures illustrate the overview. (SLD)

The National Postsecondary Student Aid Study collected data for sample students from registrar and financial aid offices at postsecondary institutions. The data were collected for students enrolled in a postsecondary institution in fall 1986. Since substantial numbers of students are not enrolled in fall and are therefore not represented in the sample, the financial aid awards were weighted to produce estimates for students enrolled any time during the academic year. The methodology used in the production of adjusted estimation weights for a subset of the sample and the results are described in this document, along with several methodological findings. Estimated federal aid recipients enrolled in fall accounted for only about 77% of recipients during the 1986-87 academic year. Fall recipients accounted for only 75% of recipients of Pell Grants and Guaranteed Student Loans (GSLs). Fall recipients were a much higher percentage of recipients for the other three campus-based programs--Supplemental Educational Opportunity Grant, Perkins Loan, and College Work-Study Program. The percent of recipients enrolled in fall varied by institution type. Estimated mean award amounts for fall and academic-year federal aid recipients were more comparable than number of recipients. Individual program data counts were also compared. (MSE)