Search

This review presents two case studies that illustrate how multivariate statistical modeling applies to the specific goal of improving educational assessment. The first case study involves the development of a new large-scale English language proficiency assessment system (called the English Language Proficiency Assessment for the 21st Century; ELPA21). The second application concerns efforts to quantify student progress in learning using conditional growth models, a topic of current debate about assessment policy. A popular measure, Student Growth Percentile (SGP), is explored through the lens of multivariate statistical analysis. It is concluded that collaboration between researchers and practice stakeholders can improve assessments that benefit student learning.

Mobilization is itself a form of political participation, chosen with attention to the effects that an intervention in politics by significant numbers of others might be expected to produce, the costs that will need to be borne in order to produce it, and the effectiveness and cost of other kinds of investments in political influence. This essay focuses on the political participation of political mobilizers, the actors behind the actors in mass politics. Considering matters from the perspective of the elites who might solicit citizen involvement, I review some of the major ways that politics in the United States has changed over the last two decades, their implications for the cost-effectiveness of mobilization as a strategy in politics, and the evidence – such as exists – for their effects on mass participation in American elections and government. The regularization of two-party competitive politics, the polarization of political elites and the electorate, and the shifting regulation of campaigns and elections, I argue, have affected the opportunities, incentives, and resources for mobilization as the strategy of choice for campaigns, parties, and advocates in America.

Ground delay programs (GDPs) are frequently used to keep U.S. air transportation safe and efficient. Most research on GDPs has focused on optimal design and implementation, with little attention given to retrospective performance evaluation. This research fills that gap by identifying GDP performance criteria, developing associated performance metrics, and evaluating GDP performance metrics across airports and over time. GDP performance criteria are established, and associated performance metrics are specified for five performance goals: capacity utilization, efficiency, predictability, equity, and flexibility. By defining multiple performance metrics, this research enables FAA traffic managers and flight operators to review GDP performance after the fact in a comprehensive way and to uncover GDP performance trends across airports and over time. Through use of data from the FAA Aggregate Demand List and the FAA Aviation System Performance Metrics, historical GDP performance is assessed for San Francisco I...

Traditional estimators of item response theory scale scores ignore uncertainty carried over from the item calibration process, which can lead to incorrect estimates of the standard errors of measurement (SEMs). Here, the authors review a variety of approaches that have been applied to this problem and compare them on the basis of their statistical methods and goals. They then elaborate on the particular flexibility and usefulness of a multiple imputation–based approach, which can be easily applied to tests with mixed item types and multiple underlying dimensions. This proposed method obtains corrected estimates of individual scale scores, as well as their SEMs. Furthermore, this approach enables a more complete characterization of the impact of parameter uncertainty by generating confidence envelopes (intervals) for item trace lines, test information functions, conditional SEM curves, and the marginal reliability coefficient. The multiple imputation–based approach is illustrated through the analysis of an artificial data set, then applied to data from a large educational assessment. A simulation study was also conducted to examine the relative contribution of item parameter uncertainty to the variability in score estimates under various conditions. The authors found that the impact of item parameter uncertainty is generally quite small, though there are some conditions under which the uncertainty carried over from item calibration contributes substantially to variability in the scores. This may be the case when the calibration sample is small relative to the number of item parameters to be estimated or when the item response theory model fit to the data is multidimensional.

Many runway capacity estimation models are available today, and developers usually claim that their models have been validated. However, information about the validation process is often limited, and different models are validated at differing levels of complexity. This paper proposes two validation methodologies for use in testing model predictions against reality. It demonstrates the two methodologies on two models [the airfield capacity model (ACM) and the runway simulator (rS)] and two airports (San Francisco International and Los Angeles International in California). The results indicate that when arrivals and departures are considered separately, both ACM and rS tend to overpredict capacities under good visibility conditions and to predict larger ranges of capacity values than are seen empirically. However, when considering total operations (arrivals and departures together), the results show that both models failed to provide good estimates of total throughput at both airports. Overall, arrival and departure capacity estimates from rS typically reflect empirical capacities better than those from ACM.

Flight cancellations are costly to airlines and air passengers, but the question is, how costly? Little quantitative analysis about cancellation cost has been done, and current studies are focused on flight delays. However, cancellations should be studied to supplement delay metrics to assess the performance of the National Airspace System, as well as to evaluate the effectiveness of air traffic management. Postoperation data of the ground delay program were explored, and relative value of cancellations in terms of delay savings were inferred. Two cancellation metrics were developed for this purpose. In addition, flight cancellation decisions were modeled with a discrete choice model by using the two cancellation metrics that were generated as well as some other flight characteristics. Three models are presented in this study. The results reveal the value of a flight cancellation and airline preference structure in making decisions in canceling a flight in response to schedule disturbance. The study contributes to the literature in a distinct way: post-operation data are used to understand airline flight cancellation behavior in the context of ground delay programs.

There has been a unique convergence of factors that have contributed to Japan's outstanding public transportation system. The paper focuses on two modes of transportation: air and high-speed rail (HSR). These two modes do not complement each other as in Europe, but compete head-on in Japan. Moreover, Japan does not have to contend with market-distorting public subsidies, because both companies operating HSR in the corridor of interest are highly profitable, primarily because of their HSR service. The Japanese transportation system therefore provides an excellent environment in which to study the nature of competition between air and HSR. (Conventional rail is also discussed where appropriate.) Individual contributors to terminal pair choice are contrasted between air and HSR, and the trade-offs between accessibility, frequency, and speed are analyzed in detail.

A strategy for airline recovery from schedule perturbation caused by adverse weather or other temporary events is proposed. Capacity reduction at a hub airport caused by these temporary events is reflected as reduced slots (or later controlled time of arrivals) for airlines from ground delay programs. It is suggested that under severe circumstances legacy airlines with hub-and-spoke networks implement ground transportation into their network to substitute for flights. This strategy is called real-time intermodal substitution (RTIMS). RTIMS is different from air-rail cooperation practiced in Europe in that it is triggered only by a severe demand and supply imbalance at major hub airports and it performs operational integration of airside and ground transportation. Mathematical programming is presented to help airlines make decisions about if and how to delay, canceling flights, or substituting flights with buses. An approximation algorithm is proposed to obtain solutions, avoiding substantial computation time required to solve large-scale nonlinear integer programming. As an evaluation, experimental data are used to compare scenarios with and without tactical intermodal substitution.

Recent studies of the single-airport ground-holding problem use static or dynamic optimization to manage uncertainty about future airport capacities. Scenario trees of airport capacity profiles provide the basis for formulating multistage recourse problems. In this paper, methodologies are presented for generating scenario trees from empirical data, and the performance of scenario-based models is examined with the scenario trees. Most U.S. airports have capacity profiles that can be classified into some nominal scenarios, and for many airports, these scenarios can be naturally combined into scenario trees. The delay costs yielded from using dynamic optimization are consistently and considerably lower than from static optimization. The results illustrate the benefit of the wait-and-see strategy in a real-world setting and suggest the need for further research on implementing scenario-based dynamic strategies.

An econometric model of average daily delay is formulated and estimated to analyze flight delay in the U.S. domestic system. The model considers the effects of arrival queuing, volume, terminal weather, en route weather, seasonal effects, and secular effects. In particular, the time-of-day effects of arrival queuing, the effects of scheduled arrivals, and the interaction between scheduled arrivals and weather conditions are investigated. The estimation results suggest that ( a) queuing has a greater delay impact in the morning than in the afternoon or evening (one unit of morning queuing delay causes about three times as much average daily delay as one unit of evening queuing delay), ( b) scheduled arrivals–-both alone and in interaction with weather conditions–-significantly affect average delay, and ( c) 31% of the total delay increase between early 2004 and early 2005 can be attributed to traffic growth.

The objective of this paper is to present the findings and a set of hypotheses that emerged in the initial steps of an assessment study of air traffic controllers’ adoption and adaptation of new technologies, with the user request evaluation tool (URET) as a case study. The hypotheses are the basis for further study on this topic. The importance of this investigation is to provide a better understanding of changes brought about by the use of decision support tools by sector controller teams. The main purpose of URET is to support sector team strategic planning and allow controllers to concentrate on more user-beneficial control actions. Actual improvements depend on the way controllers use the automation tools in their work. Three aspects were noted about URET usage: ( a) different sector teams use it in different ways, ( b) in many instances URET usage differs from what was intended, and ( c) usage varies across centers. Subject matter experts were interviewed to explore how controllers have adopted and adapted URET across the mentioned variations. The goal of this research is to draw lessons from the experience with URET that can inform technology deployments in the future.

FAA regulates the safety of the aviation industry through the safety oversight system, which is a system of rulemaking, standard-setting, certification, accident investigation, rule enforcement, and surveillance activities. Federal programs, including those of FAA, use performance indicators to measure the achievement of program goals. As part of a broader program of developing risk management methodologies, FAA is researching performance indicators that can be used to measure the performance of the safety oversight system. One of its goals is to create performance indicators that can describe the safety oversight system's influence on safety outcomes such as fatalities. Creating performance measures that link activities to safety outcomes is challenging because it is difficult to establish the causation between oversight activities and these safety outcomes. This challenge is not unique to FAA, and external reviewers such as the Government Accountability Office have recommended that other high-reliability sectors, including rail, develop such indicators. In addition to safety outcomes, other aspects of safety oversight system performance can be described with meaningful metrics. The background and motivation for oversight evaluation in the aviation industry and in general are discussed, as well as the challenges, some generic and some unique, of evaluating aviation safety oversight activities. Research is also presented on how safety oversight evaluation is conducted outside aviation.

The current demand management policy at LaGuardia Airport (LGA) New York, must be changed in 2007 under the provision of the Wendell H. Ford Aviation Investment and Reform Act for the 21st Century of April 2000 (AIR-21). As a preliminary step for developing a new policy, this study considered how past policies, along with other factors, have affected operational performance at LGA. The interaction between LGA and the rest of the aviation system was also investigated by estimating simultaneous equations of average delay for LGA and the National Airspace System (NAS) by using two-stage least squares. The results demonstrate that the arrival delay impact of AIR-21 on LGA was in the form of Increased Ground Delay Program holding and that although delay increased markedly under AIR-21, there were also observable improvements in the ability of LGA to handle traffic. Furthermore, on the basis of the simultaneous equations analysis, it was found that 1 min of arrival delay at LGA causes about 2 min of delay elsewhere in the NAS, suggesting that demand management at LGA is a national rather than a local issue.

The recent increase in flight delays in the U.S. domestic system is analyzed by estimating an econometric model of average daily delay that incorporates the effects of arrival queuing, convective weather, terminal weather conditions, seasonal effects, and secular effects (trends in delays not accounted for by other variables). From the estimation results it was possible to quantify some sources of higher delays in late 2003 and early 2004 and track changes in delays that are not attributable to major causal factors. Results suggest that when these factors are controlled for, delays decreased steadily from 2000 through early 2003, but that the trend reversed thereafter. Of the total delay increase between early 2003 and early 2004, half to two-thirds can be attributed to specific sources.

Passenger flow is an important planning factor in an air transportation system. However, forecasting segment (link) and airport flows becomes more complicated in a hub-and-spoke system since segment flows are the aggregations of route flows, and the number of routes increases with hubbing activities. This research develops an equilibrium model considering certain important characteristics of an air transportation system such as distance, airport delay, airline competition, and networks to predict segment and airport passenger flows from the viewpoint of the whole system. The major features of the model include ( a) treatment of segment flows and airport delays as endogenous by considering the feedback of assigned segment flows and their impacts on airports; ( b) reflecting the flexibility of air networks, a start with all links between all airports as the potential network and determination of the predicted network according to the equilibrium flows on segments; and ( c) connection of key elements of the system so that it can evaluate the system impacts of some element changes. The model is demonstrated by applying it to the National Airspace System of the United States. Several characteristics of the model are also investigated. In addition, a policy experiment shows that improvement of an airport not only affects the airport itself but also changes the flows and performance of other airports–-the model can be a tool for evaluating systemwide effects. Finally, the model's limitations and possible remedies are discussed.

The throughput effect of Traffic Management Advisor (TMA) metering for Los Angeles International Airport was estimated with censored regression. Under TMA metering, traffic management coordinators receive advisories about when individual flights should be scheduled to enter the terminal area, accounting for the unimpeded time and constraints related to separation requirements and capacity constraints. Tests of TMA metering at Los Angeles began in spring 2002; mandatory use during the morning peak period started in November 2002. This analysis focuses on observed counts during periods when metering is and is not in effect and takes into account other relevant factors such as weather, visibility, and demand. Demand is used as a censoring variable that places an upper limit on the observed count. A significant increase in achievable arrival throughput under instrument conditions was found in the morning peak period, but a significant decrease was observed under visual conditions. These results are consistent with field reports, which suggest that under visual meteorological conditions, TMA was assigning unnecessary metering delays. The success of the censored regression methodology in capturing these effects suggests that it would be useful for various other applications in which the effect of some event on airport capacity must be ascertained.

Explored are the benefits of integrating User Request Evaluation Tool (URET) and Controller-Pilot Data Link Communications (CPDLC). Controller-pilot voice-communication messages and aircraft traffic flows and conflicts are analyzed in great detail in one representative, URET-operating en route sector. On the basis of the mapped URET data and the real-world communication messages, a base case and two alternative scenarios were analyzed to estimate the number of clearances that are given to pilots to resolve aircraft conflicts a sufficient time before the start of the conflict, and to determine the reduction in frequency congestion possible if such messages were sent via data link. It was found that the highest frequency use, which corresponded to the first traffic peak, was reduced 27% after the second-scenario messages were removed from the base-case scenario. After removing the non-time-critical conflict-resolution messages, the total reduction was 59%. Frequency use during the highest number of aircraft conflicts was reduced 65% after all messages from the second and the third scenario were removed. Thus, the benefits of integrating CPDLC and URET are significant. If non-time-critical conflict messages were transmitted via data link in the integrated CPDLC and URET environment, they could considerably improve the frequency congestion. More important, the largest benefits would be experienced in situations involving a large number of aircraft conflicts, or during busy periods of traffic. These improvements could further help to reduce the number of communication errors (and the consequent air traffic control workload), as well as the number of operational errors.

Free Flight Phase 1 (FFP1) is an FAA program for improving the performance of the National Airspace System (NAS) through the deployment of advanced technologies for air traffic management. In addition to the deployment activities, FFP1 includes a significant evaluation component, which faces a significant hurdle. A plethora of factors—weather, demand, enhancements to the NAS infrastructure not related to FFP1, facility outages, and so on—may also cause changes in NAS performance. It is necessary to normalize for these factors in order to determine the effect of FFP1. Normalization procedures to isolate the impact of the implementation of an FFP1 technology—the Terminal Area Traffic Management Advisor (T-TMA)—are documented at the Southern California TRACON, where it is used for controlling traffic into Los Angeles International Airport. Two examples of normalization are presented. One examines the effect of T-TMA on airport arrival capacity, and the other looks at arrival delay. The results, although preliminary given the short time since implementation, are consistent: it appears that capacities have increased and delays decreased as a result of the deployment of the tool. Moreover, the magnitudes of the delay reductions and capacity increases are consistent.

It has been common for airport traffic forecasts to assume a continuing increase in average aircraft size in the future so that the anticipated growth in passenger demand will not result in a proportional increase in aircraft operations. However, to understand the likely future trends in average aircraft size, as well as strategies that an airport might pursue to encourage airlines to use larger equipment, it is necessary to understand the competitive dynamics of the various markets served by the airport and the factors that determine the size of equipment that airlines will use in those markets. Recent trends in average aircraft size at a major U.S. West Coast airport and the influence of market characteristics on the size of aircraft used in different markets are examined. The changes in the past decade in selected short- and medium-haul markets are analyzed, and the results are presented of a cross-sectional model that estimates the influence of various market characteristics, including air traffic delay levels, on the average size of aircraft deployed in a wide variety of markets throughout the United States. The prospects for airport capacity constraints to influence airlines to make use of larger aircraft are discussed, and the implications for airport strategies to encourage the use of larger aircraft are examined.

The manner in which changes in the performance of the U.S. National Airspace System (NAS) affect carrier costs is examined. The methodology includes estimation of airline cost functions that contain NAS performance metrics as arguments, using quarterly data for 10 U.S. domestic airlines. The primary interest is in the impact of delay on cost performance, and performance metrics that vary by airline and quarter are developed. The potentially large number of metrics are reduced by applying factor analysis to seven underlying variables, including average delay, delay variance, and the proportion of flights that are cancelled. The analysis reveals that variations in the seven variables can be adequately captured by three or fewer factors. The three factors used correspond to “delay,” “variability,” and “disruption,” the last two of which are merged into a single “irregularity” factor in the two-factor model. When used as arguments in an airline cost function, the “disruption” factor is found to be a significant contributor to airline costs in the three-factor model, as is the “irregularity” factor in the two-factor model. No significant effect is found for the delay factor in the multifactor models. These results challenge the prevailing assumption that delay reduction is the most important benefit from investments in NAS capital and operations rules. The carrier cost savings that would result from improved NAS performance levels are estimated, and these are compared with previously published estimates.

Activity scheduling models describe how people build daily activity schedules, given the opportunities and constraints they face. Many current models fail to adequately account for interdependence of these opportunities and constraints over time. This limits their ability to model aspects of behavior such as routine patterns, the desire for flexibility, and responses to urgent needs. Time dependence issues affect user responses to many transportation policies, including ridesharing incentives, flex-time, and paratransit strategies. Time dependence can be better captured by conceptualizing the activity scheduling process as a regulation and control process, the objective of which is to obtain an activity program that is both feasible and personally optimal. Activity schedules evolve over time as a result of regulatory action taken in response to the arrival of disturbances. An example is provided to illustrate the application of these concepts to the activity scheduling process. While it can incorporate useful ideas previously forwarded by researchers, control theory provides a framework for further theoretical and empirical development.

With the acceptance and widespread application of stated preference methods in transportation analysis, there has been a need for the analysis of choice data with repeated observations. Analysis of repeated choice data is complicated by correlation of responses across the choices made by a single individual. In the probit framework, repeated choice can easily be accommodated by way of correlation in the utilities associated with the errors. However, the probit framework has computational limitations; subsequently, researchers have looked to modeling repeated choice by using extensions of the logit model. Random parameter logit models estimated by simulation methods provide a promising modeling framework. The estimation results of some stated preference intercity business travel data are compared by using three alternative estimation techniques to account for correlations across the choices made by a single individual. It is believed that two of the alternative methods, which use random-parameter logit models (developed by researchers for use in an energy application), have not been used before in the transportation context. A comparison of the three techniques provides documentation for researchers and practitioners who plan to analyze such data.

Metropolitan regions with more than one major airport—multiple airport systems (MASs)—are important to the U.S. air transport system because of the large number of passengers they serve. Airport ground access factors strongly influence the allocation of traffic in MASs. The effects of improvements to airport ground access (by nonautomobile modes) on airport use in a MAS are analyzed. A case study of an extension of a Bay Area Rapid Transit rail link into the San Francisco International Airport (SFO) is presented. Two airport choice models were developed. One is a nested logit model in which the airport choice decision occurs at the higher level and the mode choice decision at the lower level, and the other is a multinomial logit model. The results indicated that improvements to SFO ground access would modestly strengthen SFO as the dominant airport in the San Francisco Bay Area and that most of the diversion of passengers would be from Oakland Airport.