Search

As part of the MULTITUDE project a survey was undertaken in 2011 regarding how practitioners used traffic simulation models. This revealed that 19% of those polled, conducted no calibration of their models, and of those that did, only 55% used guidelines during this process. To investigate this issue further a second survey was performed to identify which documents were being used most, and areas of concern, where it was felt that further/better guidance was needed. In this report we have examined these areas, their strengths and weaknesses, and have isolated five gaps, where improvements would allow better overall guidance to be produced: • Data, where a greater quality and quantity needs to be available not only for the performance of calibration and validation but also to allow a greater understanding of the variability of conditions likely to be encountered. • Standardisation and definitions in basic methodology, where greater clarity is required as to what (for example) MoPs are acceptable and more importantly, essential. • Illustration, Comparison and Evaluation, with a greater need for comparable case and meta studies. • Variability, where guidance is needed as to which (for example) parameters effect differing macroscopic observables, the so called ‘hierarchy of parameters’ which can be uncovered through greater sensitivity and uncertainty analysis. • Assisted calibration, where automated codes would aid in sensitivity analysis, batch analysis and, through reduced project ‘run times’, potentially an increase in the number of runs undertaken. In addition, and through stakeholder engagement, it has been revealed that there are a number of non technical issues slowing advances in this area which are related to a lack of clarity of the purpose of guidelines, their target/required audience and simple economics. This has lead to the production of five recommended cross-cutting actions: A. Agencies need to consider a better communication programme as to what is expected of contractors as regards adherence to their guidelines, when and how to depart from them. B. A greater education is needed among practitioners as regards simulation and traffic fundamentals and as such there may be a need for a core qualification of some description. C. Agencies need to ensure that contractual expectations as regards guideline observance are not in contradiction with budgetary constraints. D. Manufacturers need to be encouraged to provide software to expedite all stages of the simulation project life cycle. E. A heavily practitioner centric, pan-European forum, is needed for the ongoing debate of simulation issues with, as far as possible face to face meetings and the performance of short term working groups., JRC.F.8-Sustainable Transport

The evermore widespread use of microscopic traffic simulation in the analysis of road systems has refocused attention on submodels, including car-following models. The difficulties of microscopic-level simulation models in the accurate reproduction of real traffic phenomena stem not only from the complexity of calibration and validation operations but also from the structural inadequacies of the submodels themselves. Both of these drawbacks originate from the scant information available on real phenomena because of the difficulty with the gathering of accurate field data. In this study, the use of kinematic differential Global Positioning System instruments allowed the trajectories of four vehicles in a platoon to be accurately monitored under real traffic conditions on both urban and extraurban roads. Some of these data were used to analyze the behaviors of four microscopic traffic flow models that differed greatly in both approach and complexity. The effect of the choice of performance measures on the model calibration results was first investigated, and intervehicle spacing was shown to be the most reliable measure. Model calibrations showed results similar to those obtained in other studies that used test track data. Instead, validations resulted in higher deviations compared with those from previous studies (with peaks in cross validations between urban and extraurban experiments). This confirms the need for real traffic data. On comparison of the models, all models showed similar performances (i.e., similar deviations in validation). Surprisingly, however, the simplest model performed on average better than the others, but the most complex one was the most robust, never reaching particularly high deviations.

This chapter discusses the principles of calibration and validation and links these phases to the previous chapters on data collection and data enhancement. First, a generic outline of a typical simulation study is given in Section 4.1, then a generic procedure for calibration and validation is covered in Section 4.2, underlying three main choices: measures of performance, measures for goodness of fit, and optimization procedure. Section 4.3 presents examples of calibration for car following. Section 4.4 explains which role can be played by synthetic data while scrutinizing if the calibration and validation chain procedure is or not appropriate. Section 4.5 discusses conclusions.

A single source of information for researchers and professionals, Traffic Simulation and Data: Validation Methods and Applications offers a complete overview of traffic data collection, state estimation, calibration and validation for traffic modelling and simulation. It derives from the Multitude Project—a European Cost Action project that incorporates work packages defining traffic simulation practice and research; highway and network modeling; and synthesis, dissemination, and training. This book addresses the calibration and validation of traffic models, and introduces necessary frameworks and techniques. It also includes viable methods for sensitivity analyses, and incorporates relevant tools for application. The book begins with a brief summary of various data collection techniques that can be applied to collect different data types. It then showcases various data processing and enhancement techniques for improving the quality of collected data. It also introduces the techniques according to the type of estimation, for example microscopic data enhancement, traffic state estimation, feature extraction and parameter identification techniques, and origin–destination matrix estimation. The material discusses the measures of performance, data error and goodness of fit, and optimization algorithms. It also contains the sensitivity analyses of parameters in traffic models. Describes the various tasks of calibration and validation Considers the best use of available data Presents the sensitivity analysis method Discusses typical issues of data error in transportation system data and how these errors can impact simulation results Details various methodologies for data collection, sensitivity analysis, calibration, and validation Examines benefits that result from the application of these methods Traffic Simulation and Data: Validation Methods and Applications serves as a key resource for transport engineers and planners, researchers, and graduate students in transport engineering and planning.

convenient way to assess the traffic fuel consumption is to couple a microscopic traffic flow model with an instantaneous emission model. Traffic models are usually calibrated using goodness of fit indicators that are related to the traffic behaviour. Thus, this paper aims to investigate how the traffic model calibration influences the accuracy of fuel consumption estimation. Two traffic models are investigated: the Newell’s and the Gipps’s model. It appears that the Gipps’s model globally provides more accurate fuel consumption estimation than the Newell’s model. For both models, the fuel consumption estimation of a single vehicle is very sensitive to the calibration. This is confirmed by a global sensitivity analysis of the Gipp’s model that shows that non optimal parameters significantly increase the variance of the outputs. Fortunately, this is no longer the case when the fuel consumption of a whole platoon is calculated. In fact, using global mean value for the traffic model parameters lead to errors lower than 3% when estimation the consumption of a platoon of ten vehicles.

In recent time, in the field of traffic simulation, sensitivity analysis (SA) is starting to attract attention as an indispensible tool for simplifying the calibration of microscopic traffic flow models (1,2,3). These models, in facts, involve many sub-models and dozens of parameters (4) that need to be calibrated to make the model suitable to correctly reproduce local traffic conditions. Unfortunately, such models are quite computationally expensive (the typical duration of a simulation run being in the order of minutes) meanwhile the calibration in the high-dimensional space of model parameters usually requires several thousands of model evaluations. For this reason, it is common practice to carry out calibration only for a limited number of parameters. However, there is no established procedure for their selection, other than the personal experience. Therefore it is easy to imagine as the selection of an incomplete set of parameters for the calibration might lead to several issues, including but not limited to the inaccuracy and unreliability of model results as well as unrealistic values for the calibrated parameters. Therefore, a proper SA, including the initial screening of the parameters, can be very valuable for the subsequent calibration process (5), as it can provide both quantitative and qualitative information regarding the effects of the different model parameters (and their variations) on the simulation results. The sensitivity analysis of traffic simulation models is therefore carried out on its parameters, whereas the other inputs like the transportation network or the travel demand are kept constant. Since many sensitivity analysis techniques require a considerable number of model evaluations (5) and a SA needs to be repeated for each specific case study, the computational complexity still remains a problem. For this reason, the possibility of generalizing results of a SA of a specific traffic simulation model in a specific case study, is worth to be investigated. In the present work we present the preliminary results of an exploratory research in which the robustness of the results of a sensitivity analysis, carried out on the parameters of a car-following model, is assessed against the variation of the non-parametric inputs. Car-following models are the key components of all microscopic traffic simulation models. They describe the longitudinal motion of a vehicle by mimicking the reaction of its driver (the “follower”) to the stimuli perceived while interacting with the front vehicle (the “leader”). They are in the form of differential equations (sometimes delayed) whose basic inputs are, generally, the follower’s speed, the distance between the follower and the leader and their speed difference. Outputs of such models are usually the follower’s speed or acceleration. Traffic is therefore simulated through a system of chained coupled equations. Apart from the mentioned inputs, simulation outputs are strongly dependent on the values of the model parameters which vary among the population of drivers (that is along the chain of coupled equations) as deemed to capture the individual psycho-physical characteristics of each driver. In simulation practice such parameters are considered uncertain in order to cover all the uncertainty in the simulation process. They are usually calibrated through an inverse analysis, that is by looking for the value of parameters that allow the simulated time-space trajectory of a vehicle to be as near as possible to the measured one (6). Although these models have usually a quite simple formulation, their behavior, especially as the result of the parameter values, is not yet clear. Their SA is therefore an interesting and timely issue. As already mentioned, however, this is not the only objective of this work. Here, we do want also to ascertain how robust are the results of the analysis against the variation of the non-parametric inputs, that is, by varying the leader’s trajectory. To this aim, the sensitivity analysis included the leader’s trajectory as an additional factor which, in the Monte Carlo framework adopted, was sampled from a predefined dataset of trajectories. Such dataset, in particular, was built by picking trajectories measured in different roads (freeway and arterial) and in different traffic conditions. In this way it was possible to assess the relative effect on model outputs of the parameters and the input trajectories that was essential to understand whether results of a SA can be generalized (independently of the trajectory). In the experiment, we chose the sensitivity analysis technique based on the computation of the Sobol first order and total order sensitivity indices (5,7). Confidence intervals around the indices were also calculated in order to check for their stability. The car-following model used was the IDM model (8), while the trajectories considered were 101 trajectories selected from 10 databases available thanks to the NGSIM project which allowed us to capture a wide spectrum of driving behaviors. Some results are summarized in Figure 1. Figure 1. First and total order sensitivity indices for the parameters of the IDM model. They show that the input trajectory (identified by the PairID variable) has a prominent effect on the model outputs. At the same time, however, its effect is mainly played in combination with other model parameters. Overall, it can be said that there are few parameters exerting a certain effect on the output of the model no matter the input used. This is an important result as it opens the path for defining classes of problems, in traffic simulation, for which the parameters to calibrate can be defined a priori and not individuated case by case.

In the field of traffic simulation, the calibration of uncertain inputs against real data is usually taken to cover both the epistemic uncertainty regarding the un-modeled details of the phenomena and the aleatory not predicted by the models. For this reason, model parameters are usually indirectly estimated within an optimization framework which tries to maximize the fit between real and simulated measures of the traffic system. This is the case, for example, of the calibration of car-following models’ parameters against vehicle trajectory data. Only recently, it has been proven that the capability of the optimization framework to provide the parameters’ values that allow the car-following model reproducing real trajectories at its best is strictly connected to the setting of the optimization framework itself. This, in particular, entails the necessity to carefully choose an appropriate combination of optimization algorithm and measure of goodness of fit (GOF). In this study, the authors focus attention on this latter issue. Specifically, it is claimed here that the commonly used GOFs are not able to capture the dynamics of the time-series which calibration is performed against. Therefore, a spectral analysis based approach to evaluate the overall performance of the simulation model in the objective function is proposed. The new measure of goodness of fit is tested in the calibration of the Intelligent Driver Model against synthetic and real trajectory data. Results with synthetic data, in particular, confirm that such a new optimization setting is always able to find the global optimum of the problem., JRC.F.8-Sustainable Transport

This study evaluated the effectiveness of speed limits enforced with an automated section speed enforcement system (ASSES) through the analysis of compliance after the system implementation in the Italian Motorway A3 Naples-Salerno. The speed limits were selected taking into account the average of the following values: (1) minimum design speed; (2) minimum speed consistent with the available stopping sight distance; and (3) operating speeds before the ASSES implementation. The first two criteria take into account safety whereas the third criterion takes into account the mobility needs. The speed limits selected with these criteria were then refined basing on: maximum difference between the section speed limit and the minimum speed evaluated with criteria 1 and 2, evaluation of critical conditions, and consistency between the speed limits and the highway function. The system was activated on April 21, 2010 and the driving speeds were monitored in the two periods April 21, 2010 - May 11, 2010 and March 30, 2011 - April 20, 2011. Overall, non compliance to the speed limits was equal to 50%. Non compliance was higher for heavy vehicles (60-70%) than for light vehicles (32-65%). Non compliance of light vehicles was much higher in the sections with the most constrained alignments (64% in sections with speed limit equal to 80 kph vs. 34% in sections with speed limit equal to 100 kph). Higher compliance to the speed limits, according to the authors’ belief, might be achieved by a better strategy of communication and information to the road users and a speed limit management strategy synergic between the highway agency and the Police who actually manages the commitments of fines., JRC.F.8-Sustainable Transport

Railway designers and operators have the hard task of identifying effective solutions to achieve the established network performance levels trying to optimize as more as possible the allocation of economic and technical resources available. To this aim, a preliminary sensitivity analysis is strongly recommended to understand which design parameters are really worth focusing on, to improve a certain measure of performance, without wasting efforts for intervening on non relevant parameters. In this paper in fact, a variance-based sensitivity analysis framework is introduced for supporting decisional processes of railway system design. In particular such framework uses an “own-built” parallel microscopic simulation model to estimate network performances for all possible value combinations of the design parameters involved in the analysis. A practical application has been then carried out on a real mass transit system to understand how four different performance measures were influenced by four design parameters. Results show that the proposed framework is able to identify for a certain performance the variables which mostly affect it, addressing therefore the users towards the choice of intervention solutions which can lead to more efficient performance improvements.

This paper reports on the results of a web based survey conducted as part of the MULTITUDE project (Methods and tools for supporting the Use caLibration and validaTIon of Traffic simUlation moDEls). The project, being performed in Europe, is attempting to focus and drive forward the issue of 'model validity' and thereby calibration and validation methods and their usage in simulation. As part of this, an assessment of the state-of-the-practice has been performed, examining among other things how models are applied, what procedures are used for calibration and what guidelines are followed. The basis for this state-of-the-practice, is a web based survey which ran in the latter quarter of 2010 and obtained 215 responses. The paper details the findings of this survey which, while lengthy, substantiates two clear messages. Firstly, that Calibration, Validation and Sensitivity analysis although now more widespread than was previously thought, are still far from being required procedure, and secondly that while guidelines are increasingly in use, governing for example number of simulation runs, there are still many regions where this does not occur and indeed there is still much reliance on personal experience and habit, especially in situations where network characteristics become large. The results of the survey underpin the need for further focus on the use of models and procedures for calibration and validation, and indicate, more specifically, the areas that MULTITUDE will focus on in the near future.

Mesoscopic and microscopic simulation models of railway systems hold a fundamental role in supporting designing phases of both infrastructure components and railway operation strategies. Limits of such approaches regard the scarce accuracy in results of the former, which are unable to describe transient train dynamics, and the computational inefficiency of the latter which prevent their application to large-sized networks or probabilistic analyses. In this paper a microscopic and a mesoscopic “own built” railway simulation models are firstly introduced. Their application to a common case study and a comparison between the corresponding outputs and computing performances is then presented. Results comparison show that the dynamical integration of the approaches would overcome the respective limits improving both the range of possible applications and the accuracy of results.

In the last decades simulation optimization has received considerable attention from both researchers and practitioners. Simulation optimization is the process of finding the best values of some decision variables for a system whose performance is evaluated using the output of a simulation model. A possible example of simulation optimization is the model calibration. In traffic modelling this topic is particularly relevant since the solutions to the methodological issues arising when setting up a calibration study cannot be posed independently. This calls for methodologies able to check the robustness of a calibration framework as well as further investigations of the issue, in order to identify possible “classes” of problems to be treated in a similar way. Therefore in the present work, first we describe a general method for verifying a traffic micro-simulation calibration procedure (suitable in general for simulation optimization), based on a test with synthetic data. Then we apply it to draw inference on the effect that different combinations of parameters to calibrate, optimization algorithm, measures of Goodness of Fit and noise in the data may have on the optimization problem. The time required to perform all the simulations needed to the study would make the present procedure unfeasible. For this reason a Kriging surrogate of the simulation model has been used in its place. Results show the importance of verifying the calibration procedure with synthetic data. In addition they ascertain the need for global optimization solutions, giving new insights into the topic.

In this study, a methodology for the selection of speed limits in sections enforced with automated section speed enforcement systems (ASSESs) has been proposed. Results of the speed control system have been assessed by speed monitoring after the system implementation in the Italian Motorway A3 Naples-Salerno. The speed limits were selected taking into account the average of the following values: (1) minimum design speed; (2) minimum speed consistent with the available stopping sight distance; and (3) operating speeds before the ASSES implementation. The first two criteria take into account safety whereas the third criterion takes into account the mobility needs. The speed limits selected with these criteria were then refined basing on: maximum difference between the section speed limit and the minimum speed evaluated with criteria 1 and 2, evaluation of critical conditions, and consistency between the speed limits and the highway function. Overall, non compliance to the speed limits was equal to 50%. Non compliance was higher for heavy vehicles (60-70%) than for light vehicles (32-65%). Non compliance of light vehicles was much higher in the sections with the most constrained alignments (64% in sections with speed limit equal to 80 kph vs. 34% in sections with speed limit equal to 100 kph). Higher compliance to the speed limits, according to the authors’ belief, might be achieved by a better strategy of communication and information to the road users and a speed limit management strategy synergic between the highway agency and the Police.

In this work variance-based techniques for model sensitivity analysis have been discussed and applied to the Intelligent Driver Model (IDM) and the Gipps’ model. Throughout the paper it is argued that the application of such methods is crucial for a true comprehension and the correct use of these models. In particular, concerning the subject of their parameters estimation or calibration. Important issues arising when setting up a sensitivity analysis have been investigated and commented in the specific application to car-following models. Among these issues, the importance of the characterization of the uncertainty in the inputs, also known as data assimilation, and the influence of the data richness (i.e. the coverage of a wide spectrum of traffic patterns) on the characterization of the parameters significance and on their successive calibration. The method allowed us to show that both in the case of the IDM and the Gipps’ model, some parameters, which are generally considered fixed in the field literature, account for a high share of the output uncertainty and thus require to be calibrated. Sensitivity indices let us also to evaluate the parsimony of models, intended as the ability to describe reality with a minimum of adjusting parameters.

The increasing need, in the transportation sector, for new policy initiatives that generate positive and sustainable environmental, social and economic impacts is increasingly directed towards the promotion of intermodal integration in transportation that may prompt a rebalance in the mobility pattern in relevant areas, especially in an urban environment. An interesting example of a city with a great variety of such complex transportation challenges is the Venice area in Italy. In this paper a feasibility study for a sub‐lagoon tube connection between Venice island and its mainland is presented. This project aims to improve accessibility and sustainability in the Venice region, by favouring an intermodal re‐balance in the local transportation system. In particular, the methodology of transportation evaluation and of scenario simulations will be highlighted, while the main results of the study will be presented and discussed. Multi‐criteria analysis is used here inter alia as an important assessment and evaluation method. The case study of Venice can be considered as an illustrative study, in which a novel and challenging urban plan is followed by specific studies assessing the overall sustainability of the proposed policy intervention, even though it is not ensued by an immediate implementation.

The problem of calibrating traffic simulation models comes within the framework of “no free lunch” theorems: solutions to the methodological issues arising when setting up a calibration study cannot be posed independently. In other words, the choice of the algorithm to use, for instance, depends upon the parameters to calibrate, upon the measure of goodness of fit (GoF), and, of course, upon the micro-simulation model applied. This calls for methodologies able to check the robustness of a calibration framework as well as further investigations of the problem, in order to identify possible “classes” of problems to be treated using the same approach. Therefore in the present work, first we describe a general method for verifying a traffic micor-simulation calibration procedure, based on a test with synthetic data. Then we investigate the influence that the choice of a particular GoF measure may have on the results of a calibration exercise. In all, 16 GoF measures are analyzed by using them to create and visualize the objective function of eight different calibration configurations. It was thus also possible i) to test the hypothesis presented in Punzo and Ciuffo (2009) to calibrate different parameters independently, on different time series of measurements and ii) to check the effect on the calibration itself of random errors in traffic measurement. Results show the importance of verifying the calibration procedure with synthetic data before using real measurements. In addition they highlight limitations of some GoF measures as well as give major insights into the topic.