Assessing the robustness of sensitivity analysis results. Application to traffic simulation models

In recent time, in the field of traffic simulation, sensitivity analysis (SA) is starting to attract attention as an indispensible tool for simplifying the calibration of microscopic traffic flow models (1,2,3). These models, in facts, involve many sub-models and dozens of parameters (4) that need to be calibrated to make the model suitable to correctly reproduce local traffic conditions. Unfortunately, such models are quite computationally expensive (the typical duration of a simulation run being in the order of minutes) meanwhile the calibration in the high-dimensional space of model parameters usually requires several thousands of model evaluations. For this reason, it is common practice to carry out calibration only for a limited number of parameters. However, there is no established procedure for their selection, other than the personal experience. Therefore it is easy to imagine as the selection of an incomplete set of parameters for the calibration might lead to several issues, including but not limited to the inaccuracy and unreliability of model results as well as unrealistic values for the calibrated parameters. Therefore, a proper SA, including the initial screening of the parameters, can be very valuable for the subsequent calibration process (5), as it can provide both quantitative and qualitative information regarding the effects of the different model parameters (and their variations) on the simulation results. The sensitivity analysis of traffic simulation models is therefore carried out on its parameters, whereas the other inputs like the transportation network or the travel demand are kept constant. Since many sensitivity analysis techniques require a considerable number of model evaluations (5) and a SA needs to be repeated for each specific case study, the computational complexity still remains a problem. For this reason, the possibility of generalizing results of a SA of a specific traffic simulation model in a specific case study, is worth to be investigated. In the present work we present the preliminary results of an exploratory research in which the robustness of the results of a sensitivity analysis, carried out on the parameters of a car-following model, is assessed against the variation of the non-parametric inputs. Car-following models are the key components of all microscopic traffic simulation models. They describe the longitudinal motion of a vehicle by mimicking the reaction of its driver (the “follower”) to the stimuli perceived while interacting with the front vehicle (the “leader”). They are in the form of differential equations (sometimes delayed) whose basic inputs are, generally, the follower’s speed, the distance between the follower and the leader and their speed difference. Outputs of such models are usually the follower’s speed or acceleration. Traffic is therefore simulated through a system of chained coupled equations. Apart from the mentioned inputs, simulation outputs are strongly dependent on the values of the model parameters which vary among the population of drivers (that is along the chain of coupled equations) as deemed to capture the individual psycho-physical characteristics of each driver. In simulation practice such parameters are considered uncertain in order to cover all the uncertainty in the simulation process. They are usually calibrated through an inverse analysis, that is by looking for the value of parameters that allow the simulated time-space trajectory of a vehicle to be as near as possible to the measured one (6). Although these models have usually a quite simple formulation, their behavior, especially as the result of the parameter values, is not yet clear. Their SA is therefore an interesting and timely issue. As already mentioned, however, this is not the only objective of this work. Here, we do want also to ascertain how robust are the results of the analysis against the variation of the non-parametric inputs, that is, by varying the leader’s trajectory. To this aim, the sensitivity analysis included the leader’s trajectory as an additional factor which, in the Monte Carlo framework adopted, was sampled from a predefined dataset of trajectories. Such dataset, in particular, was built by picking trajectories measured in different roads (freeway and arterial) and in different traffic conditions. In this way it was possible to assess the relative effect on model outputs of the parameters and the input trajectories that was essential to understand whether results of a SA can be generalized (independently of the trajectory). In the experiment, we chose the sensitivity analysis technique based on the computation of the Sobol first order and total order sensitivity indices (5,7). Confidence intervals around the indices were also calculated in order to check for their stability. The car-following model used was the IDM model (8), while the trajectories considered were 101 trajectories selected from 10 databases available thanks to the NGSIM project which allowed us to capture a wide spectrum of driving behaviors. Some results are summarized in Figure 1. Figure 1. First and total order sensitivity indices for the parameters of the IDM model. They show that the input trajectory (identified by the PairID variable) has a prominent effect on the model outputs. At the same time, however, its effect is mainly played in combination with other model parameters. Overall, it can be said that there are few parameters exerting a certain effect on the output of the model no matter the input used. This is an important result as it opens the path for defining classes of problems, in traffic simulation, for which the parameters to calibrate can be defined a priori and not individuated case by case.