Search

Trajectory prediction is fundamental in computer vision and autonomous driving, particularly for understanding pedestrian behavior and enabling proactive decision-making. Existing approaches in this field often assume precise and complete observational data, neglecting the challenges associated with out-of-view objects and the noise inherent in sensor data due to limited camera range, physical obstructions, and the absence of ground truth for denoised sensor data. Such oversights are critical safety concerns, as they can result in missing essential, non-visible objects. To bridge this gap, we present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique. Our approach denoises noisy sensor observations in an unsupervised manner and precisely maps sensor-based trajectories of out-of-sight objects into visual trajectories. This method has demonstrated state-of-the-art performance in out-of-sight noisy sensor trajectory denoising and prediction on the Vi-Fi and JRDB datasets. By enhancing trajectory prediction accuracy and addressing the challenges of out-of-sight objects, our work significantly contributes to improving the safety and reliability of autonomous driving in complex environments. Our work represents the first initiative towards Out-Of-Sight Trajectory prediction (OOSTraj), setting a new benchmark for future research. The code is available at \url{https://github.com/Hai-chao-Zhang/OOSTraj}., Comment: In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (CVPR)

Trajectory prediction plays a vital role in understanding pedestrian movement for applications such as autonomous driving and robotics. Current trajectory prediction models depend on long, complete, and accurately observed sequences from visual modalities. Nevertheless, real-world situations often involve obstructed cameras, missed objects, or objects out of sight due to environmental factors, leading to incomplete or noisy trajectories. To overcome these limitations, we propose LTrajDiff, a novel approach that treats objects obstructed or out of sight as equally important as those with fully visible trajectories. LTrajDiff utilizes sensor data from mobile phones to surmount out-of-sight constraints, albeit introducing new challenges such as modality fusion, noisy data, and the absence of spatial layout and object size information. We employ a denoising diffusion model to predict precise layout sequences from noisy mobile data using a coarse-to-fine diffusion strategy, incorporating the RMS, Siamese Masked Encoding Module, and MFM. Our model predicts layout sequences by implicitly inferring object size and projection status from a single reference timestamp or significantly obstructed sequences. Achieving SOTA results in randomly obstructed experiments and extremely short input experiments, our model illustrates the effectiveness of leveraging noisy mobile data. In summary, our approach offers a promising solution to the challenges faced by layout sequence and trajectory prediction models in real-world settings, paving the way for utilizing sensor data from mobile phones to accurately predict pedestrian bounding box trajectories. To the best of our knowledge, this is the first work that addresses severely obstructed and extremely short layout sequences by combining vision with noisy mobile modality, making it the pioneering work in the field of layout sequence trajectory prediction., Comment: In Proceedings of the 31st ACM International Conference on Multimedia 2023 (MM 23)

In this paper, we propose first a mmWave channel tracking algorithm based on multidimensional orthogonal matching pursuit algorithm (MOMP) using reduced sparsifying dictionaries, which exploits information from channel estimates in previous frames. Then, we present an algorithm to obtain the vehicle's initial location for the current frame by solving a system of geometric equations that leverage the estimated path parameters. Next, we design an attention network that analyzes the series of channel estimates, the vehicle's trajectory, and the initial estimate of the position associated with the current frame, to generate a refined, high accuracy position estimate. The proposed system is evaluated through numerical experiments using realistic mmWave channel series generated by ray-tracing. The experimental results show that our system provides a 2D position tracking error below 20 cm, significantly outperforming previous work based on Bayesian filtering.

One strategy to obtain user location information in a wireless network operating at millimeter wave (mmWave) is based on the exploitation of the geometric relationships between the channel parameters and the user position. These relationships can be easily built from the LoS path and/or first order reflections, but high resolution channel estimates are required for high accuracy. In this paper, we consider a mmWave MIMO system based on a hybrid architecture, and develop first a low complexity channel estimation strategy based on MOMP suitable for high dimensional channels, as those associated to operating with large planar arrays. Then, a deep neural network (DNN) called PathNet is designed to classify the order of the estimated channel paths, so that only the line-of-sight (LOS) path and first order reflections are selected for localization purposes. Next, a 3D localization strategy exploiting the geometry of the environment is developed to operate in both LOS and non-line-of-sight (NLOS) conditions, while considering the unknown clock offset between the transmitter (TX) and the receiver (RX). Finally, a Transformer based network exploiting attention mechanisms called ChanFormer is proposed to refine the initial position estimate obtained from the geometric system of equations that connects user position and channel parameters. Simulation results obtained with realistic vehicular channels generated by ray tracing indicate that sub-meter accuracy (<= 0.45 m) can be achieved for 95% of the users in LOS channels, and for 50% of the users in NLOS conditions., Comment: Journal

Low Earth Orbit (LEO) satellite communications (SatCom) are considered a promising solution to provide uninterrupted services in cellular networks. Line-of-sight (LoS) links between the LEO satellites and the ground users are, however, easily blocked in urban scenarios. In this paper, we propose to enable LEO SatCom in non-line-of-sight (NLoS) channels, as those corresponding to links to users in urban canyons, with the aid of reconfigurable intelligent surfaces (RISs). First, we derive the near field signal model for the satellite-RIS-user link. Then, we propose two deployments to improve the coverage of a RIS-aided link: down tilting the RIS located on the top of a building, and considering a deployment with RISs located on the top of opposite buildings. Simulation results show the effectiveness of using RISs in LEO SatCom to overcome blockages in urban canyons. Insights about the optimal tilt angle and the coverage extension provided by the deployment of an additional RIS are also provided., Comment: 6 pages, 6 figures, submitted to Globecom 2022

High resolution compressive channel estimation provides information for vehicle localization when a hybrid mmWave MIMO system is considered. Complexity and memory requirements can, however, become a bottleneck when high accuracy localization is required. An additional challenge is the need of path order information to apply the appropriate geometric relationships between the channel path parameters and the vehicle, RSU and scatterers position. In this paper, we propose a low complexity channel estimation strategy of the angle of departure and time difference of arrival based on multidimensional orthogonal matching pursuit. We also design a deep neural network that predicts the order of the channel paths so only the LoS and first order reflections are used for localization. Simulation results obtained with realistic vehicular channels generated by ray tracing show that sub-meter accuracy can be achieved for 50% of the users, without resorting to perfect synchronization assumptions or unfeasible all-digital high resolution MIMO architectures.

We propose and examine the idea of continuously adapting state-of-the-art neural network (NN)-based orthogonal frequency division multiplex (OFDM) receivers to current channel conditions. This online adaptation via retraining is mainly motivated by two reasons: First, receiver design typically focuses on the universal optimal performance for a wide range of possible channel realizations. However, in actual applications and within short time intervals, only a subset of these channel parameters is likely to occur, as macro parameters, e.g., the maximum channel delay, can assumed to be static. Second, in-the-field alterations like temporal interferences or other conditions out of the originally intended specifications can occur on a practical (real-world) transmission. While conventional (filter-based) systems would require reconfiguration or additional signal processing to cope with these unforeseen conditions, NN-based receivers can learn to mitigate previously unseen effects even after their deployment. For this, we showcase on-the-fly adaption to current channel conditions and temporal alterations solely based on recovered labels from an outer forward error correction (FEC) code without any additional piloting overhead. To underline the flexibility of the proposed adaptive training, we showcase substantial gains for scenarios with static channel macro parameters, for out-of-specification usage and for interference compensation., Comment: Submitted to SPAWC 2022

Configuring millimeter wave links following a conventional beam training protocol, as the one proposed in the current cellular standard, introduces a large communication overhead, specially relevant in vehicular systems, where the channels are highly dynamic. In this paper, we propose the use of a passive radar array to sense automotive radar transmissions coming from multiple vehicles on the road, and a radar processing chain that provides information about a reduced set of candidate beams for the links between the road-infrastructure and each one of the vehicles. This prior information can be later leveraged by the beam training protocol to significantly reduce overhead. The radar processing chain estimates both the timing and chirp rates of the radar signals, isolates the individual signals by filtering out interfering radar chirps, and estimates the spatial covariance of each individual radar transmission. Then, a deep network is used to translate features of these radar spatial covariances into features of the communication spatial covariances, by learning the intricate mapping between radar and communication channels, in both line-of-sight and non-line-of-sight settings. The communication rates and outage probabilities of this approach are compared against exhaustive search and pure radar-aided beam training methods (without deep learning-based mapping), and evaluated on multi-user channels simulated by ray tracing. Results show that: (i) the proposed processing chain can reliably isolate the spatial covariances for individual radars, and (ii) the radar-to-communications translation strategy based on deep learning provides a significant improvement over pure radar-aided methods in both LOS and NLOS channels.

The high overhead of the beam training process is the main challenge when establishing mmWave communication links, especially for vehicle-to-everything (V2X) scenarios where the channels are highly dynamic. In this paper, we obtain prior information to speed up the beam training process by implementing two deep neural networks (DNNs) that realize radar-to-communication (R2C) channel information translation in a vehicle-to-infrastructure (V2I) system. Specifically, the first DNN is built to extract the information from the radar azimuth power spectrum (APS) to reconstruct the communication APS, while the second DNN exploits the information extracted from the spatial covariance of the radar channel to realize R2C covariance prediction. The achieved data rate and the similarity between the estimated and the true communication APS are used to evaluate the prediction performance. The covariance estimation method generally provides higher similarity, as the APS predictions cannot always capture the mismatch between the radar and communication APS. Compared to the beam training method which exploits directly the radar APS without an attempt to translate it to the communication channel, our proposed deep learning (DL) aided methods remarkably reduce the beam training overhead, resulting in a 13.3% and 21.9% rate increase when using the communication APS prediction and covariance prediction, respectively.

Beamforming gain is a key ingredient in the performance of LEO satellite communication systems to be integrated into cellular networks. However, beam codebooks previously designed in the context of MIMO communication for terrestrial networks, do not provide the appropriate performance in terms of inter-beam interference and gain stability as the satellite moves. In this paper, we propose a dynamic codebook that provides a stable gain during the period of time that the satellite covers a given cell, while avoiding link retraining and extra calculation as the satellite moves. In addition, the proposed codebook provides a higher signal-to-interference-plus-noise (SINR) ratio than those DFT codebooks commonly used in cellular systems., Comment: submitted to ICC 2022

5G and future cellular networks intend to incorporate low earth orbit (LEO) satellite communication systems (SatCom) to solve the coverage and availability problems that cannot be addressed by satellite-based or ground-based infrastructure alone. This integration of terrestrial and non terrestrial networks poses many technical challenges which need to be identified and addressed. To this aim, we design and simulate the downlink of a LEO SatCom compatible with 5G NR, with a special focus on the design of the beamforming codebook at the satellite side. The performance of this approach is evaluated for the link between a LEO satellite and a mobile terminal in the Ku band, assuming a realistic channel model and commercial antenna array designs, both at the satellite and the terminal. Simulation results provide insights on open research challenges related to analog codebook design and hybrid beamforming strategies, requirements of the antenna terminals to provide a given SNR, or required beam reconfiguration capabilities among others.

We compare the potential of neural network (NN)-based channel estimation with classical linear minimum mean square error (LMMSE)-based estimators, also known as Wiener filtering. For this, we propose a low-complexity recurrent neural network (RNN)-based estimator that allows channel equalization of a sequence of channel observations based on independent time- and frequency-domain long short-term memory (LSTM) cells. Motivated by Vehicle-to-Everything (V2X) applications, we simulate time- and frequency-selective channels with orthogonal frequency division multiplex (OFDM) and extend our channel models in such a way that a continuous degradation from line-of-sight (LoS) to non-line-of-sight (NLoS) conditions can be emulated. It turns out that the NN-based system cannot just compete with the LMMSE equalizer, but it also can be trained w.r.t. resilience against system parameter mismatch. We thereby showcase the conceptual simplicity of such a data-driven system design, as this not only enables more robustness against, e.g., signal-to-noise-ratio (SNR) or Doppler spread estimation mismatches, but also allows to use the same equalizer over a wider range of input parameters without the need of re-building (or re-estimating) the filter coefficients. Particular attention has been paid to ensure compatibility with the existing IEEE 802.11p piloting scheme for V2X communications. Finally, feeding the payload data symbols as additional equalizer input unleashes further performance gains. We show significant gains over the conventional LMMSE equalization for highly dynamic channel conditions if such a data-augmented equalization scheme is used., Comment: Accepted at 32nd IEEE Intelligent Vehicles Symposium

Millimeter wave provides high data rates for Vehicle-to-Everything (V2X) communications. This paper motivates millimeter wave to support automated driving and begins by explaining V2X use cases that support automated driving with references to several standardi-zation bodies. The paper gives a classification of existing V2X stand-ards: IEEE802.11p and LTE V2X, along with the status of their com-mercial deployment. Then, the paper provides a detailed assessment on how millimeter wave V2X enables the use case of cooperative perception. The explanations provide detailed rate calculations for this use case and show that millimeter wave is the only technology able to achieve the requirements. Furthermore, specific challenges related to millimeter wave for V2X are described, including cover-age enhancement and beam alignment. The paper concludes with some results from three studies, i.e. IEEE802.11ad (WiGig) based V2X, extension of 5G NR (New Radio) toward mmWave V2X, and prototypes of intelligent street with mmWave V2X.

Modern vehicular wireless technology enables vehicles to exchange information at any time, from any place, to any network -- forms the vehicle-to-everything (V2X) communication platforms. Despite benefits, V2X applications also face great challenges to security and privacy -- a very valid concern since breaches are not uncommon in automotive communication networks and applications. In this survey, we provide an extensive overview of V2X ecosystem. We also review main security/privacy issues, current standardization activities and existing defense mechanisms proposed within the V2X domain. We then identified semantic gaps of existing security solutions and outline possible open issues., Comment: Accepted for publication, IEEE Transactions on Intelligent Vehicles, March 2020. arXiv admin note: text overlap with arXiv:1610.06810 by other authors

Recently, millimeter wave (mmWave) bands have been investigated as a means to enhance automated driving and address the challenging data rate and latency demands of emerging automotive applications. For the development of those systems to operate in bands above 6 GHz, there is a need to have accurate channel models able to predict the peculiarities of the vehicular propagation at these bands, especially as far as Vehicle-to-Vehicle (V2V) communications are concerned. In this paper, we validate the channel model that the 3GPP has proposed for NR-V2X systems, which (i) supports deployment scenarios for urban/highway propagation, and (ii) incorporates the effects of path loss, shadowing, line of sight probability, and static/dynamic blockage attenuation. We also exemplify the impact of several automotive-specific parameters on the overall network performance considering realistic system-level simulation assumptions for typical scenarios. Finally, we highlight potential inconsistencies of the model and provide recommendations for future measurement campaigns in vehicular environments., Comment: 5 pages, 3 figures, 4 tables. Accepted to the IEEE 2nd Connected and Automated Vehicles Symposium

A major source of difficulty when operating with large arrays at mmWave frequencies is to estimate the wideband channel, since the use of hybrid architectures acts as a compression stage for the received signal. Moreover, the channel has to be tracked and the antenna arrays regularly reconfigured to obtain appropriate beamforming gains when a mobile setting is considered. In this paper, we focus on the problem of channel tracking for frequency-selective mmWave channels, and propose two novel channel tracking algorithms that leverage prior statistical information on the angles-of-arrival and angles-of-departure. Exploiting this prior information, we also propose a precoding and combining design method to increase the received SNR during channel tracking, such that near-optimum data rates can be obtained with low-overhead. In our numerical results, we analyze the performance of our proposed algorithms for different system parameters. Simulation results show that, using channel realizations extracted from the 5G New Radio channel model, our proposed channel tracking framework is able to achieve near-optimum data rates.

A critical requirement for automated driving systems is enabling situational awareness in dynamically changing environments. To that end vehicles will be equipped with diverse sensors, e.g., LIDAR, cameras, mmWave radar, etc. Unfortunately the sensing `coverage' is limited by environmental obstructions, e.g., other vehicles, buildings, people, objects etc. A possible solution is to adopt collaborative sensing amongst vehicles possibly assisted by infrastructure. This paper introduces new models and performance analysis for vehicular collaborative sensing and networking. In particular, coverage gains are quantified, as are their dependence on the penetration of vehicles participating in collaborative sensing. We also evaluate the associated communication loads in terms of the Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) capacity requirements and how these depend on penetration. We further explore how collaboration with sensing capable infrastructure improves sensing performance, as well as the benefits in utilizing spatio-temporal dynamics, e.g., collaborating with vehicles moving in the opposite direction. Collaborative sensing is shown to greatly improve sensing performance, e.g., improves coverage from 20% to 80% with a 20% penetration. In scenarios with limited penetration and high coverage requirements, infrastructure can be used to both help sense the environment and relay data. Once penetration is high enough, sensing vehicles provide good coverage and data traffic can be effectively `offloaded' to V2V connectivity, making V2I resources available to support other in-car services.

Accurate beam alignment is essential for beam-based millimeter wave communications. Conventional beam sweeping solutions often have large overhead, which is unacceptable for mobile applications like vehicle-to-everything. Learning-based solutions that leverage sensor data like position to identify good beam directions are one approach to reduce the overhead. Most existing solutions, though, are supervised-learning where the training data is collected beforehand. In this paper, we use a multi-armed bandit framework to develop online learning algorithms for beam pair selection and refinement. The beam pair selection algorithm learns coarse beam directions in some predefined beam codebook, e.g., in discrete angles separated by the 3dB beamwidths. The beam refinement fine-tunes the identified directions to match the peak of the power angular spectrum at that position. The beam pair selection uses the upper confidence bound (UCB) with a newly proposed risk-aware feature, while the beam refinement uses a modified optimistic optimization algorithm. The proposed algorithms learn to recommend good beam pairs quickly. When using 16x16 arrays at both the transmitter and receiver, it can achieve on average 1dB gain over the exhaustive search (over 271x271 beam pairs) on the unrefined codebook within 100 time-steps with a training budget of only 30 beam pairs., Comment: 18 pages, 11 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Efficient beam alignment is a crucial component in millimeter wave systems with analog beamforming, especially in fast-changing vehicular settings. This paper proposes a position-aided approach where the vehicle's position (e.g., available via GPS) is used to query the multipath fingerprint database, which provides prior knowledge of potential pointing directions for reliable beam alignment. The approach is the inverse of fingerprinting localization, where the measured multipath signature is compared to the fingerprint database to retrieve the most likely position. The power loss probability is introduced as a metric to quantify misalignment accuracy and is used for optimizing candidate beam selection. Two candidate beam selection methods are developed, where one is a heuristic while the other minimizes the misalignment probability. The proposed beam alignment is evaluated using realistic channels generated from a commercial ray-tracing simulator. Using the generated channels, an extensive investigation is provided, which includes the required measurement sample size to build an effective fingerprint, the impact of measurement noise, the sensitivity to changes in traffic density, and beam alignment overhead comparison with IEEE 802.11ad as the baseline. Using the concept of beam coherence time, which is the duration between two consecutive beam alignments, and parameters of IEEE 802.11ad, the overhead is compared in the mobility context. The results show that while the proposed approach provides increasing rates with larger antenna arrays, IEEE 802.11ad has decreasing rates due to the larger beam training overhead that eats up a large portion of the beam coherence time, which becomes shorter with increasing mobility., Comment: 16 pages, 19 figures, Submitted to IEEE Transactions on Vehicular Technology