Search

High data rate communication enables vehicles to perform intelligent services. Leveraging millimeter-wave (mmWave) multiple-input-multiple-output (MIMO) technology, vehicles can effectively transmit and receive sensor data from a variety of sources such as cameras, lidar, and radar. The directionality of mmWave MIMO communication, however, necessitates on-the-fly beam designs to tackle mobility and blockage in vehicular scenarios. The inherent complexity of vehicle dynamics makes this design problem a challenging sequential decision-making task. Consequently, there is a strong motivation to adopt an online learning approach that can simultaneously tackle blockage mitigation and beam management in real-time. In this dissertation, I first explore the joint relay selection and beam management in mmWave vehicular networks with analog beamforming architecture using deep reinforcement learning (DRL). The objective is to dynamically initiate beam-based transmission, track beams, and execute relay switching with minimal overhead. An integral component of the DRL algorithm involves acquiring the capability to learn adaptive thresholds that govern the decision-making process for relay switching and beam realignment. Furthermore, I propose a joint band assignment and beam management algorithm to maximize the data rate in multi-band vehicular networks with hybrid beamforming architecture. This problem necessitates effective handling of the distinct beam management procedures across bands without excessively expanding the action space. To address this, I exploit hierarchy in decision-making and develop a hierarchical reinforcement learning (HRL) algorithm to learn band assignment in the upper level and beam management in the lower level. Lastly, I revisit the mmWave joint relay selection and beam management, focusing on the beam squint effect. Array architectures based on phase shifters are susceptible to discrepancies between the steering angle and beam direction, known as beam squint, especially as the number of antennas or bandwidth increases. To address this issue in wideband communication and massive MIMO systems, I propose a learning algorithm to compute the delay-phase profile to use both true-time-delay and phase shifter arrays.