Search

This paper presents a novel approach to material classification using short-wave infrared (SWIR) imaging, aimed at applications where differentiating visually similar objects based on material properties is essential, such as in autonomous driving. Traditional vision systems, relying on visible spectrum imaging, struggle to distinguish between objects with similar appearances but different material compositions. Our method leverages SWIR’s distinct reflectance characteristics, particularly for materials containing moisture, and demonstrates a significant improvement in accuracy. Specifically, SWIR data achieved near-perfect classification results with an accuracy of 99% for distinguishing real from artificial objects, compared to 77% with visible spectrum data. In object detection tasks, our SWIR-based model achieved a mean average precision (mAP) of 0.98 for human detection and up to 1.00 for other objects, demonstrating its robustness in reducing false detections. This study underscores SWIR’s potential to enhance object recognition and reduce ambiguity in complex environments, offering a valuable contribution to material-based object recognition in autonomous driving, manufacturing, and beyond.

Providing ubiquitous connectivity for people and equipment with a wide range of service conditions is regarded as the major design goal of 6G networks. Moreover, the future 6G networks need to be extremely flexible to enable reliable and low-latency access for the dynamic number of mobile user equipment. The current base stations (BSs) and remote relay antennas are mostly, on the other hand, installed at fixed geographic locations based on long-term traffic patterns with little re-deployment flexibility. For most 6G applications, where dynamic data traffic occurs in both spatial and temporal domains, such rigid radio access networks (RANs) are unable to maintain ubiquitous connectivity. On the other hand, the rapid advancement of unmanned aerial vehicle (UAV) technology is creating a unique opportunity for the cellular operator to realize flying wireless infrastructure called wireless infrastructure UAV (WI UAV). WI UAV supports wireless connectivity and can stir itself to improve spectral efficiency, coverage, and quality of service of the end-users. This paper introduces 6G new radio in the unlicensed band (NR-U)-based WI UAV that can be used as a base station, relay, or data collector/disseminator. UAV are categorized based on their characteristics, applications, and operations. Further, this paper discusses various regulatory and standardization initiatives for integrating UAVs into the cellular network. Non-standalone NR-U network architecture is designed and explained for WI UAV. Several opportunities and design challenges of NR-U for the WI UAV are discussed and future scopes of WI UAVs are presented.

Connected and automated vehicles (CAVs) require multiple tasks in their seamless maneuverings. Some essential tasks that require simultaneous management and actions are motion planning, traffic prediction, traffic intersection management, etc. A few of them are complex in nature. Multi-agent reinforcement learning (MARL) can solve complex problems involving simultaneous controls. Recently, many researchers applied MARL in such applications. However, there is a lack of extensive surveys on the ongoing research to identify the current problems, proposed methods, and future research directions in MARL for CAVs. This paper provides a comprehensive survey on MARL for CAVs. A classification-based paper analysis is performed to identify the current developments and highlight the various existing research directions. Finally, the challenges in current works are discussed, and some potential areas are given for exploration to overcome those challenges. Future readers will benefit from this survey and can apply the ideas and findings in their research to solve complex problems.

Unmanned aerial vehicles (UAVs) and UAV traffic management (UTM) have drawn attention for applications such as parcel delivery, aerial mapping, agriculture, and surveillance based on line-of-sight (LoS) links. UTM is essential to operate multiple fully autonomous UAVs safely beyond the visual line of sight (BVLoS) in the future dense UAV traffic environment. Various research and development teams globally take UTM initiatives and work on platform testing with different industrial partners. In the future, urban airspace will be congested with various types of autonomous aerial vehicles, thereby resulting in complex air-traffic management caused by communication issues. The UTM requires an efficient communication backbone to handle all airborne communication services. Existing cellular networks are suitable only for terrestrial communication and have limitations in supporting aerial communications. These issues motivate the investigation of an appropriate communication technology for advanced UTM systems. Thus, in this study, we present a future perspective of 6G-enabled UTM ecosystems in a very dense and urban air-traffic scenario focusing on non-terrestrial features, including aerial and satellite communication. We also introduce several urban airspace segmentation and discuss a strategic management framework for dynamic airspace traffic management and conflict-free UAV operations. The UTM enhances the adaptive use of the airspace by shaping the airspace with the overall aim of maximizing the capability and efficiency of the network. We also discuss the 6G multi-layer parameters i.e., space, air, and terrestrial, for safe and efficient urban air transportation in three-dimensional space. Moreover, we discuss the issues and challenges faced by future UTM systems and provide tentative solutions. We subsequently extend the vision of the UTM system and design an advanced and fully autonomous 6G-based UTM system.

The recent growth and adoption of unmanned aerial vehicles (UAVs) is due to their low development cost, high aerial mobility, advanced battery technology, rotors, gyroscopes, GPS, cameras, sensors, and wide range of applications. The UAV offers new possibilities for business in civil and noncivil applications such as parcel delivery, aerial mapping, agriculture, wildlife conservation, surveillance, and search and rescue. However, considering the exponential growth of various types of aerial vehicles, it is clear that advanced air traffic management services are urgently needed to control and handle the ever-increasing air traffics. In this paper, we first present the two major low-altitude UAV traffic management systems in the United States and Europe and compare them thoroughly. We present the global vision and advancement of UAV traffic management system. To understand the complexity of future air transportation system, we discuss the necessary elements required for an urban air traffic management system. We provide key strategic elements for an advanced urban traffic management through connectivity, open source, safety enhancement, and automation to maximize its impacts on the innovative future air traffic and mobility system. In the end, we discuss the challenging security issues in urban traffic management system.

In this paper, we have demonstrated a robust in-cabin monitoring system (IMS) for safety, security, surveillance, and monitoring, including privacy concerns for personal and shared autonomous vehicles (AVs). It consists of a set of monitoring cameras and an onboard device (OBD) equipped with artificial intelligence (AI). Hereafter, this combination of a camera and an OBD is referred to as the AI camera. We have investigated the issues for mobility services in higher levels of autonomous driving, what needs to be monitored, how to monitor, etc. Our proposed IMS is an on-device AI system that indigenously has improved the privacy of the users. Furthermore, we have enlisted the essential actions to be considered in an IMS and developed an appropriate database (DB). Our DB consists of multifaced scenarios important for monitoring the in-cabin of the higher-level AVs. Moreover, we have compared popular AI models applied for object and occupant recognition. In addition, our DB is available on request to support the research on the development of seamless monitoring of the in-cabin higher levels of autonomous driving for the assurance of safety and security.

Driving in an adverse rain environment is a crucial challenge for vision-based advanced driver assistance systems (ADAS) in the automotive industry. The vehicle windshield wiper removes adherent raindrops that cause distorted images from in-vehicle frontal view cameras, but, additionally, it causes an occlusion that can hinder visibility at the same time. The wiper-occlusion causes erroneous judgments by vision-based applications and endangers safety. This study proposes behind-the-scenes (BTS) that detects and removes wiper-occlusion in real-time image inputs under rainy weather conditions. The pixel-wise wiper masks are detected by high-pass filtering to predict the optical flow of a sequential image pair. We fine-tuned a deep learning-based optical flow model with a synthesized dataset, which was generated with pseudo-ground truth wiper masks and flows using auto-labeling with acquired real rainy images. A typical optical flow dataset with static synthetic objects is synthesized with real fast-moving objects to enhance data diversity. We annotated wiper masks and scenes as detection ground truths from the collected real images for evaluation. BTS outperforms by achieving a 0.962 SSIM and 91.6% F1 score in wiper mask detection and 88.3% F1 score in wiper image detection. Consequently, BTS enhanced the performance of vision-based image restoration and object detection applications by canceling occlusions and demonstrated it potential role in improving ADAS under rainy weather conditions.

An authorized traffic controller (ATC) has the highest priority for direct road traffic. In some irregular situations, the ATC supersedes other traffic control. Human drivers indigenously understand such situations and tend to follow the ATC; however, an autonomous vehicle (AV) can become confused in such circumstances. Therefore, autonomous driving (AD) crucially requires a human-level understanding of situation-aware traffic gesture recognition. In AVs, vision-based recognition is particularly desirable because of its suitability; however, such recognition systems have various bottlenecks, such as failing to recognize other humans on the road, identifying a variety of ATCs, and gloves in the hands of ATCs. We propose a situation-aware traffic control hand-gesture recognition system, which includes ATC detection and gesture recognition. Three-dimensional (3D) hand model-based gesture recognition is used to mitigate the problem associated with gloves. Our database contains separate training and test videos of approximately 60 min length, captured at a frame rate of 24 frames per second. It has 35,291 different frames that belong to traffic control hand gestures. Our approach correctly recognized traffic control hand gestures; therefore, the proposed system can be considered as an extension of the operational domain of the AV.

The typical configuration of virtual reality (VR) devices consists of a head-mounted display (HMD) and handheld controllers. As such, these units have limited utility in tasks that require hand-free operation, such as in surgical operations or assembly works in cyberspace. We propose a user interface for a VR headset based on a wearer’s facial gestures for hands-free interaction, similar to a touch interface. By sensing and recognizing the expressions associated with the in situ intentional movements of a user’s facial muscles, we define a set of commands that combine predefined facial gestures with head movements. This is achieved by utilizing six pairs of infrared (IR) photocouplers positioned at the foam interface of an HMD. We demonstrate the usability and report on the user experience as well as the performance of the proposed command set using an experimental VR game without any additional controllers. We obtained more than 99% of recognition accuracy for each facial gesture throughout the three steps of experimental tests. The proposed input interface is a cost-effective and efficient solution that facilitates hands-free user operation of a VR headset using built-in infrared photocouplers positioned in the foam interface. The proposed system recognizes facial gestures and incorporates a hands-free user interface to HMD, which is similar to the touch-screen experience of a smartphone.

Developing a user interface (UI) suitable for headset environments is one of the challenges in the field of augmented reality (AR) technologies. This study proposes a hands-free UI for an AR headset that exploits facial gestures of the wearer to recognize user intentions. The facial gestures of the headset wearer are detected by a custom-designed sensor that detects skin deformation based on infrared diffusion characteristics of human skin. We designed a deep neural network classifier to determine the user’s intended gestures from skin-deformation data, which are exploited as user input commands for the proposed UI system. The proposed classifier is composed of a spatiotemporal autoencoder and deep embedded clustering algorithm, trained in an unsupervised manner. The UI device was embedded in a commercial AR headset, and several experiments were performed on the online sensor data to verify operation of the device. We achieved implementation of a hands-free UI for an AR headset with average accuracy of 95.4% user-command recognition, as determined through tests by participants.