Search

With the rapid progress of the Internet of Things (IoT) technology, human behavior recognition has become an important research topic in the field of ubiquitous computing and has obtained quite a number of research achievements. Accurate human behavior recognition can enhance the quality of human-computer interaction and facilitate the development of various sensing applications. With the popularity of smartphones and the improved performance of sensors such as speakers and microphones built in smartphones, the behavior recognition technique based on ultrasound signal of the smartphone has gained more attention and achieved several research results. In this paper, we first review the common behavior recognition techniques including light, video, sound, and frequency radio and outline their main characteristics. Then, we introduce the fundamental principle of human behavior recognition based on ultrasound signals. Specifically, these systems treat speakers and microphones embedded in smartphones as the transceiver and leverage the received signal changes caused by human movement including phase differences, frequency shift, and time of flight (ToF) to recognize human behavior. Next, we investigate the state-of-the-art studies and applications and analyze the signal processing techniques such as data collection, signal preprocessing, feature description, and behavior recognition approach. Afterward, according to the purpose of these applications, we classify them into five groups and compare them in detail including hand gesture recognition, activity recognition, hand trajectory tracking, vital sign monitoring, and lip reading. Finally, we conclude by discussing the limitations, challenges, and open issues involved in behavior recognition based on ultrasound signal of smartphone.

With the rapid development of Internet of Things, hand gesture recognition has drawn wide attention in the field of ubiquitous computing because it provides us with simple and natural human-computer interaction mode. Among these various implementations, hand gesture recognition using ultrasonic signals of smartphone has become a hot research topic due to its various advantages. In this paper, we consider the smartphone as an active sonar sensing system to identify hand movements. Specifically, the speakers emit ultrasonic signal and the microphone on the same phone receives the changed echo affected by hand movements. This paper investigates the state-of-the-art hand gesture applications and presents a comprehensive survey on the characteristics of studies using the active sonar sensing system. Firstly, we review the existing research of hand gesture recognition based on acoustic signals. After that, we introduce the characteristics of ultrasonic signal and describe the fundamental principle of hand gesture recognition. Then, we focus on the typical methods used in these studies and present a detailed analysis on signal generation, feature extraction, preprocessing, and recognition methods. Next, we investigate the state-of-the-art ultrasonic-based applications of hand gesture recognition using smartphone and analyze them in detail from dynamic gesture recognition and hand tracking. Afterwards, we make a discussion about these systems from signal acquisition, signal processing, and performance evaluation to obtain some insight into development of the ultrasonic hand gesture recognition system. Finally, we conclude by discussing the challenges, insight, and open issues involved in hand gesture recognition based on ultrasonic signal of the smartphone.

Recent years have witnessed increasing research interest in human behavior recognition as it provides attractive applications in various sensing scenarios. Among these encouraging implementations, device-free behavior recognition based on WiFi channel state information (CSI) has attracted significant attention due to the popularity of WiFi devices and abundant channel characteristics from CSI. Meanwhile, the CSI signal provides us with additional benefits because it can propagate through a wall. This through-the-wall and device-free scheme not only enables us to identify specific human actions but also to infer person activities by collecting the data from different rooms. This paper presents a survey on the state-of-art progresses in device-free through-the-wall human behavior recognition based on CSI. Specifically, this paper first introduces the basic concept of CSI and describes the signal variation caused by human behavior. Then, it illustrates that different human behaviors can cause signal transformation. Therefore, the unique map relationship between action and signal variation can be leveraged to recognize human behavior. Next, it provides the general architecture of through-the-wall behavior recognition and highlights its core characteristic. It investigates the state-of-art applications in various scenarios and analyzes specific design schemes and implementations. Afterward, it discusses the various across wall applications and makes a detailed comparison between non-through-the-wall and through-the-wall applications. Meanwhile, it analyzes many factors that affect recognition accuracy and emphasizes performance differences under across wall scenarios. Finally, this paper concludes by summarizing the issues and challenges faced and providing insights into the possible solution and future research trend.

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.