Search

Hand-raising gesture is one of the most popular signs of communication, whose frequency is related to the classroom’s atmosphere, the attractiveness of the subject, and the level of interactions between students and teachers. However, automatic hand-raising gesture detection and recognition remains still a challenging problem mainly due to low hand resolution, hand occlusion, various backgrounds, and viewpoint changes. While majority of the existing methods focus on static hand-raising posture detection, in this paper, we propose a framework for dynamic hand gesture recognition from classroom videos consisting of two main stages: hand posture detection and dynamic hand gesture recognition. In hand posture detection stage, we extend the previous work by adding relative position-aware in non-local network. After detecting the hand-raising posture on static pixels, which in turn accelerates the performance images, at the dynamic hand gesture recognition stage, we incorporate object detection and tracking to associate the hand-raising detection results at consecutive frames to supplement the missing detection due to the occlusion issue and obtain hand-raising gesture recognition at the event level. The experimental results show that the proposed method outperforms three benchmark models for static hand posture detection that are Faster-RCNN, Libra-RCNN, and Libra-RCNN+RDA on our dataset with the gains obtained for mAP being 7.68%, 5.76%, and 0.35% higher, respectively. In terms of event-level recognition results, the proposed method achieves the value of frame-wise accuracy, temporal_IoU, F1-score@0.3, and Levenshtein-score are 90.0%, 84.4%, 83.2%, and 84.3%, respectively. The code and dataset used in the paper will be made publicity available for research community.

Human action recognition (HAR) under different camera viewpoints is the most critical requirement for practical deployment. In this paper, we propose a novel method that leverages successful deep learning-based features for action representation and multi-view analysis to accomplish robust HAR under viewpoint changes. Specifically, we investigate various deep learning techniques, from 2D CNNs to 3D CNNs to capture spatial and temporal characteristics of actions at each separated camera view. A common feature space is then constructed to keep view-invariant features among extracted streams. This is carried out by learning a set of linear transformations that project private features into the common space in which the classes are well distinguished from each other. To this end, we first adopt Multi-view Discriminant Analysis (MvDA). The original MvDA suffers from odd situations in which the most class-discrepant common space could not be found because its objective is overly concentrated on pushing classes from the global mean but unaware of the distance between specific pairs of adjoining classes. We then introduce a pairwise-covariance maximizing extension that takes pairwise distances between classes into account, namely pc-MvDA. The novel method also differs in the way that could be more favorably applied for large high-dimensional multi-view datasets. Extensive experimental results on four datasets (IXMAS, MuHAVi, MICAGes, NTU RGB+D) show that pc-MvDA achieves consistent performance gain, especially for harder classes. The code is publicly available for research purpose at https://github.com/inspiros/pcmvda.

Hand gestures have been shown to be an efficient way for human-machine interaction. Existing approaches usually utilize ambient or head/chest-mounted cameras to capture hand images. This paper presents a new way to capture hand gestures using the wrist-worn camera. The wrist-worn device is designed as a watch with an integrated camera that is much easier and comfortable to wear in daily life context. We then collect a dataset of ten hand postures using the designed prototype by ten subjects. In addition, we deploy state-of-the-art lite CNN models (YOLO family, Single Shot Detector-SSD) as posture detectors and classifiers. Experimental results show that with limited camera angles, the postures are highly distinctive and easily discriminated with the highest performance of 98.85% and 97.40% in terms of precision and recall, which motivates a wide range of applications and new research directions for human-machine interaction, wearables, the Internet of Things (IoT) and so on.

Vehicle to Vehicle (V2V) and Vehicles to Infrastructure (V2I) are key communication technologies for intelligent transportation system (ITS) to exchange the transparent information such as GPS positions, velocities, traffic status and environment conditions between vehicles as well as between vehicles and infrastructure on the road. These technologies support preventing accidents by releasing a warning as well as reducing traffic congestion. In this paper, design and implementation of a small-size WiFi-based prototype of a V2V-V2I communication unit for low speed vehicles is presented. WiFi, a popular wireless communication technique, was chosen since it is not only a cost-effective choice, but it is also suited for low-speed vehicles like motorbikes, bikes, tracking UAVs, and ground robots. This work analyzes and implements essential use cases and related algorithms to demonstrate the efficiency of V2V and V2I communication using WiFi technology. The results demonstrate the proposed unit’s technological feasibility in real-world vehicles.

Detecting and understanding interactions between students and teachers in classroom is an important criterion for computer vision-based educational assistive systems. Recently, deep long-range spatial dependencies modeling techniques, such as non-local networks, have been proven to be very effective for such tasks. Yet, regarding global context generation, we analyze that the non-local operation only compares pixels using their values, which cannot pertain to structural information. In this paper, we first extend the non-local module to corporate locality attributes. We further observe that each query is treated uniformly to generate the attention map. Hence, we incorporate distance-wise representations with an efficient implementation into the non-local formulas. The proposed locality and relative distance-aware non-local module is integrated into an object detection architecture namely Libra-RCNN and is evaluated through our experiments on a pre-access hand-raising gesture dataset. Our straightforward modification achieves 0.8% and 2.8% higher performance compared to the baseline Libra-RCNN model, in terms of mAP 0.5 and mAP 0.75 respectively.

Video or image-based people counting in real-time has multiple applications in intelligent transportation, density estimation or class management, and so on. This problem is usually carried out by detecting people using conventional detectors. However, this approach can be failed when people stay in various postures or are occluded by each other. In this paper, we notice that even a main part of human body is occluded, their face and head are still observable. We then propose a method that counts people based on face and head detection and pairing. Instead of deploying only face or head detector, we apply both detectors as in many cases the human does not turn his/her face to camera then head detector takes advantage. Otherwise, face detector produces reliable results. The fact of combining both head and face detection results will lead to duplicated responses for one person. We then propose a simple yet effective alignment technique to pair a face with a head of a person. Subsequently, the remaining heads and faces which are not paired with any other faces or heads will be added to our people counter to increase the true positive rate. We evaluate our proposed method on four datasets (Hollywood, Casablanca, Wider Face, and our own dataset). The experimental results show an improvement of average precision and recall comparing to the original head or face detectors.