Search

Human activity recognition (HAR) plays a vital role in many applications, such as surveillance, in-home monitoring, and health care. Portable radar sensor has been increasingly used in HAR systems in combination with deep learning (DL). However, it is both difficult and time-consuming to obtain a large-scale radar dataset with reliable labels. Insufficient labeled data often limit the generalization of DL models. As a result, the performance of DL models will drop when being applied to a new scenario. In this sense, only labeling a small portion of data in the large-scale radar dataset is more feasible. In this article, we propose a semisupervised transfer learning (TL) algorithm, “joint domain and semantic transfer learning (JDS-TL),” for radar-based HAR, which is composed of two modules: unsupervised domain adaptation (DA) and supervised semantic transfer. By employing a sparsely labeled dataset to train the HAR model, the proposed method alleviates the need of labeling a significantly large number of radar signals. We adopt a public radar micro-Doppler spectrogram dataset including six human activities to evaluate JDS-TL. Experiments show that the proposed JDS-TL is able to recognize the six activities with an average accuracy of 87.6% when there are only 10% instances labeled in the training dataset. Ablation analysis also demonstrates the efficiency of the DA and the semantic transfer modules. [ABSTRACT FROM AUTHOR]

Micro-Doppler spectrograms are a conventional data representation domain for movement recognition such as Human Activity Recognition (HAR) or gesture detection. However, they present the problem of time-frequency resolution trade-offs of Short-Time Fourier Transform (STFT), which may have limitations due to unambiguous Doppler frequency, and the STFT computation may be onerous in constrained embedded environments. We propose in this paper an alternative classification approach based on the radar phase information directly extracted from high-resolution Range Map (RM). This novel approach does not use the aforementioned micro-Doppler processing, and yet achieves equivalent or even superior classification results. This shows a potential advantage for low-latency, real-time applications, or computationally constrained scenarios. The proposed method exploits the Histogram of Oriented Gradients (HOG) algorithm as an effective feature extraction algorithm, specifically its capability to capture the unique shape and patterns present in the wrapped phase domains, such as their contour intensity and distributions. Validation results consistently above 92% demonstrate the effectiveness of this method on two independent datasets of arm gestures and gross-motor activities. These were classified with three algorithms, namely the Nearest Neighbor (NN), the linear Support Vector Machine (SVM), and the Gaussian SVM classifiers using the proposed phase information. Feature fusion of different data domains, e.g. the modulus of the RM fused with the RM phase information, is also investigated and shows classification improvement specifically for the robustness of activity performances, such as the aspect angle and the speed of performance. [ABSTRACT FROM AUTHOR]

This paper presents a framework based on multi-layer bi-LSTM network (bidirectional Long Short-Term Memory) for multimodal sensor fusion to sense and classify daily activities’ patterns and high-risk events such as falls. The data collected in this work are continuous activity streams from FMCW radar and three wearable inertial sensors on the wrist, waist, and ankle. Each activity has a variable duration in the data stream so that the transitions between activities can happen at random times within the stream, without resorting to conventional fixed-duration snapshots. The proposed bi-LSTM implements soft feature fusion between wearable sensors and radar data, as well as two robust hard-fusion methods using the confusion matrices of both sensors. A novel hybrid fusion scheme is then proposed to combine soft and hard fusion to push the classification performances to approximately 96% accuracy in identifying continuous activities and fall events. These fusion schemes implemented with the proposed bi-LSTM network are compared with conventional sliding window approach, and all are validated with realistic “leaving one participant out” (L1PO) method (i.e. testing subjects unknown to the classifier). The developed hybrid-fusion approach is capable of stabilizing the classification performance among different participants in terms of reducing accuracy variance of up to 18.1% and increasing minimum, worst-case accuracy up to 16.2%. [ABSTRACT FROM AUTHOR]

Remote health monitoring is a key component in the future of healthcare with predictive and fall risk estimation applications required in great need and with urgency. Radar, through the exploitation of the micro-Doppler effect, is able to generate signatures that can be classified automatically. In this work, features from two different radar systems operating at C band and K band have been used together co-operatively to classify ten indoor human activities with data from 20 subjects with a support vector machine classifier. Feature selection has been applied to remove redundancies and find a set of salient features for the radar systems, individually and in the fused scenario. Using the aforementioned methods, we show improvements in the classification accuracy for the systems from 75 and 70% for the radar systems individually, up to 89% when fused. [ABSTRACT FROM AUTHOR]

In an era where man-machine interaction increasingly uses remote sensing, gesture recognition through use of the micro-Doppler (mD) effect is an emerging application which has attracted great interest. It is a sensible solution and here the authors show its potential for detecting aperiodic human movements. In this study, the authors classify ten hand gestures with a set of handcrafted features using simulated mD signatures generated from Kinect skeleton data. Data augmentation in the form of synthetic minority oversampling technique has been applied to create synthetic samples and classified with the support vector machine and K-nearest neighbour classifier with classification rate of 71.1 and 51% achieved. Finally, using weights generated by an action pair based one vs. one classification layer improves classification accuracy by 24.7 and 28.4%. [ABSTRACT FROM AUTHOR]

With the increased life expectancy and rise in health conditions related to aging, there is a need for new technologies that can routinely monitor vulnerable people, identify their daily pattern of activities and any anomaly or critical events such as falls. This paper aims to evaluate magnetic and radar sensors as suitable technologies for remote health monitoring purpose, both individually and fusing their information. After experiments and collecting data from 20 volunteers, numerical features has been extracted in both time and frequency domains. In order to analyze and verify the validation of fusion method for different classifiers, a support vector machine with a quadratic kernel, and an artificial neural network with one and multiple hidden layers have been implemented. Furthermore, for both classifiers, feature selection has been performed to obtain salient features. Using this technique along with fusion, both classifiers can detect 10 different activities with an accuracy rate of approximately 96%. In cases where the user is unknown to the classifier, an accuracy of approximately 92% is maintained. [ABSTRACT FROM AUTHOR]

Radar-based human motion recognition is crucial for many applications, such as surveillance, search and rescue operations, smart homes, and assisted living. Continuous human motion recognition in real-living environment is necessary for practical deployment, i.e., classification of a sequence of activities transitioning one into another, rather than individual activities. In this paper, a novel dynamic range-Doppler trajectory (DRDT) method based on the frequency-modulated continuous-wave (FMCW) radar system is proposed to recognize continuous human motions with various conditions emulating real-living environment. This method can separate continuous motions and process them as single events. First, range-Doppler frames consisting of a series of range-Doppler maps are obtained from the backscattered signals. Next, the DRDT is extracted from these frames to monitor human motions in time, range, and Doppler domains in real time. Then, a peak search method is applied to locate and separate each human motion from the DRDT map. Finally, range, Doppler, radar cross section (RCS), and dispersion features are extracted and combined in a multidomain fusion approach as inputs to a machine learning classifier. This achieves accurate and robust recognition even in various conditions of distance, view angle, direction, and individual diversity. Extensive experiments have been conducted to show its feasibility and superiority by obtaining an average accuracy of 91.9% on continuous classification. [ABSTRACT FROM AUTHOR]

Micro-Doppler signatures are extremely valuable in the classification of a wide range of targets. This paper investigates the effects of jamming on the micro-Doppler classification performance and explores a potential deep topology enabling low-bandwidth data fusion between nodes in a multistatic radar network. The topology is based on an array of three independent deep neural networks (DNNs) functioning cooperatively to achieve joint classification. In addition to this, a further DNN is trained to detect the presence of jamming, and from this, it attempts to remedy the degradation effects in the data fusion process. This is applied to the real experimental data gathered with the multistatic radar system, NetRAD, of a human operating with seven combinations of holding a rifle-like object and a heavy backpack that is slung on their shoulders. The resilience of the proposed network is tested by applying synthetic jamming signals into specific radar nodes and observing the networks’ ability to respond to these undesired effects. The results of this are compared with a traditional voting system topology, serving as a convenient baseline for this paper. [ABSTRACT FROM AUTHOR]

This article covers radar signal processing for sensing in the context of assisted living (AL). This is presented through three example applications: human activity recognition (HAR) for activities of daily living (ADL), respiratory disorders, and sleep stages (SSs) classification. The common challenge of classification is discussed within a framework of measurements/preprocessing, feature extraction, and classification algorithms for supervised learning. Then, the specific challenges of the three applications from a signal processing standpoint are detailed in their specific data processing and ad hoc classification strategies. Here, the focus is on recent trends in the field of activity recognition (multidomain, multimodal, and fusion), health-care applications based on vital signs (superresolution techniques), and comments related to outstanding challenges. Finally, this article explores challenges associated with the real-time implementation of signal processing/classification algorithms. [ABSTRACT FROM AUTHOR]

The accurate classification of activity patterns based on radar signatures is still an open problem and is a key to detect anomalous behavior for security and health applications. This paper presents a novel iterative convolutional neural network strategy with an autocorrelation pre-processing instead of the traditional micro-Doppler image pre-processing to classify activities or subjects accurately. The proposed strategy uses an iterative deep learning framework for the automatic definition and extraction of features. This is followed by a traditional supervised learning classifier to label different activities. Using three human subjects and their real motion captured data, 12 000 radar signatures were simulated by varying additive white Gaussian noise. In addition, 6720 experimental radar signatures were captured with a frequency-modulated continuous radar at 5.8 GHz with 400 MHz of instantaneous bandwidth from seven activities using one subject and 4800 signatures from five subjects while walking. The simulated and experimental data were both used to validate our proposed method, with signal–noise ratio varying from −20 to 20 dB and with 88.74% average accuracy at −10 dB and 100% peak accuracy at 15 dB. The proposed iterative convolutional neural networks followed with random forests not only outperform the feature-based methods using micro-Doppler images but also outperform the classification methods using other types of supervised classifiers after our proposed iterative convolutional neural network. [ABSTRACT FROM AUTHOR]

Lameness is a significant problem for performance horses and farmed animals, with severe impact on animal welfare and treatment costs. Lameness is commonly diagnosed through subjective scoring methods performed by trained veterinary clinicians, but automatic methods using suitable sensors would improve efficiency and reliability. In this letter, we propose the use of radar micro-Doppler signatures for contactless and automatic identification of lameness, and present preliminary results for dairy cows, sheep, and horses. These proof-of-concept results are promising, with classification accuracy above 85% for dairy cows, around 92% for horses, and close to 99% for sheep. [ABSTRACT FROM AUTHOR]

This paper investigates the selection of different combinations of features at different multistatic radar nodes, depending on scenario parameters, such as aspect angle to the target and signal-to-noise ratio, and radar parameters, such as dwell time, polarization, and frequency band. Two sets of experimental data collected with the multistatic radar system NetRAD are analyzed for two separate problems, namely the classification of unarmed versus potentially armed multiple personnel, and the personnel recognition of individuals based on walking gait. The results show that the overall classification accuracy can be significantly improved by taking into account feature diversity at each radar node depending on the environmental parameters and target behavior, in comparison with the conventional approach of selecting the same features for all nodes. [ABSTRACT FROM PUBLISHER]

In this letter, we investigate the use of micro-Doppler signatures experimentally recorded by a multistatic radar system to perform recognition of people walking. Three different sets of features are tested, taking into account the impact on the overall classification performance of parameters, such as aspect angle, types of classifier, different values of signal-to-noise ratio, and different ways of exploiting multistatic information. High classification accuracy of above 98% is reported for the most favorable aspect angle, and the benefit of using multistatic data at less favorable angles is discussed. [ABSTRACT FROM PUBLISHER]

In this letter, we present the use of experimental human micro-Doppler signature data gathered by a multistatic radar system to discriminate between unarmed and potentially armed personnel walking along different trajectories. Different ways of extracting suitable features from the spectrograms of the micro-Doppler signatures are discussed, particularly empirical features such as Doppler bandwidth, periodicity, and others, and features extracted from singular value decomposition (SVD) vectors. High classification accuracy of armed versus unarmed personnel (between 90% and 97% depending on the walking trajectory of the people) can be achieved with a single SVD-based feature, in comparison with using four empirical features. The impact on classification performance of different aspect angles and the benefit of combining multistatic information is also evaluated in this letter. [ABSTRACT FROM PUBLISHER]