Search

Radar-based human activity recognition (HAR) provides a non-contact method for many scenarios, such as human-computer interaction, smart security, and advanced surveillance with privacy protection. Feeding radar-preprocessed micro-Doppler signals into a deep learning (DL) network is a promising approach for HAR. Conventional DL algorithms can achieve high performance in terms of accuracy, but the complex network structure causes difficulty for their real-time embedded application. In this study, an efficient network with an attention mechanism is proposed. This network decouples the Doppler and temporal features of radar preprocessed signals according to the feature representation of human activity in the time-frequency domain. The Doppler feature representation is obtained in sequence using the one-dimensional convolutional neural network (1D CNN) following the sliding window. Then, HAR is realized by inputting the Doppler features into the attention-mechanism-based long short-term memory (LSTM) as a time sequence. Moreover, the activity features are effectively enhanced using the averaged cancellation method, which improves the clutter suppression effect under the micro-motion conditions. Compared with the traditional moving target indicator (MTI), the recognition accuracy is improved by about 3.7%. Experiments based on two human activity datasets confirm the superiority of our method compared to traditional methods in terms of expressiveness and computational efficiency. Specifically, our method achieves an accuracy close to 96.9% on both datasets and has a more lightweight network structure compared to algorithms with similar recognition accuracy. The method proposed in this article has great potential for real-time embedded applications of HAR.

The ability to sculpt complex reference waves and probe diverse radiation field patterns have facilitated the rise of metasurface antennas, while there is still a compromise between the required wide operation band and the non-overlapping characteristic of radiation field patterns. Specifically, the current computational image formation process with a classic matched filter and other sparsity-driven algorithms would inevitably face the challenge of a relatively confined scene information sampling ratio and high computational complexity. In this paper, we marry the concepts of a deep convolutional neural network with computational imaging literature. Compared with the current matched filter and compressed sensing reconstruction technique, our proposal could handle a relatively high correlation of measurement modes and low scene sampling ratio. With the delicately trained reconstruction network, point-size objects and more complicated targets can both be quickly and accurately reconstructed. In addition, the unavoidable heavy computation burden and essential large operation frequency band can be effectively mitigated. The simulated experiments with measured radiation field data verify the effectiveness of the proposed method.

ObjectiveWheat, as one of the major global food crops, plays a key role in food production and food supply. Different influencing factors can lead to different types of wheat lodging, e.g., root lodging may be due to improper use of fertilizers. While stem lodging is mostly due to harsh environments, different types of wheat lodging can have different impacts on yield and quality. The aim of this study was to categorize the types of wheat lodging by unmanned aerial vehicle (UAV) image detection and to investigate the effect of UAV flight altitude on the classification performance.MethodsThree UAV flight altitudes (15, 45, and 91 m) were set to acquire images of wheat test fields. The main research methods contained three parts: an automatic segmentation algorithm, wheat classification model selection, and an improved classification model based on EfficientNetV2-C. In the first part, the automatic segmentation algorithm was used to segment the UAV to acquire the wheat test field at three different heights and made it into the training dataset needed for the classification model. The main steps were first to preprocess the original wheat test field images acquired by the UAV through scaling, skew correction, and other methods to save computation time and improve segmentation accuracy. Subsequently, the pre-processed image information was analyzed, and the green part of the image was extracted using the super green algorithm, which was binarized and combined with the edge contour extraction algorithm to remove the redundant part of the image to extract the region of interest, so that the image was segmented for the first time. Finally, the idea of accumulating pixels to find sudden value added was used to find the segmentation coordinates of two different sizes of wheat test field in the image, and the region of interest of the wheat test field was segmented into a long rectangle and a short rectangle test field twice, so as to obtain the structural parameters of different sizes of wheat test field and then to generate the dataset of different heights. In the second part, four machine learning classification models of support vector machine (SVM), K nearest neighbor (KNN), decision tree (DT), and naive bayes (NB), and two deep learning classification models (ResNet101 and EfficientNetV2) were selected. Under the unimproved condition, six classification models were utilized to classify the images collected from three UAVs at different flight altitudes, respectively, and the optimal classification model was selected for improvement. In the third part, an improved model, EfficientNetV2-C, with EfficientNetV2 as the base model, was proposed to classify and recognized the lodging type of wheat in test field images. The main improvement points were attention mechanism improvement and loss function improvement. The attention mechanism was to replace the original model squeeze and excitation (SE) with coordinate attention (CA), which was able to embed the position information into the channel attention, aggregate the features along the width and height directions, respectively, during feature extraction, and capture the long-distance correlation in the width direction while retaining the long-distance correlation in the length direction, accurate location information, enhancing the feature extraction capability of the network in space. The loss function was replaced by class-balanced focal loss (CB-Focal Loss), which could assign different loss weights according to the number of valid samples in each class when targeting unbalanced datasets, effectively solving the impact of data imbalance on the classification accuracy of the model.Results and DiscussionsFour machine learning classification results: SVM average classification accuracy was 81.95%, DT average classification accuracy was 79.56%, KNN average classification accuracy was 59.32%, and NB average classification accuracy was 59.48%. The average classification accuracy of the two deep learning models, ResNet101 and EfficientNetV2, was 78.04%, and the average classification accuracy of ResNet101 was 81.61%. Comparing the above six classification models, the EfficientNetV2 classification model performed optimally at all heights. And the improved EfficientNetV2-C had an average accuracy of 90.59%, which was 8.98% higher compared to the average accuracy of EfficientNetV2. The SVM classification accuracies of UAVs at three flight altitudes of 15, 45, and 91 m were 81.33%, 83.57%, and 81.00%, respectively, in which the accuracy was the highest when the altitude was 45 m, and the classification results of the SVM model values were similar to each other, which indicated that the imbalance of the input data categories would not affect the model's classification effect, and the SVM classification model was able to solve the problem of high dimensionality of the data efficiently and had a good performance for small and medium-sized data sets. The SVM classification model could effectively solve the problem of the high dimensionality of data and had a better classification effect on small and medium-sized datasets. For the deep learning classification model, however, as the flight altitude increases from 15 to 91 m, the classification performance of the deep learning model decreased due to the loss of image feature information. Among them, the classification accuracy of ResNet101 decreased from 81.57% to 78.04%, the classification accuracy of EfficientNetV2 decreased from 84.40% to 81.61%, and the classification accuracy of EfficientNetV2-C decreased from 97.65% to 90.59%. The classification accuracy of EfficientNetV2-C at each of the three altitudes. The difference between the values of precision, recall, and F1-Score results of classification was small, which indicated that the improved model in this study could effectively solve the problems of unbalanced model classification results and poor classification effect caused by data imbalance.ConclusionsThe improved EfficientNetV2-C achieved high accuracy in wheat lodging type detection, which provides a new solution for wheat lodging early warning and crop management and is of great significance for improving wheat production efficiency and sustainable agricultural development.

Machine vision has been increasingly used for agricultural sensing tasks. The detection method based on deep learning for infield corn kernels can improve the detection accuracy. In order to obtain the number of lost corn kernels quickly and accurately after the corn harvest, and evaluate the corn harvest combine performance on grain loss, the method of directly using deep learning technology to count corn kernels in the field was developed and evaluated. Firstly, an RGB camera was used to collect image with different backgrounds and illuminations, and the datasets were generated. Secondly, different target detection networks for kernel recognition were constructed, including Mask R-CNN, EfficientDet-D5, YOLOv5-L and YOLOX-L, and the collected 420 effective images were used to train, verify and test each model. The number of images in train, verify and test datasets were 200, 40 and 180, respectively. Finally, the counting performances of different models were evaluated and compared according to the recognition results of test set images. The experimental results showed that among the four models, YOLOv5-L had overall the best performance, and could reliably identify corn kernels under different scenes and light conditions. The average precision (AP) value of the model for the image detection of the test set was 78.3%, and the size of the model was 89.3 MB. The correct rate of kernel count detection in four scenes of non-occlusion, surface mid-level-occlusion, surface severe-occlusion and aggregation were 98.2%, 95.5%, 76.1% and 83.3%, respectively, and F1 values were 94.7%, 93.8%, 82.8% and 87%, respectively. The overall detection correct rate and F1 value of the test set were 90.7% and 91.1%, respectively. The frame rate was 55.55 f/s, and the detection and counting performance were better than Mask R-CNN, EfficientDet-D5 and YOLOX-L networks. The detection accuracy was improved by about 5% compared with the second best performance of Mask R-CNN. With good precision, high throughput, and proven generalization, YOLOv5-L can realize real-time monitoring of corn harvest loss in practical operation.

Active deception jamming recognition has gained significant attention as a crucial aspect of modern electronic warfare, and a large quantity of jamming recognition methods based on either artificial or deep learning methods have been proposed to date. In actual complex battlefields, the abundant deceptive jamming signals are extremely difficult to obtain and the deceptive jamming signals leveraged by the adversarial jammers are often significantly influenced by noise. To address these challenges a Siamese squeeze wavelet attention network (SSWAN) for radar active deception jamming recognition method is proposed. By constructing a wavelet attention module (WAM), the fine‐grained time‐frequency texture features of echo and jamming signals are preferably extracted even under prohibitively strong noise scenarios. Specifically, the Siamese structure is employed as the main backbone network to measure the similarity between jamming signals and make full use of the training samples; besides, the squeeze learning module is embedded to maintain lightweight and prevent overfitting. Experimental results demonstrate that at a jamming‐to‐noise ratio (JNR) of −8 dB, the proposed method achieves recognition accuracies above 96.3% for 6‐class and multi‐class combinational active deceptive jamming types. Overall, compared to mainstream deep networks, the proposed method exhibits superior advantages in lower JNR ratio and small samples of actual battlefield scenarios. [ABSTRACT FROM AUTHOR]

The millimeter-wave frequency-diverse imaging regime has recently received considerable attention in both the security screening and synthetic aperture radar imaging literature. Considering that the minor systematic errors and alignment errors could still produce heavily corrupted images, these complex-based imaging reconstructions rely heavily on the precise measurement of both phase and amplitude of radiation field patterns and echo signals. In the literature, it is shown that by leveraging phase-retrieval techniques, salient reconstruction images can still be acquired, even in the presence of significant phase errors, which could ease the phase error calibration pressure to a large extent in practical imaging applications. In this paper, in the regime of phaseless frequency-diverse imaging, with the powerful feature inference and generation power of unsupervised generative models, an end-to-end deep prior generative neural network is designed to achieve near real-time imaging. The harsh imaging reconstruction with both the high radiation mode correlations and extremely low scene compression sampling ratio, which are extremely troublesome to tackle for generally applied matched-filter and compressed sensing approach in the current frequency-diverse imaging literature, can still be preferably handled with our reconstruction network. The well-trained reconstruction network is constituted by prior inference and deep generative modules with excellent generative capabilities and significant prior inference abilities. Using simulation experiments with radiation field data, we verify that the integration of phase-free frequency-change imaging with deep learning networks can effectively improve reconstruction capabilities and improve robustness to systematic phase errors. Compared with existing imaging methods, our imaging method has high imaging performance and can even reconstruct targets under low compression ratio conditions, which is somewhat competitive with current state-of-the-art algorithms. Moreover, we find that the proposed method has good anti-noise and stability. [ABSTRACT FROM AUTHOR]

Orchard machinery autonomous navigation is helpful for improving the efficiency of fruit production and reducing labor costs. Path planning is one of the core technologies of autonomous navigation for orchard machinery. As normally planted in straight and parallel rows, fruit trees are natural landmarks that can provide suitable cues for orchard intelligent machinery. This paper presents a novel method to realize path planning based on computer vision technologies. We combine deep learning and the least-square (DL-LS) algorithm to carry out a new navigation line extraction algorithm for orchard scenarios. First, a large number of actual orchard images are collected and processed for training the YOLO V3 model. After the training, the mean average precision (MAP) of the model for trunk and tree detection can reach 92.11%. Secondly, the reference point coordinates of the fruit trees are calculated with the coordinates of the bounding box of trunks. Thirdly, the reference lines of fruit trees growing on both sides are fitted by the least-square method and the navigation line for the orchard machinery is determined by the two reference lines. Experimental results show that the trained YOLO V3 network can identify the tree trunk and the fruit tree accurately and that the new navigation line of fruit tree rows can be extracted effectively. The accuracy of orchard centerline extraction is 90.00%. [ABSTRACT FROM AUTHOR]

The number of wheat spikelets is an important phenotypic trait and can be used to assess the grain yield of the wheat crop. However, manual counting of spikelets is time-consuming and labor-intensive. To develop a cost-effective and highly efficient phenotyping system for counting the number of spikelets under laboratory conditions, methods based on imaging processing techniques and deep learning were proposed to accurately detect and count spikelets from color images of wheat spikes captured at the grain filling stage. An unsupervised learning-based method was first developed to automatically detect and label spikelets from spike color images and build the datasets for the model training. Based on the constructed datasets, a deep convolutional neural network model was retrained using transfer learning to detect the spikelets. Testing results showed that the root mean squared errors, relative root mean squared errors, and the coefficients of determination between the automatic and manual counted spikelets for four wheat lines were 0.62, 0.58, 0.54, and 0.77; 3.96, 3.73, 3.34, and 4.94%; and 0.73, 0.78, 0.84, and 0.67, respectively. We demonstrated that the proposed methods can effectively estimate the number of wheat spikelets, which improves the counting efficiency of wheat spikelets and contributes to the analysis of the developmental characteristics of wheat spikes. [ABSTRACT FROM AUTHOR]

Traditional Artificial Intelligence (AI) based fault detection approaches need a large amount of data for the model learning. However, in a real-world system, it is very difficult and expensive to obtain massive labeled fault data. In addition, the working conditions of a motor are usually variable, conventional fault diagnosis models with weak generalization ability can only be used for fault detection under constant working condition. The performance of traditional AI based approaches decreases when the working condition changes. To this end, a novel deep Gaussian process (GP) kernel transfer based few-shot learning method (RNGPT) is proposed in this paper for the fault detection of electric machines. First, a deep residual network (ResNet) is used to extract the features of the raw data. Then, the encoded latent feature vector is fed into the GP with kernel transfer ability to make the motor fault detection and classification. The proposed method uses much less data than the traditional AI based method to achieve fault diagnosis under variable working condition, and does not cause an overfitting problem. Experimental results of two case studies demonstrate that the proposed RNGPT model can accurately and effectively detect motor faults with limited labeled data under different working conditions. Experimental results of RNGPT with radial basis function (RBF) kernel model on simulation data present that the fault detection accuracy of the proposed method is about 16% higher than the conventional deep learning methods, 6% higher than other few-shot learning based methods in 5-shot and 4% higher in 1-shot. Finally, experimental on a real-world dataset, the RNGPT-RBF model still has the highest fault diagnosis accuracy in 5-shot (99.39 $ \pm $ 0.09%) and 1-shot (98.55 $ \pm $ 0.16%). [ABSTRACT FROM AUTHOR]

Accurate real-time water quality prediction is of great significance for local environmental managers to deal with upcoming events and emergencies to develop best management practices. In this study, the performances in real-time water quality forecasting based on different deep learning (DL) models with different input data pre-processing methods were compared. There were three popular DL models concerned, including the convolutional neural network (CNN), long short-term memory neural network (LSTM), and hybrid CNN–LSTM. Two types of input data were applied, including the original one-dimensional time series and the two-dimensional grey image based on the complete ensemble empirical mode decomposition algorithm with adaptive noise (CEEMDAN) decomposition. Each type of input data was used in each DL model to forecast the real-time monitoring water quality parameters of dissolved oxygen (DO) and total nitrogen (TN). The results showed that (1) the performances of CNN–LSTM were superior to the standalone model CNN and LSTM; (2) the models used CEEMDAN-based input data performed much better than the models used the original input data, while the improvements for non-periodic parameter TN were much greater than that for periodic parameter DO; and (3) the model accuracies gradually decreased with the increase of prediction steps, while the original input data decayed faster than the CEEMDAN-based input data and the non-periodic parameter TN decayed faster than the periodic parameter DO. Overall, the input data preprocessed by the CEEMDAN method could effectively improve the forecasting performances of deep learning models, and this improvement was especially significant for non-periodic parameters of TN. [ABSTRACT FROM AUTHOR]

Machine vision has been increasingly used to address agricultural issues. One such case is corn field harvest losses and image-based object detection approaches, namely image processing, machine learning, and deep learning were investigated to detect and count infield corn kernels, immediately after harvest for combine harvester performance evaluation. A hand-held low-cost RGB camera was used to collect images with kernels of different backgrounds, based on which a 420 images dataset (200, 40, and 180 for training, validation, and testing, respectively) was generated. Three different models for kernel detection were constructed based on image processing, machine learning, and deep learning. For the imaging processing method, the images were preprocessed (color thresholding, graying, and erosion), followed by Hough circle detection to identify kernels. For the machine learning (cascade detector) and deep learning (Mask R-CNN, EfficientDet, YOLOv5, and YOLOX), models were trained, validated, and tested. Experimental results showed the overall performance of the deep learning network YOLOv5 was superior to the other approaches, with a small model size (89.3 MB) and a high model average precision (78.3 %) for object detection. The detection accuracy, undetection rate and F 1 value were 90.7 %, 9.3 %, and 91.1 %, respectively, and the average detection rate was 55 fps. This study demonstrates that the YOLOv5 model has the potential to be used as a real-time, reliable, and robust method for infield corn kernel detection. [ABSTRACT FROM AUTHOR]

• Comprehensive wheat lodging analysis in terms of ratio, location, and angle was conducted. • Auto crop plot dataset generation method was developed. • Imbalanced data challenge was addressed by changing the conventional loss function. • EfficientNet-B7 produced exceptional performance in identifying and categorizing wheat lodging. • LRCN gave a high performance for the classification of concatenated dataset. Crop lodging in agricultural fields is one of the major factors that limit cereal crop yields. Wheat, the most popular cereal crop in most countries, is also affected by this phenomenon, which may result in a significant decrease in both yield and quality. Therefore, addressing wheat lodging is crucial for producers. This study aims to detect and identify wheat lodging through aerial images and classify its severity based on ratio, angle and location of the lodging. To achieve this goal, a multi-task approach was proposed involving three phases. First, automatic dataset generation methodology was conducted on orthomosaic imagery of three dates. Next, a comprehensive assessment of wheat lodging (ratio, angle and location) was performed, which has received little research attention. Third, applying and improving selected classification models for classifying image datasets was conducted. Combining convolutional neural networks and temporal sequences in a single model provided an opportunity to use spatiotemporal information extracted from the wheat image datasets. Time dependency and individual dates were both considered in the classification task. The limited number of data and imbalanced classes challenges, resulting from real field conditions data collection, were overcome by applying a new loss function to the classifier models. The overall accuracy of wheat lodging classification reached over 91% in these two states using the proposed approach. Based on this research, wheat lodging was detected more accurately by the proposed models despite the small and imbalanced dataset. The developed methodology paves the way for comprehensive and automatic wheat lodging detection, and the methodology can be adapted for similar crops that suffer lodging issues with suitable modifications. [ABSTRACT FROM AUTHOR]

• We propose a GAN-based method for image small-sample augmentation named WGAN-CL. • WGAN-CL designs shortcut-stream connections to broaden the model's solution space. • We design a confidence loss to improve the model's learning capability. • Experiments achieve state-of-the-art performance in image quality and diversity. The small-sample task is a current challenge in the field of deep learning, due to the huge annotation cost and the inherent limitations of targets, such as the acquisition of rare animal and plant images. Data augmentation is an effective method to solve the semantic sparseness and overfitting of deep convolution neural network in small-sample classification, but its effectiveness remains to be improved. We propose a Wasserstein GAN with confidence loss (WGAN-CL) to implement the expansion of small-sample plant dataset. Firstly, a shallower GAN's structure is designed to adapt to less plant data. Meanwhile, shortcut-stream connections are brought into the basic network to enlarge the solution space of the model without producing additional training parameters. Secondly, the Wasserstein distance combined with confidence loss is used for optimizing the model. Experiments demonstrate that the Wasserstein distance with gradient penalty guarantees the stability of model training and the diversity of outputs. And the sample screening strategy based on confidence loss can ensure that the generated image is close to the real image in semantic features, which is critical for subsequent image classification. To verify the effectiveness of the WGAN-CL in plant small-sample augmentation, 2000 flower images of 5 categories in the "Flowers" dataset are utilized as training samples, while 2000 augmented images are employed for model training as well to improve the performance of a classical classifier. WGAN-CL has a significant performance improvement over state-of-the-art technologies, i.e., a 2.2% improvement in recall and a 2% improvement in F1-score. Experiments on the "Plant Leaves" dataset also achieved excellent results demonstrating that WGAN-CL can be migrated to other tasks. WGAN-CL uses less computational resources while considering both effectiveness and robustness, proved the practicality of our model. [ABSTRACT FROM AUTHOR]

• Designed an Expert System for high throughput soybean pods phenotype investigation. • A novel method for synthesizing training datasets automatically. • A hybrid sim/real dataset designed for soybean pods instance segmentation. • Significant improvement on accuracy of phenotype by segmented-mask based method. The segmentation of individual soybean pod is the prerequisite step for obtaining phenotypic traits such as pod length, width, and the number of seeds per pod. Nevertheless, handcrafted features-based image methods are not robust and practical in segmenting soybean pods with several morphological peculiarities in varying degrees. Although deep learning-based algorithms can achieve accurate training and strong generalization capabilities, it requires massive hand-labeled datasets, especially in the agricultural realm. Hence, we present a novel image synthesis method for rapidly generating a high throughput soybean pods dataset with plenty of raw images and mask images, where the soybean pods are densely sampled for simulating frequently physically touching. Then, we train a variant of Mask R-CNN on our synthetic dataset without any manual-labeled data and evaluate the trained model on our synthetic test images dataset and a real-world test image dataset of densely-cluttered soybean pods with Average Precision and Average Recall. Finally, a new segmented mask-based method is proposed for calculating the pod length and width and validation is carried out. The experimental results show that the proposed expert system could be used to quickly and efficiently segment and calculate the morphological parameters of each pod and it is practical to use this approach for high-throughput object instance segmentation and measurement. [ABSTRACT FROM AUTHOR]