Search

The number of pollen-bearing honey bees serves as a vital indicator for assessing colony balance and health. Despite its significance, prevailing detection techniques still rely heavily on manual observation and annotation, leading to time-consuming processes that cannot sustain long-term, continuous monitoring efforts. To facilitate automatic beehive monitoring, this study introduces an efficient method for pollen-bearing bee detection. Initially, we furnish a comprehensive dataset, dubbed VnPollenBee, meticulously annotated for pollen-bearing honey bee detection and classification. The dataset comprises 60,826 annotated boxes that delineate both pollen-bearing and non-pollen-bearing bees in 2051 images captured at the entrances of beehives under various environmental conditions. To the best of our knowledge, this represents the first dedicated dataset for pollen-bearing bee detection. The VnPollenBee dataset is publicly accessible to the research community at https://comvis-hust.github.io/datasets/pollenbee.html. Subsequently, we propose the incorporation of diverse techniques into two baseline models, namely YOLOv5 and Faster RCNN, to effectively address the imbalance that arises during the detection of pollen-bearing bees due to their number being typically much lower than the total number of bees present at hive entrances. The experimental results demonstrate that our proposed method outperforms the baseline models on the VnPollenBee dataset, yielding Precision, Recall, and F1 score of 99%, 93%, and 95%, respectively. Specifically, the improvements obtained are 3% and 2% in Recall and F1 score when using YOLOv5, and 3%, 2%, and 2% in Precision, Recall, and F1 score when using Faster RCNN. These findings confirm the potential of our approach to facilitate bee foraging behavior analysis and automated bee monitoring.

This study aims at developing a computer-aided diagnostic system based on deep learning techniques for detecting various typical lesions during endoscopic examinations in the human gastrointestinal tract. We propose a lesion detection model, namely called GIFCOS-DT, that is built upon a one-stage backbone for object detection (Fully Convolutional One-Stage Object Detection - FCOS). For the proposed model, to deal with the diverse shapes and appearance of the lesions, we introduce a new loss function based on Distance Transform, that better describes the elongated or curved shapes of lesions than the common loss functions like Intersection of Union or centroid loss. We then deploy the detection model on an embedded device that connects to the endoscopic machine to assist endoscopists during examinations. A multithread technique is employed to accelerate the processing times of all steps of the system. Extensive experiments have been conducted on two challenging datasets, the benchmark dataset (Kvasir-SEG) and our newly collected dataset (IGH_GIEndoLesion-SEG), which include various typical lesions of the gastrointestinal (GI) tract (reflux esophagitis, esophageal cancer, helicobacter pylori negative gastritis, helicobacter pylori positive gastritis, gastric cancer, duodenal ulcer, and colorectal polyps). Experimental results show that our proposed methods outperform the original FCOS by 4.2% and 7.2% on Kvasir-SEG and our collected dataset respectively in terms of the average $AP_{50}$ score. On the Kvasir-SEG dataset, the GIFCOS-DT outperforms state-of-the-art detectors such as Faster R-CNN, DETR, YOLOv3, and YOLOv4. Our developed supporting system for lesion detection can run at 14.85 FPS on an embedded Jetson AGX Xavier or 31.92 FPS on an RTX 3090. The detection results of various types of lesions are promising, mostly on malignant lesions such as gastric cancers. The proposed system can be deployed as an assistant tool in endoscopy to reduce missed detection of lesions. Our code is available at https://github.com/hanhtran201/GIFCOS-DT.

Automatic analysis of medical images and endoscopic images, in particular, is an attractive research topic in recent years. To achieve this goal, many tasks must be conducted for example lesion detection, segmentation, and classification. However, existing methods for such problems still face challenges due to the spreading characteristic of the lesions as well as some artifacts caused by motion, specularities, low contrast, bubbles, debris, bodily fluid, and blood. As a consequence, single segmentation or detection could not deal with those issues. Besides, although deep learning has achieved impressive results in many tasks, lacking a large annotated dataset could lead to overfitting problems. In this paper, we tackle these issues by taking particular characteristics of lesions of interest and advantages of deep segmentation models into account. We propose a dual-path framework (namely, DCS-UNet) that combines both segmentation and classification. We first segment lesions from the image using U-Net architecture with different encoder backbones (ResNet-50, VGG-16, DenseNet-201). The segmented regions will be then refined in the second path where we classify every patch in the whole image or the Inner or Outer regions extended from the contours given by the segmentation results. For the refinement scheme, we utilize the color-dependent and texture features as doctors’ advice and deploy the support vector machine (SVM) technique to classify a patch into disease region or not. Extensive experiments were conducted on three datasets of gastroesophageal reflux disease (GERD) endoscopic images as GradeM, GradeA, GradeB, which are defined as modified Los Angeles classification. The experimental results show the improvements of the proposed method compared to the single U-Net or a segmentation using the hand-designed features-based scheme. The proposed method improves mDice and mIoU by 0.5% and 0.36% on GradeA dataset, which is the most challenging dataset with ambiguous separations of GERD vs. normal regions, and by 0.82% and 0.81% on GradeM dataset, which is clearer to segment GERD regions. This proposed framework shows the potential to develop a diagnosis assistance system to help endoscopists reduce burden in examining GERD in the future.

Hand-raising gesture is one of the most popular signs of communication, whose frequency is related to the classroom’s atmosphere, the attractiveness of the subject, and the level of interactions between students and teachers. However, automatic hand-raising gesture detection and recognition remains still a challenging problem mainly due to low hand resolution, hand occlusion, various backgrounds, and viewpoint changes. While majority of the existing methods focus on static hand-raising posture detection, in this paper, we propose a framework for dynamic hand gesture recognition from classroom videos consisting of two main stages: hand posture detection and dynamic hand gesture recognition. In hand posture detection stage, we extend the previous work by adding relative position-aware in non-local network. After detecting the hand-raising posture on static pixels, which in turn accelerates the performance images, at the dynamic hand gesture recognition stage, we incorporate object detection and tracking to associate the hand-raising detection results at consecutive frames to supplement the missing detection due to the occlusion issue and obtain hand-raising gesture recognition at the event level. The experimental results show that the proposed method outperforms three benchmark models for static hand posture detection that are Faster-RCNN, Libra-RCNN, and Libra-RCNN+RDA on our dataset with the gains obtained for mAP being 7.68%, 5.76%, and 0.35% higher, respectively. In terms of event-level recognition results, the proposed method achieves the value of frame-wise accuracy, temporal_IoU, F1-score@0.3, and Levenshtein-score are 90.0%, 84.4%, 83.2%, and 84.3%, respectively. The code and dataset used in the paper will be made publicity available for research community.

In this paper, we present an unified framework for understanding hand action from the first-person video. The proposed framework composes two main components. The first component estimates three-dimensional (3D) hand joints from RGB images. Two network structures derived from the baseline HopeNet network are proposed: convolutional neural networks (CNNs) which are traditional multi-layer CNN and CNN combining with GraphCNN to perform 3D hand pose estimation, without the use of GraphUNet as in baseline HopeNet method. The second component of the framework recognizes hand action from skeleton stream. We first deploy two recent advanced neuronal networks that are PA-ResGCN and Double-feature Double-motion (DDNet). To focus more on the hand pose changes, we improve DDNet with two normalization strategies of the hand joints. Finally, we fuse PA-ResGCN with our improved DDNet to still boost the recognition performance. We evaluate our proposed methods on First-Person Hand Action Benchmark dataset. Experiments show that our model for 3D hand joints estimation achieves the best precision (36.6 mm). Our hand joint normalization strategies improve the original DDNet from 0.71% to 4.05% of accuracy with the ground-truth hand pose while the improvement is significantly larger (from 2.96% to 10.98%) with the estimated hand pose. The late fusion schemes outperform different state-of-the-art methods for the hand action recognition with the highest accuracy of 86.67%. These experimental results show potential and extendable possibilities for developing practical first-person vision applications.

In this work, we study the ability to use hand gestures for human-machine interaction from wrist-worn sensors. Towards this goal, we design a wrist-worn prototype to capture RGB video stream of hand gestures. Then we built a new wrist-worn gesture dataset (named WiGes) with various subjects in interaction with home appliances in different environments. To the best of our knowledge, this is the first benchmark released for studying hand gestures from a wrist-worn camera. We then evaluate various CNN models for vision-based recognition. Furthermore, we deeply analyze the models that produce the best trade-off between accuracy, memory requirement, and computational cost. We point out that among studied architectures, MoviNet produces the highest accuracy. Then, we introduce a new MoviNet-based two-stream architecture that takes both RGB and optical flow into account. Our proposed architecture increases the Top-1 accuracy by 1.36% and 3.67% according to two evaluation protocols. Our dataset, baselines, and proposed model analysis give instructive recommendations for human-machine interaction using hand-held devices.

Abstract Skeleton‐based human action recognition has emerged recently thanks to its compactness and robustness to appearance variations. Although impressive results have been obtained in recent years, the performance of skeleton‐based action recognition methods has to be improved to be deployed in real‐time applications. Recently, a lightweight network structure named Double‐feature Double‐motion Network (DD‐Net) has been proposed for the skeleton‐based human action recognition. With high speed, the DD‐Net achieves state‐of‐the‐art performance on hand and body actions. The DD‐Net could not distinguish actions if they have a weak connection with the global trajectories. However, the DD‐Net is suitable for human action recognition where actions strongly correlate to the global trajectories. In this paper, the authors propose TD‐Net, an improved version of the DD‐Net in which a new branch is added. The new branch takes the normalised coordinates of joints (NCJ) to enrich the spatial information. On five datasets for skeleton‐based human activity recognition that are MSR‐Action3D, CMDFall, JHMDB, FPHAB, and NTU RGB + D, the TD‐Net consistently obtains superior performance compared with the baseline model DD‐Net. The proposed method outperforms different state‐of‐the‐art methods, including both hand‐designed and deep learning‐based methods on four datasets (MSR‐Action3D, CMDFall, JHMDB, and FPHAB). Furthermore, the generalisation of the proposed method is confirmed through cross‐dataset evaluation. To illustrate the potential use of the model for real‐time human action recognition, the authors have deployed an application on an edge device. The experimental result shows that the application can process up to 40 fps for pose estimation using MediaPipe. It takes only 0.04 ms to recognise an action from skeleton sequences.

Human action recognition (HAR) under different camera viewpoints is the most critical requirement for practical deployment. In this paper, we propose a novel method that leverages successful deep learning-based features for action representation and multi-view analysis to accomplish robust HAR under viewpoint changes. Specifically, we investigate various deep learning techniques, from 2D CNNs to 3D CNNs to capture spatial and temporal characteristics of actions at each separated camera view. A common feature space is then constructed to keep view-invariant features among extracted streams. This is carried out by learning a set of linear transformations that project private features into the common space in which the classes are well distinguished from each other. To this end, we first adopt Multi-view Discriminant Analysis (MvDA). The original MvDA suffers from odd situations in which the most class-discrepant common space could not be found because its objective is overly concentrated on pushing classes from the global mean but unaware of the distance between specific pairs of adjoining classes. We then introduce a pairwise-covariance maximizing extension that takes pairwise distances between classes into account, namely pc-MvDA. The novel method also differs in the way that could be more favorably applied for large high-dimensional multi-view datasets. Extensive experimental results on four datasets (IXMAS, MuHAVi, MICAGes, NTU RGB+D) show that pc-MvDA achieves consistent performance gain, especially for harder classes. The code is publicly available for research purpose at https://github.com/inspiros/pcmvda.

Recently, skeleton-based gesture and action recognition have emerged thanks to the progress in human pose estimation. Gesture representation using skeletal data is robust since skeletal data are invariant to the individual’s appearance. Among different approaches proposed for skeleton-based action/gesture recognition, Graph Convolutional Network (GCN) and its variations have obtained great attention thanks to its ability to capture the graph essence of the skeletal data. In this paper, we aim to design an efficient scheme using relative joints of skeleton sequences adapted in a GCN framework. Both spatial features (i.e., joint positions) and temporal ones (i.e., the velocity of joints) are combined to form the input of Attention-enhanced Adaptive GCN (AAGCN). The proposed framework deals with limitations of the original AAGCN when it works on challenging datasets with incomplete and noisy skeletal data. Extensive experiments are carried out on three datasets CMDFALL, MICA-Action3D, NTU-RGBD. Experimental results show that the proposed method achieves superior performance compared with existing methods. Moreover, to illustrate the application of the proposed method in real-time traffic control gesture recognition for autonomous vehicles, we have evaluated the proposed method on the TCG dataset. The obtained results show that the proposed method offers real-time computation capability and good recognition results. These results suggest a promising solution to deploy a real-time and robust recognition technique for gesture-based traffic control in autonomous vehicles.

Background: Newborns commonly experience pain due to a variety of reasons. Non-nutritive sucking (NNS) is thought to be an effective non-pharmacological method of pain-relief. However, the significant heterogeneity in some systematic reviews limited the certainty of the findings about NNS. Hence, this study was aimed at evaluating the analgesic effect and safety of NNS in healthy term neonates during the heel prick test, using the Neonatal Pain Agitation and Sedation Scale (N-PASS). Methods: In this randomized controlled trial, 42 term infants were included: 22 infants in the intervention group (receiving NNS) and 20 infants in the control group (not receiving NNS). In the intervention group, neonates were given a silicone pacifier 120 s before, during, and 120 s after the heel puncture, while neonates in the control group received routine care. Pain-related outcomes were assessed using the N-PASS at 30s, 60s, 90s, and 120s after the heel prick. Any adverse events during the procedure were noted. The Spearman correlation coefficients between the pain scores on the N-PASS and two other scales (NFCS and NIPS) were calculated at all four moments of evaluation. Results: The mean N-PASS pain scores at 30s, 60s, 90s, and 120s after heel prick were significantly lower in the NNS group than in the control group: 4.73 ± 2.78 vs. 7.90 ± 1.52 (p = 0.0002); 3.64 ± 3.06 vs. 5.55 ± 2.95 (p = 0.052); 2.59 ± 3.08 vs. 5.25 ± 3.51 (p = 0.011); and 2.05 ± 2.94 vs. 4.90 ± 3.99 (p = 0.013), respectively. No adverse events were detected in either group. Our findings revealed a strong positive correlation between the pain scores on the N-PASS and two other scales (NFCS and NIPS), which was statistically significant (p

Inspection of rice seeds is a crucial task for plant nurseries and farmers since it ensures seed quality when growing seedlings. Conventionally, this process is performed by expert inspectors who manually screen large samples of rice seeds to identify their species and assess the cleanness of the batch. In the quest to automate the screening process through machine vision, a variety of approaches utilise appearance-based features extracted from RGB images while others utilise the spectral information acquired using Hyperspectral Imaging (HSI) systems. Most of the literature on this topic benchmarks the performance of new discrimination models using only a small number of species. Hence, it is unclear whether or not model performance variance confirms the effectiveness of proposed algorithms and features, or if it can be simply attributed to the inter-class/intra-class variations of the dataset itself. In this paper, a novel method to automatically screen and classify rice seed samples is proposed using a combination of spatial and spectral features, extracted from high resolution RGB and hyperspectral images. The proposed system is evaluated using a large dataset of 8,640 rice seeds sampled from a variety of 90 different species. The dataset is made publicly available to facilitate robust comparison and benchmarking of other existing and newly proposed techniques going forward. The proposed algorithm is evaluated on this large dataset and the experimental results show the effectiveness of the algorithm to eliminate impure species by combining spatial features extracted from high spatial resolution images and spectral features from hyperspectral data cubes.

Recently, the recent advancement of deep learning with the capacity to perform automatic high-level feature extraction has achieved promising performance for sensor-based human activity recognition (HAR). Among different deep learning methods, Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) have been widely adopted. However, scalar outputs and pooling in CNN only allow to get the invariance but not the equivariance. The capsule networks (CapsNet) with the vector output and routing by agreement is able to capture the equivariance. In this paper, we propose a method for recognizing human activity from wearable sensors based on a capsule network named SensCapsNet. The architecture of SensCapsNet is designed to be suitable for spatial-temporal data coming from wearable sensors. Experimental results show that the proposed network outperforms CNN and LSTM methods. The performance of the proposed CapsNet architecture is assessed by altering dynamic routing between capsule layers. The proposed SensCapsNet yields improved accuracy values of 77.7% and 70.5% for 1 routing on two testing datasets in comparison with the baseline methods based on CNN and LSTM that yields the F1-score of 67.7% and 69.2% for the first dataset and 65.3% and 67.6% for the second dataset respectively. Moreover, even several human activity datasets are available, privacy invasion and obtrusive concerns have not been carefully taken in to consideration in dataset building. Toward to build a non-obstructive sensing based human activity recognition method, in this paper, a dataset named 19NonSens is designed and collected from twelve subjects wearing e-Shoes and a smart watch to perform 19 activities under multiple contexts. This dataset will be made publicity available. Finally, thanks to the promising results obtained by the proposed method, we develop a life logging application which achieves a real-time computation and the accuracy rate greater than 80% for 5 common upper body activities.

Unmanned aerial vehicles (UAV) are used to conduct a variety of recognition as well as specific missions such as target tracking and safe landing. The transmitted image sequences to be interpreted at ground station usually face limited requirements of the data transmission. In this paper, on the one hand, we handle a surveillance mission with segmenting an UAV video's content into semantic regions. A multi-class image segmentation approach is proposed considering UAV video-specific characteristics. After post-processing steps on the segmentation results, a support vector-machine classifier is used to recognize regions. A Markov model is introduced to combine the results from the previous frames in order to improve the accuracy. On the other hand, this study also assesses the influences of data reduction techniques on the proposed techniques. The comparisons between the untreated configuration and control conditions under manipulations of the frame rate, spatial resolution, and compression ratio, demonstrate how these data reduction techniques adversely influence the algorithm's performance. The experiments also point out the optimal configuration in order to obtain a trade-off between the target performance and requirements of the data transmission.

Antimicrobial viroblock/polyacrylonitrile nanofiber webs fabricated using the electrospinning method were assessed in terms of the antimicrobial activity against infectious agents as a potential material used in mask production. Viroblock (VB) is an amalgam of lipid vesicle and silver. Lipid vesicle depletes the virus outer membrane, which contains cholesterol, while silver ions penetrate the virus, interact with sulfur-bearing moieties, and possess the virus bactericidal property. VB, having anti-coronavirus and anti-influenza properties, was prepared in four different concentrations, 0.5 wt%, 1 wt%, 1.5 wt%, and 2 wt%, in regard to nanofiber weight. The resultant nanofibers were characterized by scanning electron microscope (SEM), Fourier transform infrared (FTIR), thermogravimetric analysis (TGA), water contact angle, water content, and thermogravimetric analysis (TGA). Moreover, all nanofibrous samples were evaluated for cell proliferation assay and ATCC antibacterial tests. Based on characterization results and cytotoxicity, the developed composite nanofibers-based webs showed good promise for utilization in anti-viral masks. Particularly, 2 wt% VB/PAN nanofibers have the highest antibacterial properties against negative and positive bacteria along with excellent cell viability.

Two 5 vol% Ni/Al2O3 composites with the difference in Al2O3 grain size were fabricated by pulsed electric current sintering technique to investigate the influence of Al2O3 grain size on oxidation behavior of the composites. Average Al2O3 grain sizes of two fabricated composites were 1.1 μm and 0.5 μm after sintering. Oxidation tests were conducted at temperatures ranging from 1100 to 1350 °C for 1–48 h in air. A thin NiAl2O4 layer was observed in exposed surface of samples after oxidation. An oxidized zone that consisted of Al2O3 matrix and NiAl2O4 grains was defined. Growth of the oxidized zone obeyed the parabolic law. Influences of Al2O3 grain size on high-temperature oxidation of the composites were discussed.

Systematic reviews and/or meta-analyses generally provide the best evidence for medical research. Authors are recommended to use flow diagrams to present the review process, allowing for better understanding among readers. However, no studies as of yet have assessed the quality of flow diagrams in systematic review/meta-analyses. Our study aims to evaluate the quality of systematic review/meta-analyses over a period of ten years, by assessing the quality of the flow diagrams, and the correlation to the methodological quality. Two hundred articles of "systematic review" and/or "meta-analysis" from January 2004 to August 2015 were randomly retrieved in Pubmed to be assessed for the flow diagram and methodological qualities. The flow diagrams were evaluated using a 16-grade scale corresponding to the four stages of PRISMA flow diagram. It composes four parts: Identification, Screening, Eligibility and Inclusion. Of the 200 articles screened, 154 articles were included and were assessed with AMSTAR checklist. Among them, 78 articles (50.6%) had the flow diagram. Over ten years, the proportion of papers with flow diagram available had been increasing significantly with regression coefficient beta = 5.649 (p = 0.002). However, the improvement in quality of the flow diagram increased slightly but not significantly (regression coefficient beta = 0.177, p = 0.133). Our analysis showed high variation in the proportion of articles that reported flow diagram components. The lowest proportions were 1% for reporting methods of duplicates removal in screening phase, followed by 6% for manual search in identification phase, 22% for number of studies for each specific/subgroup analysis, 27% for number of articles retrieved from each database, and 31% for number of studies included in qualitative analysis. The flow diagram quality was correlated with the methodological quality with the Pearson's coefficient r = 0.32 (p = 0.0039). Therefore, this review suggests that the reporting quality of flow diagram is less satisfactory, hence not maximizing the potential benefit of the flow diagrams. A guideline with standardized flow diagram is recommended to improve the quality of systematic reviews, and to enable better reader comprehension of the review process.

Leaves were harvested from four different cultivars of Colocasia esculenta and three cultivars of Alocasia odora that were growing on nine different farms in central Vietnam. The total, soluble and insoluble oxalate contents of the leaves were extracted and measured using HPLC chromatography. Total calcium determinations were also carried out on the same samples. The total oxalate content of the leaves ranged from 433.8 to 856.1 mg/100 g wet matter (WM) while the soluble oxalate ranged from 147.8 to 339.7 mg/100 g WM. The proportion of soluble oxalate ranged from 28% to 41% (overall mean 35%) of the total oxalate content of the leaves. The equivalent insoluble oxalate proportion ranged from 59% to 72% of the total (overall mean 65%). There was little difference between the Colocasia esculenta and Alocasia odora taro cultivars, although the total oxalate content was significantly higher in Alocasia odora cultivars. The overall mean total calcium content was 279.5 mg/100 WM and the percentage of insoluble calcium bound as calcium oxalate ranged from 31.7% to 57.3% of the total calcium content (overall mean 47.1%). The oxalate content in taro leaves is a major factor to consider when different cultivars of taro are recommended for human or animal consumption.