Search

Recently, the research community has taken great interest in human activity recognition (HAR) due to its wide range of applications in different fields of life, including medicine, security, and gaming. The use of sensory data for HAR systems is most common because the sensory data are collected from a person’s wearable device sensors, thus overcoming the privacy issues being faced in data collection through video cameras. Numerous systems have been proposed to recognize some common activities of daily living (ADLs) using different machine learning, image processing, and deep learning techniques. However, the existing techniques are computationally expensive, limited to recognizing short-term activities, or require large datasets for training purposes. Since an ADL is made up of a sequence of smaller actions, recognizing them directly from raw sensory data is challenging. In this paper, we present a computationally efficient two-level hierarchical framework for recognizing long-term (composite) activities, which does not require a very large dataset for training purposes. First, the short-term (atomic) activities are recognized from raw sensory data, and the probabilistic atomic score of each atomic activity is calculated relative to the composite activities. In the second step, the optimal features are selected based on atomic scores for each composite activity and passed to the two classification algorithms: random forest (RF) and support vector machine (SVM) due to their well-documented effectiveness for human activity recognition. The proposed method was evaluated on the publicly available CogAge dataset that contains 890 instances of 7 composite and 9700 instances of 61 atomic activities. The data were collected from eight sensors of three wearable devices: a smartphone, a smartwatch, and smart glasses. The proposed method achieved the accuracy of 96.61% and 94.1% by random forest and SVM classifiers, respectively, which shows a remarkable increase in the classification accuracy of existing HAR systems for this dataset.

This paper addresses the problem of feature encoding for gait analysis using multimodal time series sensory data. In recent years, the dramatic increase in the use of numerous sensors, e.g., inertial measurement unit (IMU), in our daily wearable devices has gained the interest of the research community to collect kinematic and kinetic data to analyze the gait. The most crucial step for gait analysis is to find the set of appropriate features from continuous time series data to accurately represent human locomotion. This paper presents a systematic assessment of numerous feature extraction techniques. In particular, three different feature encoding techniques are presented to encode multimodal time series sensory data. In the first technique, we utilized eighteen different handcrafted features which are extracted directly from the raw sensory data. The second technique follows the Bag-of-Visual-Words model; the raw sensory data are encoded using a pre-computed codebook and a locality-constrained linear encoding (LLC)-based feature encoding technique. We evaluated two different machine learning algorithms to assess the effectiveness of the proposed features in the encoding of raw sensory data. In the third feature encoding technique, we proposed two end-to-end deep learning models to automatically extract the features from raw sensory data. A thorough experimental evaluation is conducted on four large sensory datasets and their outcomes are compared. A comparison of the recognition results with current state-of-the-art methods demonstrates the computational efficiency and high efficacy of the proposed feature encoding method. The robustness of the proposed feature encoding technique is also evaluated to recognize human daily activities. Additionally, this paper also presents a new dataset consisting of the gait patterns of 42 individuals, gathered using IMU sensors.

Industries are rapidly moving toward mitigating errors and manual interventions by automating their process. The same motivation is carried out in this research which targets to study a conveyor system installed in soda ash manufacturing plants. Our aim is to automate the determination of optimal parameters, which are chosen by identifying the flow rate of the materials available on the conveyor belt for maintaining the ratio between raw materials being carried. The ratio is essential to produce 40% pure carbon dioxide gas needed for soda ash production. A visual sensor mounted on the conveyor belt is used to estimate the flow rate of the raw materials. After selecting the region of interest, a segmentation algorithm is defined based on a voting-based technique to segment the most confident region. Moments and contour features are extracted and passed to machine learning algorithms to estimate the flow rate of different experiments. An in-depth analysis is completed on various techniques and convincing results are achieved on the final data split with the best parameters using the Bagging regressor. Each step of the process is made resilient enough to work in a challenging environment even if the belt is placed in an outdoor environment. The proposed solution caters to the current challenges and serves as a practical solution for estimating material flow without manual intervention.

Lung cancer is the world’s second-largest cause of cancer mortality. Patients’ lives can be saved if this malignancy is detected early. Doctors, however, encounter difficulties in detecting cancer in computed tomography (CT) images. In recent years, significant research has been devoted to producing automated lung nodule detection methods that can help radiologists. Most of them use only the lung window in their analysis and generally do not consider the mediastinal windows, which, according to recent research, carry important information. In this paper, we propose a simple yet effective algorithm to analyze multi-window CT images for lung nodules. The algorithm works in three steps. First, the CT image is preprocessed to suppress any noise and improve the image quality. Second, the lungs are extracted from the preprocessed image. Based on the histogram analysis of the lung windows, we propose a multi-Otsu-based approach for lung segmentation in lung windows. The case of mediastinal windows is rather difficult due to irregular patterns in the histograms. To this end, we propose a global–local-mean-based thresholding technique for lung detection. In the final step, the nodule candidates are extracted from the segmented lungs using simple intensity-based thresholding. The radius of the extracted objects is computed to separate the nodule from the bronchioles and blood vessels. The proposed algorithm is evaluated on the benchmark LUNA16 dataset and achieves accuracy of over 94% for lung tumor detection, surpassing that of existing similar methods.

Farmland trees are a vital part of the local economy as trees are used by farmers for fuelwood as well as food, fodder, medicines, fibre, and building materials. As a result, mapping tree species is important for ecological, socio-economic, and natural resource management. The study evaluates very high-resolution remotely sensed WorldView-2 (WV-2) imagery for tree species classification in the agroforestry landscape of the Kano Close-Settled Zone (KCSZ), Northern Nigeria. Individual tree crowns extracted by geographic object-based image analysis (GEOBIA) were used to remotely identify nine dominant tree species (Faidherbia albida, Anogeissus leiocarpus, Azadirachta indica, Diospyros mespiliformis, Mangifera indica, Parkia biglobosa, Piliostigma reticulatum, Tamarindus indica, and Vitellaria paradoxa) at the object level. For every tree object in the reference datasets, eight original spectral bands of the WV-2 image, their spectral statistics (minimum, maximum, mean, standard deviation, etc.), spatial, textural, and color-space (hue, saturation), and different spectral vegetation indices (VI) were used as predictor variables for the classification of tree species. Nine different machine learning methods were used for object-level tree species classification. These were Extra Gradient Boost (XGB), Gaussian Naïve Bayes (GNB), Gradient Boosting (GB), K-nearest neighbours (KNN), Light Gradient Boosting Machine (LGBM), Logistic Regression (LR), Multi-layered Perceptron (MLP), Random Forest (RF), and Support Vector Machines (SVM). The two top-performing models in terms of highest accuracies for individual tree species classification were found to be SVM (overall accuracy = 82.1% and Cohen’s kappa = 0.79) and MLP (overall accuracy = 81.7% and Cohen’s kappa = 0.79) with the lowest numbers of misclassified trees compared to other machine learning methods.

Nitrogen is an essential nutrient element required for optimum crop growth and yield. If a specific amount of nitrogen is not applied to crops, their yield is affected. Estimation of nitrogen level in crops is momentous to decide the nitrogen fertilization in crops. The amount of nitrogen in crops is measured through different techniques, including visual inspection of leaf color and texture and by laboratory analysis of plant leaves. Laboratory analysis-based techniques are more accurate than visual inspection, but they are costly, time-consuming, and require skilled laboratorian and precise equipment. Therefore, computer-based systems are required to estimate the amount of nitrogen in field crops. In this paper, a computer vision-based solution is introduced to solve this problem as well as to help farmers by providing an easier, cheaper, and faster approach for measuring nitrogen deficiency in crops. The system takes an image of the crop leaf as input and estimates the amount of nitrogen in it. The image is captured by placing the leaf on a specially designed slate that contains the reference green and yellow colors for that crop. The proposed algorithm automatically extracts the leaf from the image and computes its color similarity with the reference colors. In particular, we define a green color value (GCV) index from this analysis, which serves as a nitrogen indicator. We also present an evaluation of different color distance models to find a model able to accurately capture the color differences. The performance of the proposed system is evaluated on a Spinacia oleracea dataset. The results of the proposed system and laboratory analysis are highly correlated, which shows the effectiveness of the proposed system.

Emerging 3D-related technologies such as augmented reality, virtual reality, mixed reality, and stereoscopy have gained remarkable growth due to their numerous applications in the entertainment, gaming, and electromedical industries. In particular, the 3D television (3DTV) and free-viewpoint television (FTV) enhance viewers’ television experience by providing immersion. They need an infinite number of views to provide a full parallax to the viewer, which is not practical due to various financial and technological constraints. Therefore, novel 3D views are generated from a set of available views and their depth maps using depth-image-based rendering (DIBR) techniques. The quality of a DIBR-synthesized image may be compromised for several reasons, e.g., inaccurate depth estimation. Since depth is important in this application, inaccuracies in depth maps lead to different textural and structural distortions that degrade the quality of the generated image and result in a poor quality of experience (QoE). Therefore, quality assessment DIBR-generated images are essential to guarantee an appreciative QoE. This paper aims at estimating the quality of DIBR-synthesized images and proposes a novel 3D objective image quality metric. The proposed algorithm aims to measure both textural and structural distortions in the DIBR image by exploiting the contrast sensitivity and the Hausdorff distance, respectively. The two measures are combined to estimate an overall quality score. The experimental evaluations performed on the benchmark MCL-3D dataset show that the proposed metric is reliable and accurate, and performs better than existing 2D and 3D quality assessment metrics.

Human activity recognition (HAR) aims to recognize the actions of the human body through a series of observations and environmental conditions. The analysis of human activities has drawn the attention of the research community in the last two decades due to its widespread applications, diverse nature of activities, and recording infrastructure. Lately, one of the most challenging applications in this framework is to recognize the human body actions using unobtrusive wearable motion sensors. Since the human activities of daily life (e.g., cooking, eating) comprises several repetitive and circumstantial short sequences of actions (e.g., moving arm), it is quite difficult to directly use the sensory data for recognition because the multiple sequences of the same activity data may have large diversity. However, a similarity can be observed in the temporal occurrence of the atomic actions. Therefore, this paper presents a two-level hierarchical method to recognize human activities using a set of wearable sensors. In the first step, the atomic activities are detected from the original sensory data, and their recognition scores are obtained. Secondly, the composite activities are recognized using the scores of atomic actions. We propose two different methods of feature extraction from atomic scores to recognize the composite activities, and they include handcrafted features and the features obtained using the subspace pooling technique. The proposed method is evaluated on the large publicly available CogAge dataset, which contains the instances of both atomic and composite activities. The data is recorded using three unobtrusive wearable devices: smartphone, smartwatch, and smart glasses. We also investigated the performance evaluation of different classification algorithms to recognize the composite activities. The proposed method achieved 79% and 62.8% average recognition accuracies using the handcrafted features and the features obtained using subspace pooling technique, respectively. The recognition results of the proposed technique and their comparison with the existing state-of-the-art techniques confirm its effectiveness.

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

Unmanned Aerial Vehicle (UAV) is one of the latest technologies for high spatial resolution 3D modeling of the Earth. The objectives of this study are to assess low-cost UAV data using image radiometric transformation techniques and investigate its effects on global and local accuracy of the Digital Surface Model (DSM). This research uses UAV Light Detection and Ranging (LIDAR) data from 80 m and UAV Drone data from 300 and 500 m flying height. RAW UAV images acquired from 500 m flying height are radiometrically transformed in Matrix Laboratory (MATLAB). UAV images from 300 m flying height are processed for the generation of 3D point cloud and DSM in Pix4D Mapper. UAV LIDAR data are used for the acquisition of Ground Control Points (GCP) and accuracy assessment of UAV Image data products. Accuracy of enhanced DSM with DSM generated from 300 m flight height were analyzed for point cloud number, density and distribution. Root Mean Square Error (RMSE) value of Z is enhanced from ±2.15 m to ±0.11 m. For local accuracy assessment of DSM, four different types of land covers are statistically compared with UAV LIDAR resulting in compatibility of enhancement technique with UAV LIDAR accuracy.

Multi-focus image fusion is the process of combining focused regions of two or more images to obtain a single all-in-focus image. It is an important research area because a fused image is of high quality and contains more details than the source images. This makes it useful for numerous applications in image enhancement, remote sensing, object recognition, medical imaging, etc. This paper presents a novel multi-focus image fusion algorithm that proposes to group the local connected pixels with similar colors and patterns, usually referred to as superpixels, and use them to separate the focused and de-focused regions of an image. We note that these superpixels are more expressive than individual pixels, and they carry more distinctive statistical properties when compared with other superpixels. The statistical properties of superpixels are analyzed to categorize the pixels as focused or de-focused and to estimate a focus map. A spatial consistency constraint is ensured on the initial focus map to obtain a refined map, which is used in the fusion rule to obtain a single all-in-focus image. Qualitative and quantitative evaluations are performed to assess the performance of the proposed method on a benchmark multi-focus image fusion dataset. The results show that our method produces better quality fused images than existing image fusion techniques.

Image fusion is a process that integrates similar types of images collected from heterogeneous sources into one image in which the information is more definite and certain. Hence, the resultant image is anticipated as more explanatory and enlightening both for human and machine perception. Different image combination methods have been presented to consolidate significant data from a collection of images into one image. As a result of its applications and advantages in variety of fields such as remote sensing, surveillance, and medical imaging, it is significant to comprehend image fusion algorithms and have a comparative study on them. This paper presents a review of the present state-of-the-art and well-known image fusion techniques. The performance of each algorithm is assessed qualitatively and quantitatively on two benchmark multi-focus image datasets. We also produce a multi-focus image fusion dataset by collecting the widely used test images in different studies. The quantitative evaluation of fusion results is performed using a set of image fusion quality assessment metrics. The performance is also evaluated using different statistical measures. Another contribution of this paper is the proposal of a multi-focus image fusion library, to the best of our knowledge, no such library exists so far. The library provides implementation of numerous state-of-the-art image fusion algorithms and is made available publicly at project website.

Movement analysis of human body parts is momentous in several applications including clinical diagnosis and rehabilitation programs. The objective of this research is to present a low-cost 3D visual tracking system to analyze the movement of various body parts during therapeutic procedures. Specifically, a marker based motion tracking system is proposed in this paper to capture the movement information in home-based rehabilitation. Different color markers are attached to the desired joints’ locations and they are detected and tracked in the video to encode their motion information. The availability of this motion information of different body parts during the therapy can be exploited to achieve more accurate results with better clinical insight, which in turn can help improve the therapeutic decision making. The proposed framework is an automated and inexpensive motion tracking system with execution speed close to real time. The performance of the proposed method is evaluated on a dataset of 10 patients using two challenging matrices that measure the average accuracy by estimating the joints’ locations and rotations. The experimental evaluation and its comparison with the existing state-of-the-art techniques reveals the efficiency of the proposed method.

The lung tumor is among the most detrimental kinds of malignancy. It has a high occurrence rate and a high death rate, as it is frequently diagnosed at the later stages. Computed Tomography (CT) scans are broadly used to distinguish the disease; computer aided systems are being created to analyze the ailment at prior stages productively. In this paper, we present a fully automatic framework for nodule detection from CT images of lungs. A histogram of the grayscale CT image is computed to automatically isolate the lung locale from the foundation. The results are refined using morphological operators. The internal structures are then extracted from the parenchyma. A threshold-based technique is proposed to separate the candidate nodules from other structures, e.g., bronchioles and blood vessels. Different statistical and shape-based features are extracted for these nodule candidates to form nodule feature vectors which are classified using support vector machines. The proposed method is evaluated on a large lungs CT dataset collected from the Lung Image Database Consortium (LIDC). The proposed method achieved excellent results compared to similar existing methods; it achieves a sensitivity rate of 93.75%, which demonstrates its effectiveness.

Movement analysis of infants’ body parts is momentous for the early detection of various movement disorders such as cerebral palsy. Most existing techniques are either marker-based or use wearable sensors to analyze the movement disorders. Such techniques work well for adults, however they are not effective for infants as wearing such sensors or markers may cause discomfort to them, affecting their natural movements. This paper presents a method to help the clinicians for the early detection of movement disorders in infants. The proposed method is marker-less and does not use any wearable sensors which makes it ideal for the analysis of body parts movement in infants. The algorithm is based on the deformable part-based model to detect the body parts and track them in the subsequent frames of the video to encode the motion information. The proposed algorithm learns a model using a set of part filters and spatial relations between the body parts. In particular, it forms a mixture of part-filters for each body part to determine its orientation which is used to detect the parts and analyze their movements by tracking them in the temporal direction. The model is represented using a tree-structured graph and the learning process is carried out using the structured support vector machine. The proposed framework will assist the clinicians and the general practitioners in the early detection of infantile movement disorders. The performance evaluation of the proposed method is carried out on a large dataset and the results compared with the existing techniques demonstrate its effectiveness.