Search

Robotic grasp in complex open-world scenarios requires an effective and generalizable perception. Estimating object’s pose is needed in a variety of practical grasping scenarios. Here we present a novel approach of pose estimation of textureless and textured objects. The algorithm utilizes a single RGB-D image to exploit depth invariant, oriented point pair feature as well as local contextual sensitivity in cluttered environments. To enhance the performance of the voting process and improve learning efficiency, we employ a global pruning algorithm that reduces the risk of overfitting and simplifies the structure of decision trees after compensating for the complementary information among multiple trees by optimizing a designed global objective function. Finally, we also refine the pose obtained from the above stage. The proposed approach of estimating 6-D (degree of freedom) poses of textured and textureless objects is evaluated on publicly available data sets against the recent works under various conditions. It illustrates that our framework is superior to these recent works. Further, we perform extensive qualitative experiments of robotic grasp to illustrate the proposed approach can be applied to practical scenarios. Note to Practitioners —This article is motivated by the problem of the pose estimation of textured and textureless objects in clutter environments. It is difficult for conventional works to address the issue of estimating textured or textureless objects’ poses in such scenarios. We considered that a novel system should be able to obtain the 6-D poses of objects. Therefore, we investigate the combined use of multiple split functions with different characteristics. Learning the model based on Hough forests always cost much computational resource; therefore, we construct a novel pruned Hough forest for solving this issue. Through the comparison and robotic grasp verifications, the behavior of our system can be used in practical applications. In future, we will deploy the proposed system in robotic assembling tasks.

We often locate ourselves in a trade-off situation between what is predicted and understanding why the predictive modeling made such a prediction. This high-risk medical segmentation task is no different where we try to interpret how well has the model learned from the image features irrespective of its accuracy. We propose image-specific fine-tuning to make a deep learning model adaptive to specific medical imaging tasks. Experimental results reveal that: a) proposed model is more robust to segment previously unseen objects (negative test dataset) than state-of-the-art CNNs; b) image-specific fine-tuning with the proposed heuristics significantly enhances segmentation accuracy; and c) our model leads to accurate results with fewer user interactions and less user time than conventional interactive segmentation methods. The model successfully classified ’no polyp’ or ’no instruments’ in the image irrespective of the absence of negative data in training samples from Kvasir-seg and Kvasir-Instrument datasets.

This work presents a novel approach to detecting stress differences between experts and novices in Situation Awareness (SA) tasks during maritime navigation using one type of wearable sensor, Empatica E4 Wristband. We propose that for a given workload state, the values of biosignal data collected from wearable sensor vary in experts and novices. We describe methods to conduct a designed SA task experiment, and collected the biosignal data on subjects sailing on a 240° view simulator. The biosignal data were analysed by using a machine learning algorithm, a Convolutional Neural Network. The proposed algorithm showed that the biosingal data associated with the experts can be categorized as different from that of the novices, which is in line with the results of NASA Task Load Index (NASA-TLX) rating scores. This study can contribute to the development of a self-training system in maritime navigation in further studies.

There is a growing interest in automatic crafting of neural network architectures as opposed to expert tuning to find the best architecture. On the other hand, the problem of stock trading is considered one of the most dynamic systems that heavily depends on complex trends of the individual company. This paper proposes a novel self-evolving neural network system called self-evolving Multi-Layer Perceptron (seMLP) which can abstract the data and produce an optimum neural network architecture without expert tuning. seMLP incorporates the human cognitive ability of concept abstraction into the architecture of the neural network. Genetic algorithm (GA) is used to determine the best neural network architecture that is capable of knowledge abstraction of the data. After determining the architecture of the neural network with the minimum width, seMLP prunes the network to remove the redundant neurons in the network, thus decreasing the density of the network and achieving conciseness. seMLP is evaluated on three stock market data sets. The optimized models obtained from seMLP are compared and benchmarked against state-of-the-art methods. The results show that seMLP can automatically choose best performing models.

While we know that motivated students learn better than non-motivated students but detecting motivation is challenging. Here we present a game-based motivation detection approach from the EEG signals. We take an original approach of using EEG-based brain computer interface to assess if motivation state is manifest in physiological EEG signals as well, and what are suitable conditions in order to achieve the goal? To the best of our knowledge, detection of motivation level from brain signals is proposed for the first time in this paper. In order to resolve the central obstacle of small EEG datasets containing deep features, we propose a novel and unique ‘residual-in-residual architecture of convolutional neural network (RRCNN)’ that is capable of reducing the problem of over-fitting on small datasets and vanishing gradient. Having accomplished this, several aspects of using EEG signals for motivation detection are considered, including channel selection and accuracy obtained using alpha or beta waves of EEG signals. We also include a detailed validation of the different aspects of our methodology, including detailed comparison with other works as relevant. Our approach achieves 89% accuracy in using EEG signals to detect motivation state while learning, where alpha wave signals of frontal asymmetry channels are employed. A more robust (less sensitive to learning conditions) 88% accuracy is achieved using beta waves signals of frontal asymmetry channels. The results clearly indicate the potential of detecting motivation states using EEG signals, provided suitable methodologies such as proposed in this paper, are employed. Published version SC acknowledges internship funding support from Research Council Norway’s INTPART grant no. 309802

Examining specific sub-cellular structures while minimizing cell perturbation is important in the life sciences. Fluorescence labeling and imaging is widely used. With the advancement of deep learning, digital staining routines for label-free analysis have emerged to replace fluorescence imaging. Nonetheless, digital staining of sub-cellular structures such as mitochondria is sub-optimal. This is because the models designed for computer vision are directly applied instead of optimizing them for microscopy data. We propose a new loss function with multiple thresholding steps to promote more effective learning for microscopy data. We demonstrate a deep learning approach to translate the labelfree brightfield images of living cells into equivalent fluorescence images of mitochondria with an average structural similarity of 0.77, thus surpassing the state-of-the-art of 0.7 with L1. We provide insightful examples of unique opportunities by data-driven deep learning-enabled image translations.

Diabetes mellitus affected an estimated 463 million people in the year 2019. The number of diabetic patients is projected to increase to an alarming figure of 700 million by the year 2045, out of which 90–95% of them are expected to be type 2 diabetes mellitus (T2DM) patients. The research presented an alternative way of state-of-the-art insulin therapy using manual insulin infusion. The T2DM model that simulates the body reaction of a T2DM patient has been developed using real human clinical data that uses insulin pump therapy. The proposed system uses a closed-loop control together with fuzzy gain scheduling and recurrent self-evolving Takagi–Sugeno–Kang fuzzy neural network (RST-FNN). Such a system will help the patient remove the need for manual insulin infusion. This proposed system will record the blood glucose level and predict the next iteration’s blood glucose level. The change in blood glucose level will help detect the food intake (carbohydrates) with reference to the gain scheduler and the controller will communicate with the insulin pump to infuse the corresponding amount of insulin.

Text classification is one of the widely used phenomena in different natural language processing tasks. State-of-the-art text classifiers use the vector space model for extracting features. Recent progress in deep models, recurrent neural networks those preserve the positional relationship among words achieve a higher accuracy. To push text classification accuracy even higher, multi-dimensional document representation, such as vector sequences or matrices combined with document sentiment, should be explored. In this paper, we show that documents can be represented as a sequence of vectors carrying semantic meaning and classified using a recurrent neural network that recognizes long-range relationships. We show that in this representation, additional sentiment vectors can be easily attached as a fully connected layer to the word vectors to further improve classification accuracy. On the UCI sentiment labelled dataset, using the sequence of vectors alone achieved an accuracy of 85.6%, which is better than 80.7% from ridge regression classifier—the best among the classical technique we tested. Additional sentiment information further increases accuracy to 86.3%. On our suicide notes dataset, the best classical technique—the Naíve Bayes Bernoulli classifier, achieves accuracy of 71.3%, while our classifier, incorporating semantic and sentiment information, exceeds that at 75% accuracy.

In fields such as finance, medicine, engineering, and science, making real-time predictions during transient periods characterized by sudden and large changes is a hard challenge for machine learning. Humans keep memory of these transient events, abstractly learn the most relevant rules and reuse them when similar events occur, which stems from episodic memory that allows storage and recall of similar events. This paper proposes a novel online general episodic memory mechanism (GEMM) and demonstrates its integration into the Neuro-Fuzzy system (NFS) architecture called evolving Mamdani Fuzzy Inference System (eMFIS) with Fuzzy Rule Interpolation and Extrapolation (FRI/E). Our proposition, called GEMM-eMFIS(FRI/E), learns from past events by storing and retrieving them from an episodic memory cache during event-driven transient behavior, thereby boosting performance while using a few rules only. GEMM-eMFIS(FRI/E) further has several in-built mechanisms that enable it to learn effectively from continuous stream of online data. They include associative-dissociative learning theory to keep its rule base updated, 2-stage incremental clustering; (2SIC) to determine cluster width, interpolation and extrapolation of rules to deal with concept shifts and drifts in the time-variant data, and rule pruning and merging to keep the rule base compact. GEMM-eMFIS (FRI/E) is benchmarked against other NFS’ on various time-variant datasets such as stock index prices and rainfall runoff with 3%-5% improvement during transient period and shows strong forecasting performances with 4%-5% more interpretability with lesser rules.

This paper proposes a novel human-inspired methodology called IRON-MAN (Integrated RatiONal prediction and Motionless ANalysis of videos) on mobile multi-processor systems-on-chips (MPSoCs). The methodology integrates analysis of the previous image frames of the video to represent the analysis of the current frame in order to perform Temporal Motionless Analysis of the Video (TMAV). This is the first work on TMAV using Convolutional Neural Network (CNN) for scene prediction in MPSoCs. Experimental results show that our methodology outperforms state-of-the-art. We also introduce a metric named, Energy Consumption per Training Image (ECTI) to assess the suitability of using a CNN model in mobile MPSoCs with a focus on energy consumption of the device.

Interpretabilty is one of the desired characteristics in various classification task. Rule-based system and fuzzy logic can be used for interpretation in classification. The main drawback of rule-based system is that it may contain large complex rules for classification and sometimes it becomes very difficult in interpretation. Rule reduction is also difficult for various reasons. Removing important rules may effect in classification accuracy. This paper proposes a hybrid fuzzy-rough set approach named RS-HeRR for the generation of effective, interpretable and compact rule set. It combines a powerful rule generation and reduction fuzzy system, called Hebbian-based rule reduction algorithm (HeRR) and a novel rough-set-based attribute selection algorithm for rule reduction. The proposed hybridization leverages upon rule reduction through reduction in partial dependency as well as improvement in system performance to significantly reduce the problem of redundancy in HeRR, even while providing similar or better accuracy. RS-HeRR demonstrates these characteristics repeatedly over four diverse practical classification problems, such as diabetes identification, urban water treatment monitoring, sonar target classification, and detection of ovarian cancer. It also demonstrates excellent performance for highly biased datasets. In addition, it competes very well with established non-fuzzy classifiers and outperforms state-of-the-art methods that use rough sets for rule reduction in fuzzy systems.

Artificial intelligent systems often model the solutions of typical machine learning problems, inspired by biological processes, because of the biological system is faster and much adaptive than deep learning. The utility of bio-inspired learning methods lie in its ability to discover unknown patterns, and its less dependence on mathematical modeling or exhaustive training. In this paper, we propose a new bio-inspired learning model for a single-class classifier to detect abnormality in video object trajectories. The method uses a simple but dynamic extreme learning machine (ELM) and hierarchical temporal memory (HTM) together referred to as ELM-HTM in an unsupervised way to learn and classify time series patterns. The method has been tested on trajectory sequences in traffic surveillance to find abnormal behaviors such as high-speed, unusual stops, driving in wrong directions, loitering, etc. Experiments have also been performed with 3D air signatures captured using sensors and used for biometric authentication(forged/genuine). The results indicate a significant gain over training time and classification accuracy. The proposed method outperforms in predicting long-time patterns by observing small steps with an average accuracy gain of 15% as compared to the state-of-the-art HTM. The method has applications in detecting abnormal activities in videos by learning the movement patterns as well as in biometric authentication.

Automated feature extraction from program source-code such that proper computing resources could be allocated to the program is very difficult given the current state of technology. Therefore, conventional methods call for skilled human intervention in order to achieve the task of feature extraction from programs. This research is the first to propose a novel human-inspired approach to automatically convert program source-codes to visual images. The images could be then utilized for automated classification by visual convolutional neural network (CNN) based algorithm. Experimental results show high prediction accuracy in classifying the types of program in a completely automated manner using this approach.

Accurate ellipse detection in image streams at real-time execution is an open challenge. We present a novel fast and robust ellipse detection method. The method adopts arcs selection, smart grouping, and repeated utilization of gradient information to significantly reduce the computations otherwise needed without compromising the detection effectiveness. Geometric properties calculable with few computations, such as arc smoothness, relative placement of curves, and region of confidence for ellipse centres, are utilized for this purpose. An exhaustive sensitivity analysis of the method's control parameters has been performed. It reveals range of values that support consistent performance over diverse challenging datasets with complex background, multiple differently sized ellipses, and occluded, overlapping ellipses. The method's performance is compared with six state-of-the-art detectors over four diverse datasets. Among all the tested methods, the proposed method demonstrates the best balance between detection effectiveness (the best or the second best F-measure scores) and computation time (>40 Hz) across all the datasets.

A novel pseudo-incremental ensemble rough set pseudo-outer product (PIE-RSPOP) fuzzy neural network is proposed for learning and prediction of trends in complex temporal series. It incorporates four theories pertaining to the learning and memorizing mechanisms in human beings. BCM theory of synaptic metaplasticity replaces Hebbian learning of weights of fuzzy rules and allows for a more natural associative–dissociative learning of weights. Short term forgetting of weights of the fuzzy rules is integrated for rules that are recalled and rules that are not recalled by an instance of data through Ebbinghaus theory of forgetting due to decay and displacement. The long term forgetting of networks in the ensemble is also incorporated through exponential decay of weights based on the age and strengths of the networks. Lastly, a hippocampal mechanism for caching the memories for subsequent recall is proposed. Another contribution of PIE-RSPOP is its ability to deal with concept drift and less storage of historical data. Lastly, despite the usually uninterpretable nature of incremental ensembles of fuzzy networks, a scheme to derive simple interpretable single knowledge base is also proposed. Variety of numerical results on standard datasets are used to demonstrate the advantages of PIE-RSPOP over other incremental learning methods.

We present a survey on maritime object detection and tracking approaches, which are essential for the development of a navigational system for autonomous ships. The electro-optical (EO) sensor considered here is a video camera that operates in the visible or the infrared spectra, which conventionally complements radar and sonar for situational awareness at sea and has demonstrated its effectiveness over the last few years. This paper provides a comprehensive overview of various approaches of video processing for object detection and tracking in the maritime environment. We follow an approach-based taxonomy wherein the advantages and limitations of each approach are compared. The object detection system consists of the following modules: horizon detection, static background subtraction, and foreground segmentation. Each of these has been studied extensively in maritime situations and has been shown to be challenging due to the presence of background motion especially due to waves and wakes. The key processes involved in object tracking include video frame registration, dynamic background subtraction, and the object tracking algorithm itself. The challenges for robust tracking arise due to camera motion, dynamic background, and low contrast of tracked object, possibly due to environmental degradation. The survey also discusses multisensor approaches and commercial maritime systems that use EO sensors. The survey also highlights methods from computer vision research, which hold promise to perform well in maritime EO data processing. Performance of several maritime and computer vision techniques is evaluated on Singapore Maritime Dataset.

Learning to dehaze single hazy images, especially using a small training dataset is quite challenging. We propose a novel generative adversarial network architecture for this problem, namely back projected pyramid network (BPPNet), that gives good performance for a variety of challenging haze conditions, including dense haze and inhomogeneous haze. Our architecture incorporates learning of multiple levels of complexities while retaining spatial context through iterative blocks of UNets and structural information of multiple scales through a novel pyramidal convolution block. These blocks together for the generator and are amenable to learning through back projection. We have shown that our network can be trained without over-fitting using as few as 20 image pairs of hazy and non-hazy images. We report the state of the art performances on NTIRE 2018 homogeneous haze datasets for indoor and outdoor images, NTIRE 2019 denseHaze dataset, and NTIRE 2020 non-homogeneous haze dataset.

This paper proposes a novel human-inspired methodology called IRON-MAN ( I ntegrated R ati ON al prediction and M otionless AN alysis ) for mobile multi-processor systems-on-chips (MPSoCs). The methodology integrates analysis of the previous image frames of the video to represent the analysis of the current frame in order to perform Temporal Motionless Analysis of the Video ( TMAV ). This is the first work on TMAV using Convolutional Neural Network (CNN) for scene prediction in MPSoCs. Experimental results show that our methodology outperforms state-of-the-art. We also introduce a metric named, Energy Consumption per Training Image ( ECTI ) to assess the suitability of using a CNN model in mobile MPSoCs with a focus on energy consumption and lifespan reliability of the device.

Diagrammatic reasoning (DR) problems are well known. However, solving DR problems represented in 4 × 1 Raven’s Progressive Matrix (RPM) form using computer vision and pattern recognition has not yet been tried. Emergence of deep learning techniques aided by advanced computing can be exploited to solve such DR problems. In this paper, we propose a new learning framework by combining LSTM and Convolutional LSTM to solve 4 × 1 DR problems. Initially, the elementary geometrical shapes in such problems are detected using a typical CNN-based detector. Next, relations of various shapes are analyzed and a high-level feature set is produced and processed in the LSTM framework. A new 4 × 1 DR dataset has been prepared and made available to the research community. We believe, it will be helpful in advancing this research further. We have compared our method with some of the existing frameworks that can be used for solving RPM-guided DR problems. We have recorded 18–20% increase in the average prediction accuracy as compared to the prior frameworks when applied to RPM-guided DR problems. We believe the CV research community will be interested to carry out similar research, particularly to investigate the feasibility of solving other types of known DR problems.

This paper introduces a fusion convolutional architecture for efficient learning of spatio-temporal features in video action recognition. Unlike 2D convolutional neural networks (CNNs), 3D CNNs can be applied directly on consecutive frames to extract spatio-temporal features. The aim of this work is to fuse the convolution layers from 2D and 3D CNNs to allow temporal encoding with fewer parameters than 3D CNNs. We adopt transfer learning from pre-trained 2D CNNs for spatial extraction, followed by temporal encoding, before connecting to 3D convolution layers at the top of the architecture. We construct our fusion architecture, semi-CNN, based on three popular models: VGG-16, ResNets and DenseNets, and compare the performance with their corresponding 3D models. Our empirical results evaluated on the action recognition dataset UCF-101 demonstrate that our fusion of 1D, 2D and 3D convolutions outperforms its 3D model of the same depth, with fewer parameters and reduces overfitting. Our semi-CNN architecture achieved an average of 16&ndash, 30% boost in the top-1 accuracy when evaluated on an input video of 16 frames.

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Maritime vessels equipped with visible and infrared cameras can complement other conventional sensors for object detection. However, application of computer vision techniques in maritime domain received attention only recently. The maritime environment offers its own unique requirements and challenges. Assessment of the quality of detections is a fundamental need in computer vision. However, the conventional assessment metrics suitable for usual object detection are deficient in the maritime setting. Thus, a large body of related work in computer vision appears inapplicable to the maritime setting at the first sight. We discuss the problem of defining assessment metrics suitable for maritime computer vision. We consider new bottom edge proximity metrics as assessment metrics for maritime computer vision. These metrics indicate that existing computer vision approaches are indeed promising for maritime computer vision and can play a foundational role in the emerging field of maritime computer vision.

Identification of the seed varieties is essential in the quality control and high yield crop growth. The existing methods of varietal identification rely primarily on visual examination and DNA fingerprinting. Although the pattern of DNA fingerprinting allows precise classification of seed varieties but fraught with challenges such as low rate of polymorphism amongst closely related species, destructive method of analysis and a huge cost involved in identification of robust markers such as simple sequence repeat (SSR) and single nucleotide polymorphisms. Here, we propose a fast, non-contact and non-invasive technique, deep learning assisted optical coherence tomography (OCT) for subsurface imaging in order to distinguish different seed varieties. The volumetric dataset of, (a) four rice varieties (PUSA Basmati 1, PUSA 1509, PUSA 44 and IR 64) and, (b) seven morphologically similar seeds of rice landrace Pokkali was acquired using OCT technique. A feedforward deep neural network is implemented for deep feature extraction and to classify the OCT images into their relevant classes. The proposed method provides the classification accuracy of 89.6% for the dataset of total 158,421 OCT images and 82.5% in classifying the dataset of total 56,301 OCT images collected from Pokkali seeds. The current technique can accurately classify seed varieties irrespective of the morphological similarities and can be adopted for the removal of varietal duplication and assessment of the purity of the seeds.

Quantitative phase microscopy (QPM) is a label-free technique that enables monitoring of morphological changes at the subcellular level. The performance of the QPM system in terms of spatial sensitivity and resolution depends on the coherence properties of the light source and the numerical aperture (NA) of objective lenses. Here, we propose high space-bandwidth quantitative phase imaging using partially spatially coherent digital holographic microscopy (PSC-DHM) assisted with a deep neural network. The PSC source synthesized to improve the spatial sensitivity of the reconstructed phase map from the interferometric images. Further, compatible generative adversarial network (GAN) is used and trained with paired low-resolution (LR) and high-resolution (HR) datasets acquired from the PSC-DHM system. The training of the network is performed on two different types of samples, i.e. mostly homogenous human red blood cells (RBC), and on highly heterogeneous macrophages. The performance is evaluated by predicting the HR images from the datasets captured with a low NA lens and compared with the actual HR phase images. An improvement of 9× in the space-bandwidth product is demonstrated for both RBC and macrophages datasets. We believe that the PSC-DHM + GAN approach would be applicable in single-shot label free tissue imaging, disease classification and other high-resolution tomography applications by utilizing the longitudinal spatial coherence properties of the light source.

The recent worldwide outbreak of the novel coronavirus (COVID-19) has opened up new challenges to the research community. Artificial intelligence (AI) driven methods can be useful to predict the parameters, risks, and effects of such an epidemic. Such predictions can be helpful to control and prevent the spread of such diseases. The main challenges of applying AI is the small volume of data and the uncertain nature. Here, we propose a shallow long short-term memory (LSTM) based neural network to predict the risk category of a country. We have used a Bayesian optimization framework to optimize and automatically design country-specific networks. The results show that the proposed pipeline outperforms state-of-the-art methods for data of 180 countries and can be a useful tool for such risk categorization. We have also experimented with the trend data and weather data combined for the prediction. The outcome shows that the weather does not have a significant role. The tool can be used to predict long-duration outbreak of such an epidemic such that we can take preventive steps earlier.

We propose a high efficient learning approach to estimating 6D (Degree of Freedom) pose of the textured or texture-less objects for grasping purposes in a cluttered environment where the objects might be partially occluded. The method comprises three main steps. Given a single RGB-D image, we first deploy appropriate features and the random forest to deduce the object class probability and cast votes for the 6D pose in Hough space by joint regression and classification framework, adopting reservoir sampling and summarizing the pose distribution by clustering. Next, we integrate the auto-context into cascaded Hough forests to improve the efficiency of learning. Extensive experiments on various public datasets and robotic grasps indicate that our method presents some improvements over the state-of-art and reveals the capability for estimating poses in practical applications efficiently. Accepted version

Source at https://doi.org/10.1038/s41598-018-23374-7. Localization microscopy and multiple signal classification algorithm use temporal stack of image frames of sparse emissions from fluorophores to provide super-resolution images. Localization microscopy localizes emissions in each image independently and later collates the localizations in all the frames, giving same weight to each frame irrespective of its signal-to-noise ratio. This results in a bias towards frames with low signal-to-noise ratio and causes cluttered background in the super-resolved image. User-defined heuristic computational filters are employed to remove a set of localizations in an attempt to overcome this bias. Multiple signal classification performs eigen-decomposition of the entire stack, irrespective of the relative signal-to-noise ratios of the frames, and uses a threshold to classify eigenimages into signal and null subspaces. This results in under-representation of frames with low signal-to-noise ratio in the signal space and over-representation in the null space. Thus, multiple signal classification algorithms is biased against frames with low signal-to-noise ratio resulting into suppression of the corresponding fluorophores. This paper presents techniques to automatically debias localization microscopy and multiple signal classification algorithm of these biases without compromising their resolution and without employing heuristics, user-defined criteria. The effect of debiasing is demonstrated through five datasets of invitro and fixed cell samples.

This paper provides a benchmark of the performance of 23 classical and state-of-the-art background subtraction (BS) algorithms on visible range and near infrared range videos in the Singapore Maritime dataset. Importantly, our study indicates the limitations of the conventional performance evaluation criteria for maritime vision and proposes new performance evaluation criteria that is better suited to this problem. This paper provides insight into the specific challenges of BS in maritime vision. We identify four open challenges that plague BS methods in maritime scenario. These include spurious dynamics of water, wakes, ghost effect, and multiple detections. Poor recall and extremely poor precision of all the 23 methods, which have been otherwise successful for other challenging BS situations, allude to the need for new BS methods custom designed for maritime vision. NRF (Natl Research Foundation, S’pore) Accepted version

Robotic manipulation employs the object detection in images to create a scene awareness and locate an object's pose. In dynamic scenarios, fast multiobject detection and tracking are crucial. Many objects commonly found in household and industrial environments are represented by cylindrical shapes. Thus, it is available for robots to manipulate them through the real-Time detection of elliptic shape primitives formed by the circular tops of these objects. We devise an efficient algorithm of the detection of elliptic shape primitives, which in turn enables robust and real-Time robotic manipulations of such objects. The proposed algorithm incorporates the information of elliptic edge curvature, splits complex curves into arcs, classifies the arcs into different quadrants of a candidate elliptic shape, determines the quality of arc selection for ellipse fitting, and then retrieves the corresponding elliptic shape primitive. Our algorithm provides either faster or more accurate ellipse detection results than the current state-of-The-Art methods, irrespective of challenging scenarios such as occluded or overlapping ellipses. This is verified by performance comparison with six state-of-The-Art elliptic shape detection algorithms on four public image datasets. The algorithm has been integrated on robots to demonstrate the ability to carry out accurate robotic manipulations (tracking, grasping, and stacking) of cylindrical objects in real time. We show that the robotic manipulator, empowered by the elliptic shape primitive algorithm, performs well in complex manipulation experiments as well as dynamic scenarios. Accepted version

Robotic manipulation of objects requires a fast recognition from image stream. For many cylindrical object (e.g., cans, cups, pipes, bottles, etc.) this is possible through detection of ellipse depicting the circular top of the cylinder. Growing industrial and warehouse applications of robots drive the demand for fast and reliable detection of ellipses, while state-of-the-art methods are lacking in either speed or accuracy strength. We present a novel algorithm to perform fast and robust ellipse detection. First, the method utilizes the information of edge curvature to split curves into arcs. Next, the arc convexity–concavity is used to classify arcs into different quadrants of ellipses. Then, based on multiple geometric constraints the arcs can be grouped at low computational cost. Our method is compared with six state-of-the-art methods using three public image datasets. The comparison results show that the proposed algorithm outperforms other methods with high detection accuracy and fast detection speed. Lastly, the algorithm is applied to identifying cylindrical objects in real-time for arranging and tracking purposes.

Optical coherence tomography (OCT) is being increasingly adopted as a label-free and non-invasive technique for biomedical applications such as cancer and ocular disease diagnosis. Diagnostic information for these tissues is manifest in textural and geometric features of the OCT images, which are used by human expertise to interpret and triage. However, it suffers delays due to the long process of the conventional diagnostic procedure and shortage of human expertise. Here, a custom deep learning architecture, LightOCT, is proposed for the classification of OCT images into diagnostically relevant classes. LightOCT is a convolutional neural network with only two convolutional layers and a fully connected layer, but it is shown to provide excellent training and test results for diverse OCT image datasets. We show that LightOCT provides 98.9% accuracy in classifying 44 normal and 44 malignant (invasive ductal carcinoma) breast tissue volumetric OCT images. Also, >96% accuracy in classifying public datasets of ocular OCT images as normal, age-related macular degeneration and diabetic macular edema. Additionally, we show ∼96% test accuracy for classifying retinal images as belonging to choroidal neovascularization, diabetic macular edema, drusen, and normal samples on a large public dataset of more than 100,000 images. The performance of the architecture is compared with transfer learning based deep neural networks. Through this, we show that LightOCT can provide significant diagnostic support for a variety of OCT images with sufficient training and minimal hyper-parameter tuning. The trained LightOCT networks for the three-classification problem will be released online to support transfer learning on other datasets.

In this paper, we propose a novel ellipse detection algorithm for synthetic and real images. Existing ellipse detection methods are too slow when used with limited hardware resources. The proposed method demonstrates the capability of detecting ellipses with an excellent accuracy at an acceptable speed level in three public datasets. The excellent performance is attributed to the novel combination of classification of arcs into different quadrants of a candidate ellipse, edge curvature and convexity-concavity analysis, and an elliptic geometry constraint.

Ultrasound based structural health monitoring of piezoelectric material is challenging if a damage changes at a microscale over time. Classifying geometrically similar damages with a difference in diameter as small as 100 &mu, m is difficult using conventional sensing and signal analysis approaches. Here, we use an unconventional ultrasound sensing approach that collects information of the entire bulk of the material and investigate the applicability of machine learning approaches for classifying such similar defects. Our results show that appropriate feature design combined with simple k-nearest neighbor classifier can provide up to 98% classification accuracy even though conventional features for time-series data and a variety of classifiers cannot achieve close to 70% accuracy. The newly proposed hybrid feature, which combines frequency domain information in the form of power spectral density and time domain information in the form of sign of slope change, is a suitable feature for achieving the best classification accuracy on this challenging problem.

Video-based person reidentification is a challenging and important task in surveillance-based applications. Toward this, several shallow and deep networks have been proposed. However, the performance of existing shallow networks does not generalize well on large datasets. To improve the generalization ability, we propose a shallow end-to-end network which incorporates two stream convolutional neural networks, discriminative visual attention and recurrent neural network with triplet and softmax loss to learn the spatiotemporal fusion features. To effectively use both spatial and temporal information, we apply spatial, temporal, and spatiotemporal pooling. In addition, we contribute a large dataset of airborne videos for person reidentification, named DJI01. It includes various challenging conditions, such as occlusion, illuminationchanges, people with similar clothes, and the same people on different days. We perform elaborate qualitative and quantitative analyses to demonstrate the robust performance of the proposed model. Published version

A novel method that uses both the local and the global nature of fit for dominant point detection is proposed. Most other methods use local fit to detect dominant points. The proposed method uses simple metrics like precision (local nature of fit) and reliability (global nature of fit) as the optimization goals for detecting the dominant points. Depending on the desired level of fitting (very fine or crude), the threshold for precision and reliability can be chosen in a very simple manner. Extensive comparison of various line fitting algorithms based on metrics such as precision, reliability, figure of merit, integral square error, and dimensionality reduction is benchmarked on publicly available and widely used datasets (Caltech 101, Caltech 256, and Pascal (2007, 2008, 2009, 2010) datasets) comprising 102628 images. Such work is especially useful for segmentation, shape representation, activity recognition, and robust edge feature extraction in object detection and recognition problems.

This paper addresses the problem of horizon detection, a fundamental process in numerous object detection algorithms, in a maritime environment. The maritime environment is characterized by the absence of fixed features, the presence of numerous linear features in dynamically changing objects and background and constantly varying illumination, rendering the typically simple problem of detecting the horizon a challenging one. We present a novel method called multi-scale consistence of weighted edge Radon transform, abbreviated as MuSCoWERT. It detects the long linear features consistent over multiple scales using multi-scale median filtering of the image followed by Radon transform on a weighted edge map and computing the histogram of the detected linear features. We show that MuSCoWERT has excellent performance, better than seven other contemporary methods, for 84 challenging maritime videos, containing over 33,000 frames, and captured using visible range and near-infrared range sensors mounted onboard, onshore, or on floating buoys. It has a median error of about 2 pixels (less than 0.2%) from the center of the actual horizon and a median angular error of less than 0.4 deg. We are also sharing a new challenging horizon detection dataset of 65 videos of visible, infrared cameras for onshore and onboard ship camera placement.

This paper proposes a new method for horizon detection called the multi-scale cross modal linear feature. This method integrates three different concepts related to the presence of horizon in maritime images to increase the accuracy of horizon detection. Specifically it uses the persistence of horizon in multi-scale median filtering, and its detection as a linear feature commonly detected by two different methods, namely the Hough transform of edgemap and the intensity gradient.We demonstrate the performance of the method over 13 videos comprising of more than 3000 frames and show that the proposed method detects horizon with small error in most of the cases, outperforming three state-of-the-art methods.

This paper discusses the problem of reducing camera metamerism by using a fourth spectral channel in addition to the typical red, green, and blue channels of the tristimulus sensor used in the commercial consumer cameras. Specifically, we consider three options for this fourth channel. The first option is to use a color channel from another camera to reduce metamerism. The second option is to use a color channel from an image captured by the same camera but with a color filter as the fourth channel. The third option is to design a specific spectral channel to be fabricated with the existing camera sensor. This option uses the metameric black space to design the channel. The commercial cameras' original metamerism is typically more than 20%, as observed in a dataset of 335 spectral images captured in 5 different indoor illuminations. Our results show that the third option is the best since it reduces metamerism down to about 5%. Among the first and second options, the first option is more effective and it reduces metamerism down to about 15%. The channel designed using the third option can be used for advanced applications such as distinguishing objects with different spectral reflectances but similar colors.

In this paper, we propose a novel ellipse detection method for real images. The proposed method uses the information of edge curvature and their convexity in relation to other edge contours as clues for identifying edge contours that can be grouped together. A search region is computed for every edge contour that contains other edge contours eligible for grouping with the current edge contour. A two-dimensional Hough transform is performed in an intermediate step, in which we use a new ‘relationship score’ for ranking the edge contours in a group, instead of the conventional histogram count. The score is found to be more selective and thus more efficient. In addition, we use three novel saliency criteria, that are non-heuristic and consider various aspects for quantifying the goodness of the detected elliptic hypotheses and finally selecting good elliptic hypotheses. The thresholds for selection of elliptic hypotheses are determined by the detected hypotheses themselves, such that the selection is free from human intervention. The method requires a few seconds in most cases. So, it is suitable for practical applications. The performance of the proposed ellipse detection method has been tested on a dataset containing 1200 synthetic images and the Caltech 256 dataset containing real images. In both cases, the results show that the proposed ellipse detection method performs far better than existing methods and is close to the ideal results, with precision, recall, and F-measure, all very close to 1. Further, the method is robust to the increase in the complexity of the images (such as overlapping ellipses, occluded ellipses), while the performance of the contemporary methods deteriorates significantly.

Despite many efforts, there are no computational models of consciousness that can be used to design conscious intelligent machines. This is mainly attributed to available definitions of consciousness being human centered, vague, and incomplete. Through a biological analysis of consciousness and concept of machine intelligence, we propose a physical definition of consciousness with the hope to model it in intelligent machines. We propose a computational model of consciousness driven by competing motivations, goals, and attention-switching. We propose a concept of mental saccades that is useful for explaining the attention-switching and focusing mechanism from computational perspective. Finally, we compare our model with other computational models of consciousness.

This paper discusses the design of an additional spectral filter (i.e., a fourth channel) to be used with existing camera sensors such that the camera's modified color gamut overlaps almost the full CIE XYZ color gamut. The proposed approach leverages on the matrix-R theory that states that the space of metamerism of a sensor, known as the metameric black space, can be determined directly from the camera's spectral sensitivities. Using this metameric black space, a novel fourth channel has been designed on the sensor that can expand the camera's gamut. The effectiveness of this idea has been demonstrated for five commercial cameras, Munsell color chips, and images taken under various illuminations. It is shown that the designed fourth channel is very effective in fitting the camera's color gamuts to CIE XYZ color gamut, reducing CIE LAB colorimetric distances, as well as the color differences between the camera's XYZ images and the true CIE XYZ images.

Camera images saved in raw format are being adopted in computer vision tasks since raw values represent minimally processed sensor responses. Camera manufacturers, however, have yet to adopt a standard for raw images and current raw-rgb values are device specific due to different sensors spectral sensitivities. This results in significantly different raw images for the same scene captured with different cameras. This paper focuses on estimating a mapping that can convert a raw image of an arbitrary scene and illumination from one camera's raw space to another. To this end, we examine various mapping strategies including linear and non-linear transformations applied both in a global and illumination-specific manner. We show that illumination-specific mappings give the best result, however, at the expense of requiring a large number of transformations. To address this issue, we introduce an illumination-independent mapping approach that uses white-balancing to assist in reducing the number of required transformations. We show that this approach achieves state-of-the-art results on a range of consumer cameras and images of arbitrary scenes and illuminations.

A macro-filter-lens design that can correct for chromatic and geometric aberrations simultaneously while providing for a long focal length is presented. The filter is easy to fabricate since it involves two spherical surfaces and a planar surface. Chromatic aberration correction is achieved by making all the rays travel the same optical distance inside the filter element (negative meniscus). Geometric aberration is corrected for by the lens element (plano–convex), which makes the output rays parallel to the optic axis. This macro-filter-lens design does not need additional macro lenses and it provides an inexpensive and optically good (aberration compensated) solution for macro imaging of objects not placed close to the camera.

This paper focuses on a training-based method to reconstruct a scene’s spectral reflectance from a single RGB image captured by a camera with known spectral response. In particular, we explore a new strategy to use training images to model the mapping between camera-specific RGB values and scene reflectance spectra. Our method is based on a radial basis function network that leverages RGB white-balancing to normalize the scene illumination to recover the scene reflectance. We show that our method provides the best result against three state-of-art methods, especially when the tested illumination is not included in the training stage. In addition, we also show an effective approach to recover the spectral illumination from the reconstructed spectral reflectance and RGB image. As a part of this work, we present a newly captured, publicly available, data set of hyperspectral images that are useful for addressing problems pertaining to spectral imaging, analysis and processing.

This paper deals with tracking of deformable objects in the presence of occlusion using dominant point representation of the boundary contour. A novel nonintegral time propagation model for propagating the dominant points is proposed. It uses an initial guess generated from a linear operation and an analytical conjugate gradient approach for online robust learning of the shape deformation and motion model. A scheme is presented to automatically detect and correct the region of large local deformation. In order to deal with occlusion, admissible restrictions on deformation and motion of the object are automatically determined. The proposed method overcomes the need of offline learning and learns the deformation and motion model of the object using very few initial frames of the input video. The performance of the method is demonstrated using varieties of videos of different objects.

Knowing the RGB spectral sensitivities of a camera is useful for several image processing applications. However, camera manufacturers seldom provide this information. Calibration methods for determining them can be daunting, requiring either sophisticated instruments or carefully controlled lighting. This paper presents a quick and easy method that provides a reasonable approximation of the camera response functions with only a color chart and casual unknown illuminations. Our approach is enabled by careful design of the cost function that imposes several constraints on the problem to make it tractable. In the components of the cost function, the Luther condition provides the global shape prior and the commercial lighting systems with unknown spectra provide narrow spectral windows for local reconstruction. The quality of reconstruction is comparable with other methods that use known illumination.

Detecting geometric shapes like ellipses from real images have many potential applications. Some examples include pupil tracking, detecting spherical or ellipsoidal objects like fruits, pebbles, golf balls, etc. from a scene for robotic applications, detecting ellipsoidal objects in underwater images, detecting fetal heads, cells, or nuclei in biological and biomedical images, identifying the eddy currents and zones using oceanic images, forming structural descriptors for objects and faces, traffic sign interpretation1, etc.

We present an extended architecture for adaptive traffic signal control system with online learning feature. The presented architecture is modified to incorporate the advantages of cloud computing such that the resultant architecture has high accuracy, online learning capability, scalability, robustness, and real time performance.

We prove that when a line is approximated using digital line, the error in the slope of the digital line has a definite upper bound and is strongly dependent on the two pixels chosen for defining the digital line. Thus, an analytical expression of the maximum deviation of the pixels from the digital line can be derived. Using this, the conventional line fitting methods that use maximum tolerable deviation as the optimization goal can be made control-parameter independent. This error bound can be used to make the most recent and sophisticated line fitting methods parameter independent and more robust to digitization noises. In our knowledge, this is the first line fitting method completely devoid of any control parameter. Such control-parameter independent line fitting algorithm retains the characteristics of the digital curve with sufficient reliability and precision and provides good dimensionality reduction in representing the digital curves. Extensive results have been generated for 9 datasets comprising of about a hundred thousand images. The proposed method shows robust and repeatable performance across all the datasets with low standard deviation in the performance.

A novel ellipse detection method for real images is proposed here. The method is novel in various respects. It uses a combination of geometric and Hough transform techniques for generating an initial guess of the centers of ellipses. This initial guess is used in a novel grouping technique in which the edges within a group are ranked in the order of their strength of relationship with the group. The novel relationship score is more selective and thus more efficient than the typical histogram count used in Hough transform methods. A least squares method has been adapted into an optimization scheme that reduces the number of outliers. The elliptic hypotheses are then subjected to non-heuristic saliency criteria. The thresholds for selection of elliptic hypotheses are determined by the detected hypotheses themselves, such that the selection is image independent and free of human intervention. Since only two control parameters are needed and the method requires a few seconds in most cases, it is suitable for practical applications. The experimental results for 1200 synthetic images and 400 real images from Caltech 256 dataset clearly demonstrate the superior accuracy and robustness of the proposed method as compared to contemporary ellipse detection methods.

The time of delivering the embedded system solution to market is very important, because of the highly competitive market and demand of the consumer needs. In Real Time Operating System (RTOS) based embedded systems, the overall correctness of the embedded system depends on logically correct output within the specified time. Developing, testing and debugging of an RTOS based embedded system takes lots of time. The virtual platform provides the great advantage in developing, testing and debugging the RTOS based embedded system. This work describes the design and development of a reconfigurable virtual platform for real-time kernel. Eclipse IDE platform, NXP LPC-2129 target, OpenOCD have been chosen to develop this reconfigurable virtual platform for real-time kernels. For the selected target, GNU Toolchain has been configured as a plug-in in Eclipse IDE for both Windows and Linux to get own virtual platform. The μC/OS-II Board Support Package and the virtual timer for gdb debugger to support selected target has been developed and added in this virtual platform. This virtual platform facilitates the reconfiguration of the various RTOS such as μC/OS-II, DrRTOS and AVX. From the implementation of this design framework on real system, it can be concluded that the virtual platform increases the efficiency of the RTOS based applications. It gives the single solution for the developers to develop, execute, test, debug and verify the applications for various RTOS in a shorter span of time and with substantial reduction of developmental cost.