Search

Egocentric vision holds great promise for increasing access to visual information and improving the quality of life for blind people. While we strive to improve recognition performance, it remains difficult to identify which object is of interest to the user; the object may not even be included in the frame due to challenges in camera aiming without visual feedback. Also, gaze information, commonly used to infer the area of interest in egocentric vision, is often not dependable. However, blind users tend to include their hand either interacting with the object they wish to recognize or simply placing it in proximity for better camera aiming. We propose a method that leverages the hand as the contextual information for recognizing an object of interest. In our method, the output of a pre-trained hand segmentation model is infused to later convolutional layers of our object recognition network with separate output layers for localization and classification. Using egocentric datasets from sighted and blind individuals, we show that the hand-priming achieves more accurate localization than other approaches that encode hand information. Given only object centers along with labels, our method achieves comparable classification performance to the state-of-the-art method that uses bounding boxes with labels.

Robot manipulation, for example, pick-and-place manipulation, is broadly used for intelligent manufacturing with industrial robots, ocean engineering with underwater robots, service robots, or even healthcare with medical robots. Most traditional robot manipulations adopt 2-D vision systems with plane hypotheses and can only generate 3-DOF (degrees of freedom) pose accordingly. To mimic human intelligence and endow the robot with more flexible working capabilities, 3-D vision-based robot manipulation has been studied. However, this task is still challenging in the open world especially for general object recognition and pose estimation with occlusion in cluttered backgrounds and human-like flexible manipulation. In this article, we propose a comprehensive analysis of recent progress about the 3-D vision for robot manipulation, including 3-D data acquisition and representation, robot-vision calibration, 3-D object detection/recognition, 6-DOF pose estimation, grasping estimation, and motion planning. We then present some public datasets, evaluation criteria, comparisons, and challenges. Finally, the related application domains of robot manipulation are given, and some future directions and open problems are studied as well.

In transfer learning model, the source domain samples and target domain samples usually share the same class labels but have different distributions. In general, the existing transfer learning algorithms ignore the interclass differences and intraclass similarities across domains. To address these problems, this article proposes a transfer learning algorithm based on discriminative Fisher embedding and adaptive maximum mean discrepancy (AMMD) constraints, called discriminative Fisher embedding dictionary transfer learning (DFEDTL). First, combining the label information of source domain and part of target domain, we construct the discriminative Fisher embedding model to preserve the interclass differences and intraclass similarities of training samples in transfer learning. Second, an AMMD model is constructed using atoms and profiles, which can adaptively minimize the distribution differences between source domain and target domain. The proposed method has three advantages: 1) using the Fisher criterion, we construct the discriminative Fisher embedding model between source domain samples and target domain samples, which encourages the samples from the same class to have similar coding coefficients; 2) instead of using the training samples to design the maximum mean discrepancy (MMD), we construct the AMMD model based on the relationship between the dictionary atoms and profiles; thus, the source domain samples can be adaptive to the target domain samples; and 3) the dictionary learning is based on the combination of source and target samples which can avoid the classification error caused by the difference among samples and reduce the tedious and expensive data annotation. A large number of experiments on five public image classification datasets show that the proposed method obtains better classification performance than some state-of-the-art dictionary and transfer learning methods. The code has been available at https://github.com/shilinrui/DFEDTL.

Object detection in underwater acoustics especially sea floor object has been overwhelming mission chiefly owing to strident environment of sonar images as well as because of visibly existing sonar images. Side Scan Sonar is the primary sensor for Autonomous Underwater Vehicles to perform survey on sea water. Hence, we are using this SSS images for categorizing several objects like sand, mud, clay, graves, ridges and sediments in underwater sea through any size subsequent to training. We applied two-layer CNN architecture to train the model as well as we utilized three pre-trained network models such as VGG-19, ResNet50 and EfficientNet model for evaluating the performance of the model based on training and validation accuracy measures. Moreover, we utilized deep learning based transfer learning approach in which the parameters are tuned for classifying the images into sediments, clay, mud, stones etc. Our experimental outcomes shows that pre-trained EfficientNet model generates better accuracy of 100% after fine tuning the parameter in object recognition along with classification using SSS images.

With the emergence of various techniques involved in deep learning the researchers of computer vision tends to focus on the strategies such as object recognition and segmentation of image. This has inclined to accomplish the deep learning techniques in 3D reconstruction of both specific and generic objects. As the space for reconstruction of 3D images either in single or multi view has envisioned the researchers to concentrate on the available technologies used for reconstruction. With the available built in methods and technologies in deep learning, the performance of the proposed methods were reviewed and analyzed using several parameters. As the remaking of 2D images is still in the beginning stage, it is important to study the 3D shape representations, various network architecture, methodologies and approaches behind 3D reconstruction. In this work a review of deep learning methods for single or multiple RGB images of specific and generic object 3D reconstruction was done. Several methods and their importance were also discussed along with the challenges encountered and with further research directions. This paper critically analyzes the various 3D Shaped Representations, 3D Data Network Architectures, Depth Estimation methods, Multi View Representations and the Data Representation Techniques.

Event-based object recognition has drawn increasing attention for event cameras’ distinguished advantages of low power consumption and high dynamic range. For this new modality, previous works based on customizing low-level descriptors are vulnerable to noise and with limited generalizability. Although recent works turn to design various deep neural networks to extract event features, they either suffer from data insufficiency to fully train the event-based model or fail to encode spatial and temporal cues simultaneously with their single view network. In this work, we address these limitations by proposing a multi-view attention-aware network, in which an event stream is projected to multi-view 2D maps to utilize well-trained 2D models and explore spatio-temporal complements. Besides, the attention mechanism is used to boost the complements in different streams for better joint inference. Comprehensive experiments show the large superiority of our model over state-of-the-art methods as well as the efficacy of our multi-view fusion framework for event data.

Event cameras sense brightness changes in each pixel and yield asynchronous event streams instead of producing intensity images. They have distinct advantages over conventional cameras, such as a high dynamic range (HDR) and no motion blur. To take advantage of event cameras with existing image-based algorithms, a few methods have been proposed to reconstruct images from event streams. However, the output images have a low resolution (LR) and are unrealistic. Low-quality outputs stem from broader applications of event cameras, where high-quality and high-resolution (HR) images are needed. In this work, we consider the problem of reconstructing and super-resolving images from LR events when no ground truth (GT) HR images and degradation models are available. We propose a novel end-to-end joint framework for single image reconstruction and super-resolution from LR event data. Our method is primarily unsupervised to handle the absence of real inputs from GT and deploys adversarial learning. To train our framework, we constructed an open dataset, including simulated events and real-world images. The use of the dataset boosts the network performance, and the network architectures and various loss functions in each phase help improve the quality of the resulting image. Various experiments showed that our method surpasses the state-of-the-art LR image reconstruction methods for real-world and synthetic datasets. The experiments for super-resolution (SR) image reconstruction also substantiate the effectiveness of the proposed method. We further extended our method to more challenging problems of HDR, sharp image reconstruction, and color events. In addition, we demonstrate that the reconstruction and super-resolution results serve as intermediate representations of events for high-level tasks, such as semantic segmentation, object recognition, and detection. We further examined how events affect the outputs of the three phases and analyze our method's efficacy through an ablation study.

This article presents a versatile soft robotic gripper system whereby its fingers can be reconfigured into different poses such as scoop, pinch, and claw. This allows the gripper to efficiently and safely handle food samples of different shapes, sizes and stiffness such as uncooked tofu and broccoli floret. The 3D-printed fingers were tested to last up to 25 000 cycles without significant changes in the curvature profile and force output profile. A benchmark experiment was conducted to evaluate the performance of the gripper and state-of-the-art gripping solutions. Capability of versatile soft gripper was optimized by integrating vision and tactile sensing facilities. An object recognition system was developed to identify food samples such as potato, broccoli, and sausage. Position and orientation of food samples were identified and pick-and-place pathway was optimized to achieve the best gripping performance. Flexible tactile sensors were integrated into soft fingers and closed-loop force feedback control system was developed. This allowed the gripper to automatically explore and select the most stable grip pose for different food samples. Integration of vision and force feedback system ensure that objects detected by the system would be firmly gripped. The reconfigurable soft robotic gripper system has been demonstrated to perform high-speed pick-and-place tasks (∼3 s per item) with object recognition system, making it a potential solution to food and grocery supply chain needs.

Acoustic fingerprinting aims to identify a mobile device based on its internal microphone(s) and speaker(s) which are unique due to manufacturing imperfection. This paper seeks a thorough understanding of the (in)security of exploring acoustic fingerprints for achieving distributed mobile authentication. Our contributions are threefold. First, we present a new acoustic fingerprint-emulation attack and demonstrate that it is a common vulnerability of acoustic mobile authentication systems. Second, we propose a dynamic challenge-response defense to secure acoustic mobile authentication systems against the acoustic fingerprint-emulation attack. Finally, we thoroughly investigate existing acoustic fingerprinting schemes and identify the best option for accurate, secure, and deployable acoustic mobile authentication systems.

Lately, variational autoencoder (VAE) has arisen as one of the most prevalent methods for unsupervised learning of intricate distributions. Despite being successful in deep feature extraction and uncertain data modelling, it still suffers from the instability and reconstruction error due to random sampling in the latent subspace representation of original input space. In this paper, to deal with those limitations, constrained VAE (CVAE) is proposed by utilizing input sample information. Enthused by parallel interaction mechanism between the ventral and dorsal stream of human brain in object recognition, parallel interaction spatial-temporal CVAE (PIST-CVAE) is proposed to extract spatial and temporal features from input samples. Lower dimensional nonlinear features extracted from PIST-CVAE are used to build the soft sensor. The effectiveness of CVAE and PIST-CVAE is demonstrated in an industrial case study, a polyester polymerization process.

Currently, visual-tactile fusion learning for robotic object recognition has achieved appealing performance, due to the fact that visual and tactile data can offer complementary information. However: 1) the distinct gap between vision and touch makes it difficult to fully explore the complementary information, which would further lead to performance degradation and 2) most of the existing visual-tactile fused learning methods assume that visual and tactile data are complete, which is often difficult to be satisfied in many real-world applications. In this article, we propose a partial visual-tactile fused (PVTF) framework for robotic object recognition to address these challenges. Specifically, we first employ two modality-specific (MS) encoders to encode partial visual-tactile data into two incomplete subspaces (i.e., visual subspace and tactile subspace). Then, a modality gap mitigated (MGM) network is adopted to discover modality-invariant high-level label information, which is utilized to generate gap loss and further help updating the MS encoders for relatively consistent visual and tactile subspaces generation. In this way, the huge gap between vision and touch is mitigated, which would further contribute to mine the complementary visual-tactile information. Finally, to achieve data completeness and complementary visual-tactile information exploration simultaneously, a cycle subspace leaning technique is proposed to project the incomplete subspaces into a complete subspace by fully exploiting all the obtainable samples, where complete latent representations with maximum complementary information can be learned. A lot of comparative experiments conducted on three visual-tactile datasets validate the advantage of the proposed PVTF framework, by comparing with state-of-the-art baselines.

Novel high-resolution pressure-sensor arrays allow treating pressure readings as standard images. Computer vision algorithms and methods such as convolutional neural networks (CNN) can be used to identify contact objects. In this paper, a high-resolution tactile sensor has been attached to a robotic end-effector to identify contacted objects. Two CNN-based approaches have been employed to classify pressure images. These methods include a transfer learning approach using a pre-trained CNN on an RGB-images dataset and a custom-made CNN (TactNet) trained from scratch with tactile information. The transfer learning approach can be carried out by retraining the classification layers of the network or replacing these layers with an SVM. Overall, 11 configurations based on these methods have been tested: eight transfer learning-based, and three TactNet-based. Moreover, a study of the performance of the methods and a comparative discussion with the current state-of-the-art on tactile object recognition is presented.

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems. The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

Deep neural networks (DNNs) are promising models of the cortical computations supporting human object recognition. However, despite their ability to explain a significant portion of variance in neural data, the agreement between models and brain representational dynamics is far from perfect. We address this issue by asking which representational features are currently unaccounted for in neural timeseries data, estimated for multiple areas of the ventral stream via source-reconstructed magnetoencephalography (MEG) data acquired in human participants (9 females, 6 males) during object viewing. We focus on the ability of visuo-semantic models, consisting of human-generated labels of object features and categories, to explain variance beyond the explanatory power of DNNs alone. We report a gradual reversal in the relative importance of DNN versus visuo-semantic features as ventral-stream object rep-resentations unfold over space and time. While lower-level visual areas are better explained by DNN features, especially during the early phase of the response (< 128 ms after stimulus onset), higher-level cortical dynamics are best accounted for by visuo-semantic features during a later time window (starting 146 ms after stimulus onset). Among the visuo-semantic features, object parts and basic categories drive the advantage over DNNs. These results show that a significant component of the variance unexplained by DNNs in higher-level cortical dynamics is structured, and can be explained by readily nameable aspects of the objects. We conclude that current DNNs fail to fully capture dynamic representations in higher-level human visual cortex and suggest a path toward more accurate models of ventral stream computations.SIGNIFICANCE STATEMENTWhen we view objects such as faces and cars in our visual environment, their neural representations dynamically unfold over time at a millisecond scale. These dynamics reflect the cortical computations that support fast and robust object recognition. Deep neural networks (DNNs) have emerged as a promising framework for modeling these computations but cannot yet fully account for the neural dynamics. Using magnetoencephalography data acquired in human observers during object viewing, we show that readily nameable aspects of objects, such as “eye”, “wheel”, and “face”, can account for variance in the neural dynamics over and above DNNs. These findings suggest that DNNs and humans may in part rely on different object features for visual recognition and provide guidelines for model improvement.

To enhance the accuracy of object recognition, various combination of recognition algorithms are used in recent literature. In this paper coherence of Discrete Wavelet Transform (DWT) is combined with four different algorithms: error vector of principal component analysis (PCA), eigen vector of PCA, eigen vector of Linear Discriminant Analysis (LDA) and Convolutional Neural Network (CNN) then combination of four results are done using entropy of detection probability and Fuzzy system. From this research the accuracy of recognition is found dependent on image and diversity of database. The combined method of the paper provides recognition rate of 89.56% for the worst case and 93.34% for the best case both can be said better in comparison with the previous works where individual method has been implemented on a specific set of images.

There is recent evidence for a domain-general object recognition ability, called O, which is distinct from general intelligence and other cognitive and personality constructs. We extend the study of O by characterizing how it generalizes to the ability to recognize familiar objects and to the ability to make judgments of the average identity of ensembles of objects. We applied latent variable modeling to data collected from a sample of adults (N = 284) in three different tasks and for six different object domains (three novel and three familiar). The results replicated prior work in finding that on average 88% of the variance of lower-order factors could be accounted by O for novel objects. The latent constructs recruited by the higher-order factor for novel objects and for familiar objects were almost perfectly correlated and therefore functionally identical. A latent factor for ensemble perception shared about 42% of the variance with O, suggesting at least strong overlap between abilities supporting judgments about individual objects and ensemble of objects. This work extends the theoretical reach of O by showing generalization across two dimensions (familiar vs. novel objects; individual vs. ensemble object perception). With respect to the structure of individual differences in high-level vision, researchers would benefit from accounting for the contribution of O when seeking to understand various domain-specific abilities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

3D object recognition is a hot research topic. Particularly, view-based methods, which represent a 3D object with a collection of its rendered views on the 2D domain, play an important role in this field. Currently, view-based researches tend to aggregate information from multiple views via pooling based strategies to endow the models with the characteristic of view permutation invariance, at the cost of inevitable loss of useful features. In this paper, we introduce a new method that learns a more comprehensive descriptor for a 3D object from its views while successfully keeping its robustness to the variation of view permutation. Our method disentangles the information in the set of multi-view images into a global category-related feature and a set of view-permutation related features. To unbind these two parts, an encode-decoder based disentangling architecture is proposed, which barely bring extra computations compared to the baseline model. Systematic experiments are conducted for this new method to demonstrates the effectiveness and the competitive performance based on ModelNet40, ModelNet10, and ShapeNetCore55 datasets. Codes for our paper will be released soon on “https://github.com/hjjpku/multi view sort”.

With the popularization of Internet of Things (IoT) devices in smart home and industry fields, a huge number of IoT devices are connected to the Internet. However, what devices are connected to a network may not be known by the Internet Service Provider (ISP), since many IoT devices are placed within small networks (e.g., home networks) and are hidden behind network address translation (NAT). Without pinpointing IoT devices in a network, it is unlikely for the ISP to appropriately configure security policies and effectively manage the network. Additionally, inferring fine-grained user interactions of IoT devices is also an interesting yet unresolved problem. In this paper, we design an efficient and scalable system via spatial-temporal traffic fingerprinting from an ISP's perspective in consideration of practical issues like learning-testing asymmetry. Our system can accurately identify typical IoT devices in a network, with the additional capability of identifying what devices are hidden behind NAT and the number of each type of device that share the same IP address. Our system can also detect user interactions and meanwhile identify their (concurrent) number through a multi-output regression model. Through extensive evaluation, we demonstrate that the system can generally identify IoT devices with an F1-Score above 0.999, and estimate the number of the same type of IoT device behind NAT with an average error below 5%. By studying 29 user interactions of 7 devices, we show that our system is promising in detecting user interactions.

Applicable human-robot collaboration requires intuitive recognition of human intention during shared work. A grasped object such as a tool held by the human provides vital information about the upcoming task. In this letter, we explore the use of a wearable device to non-visually recognize objects within the human hand in various possible grasps. The device is based on Force-Myography (FMG) where simple and affordable force sensors measure perturbations of forearm muscles. We propose a novel Deep Neural-Network architecture termed Flip-U-Net inspired by the familiar U-Net architecture used for image segmentation. The Flip-U-Net is trained over data collected from several human participants and with multiple objects of each class. Data is collected while manipulating the objects between different grasps and arm postures. The data is also pre-processed with data augmentation and used to train a Variational Autoencoder for dimensionality reduction mapping. While prior work did not provide a transferable FMG-based model, we show that the proposed network can classify objects grasped by multiple new users without additional training efforts. Experiment with 12 test participants show classification accuracy of approximately 95% over multiple grasps and objects. Correlations between accuracy and various anthropometric measures are also presented. Furthermore, we show that the model can be fine-tuned to a particular user based on an anthropometric measure.

Existing multi-task models for object recognition and segmentation have verified the effectiveness of joint optimization of two semantic tasks. However, learning discriminative representations with insufficient training data and redundant contextual information from the background remains challenging. Semantic attributes are designed as powerful and informative mid-level features that 1) share information across categories to model the interclass correlation and that 2) can be localized in the object region to benefit foreground extraction. This paper introduces a novel attribute-aware feature encoding (AFE) module to a multi-task network for object recognition and segmentation with the aim of improving both semantic tasks by regularizing feature encoding with auxiliary attribute learning. Intuitively, attribute learning in our method not only provides extra supervision signals to capture interclass correlation in object classification but also refines the output of object segmentation via weakly supervised attribute localization. The experimental results on two public benchmarks show that our method yields remarkable improvement in both semantic tasks and auxiliary attribute estimation over existing methods.

Public speaking is a critical skill in daily communication. While more practicing such as rehearsal is helpful to improve such a skill, lack of personalized feedback limits the effectiveness of practicing. Therefore, we formulate the task of personalized feedback as an affective audio annotation problem by learning knowledge from online public speech videos. Considering the great success of deep learning techniques such as convolutional neural networks in a wide range of applications including speech recognition and object recognition, we propose a novel convolutional clustering neural network (CCNN) to solve this multi-label classification problem. Instead of aggregating the features of different channels through pooling, we introduce a novel clustering layer to derive intermediate representation for improved annotation performance. In order to evaluate the performance of our proposed method, we purposely built an affective audio annotation dataset by collecting more than 2,000 video clips from the TED website. Experimental results on this dataset demonstrate that our proposed method outperforms traditional CNN-based approaches with a lower hamming loss for affective annotation.

RGB-D object recognition is an essential but challenging task in computer vision. Most existing methods often exploit popular neural network architectures and various multi-modal fusion manners, which fail to explore the relation between low-level and high-level feature representations. To address this issue, we propose a convolutional recurrent fusion method for RGB-D object recognition. Firstly, a novel regularization strategy, called CutResize, is utilized to generate more reliable multi-scale RGB-D fusion input. Secondly, a convolutional attention gate recurrent unit (ConvAGRU) is proposed to fuse multi-modal features, which can capture the relation between low-level and high-level feature representations. Finally, the network recognizes RGB-D objects in an end-to-end manner. Especially, the CutResize uses the scaled image from another patch to mix with the input image, which avoids the mixed image only contains background and improves the localization ability of the model for small-scale features. The proposed ConvAGRU enhances the context correlation of the network and produces more compact and robust multi-modal fusion features. The proposed method is evaluated on several public datasets (including CIFAR-10, CIFAR-100, JHUIT-50, and OCID datasets). The experiment results show that the proposed method obtains better recognition performance than existing methods and achieves state-of-the-art results.

Object detection and instance segmentation with a large number of object categories and long-tailed data distribution are challenging for most existing deep learning models. As the number of classes increases, the outputs of a classifier become sensitive to likely noisy logits, which can easily result in an incorrect recognition. To alleviate the large-vocabulary problem, we cluster fine-grained classes into coarser parent classes and then build a classification tree to classify an object into a fine-grained class via its parent class. Because the number of parent class is much fewer, their logits are more stable to suppress the wrong/noisy logits existed in the fine-grained class nodes. Due to a variety of ways for clustering fine-grained classes into parent classes, we can further construct multiple trees to build a classification forest where each single tree contributes its vote to the fine-grained classification. Moreover, a simple yet effective resampling method, termed as NMS Resampling, is proposed aiming at solving the long tail (data imbalance) problem. Our method, coined as ForestDet, serves as a plug-and-play module, which can be readily employed in both one-stage and two-stage object recognition models for recognizing more than 1000 categories. Extensive experiments are conducted on the large vocabulary dataset LVIS. Compared to the Mask R-CNN baseline, our two-stage counterpart Forest R-CNN significantly boosts the performance by 11.5% and 3.9% AP improvements on the rare categories and overall categories, respectively. Compared to the RetinaNet baseline, our one-stage counterpart Forest RetinaNet improves 2.1% AP on overall categories. Moreover, we achieve state-of-the-art results on the LVIS dataset. Code and models are available at https://github.com/JialianW/Forest_RCNN.

For various remote sensing objects, the current detection framework based on a single detection pipeline fails to provide satisfactory detection accuracy. In order to further improve the object recognition ability of the detection model, this article introduces the effective ``multiexpert'' mechanism into the field of remote sensing object detection and then constructs a multiexpert detection network (MEDNet). In this model, we first construct multiple feature pyramids (MFPs) to replace the traditional single feature pyramid to enrich the semantic representation ability of the model. Then, we equip multiple detection experts (MDEs) to leverage multiple kinds of features from MFP to perform different semantic predictions. As the first CNN-based multiexpert detection model for remote sensing images, we tailor a loss distance-based k-experts clustering (LD-kEC) strategy to assign training samples to different detection experts in an unsupervised fashion. By this strategy, we can directly use the existing remote sensing dataset without expert labels for end-to-end training of our multiexpert model. The experimental results prove that the proposed multiexpert-based detector can indeed significantly improve the object detection performance for remote sensing images.

To investigate the identification method of magnetic moving targets and provide theoretical explanations, a magnetic target motion model was established based on the magnetic dipole model, incorporating the target moving velocity, direction angle, and CPA (the Close Proximity Approach), and magnetic signature waveforms were obtained under different conditions. The control variable method was used to study the correlation between the magnetic signature waveforms and the target moving speed, direction angle, and CPA, as the basis for realizing the recognition of the magnetic moving target. The highly sensitive tunneling magnetoresistive sensor (TMR) was used for experiments, and the results are in agreement with simulations. The waveform width and peak value have an obvious corresponding relationship with the above three motion states, which provides an efficient method for extracting the motion state of a moving target.

Traditional super-resolution (SR) methods by minimize the mean square error usually produce images with oversmoothed and blurry edges, due to the lack of high-frequency details. In this paper, we propose two novel techniques within the generative adversarial network framework to encourage generation of photo-realistic images for image super-resolution. Firstly, instead of producing a single score to discriminate real and fake images, we propose a variant, called Fine-grained Attention Generative Adversarial Network (FASRGAN), to discriminate each pixel of real and fake images. FASRGAN adopts a UNetlike network as the discriminator with two outputs: an image score and an image score map. The score map has the same spatial size as the HR/SR images, serving as the fine-grained attention to represent the degree of reconstruction difficulty for each pixel. Secondly, instead of using different networks for the generator and the discriminator, we introduce a feature-sharing variant (denoted as Fs-SRGAN) for both the generator and the discriminator. The sharing mechanism can maintain model express power while making the model more compact, and thus can improve the ability of producing high-quality images. Quantitative and visual comparisons with state-of-the-art methods on benchmark datasets demonstrate the superiority of our methods. We further apply our super-resolution images for object recognition, which further demonstrates the effectiveness of our proposed method. The code is available at https://github.com/Rainyfish/FASRGAN-and-Fs-SRGAN.

Achieving human-like visual abilities is a holy grail for machine vision, yet precisely how insights from human vision can improve machines has remained unclear. Here, we demonstrate two key conceptual advances: First, we show that most machine vision models are systematically different from human object perception. To do so, we collected a large dataset of perceptual distances between isolated objects in humans and asked whether these perceptual data can be predicted by many common machine vision algorithms. We found that while the best algorithms explain ∼ 70 percent of the variance in the perceptual data, all the algorithms we tested make systematic errors on several types of objects. In particular, machine algorithms underestimated distances between symmetric objects compared to human perception. Second, we show that fixing these systematic biases can lead to substantial gains in classification performance. In particular, augmenting a state-of-the-art convolutional neural network with planar/reflection symmetry scores along multiple axes produced significant improvements in classification accuracy (1-10 percent) across categories. These results show that machine vision can be improved by discovering and fixing systematic differences from human vision.

Haptic and proprioceptive feedback is critical for sensorimotor integration when we use our hand to perform daily tasks. Here, we evaluated how externally evoked haptic and proprioceptive feedback and myoelectric control strategies affected the recognition of object properties when participants controlled a prosthetic hand. Fingertip haptic sensation was elicited using a transcutaneous nerve stimulation grid to encode the prosthetic's fingertip forces. An array of tactors elicited patterned vibratory stimuli to encode tactile-proprioceptive kinematic information of the prosthetic finger joint. Myoelectric signals of the finger flexor and extensor were used to control the position or velocity of joint angles of the prosthesis. Participants were asked to perform object property (stiffness and size) recognition, by controlling the prosthetic hand with concurrent haptic and tactile-proprioceptive feedback. With the evoked feedback, intact and amputee participants recognized the object stiffness and size at success rates ranging from 50% to 100% in both position and velocity control with no significant difference across control schemes. Our findings show that evoked somatosensory feedback in a non-invasive manner can facilitate closed-loop control of the prosthetic hand and allowed for simultaneous recognition of different object properties. The outcomes can facilitate our understanding on the role of sensory feedback during bidirectional human-machine interactions, which can potentially promote user experience in object interactions using prosthetic hands.

Accurate object recognition and pose estimation models are essential for practical applications of robot arms, such as picking products on a shelf. Training such a model often requires a large-scale dataset with qualified labels for both object classes and pose parameters, and collecting accurate pose labels is particularly costly. A recent paper Ueno et al.(2019)[28] proposed a triplet learning framework for joint object recognition and pose estimation without explicit pose labels by learning a spatial transformer network to estimate the pose difference of an input image from an anchor image depicting the same object in a reference pose. However, our analysis suggests that the pose estimation accuracy is severely degraded for input images with large pose differences. To address this problem, we propose a new learning approach called multiple-anchor triplet learning. The basic idea is to give dense reference poses by preparing multiple anchors so that there is at least one anchor image having a small pose difference to the input image. Our multiple-anchor triplet learning is an extension of the standard single-anchor triplet learning to the multiple-anchor case. Inspired by the idea of multiple instance learning, we introduce a selection layer that automatically chooses the best anchor for each input image and allows the network to be trained end-to-end to minimize triplet-based losses. Experiments with three benchmark datasets in product picking scenarios demonstrate that our method significantly outperforms existing methods in both object recognition and pose estimation accuracy.

In recent years, researches of remote sensing data have begun to develop towards mass information and diversification, and traditional analytical methods alone could not meet the needs of modern data processing. Deep learning methods have developed a lot in the field of computer vision which has provided a new target recognition method for remote sensing images. This paper concentrates on the application of convolutional neural networks(CNN) in object recognition of Unmanned Aerial Vehicle(UAV) remote sensing images: YOLOv3 and Faster R-CNN network model are built on computer and NVIDIA JETSON TX2, and NWPU VHR-10 dataset is used as training and test samples for simulation verification. The results show that the accuracy of remote sensing multiclass target recognition using the YOLOv3 network model is 73.8%, and the detection rate is 0.25s per image on average. The accuracy of using YOLOv3 network model in several network models is second only to Faster RCNN, but its real-time performance is better than one of Faster RCNN. The YOLOv3 network model is more appropriate for carrying on the UAVs.

Computational neuroscience models of vision and neural network models for object recognition are often framed by different research agendas. Computational neuroscience mainly aims at replicating experimental data, while (artificial) neural networks target high performance on classification tasks. However, we propose that models of vision should be validated on object recognition tasks. At some point, mechanisms of realistic neuro-computational models of the visual cortex have to convince in object recognition as well. In order to foster this idea, we report the recognition accuracy for two different neuro-computational models of the visual cortex on several object recognition datasets. The models were trained using unsupervised Hebbian learning rules on natural scene inputs for the emergence of receptive fields comparable to their biological counterpart. We assume that the emerged receptive fields result in a general codebook of features, which should be applicable to a variety of visual scenes. We report the performances on datasets with different levels of difficulty, ranging from the simple MNIST to the more complex CIFAR-10 or ETH-80. We found that both networks show good results on simple digit recognition, comparable with previously published biologically plausible models. We also observed that our deeper layer neurons provide for naturalistic datasets a better recognition codebook. As for most datasets, recognition results of biologically grounded models are not available yet, our results provide a broad basis of performance values to compare methodologically similar models.

We present an automated, visual question answering based companion – VisQuelle - to facilitate elementary learning of word-object associations. In particular, we attempt to harness the power of machine learning models for object recognition and the understanding of combined processing of images and text data from visual-question answering to provide variety and nuance in the images associated with letters or words presented to the elementary learner. We incorporate elements such as gamification to motivate the learner by recording scores, errors, etc., to track the learner’s progress. Translation is also provided to reinforce word-object associations in the user’s native tongue, if the learner is using VisQuelle to learn a second language.

Semantic edge detection (SED), which aims at jointly extracting edges as well as their category information, has far-reaching applications in domains such as semantic segmentation, object proposal generation, and object recognition. SED naturally requires achieving two distinct supervision targets: locating fine detailed edges and identifying high-level semantics. Our motivation comes from the hypothesis that such distinct targets prevent state-of-the-art SED methods from effectively using deep supervision to improve results. To this end, we propose a novel fully convolutional neural network using diverse deep supervision (DDS) within a multi-task framework where bottom layers aim at generating category-agnostic edges, while top layers are responsible for the detection of category-aware semantic edges. To overcome the hypothesized supervision challenge, a novel information converter unit is introduced, whose effectiveness has been extensively evaluated on SBD and Cityscapes datasets., International Journal of Computer Vision

PURPOSE: In the multi-atlas segmentation (MAS) method, a large enough atlas set, which can cover the complete spectrum of the whole population pattern of the target object will benefit the segmentation quality. However, the difficulty in obtaining and generating such a large set of atlases and the computational burden required in the segmentation procedure make this approach impractical. In this paper, we propose a method called SOMA to select subject-, object-, and modality-adapted precision atlases for automatic anatomy recognition in medical images with pathology, following the idea that different regions of the target object in a novel image can be recognized by different atlases with regionally best similarity, so that effective atlases have no need to be globally similar to the target subject and also have no need to be overall similar to the target object. METHODS: The SOMA method consists of three main components: atlas building, object recognition, and object delineation. Considering the computational complexity, we utilize an all-to-template strategy to align all images to the same image space belonging to the root image determined by the minimum spanning tree (MST) strategy among a subset of radiologically near-normal images. The object recognition process is composed of two stages: rough recognition and refined recognition. In rough recognition, subimage matching is conducted between the test image and each image of the whole atlas set, and only the atlas corresponding to the best-matched subimage contributes to the recognition map regionally. The frequency of best match for each atlas is recorded by a counter, and the atlases with the highest frequencies are selected as the precision atlases. In refined recognition, only the precision atlases are examined, and the subimage matching is conducted in a nonlocal manner of searching to further increase the accuracy of boundary matching. Delineation is based on a U-net-based deep learning network, where the original gray scale image together with the fuzzy map from refined recognition compose a two-channel input to the network, and the output is a segmentation map of the target object. RESULTS: Experiments are conducted on computed tomography (CT) images with different qualities in two body regions – head and neck (H&N) and thorax, from 298 subjects with nine objects and 241 subjects with six objects, respectively. Most objects achieve a localization error within two voxels after refined recognition, with marked improvement in localization accuracy from rough to refined recognition of 0. 6–3 mm in H&N and 0. 8–4. 9 mm in thorax, and also in delineation accuracy (Dice coefficient) from refined recognition to delineation of 0. 01–0. 11 in H&N and 0. 01–0. 18 in thorax. CONCLUSIONS: The SOMA method shows high accuracy and robustness in anatomy recognition and delineation. The improvements from rough to refined recognition and further to delineation, as well as immunity of recognition accuracy to varying image and object qualities, demonstrate the core principles of SOMA where segmentation accuracy increases with precision atlases and gradually refined object matching.

As augmented reality technologies develop, real-time interactions between objects present in the real world and virtual space are required. Generally, recognition and location estimation in augmented reality are carried out using tracking techniques, typically markers. However, using markers creates spatial constraints in simultaneous tracking of space and objects. Therefore, we propose a system that enables camera tracking in the real world and visualizes virtual visual information through the recognition and positioning of objects. We scanned the space using an RGB-D camera. A three-dimensional (3D) dense point cloud map is created using point clouds generated through video images. Among the generated point cloud information, objects are detected and retrieved based on the pre-learned data. Finally, using the predicted pose of the detected objects, other information may be augmented. Our system estimates object recognition and 3D pose based on simple camera information, enabling the viewing of virtual visual information based on object location.

In today’s world, accurate and fast information is vital for safe aircraft landings. The purpose of an EMAS (Engineered Materials Arresting System) is to prevent an aeroplane from overrunning with no human injury and minimal damage to the aircraft. Although various algorithms for object detection analysis have been developed, only a few researchers have examined image analysis as a landing assist. Image intensity edges are employed in one system to detect the sides of a runway in an image sequence, allowing the runway’s 3-dimensional position and orientation to be approximated. A fuzzy network system is used to improve object detection and extraction from aerial images. In another system, multi-scale, multiplatform imagery is used to combine physiologically and geometrically inspired algorithms for recognizing objects from hyper spectral and/or multispectral (HS/MS) imagery. However, the similarity in the top view of runways, buildings, highways, and other objects is a disadvantage of these methods. We propose a new method for detecting and tracking the runway based on pattern matching and texture analysis of digital images captured by aircraft cameras. Edge detection techniques are used to recognize runways from aerial images. The edge detection algorithms employed in this paper are the Hough Transform, Canny Filter, and Sobel Filter algorithms, which result in efficient detection.

We proposes a multiple fractal dimensions (MFD) method for robust object description. MFD is an effective feature extraction approach, which is first calculated based on a phase angle quantization method to categorize the points of the input image. And then fractal dimensions are calculated to describe the distribution of feature pattern characterized as the intrinsic property of the general objects, i.e., land scene, face and pedestrian. We theoretically proven that our MFD is shown to be invariant to local variations, i.e., Bi-Lipschitz, which is a desirable characteristic for objects, such as land-scene images, face and pedestrian due to the existence of scale variations, local variations and illumination variations in those images. The proposed method is extensively evaluated on land-use scene recognition, face recognition, expression recognition, and pedestrian detection. The experimental results on UC Merced 21-class scene dataset, AR, JAFFE and INRIA pedestrian databases show that our method achieves superior performances over several state-of-the-art methods in terms of recognition rates.

In unsupervised domain adaptation (UDA), a classifier for a target domain is trained with labeled source data and unlabeled target data. Existing UDA methods assume that the source data come from the same source domain (i.e., single-source scenario) or from multiple source domains, whose feature spaces have the same dimension ( homogeneous ) but different distributions (i.e., multihomogeneous-source scenario). However, in the real world, for a specific target domain, we probably have multiple different-dimension ( heterogeneous ) source domains, which do not satisfy the assumption of existing UDA methods. To remove this assumption and move forward to a realistic UDA problem, this article presents a shared-fuzzy-equivalence-relation neural network (SFERNN) for addressing the multisource heterogeneous UDA problem. The SFERNN is a five-layer neural network containing $c$ source branches and one target branch. The network structure of the SFERNN is first confirmed by a novel fuzzy relation called multisource shared fuzzy equivalence relation. Then, we optimize parameters of the SFERNN via minimizing cross-entropy loss on $c$ source branches and the distributional discrepancy between each source branch and the target branch. Experiments distributed across eight real-world datasets are conducted to validate the SFERNN. This testing regime demonstrates that the SFERNN outperforms the existing single-source heterogeneous UDA methods, especially when the target domain contains few data.

Generalized formalization of recognition algorithm for specialized computer systems is presented in this paper. The structure features of image recognition methods, which have to be taken into account when developing classifiers for object recognition in specialized computer systems, are described. The fundamental types of images characteristics-features, which are used in various methods of image recognition, are discussed. Approaches to development of classifiers for recognizing robotization objects, which are implemented on basis of Haar classifiers, are discussed. The issues of using machine learning algorithm of adaptive gain AdaBoost for development of such classifiers are also considered. Utilities have been developed for implementation of classifiers for object recognition in specialized computer systems.

Zero-shot learning (ZSL) realizes unseen object recognition by transferring knowledge from seen classes to unseen classes under common semantic space assumption, such as attribute space and semantic word vector space. Previous works used seen image feature and semantic representation but ignored unseen test image features to learn a projection from visual space to semantic space, which may lead to the domain shift problem, i.e., due to disjoint seen and unseen classes, the projection learnt from auxiliary dataset is biased when applied directly to the target dataset. In this paper, adversarial strategy is proposed with an instantiation to deal with domain shift problem. It is described as a two-player game in which player 1 is projector while player 2 is classifier . Projector expects to learn a projection from visual space to semantic space with good semantic preservation property while classifier expects to achieve high accuracy. The adversarial meaning comes from the design of parallel structure between loss function on training samples and that on test samples, semantic compatibility of these two loss functions and loss function in classifier, and distribution alignment. A theoretical analysis is provided and experiments are performed on benchmark datasets to ascertain the effectiveness of adversarial strategy.

Three-dimensional (3-D) object recognition task focuses on detecting the objects of a scene and estimating their 6-DOF pose via effective feature extraction methods. Most recent feature extraction methods are based on the deep neural networks and show good performances. However, these methods require rendering engine to assist in generating a large amount of training data, which need much time to converge and further lead to the block in a rapid industrial production line. Besides, for the common hand-crafted features, the lack of discriminant feature-points amongst various texture-less and surface-smooth objects can cause ambiguity in the process of feature-points matching. To address these challenges above, a hand-crafted 3-D feature descriptor with center offset and pose annotations is proposed in this article, which is called view-specific local projection statistics (VSLPSs). By relying on these annotations as seeds, a voting strategy is then used to transform the feature-points matching problem into the problem of voting an optimal model-view in the 6-DOF space. In this way, the ambiguity of feature-points matching caused by poor feature discrimination is eliminated. To the end, various experiments on three public datasets and our built 3-D bin-picking dataset demonstrate that our proposed VSLPS method performs well in comparison with the state-of-the-art.

Detection and recognition of objects in images is the main problem to be solved by computer vision systems. As part of solving this problem, the model of object recognition in aerial photographs taken from unmanned aerial vehicles has been improved. A study of object recognition in aerial photographs using deep convolutional neural networks has been carried out. Analysis of possible implementations showed that the AlexNet 2012 model (Canada) trained on the ImageNet image set (China) is most suitable for this problem solution. This model was used as a basic one. The object recognition error for this model with the use of the ImageNet test set of images amounted to 15%. To solve the problem of improving the effectiveness of object recognition in aerial photographs for 10 classes of images, the final fully connected layer was modified by rejection from 1,000 to 10 neurons and additional two-stage training of the resulting model. Additional training was carried out with a set of images prepared from aerial photographs at stage 1 and with a set of VisDrone 2021 (China) images at stage 2. Optimal training parameters were selected: speed (step) (0.0001), number of epochs (100). As a result, a new model under the proposed name of AlexVisDrone was obtained. The effectiveness of the proposed model was checked with a test set of 100 images for each class (the total number of classes was 10). Accuracy and sensitivity were chosen as the main indicators of the model effectiveness. As a result, an increase in recognition accuracy from 7% (for images from aerial photographs) to 9% (for the VisDrone 2021 set) was obtained which has indicated that the choice of neural network architecture and training parameters was correct. The use of the proposed model makes it possible to automate the process of object recognition in aerial photographs. In the future, it is advisable to use this model at ground stations of unmanned aerial vehicle complex control when processing aerial photographs taken from unmanned aerial vehicles, in robotic systems, in video surveillance complexes and when designing unmanned vehicle systems

Underwater explorations and probes have now become frequent for marine discovery and endangered resources protection. The decrease in natural light with an increase in water depth and the characteristic of the medium to absorb and scatter light pose crucial challenges to underwater vision systems. Autonomous Underwater Vehicles (AUVs) depend upon their imaging systems for navigation and environmental resource exploration. This paper proposes DeepRecog—an integrated underwater image deblurring and object recognition framework for AUV vision systems. The principle behind the image deblurring module involves a threefold approach consisting of CNNs, adaptive and transformative filters. The ensemble object detection and recognition module identifies marine life and other frequently existent underwater assets from AUV images and achieves mean Average Precision (mAP) of 0.95 and was found to be 6.42% more precise than YOLOv3, 8.43% more than FasterRCNN + VGG16 and 15.78% more than FasterRCNN. This framework was created with the purpose of providing real-time detection and recognition with minimal delay. The system can also be employed for former images acquired from AUVs and hopes to facilitate efficient solutions for marine image post-processing.

The existing view-based 3D object classification and recognition methods ignore the inherent hierarchical correlation and distinguishability of views, making it difficult to further improve the classification accuracy. In order to solve this problem, this paper proposes an end-to-end multi-view dual attention network framework for high-precision recognition of 3D objects. On one hand, we obtain three feature layers of query, key, and value through the convolution layer. The spatial attention matrix is generated by the key-value pairs of query and key, and each feature in the value of the original feature space branch is assigned different importance, which clearly captures the prominent detail features in the view, generates the view space shape descriptor, and focuses on the detail part of the view with the feature of category discrimination. On the other hand, a channel attention vector is obtained by compressing the channel information in different views, and the attention weight of each view feature is scaled to find the correlation between the target views and focus on the view with important features in all views. Integrating the two feature descriptors together to generate global shape descriptors of the 3D model, which has a stronger response to the distinguishing features of the object model and can be used for high-precision 3D object recognition. The proposed method achieves an overall accuracy of 96.6% and an average accuracy of 95.5% on the open-source ModelNet40 dataset, compiled by Princeton University when using Resnet50 as the basic CNN model. Compared with the existing deep learning methods, the experimental results demonstrate that the proposed method achieves state-of-the-art performance in the 3D object classification accuracy.