Search

Deep Learning (DL) revolutionized a wide variety of research fields and applications of Artificial Intelligence (AI) in the last years. Computer vision is among those fields that witnessed a stronger impact, especially in some tasks like image equalization, classification and segmentation, that have several applications in medical imaging. Such applications often focus on microscopy images, but they can be successfully extended to other fields, like the detection of diseases involving the ocular surface and the cornea. One of the reasons of the success of deep learning applied to image processing relies in the end‐to‐end data‐driven approach. This means that the whole image processing and classification/segmentation pipeline can be adapted to the data being processed. This is particularly powerful when dealing with medical images, whose processing is often based on the detection of low‐ and mid‐level patterns that might be difficult to define in a formal manner. This means that deep learning has the chance to select and consider some image elements that would be nearly impossible to define using standard computer vision techniques. However, this becomes possible if large amounts of data are available for training the network, which became reality in the last years thanks to the wide digitalization of medical systems, including ophthalmology equipment. Based on a substantial amount of literature that has rapidly grown in the last years, this talk will focus on the new perspectives of eye disease diagnosis that were opened by the advent of deep learning. At the same time, the limitations and risks that must be considered in the development of DL‐based medical systems, like overfitting and lack of generalization, will be highlighted and discussed. Keeping track of such limitations is crucial in order not to consider DL as a sort of magic box, but rather, as powerful image analysis and processing tool. The talk will also provide a qualitative and quantitative evaluation of the diagnosis systems currently available that exploit DL and AI systems. The current and future impact of automatic image processing systems dedicated to ophthalmology will also be discussed. When automatic medical image analysis will be widely available, the patterns of medical diagnosis will need to change, taking advantage of the possibility to perform larger screenings that will be automatically processed. [ABSTRACT FROM AUTHOR]

This work presents a generic computer vision system designed for exploiting trained deep Convolutional Neural Networks (CNN) as a generic feature extractor and mixing these features with more traditional hand-crafted features. Such a system is a single structure that can be used for synthesizing a large number of different image classification tasks. Three substructures are proposed for creating the generic computer vision system starting from handcrafted and non-handcrafter features: i) one that remaps the output layer of a trained CNN to classify a different problem using an SVM; ii) a second for exploiting the output of the penultimate layer of a trained CNN as a feature vector to feed an SVM; and iii) a third for merging the output of some deep layers, applying a dimensionality reduction method, and using these features as the input to an SVM. The application of feature transform techniques to reduce the dimensionality of feature sets coming from the deep layers represents one of the main contributions of this paper. Three approaches are used for the non-handcrafted features: deep transfer learning features based on convolutional neural networks (CNN), principal component analysis network (PCAN), and the compact binary descriptor (CBD). For the handcrafted features, a wide variety of state-of-the-art algorithms are considered: Local Ternary Patterns, Local Phase Quantization, Rotation Invariant Co-occurrence Local Binary Patterns, Completed Local Binary Patterns, Rotated local binary pattern image, Globally Rotation Invariant Multi-scale Co-occurrence Local Binary Pattern, and several others. The computer vision system based on the proposed approach was tested on many different datasets, demonstrating the generalizability of the proposed approach thanks to the strong performance recorded. The Wilcoxon signed rank test is used to compare the different methods; moreover, the independence of the different methods is studied using the Q-statistic. To facilitate replication of our experiments, the MATLAB source code will be available at ( https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0 ). [ABSTRACT FROM AUTHOR]

Thanks to the increasing popularity of 3D sensors, robotic vision has experienced huge improvements in a wide range of applications and systems in the last years. Besides the many benefits, this migration caused some incompatibilities with those systems that cannot be based on range sensors, like intelligent video surveillance systems, since the two kinds of sensor data lead to different representations of people and objects. This work goes in the direction of bridging the gap, and presents a novel re-identification system that takes advantage of multiple video flows in order to enhance the performance of a skeletal tracking algorithm, which is in turn exploited for driving the re-identification. A new, geometry-based method for joining together the detections provided by the skeletal tracker from multiple video flows is introduced, which is capable of dealing with many people in the scene, coping with the errors introduced in each view by the skeletal tracker. Such method has a high degree of generality, and can be applied to any kind of body pose estimation algorithm. The system was tested on a public dataset for video surveillance applications, demonstrating the improvements achieved by the multi-viewpoint approach in the accuracy of both body pose estimation and re-identification. The proposed approach was also compared with a skeletal tracking system working on 3D data: the comparison assessed the good performance level of the multi-viewpoint approach. This means that the lack of the rich information provided by 3D sensors can be compensated by the availability of more than one viewpoint. [ABSTRACT FROM AUTHOR]

In this work, we propose an unorthodox approach for describing a given image. Each image is represented by a feature vector whose elements are the scores assigned to object classes by deep convolutional neural networks that were not related to those that built the given image classification problem. The deep neural networks are trained using 1000 classes; therefore, each image is described by 1000 scores, which are fed to a support vector machine. The proposed approach could be considered a transfer learning method, where, instead of repurposing the learned features to a second classification problem, we use the scores obtained by trained convolutional neural networks. Methods based on state of the art handcrafted descriptors, and the novel approach presented here are compared, together with selected ensembles of such methods. The fusion between a standard approach and the new unorthodox method boosts the performance of the standard approach. The Wilcoxon signed rank test is used to compare the different methods. The novel method is applied to 21 different datasets to demonstrate its generality. The MATLAB source code to replicate our experiments will be available at ( https://www.dei.unipd.it/node/2357 +Pattern Recognition and Ensemble Classifiers). [ABSTRACT FROM AUTHOR]

Cracks are the main source of failure in the production of metal parts: systems for checking their presence are therefore crucial for defect-free production. In this paper, an autonomous system for performing this quality control is presented. The system is equipped with a heating tool, a thermocamera, and a robot to handle the part. The inspection process is based on the observation of the propagation of thermal waves through the inspected part, a method that can highlight very small cracks with high reliability. A knowledge-based approach to visual inspection is exploited for detecting the cracks: all the system parameters are known by means of an accurate calibration of the workcell. The system was tested on a large dataset and demonstrated its capability of detecting tiny production defects, that can lead to dangerous failures when the metal components are put under strong mechanical stress. [ABSTRACT FROM AUTHOR]

Multi-camera systems are an effective solution for perceiving large areas or complex scenarios with many occlusions. In such a setup, an accurate camera network calibration is crucial in order to localize scene elements with respect to a single reference frame shared by all the viewpoints of the network. This is particularly important in applications such as object detection and people tracking. Multi-camera calibration is a critical requirement also in several robotics scenarios, particularly those involving a robotic workcell equipped with a manipulator surrounded by multiple sensors. Within this scenario, the robot-world hand-eye calibration is an additional crucial element for determining the exact position of each camera with respect to the robot, in order to provide information about the surrounding workspace directly to the manipulator. Despite the importance of the calibration process in the two scenarios outlined above, namely (i) a camera network, and (ii) a camera network with a robot, there is a lack of standard datasets available in the literature to evaluate and compare calibration methods. Moreover they are usually treated separately and tested on dedicated setups. In this paper, we propose a general standard dataset acquired in a robotic workcell where calibration methods can be evaluated in two use cases: camera network calibration and robot-world hand-eye calibration. The Multi-Eye To Robot Indoor Calibration (METRIC) dataset consists of over 10,000 synthetic and real images of ChAruCo and checkerboard patterns, each one rigidly attached to the robot end-effector, which was moved in front of four cameras surrounding the manipulator from different viewpoints during the image acquisition. The real images in the dataset includes several multi-view image sets captured by three different types of sensor networks: Microsoft Kinect V2, Intel RealSense Depth D455 and Intel RealSense Lidar L515, to evaluate their advantages and disadvantages for calibration. Furthermore, in order to accurately analyze the effect of camera-robot distance on calibration, we acquired a comprehensive synthetic dataset, with related ground truth, with three different camera network setups corresponding to three levels of calibration difficulty depending on the cell size. An additional contribution of this work is to provide a comprehensive evaluation of state-of-the-art calibration methods using our dataset, highlighting their strengths and weaknesses, in order to outline two benchmarks for the two aforementioned use cases. [ABSTRACT FROM AUTHOR]

In this paper, an automatic optical inspection system for checking the sequence of colored wires in electric cable is presented. The system is able to inspect cables with flat connectors differing in the type and number of wires. This variability is managed in an automatic way by means of a self-learning subsystem and does not require manual input from the operator or loading new data to the machine. The system is coupled to a connector crimping machine and once the model of a correct cable is learned, it can automatically inspect each cable assembled by the machine. The main contributions of this paper are: (i) the self-learning system; (ii) a robust segmentation algorithm for extracting wires from images even if they are strongly bent and partially overlapped; and (iii) a color recognition algorithm able to cope with highlights and different finishing of the wire insulation. We report the system evaluation over a period of several months during the actual production of large batches of different cables; tests demonstrated a high level of accuracy and the absence of false negatives, which is a key point in order to guarantee defect-free productions. [ABSTRACT FROM PUBLISHER]

This paper presents an ambient intelligence system designed for assisted living. The system processes the audio and video data acquired from multiple sensors spread in the environment to automatically detect dangerous events and generate automatic warning messages. The paper presents the distributed perception infrastructure that has been implemented by means of an open-source software middleware called NMM. Different processing nodes have been developed which can cooperate to extract high level information about the environment. Examples of implemented nodes running algorithms for people detection or face recognition are presented. Experiments on novel algorithms for people fall detection and sound classification and localization are discussed. Eventually, we present successful experiments in two test bed scenarios. [ABSTRACT FROM AUTHOR]

This paper focuses on the key role played by the adoption of a framework in teaching robotics with a computer science approach in the master in Computer Engineering. The framework adopted is the Robot Operating System (ROS), which is becoming a standard de facto inside the robotics community. The educational activities proposed in this paper are based on a constructionist approach. The Mindstorms NXT robot kit is adopted to trigger the learning challenge. The ROS framework is exploited to drive the students programming methodology during the laboratory activities and to allow students to exercise with the major computer programming paradigms and the best programming practices. The major robotics topics students are involved with are: acquiring data from sensors, connecting sensors to the robot, and navigate the robot to reach the final goal. The positive effects given by this approach are highlighted in this paper by comparing the work recently produced by students with the work produced in the previous years in which ROS was not yet adopted and many different software tools and languages were used. The results of a questionnaire are reported showing that we achieved the didactical objectives we expected as instructors. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a crucial task in emerging robotic applications like autonomous driving and social robotics. State-of-the-art methods in this field rely on deep learning, with several works in the literature following the trend of using larger networks to achieve higher performance. However, this leads to greater model complexity and higher computational costs, which make it difficult to integrate such models on mobile robots. In this work we investigate how it is possible to obtain lighter performing deep models introducing additional data at a very low computational cost, instead of increasing the network complexity. We consider the features used in the 3D Entangled Forests algorithm, proposing different strategies to integrate such additional information into different deep networks. The new features allow to obtain lighter and performing segmentation models, either by shrinking the network size or improving existing networks proposed for real-time segmentation. Such result represents an interesting alternative in mobile robotics application, where computational power and energy are limited. [ABSTRACT FROM AUTHOR]

Recognizing human actions is crucial for an effective and safe collaboration between humans and robots. For example, in a collaborative assembly task, human workers can use gestures to communicate with the robot, and the robot can use the recognized actions to anticipate the next steps in the assembly process, leading to improved safety and productivity. In this work, we propose a general framework for human action recognition based on 3D pose estimation and ensemble techniques, which allows to recognize both body actions and hand gestures. The framework relies on OpenPose and 2D to 3D lifting methods to estimate 3D joints for the human body and the hands, feeding then these joints into a set of graph convolutional networks based on the Shift-GCN architecture. The output scores of all networks are combined using an ensemble approach to predict the final human action. The proposed framework was evaluated on a custom dataset designed for human–robot collaboration tasks, named IAS-Lab Collaborative HAR dataset. The results showed that using an ensemble of action recognition models improves the accuracy and robustness of the overall system; moreover, the proposed framework can be easily specialized on different scenarios and achieve state-of-the-art results on the HRI30 dataset when coupled with an object detector or classifier. • General framework for human action recognition based on 3D pose estimation. • 3D pose estimation based on 2D pose estimator and 2D to 3D lifting methods. • Ensemble of skeleton-based classifiers to improve accuracy and robustness. [ABSTRACT FROM AUTHOR]

CNNs and other deep learners are now state-of-the-art in medical imaging research. However, the small sample size of many medical data sets dampens performance and results in overfitting. In some medical areas, it is simply too labor-intensive and expensive to amass images numbering in the hundreds of thousands. Building Deep CNN ensembles of pre-trained CNNs is one powerful method for overcoming this problem. Ensembles combine the outputs of multiple classifiers to improve performance. This method relies on the introduction of diversity, which can be introduced on many levels in the classification workflow. A recent ensembling method that has shown promise is to vary the activation functions in a set of CNNs or within different layers of a single CNN. This study aims to examine the performance of both methods using a large set of twenty activations functions, six of which are presented here for the first time: 2D Mexican ReLU, TanELU, MeLU + GaLU, Symmetric MeLU, Symmetric GaLU, and Flexible MeLU. The proposed method was tested on fifteen medical data sets representing various classification tasks. The best performing ensemble combined two well-known CNNs (VGG16 and ResNet50) whose standard ReLU activation layers were randomly replaced with another. Results demonstrate the superiority in performance of this approach. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the effects that different preprocessing techniques have on the performance of features extracted from Haralick's co-occurrence matrix, one of the best known methods for analyzing image texture. In addition, we compare and combine different strategies for extracting descriptors from the co-occurrence matrix. We propose an ensemble of different preprocessing methods, where, for each descriptor, a given Support Vector Machine (SVM) classifier is trained. The set of classifiers is then combined by weighted sum rule. The best result is obtained by combining the extracted descriptors using the following preprocessing methods: wavelet decomposition, local phase quantization, orientation, and the Weber law descriptor. Texture descriptors are extracted from the entire co-occurrence matrix, as well as from sub-windows, and evaluated at multiple scales. We validate our approach on eleven image datasets representing different image classification problems using the Wilcoxon signed rank test. Results show that our approach improves the performance of standard methods. All source code for the approaches tested in this paper will be available at: https://www.dei.unipd.it/node/2357 [ABSTRACT FROM AUTHOR]

We perform an extensive study of the performance of different classification approaches on twenty-five datasets (fourteen image datasets and eleven UCI data mining datasets). The aim is to find General-Purpose (GP) heterogeneous ensembles (requiring little to no parameter tuning) that perform competitively across multiple datasets. The state-of-the-art classifiers examined in this study include the support vector machine, Gaussian process classifiers, random subspace of adaboost, random subspace of rotation boosting, and deep learning classifiers. We demonstrate that a heterogeneous ensemble based on the simple fusion by sum rule of different classifiers performs consistently well across all twenty-five datasets. The most important result of our investigation is demonstrating that some very recent approaches, including the heterogeneous ensemble we propose in this paper, are capable of outperforming an SVM classifier (implemented with LibSVM), even when both kernel selection and SVM parameters are carefully tuned for each dataset. [ABSTRACT FROM AUTHOR]

Highlights: [•] Cell phenotype image classification by ensemble of descriptors. [•] Compare some recently proposed methods that are based on the co-occurrence matrix. [•] Investigate the correlation among the features that can be extracted from the co-occurrence matrix. [•] Determine the best way to combine co-occurrence matrix based feature sets. [ABSTRACT FROM AUTHOR]

In 1979 Haralick famously introduced a method for analyzing the texture of an image: a set of statistics extracted from the co-occurrence matrix. In this paper we investigate novel sets of texture descriptors extracted from the co-occurrence matrix; in addition, we compare and combine different strategies for extending these descriptors. The following approaches are compared: the standard approach proposed by Haralick, two methods that consider the co-occurrence matrix as a three-dimensional shape, a gray-level run-length set of features and the direct use of the co-occurrence matrix projected onto a lower dimensional subspace by principal component analysis. Texture descriptors are extracted from the co-occurrence matrix evaluated at multiple scales. Moreover, the descriptors are extracted not only from the entire co-occurrence matrix but also from subwindows. The resulting texture descriptors are used to train a support vector machine and ensembles. Results show that our novel extraction methods improve the performance of standard methods. We validate our approach across six medical datasets representing different image classification problems using the Wilcoxon signed rank test. The source code used for the approaches tested in this paper will be available at: http://www.dei.unipd.it/wdyn/?IDsezione=3314&IDgruppo_pass=124&preview=. [ABSTRACT FROM AUTHOR]

• Ensemble of convolutional neural networks. • New activation function is proposed. • Activations functions are compared and combined. • Rectified Linear Units and different variants are compared. Activation functions play a vital role in the training of Convolutional Neural Networks. For this reason, developing efficient and well-performing functions is a crucial problem in the deep learning community. The idea of these approaches is to allow a reliable parameter learning, avoiding vanishing gradient problems. The goal of this work is to propose an ensemble of Convolutional Neural Networks trained using several different activation functions. Moreover, a novel activation function is here proposed for the first time. Our aim is to improve the performance of Convolutional Neural Networks in small/medium sized biomedical datasets. Our results clearly show that the proposed ensemble outperforms Convolutional Neural Networks trained with a standard ReLU as activation function. The proposed ensemble outperforms with a p-value of 0.01 each tested stand-alone activation function; for reliable performance comparison we tested our approach on more than 10 datasets, using two well-known Convolutional Neural Networks: Vgg16 and ResNet50. [ABSTRACT FROM AUTHOR]

This paper proposes RUR53: an Unmanned Ground Vehicle able to navigate through, identify, and reach areas of interest. There, it can recognize, localize, and manipulate work tools to perform both indoor and outdoor complex tasks. Indeed, a wide range of sensors composes the robot and enables it to perceive vast workspaces, reach distant targets, and face the uncertainties of the real world. Precise object detection is also guaranteed, essential to manipulate objects of different shapes and materials. Moreover, a customized 3-finger gripper makes the gripping mode suitable for any lightweight object. Two modalities are proposed: autonomous and teleoperated, letting both unskilled and skilled human operators easily adapt the system to complete personalized tasks. The paper exhaustively describes RUR53 architecture and demonstrates its good performance while executing both indoor and outdoor navigation and manipulation tasks. A specific case study is described where the proposed modular architecture allows to easily switch to a semi-teleoperated mode: the 2017 Mohamed Bin Zayed International Robotics Challenge, where our team ranked third in the Grand Challenge in collaboration with the Czech Technical University in Prague, the University of Pennsylvania, and the University of Lincoln (UK). [ABSTRACT FROM AUTHOR]

To foster human–robot interaction, autonomous robots need to understand the environment in which they operate. In this context, one of the main challenges is semantic segmentation, together with the recognition of important objects, which can aid robots during exploration, as well as when planning new actions and interacting with the environment. In this study, we extend a multi-view semantic segmentation system based on 3D Entangled Forests (3DEF) by integrating and refining two object detectors, Mask R-CNN and You Only Look Once (YOLO), with Bayesian fusion and iterated graph cuts. The new system takes the best of its components, successfully exploiting both 2D and 3D data. Our experiments show that our approach is competitive with the state-of-the-art and leads to accurate semantic segmentations. • Introduction of a detector in a multi-view semantic segmentation pipeline. • Semantic segmentation of coarse scene elements and objects with clear boundaries. • Refinement of detection priors for accurate object contours. [ABSTRACT FROM AUTHOR]

In recent years, the field of deep learning has achieved considerable success in pattern recognition, image segmentation, and many other classification fields. There are many studies and practical applications of deep learning on images, video, or text classification. Activation functions play a crucial role in discriminative capabilities of the deep neural networks and the design of new "static" or "dynamic" activation functions is an active area of research. The main difference between "static" and "dynamic" functions is that the first class of activations considers all the neurons and layers as identical, while the second class learns parameters of the activation function independently for each layer or even each neuron. Although the "dynamic" activation functions perform better in some applications, the increased number of trainable parameters requires more computational time and can lead to overfitting. In this work, we propose a mixture of "static" and "dynamic" activation functions, which are stochastically selected at each layer. Our idea for model design is based on a method for changing some layers along the lines of different functional blocks of the best performing CNN models, with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLU layer) by a different activation function stochastically drawn from a set of activation functions: in this way, the resulting CNN has a different set of activation function layers. [ABSTRACT FROM AUTHOR]

This paper presents an improved version of a recent state-of-the-art texture descriptor called Gaussians of Local Descriptors (GOLD), which is based on a multivariate Gaussian that models the local feature distribution that describes the original image. The full rank covariance matrix, which lies on a Riemannian manifold, is projected on the tangent Euclidean space and concatenated to the mean vector for representing a given image. In this paper, we test the following features for describing the original image: scale-invariant feature transform (SIFT), histogram of gradients (HOG), and weber's law descriptor (WLD). To improve the baseline version of GOLD, we describe the covariance matrix using a set of visual features that are fed into a set of Support Vector Machines (SVMs). The SVMs are combined by sum rule. The scores obtained by an SVM trained using the original GOLD approach and the SVMs trained with visual features are then combined by sum rule. Experiments show that our proposed variant outperforms the original GOLD approach. The superior performance of the proposed system is validated across a large set of datasets. Particularly interesting is the performance obtained in two widely used person re-identification datasets, CAVIAR4REID and IAS, where the proposed GOLD variant is coupled with a state-of-the-art ensemble to obtain an improvement of performance on these two datasets. Moreover, we performed further tests that combine GOLD with non-binary features (local ternary/quinary patterns) and deep transfer learning. The fusion among SVMs trained with deep features and the SVMs trained using the ternary/quinary coding ensemble is demonstrated to obtain a very high performance across datasets. The MATLAB code for the ensemble of classifiers and for the extraction of the features will be publicly available 1 1 https://www.dei.unipd.it/node/2357 (+Pattern Recognition and Ensemble Classifiers) to other researchers for future comparisons. [ABSTRACT FROM AUTHOR]