Search

This paper is devoted to studying the capabilities of modern neural networks in image processing for solving the problem of monitoring the state of steel and reinforced concrete structures. The article presents a method for solving monitoring problems based on the use of a combination of several neural networks focused on recognizing a fragment of a structure and parts of a structure. Methods for training neural networks on small training samples are proposed. The results of the operation of the algorithms on real images are presented, showing the consistency and efficiency of the proposed solution.

It is crucial to analyze the tire under dynamic conditions to observe a performance close to realistic situations. Particularly, the temperature generated by the interaction of tire and pavement can provide useful information and understanding into how tire components can be optimized. However, in such a situation, data generation and measurement are challenging tasks. High-resolution thermal infrared imaging is a non-contact technology that transforms radiation patterns into a visible image and allows measuring the temperature changes on the surface of the tire. Therefore, the first step towards a systematic analysis of the performance of the tire is to segment the surface. To this end, we present an approach that combines traditional image processing methods with convolutional neural networks. We further investigate transfer learning techniques to improve the prediction of the proposed model on a different dataset. Our ultimate goal is to implement a robust network to segment a broad variety of tires. A segmentation accuracy >0.97 and a validation error

Apart from machine learning and knowledge engineering, there is a third way of challenging machine vision – the Gestalt-law school. In an interdisciplinary effort between psychology and cybernetics, compositionality in perception has been studied for at least a century along these lines. Hierarchical compositions of parts and aggregates are possible in this approach. This is particularly required for high-quality high-resolution imagery becoming more and more common, because tiny details may be important as well as large-scale interdependency over several thousand pixels distance. The contribution at hand studies the depth of Gestalt-hierarchies in typical social image genres—portraits and group pictures—exemplarily, and outlines technical means for their automatic extraction. The practical part applies bottom-up hierarchical Gestalt grouping as well as top-down search focusing and constraint enforcement, listing as well success as failure. In doing so, the paper discusses exemplarily the depth and nature of such compositions in imagery relevant to human beings.

This article presents the results of the joint work of image analysis specialists and ophthalmologists on the problem of analyzing images obtained by optical coherence tomography (angiography). A descriptive algorithmic scheme for analyzing images obtained using optical coherence tomography angiography was constructed to automate detection of pathological changes in the morphometric characteristics of the fundus. The algorithmic scheme is based on the image processing, analysis, and recognition methods. A feature space has been developed that makes it possible to identify pathological changes in the structure of the vascular plexuses of the human retina. It was possible to achieve high-accuracy classification of images of healthy eyes and those with pathology. Analysis of correlation between the features and investigated structural, clinical, and functional parameters can be used to confirm that the selected features are informative diagnostic and prognostic markers, and automated analysis using the proposed algorithm makes it possible to speed up and optimize the procedure for interpreting images of this class. Software has been created that allows, with high accuracy in an autonomous mode, performing differential diagnostics of the normal state of vessels from the pathological one, which increases its diagnostic value. In the future, more detailed work is planned with “borderline” conditions, which constitute the main problem in screening patients with diabetes mellitus.

In order to improve the accuracy of conical target pose parameters measured by monocular vision, we propose a high precision visual measurement method for conical target based on the minimum reconstruction error of 3D. Firstly, we select the main and auxiliary features used for pose parameters’ measurement by analyzing target’s imaging process. Secondly, the algebraic and geometric methods for measuring pose parameters are determined based on the selected features. Finally, we utilize the geometric constraints of target to establish the reconstruction error formula and optimize the pose parameters based on the minimum reconstruction error principle to improve the measurement accuracy. To verify the effectiveness of our method, we use 3ds Max software to generate target’s simulation images under different pose parameters and compare the pose parameters before and after optimization. Experimental results show that the proposed method can significantly improve the accuracy of pose parameters when the measurement distance is far (more than 30 m), which is up to about 40%.

Diabetic retinopathy is one of the dangerous fundus diseases that leads to irreversible loss of vision. In the case of untimely or incorrect treatment, blindness occurs. Currently, laser coagulation is a common treatment method. An ophthalmologist uses a laser to apply a series of burns to the retina. The success of the operation depends entirely on the experience of the doctor. The automatic formation of a preliminary plan of coagulates allows us to solve a number of problems related to the operation, such as long manual placement of coagulates or adjustment of laser power. Thus, the probability of a doctor’s error is reduced, and the preparation time for the operation is significantly reduced. One of the key stages in the formation of the plan is the segmentation of the fundus image. This stage is carried out with the help of texture features, the calculation of which takes a long time. In relation to this, this study proposes high-performance algorithms for the segmentation of fundus images using CUDA technologies, which significantly speed up sequential versions and outperform parallel algorithms.

The article is devoted to the problem of the unity and integrity of the image of a semantic standard, allocated by phrases for a topical text. Herewith, the proximity of the text to the standard is assessed without searching for paraphrases, and the base for assessing the proximity of the text to the standard is the division of words of each of its phrases into classes according to the value of the TF-IDF measure relative to the texts of the corpus, previously formed by the expert. The analyzed texts are abstracts of scientific articles together with their titles. The core of the problem is as follows: for each phrase, the maximum proximity to the standard is achieved with respect to its corpus document and, as a consequence, it is required to assess the mutual relevance of such documents for different phrases of the analyzed text. In this study, this problem is solved by introducing the distances between the vectors of the values of the TF-IDF measure of the words of a separate phrase with respect to different documents in the corpus. In this case, the distance between documents, relative to which the closest proximity to the standard of phrases of the analyzed text was achieved, should be minimal. Using the Euclidean metric and Manhattan distance as an example, this study illustrates the application of the proposed approach to the problem of choosing a higher-level text for the given one in the hierarchy being formed in terms of semantic standard complementarity.

Feature extraction based traditional human action recognition algorithms are complicated, leading to low recognition accuracy. We present an algorithm for the recognition of human actions in videos based on spatio-temporal fusion using 3D convolutional neural networks (3D CNNs). The algorithm contains two subnetworks, which extract deep spatial information and temporal information, respectively, and bilinear fusion policy is applied to obtain the final fused spatio-temporal information. Spatial information is represented by a gradient feature, and the temporal information is represented by optical flow. The fused spatio-temporal information can retrieve deep features from multiple angles by constructing a new 3D CNNs. The proposed algorithm is compared with the current mainstream algorithms in the KTH and UCF101 datasets, showing effectiveness and high recognition accuracy.

This article explores the use of computer vision for recognizing human fatigue by the eyes. The primary attention is paid to the development of a software package that can be used in the future as a system for monitoring the condition of drivers. Face detection is based on the Viola–Jones method and Haar cascades. This allows the algorithm to work in real time. However, convolutional neural networks are used to recognize eye conditions. Such networks training takes place on the eyes cut out from images of faces. Learning occurs in two eye states: open and closed. Moreover, the left and right eyes are analyzed separately. Different illumination characteristics have resulted in different accuracy rates for each eye. To develop the program, the Python programming language was used, the Jupyter Notebook was chosen as the development environment, and OpenCV was used as the main library, since it allows us to receive and process data from a USB camera. The developed software package allows us to detect closed eyes with precision and recall of about 90%. This uses a simple camera with a low resolution of 640 × 480 pixels. The proposed algorithm requires an additional increase in the accuracy and completeness of recognition of the closed eyes.

Decomposition, i.e., partitioning an image into components with different information content, is used to solve many problems of processing and analyzing of video information. The separation of an image into piecewise smooth structural and texture-noise components is of interest. The problem is the preservation of the signal structure, since independent smoothing is required in each of the image areas that reflect a particular object in the scene. Most of the known smoothing-used decomposition methods are based on the indirect characteristics of the local area of the image, for example, the distribution of signal values, which does not reflect its structural features sufficiently well. The criterion for limiting the smoothing area should be the fact that the target and surrounding points belong to the same spatial area of the image. A sufficient condition for the connectivity of the points of the region is the absence of contour lines between them. An approach to decomposition based on the preliminary delineation of the image areas by detecting contour (brightness) differences between them and subsequent contour-constrained smoothing inside each of the areas is proposed. The concept of the affinity of points in the image is introduced, based on which the smoothing algorithm is built. Experimental comparisons of the proposed algorithm with other well-known image smoothing algorithms are carried out.

Imaging problems are becoming increasingly important due to the development of systems for aerospace monitoring of the Earth, radio and sonar location, medical devices for early diagnosis of diseases, etc. However, most of the work on image processing is related to images defined on rectangular two-dimensional grids or grids of higher dimensions. In some practical situations, images are defined on a cylinder (for example, images of pipelines, blood vessels, rotation details) or on a circle (for example, images of a facies (thin film) of dried biological fluid, an eye, a cut of a tree trunk). The specifics of the field of assignment of such images must be taken into account in their models and processing algorithms. In this paper, autoregressive models of cylindrical and circular images are considered and expressions of the correlation function are given depending on the autoregressive parameters. Spiral scanning of a cylindrical image can be viewed as a quasi-periodic process due to the correlation of image lines. To represent heterogeneous images with random heterogeneity, “double stochastic” models are used, in which one or several control images set the parameters of the resulting image. The available image can be used to estimate the parameters of the model of its control images. However, this is not sufficient to fully identify the hidden control images. It is also necessary to evaluate their covariance functions and find out whether they correspond to the hypothetical ones. The paper proposes a test for testing the hypotheses about the covariance functions of cylindrical and circular images with a study of its power relative to the parameters of the image model.

To observe biodiversity, the variety of plant and animal life in the world or in a particular habitat, human observers make the most common examinations, often assisted by technical equipment. Measuring objectively the number of different species of animals, plants, fungi, and microbes that make up the ecosystem can be difficult. In order to monitor changes in biodiversity, data have to be compared across space and time. Cameras are an essential sensor to determine the species range, abundance, and behavior of animals. The millions of recordings from camera traps set up in natural environments can no longer be analyzed by biologists. We started research on doing this analysis automatically without human interaction. The focus of our present sensor is on image capture of wildlife and moths. Special hardware elements for the detection of different species are designed, implemented, tested, and improved, as well as the algorithms for classification and counting of samples from images and image sequences, e.g., to calculate presence, absence, and abundance values or the duration of characteristic activities related to the spatial mobilities. For this purpose, we are developing stereo camera traps that allow spatial reconstruction of the observed animals. This allows three-dimensional coordinates to be recorded and the shape to be characterized. With this additional feature data, species identification and movement detection are facilitated. To classify and count moths, they are attracted to an illuminated screen, which is then photographed at intervals by a high-resolution color camera. To greatly reduce the volume of data, redundant elements and elements that are consistent from image to image are eliminated. All design decisions take into account that at remote sites and in fully autonomous operation, power supply on the one hand and possibilities for data exchange with central servers on the other hand are limited. Installation at hard-to-reach locations requires a sophisticated and demanding system design with an optimal balance between power requirements, bandwidth for data transmission, required service and operation in all environmental conditions for at least ten years.

This special issue of PRIA is devoted to some scientific results and trends of the 25th International Conference on Pattern Recognition (Virtual, Milano, Italy, January 10–15, 2021). Two important events of ICPR-2020 are represented in this special issue: (1) The paper of Professor Ching Yee Suen (Centre for Pattern Recognition and Machine Intelligence, Department of Computer Science and Software Engineering, Concordia University, Montreal, QC, Canada)–the recent winner of IAPR very prestigious K.S. Fu Prize for a year of 2020. The paper based on his lecture “From handwriting to human personality and facial beauty” presented at the ICPR 2020; (2) Special issue “ICPR-2020 Workshop “Image Mining. Theory and Applications.” The analysis of the scientific contribution of IMTA-VII-2021 allows us to draw the following conclusions: (1) The construction of a unified mathematical theory of image analysis is still far from complete. (2) There is considerable interest in the development of new mathematical methods for analyzing and evaluating information presented in the form of images. (3) There is a tendency to expand the mathematical apparatus in the development of new methods of image analysis and recognition by involving in this process areas of mathematics that were not previously used in image analysis. (4) The gap between the capabilities of new mathematical methods of image analysis and recognition and their actual use in solving applied problems remains significant. (5) There is an excessive use of neural networks in solving applied problems of image analysis and image recognition, and quite often without proper justification and interpretation of the results. The special issue includes articles based on the workshop papers selected by the IMTA-VII-2021 Program Committee for publication in PRIA. The PRIA special issue “Scientific Resume of the 25th International Conference on Pattern Recognition” is prepared by the National Committee for Pattern Recognition and Image Analysis of the Russian Academy of Sciences, the IAPR member society, and by the IAPR Technical Committee no. 16 on Algebraic and Discrete Mathematical Techniques in Pattern Recognition and Image Analysis.

This article is devoted to the basic models of descriptive image analysis, which is the leading section of the modern mathematical theory of image analysis and recognition. The fundamental problem that the subject of the article addresses is automated extraction of information from images necessary for making intelligent decisions. Descriptive analysis envisages the implementation of image analysis processes in the image formalization space, the elements of which are various forms (states, phases) of the image representation, which is converted from the original form to one convenient for recognition (i.e., to a model), and data representation conversion models. The image analysis processes are considered to be sequences of transformations implemented in the phase space and providing the construction of phase states of the image, which together form the phase trajectory of image transfer from the original form to the model. The study of the image formalization space leads to formalization of the concepts of representation/image model, as well as to construction of models of the image recognition and analysis processes and formulation of mathematical statements of image recognition and analysis problems. Two types of image analysis models are considered: (1) models reflecting the methodology and mathematical foundations of image recognition and analysis—the problem statement, used mathematical and heuristic methods, algorithmic content of the process: (a) a model based on reverse algebraic closure, (b) a model based on the property of equivalence of images of the same class, (c) a model based on multiple image models and multiple classifiers; (2) models characterizing the architecture and structure of the recognition process: (a) a multilevel model for combining algorithms and initial data for image recognition, (b) information structure for generating descriptive algorithmic image recognition schemes. A brief description of the above models is given. A comparative analysis of the relationships and specifics of the models is carried out. Directions for further research are discussed.

A new type of coronavirus called (SARS-CoV-2) causes the COVID-19 coronavirus disease. The World Health Organization (WHO) declared this COVID-19 disease as pandemic because the disease got spread over several countries. At present situation, there is no medicine available for prevention or cure of the infectious disease. Samples taken from persons with COVID-19 symptoms are commonly tested using Reverse Transcription–Polymerase Chain Reaction (RT-PCR) process which is costlier and also take a minimum of 24 h to get the test result as either negative or positive. The proposed work suggests the possibility of using X-ray images of persons having COVID-19 symptoms to be classified as 1) healthy, 2) COVID-19 affected, or 3) Pneumonia affected. Experimentation is carried out with data samples from each category and classification done using Convolutional Neural Network (CNN), transfer learning using VGG Net, and machine learning techniques such as Support Vector Machine (SVM) and XGBoost which utilizes features extracted with the help of Convolutional Neural Network. Out of the models compared, the SVM with CNN extracted features was able to produce a highest precision, recall, F1-score and accuracy of 95.27, 94.52, 94.94, and 95.81%, respectively in identifying healthy, Pneumonia, and COVID-19 affected persons while experimented with 5-fold cross validation.

A coordinated ranking as an opinion of an expert group can be represented by the well-known Kemeny’s median. Such a decision is a least different from other rankings, and is free of some contradictions of the majority rule problem. As a mathematical principle, the Kemeny’s median gives a decision in any case, in particular, for conflicting experts or groups. In practice, competing opinions are usually identified, and special procedures are used to achieve the required level of consensus. One known approach consists in assigning weights to expert opinions. In this paper, the correct mathematical basis is formulated for new types on the Kemeny’s median, like metric and weighted ones. The problem to find the median for a linear combination of expert rankings is investigated based on the well-known locally optimal Kemeny’s algorithm. It is proposed to investigate the rank aggregation problem based on new types of the median.

In this research paper, change detection based methods were considered to find collapsed and intact buildings using radar remote sensing data or radar imageries. The main task of this research paper is the collection of most relevant scientific research in the field of building damage assessment using radar remote sensing data. Several methods are selected and presented as best methods in the present time, there are methods with using interferometric coherence, backscattering coefficients in different spatial resolution. In conclusion, methods are given in the end, which show, which methods and radar remote sensing data give more accuracy and more available for building damage assessment. Low-resolution Sentinel-1A/B radar remote sensing data are recommended as free available for monitoring of destruction on a degree microregion level. Change detection and texture-based methods are used together to increase overall accuracy. Homogeneity and Dissimilarity GLCM texture parameters found better for the separation of collapsed and intact buildings. Dual polarization (VV, VH) backscattering coefficients and coherence coefficients (before the earthquake and coseismic) were fully utilized for this study. There were defined the better multi variable for supervised classification of none building, damaged and intact buildings features in urban areas. In this work, we were achieved overall accuracy 0.77, producer’s accuracy for none building is 0.84, for damaged building case 0.85, for intact building 0.64. Amatrice town was chosen as most damaged from the 2016 Central Italy Earthquake.

Nonverbal communication primarily by the way of hand gestures is as old as human evolution. Before languages were developed and texts, which came much after that, hand gestures were the only way of human interaction. And hence in historical artefacts across various civilizations and religions across the world hand gestures play a predominant role, some more than others. While selecting a subject for the application of this research we wanted to zero in on a practice or religion where non-verbal communication is a prevalent part and hence our inclination towards Buddhism. In Buddhism, hand gestures (mudras) are considered as a sacred gesture that represent the different Buddha deities and their significance. The article proposes a system that identify the Buddhist hand gesture (mudras) using computer-aided technology. The system comprises a preprocessing stage, which creates a contour plot of the image to obtain the boundary of the region of interest. The features are extracted by generating a star skeleton from the preprocessed image. The star skeleton calculated by considering the local maxima of the distance signal obtained by joining the centroid with the boundary pixels describes the mudras. Each of these mudras has a different star skeleton. The star skeletons computed from the known sample images are used to build a database for the recognition system. The recognition is achieved by choosing the template with the most similar skeleton retrieved from the database.

The work is devoted to the research that was carried out within the framework of computer vision problems applicable to the analysis of images and video information with vehicles. We solve the problem of classifying vehicles. We analyze the drawbacks of Haar features and convolutional neural networks and test the obtained networks using the key point method; we construct an integral algorithm that includes several networks, and we further validate it on a large number of real photographs and types of vehicles. Next, we solve the task to develop a software framework for tracking vehicles by analyzing adjacent photographs from a video sequence. After that, we consider the tracking task in more detail. We analyze modern tracking algorithms using machine learning and describe our implemented tracker with support for the appearance of obstacles between the camera and a moving vehicle. As a result, we propose algorithms and open-source software that, after being configured for specific cameras, can be used in traffic analysis systems.

Texture analysis is a significant area in image processing, but feature point extraction is pretty susceptible to noise in texture images. In this paper, a new method for extracting feature points, especially from the paper overlapping images, is presented, which is using dynamic threshold, mathematical morphology and image thinning to extract potential feature points. And an optimization algorithm is also proposed to promote the repeatability of feature points via analyzing corresponding skeletons. Results show that the proposed algorithm could depress noise, and the repeatability of this method (60%) outperforms traditional feature extraction algorithms, like Harris (46%), FAST (57%), and SUSAN (45%), in paper overlapping images.

An algorithm for statistical analysis of aerial photography data obtained by unmanned aerial vehicles is proposed. Segmentation of multispectral images by a complex of spectral and textural features makes it possible to identify areas of historical anthropogenic impact on the natural environment. The test site was the territory of the economic district of the Pudemsky ironworks (Udmurt Republic), where the arable lands of factory peasants were located in the first half of the 19th century. The location of the arable land and its configuration were restored as a result of the transformation of historical cartographic materials from 1817–1832. At the first stage of the algorithm, it is supposed to calculate features according to multispectral survey data (Haralick’s features, NDVI index); at the second stage, it is to reduce the number of features by the principal component analysis; at the third stage, it is to segment images based on the received features by the method k-means. The initial data were the results of multispectral imaging in Green, Red, RedEdge, and near infrared (NIR) spectral ranges. The efficiency of the proposed algorithm is shown by comparing the segmentation results with reference data (historical cartographic materials and aerial photographs in the visible range).

Early detection of prostate cancer is crucial for its successful treatment. However, it is not always an easy task because of the various image capturing configurations, like acquisition protocols, magnetic field strength, presence/absence of endorectal coil, and resolution. The major bottleneck in the process is the delineation of the prostate boundary for its localization, which is required for the detection of abnormalities and performing radiotherapy accurately. Phenomenal development in Artificial Intelligence and Deep Learning has been contributing significantly to medical diagnostics using Computer Vision and the self-learning capabilities of Deep Learning has been explored to present a viable solution to automate this repetitive task of prostate segmentation. The previous approaches of 2D segmentation do not capture volumetric information and are very time consuming too. Hence, we have developed a Deep Learning based automated solution called DV-Net (Dilated Volumetric Network) for volumetric segmentation of prostate cancer. The proposed method considers the full prostate volume in 3D and requires minimal post-processing, which makes it less dependent on the type of input. We also focus on increasing the receptive field of the network and use deep supervision for better segmentation accuracy. Owing to all these features, DV-Net has shown to outperform the accuracy of the baseline V-Net model on the Prostate MR Image Segmentation (PROMISE) data set.

The problem of choosing an attribute space for solving classification problems is considered. When choosing a space, nonlinear transformations of the values of both individual features and their combinations are used. The fuzziness of dividing objects into classes according to the values of individual features is investigated using membership functions. The generality of the procedure for calculating the values of the membership function for data measured in nominal and interval scales is shown. The efficiency of the feature space selection is demonstrated through the indicators of the generalizing ability of the algorithms calculated through the estimates of the structure of relations of class objects.

In this paper, we present an improved Efficient Neural Network (ENet) for semantic segmentation, and named the proposed network as Efficient Residual Neural Network (ERNet). The ERNet network contains two processing streams: one is pooling stream, which is used to obtain high-dimensional semantic information; the other is residual stream which is used to record low-dimensional boundary information. The ERNet has five stages, each stage contains several bottleneck modules. The output of each bottleneck in the ERNet network is fed into the residual stream. Starting from the second stage of ERNet, pooling stream and residual stream through concatenating are used as inputs for each down-sampling or up-sampling bottleneck. The identity mapping of residual stream shortens the distance between the near input and output terminals of each stage network in ERNet, alleviates the problem of vanishing gradient, strengthens the propagation of low-dimensional boundary features, and encourages feature reuse of low-dimensional boundary features. We tested ERNet on CamVid, Cityscape, and SUN RGB-D datasets. The segmentation speed of ERNet is close to that of ENet, but the segmentation accuracy is higher than that of ENet.

Facial video presentation is a topic of interest in many security systems due to its non-intrusive nature. However, such systems are vulnerable to spoof attacks made by fake face videos and thereby gaining unauthorized access in the system. For a robust biometric system anti spoofing approaches like liveness detection ought to be implemented in order to counter the aforesaid print and replay attacks. This article proposes a novel approach of anti spoofing using Multivariate histogram of oriented gradients descriptor in the auto detected micro expression (μE) regions of human facial videos. Facial μE are very brief, spontaneous facial expressions that highlight the face of humans when they either unconsciously or deliberately conceal an emotion. The work emphasizes the variations in μE in fake and original video representation by a considerable amount and claims such a variance is a tool to combat against presentation attacks. In particular, the method automatically extracts the ROI of major changes in μE using the Multivariate orientation gradients parameter and thus proposes this descriptor as one of the most suitable tools to characterize the liveness. The entire implementation is carried out on the self-created Database for replay attacks. The result obtained is satisfactory and tested statistically significant.

In the environment, water is present in a wide variety of scenes, and its detection enables a broad range of applications such as outdoor/indoor surveillance, maritime surveillance, scene understanding, content-based video retrieval, and automated driving as for unmanned ground and aerial vehicles. However, the detection of water is a delicate issue since it can appear in quite varied forms. Moreover, it can adopt simultaneously several textural, chromatic and reflectance properties. The diversification of water appearances makes their classification in the same class a quite difficult task using the general dynamic texture recognition methods. In this work, we propose a new more appropriate approach for recognition and segmentation of water in videos. In this approach, we start with a preprocessing phase in which we homogenize the aspect of the different aquatic surfaces by eliminating any differences in coloration, reflection and illumination. In this phase, a pixel-wise comparison is introduced leading to a unidirectional binarization. Two segmentation steps are then deployed: preliminary segmentation and final segmentation. In the first segmentation, candidate regions are first generated and then classified by applying our spatiotemporal descriptor. This descriptor investigates both spatial and temporal behavior of textures on a local scale through a sliding window. Secondly, a superpixel segmentation is applied in order to regularize the classification results. The proposed approach has been tested using both the recent Video Water and the DynTex databases. Furthermore, it has been compared with other similar work, and with other dynamic texture and material recognition methods. The obtained results show the efficiency of the proposed approach over the other methods. Additionally, its low computational cost makes it suitable for real-time applications.

Advent of convolutional neural network (CNN) in the field of single image super-resolution (SISR) has shown immense improvement in the process of high resolution (HR) image generation. It involves an end-to-end mathematical mapping through non-linear feature extraction between low resolution (LR) and HR image. Performance of CNN can be improved by increasing depth of the architecture, which generally results in higher computational cost and running time. Also, the performance of CNN can be improved by providing more appropriate input. Presently, in most of the cases input image to a CNN is a LR image that is bi-cubically interpolated to the desired size of HR image. However, bicubic interpolation results into detail smoothing of the image. Therefore, in this paper, a hybrid of CNN and fractal interpolation based SISR algorithm is proposed for reconstruction of HR image. Here, a three layered light-weight CNN architecture is utilize which is capable of producing comparable performance with the traditional SISR techniques and fractal interpolation helps in better preservation of structural and textural properties of the HR image. Experimental results are provided to prove the efficacy of the algorithm proposed in the paper.

This paper offers a brief overview of the main theoretical and practical results in digital image processing in the manufacturing of integrated circuits (ICs), obtained by the authors and staff of the Laboratory for Systems Identification of the United Institute for Informatics Problems of the National Academy of Sciences of Belarus. The developed methods, algorithms, and computer analysis technology combine conventional digital processing methods and neural networks. This ensures robust recognition and optical inspection of the objects of layouts of ICs with the desired accuracy. The obtained results are used at the leading enterprises of electronic engineering of the Republic of Belarus in VLSI designing and manufacturing, as well as in the fabrication of optical inspection units for VLSI semiconductor wafers.

The article offers a brief overview of the main theoretical and practical results obtained by its authors and their scientific followers. The results concern mainly the deterministic theory of pattern recognition. In particular, the main results on the logical and algebraic correction of heuristic algorithms are presented. Possible directions for the development of recognition algorithms based on inductive inference, similarity, and precedence metrics are also proposed. The practical part deals with some of the decision-making tasks in areas such as medical diagnostics and various technical areas. The nature of the review and value judgments is based on the experience of the authors.

According to World Health Organization in 2017 nearly 253 million people are visually impaired of whom 36 million are blind. Braille books involve the tactile format that helps the visually impaired people to gain knowledge but only a limited resource is available. Enormous papers and studies describe the method for obtaining machine readable document from textual image. In upcoming days character recognition might serve a key role to create a paperless environment that helps the visually impaired people to gain enormous amount of educational material. Handwritten script recognition is gaining vital importance in today’s electronically interconnected society. In the field of machine learning and pattern matching handwritten has gained lot of attention. This paper first summarizes the pros and cons of technologies developed for the visually impaired people in terms of education material obtained from text image and handwritten image. Along with that it presents a performance comparison of different methods. Finally, it describes the future research work in this domain.

With the development of iris recognition technology, sensors of iris images acquisition are being constantly developed and updated. Re-register users every time a new sensor is deployed is time-consuming and complicated, especially in applications with large-scale registered users. Therefore, it is a challenging problem to choose the common recognition model which is effective for multi-source heterogeneous iris recognition(MSH-IR). The paper proposes a efficient neural network model of stacked Convolutional Deep Belief Networks-Deep Belief Network (CDBNs-DBN) for MSH-IR. The main improvements are two parts: firstly, this model uses the region-by-region extraction method and positions the convolution kernel through the offset of the hidden layer to locate the effective local texture feature structure. Secondly, the model uses DBN as a classifier in order to reduce the reconstruction error through the negative feedback mechanism of the auto-encoder. Experimental results have been implemented on publicly available IIT Delhi iris database, which is captured by three different iris captured sensors. Experiments shows the model performs strong robustness performance and recognition ability.

Single image super-resolution via convolutional neural network (CNN) has demonstrated superior performance. In this paper, we propose a deep CNN model named super-resolution dense residual convolutional network (SRDCR) with the goal of reconstructing high quality high-resolution (HR) image. We propose a dense residual block (DRB) to learn residual information by residual connected layers. The local fusion layer (LFL) is then used to adaptively fuse the input of DRB and the output of the last residual layer. After multiple DRBs residual learning, the global fusion layer (GFL) reconstructs an HR image by adaptively combining the original low-resolution (LR) information and learned information. Experiments on extensive benchmark show that our method achieves favorable performance with much less CNN layers than DRB network.

We generalize the experience of the applied and theoretical investigations performed in the last 7‒8 years, including the generated algorithms and procedures for analyzing the behavior of the cells’ population as a system of dynamical objects by using a concept of the integral optical flow. We determine the main types of motion which make it possible to separate the key moments of the motion of cells in the population and describe the stages of the cells’ development and their interaction with each other. In order to describe the motion based on the integral optical flow, we generate maps which make possible to determine the moment at which the state starts to change and classify the main types of motion in the cells’ population: directed motion, aggregation (cells motion towards each other), dispersion (cells’ motion in different directions with respect to the common center), division (formation of several new cells in the place where the old one was), and apoptosis (cell destruction). We develop the algorithms for analyzing cells’ motion by using the integral optical flow. The generated procedure is intended for nondestructively checking the development of the cells’ population and it can be used in automatic systems of cell engineering. The procedure based on integral optical flow analysis and motion maps makes it possible to monitor the development of the cells’ population at the new level by considering the interaction of cells.

An automatic segmentation scheme for accurate segmentation of characters from Historical Handwritten Kannada Stone Inscription images is presented in this paper. The accuracy of Character Segmentation plays a vital role in facilitating Optical Character Recognition from these text documents. Such an Optical Character Recognition system is a crucial part in the design of an intelligent system which can perform automatic transliteration of the old Kannada text in the inscriptions to its modern counterpart. The stone inscription documents are heavily degraded due to various factors like aging, depositions, risky handling, non uniform illumination and unclear separation between the foreground text and the background. An attempt to suppress these noise degradations might also remove useful text edges thus undermining the performance of the segmentation process. Hence an appropriate image enhancement phase is crucial for these document images. Total Variation Regularization (TVR) is an image contrast enhancement method which denoises the image by smoothing without blurring the sharp edges. A framework of TVR enhancement and Connected Component Labeling segmentation was implemented and evaluated on the dataset of digitized Estampages of Historical Handwritten Kannada Stone Inscriptions (EHHKSI). These Estampages are preserved at the Archeological Survey of India. A global contrast enhancement factor of 1.1528 and character segmentation accuracy of 79.2% was achieved on this dataset. Experimentation validated the improvements over previously reported state of the art methods.

We propose a new copy-move forgery detection method, which can solve the problems of multiple copy-move forgery, low accuracy and inaccurate tampered region location. First, keypoints and corresponding features of the image are extracted by using AKAZE (accelerated KAZE). Second, features are matched by using the Hamming distance and g2NN which can detect the multiple copy-move forgery. Then, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to cluster the keypoints and remove false matching. Finally, PSNR (peak signal-to-noise ratio) and morphological processing is used to locate tampered region accurately. Experimental results show that the proposed method performs well on geometric transformation, post-processing, multiple copy-move forgery, and tampered region localization.

Convolutional neural networks (CNN) have achieved unprecedented results in the fields of pattern recognition and image processing. CNN methods have also been gradually applied to iris recognition. However, in the case of insufficient training samples, training deep CNN models is prone to overfitting. In order to solve the above problem, this paper proposes an iris recognition method based on fine-tune SquIrisNet. When the pre-trained SqueezNet is migrated to the iris dataset, the fully connected layers are used instead of the convolutional layer and the global pooling layer, then the parameters are adjusted using the error back propagation algorithm, and finally the images are classified by the Softmax classifier. The fully connected layers added in this model can play the role of “firewall” in the fine-tune process, retaining the model complexity to a certain extent. The experimental results on the IIT Delhi and SDUMLA-HMT iris databases show that the proposed method has higher correct recognition rate, faster convergence speed and better robustness than AlexNet and VGGNet.

Training methods for deep neural networks (DNNs) are analyzed. It is shown that maximizing the likelihood function of the distribution of the input data P(x) in the space of synaptic connections of a restricted Boltzmann machine (RBM) is equivalent to minimizing the cross-entropy (CE) of the network error function and minimizing the total mean squared error (MSE) of the network in the same space using linear neurons. The application of DNNs for the detection and recognition of productmarking is considered.

Sentiment analysis and sarcasm detection are specialized areas in the field of information retrieval and natural language processing. Sentiment classification is closely correlated with sarcasm detection, where people usually adopt sarcasm to highlight their negative feeling. This paper proposes a novel multi-task deep neural networks for joint sarcasm detection and sentiment analysis (MT_SS). MT_SS train both tasks jointly using bidirectional gated recurrent unit with attention network module to obtain task-specific local feature representation while using convolutional neural networks to obtain global feature representation. The experiments on two datasets show that our proposed model outperforms the state-of-the-art approaches.

Lesion segmentation is a vital step in a melanoma recognition system. Many algorithms were developed for the efficient skin lesion segmentation. Most of them fails to realize a perfect segmentation. This paper proposes a novel, fully automatic system, for the lesion segmentation in dermatograms. The proposed approach executes in two steps. Selection of root seed is the first step. All the lesion pixels in the dermatogram are identified during the second step. Traversal through a predefined lesion pixel path ensures the reachability of all lesion pixels irrespective of the possible lesion discontinuity. The proposed algorithm is tested with two publically available dataset, PH2 and images of ISBI2016 challenge. Out of the six evaluation parameters, the proposed method shows the best values for specificity, accuracy, Hammuode distance and XOR. This confirms the merit of the proposal with respect to existing popular methods.

Contrast enhancement is a primitive step in image processing application for enhancing the visual perceptibility which can be treated as the increase in gray level differences between background and foreground with suppression of noises. In literature, various methods have been proposed for contrast enhancement such as bi- histogram and two-dimensional histogram-based methods, etc. Among various methods, the suitable contrast enhancement algorithm should be chosen based on different applications. To find suitability, qualitative and quantitative analysis is also needed. The quantitative analysis can be done with the help of the various performance measures. In this paper, a complete taxonomy of performance measure for contrast enhancement has been discussed and analyzed for the CSIQ database. Among the various performance measures, it is found that Absolute Mean Brightness Error (AMBE), entropy, contrast improvement index, and gradient magnitude similarity deviation (GMSD) are better in terms of the range of operation and the wide variation of low contrast images. It is interpreted that absolute mean brightness requires less time for computation and it is widely used by the researchers for quantitative analysis of contrast enhancement algorithm. This study provides a critical analysis of the several quantitative measures that validate the effectiveness of qualitative measures and motivates to propose an alternate measure that considers the strength of various measures discussed in the literature. These measures quantify the mean brightness shifting, contrast, and entropy preservation.

Microblog is different from traditional social networks and e-commerce website for its low user activity, data sparsity and dynamic of users’ preference. So traditional recommendation algorithms effect certainly is not very aridly ideal for Microblog. In this paper, we develop a time-aware social hierarchical Poisson factorization (HPF_TS) model to make personalized micro-blog recommendation to each user. HPF_TS is a new Bayesian factorization model based hierarchical Poisson factorization that accounts for socialization information, the time characteristics of user preferences to recommend micro-blogs to users whom are interested in. The interpret ability of the recommendation is also attached more importance to latent topics by utilizing gamma-Poisson structure for modeling items’ content. We studied our models with four real-world data sets and the results show that the superior performance of the proposed model, compared with several alternative methods.

Iris biometric is a widely deployed tool for biometric based user authentication. Its success has paved path for several security related attacks on this biometric. In this paper, we propose a deep learning perspective to solve two of these attacks—replay attack and database attack simultaneously. The proposed architecture HsIrisNet, a convolutional neural network (CNN), is trained and tested on two publicly available databases Casia-Iris-Interval v4 DB and IIT Delhi DB. The proposed approach HsIrisNet uses histograms of the selective robust regions followed by image upsampling to authenticate the user. These robust regions have been found to be sufficient enough to authenticate the user, enabling non-deterministic approach. The use of these selective regions for authentication enables the system to mitigate replay attack. While, the CNN model does not require saving any biometric data to the database, which mitigates database attack from the system. The comparison of proposed approach with existing state-of-art techniques show better performance of the proposed approach.

Smart Electromechanical Systems (SEMS) are used in cyber-physical systems and, in particular, in intelligent robots. These systems have the ability to integrate the functions of computing, controlling, communicating, storing information, monitoring, measuring, and controlling their own parameters and the environmental parameters. It is important to keep in mind that the behavior of the system is based on information received from the sense sensors of the robot’s central nervous system (CNS) about the state of the environment and the state of the system. The aim of the paper is to describe an algebraic approach to form images based on the sensor data of robots built based on the SEMS modules. An algebraic approach for the formation of images based on the sensory data of robots built based on the SEMS modules is proposed. The algebraic approach to the formation of images based on the sensory data proposed in the article can be used to form the strategy and tactics for controlling intelligent robots.

Ultrasonography images have a high impact in the medical field for faster and accurate diagnosis of the diseases. The analysis and processing of ultrasound images is a tedious task. The proposed work focuses on automatic segmentation and analysis of renal calculi in digital ultrasound kidney images. The developed methodology includes steps such as preprocessing, segmentation and analysis. Preprocessing includes despeckling of input ultrasound images and is performed by using contourlet transform. Preprocessed images undergo automatic segmentation using the level set method. Analysis of the segmented stones is also carried out to obtain metrics such as the number of stones and their sizes. These metrics are essential to decide about the further plan of treatment by urologists and nephrologists. Performance of the developed algorithm is evaluated by the medical experts and also by using the various parameters such as dice similarity coefficient, Jaccard index, specificity, sensitivity, and accuracy.

In the past two decades, there has been an increasing research in recognition and analysis of human emotions in areas of neuroscience, psychology, cognitive science, and computer science. Facial expressions play an important role in assessment of several neuropsychiatric disorders where patients are having impairment in recognition as well as expression of emotional facial expressivity. The majority of the previous work in the domain of machine analysis of human emotion was focused on recognition of prototypic expressions of six basic emotions or also on finding out movements of the individual muscle that the face can produce. Most of these automated facial expression recognition methods were based on data that has been posed on demand and acquired in laboratory settings. But, for real-world applications, instead of classifying a face image into one of the facial expression categories, focus needs to be given on the problem of estimating the depth of facial expression for further grading the emotions leading to applicability in various clinical research. In this paper we have proposed a deep learning-based Emotion Detection and Grading System (D-EDGS) for estimation of facial emotional expression into one of the seven basic categories of emotion. Along with this the proposed model D-EDGS also works towards grading the emotions into low, medium and high scale. In future, effectiveness of D-EDGS can be applied for the development of decision support system in assessment of neuro psychiatric disorders.

The term “Aesthetic” means the philosophy of beauty. In the field of art and photography, it relates to the artistic choice and appreciation of artistry. Computational assessment of aesthetic appeal of photographs is a highly idiosyncratic job. Because of this lot of researchers are interested in this area in present days. It is an interdisciplinary area related to computer vision, art and psychology. Since there is no universal standard for evaluation of aesthetic value, hence many common viewer believe that some certain features of a photograph play a vital role to make a photograph aesthetically pleasant. This survey focuses on a systematic audit of the newfangled techniques for the evaluation of aesthetic quality of photographs. The existing studies reveal that aesthetic quality of photographs depend on the compositional rules like Rule of Third, Golden ratio, Depth of field, and the color space features such as hue, saturation, texture. The purposes of this paper are to present current state of the art in aesthetic evaluation of photographs and provide few future research directions in this area. These future research directions are based on some interesting and unexplored rules of composition like Spiral structure, Framing, Symmetry, Pattern and others. These can be used for better assessment of aesthetic values of photographs.

Facial expression recognition for frontal faces has become a well-established research area in the last two decades. However, non-frontal facial expression recognition hasn’t been paid much attention until recently. In this paper, we propose an MVFE-LightNet (Multi-View Facial Expression Light Weight Network) for multi-view facial expression recognition. To this end, we first applied MTCNN for facial detection and alignment and then did preprocessing like normalization and data augmentation. Finally, we put the images into MVFE-LightNet to extract sub-space features of facial expressions with various poses. A depthwise separable residual convolution module architecture was designed to reduce the parameters of the model and lessen the chance of overfitting. Experiments were implemented on Radboud Faces Database and BU-3DFE dataset. We demonstrated that our method could effectively improve the recognition accuracy, and achieved the accuracy of 95.6% and 88.7% respectively for the Radboud and BU-3DFE.

With the development of information technology, there is a high demand for high-resolution images. Image super-resolution reconstruction technology is to estimate a high-resolution image with better quality from one or a sequence of low-resolution images, with the help of signal processing technology. The core idea is to integrate useful information with strong correlations and complementarities from single image or multiple images as desired. Learning based single image super-resolution reconstruction technology is the current research hotspot. The paper systematically overviews this technology and discuss some main categories of it, such as super-resolution reconstruction based on neighbors, based on sparse representation, based on deep learning. At the end of the paper, challenge issues and future research directions for super-resolution image reconstruction are put forward.

Image captioning is a growing field of research that has taken hold of the research community. It is a challenging task owing to the complexity of natural language generation and the difficulty involved in feature extraction from a diverse collection of images. Many models have been proposed to tackle the problem, like state-of-the-art encoder-decoder (Sequential CNN-RNN) systems that have proved to be capable of obtaining results. Recently, Reinforcement learning has made itself the new approach to the problem and has been successful in surpassing many of the state-of-the-art paradigms. We have come up with a new reward system known as the BLUDEr metric, which is a linear combination of the non-differentiable metrics BLEU and CIDEr. We directly optimize this metric for our model, on natural language generation tasks. In our experiments, we use the Flickr30k and Flickr8k datasets, which have become two of the benchmark datasets when it comes to image captioning systems. We have achieved state-of-the-art results on these two datasets, when compared with other models.

For practical use, the relevance of indoor surveillance from multiple cameras to track the movement of people and reidentify them in video sequences is constantly increasing. This is a complex task due to the effect of uneven illumination, background inhomogeneity, overlap, uncertainty of the trajectories of people, and the similarity of their visual features. The article presents an approach to track people by video sequences and reidentify them in multicamera video surveillance systems that are used indoors. At the first step, people are detected using a YOLO v4 convolution neural network (CNN) and described by a rectangular area. Further, the search for the face area and the calculation of its features are carried out, which in the developed method are used when accompanying a person in a video sequence and during his intercamera reidentification. This approach improves the accuracy of tracking with a complex movement trajectory and multiple intersections of people with similar characteristics. The search for faces is carried out on the detected areas based on the multitasking MTCNN, and the MobileFaceNetwork model is used to form the vector of the features of the face. Human features are generated using a modified CNN based on ResNet34 and an HSV color tone channel histogram. The correspondence between people on different frames is established based on the analysis of the spatial coordinates of faces and people, as well as their CNN features, using the Hungarian algorithm. To ensure the accuracy of intercamera tracking, reidentification is performed based on the facial features. Five test video sequences of different numbers of people captured indoors with a fixed video camera were used to test and compare different approaches. The obtained experimental results confirmed the strength of the characteristics of the proposed approach.