Search

Edge-preserving image smoothing is vital in the field of image processing and computational photography. The state-of-the-art filters based on optimization models have achieved promising performance. However, most of them fail to consider the spatial support in the regularization term, thus limiting the edge-preserving capabilities. In this paper, inspired by the bilateral filter, which consists of a range kernel and a spatial kernel. we propose to leverage bilateral kernel as a penalty function, and embed it into an optimization model for edge-preserving image smoothing. Furthermore, we propose to incorporate an edge-aware weighted scheme in the data term design, which further improves the edge-preserving capability. The bilateral function is non-convex and can be non-trivial to solve. In this paper, we propose a novel iterative solution based on fixed point iteration, where the main burden in each iteration is a bilateral filtering process. We have conducted extensive experiments to evaluate the proposed filter. Experiment results indicate that our filter benefits a variety of image processing tasks. Moreover, we propose an efficient approximation of the proposed filter, which is able to significantly accelerate the filtering process with neglectable sacrifice of smoothing quality. J is the objective function in this paper. Our solver is essentially an iterative procedure, where each iteration selectively adding the normalized detail layer C. ∗ I − C. ∗ O to the bilateral filtered result P. / Q. k indicates the k -th iteration. [Display omitted] • We propose a bilateral regularized optimization model for edge-preserving smoothing. • We incorporate the spatial support in the design of the regularization term. • We incorporate an edge-aware weighted scheme in the design of the data term. • We propose a novel iterative solution for our model based on fixed point iteration. [ABSTRACT FROM AUTHOR]

Medical image segmentation is a crucial aspect of medical image processing, and has been widely used in the detection and clinical diagnosis for brain, lung, liver, heart and other diseases. In this paper, we propose a novel multimodal parallel attention network, called MPA-Net, for medical image segmentation. MPA-Net is divided into two parts. The first part extracts more high-dimensional features by improved network structure, which contains the skip connection, the output of the multimodal parallel attention and the output of the previous upsampling layer. The second part incorporates a multimodal parallel attention mechanism, encompassing feature parallel attention, spatial parallel attention and channel parallel attention. This mechanism facilitates the effective fusion of high-dimensional and low-dimensional features, leading to enhanced context information. Experimental results on Kagglelung dataset, Liver dataset, Cell dataset, Drive dataset and Kvasir-SEG dataset show that MPA-Net has achieved better segmentation performance than that of other baseline methods, on lung, liver, cell contour, retinal vessel and polyps. • Architecture integrates multimodal parallel attention for segmentation. • Network employs feature, spatial, and channel parallel attention. • Radiologists aided in precise lesion localization across organs. • Identification rate of lesions improved, aiding disease diagnosis. [ABSTRACT FROM AUTHOR]

Breast cancer (BC) is a common and highly lethal ailment. It stands as the second leading contributor to cancer-related deaths in women worldwide. The timely identification of this condition is of utmost importance in mitigating mortality rates. This research paper presents a novel framework for the precise identification of BC, utilising a combination of image and numerical data features with explainable Artificial Intelligence (XAI). The utilisation of the U-NET transfer learning model is employed for image-based prediction. Additionally, an ensemble model is constructed by integrating characteristics from a customised convolutional neural network (CNN) model with an ensemble comprising random forest (RF) and support vector machine (SVM). The experiments aim to evaluate the influence of original features compared to convoluted features. A comparative analysis is carried out to assess the efficacy of various classifiers in accurately detecting BC, utilising the Wisconsin dataset. The model under consideration exhibits promising capabilities in enhancing BC diagnosis, with a remarkable accuracy rate of 99.99%. The present study contributes to the advancement of BC diagnosis by introducing a novel strategy based on machine learning and discussing the interpretation of the variables using XAI. The primary objective of this approach is to get a notable level of precision, hence facilitating the early and reliable identification of BC. Ultimately, the implementation of this approach is expected to enhance patient outcomes. • This study presents a framework for breast cancer diagnosis using image & numerical features. • For image-based prognosis, the predictive model employs U-NET transfer learning. • An ensemble strategy (RF + SVM) is proposed using CNN features for breast cancer detection • The experiments involve both numerical features of dataset and those derived from the CNN model. • Proposed model is assessed by comparing its performance with state-of-the-art methodologies. [ABSTRACT FROM AUTHOR]

Domain adaptation has found extensive applications in diverse fields, including transfer learning, deep learning, and image processing, to effectively address the challenge of mismatched data distributions. Nevertheless, persistent challenges such as limited transfer data and the occurrence of negative transfer continue to exist. These issues not only increase the complexity of model learning but also affect the accuracy of the transfer results. To tackle these challenges, this paper introduces a novel approach called the Related Weighted Subdomain Adaptive Network (RWSAN). This network constructs related subdomain clusters in the source domain to encompass a wider spectrum of associated information. It aims to align target domain subdomains with the related subdomain clusters in the source domain as closely as possible, thus increasing the number of transferable samples from the source domain and solving insufficient transferable samples in some adaptations. To address the issue of negative transfer arising from irrelevant subdomains in the source domain during the transfer process, a related weighted maximum mean discrepancy method is introduced. This method optimizes transfer weights, minimizes the feature distribution disparity between the target domain and the source domain, and mitigates the negative impact of irrelevant subdomains on the target domain, thereby enhancing the accuracy of domain adaptation during transfer. To evaluate the proposed RWSAN method, comprehensive performance tests were conducted using widely used red-green-blue (RGB) and hyperspectral image (HIS) image datasets. The results demonstrate that the RWSAN method effectively resolves insufficient transfer data and negative transfer, proving its high reliability. Additionally, it exhibits excellent performance in error analysis, network convergence analysis, data visualization, and distribution difference. • New Related Weighted Subdomain Adaptive Network (RWSAN) proposed. • RWSAN constructs related subdomain clusters in source domain. • RWSAN addresses issue of limited transfer data. • Method proposed to minimize feature distribution disparity. • Mitigates negative transfer from irrelevant subdomains. [ABSTRACT FROM AUTHOR]

Automating polyps' detection and segmentation is very helpful in achieving early detection and potentially effective treatment of colon cancer. Because of the constant rise in processing power at a reasonable cost and the availability of training data, deep learning more than ever is becoming very powerful and increasingly widespread in many domains, including computer vision and medical image processing and segmentation. In this paper, we propose a novel method for polyp image segmentation using convolutional neural networks (CNNs) and utilizing multi-parallel U-Net encoder architecture. The proposed method uses multiple encoders that utilize pre-trained CNNs to enrich the extracted features. Skip connections from these parallel encoders are concatenated and propagated to the decoder in partially and fully meshed fashions allowing for data from shallow to deeper layers in the network to be exchanged and vice versa. We trained and tested the method on five publicly available datasets: Kvasir, CVC-Clinic DB, CVC-Colon DB, CVC-T, and ETIS-Larib, which are well-known benchmark datasets for polyp image segmentation. We tested the performance of the method on these five datasets using multiple testing scenarios including the effect of using attention, vision transformers, and multiple decoders. Experimental results showed that the proposed method outperforms other state-of-the-art methods on two of the used benchmark datasets and ranked second on a third. • A novel CNN-based method using parallel encoders for polyp segmentation is proposed. • Combinations of two and three parallel encoders, and decoders with different pre-trained networks are investigated. • The effect of attention and vision transformers is examined. • Five publicly available polyp image databases were used in the experiments. • Dual encoder with attention block and partial mesh skip connections achieved the best result. [ABSTRACT FROM AUTHOR]

• Image processing tehnique for water surface velocity prediction. • Temporal feature extraction for dynamic image sequence using XOR operator. • Machine learning method for image texture analysis. Water surface velocity (WSV) is one of the critical parameters in hydrology. The development of non-intrusive measurement of this parameter using cameras is increasing. Traditionally, measuring WSV using a camera utilizes tracking of moving objects on the water surfaces. Recently, another method has emerged that utilizes the movement of water ripples to estimate the WSV. This paper proposes a novel method for estimating WSV based on camera measurements. The authors call this method Temporal Gray-scale Pattern Image (TGPI) since it extracts Gray-scale patterns of pixels of water flow video in the temporal domain using the XOR operator and create new image. The new image pattern formed is then used to predict WSV using predictor. There are four predictors being compared, namely Multiple Input Linear Regression (MILR), Multiple Input Logistic Regression (MILgR), Convolutional Neural Network Regression (CNN-R), and Convolutional Neural Network Classification (CNN-K). CNN-R and CNN-K predict WSV directly from the results of TGPI. Meanwhile, MILR and MILgR predict WSV from five TGPI features. The five features are the Mean and Median of the Histogram, Mean and Median of the Histogram of Oriented Gradient (HOG), and the Maximum Mid-Value of the Fast Fourier Transform (FFT). MILR and CNN-R are predictors for regression problems, so the testing metrics for them are Trend-Line equation and R2 from the predicted WSV and actual values graph. Meanwhile, for MILgR and CNN-K, which are predictors for classification problems, the testing metrics are Confusion Matrix (CM), Accuracy, Precision, Recall, and F1-Score. To test the four methods without distinguishing their predictor types, a 2-dimensional histogram graph is used. The data-set used for training and testing is video footage of water flow with known WSV. The WSV measurement points used in this study are 1.7 m/s, 3.1 m/s, and 4.2 m/s. The video dataset and these three points are generated by the Mini Open Channel Water Flow Simulator (MOCWFS) developed by the author in this study. From the comparison results, it can be seen that the classification type predictor is superior to the regression type. For the regression type predictor, MILR is better than CNN-R. Meanwhile, for the classification type, CNN-K is superior to MILgR. The best accuracy produced by CNN-K is 98.4%. Although there are shortcomings, the TGPI method is quite feasible for predict Water Surface Velocity. [ABSTRACT FROM AUTHOR]

Automated face recognition (FR) is a well-studied problem, with a history of more than three decades. Facial recognition software that uses two dimensional (2D) images has advanced significantly in terms of accuracy over recent years. Despite the steps forward, face recognition is still not considered to be a solved problem for the cases of (i) difficult exposure conditions, such as during night-time, or in environments with unconstrained lighting, (ii) when operating at long standoff distances or at variable distances from the camera, and (iii) when using different camera sensors that operate in the same or different spectral bands. In practical forensic scenarios, it is often the case that investigators need to operate in difficult conditions, where face images captured in the aforementioned challenging cases need to be matched against good quality face images (gallery set) and where, grouping of the data in the context of demographic information (in terms of gender, ethnicity or other soft biometrics) may also be used in order to assist law enforcement officials, forensic investigators and security personnel in subject identification. In fact, in such practical scenarios, human recognition based solely on visible spectral images may not be feasible. Hence, an attractive solution is to develop efficient FR systems that can work in different practical scenarios. While different systems have been discussed and proposed over the years, an interesting approach is to increase the plurality of the source data, or in other words, start using data captured by camera sensors that cover a broad range of the electromagnetic spectrum that falls outside the visible band, including the ultraviolet (UV) and the infrared (IR) band. Thus, FR outside the visible spectrum is considered an area of growing research interest. This is further supported by the burgeoning market of camera sensors that continues to open new opportunities, at different spectral and spatial scales, in a large number of applications including those in security, forensics and defense. In light of these thoughts, the focus of this work is to expose readers to a plethora of FR advancements when operating outside the visible spectrum and our perspective on where this field is going. The discussion will include the latest studies in the UV (100–400 nm), active IR band (0.7–2.5 μ m) and, finally, the passive IR band, or more specifically, the mid-wave IR (3–5 μ m) and long-wave IR (8–14 μ m) bands. The discussion will also include research work that involves same-spectral and cross-spectral matching scenarios, where, first, face images (probes) are acquired under controlled or difficult conditions as explained above, including dealing with uncooperative subjects, and adverse environments. Then, depending on the FR scenario, the acquired probe images are matched against mug shots collected in a controlled indoor environment using a conventional camera. This paper will start by briefly covering different multi-spectral FR systems and, then, discussing various related topics (ranging from data acquisition sets, up to system performance). The paper will also suggest needed new avenues of inquiry in the context of current work in multi-spectral FR system. [ABSTRACT FROM AUTHOR]

Abstract Image super-resolution aims to generate high-resolution image based on the given low-resolution image and to recover the details of the image. The common approaches include reconstruction-based methods and interpolation-based methods. However, these existing methods show difficulty in processing the regions of an image with complicated texture. To tackle such problems, fractal geometry is applied on image super-resolution, which demonstrates its advantages when describing the complicated details in an image. The common fractal-based method regards the whole image as a single fractal set. That is, it does not distinguish the complexity difference of texture across all regions of an image regardless of smooth regions or texture rich regions. Due to such strong presumption, it causes artificial errors while recovering smooth area and texture blurring at the regions with rich texture. In this paper, the proposed method produces rational fractal interpolation model with various setting at different regions to adapt to the local texture complexity. In order to facilitate such mechanism, the proposed method is able to segment the image region according to its complexity which is determined by its local fractal dimension. Thus, the image super-resolution process is cast to an optimization problem where local fractal dimension in each region is further optimized until the optimization convergence is reached. During the optimization (i.e. super-resolution), the overall image complexity (determined by local fractal dimension) is maintained. Compared with state-of-the-art method, the proposed method shows promising performance according to qualitative evaluation and quantitative evaluation. [ABSTRACT FROM AUTHOR]

Abstract Face detection plays an important role in many computer vision applications. In recent years, much research has focused on extending the well-established Adaboost face detector algorithm for multi-view face detection. However, detecting in-plane and out-of-plane rotated faces simultaneously is still a challenging task today. In this paper, a very robust multi-view face detection algorithm, with its core functionality based on frontal face detection, is proposed to simultaneously detect in-plane and out-of-plane rotated faces. Moreover, only the training data in the frontal face is needed and we do not require the training data from different in-plane or out-of-plane rotation angles. In the proposed algorithm, first, techniques such as the skin filter and entropy rate superpixels (ERSs) are applied to obtain face candidates. Then, angle compensation and refinement are applied to improve the accuracy of face detection in the in-plane case. Moreover, the symmetry extension technique, i.e., extending the face candidate with its flipped version to create a face similar to the frontal one, is applied to detect out-of-plane faces without the need of training data. Simulations on the FEI, Pointing'04, BaoFace, Group, Utrecht, and WWW datasets demonstrate the proposed algorithm's effectiveness and superior performance as compared to state-of-the-art face detection methods. Graphical abstract Unlabelled Image Highlights • We detect in-plane and out-of-plane rotated faces by only the frontal face detector. • Training data from different in and out-of-plane rotation angles are not required. • Skin filters and ERSs can preserve the faces from different illumination and races. • Angle compensation computes the rotation angle for each rotated face. • Symmetry extension creates pseudo faces to well detect out-of-plane rotated faces. [ABSTRACT FROM AUTHOR]

Visual tracking is an important task in various computer vision applications including visual surveillance, human computer interaction, event detection, video indexing and retrieval. Recent state of the art sparse representation (SR) based trackers show better robustness than many of the other existing trackers. One of the issues with these SR trackers is low execution speed. The particle filter framework is one of the major aspects responsible for slow execution, and is common to most of the existing SR trackers. In this paper, 1 1 An earlier brief version of the paper has appeared in ICIP'13 (R. Venkatesh Babu and P. Priti, “Interest points based object tracking via sparse representation”, in proceedings of International Conference on Image Processing (ICIP), Melbourne, Australia, 2013). we propose a robust interest point based tracker in l 1 minimization framework that runs at real-time with performance comparable to the state of the art trackers. In the proposed tracker, the target dictionary is obtained from the patches around target interest points. Next, the interest points from the candidate window of the current frame are obtained. The correspondence between target and candidate points is obtained via solving the proposed l 1 minimization problem. In order to prune the noisy matches, a robust matching criterion is proposed, where only the reliable candidate points that mutually match with target and candidate dictionary elements are considered for tracking. The object is localized by measuring the displacement of these interest points. The reliable candidate patches are used for updating the target dictionary. The performance and accuracy of the proposed tracker is benchmarked with several complex video sequences. The tracker is found to be considerably fast as compared to the reported state of the art trackers. The proposed tracker is further evaluated for various local patch sizes, number of interest points and regularization parameters. The performance of the tracker for various challenges including illumination change, occlusion, and background clutter has been quantified with a benchmark dataset containing 50 videos. [ABSTRACT FROM AUTHOR]

This paper proposes a hierarchical multi-label matcher for patch-based face recognition. In signature generation, a face image is iteratively divided into multi-level patches. Two different types of patch divisions and signatures are introduced for 2D facial image and texture-lifted image, respectively. The matcher training consists of three steps. First, local classifiers are built to learn the local matching of each patch. Second, the hierarchical relationships defined between local patches are used to learn the global matching of each patch. Three ways are introduced to learn the global matching: majority voting, ℓ 1 -regularized weighting, and decision rule. Last, the global matchings of different levels are combined as the final matching. Experimental results on different face recognition tasks demonstrate the effectiveness of the proposed matcher at the cost of gallery generalization. Compared with the UR2D system, the proposed matcher improves the Rank-1 accuracy significantly by 3% and 0.18% on the UHDB31 dataset and IJB-A dataset, respectively. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel tracking system based on edge-based object proposal and data association called object proposal association. Our object proposal method accurately detects and localizes objects in an image by searching for object-like regions, with the assumption that an object is represented by a closed boundary. To search for closed boundaries in an image, we present a new Edge Fields (EFs) technique. Using this technique, our method can extract high-quality edges and can obtain accurate boundaries from the image. The EFs technique consists of blurring and thresholding steps, where the former helps extract high-quality edges and the latter prevents the method from losing image details while blurring. After the method extracts object-like regions, we associate the regions in the previous frame with those in the current frame. For this purpose, using the Markov chain Monte Carlo data association (MCMCDA) algorithm, we can find pairs of similar regions across two frames. Experimental results demonstrate that our object proposal method is competitive with state-of-the-art object proposal methods on the PASCAL VOC 2007 dataset. Our tracking method is also competitive with state-of-the-art tracking methods on Object Tracking Benchmark dataset. [ABSTRACT FROM AUTHOR]

Cameras are a crucial exteroceptive sensor for self-driving cars as they are low-cost and small, provide appearance information about the environment, and work in various weather conditions. They can be used for multiple purposes such as visual navigation and obstacle detection. We can use a surround multi-camera system to cover the full 360-degree field-of-view around the car. In this way, we avoid blind spots which can otherwise lead to accidents. To minimize the number of cameras needed for surround perception, we utilize fisheye cameras. Consequently, standard vision pipelines for 3D mapping, visual localization, obstacle detection, etc. need to be adapted to take full advantage of the availability of multiple cameras rather than treat each camera individually. In addition, processing of fisheye images has to be supported. In this paper, we describe the camera calibration and subsequent processing pipeline for multi-fisheye-camera systems developed as part of the V-Charge project. This project seeks to enable automated valet parking for self-driving cars. Our pipeline is able to precisely calibrate multi-camera systems, build sparse 3D maps for visual navigation, visually localize the car with respect to these maps, generate accurate dense maps, as well as detect obstacles based on real-time depth map extraction. [ABSTRACT FROM AUTHOR]

Decreasing costs of vision sensors and advances in embedded hardware boosted lane related research – detection, estimation, tracking, etc. – in the past two decades. The interest in this topic has increased even more with the demand for advanced driver assistance systems (ADAS) and self-driving cars. Although extensively studied independently, there is still need for studies that propose a combined solution for the multiple problems related to the ego-lane, such as lane departure warning (LDW), lane change detection, lane marking type (LMT) classification, road markings detection and classification, and detection of adjacent lanes (i.e., immediate left and right lanes) presence. In this paper, we propose a real-time Ego-Lane Analysis System (ELAS) capable of estimating ego-lane position, classifying LMTs and road markings, performing LDW and detecting lane change events. The proposed vision-based system works on a temporal sequence of images. Lane marking features are extracted in perspective and Inverse Perspective Mapping (IPM) images that are combined to increase robustness. The final estimated lane is modeled as a spline using a combination of methods (Hough lines with Kalman filter and spline with particle filter). Based on the estimated lane, all other events are detected. To validate ELAS and cover the lack of lane datasets in the literature, a new dataset with more than 20 different scenes (in more than 15,000 frames) and considering a variety of scenarios (urban road, highways, traffic, shadows, etc.) was created. The dataset was manually annotated and made publicly available to enable evaluation of several events that are of interest for the research community (i.e., lane estimation, change, and centering; road markings; intersections; LMTs; crosswalks and adjacent lanes). Moreover, the system was also validated quantitatively and qualitatively on other public datasets. ELAS achieved high detection rates in all real-world events and proved to be ready for real-time applications. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel 2D homography computation method based on two real points. The homography is thus decomposed into three parts. The two real points and their images can be utilized to compute the first and the last parts respectively, while other primitives (could be point(s), line(s) and conic) can be utilized to compute the middle part which is a hyperbolic similarity transformation. We introduce the proposed method in a 2D pattern with a conic and a coplanar line, and apply the method in various other geometric patterns. Subsequently, many plane-based vision tasks, such as camera calibration, pose estimation and metric rectification can be solved in a unified way as polynomial systems. The experiments with simulated and real data verify the correctness and the versatility of our algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we propose a robust local descriptor for face recognition. It consists of two components, one based on a shearlet-decomposition and the other on local binary pattern (LBP). Shearlets can completely analyze the singular structures of piecewise smooth images, which is useful since singularities and irregular structures carry useful information in an underlying image. Furthermore, LBP is effective for describing the edges extracted by shearlets even when the images contain high level of noise. Experimental results using the Face Recognition Grand Challenge dataset show that the proposed local descriptor significantly outperforms many widely used features (e.g., Gabor and deep learning-based features) when the images are corrupted by random noise, demonstrating robustness to noise. In addition, experimental results show promising results for two challenging datasets which have poor image quality, i.e., a remote face dataset and the Point and Shoot Face Recognition Challenge dataset. [ABSTRACT FROM AUTHOR]

Robust detection of infrared dim and small target contributes significantly to the infrared systems in many applications. Due to the diversity of background scene and unique characteristic of target, the detection of infrared targets remains a challenging problem. In this paper, a novel approach based on total variation regularization and principal component pursuit (TV-PCP) is presented to deal with this problem. The principal component pursuit model only considers the low-rank feature of background images, which will result in poor detection ability in non-uniform and non-smooth scenes. We take into account the total variation regularization term to thoroughly describe background feature, which can achieve good detection result as well as good background estimation result. Firstly, the input infrared image is transformed to a patch image model. Secondly, the TV-PCP model is presented on the patch image. An effective optimization algorithm is proposed to solve this model. Experiments on six real datasets show that the proposed method has superior detection ability under various backgrounds, especially with good background suppression performance and low false alarm rate. [ABSTRACT FROM AUTHOR]

This paper presents a new set of three-dimensional rotation invariant texture descriptors based on the well-known local binary patterns (LBPs). In the approach proposed here, we extend an existing three-dimensional LBP based on the region growing algorithm using existing features developed exquisitely for two-dimensional LBPs (pixel intensities and differences). We have conducted experiments on a synthetic dataset of three-dimensional randomly rotated texture images in order to evaluate the discriminatory power and the rotation invariant properties of our descriptors as well as those of other two-dimensional and three-dimensional texture descriptors. Our results demonstrate the effectiveness of the extended LBPs and improvements against other state-of-the-art hand-crafted three-dimensional texture descriptors on this dataset. Furthermore, we prove that the extended LBPs can be used in medical datasets to discriminate between MR images of oxygenated and non-oxygenated brain tissues of newborn babies. [ABSTRACT FROM AUTHOR]

In this paper, we propose an unsupervised salient object segmentation approach using saliency and object features. In the proposed method, we utilize occlusion boundaries to construct a region-prior map which is then enhanced using object properties. To reject the non-salient regions, a region rejection strategy is employed based on the amount of detail (saliency information) and density of KAZE keypoints contained in them. Using the region rejection scheme, we obtain a threshold for binarizing the saliency map. The binarized saliency map is used to form a salient superpixel cluster. Finally, an iterative grabcut segmentation is applied with salient texture keypoints (SIFT keypoints on the Gabor convolved texture map) supplemented with salient KAZE keypoints (keypoints inside saliency cluster) as the foreground seeds and the binarized saliency map (obtained using the region rejection strategy) as a probably foreground region. We perform experiments on several datasets and show that the proposed segmentation framework outperforms the state of the art unsupervised salient object segmentation approaches on various performance metrics. [ABSTRACT FROM AUTHOR]

Face recognition based on image set has attracted much attention due to its promising performance to overcome various variations. Recently, classifiers of regularized nearest points, including sparse approximated nearest points (SANP), regularized nearest points (RNP) and collaborative regularized nearest points (CRNP), have achieved state-of-the-art performance for image set based face recognition. From a query set and a single-class gallery set, SANP and RNP both generate a pair of nearest points, between which the distance is regarded as the between-set distance. However, the computing of nearest points for each single-class gallery set in SANP and RNP ignores collaboration and competition with other classes, which may cause a wrong-class gallery set to have a small between-set distance. CRNP used collaborative representation to overcome this shortcoming but it doesn't explicitly minimize the between-set distance. In order to solve these issues and fully exploit the advantages of nearest points based approaches, in this paper a novel joint regularized nearest points (JRNP) is proposed for face recognition based on image sets. In JRNP, the nearest point in the query set is generated by considering the entire gallery set of all classes; at the same time, JRNP explicitly minimizes the between-set distance of the query set and a single-class gallery set. Furthermore, we proposed algorithms of greedy JRNP and adaptive JRNP to solve the presented model, and the classification is then based on the joint distance between the regularized nearest points in image sets. Extensive experiments were conducted on benchmark databases (e.g., Honda/UCSD, CMU Mobo, You Tube Celebrities databases, and the large-scale You Tube Face datasets). The experimental results clearly show that our JRNP leads the performance in face recognition based on image sets. [ABSTRACT FROM AUTHOR]

Due to the great progress made by local feature matching in both performance and robustness of dorsal hand vein recognition, this paper proposes a novel and effective approach for such an issue by improving two major steps of the SIFT-like framework, i.e. key-point detection and matching. For the former, a new key-point generation pattern, namely Centroid-based Circular Key-point Grid (CCKG), is presented, which efficiently localizes a certain number of points on the dorsal hand for the following SIFT feature extraction, leading to a discriminative description. In contrast to the existing key-point detectors, CCKG comprehensively accounts for the properties of the dorsal hand, including the vein network as well as the surrounding corium region, and hence achieves both good representativeness and low complexity. For the latter, a fine-grained matching process is introduced which makes use of Multi-task Sparse Representation Classifier (MtSRC). Compared with the traditional coarse-grained one that counts the number of associated SIFT features between the gallery and probe dorsal hand images, MtSRC precisely calculates the error of each feature of the probe as reconstructed by the gallery features, and all the errors of the probe features are combined for similarity measurement, reaching a better accuracy in recognition. The proposed approach is evaluated on the NCUT Part A database and shows its effectiveness in both the identification and verification scenarios. Additionally, the experimental results achieved on the NCUT Part B dataset highlight its generality and robustness to low quality images. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of image set classification, where each set contains a different number of images acquired from the same subject. In most of the existing literature, each image set is modeled using all its available samples. As a result, the corresponding time and storage costs are high. To address this problem, we propose a joint prototype and metric learning approach. The prototypes are learned to represent each gallery image set using fewer samples without affecting the recognition performance. A Mahalanobis metric is learned simultaneously to measure the similarity between sets more accurately. In particular, each gallery set is represented as a regularized affine hull spanned by the learned prototypes. The set-to-set distance is optimized via updating the prototypes and the Mahalanobis metric in an alternating manner. To highlight the importance of representing image sets using fewer samples, we analyzed the corresponding test time complexity with respect to the number of images used per set. Experimental results using YouTube Celebrity, YouTube Faces, and ETH-80 datasets illustrate the efficiency on the task of video face recognition, and object categorization. [ABSTRACT FROM AUTHOR]

Salient object detection is an important issue in computer vision and image procession in that it can facilitate humans to locate conspicuous visual regions in complex scenes rapidly and improve the performance of object detection and video tracking. In recent years, low-rank matrix approximation has been proved to be favorable in image saliency detection and gained a great deal of attention. An underlying assumption of low-rank recovery is that an image is a combination of background regions being low-rank and salient objects being sparse, which corresponds to tough non-smooth optimization problems. In this paper, by incorporating ℓ 2,1 -norm minimization, we obtain the corresponding smooth optimization problems and propose two effective algorithms with proved convergence. To guarantee the robustness of the proposed methods, the input image is divided into patches and each patch is approximately represented by its mean value. Besides, multi-scale visual features of each patch of the given image are extracted to capture common low-level features such as color, edge, shape and texture. The salient objects of a given image are indicated with sparse coefficients solved by the low-rank matrix approximation problem. Saliency maps are further produced with integration of the high-level prior knowledge. Finally, extensive experiments in four real-world datasets demonstrate that the proposed methods come with competitive performance over the eight compared state-of-the-arts. [ABSTRACT FROM AUTHOR]

“What is the state-of-the-art in sketch recognition and what are some important future research directions in matching sketches with digital face images?” This opinion paper focuses on answering these questions through proposing three important steps that need to move the field forward: (i) create a large, real world forensic sketch database, (ii) develop fundamental understanding of human cognition of processing sketches, and (iii) develop improved algorithms for matching sketches with mugshot photos. [ABSTRACT FROM AUTHOR]

Visual surveillance cameras have been massively deployed in public urban environments over the recent years, as a crime prevention and law enforcement solution. This fact raised the interest in developing automata to infer useful information from such crowded scenes (from abnormal behavior detection to human identification). In order to cover wide outdoor areas, one interesting possibility is to combine wide-angle and pan–tilt–zoom (PTZ) cameras in a master–slave configuration. The use of fish-eye lenses allows the master camera to maximize the coverage area while the PTZ acts as a foveal sensor, providing high-resolution images of the interest regions. This paper addresses the feasibility of using this type of data acquisition paradigm for imaging iris/periocular data with enough discriminating power to be used for biometric recognition purposes. [ABSTRACT FROM AUTHOR]

The selection of canonical images that best represent a scene type is very important for efficiently visualizing search results and re-ranking them. In this paper, we propose the selection of canonical images based on human affects that are hidden in the image. One is a probabilistic affective model (PAM) based probabilistic latent semantic analysis (PLSA) learning to annotate the image by human affects and the other is the cluster ranking algorithm to select the informative summary from vast search results. The PAM first extract the dominant color compositions (CCs) that constitute the image itself, through image segmentation and RAG analysis, then to infer numerical ratings from CCs for affective classes, a PLSA is employed that is well-known method in finding latent semantics from documents. Once converting the images to the affective space using PAM, the clustering is performed. Then to select the images that are representative among the images and are distinctive from each other, we identify three dominant properties such as coverage, affective coherence, and distinctiveness. Based on these, cluster ranking is performed. Finally, the representative images for each cluster are selected, all of which are displayed as canonical images to the user. Experiments were performed on Photo.Net and Google images and compared the results with other existing methods. Then our PAM showed the F 1 -scores of 0.667 on averages, which can improve 14% of the existing method. In addition, it is proven that the proposed system is superior to the others in selecting the canonical images, when comparing its performance with two baselines in terms of representative and diverse scores. [ABSTRACT FROM AUTHOR]

The non-linearity in medical image processing is a critical issue. Because the privacy of the medical image and loss of data is a major concern in recent years. Federated learning is a most advanced form of machine learning in which, rather than transmitting data to local server, a machine learning (ML) algorithm is installed to various devices to train on the information. The parameters from the separate modules will then be transferred to a master ML/ (deep learning) DL model for global training. The research of Image Quality Assessment (IQA) aims to simulate the process of human perception of image quality and construct an objective image quality model as consistent as possible with subjective assessment. The existing IQA methods can be roughly divided into traditional methods and deep learning methods. Traditional methods are knowledge-driven, using prior knowledge or assumptions about the human visual system (HVS) to heuristically design image quality index. Deep learning methods are data-driven, using a large amount of annotated data to learn the mapping from the image to its visual quality end-to-end. To effectively integrate traditional methods into deep networks and investigate the knowledge (model)-driven deep learning methods are the current mainstream trends in IQA research. In this paper, we take the contrary direction and improve traditional methods guided by the cue from deep learning methods. The main works include: 1. the employment of activation function ensure the nonlinear approximation ability of the neural network, here we first extend the two-stage framework widely used in the field of full-reference image quality assessment and propose a nonlinear two-stage framework. 2. Within this framework, we revisit the Edge Strength SIMlarity (ESSIM) algorithm that we previously published in IEEE Signal Processing Letters, and proposed the Nonlinear Edge Strength SIMlarity (NESSIM) algorithm. Experiments on public databases show that NESSIM can obtain good assessment results in traditional methods. • Improve traditional methods guided by the cue from deep learning methods. • Employment of activation function ensure the nonlinear approximation ability of the neural network. • Propose a nonlinear two-stage framework • Proposed the Nonlinear Edge Strength SIMlarity (NESSIM) algorithm. [ABSTRACT FROM AUTHOR]

Computer Vision has recently witnessed great research advance towards automatic facial points detection. Numerous methodologies have been proposed during the last few years that achieve accurate and efficient performance. However, fair comparison between these methodologies is infeasible mainly due to two issues. (a) Most existing databases, captured under both constrained and unconstrained (in-the-wild) conditions have been annotated using different mark-ups and, in most cases, the accuracy of the annotations is low. (b) Most published works report experimental results using different training/testing sets, different error metrics and, of course, landmark points with semantically different locations. In this paper, we aim to overcome the aforementioned problems by (a) proposing a semi-automatic annotation technique that was employed to re-annotate most existing facial databases under a unified protocol, and (b) presenting the 300 Faces In-The-Wild Challenge (300-W), the first facial landmark localization challenge that was organized twice, in 2013 and 2015. To the best of our knowledge, this is the first effort towards a unified annotation scheme of massive databases and a fair experimental comparison of existing facial landmark localization systems. The images and annotations of the new testing database that was used in the 300-W challenge are available from http://ibug.doc.ic.ac.uk/resources/300-W_IMAVIS/ . [ABSTRACT FROM AUTHOR]

Pedestrian detection is an important image understanding problem with many potential applications. There has been little success in creating an algorithm which exhibits a high detection rate while keeping the false alarm in a relatively low rate. This paper presents a method designed to resolve this problem. The proposed method uses the Kinect or any similar type of sensors which facilitate the extraction of a distinct foreground. Then potential regions, which are candidates for the presence of human(s), are detected by employing the widely used Histogram of Oriented Gradients (HOG) technique, which performs well in terms of good detection rates but suffers from significantly high false alarm rates. Our method applies a sequence of operations to eliminate the false alarms produced by the HOG detector based on investigating the fine details of local shape information. Local shape information can be identified by efficient utilization of the edge points which, in this work, are used to formulate the so called Shape Context (SC) model. The proposed detection framework is divided in four sequential stages, with each stage aiming at refining the detection results of the previous stage. In addition, our approach employs a pre-evaluation stage to pre-screen and restrict further detection results. Extensive experimental results on the dataset created by the authors, involves 673 images collected from 11 different scenes, demonstrate that the proposed method eliminates a large percentage of the false alarms produced by the HOG pedestrian detector. [ABSTRACT FROM AUTHOR]

We propose to represent a time-of-flight (TOF) camera by the map of "internal radial distances" (IRD), associating an intrinsic distance to each pixel, as an alternative for the classic pinhole model. This representation is more general than the perspective model and appears to be a natural concept for 3D reconstruction and other applications of TOF cameras. In this new framework, calibrating a ToF camera comes down to the determination of this IRD map. We show how this can be accomplished by images of flat surfaces, without performing any feature detection. We prove deterministic calibration formulas, using one or more plane images. We also offer a numerical optimization method that in principle needs only one image of a flat surface. This paper has been recommended for acceptance by Peter Sturm. [ABSTRACT FROM AUTHOR]

This paper presents a novel structure-aware method for visual tracking. The proposed tracker relies on keypoint regions as salient and stable elements that encode the object structure efficiently. In addition to the object structural properties, the appearance model also includes global color features that we first use in a probabilistic approach to reduce the search space. The second step of our tracking procedure is based on keypoint matching to provide a preliminary prediction of the target state. Final prediction is then achieved by exploiting object structural constraints, where target keypoints vote for the corrected object location. Once the object location is obtained, we update the appearance model and structural properties, allowing tracking targets with changing appearance and non-rigid structures. Extensive experiments demonstrate that the proposed Structure-Aware Tracker (SAT) outperforms recent state-of-the-art trackers in challenging scenarios, especially when the target is partly occluded and in moderately crowded scenes. [ABSTRACT FROM AUTHOR]

Semi-supervised sparse feature selection, which can exploit the large number unlabeled data and small number labeled data simultaneously, has placed an important role in web image annotation. However, most of the semi-supervised sparse feature selection methods are developed for single-view data and these methods cannot naturally deal with the multi-view data, though it has shown that leveraging information contained in multiple views can dramatically improve the feature selection performance. Recently, multi-view learning has obtained much research attention because it can reveal and leverage the correlated and complementary information between different views. So in this paper, we apply multi-view learning into semi-supervised sparse feature selection and propose a semi-supervised sparse feature selection method based on multi-view Laplacian regularization, namely, multi-view Laplacian sparse feature selection (MLSFS). 1 1 MLSFS: Multi-view Laplacian Sparse Feature Selection. MLSFS utilizes multi-view Laplacian regularization to boost semi-supervised sparse feature selection performance. A simple iterative method is proposed to solve the objective function of MLSFS. We apply MLSFS algorithm into image annotation task and conduct experiments on two web image datasets. The experimental results show that the proposed MLSFS outperforms the state-of-art single-view sparse feature selection methods. [ABSTRACT FROM AUTHOR]

In this paper, we address the document image binarization problem with a three-stage procedure. First, possible stains and general document background information are removed from the image through a background removal stage. The remaining misclassified background and character pixels are then separated using a Local Co-occurrence Mapping, local contrast and a two-state Gaussian Mixture Model. Finally, some isolated misclassified components are removed by a morphology operator. The proposed scheme offers robust and fast performance, especially for both handwritten and printed documents, which compares favorably with other binarization methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a method for face recognition by using the two-dimensional discrete wavelet transform (2D-DWT) and a new patch strategy. Based on the average image of all training samples, by using integral projection technique for two top-level's high-frequency sub-bands of 2D-DWT, we propose a non-uniform patch strategy for the top-level's low-frequency sub-band. This patch strategy is more suitable to reflect the structure feature of face image; and it is better for retaining the integrity of local information. By applying the obtained patch strategy to all samples, we obtain patches of training samples and testing samples; and then, give the final decision by using the nearest neighbor classifier and the majority voting. Experiments are run on the AR, FERET, Extended Yale B and LFW face databases. The obtained numerical results show that the new face recognition method outperforms the traditional 2D-DWT method and some state-of-the-art patch based methods. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method to address the registration of images with affine transformation. Firstly, the Maximally Stable Extremal Region (MSER) detection method is performed on the reference image and the image to be registered, respectively. And the coarse affine transformation matrix between the two images is estimated by the matched MSER pairs. Two circular regions containing roughly the same image content are also obtained by fitting and normalizing the centroids of the matched MSERs from the two images. Secondly, a scale invariant and approximate affine transformation invariant feature point detection algorithm based on the Gabor filter decomposition and phase congruency is performed on the two coarsely aligned regions, and two feature point sets are achieved, respectively. Finally, the affine transformation matrix between the two feature point sets is obtained by using a probabilistic point set registration algorithm, and the final affine transformation matrix between the reference image and the image to be registered is achieved according to the coarse affine transformation matrix and the affine transformation matrix between the two feature point sets. Several sets of experiments demonstrate that our proposed method performs competitively with the classical scale-invariant feature transform (SIFT) method for images with scale changes, and performs better than the traditional MSER and Affine-SIFT (ASIFT) methods for images with affine distortions. Moreover, the proposed method shows higher computation efficiency and robustness to illumination change than some existing area-based or feature-based methods do. [ABSTRACT FROM AUTHOR]

In this paper, a novel method is proposed for real-world pose-invariant face recognition from only a single image in a gallery. A 3D Facial Expression Generic Elastic Model (3D FE-GEM) is proposed to reconstruct a 3D model of each human face using only a single 2D frontal image. Then, for each person in the database, a Sparse Dictionary Matrix (SDM) is created from all face poses by rotating the 3D reconstructed models and extracting features in the rotated face. Each SDM is subsequently rendered based on triplet angles of face poses. Before matching to SDM, an initial estimate of triplet angles of face poses is obtained in the probe face image using an automatic head pose estimation approach. Then, an array of the SDM is selected based on the estimated triplet angles for each subject. Finally, the selected arrays from SDMs are compared with the probe image by sparse representation classification. Convincing results were acquired to handle pose changes on the FERET, CMU PIE, LFW and video face databases based on the proposed method compared to several state-of-the-art in pose-invariant face recognition. [ABSTRACT FROM AUTHOR]

In this paper, a novel and effective lip-based biometric identification approach with the Discrete Hidden Markov Model Kernel (DHMMK) is developed. Lips are described by shape features (both geometrical and sequential) on two different grid layouts: rectangular and polar. These features are then specifically modeled by a DHMMK, and learnt by a support vector machine classifier. Our experiments are carried out in a ten-fold cross validation fashion on three different datasets, GPDS-ULPGC Face Dataset, PIE Face Dataset and RaFD Face Dataset. Results show that our approach has achieved an average classification accuracy of 99.8%, 97.13%, and 98.10%, using only two training images per class, on these three datasets, respectively. Our comparative studies further show that the DHMMK achieved a 53% improvement against the baseline HMM approach. The comparative ROC curves also confirm the efficacy of the proposed lip contour based biometrics learned by DHMMK. We also show that the performance of linear and RBF SVM is comparable under the frame work of DHMMK. [ABSTRACT FROM AUTHOR]

Deep learning-based object detection solutions emerged from computer vision has captivated full attention in recent years. The growing UAV market trends and interest in potential applications such as surveillance, visual navigation, object detection, and sensors-based obstacle avoidance planning have been holding good promises in the area of deep learning. Object detection algorithms implemented in deep learning framework have rapidly became a method for processing of moving images captured from drones. The primary objective of the paper is to provide a comprehensive review of the state of the art deep learning based object detection algorithms and analyze recent contributions of these algorithms to low altitude UAV datasets. The core focus of the studies is low-altitude UAV datasets because relatively less contribution was seen in the literature when compared with standard or remote-sensing based datasets. The paper discusses the following algorithms: Faster RCNN, Cascade RCNN, R-FCN etc. into two-stage, YOLO and its variants, SSD, RetinaNet into one-stage and CornerNet, Objects as Point etc. under advanced stages in deep learning based detectors. Further, one-two and advanced stages of detectors are studied in detail focusing on low-altitude UAV datasets. The paper provides a broad summary of low altitude datasets along with their respective literature in detection algorithms for the potential use of researchers. Various research gaps and challenges for object detection and classification in UAV datasets that need to deal with for improving the performance are also listed. [ABSTRACT FROM AUTHOR]

Nowadays, the technology of using computer vision to calibrate objects is widely used, which has a huge market demand in many fields. This paper provides a calibration method of computer vision system based on dual attention neural network. This paper uses the camera to simulate human eyes to obtain three-dimensional images. After obtaining the three-dimensional images, the images are input into the Residual Network (ResNet) model, and the weight of ResNet is repeatedly updated so as to accurately identify the images. On this basis, introduces dual attention mechanism that an algorithm is used in natural language to the visual image processing, using multistage feature extraction method to extract the three-dimensional image for each characteristic of regional. After extracting the feature area, the accuracy of the feature area is constantly updated to the minimum. Besides, the feature areas are brought into the calibration algorithm of Zhang Zhengyou system to obtain the spatial coordinates of the objects in the attention area. This method can realize the space position calibration of specific objects under various complex backgrounds and calculate the distance from the calibrated objects, which can not only calibrate the system but also identify it, and greatly improve the reliability and accuracy of the calibration process. • The dual attention model realizes object calibration in complex background. • The dual attention mechanism can be used to calibrate and identify the system. • The proposed algorithm makes the calibration error less than 0.1%. • The algorithm is used to identify and calibrate parking space. [ABSTRACT FROM AUTHOR]

Fire is one of the main disasters in the world. A fire detection system should detect fires in various environments (e.g., buildings, forests, and rural areas) in the shortest time in order to reduce financial losses and humanistic disasters. Fire sensors are, in fact, complementary to conventional point sensors (e.g., smoke and heat detectors), which provide people the early warnings of fire occurrences. Cameras combined with image processing techniques detect fire occurrences more quickly than point sensors. Moreover, they provide the size, growth, and direction of fires more easily than their conventional detectors. This paper, initially, presents a glance view on the main features of various environments including buildings, forests, and mines that should be considered in the design of fire detection systems. Afterwards, it describes some of the intelligent and vision-based fire detection systems that have been presented by researchers in the last decade. These systems are categorized, in this paper, into two groups: intelligent detection systems for forest fires and intelligent fire detection systems for all of the environments. They use various intelligent techniques (e.g., convolutional neural networks, color models, and fuzzy logic) to detect fire occurrences with a high accuracy in various environments. Performances of the fire detection systems are compared to each other in terms of detection rate, precision, true-positive rate, false-positive rate, etc. under different evaluation scenarios. • Reduce financial loss and humanistic disasters by intelligent fire detection systems • Combination of the cameras and image processing to detect fire occurrences quickly • Reduction of the fires using the intelligent and vision-based detection processes • Improvement of the fire detection processes by intelligent techniques (e.g., CNN) • Enhance accuracy, detection rate, and precision of the fire detection systems [ABSTRACT FROM AUTHOR]

Superpixel is one of the most popular image over-segmentations with broad applications in the computer vision field to reduce their computations by replacing pixels as primitives. The main concerns of one superpixel generation algorithm are its accuracy and efficiency. One of the most important things in superpixel accuracy is to fit the image boundaries tightly with a few pixels as possible (namely minimal contour density, which is measured by the percent of superpixel contour pixels in the whole image). In this paper, we propose a new fast algorithm based on the clustering method to produce superpixels accurately with low contour density. First, we adopt the linear path from a pixel to one superpixel seed to define a regular term and propose a new distance measurement between them. In addition, we introduce the gradient and Local Binary Pattern (LBP) features and propose formulas of parameters in the proposed method adaptively. In this way, we can use the new distance measurement to group pixels as initial regions adaptively and produce the final superpixels by merging those small ones. Finally, we test the new algorithm on two public datasets and compare it with the state-of-the-art. Our method can generate superpixels with lower contour density while being competitive in accuracy and computational time. [ABSTRACT FROM AUTHOR]

Matrix data has arised in many field, especially in the field of image processing and computer vision. In traditional approaches, the original images need to be vectorized to one-dimension vectors, which may destroy the inherent structure of images. A novel geometrical sparse representation (GSR) model with single image is introduced in this paper that solves a model to measure the similarity between the input image and the single dictionary image. Unlike the traditional sparse representation model, the proposed model does not need to vectorize the image, so as to preserve the inherent geometrical structure of the image. We further introduce a binary coding method to preserve the local patterns of the image and enhance the sparsity of the GSR coefficients. Our method is used for face images with variations of structural noise (occlusion, illumination, etc.), extensive experiments show that our method can be competitive with or even superior to the baseline methods. • A novel GSR model that explores inherent geometric structure of the image. • An analysis shows that the GSR model is a generalized solution to illumination reflectance model. • An analysis shows that the proposed method can alleviate the effects of occlusion effectively. • Extensive experiments prove the proposed method is robust to occlusion and illumination changes. [ABSTRACT FROM AUTHOR]

Research on human eye image processing and iris recognition has grown steadily over the last few decades. It is important for researchers interested in this discipline to know the relevant datasets in this area to (i) be able to compare their results and (ii) speed up their research using existing datasets rather than creating custom datasets. In this paper, we provide a comprehensive overview of the existing publicly available datasets and their popularity in the research community using a bibliometric approach. We reviewed 158 different iris datasets referenced from the 689 most relevant research articles indexed by the Web of Science online library. We categorized the datasets and described the properties important for performing relevant research. We provide an overview of the databases per category to help investigators conducting research in the domain of iris recognition to identify relevant datasets. [Display omitted] • We reviewed 158 existing iris datasets and provide critical discussion • We performed bibliometric research to estimate the popularity of the identified datasets • We discuss limitations of existing datasets • We identified common properties of successful datasets and provide recommendations for collecting new datasets [ABSTRACT FROM AUTHOR]

In view of the difficulty in training the algorithm of image oil painting style migration and reconstruction based on the generative adversarial network, and the loss gradient of generator and discriminator disappears, this paper proposes an improved generative adversarial network based on gradient penalty, and constructs the total variance loss function to carry out the research of image oil painting style migration and reconstruction. Firstly, the Wasserstein distance (WGAN) is added to the loss function of the generative adversarial network to improve the stability of the alternative iterative training; secondly, the gradient penalty (WGAN-GP) is added to the loss function to deal with the problem of gradient disappearance in the training; finally, the LBP texture feature and total variation of the prototype are introduced based on the CycleGAN Loss noise constraint is used to improve the edge and texture strength of the image after migration of oil painting style. The experimental results show that the WGAN-GP algorithm constructed in this study has the ability of stable gradient and alternating iterative convergence, and the total variation loss noise constraint can provide good edge and texture details for the migration process of image oil painting style. Compared with the existing mainstream algorithm, the algorithm proposed in this study has better performance of image oil painting style migration and reconstruction, and better effect of image oil painting style migration and reconstruction. • WGAN-GP has stable gradient and alternating iterative convergence ability. • WGAN-GP provide good edge and texture details for the migration process • WGAN-GP provide better performance of image oil painting style migration & reconstruction. [ABSTRACT FROM AUTHOR]

Multiple object tracking is an important but challenging computer vision problem. The complex motion of objects makes tracking difficult during long periods of object occlusion, and as a result occlusions frequently cause fragmented tracks with gaps. Previous works use linear interpolation to fill in such gaps, a technique which is only able to model simple motion. As a result, tracked bounding box locations can be quite poor in these situations. In this paper, we propose a 1D CNN based solution to filling gaps which models complex motion in a data-driven way. Our proposed solution uses only bounding box coordinates as input, and as such does not incur the computational cost of processing image features directly. We show that our model significantly outperforms linear interpolation on dynamic sports datasets in terms of mean intersection over union between predicted and ground truth bounding boxes. • Overlap between athletes in team sports creates fragmented tracks. • A learned method to fill fragmented tracks. • Repair fragmented tracks using 1-dimensional convolutional neural networks. • Track highly complex motion of players in team sports. [ABSTRACT FROM AUTHOR]

Recent advances in system resources provide ease in the applicability of deep learning approaches in computer vision. In this paper, we propose a deep learning-based unsupervised image segmentation approach for natural image segmentation. Image segmentation aims to transform an image into regions, representing various objects in the image. Our method consists of a fully convolutional dense network-based unsupervised deep representation oriented clustering, followed by shallow features based high-dimensional region merging to produce the final segmented image. We evaluate our proposed approach on the BSD300 database and perform a comparison with several classical and some recent deep learning-based unsupervised segmentation methods. The experimental results represent that the proposed method is comparable and confirm the efficacy of the proposed approach. • Unsupervised image segmentation employing deep learning approaches is a recent advancement. • FC dense u-net deep network is employed to generate a high-quality clustering-based over-segmented image. • Feature fusion based high-dimensional feature vector is employed to effectively merge similar regions. • High-dimensional feature vector based region merging performs far better as compared to simple feature region merging. [ABSTRACT FROM AUTHOR]

An individual's privacy is a significant concern in surveillance videos. Existing research work into the location of individuals on the basis of detecting their skin is focused either on different techniques for detecting human skin on protecting individuals from the consequences of applying such techniques. This paper considers both lines of research and proposes a hybrid scheme for human skin detection and subsequent privacy protection by utilizing color information in dynamically varying illumination and environmental conditions. For those purposes, dynamic and explicit skin-detection approaches are implemented, simultaneously considering multiple color-spaces, i.e. RGB, perceptual (HSV) and orthogonal (YCbCr) color-spaces, and then detecting the human skin by the proposed Combined Threshold Rule (CTR)-based segmentation. Comparative qualitative and quantitative detection results with an average 93.73% accuracy, imply that the proposed scheme achieves considerable accuracy without incurring a training cost. Once skin detection has been performed, the detected skin pixels (including false positives) are encrypted, when standard AES-CFB encryption of skin pixels is shown to be preferable compared to selective encryption of a whole video frame. The scheme preserves the behavior of the subjects within the video. Hence, subsequent image processing and behavior analysis, if required, can be performed by an authorized user. The experimental results are encouraging, as they show that the average encryption time is 8.268 s and the Encryption Space Ratio (ESR) is an average 7.25% for a high definition video (1280 × 720 pixels/frame). A performance comparison in terms of Correct Detection Rate (CDR) showed an average 91.5% for CTB-based segmentation compared to using only one color space for segmentation, such as using RGB with 85.86%, HSV with 80.93% and YCbCr with an average 84.8%, which implies that the proposed method of combining color-space skin identifications has a higher ability to detect skin accurately. Security analysis confirmed that the proposed scheme could be a suitable choice for real-time surveillance applications operating on resource-constrained devices. [ABSTRACT FROM AUTHOR]

The technique of sparse representation (SR) has achieved enormous successes in multi-source image fusion. However, using SR-based fusion methods, there exists the performance degradation in limited detail preservation caused by the independent processing of image patches. To remedy this deficiency, in this paper, a novel method based on spatially masked convolutional sparse representation (SMCSR-based) is proposed for image fusion, which is composed of three steps as follows. Firstly, low-frequency and high-frequency bands are separated from each source image by the designed two-scale gradient optimization approach. Secondly, the SMCSR model is employed to fuse the high-frequency bands, and the "average" rule is applied to the combination of the low-frequency bands. At last, the fused image is reconstructed. Compared with traditional SR-based algorithms, the proposed SMCSR-based method is focused on entire images instead of those divided patches to reduce detail-loss. In addition, this method can overcome the difficulty in the selection of decomposition levels originated by multi-scale transform (MST) based fusion strategies, as well as also suppress the boundary artifacts produced by the traditional convolutional sparse representation (CSR) model. Extensive experiments and related analysis are given to verify the effectiveness of the proposed SMCSR-based fusion method. • A general two-scale decomposition framework is built. • The convolutional sparse processing is used for multi-source fusion. • The boundary handing with spatial masking is added into the convolutional sparse model. [ABSTRACT FROM AUTHOR]