Search

Brain oedema is a common pathological phenomenon following many diseases and may lead to severe secondary damage. Astrocytes are the most numerous cells in the brain. Five aquaporins (AQPs) have been found in mature astrocytes, which play crucial roles in water transportation. However, most studies have focused on AQP4 or AQP9 and whether another aquaporin such as AQP5 involved in brain oedema is unclear. Here, we addressed the issue that the expression pattern of AQP5 in rat astrocytes in vitro was altered in the hypotonic condition through some mitogen‐activated protein kinases (MAPK) pathways. Primary astrocytes were randomly divided into the control group and the hypotonic group. Cell viability was evaluated by MTT test. Immunofluorescence, Western blotting and real‐time PCR were used to detect the expression of AQP5. Western blotting was used to detect the variation of MAPK pathway. The present study demonstrated that incubation of astrocytes in the hypotonic medium produced an increase inAQP5 expression, and AQP5 peaked at 6–12 h after hypotension solution exposure. In addition, MAPK pathways were set in motion under hypotension, but not all branches. Only the p38 inhibitor can inhibit AQP5 expression in cultured astrocytes. AQP5 is directly related to the extracellular hypotonic stimuli in astrocytes, which could be regulated through the p38 MAPK pathway. [ABSTRACT FROM AUTHOR]

Image clustering is a very useful technique that is widely applied to various areas, including remote sensing. Recently, visual representations by self-supervised learning have greatly improved the performance of image clustering. To further improve the well-trained clustering models, this paper proposes a novel method by first ranking samples within each cluster based on the confidence in their belonging to the current cluster and then using the ranking to formulate a weighted cross-entropy loss to train the model. For ranking the samples, we developed a method for computing the likelihood of samples belonging to the current clusters based on whether they are situated in densely populated neighborhoods, while for training the model, we give a strategy for weighting the ranked samples. We present extensive experimental results that demonstrate that the new technique can be used to improve the state-of-the-art image clustering models, achieving accuracy performance gains ranging from 2.1 % to 15.9 % . Performing our method on a variety of datasets from remote sensing, we show that our method can be effectively applied to remote sensing images. [ABSTRACT FROM AUTHOR]

One of the challenges in the study of generative adversarial networks (GANs) is the difficulty of its performance control. Lipschitz constraint is essential in guaranteeing training stability for GANs. Although heuristic methods such as weight clipping, gradient penalty and spectral normalization have been proposed to enforce Lipschitz constraint, it is still difficult to achieve a solution that is both practically effective and theoretically provably satisfying a Lipschitz constraint. In this paper, we introduce the boundedness and continuity (BC) conditions to enforce the Lipschitz constraint on the discriminator functions of GANs. We prove theoretically that GANs with discriminators meeting the BC conditions satisfy the Lipschitz constraint. We present a practically very effective implementation of a GAN based on a convolutional neural network (CNN) by forcing the CNN to satisfy the BC conditions (BC–GAN). We show that as compared to recent techniques including gradient penalty and spectral normalization, BC–GANs have not only better performances but also lower computational complexity. [ABSTRACT FROM AUTHOR]

Label ranking aims to learn a mapping from instances to rankings over a finite number of predefined labels. Random forest is a powerful and one of the most successful general-purpose machine learning algorithms of modern times. In this paper, we present a powerful random forest label ranking method which uses random decision trees to retrieve nearest neighbors. We have developed a novel two-step rank aggregation strategy to effectively aggregate neighboring rankings discovered by the random forest into a final predicted ranking. Compared with existing methods, the new random forest method has many advantages including its intrinsically scalable tree data structure, highly parallel-able computational architecture and much superior performance. We present extensive experimental results to demonstrate that our new method achieves the highly competitive performance compared with state-of-the-art methods for datasets with complete ranking and datasets with only partial ranking information. [ABSTRACT FROM AUTHOR]

Symmetric positive definite (SPD) matrices have achieved considerable success in numerous computer vision applications including activity recognition, texture classification, and diffusion tensor imaging. Traditional pattern recognition methods developed in Euclidean space are not suitable for direct processing of SPD matrices because they lie in a Riemannian manifold of negative curvature. In this paper, we draw inspiration from neural network based simple competitive learning, which updates the weight of a neural network in Euclidean space, and propose Riemannian competitive learning which updates the weight in the Riemannian weight space, for SPD manifold clustering. This framework is based on the Fisher-Rao metric which is theoretically very good. We further introduce a conscious competition mechanism to enhance the performance of the RCL algorithm and have developed a robust algorithm termed Riemannian Frequency Sensitive Competitive Learning (rFSCL). Compared with existing methods, the advantages are threefold: (1) rFSCL inherits the online nature of competitive learning, which makes it capable of handling very large data sets; (2) rFSCL inherits the advantage of conscious competitive learning, which indicates that it is less sensitive to the initial values of the cluster centers and that all clusters are fully utilized without the “dead unit” problem associated with many clustering algorithms; and (3) As an intrinsic Riemannian clustering method, rFSCL operates along the geodesic on the manifold in an closed-form analytic manner and is completely independent of the choice of local coordinate systems. Extensive experiments on both synthetic data sets and real data sets show its superior performance to that of other state-of-the-art SPD clustering methods. [ABSTRACT FROM AUTHOR]

The classification of habitats is crucial for structuring knowledge and developing our understanding of the natural world. Currently, most successful methods employ human surveyors -a laborious, expensive and subjective process. In this paper, we formulate habitat classification as a fine-grained visual categorization problem. We build on previous work and propose an image annotation framework that uses a novel automatic random forest-based method and that takes into consideration visual and geographical closeness in the classification process. During training, low-level visual features and medium-level contextual information are extracted. For the latter, we use a human-in-the-loop methodology by asking humans a set of 17 questions about the appearances of the image that can be easily answered by non-ecologists to extract medium-level knowledge about the images. During testing, and considering that close areas have similar ecological properties, we weigh the influence of the prediction of each tree of the forest according to their distance to the unseen test photography. Additionally, we present an updated version of a geo-referenced habitat image database containing over 1,000 high-resolution ground photographs that have been manually annotated by habitat classification experts. This has been made publicly available image database specifically designed for the development of multimedia analysis techniques for ecological applications. We show experimental recall and precision results which illustrate that our image annotation framework is able to annotate with a reasonable degree of confidence four of the main habitat classes: woodland and scrub, grassland and marsh, heathland and miscellaneous. [ABSTRACT FROM AUTHOR]

Objective The white matter injury caused by intracerebral hemorrhage (ICH) includes demyelination and axonal injury. Oligodendrocyte apoptosis is reported to be involved in triggering demyelination. Experimental observations indicate that both endoplasmic reticulum and mitochondrial pathways could mediate cell apoptosis. The purpose of this study was to investigate the demyelination and the possible mechanisms in an autologous blood-injected rat model of internal capsule hemorrhage. Methods Transmission electron microscope was applied to examine the pathological changes of myelinated nerve fibers in internal capsule. Western blotting was used to detect the myelin basic protein (MBP) which was an important component of myelin sheath. Double immunofluorescence and Western blotting were used to determine the apoptosis and apoptotic pathways. The levels of caspase-12 (a representative protein of endoplasmic reticulum stress) and cytochrome c (an apoptosis factor released from mitochondria) were assessed in this study. Results Demyelination occurred on day 1, 3, and 7 after ICH onset. Myelin sheaths of internal capsule nerve fibers were swollen and broken down in ICH groups. MBP expression showed a downregulation after ICH with its minimum value occurred on day 7 post-ICH. Besides, neuron and oligodendrocyte apoptosis were observed at different time intervals post-ICH accompanied with an upregulated caspase-12 expression and enhanced cytochrome c release. Conclusions These results suggested that oligodendrocyte and neuron apoptosis may contribute to the demyelination induced by internal capsule hemorrhage and oligodendrocyte apoptosis is positively mediated through both endoplasmic reticulum and mitochondrial pathways. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel method to detect glandular structures in microscopic images of human tissue. We first convert the image from Cartesian space to polar space and then introduce a novel random field model to locate the possible boundary of a gland. Next, we develop a visual feature-based support vector regressor to verify if the detected contour corresponds to a true gland. And finally, we combine the outputs of the random field and the regressor to form the GlandVision algorithm for the detection of glandular structures. Our approach can not only detect the existence of the gland, but also can accurately locate it with pixel accuracy. In the experiments, we treat the task of detecting glandular structures as object (gland) detection and segmentation problems respectively. The results indicate that our new technique outperforms state-of-the-art computer vision algorithms in respective fields. [ABSTRACT FROM PUBLISHER]

Abstract: The aim of semantic segmentation is to assign each pixel a semantic label. Numerous methods for semantic segmentation have been proposed in recent years and most of them chose pixel or superpixel as the processing primitives. However, as the information contained in a pixel or a superpixel is not discriminative enough, the outputs of these algorithms are usually not object consistent. To tackle this problem, we introduce the concept of object-like regions as a new and higher level processing primitive. We first experimentally showed that using groundtruth segments as processing primitives can boost semantic segmentation accuracy, and then proposed a novel method to produce regions that resemble the groundtruth regions, which we named them as ‘object-like regions’. We achieve this by integrating state of the art low-level segmentation algorithms with typical semantic segmentation algorithms through a novel semantic feature feedback mechanism. We present experimental results on the publicly available image understanding dataset MSRC21 and stanford background dataset, showing that the new method can achieve relatively good semantic segmentation results with far fewer processing primitives. [Copyright &y& Elsevier]

This paper presents new designs of graph convolutional neural networks (GCNs) on 3D meshes for 3D object segmentation and classification. We use the faces of the mesh as basic processing units and represent a 3D mesh as a graph where each node corresponds to a face. To enhance the descriptive power of the graph, we introduce a 1-ring face neighborhood structure to derive novel multi-dimensional spatial and structure features to represent the graph nodes. Based on this new graph representation, we then design a densely connected graph convolutional block which aggregates local and regional features as the key construction component to build effective and efficient practical GCN models for 3D object classification and segmentation. We present experimental results to show that our new technique performs comparably to state of the art across a number of benchmark datasets where our models are also shown to have smaller number of parameters. We also present ablation studies to demonstrate the soundness of our design principles and the effectiveness of our practical models. • A novel graph data structure for representing 3D mesh data. • A novel densely connected graph convolutuional neural network for 3D meshes. • 3D object segmentation and classification. [ABSTRACT FROM AUTHOR]

Abstract: We have developed an informative sample subspace (ISS) method that is suitable for projecting high-dimensional data onto a low-dimensional subspace for classification purposes. In this paper, we present an ISS algorithm that uses a maximal mutual information criterion to search a labelled training data set directly for the subspace''s projection base vectors. We evaluate the usefulness of the ISS method using synthetic data as well as real world problems. Experimental results demonstrate that the ISS algorithm is effective and can be used as a general method for representing high-dimensional data in a low-dimensional subspace for classification. [Copyright &y& Elsevier]

Abstract: In this paper, we present a learning-based image processing technique. We have developed a novel method to map high dynamic range scenes to low dynamic range images for display in standard (low dynamic range) reproduction media. We formulate the problem as a quantization process and employ an adaptive conscience learning strategy to ensure that the mapped low dynamic range displays not only faithfully reproduce the visual features of the original scenes, but also make full use of the available display levels. This is achieved by the use of a competitive learning neural network that employs a frequency sensitive competitive learning mechanism to adaptively design the quantizer. By optimizing an distortion function, we ensure that the mapped low dynamic images preserve the visual characteristics of the original scenes. By incorporating a frequency sensitive competitive mechanism, we facilitate the full utilization of the limited displayable levels. We have developed a deterministic and practicable learning procedure which uses a single variable to control the display result. We give a detailed description of the implementation procedure of the new learning-based high dynamic range compression method and present experimental results to demonstrate the effectiveness of the method in displaying a variety of high dynamic range scenes. [Copyright &y& Elsevier]

Abstract: In this work, we are interested in technologies that will allow users to actively browse and navigate large image databases and to retrieve images through interactive fast browsing and navigation. The development of a browsing/navigation-based image retrieval system has at least two challenges. The first is that the system''s graphical user interface (GUI) should intuitively reflect the distribution of the images in the database in order to provide the users with a mental picture of the database content and a sense of orientation during the course of browsing/navigation. The second is that it has to be fast and responsive, and be able to respond to users actions at an interactive speed in order to engage the users. We have developed a method that attempts to address these challenges of a browsing/navigation based image retrieval systems. The unique feature of the method is that we take an integrated approach to the design of the browsing/navigation GUI and the indexing and organization of the images in the database. The GUI is tightly coupled with the algorithms that run in the background. The visual cues of the GUI are logically linked with various parts of the repository (image clusters of various particular visual themes) thus providing intuitive correspondences between the GUI and the database contents. In the backend, the images are organized into a binary tree data structure using a sequential maximal information coding algorithm and each image is indexed by an -bit binary index thus making response to users’ action very fast. We present experimental results to demonstrate the usefulness of our method both as a pre-filtering tool and for developing browsing/navigation systems for fast image retrieval from large image databases. [Copyright &y& Elsevier]

Organizing images into semantic categories can be very useful for searching and browsing through large image repositories. In this work, we use machine learning to associate low level colour representations of digital colour photos with their high level semantic categories. We investigate the redundancy and performance of a number of histogram-based colour image content representations in the context of automatic colour photo categorization using support vector machines. We use principal component analysis to reduce the dimensionality of (high dimensional) histogram based colour descriptors and use support vector machines to learn to classify the images into various high level categories in the histograms subspaces. We present experimental results to demonstrate the usefulness of such an approach to organizing colour photos into semantic categories. Our results show that the colour content descriptors constructed in different ways perform quite differently and the performances are data dependent hence it is difficult to pick a “winning” descriptor. Our results demonstrate conclusively that all descriptors studied in this paper are highly redundant and that regardless of their performances, the dimensionalities of these histogram based colour content descriptors can be significantly reduced without affecting their classification performances. [Copyright &y& Elsevier]

Presents information on a study which focused on the progressively predictive pyramid (PPP), a low entropy pyramidal image data structure, suited for lossless coding and progressive transmission. Progressive construction of reduced entropy Laplacian pyramid; Design of PPP predictor; Experimental results.

Several compressor inlet guide vanes (IGV) with cracks of 9F gas turbine generator unit have been found fractured during the inspection process. The purpose of this paper is to find out the failure cause of those fractured IGV in terms of visual inspection, metallographic examination, chemical component analysis, microscopic fracture analysis, hardness test and energy spectrum analysis and so on. Macro-observations showed that all the breaking points were located at the chamfering position of IGV root, where exits the stress concentration confirmed by the stress analysis. According to the results of energy spectrum analysis, the corrosive element such as chlorine and sulfur were enriched on the IGV surface, producing local pitting corrosion and formation of corrosion fatigue source. In addition, the coarse martensite structure and organizational segregation of individual IGV by means of metallographic examination resulted in the performance degradation including strength and corrosion resistance, which accelerated the formation and propagation of the stress corrosion fatigue crack. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) have been enjoying considerable success in recent years. However, mode collapse remains a major unsolved problem in training GANs and is one of the main obstacles hindering progress. In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. We first perform theoretical analysis to show that the spectral distributions of the weight matrix in the discriminator affect how the equality of Lipschitz constraint can be fulfilled, thus will have an impact on the performance of the discriminator. Prompted by these analysis, we set out to monitor the spectral distributions in the discriminators of spectral normalized GANs (SN-GANs), and discover a phenomenon which we refer to as spectral collapse, where a large number of singular values of the weight matrices drop dramatically when mode collapse occurs. We show that there are strong evidences linking mode collapse to spectral collapse; and based on this link, we set out to tackle spectral collapse as a surrogate of mode collapse. We have developed a spectral regularization method where we introduce two schemes, one static and one dynamic, to compensate the spectral distributions of the weight matrices to prevent them from collapsing, which in turn successfully prevents mode collapse in GANs. Through gradient analysis, we provide theoretical explanations for why SR-GANs are more stable and can provide better performances than SN-GANs. We also present extensive experimental results and analysis to show that SR-GANs not only always outperform SN-GANs but also always succeed in combating mode collapse where SN-GANs fail. The code is available at https://github.com/max-liu-112/SRGANs-Spectral-Regularization-GANs- [ABSTRACT FROM AUTHOR]

Given a new 6 D o F camera pose in an indoor environment, we investigate the challenging problem of predicting the view from that pose based on a set of reference RGBD views. Existing explicit or implicit 3D geometry construction methods are computationally expensive, while those based on learning have primarily focused on isolated views of object categories with regular geometric structures. In contrast to the traditional render-inpaint approach for new view synthesis in real indoor environments, we propose a conditional generative adversarial neural network (P2I-NET) that directly predicts the new view from the given pose. P2I-NET learns the conditional distribution of environment images to establish the correspondence between the camera pose and its view, achieved through innovative designs in its architecture and training loss function. Two auxiliary discriminator constraints are incorporated to ensure consistency between the pose of the generated image and that of the corresponding real-world image in both the latent feature space and the real-world pose space. Moreover, a deep convolutional neural network (CNN) is introduced to further reinforce this consistency in the pixel space. Extensive new view synthesis experiments and pose estimation experiments have been conducted on real indoor datasets. The results demonstrate that P2I-NET outperforms a number of strong baseline models based on NeRF. In particular, it is shown that P2I-NET is 40 to 100 times faster than these competitor techniques while producing images of similar quality. The results also reveal that the discriminator in P2I-NET achieves comparable positioning performance to other positioning methods, despite having a smaller model size. Furthermore, we contribute a new publicly available indoor environment dataset containing 22 high resolution RGBD videos where each frame also has accurate camera pose parameters. [ABSTRACT FROM AUTHOR]

Camera localization is a fundamental requirement in robotics and computer vision. This paper introduces a pose-to-image translation framework to tackle the camera localization problem. We present PoseGANs, a conditional generative adversarial networks (cGANs) based framework for the implementation of pose-to-image translation. PoseGANs feature a number of innovations including a distance metric based conditional discriminator to conduct camera localization and a pose estimation technique for generated camera images as a stronger constraint to improve camera localization performance. Compared with learning-based regression methods such as PoseNet, PoseGANs can achieve better performance with model sizes that are 70% smaller. In addition, PoseGANs introduce the view synthesis technique to establish the correspondence between the 2D images and the scene, i.e. , given a pose, PoseGANs are able to synthesize its corresponding camera images. Furthermore, we demonstrate that PoseGANs differ in principle from structure-based localization and learning-based regressions for camera localization, and show that PoseGANs exploit the geometric structures to accomplish the camera localization task, and is therefore more stable than and superior to learning-based regressions which rely on local texture features instead. In addition to camera localization and view synthesis, we also demonstrate that PoseGANs can be successfully used for other interesting applications such as moving object elimination and frame interpolation in video sequences. [ABSTRACT FROM AUTHOR]

• Through analyzing the procedure of guided filtering, we show that the traditional filtering practice of placing the center of the filtering window on the pixel being processed is a fundamental cause of GF to introduce artifacts such as edge blurring and halos. And through analyzing typical edge models, we show that aligning the sides of the filtering window with the pixel under processing can improve the edge preservation capability of GF. • Based on these new insights, we propose that guided filtering should align the sides or the corners rather than the center of the operation windows with the current pixel and develop the Side Window Guided filtering (SWGF) technique as an effective and practical edge-preserving filtering method. • We analyze the outputs of GF and SWGF on typical edges and corners to formally prove that SWGF can better preserve edges and corners than GF. • We show that our new side window strategy is a generic method and can be applied to enhance the performances of other forms of guided filtering schemes such as weighted guided filtering (WGF) and gradient domain guided filtering (GGF). • We show that SWGF provides state of the art performances in a variety of image processing applications including edge-preserving smoothing, detail enhancement, luminance adjustment and high dynamic range (HDR) image display. Importantly, we show that under the same parameter settings, SWGF can always improve the edge preservation performances of traditional GF and its variations. This paper presents an improved version of the popular guided filtering (GF) technique. Although very effective, GF can suffer from artifacts such as edge blurring and halos. Through analyzing GF, we discover that traditional filtering practices of placing the center of the filtering window over the pixel being processed is one of the causes of these artifacts. This leads GF to perform linear combination and averaging operations on pixels on both sides of the edges thus causing edge blurring. Based on this discovery, we introduce the Side Window Guided Filtering (SWGF) technique. Treating each pixel as a potential edge, SWGF aligns the sides or corners of the filtering window with the pixels under processing. There are left, right, upper and bottom windows placed on one side of the pixel and northwest, northeast, southwest and southeast windows placed on one corner of the pixel. The one that closest approximates the current pixel is chosen as the output. We will present mathematical analysis to demonstrate the correctness of this method. SWGF is applied to a variety of image processing applications. Results demonstrate that under exactly the same parameter settings, SWGF consistently improves the edge preservation performances of GF and its other variations. [ABSTRACT FROM AUTHOR]

Camera-based indoor localization is a fundamental aspect of indoor navigation, virtual reality, and location-based services. Deep learning methods have exhibited remarkable performance with low storage requirements and high efficiency. However, existing methods mainly derive features implicitly for pose regression without considering explicit structure information from images. This paper proposes that incorporating such information can improve the localization performance of learning-based approaches. We extract structure information from RGB images in the form of depth maps and edge maps and design two modules for depth-weighted and edge-weighted feature fusion. These modules are integrated into the pose regression network to enhance pose prediction. Furthermore, we employ a self-attention module for high-level feature extraction to augment the network capacity. Extensive experiments are conducted on the publicly available 7Scenes and 12Scenes datasets, and the results demonstrate that the proposed method achieves high localization performance, with an average positional error of 0. 19 m and 0. 16 m , respectively. The code for this work is available at https://github.com/lqing900205/structureLoc. [ABSTRACT FROM AUTHOR]

• Bumetanide rescues aquaporin-4 depolarization in a rat retinal I/R injury model. • BU suppresses AQP4 protein/mRNA and β-DG mRNA expression. • BU reduces retinal apoptotic cells and downregulates p-ERK and MMP9 expression. • BU suppresses β-DG cleavage, in part, via the ERK/MMP9 pathway. • BU protects against retinal cytotoxic edema in a rat retinal I/R injury model. Aquaporin-4 (AQP4) regulates retinal water homeostasis and participates in retinal oedema pathophysiology. β-dystroglycan (β-DG) is responsible for AQP4 polarization and can be cleaved by matrix metalloproteinase-9 (MMP9). Retinal oedema induced by ischemia–reperfusion (I/R) injury is an early complication. Bumetanide (BU) has potential efficacy against cytotoxic oedema. Our study investigated the effects of β-DG cleavage on AQP4 and the roles of BU in a rat retinal I/R injury model. The model was induced by applying 110 mm Hg intraocular pressure to the anterior eye chamber. BU and U0126 (a selective ERK inhibitor) were intraperitoneally administered 15 and 30 min, respectively, before I/R induction. Rhodamine isothiocyanate extravasation detection, quantitative real-time PCR, transmission electron microscopy, hematoxylin-eosin staining, immunofluorescence staining, western blotting, and TUNEL staining were performed. AQP4 lost its polarization in the retinal perivascular domain as a result of β-DG cleavage. BU rescued AQP4 depolarization, suppressed AQP4 protein expression, attenuated retinal cytotoxic oedema, and downregulated β-DG and AQP4 mRNA expression. BU suppressed glial responses and mitochondria-mediated apoptotic protein expression, including that of Caspase-3 and Cyto C, raised the Bcl-2/Bax ratio, and lowered the number of apoptotic cells in the retina. Both BU and U0126 downregulated p-ERK and MMP9 expression. Thus, BU treatment suppressed β-DG cleavage, recovered AQP4 polarization partially via inhibiting ERK/MMP9 signaling pathway, and possess potential neuroprotective efficacy in the rat retinal ischemia–reperfusion injury model. [ABSTRACT FROM AUTHOR]

In supervised deep learning, learning good representations for remote-sensing images (RSI) relies on manual annotations. However, in the area of remote sensing, it is hard to obtain huge amounts of labeled data. Recently, self-supervised learning shows its outstanding capability to learn representations of images, especially the methods of instance discrimination. Comparing methods of instance discrimination, clustering-based methods not only view the transformations of the same image as "positive" samples but also similar images. In this paper, we propose a new clustering-based method for representation learning. We first introduce a quantity to measure representations' discriminativeness and from which we show that even distribution requires the most discriminative representations. This provides a theoretical insight into why evenly distributing the images works well. We notice that only the even distributions that preserve representations' neighborhood relations are desirable. Therefore, we develop an algorithm that translates the outputs of a neural network to achieve the goal of evenly distributing the samples while preserving outputs' neighborhood relations. Extensive experiments have demonstrated that our method can learn representations that are as good as or better than the state of the art approaches, and that our method performs computationally efficiently and robustly on various RSI datasets. [ABSTRACT FROM AUTHOR]

Facial attributes in StyleGAN generated images are entangled in the latent space which makes it very difficult to independently control a specific attribute without affecting the others. Supervised attribute editing requires annotated training data which is difficult to obtain and limits the editable attributes to those with labels. Therefore, unsupervised attribute editing in an disentangled latent space is key to performing neat and versatile semantic face editing. In this paper, we present a new technique termed Structure-Texture Independent Architecture with Weight Decomposition and Orthogonal Regularization (STIA-WO) to disentangle the latent space for unsupervised semantic face editing. By applying STIA-WO to GAN, we have developed a StyleGAN termed STGAN-WO which performs weight decomposition through utilizing the style vector to construct a fully controllable weight matrix to regulate image synthesis, and employs orthogonal regularization to ensure each entry of the style vector only controls one independent feature matrix. To further disentangle the facial attributes, STGAN-WO introduces a structure-texture independent architecture which utilizes two independently and identically distributed (i.i.d.) latent vectors to control the synthesis of the texture and structure components in a disentangled way. Unsupervised semantic editing is achieved by moving the latent code in the coarse layers along its orthogonal directions to change texture related attributes or changing the latent code in the fine layers to manipulate structure related ones. We present experimental results which show that our new STGAN-WO can achieve better attribute editing than state of the art methods. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a face-hallucination method, namely face hallucination based on sparse local-pixel structure. In our framework, a high resolution (HR) face is estimated from a single frame low resolution (LR) face with the help of the facial dataset. Unlike many existing face-hallucination methods such as the from local-pixel structure to global image super-resolution method (LPS-GIS) and the super-resolution through neighbor embedding, where the prior models are learned by employing the least-square methods, our framework aims to shape the prior model using sparse representation. Then this learned prior model is employed to guide the reconstruction process. Experiments show that our framework is very flexible, and achieves a competitive or even superior performance in terms of both reconstruction error and visual quality. Our method still exhibits an impressive ability to generate plausible HR facial images based on their sparse local structures. [Copyright &y& Elsevier]

Although the deep learning-based indoor image localization has made significant improvement in terms of accuracy, efficiency, and storage requirement of large indoor scenes, the need for collecting huge labeled training data severely limits its practical application. Recently, the synthetic images rendered from widely available 3D models have shown promising potential to relieve the data collection problem. However, due to the dramatic differences between the synthetic and real images, the localization accuracy of approaches trained on synthetic images is not comparable to the methods trained on real images. In this paper, we propose a domain adaptation-based approach to address this issue. Specifically, the proposed approach mainly contains a model consisting of a multi-level constrained pose regression network and a feature-level discriminator network. The discriminator network forces the pose regression network to generate domain-invariant features from real and synthetic images by adversarial learning and thus reduces the performance gaps. In addition, the multi-level constraints further enhance the localization accuracy of pose regression. We perform extensive experiments on open-source rendering images in different settings. The results show that the proposed method significantly improves the performance. The code for the proposed work is available at https://github.com/lqing900205/BIM_domainadaptation. [ABSTRACT FROM AUTHOR]

We have developed a new face hallucination framework termed from local pixel structure to global image super-resolution (LPS-GIS). Based on the assumption that two similar face images should have similar local pixel structures, the new framework first uses the input low-resolution (LR) face image to search a face database for similar example high-resolution (HR) faces in order to learn the local pixel structures for the target HR face. It then uses the input LR face and the learned pixel structures as priors to estimate the target HR face. We present a three-step implementation procedure for the framework. Step 1 searches the database for K example faces that are the most similar to the input, and then warps the K example images to the input using optical flow. Step 2 uses the warped HR version of the K example faces to learn the local pixel structures for the target HR face. An effective method for learning local pixel structures from an individual face, and an adaptive procedure for fusing the local pixel structures of different example faces to reduce the influence of warping errors, have been developed. Step 3 estimates the target HR face by solving a constrained optimization problem by means of an iterative procedure. Experimental results show that our new method can provide good performances for face hallucination, both in terms of reconstruction error and visual quality; and that it is competitive with existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes to employ the visual saliency for moving object detection via direct analysis from videos. Object saliency is represented by an information saliency map (ISM), which is calculated from spatio-temporal volumes. Both spatial and temporal saliencies are calculated and a dynamic fusion method developed for combination. We use dimensionality reduction and kernel density estimation to develop an efficient information theoretic based procedure for constructing the ISM. The ISM is then used for detecting foreground objects. Three publicly available visual surveillance databases, namely CAVIAR, PETS and OTCBVS-BENCH are selected for evaluation. Experimental results show that the proposed method is robust for both fast and slow moving object detection under illumination changes. The average detection rates are and while the false detection rates are and in CAVIAR (INRIA entrance hall and shopping center) dataset and OTCBVS-BENCH database, respectively. The average processing speed is 6.6fps with frame resolution in a typical Pentium IV computer. [Copyright &y& Elsevier]

Abstract: Example-based super-resolution is a promising approach to solving the image super-resolution problem. However, the learning process can be slow and prediction can be inaccurate. In this paper, we present a novel learning-based algorithm for image super-resolution to improve the computational speed and prediction accuracy. Our new method classifies image patches into several classes, for each class, a class-specific predictor is designed. A class-specific predictor takes a low-resolution image patch as input and predicts a corresponding high-resolution patch as output. The performances of the class-specific predictors are evaluated using different datasets formed by face images and natural-scene images. We present experimental results which demonstrate that the new method provides improved performances over existing methods. [Copyright &y& Elsevier]

The Huff model is designed to estimate the probability of shopping centre patronage based on a shopping centre's attractiveness and the cost of a customer's travel. In this paper, we attempt to discover some general shopping trends by calibrating the Huff model in Shenzhen, China, and New York, USA, using taxi trajectory GPS data and sharing bikes GPS data. Geographical and Temporal Weighted Regression (GTWR) is used to fit the model, and calibration results are compared with Ordinary Least Squares (OLS) regression, Geographical Weighted Regression (GWR), and Temporal Weighted Regression (TWR). Results show that GTWR gives the highest performance due to significant geographical and temporal variation in the Huff model parameters of attractiveness and travel cost. To explain the geographical variation, we use residential sales' and rental prices in Shenzhen and New York as a proxy for customers' wealth in each region. Pearson product-moment correlation results show a medium relationship between localised sales' and rental prices and the Huff model parameter of attractiveness: that is, customer wealth explains geographic sensitivity to shopping area attractiveness. To explain temporal variation, we use census data in both Shenzhen and New York to provide job profile distributions for each region as a proxy to estimate customers' spare leisure time. Regression results demonstrate that there is a significant linear relationship between the length of spare time and the parameter of shopping area attractiveness. In particular, we demonstrate that wealthy customers with less spare time are more sensitive to a shopping centre's attractiveness. We also discover customers' sensitivities to travel distance are related to their travel mode. In particular, people riding bikes to shopping areas care much more about trip distance compared with people who take taxi. Finally, results show a divergence in behaviours between customers in New York and Shenzhen at weekends. While customers in New York prefer to shop more locally at weekends, customers in Shenzhen care less about trip distance. We provide the GTWR calibration of the Huff model as our theoretical contribution. GTWR extends the Huff model to two dimensions (time and space), so as to analyse the differences of residents' travel behaviours in different time and locations. We also provide the discoveries of factors affecting urban travel behaviours (wealth and employment) as practical contributions that may help optimise urban transportation design. In particular, the sensitivity of residents to the attraction of shopping areas has a significant positive linear relationship with the housing price and a significant negative linear relationship with the residents' length of spare time. [ABSTRACT FROM AUTHOR]

6DOF camera relocalization is an important component of autonomous driving and navigation. Deep learning has recently emerged as a promising technique to tackle this problem. In this paper, we present a novel relative geometry-aware Siamese neural network to enhance the performance of deep learning-based methods through explicitly exploiting the relative geometry constraints between images. We perform multi-task learning and predict the absolute and relative poses simultaneously. We regularize the shared-weight twin networks in both the pose and feature domains to ensure that the estimated poses are globally as well as locally correct. We employ metric learning and design a novel adaptive metric distance loss to learn a feature that is capable of distinguishing poses of visually similar images from different locations.We evaluate the proposed method on public indoor and outdoor benchmarks and the experimental results demonstrate that our method can significantly improve localization performance. Furthermore, extensive ablation evaluations are conducted to demonstrate the effectiveness of different terms of the loss function. [ABSTRACT FROM AUTHOR]

Fog removal from an image is an active research topic in computer vision. However, current literature is weak in the following two areas which in many ways are hindering progress for developing defogging algorithms. First, there is no true real-world and naturally occurring foggy image datasets suitable for developing defogging models. Second, there is no suitable mathematically simple and easy to use image quality assessment (IQA) methods for evaluating the visual quality of defogged images. We address these two aspects in this paper. We first introduce a new foggy image dataset called multiple real-world foggy image dataset (MRFID). MRFID contains foggy and clear images of 200 outdoor scenes. For each scene, one clear image and 4 foggy images of different densities defined as slightly foggy, moderately foggy, highly foggy, and extremely foggy, are manually selected from images taken from these scenes over the course of one calendar year. We then process the foggy images of MRFID using 16 defogging methods to obtain 12,800 defogged images (DFIs) and perform a comprehensive subjective evaluation of the visual quality of the DFIs. Through collecting the mean opinion score (MOS) of 120 subjects and evaluating a variety of fog-relevant image features, we have developed a new Fog-relevant Feature based SIMilarity index (FRFSIM) for assessing the visual quality of DFIs. We present extensive experimental results to show that our new visual quality assessment measure, the FRFSIM, is more consistent with the MOS than other IQA methods and is therefore more suitable for evaluating defogged images than other state-of-the-art IQA methods. Our dataset and relevant code are available at http://www.vistalab.ac.cn/MRFID-for-defogging/. [ABSTRACT FROM AUTHOR]

Accurately locating the fovea is a prerequisite for developing computer aided diagnosis (CAD) of retinal diseases. In colour fundus images of the retina, the fovea is a fuzzy region lacking prominent visual features and this makes it difficult to directly locate the fovea. While traditional methods rely on explicitly extracting image features from the surrounding structures such as the optic disc and various vessels to infer the position of the fovea, deep learning based regression technique can implicitly model the relation between the fovea and other nearby anatomical structures to determine the location of the fovea in an end-to-end fashion. Although promising, using deep learning for fovea localisation also has many unsolved challenges. In this paper, we present a new end-to-end fovea localisation method based on a hierarchical coarse-to-fine deep regression neural network. The innovative features of the new method include a multi-scale feature fusion technique and a self-attention technique to exploit location, semantic, and contextual information in an integrated framework, a multi-field-of-view (multi-FOV) feature fusion technique for context-aware feature learning and a Gaussian-shift-cropping method for augmenting effective training data. We present extensive experimental results on two public databases and show that our new method achieved state-of-the-art performances. We also present a comprehensive ablation study and analysis to demonstrate the technical soundness and effectiveness of the overall framework and its various constituent components. [ABSTRACT FROM AUTHOR]

• Multiple-origin fatigue fracture occurred on the 17-4PH feed water pump shaft. • Crack origin were located on stress concentration area where matched lock nut. • The failure mechanism was revealed systematically in micro-level. • The materials at microscopic level determines the intrinsic physical mechanism of macroscopic behavior. The failure of feed water pump shaft failure uesed in HRSG (heat recovery steam generator) of 9F class gas turbine generator unit has been systematically investigated in terms of visual inspection, fractographic, chemical component, micro structure and mechanical properties analysis using optical microscope (OM), scanning electron microscope (SEM) equipped with energy-dispersive X-ray (EDX) analysis, transmission electron microscope (TEM), elemental analyzer and universal tensile testing machine, respectively. Results suggest that the failure of feed water pump shaft was typical multiple-origin fatigue fracture resulting from torsion stress, owing to the lower mechanical properties caused by the less alloying element and alloy strengthening phase (ε-Cu, η-Ni). A detailed analysis of the micro structure revealed the existence of severe microsegregation and coarse M 23 C 6 carbide at the grain boundaries, which accelerated the nucleation of micro holes and led to the formation of microcracks at the junction of micro holes. This failure analysis can help identification of root cause and its elimination with corrective measures undertaken. [ABSTRACT FROM AUTHOR]

• We propose a generic side window filtering (GSWF) framework. It generically abstracts the essence of edge-preserving filtering by feature calculation and fusion. The features are calculated in side windows which can prevent the edges from blurring. By setting proper weighting coefficients, the features can be fused to get the filtering result. In theory, given the right filtering kernel and weighting coefficients, GSWF can be used to approximate arbitrary edge-preserving filters. • Based on GSWF and by taking advantage of the powerful representative capability of convolutional neural network, we design a side window convolutional neural network (SW-CNN). It implements the feature calculation by a new convolutional strategy called side kernel convolution and the feature fusion by two convolutions. It is a fast and efficient implementation of image filtering by convolutional neural network. By supervised learning, the kernel and the weighting coefficients can be learned to simulate and accelerate arbitrary filtering operator. • SKC is a new convolutional strategy proposed in this paper. Different from the convolutional operation in traditional CNN, in SKC, the pixel to be processed is placed on a side or a corner rather than the center of the convolutional operation window. This new convolutional operation simulates the operation in SWF. In this way, the edges will not be blurred during feature calculation of SW-CNN, which enables it to solve the low-level processing problem by low-level operations. Moreover, SKC can be used as a module to replace the traditional convolution module in other neural networks. • Compared to state-of-art networks, such as DEAF and CAN, SW-CNN can achieve better or comparable results on implementing many local or global based filters with its trainable parameters only 0.65% and 4% of DEAF and CAN, respectively. On graphics processing unit (GPU), our implementation achieves more than 900 times acceleration for several filters and leads to state-of-the-art speed for neural network implementations of filters. Convolutional neural networks (CNNs) designed for object recognition have been successfully applied to low-level tasks such as image filtering. However, these networks are usually very large which occupy large memory space and demand very high computational capacity. This makes them unsuitable for real time low-level applications on smart and portable devices with limited memory and computational capacities. In this paper, we specifically design a novel CNN, side window convolutional neural network (SW-CNN), for the fast and efficient implementation of image filtering. In SW-CNN, a new convolutional strategy, called side kernel convolution (SKC) is proposed which aligns the side or corner of the convolutional window with the pixels under processing to preserve edges during convolution. By combining SKC and the representational power of CNNs, SW-CNN can learn various image-filtering tasks very effectively. Compared to the state-of-the-art networks, the superiority of SW-CNN includes three aspects. First, the number of learnable parameters is reduced by 96%. Second, the memory consumption is reduced to 50%. Third, the running time is decreased to 50%. Results of extensive experiments demonstrate that SW-CNN not only has good performance on implementing various edge-preserving filters, but also has the adaptability and flexibility on other low-level image processing applications. [ABSTRACT FROM AUTHOR]

Background: Specific highly polarized aquaporin‐4 (AQP4) expression is reported to play a crucial role in blood‐brain barrier (BBB) integrity and brain water transport balance. The upregulation of polymerase δ‐interacting protein 2 (Poldip2) was involved in aggravating BBB disruption following ischemic stroke. This study aimed to investigate whether Poldip2‐mediated BBB disruption and cerebral edema formation in mouse bacterial meningitis (BM) model occur via induction of AQP4 polarity loss. Methods and Results: Mouse BM model was induced by injecting mice with group B hemolytic streptococci via posterior cistern. Recombinant human Poldip2 (rh‐Poldip2) was administered intranasally at 1 hour after BM induction. Small interfering ribonucleic acid (siRNA) targeting Poldip2 was administered by intracerebroventricular (i.c.v) injection at 48 hours before BM induction. A specific inhibitor of matrix metalloproteinases (MMPs), UK383367, was administered intravenously at 0.5 hour before BM induction. Western blotting, immunofluorescence staining, quantitative real‐time PCR, neurobehavioral test, brain water content test, Evans blue (EB) permeability assay, transmission electron microscopy (TEM), and gelatin zymography were carried out. The results showed that Poldip2 was upregulated and AQP4 polarity was lost in mouse BM model. Both Poldip2 siRNA and UK383367 improved neurobehavioral outcomes, alleviated brain edema, preserved the integrity of BBB, and relieved the loss of AQP4 polarity in BM model. Rh‐Poldip2 upregulated the expression of MMPs and glial fibrillary acidic protein (GFAP) and downregulated the expression of β‐dystroglycan (β‐DG), zonula occludens‐1 (ZO‐1), occludin, and claudin‐5; whereas Poldip2 siRNA downregulated the expression of MMPs and GFAP, and upregulated β‐DG, ZO‐1, occludin, and claudin‐5. Similarly, UK383367 downregulated the expression of GFAP and upregulated the expression of β‐DG, ZO‐1, occludin, and claudin‐5. Conclusion: Poldip2 inhibition alleviated brain edema and preserved the integrity of BBB partially by relieving the loss of AQP4 polarity via MMPs/β‐DG pathway. [ABSTRACT FROM AUTHOR]

In the field of single image defogging, there are two main methods. One is the image restoration method based on the atmospheric scattering theory which can recover the image texture details well. The other is the image enhancement method based on Retinex theory which can improve the image contrast well. In practice, however, the former can easily lead to low contrast images; the latter is prone to losing texture details. Therefore, how to effectively combine the advantages of both to remove fog is a key issue in the field. In this paper, we have developed a physics based generative adversarial network (PBGAN) to exploit the advantages between those two methods in parallel. To our knowledge, it is the first learning defogging framework that incorporates these two methods and to enable them to work together and complement each other. Our method has two generative adversarial modules, the Contrast Enhancement (CE) module and the Texture Restoration (TR) module. To improve contrast in the CE module, we introduced a novel inversion-adversarial loss and a novel inversion-cycle consistency loss for training the generator. To improve the texture in the TR module, we introduced two convolutional neural networks to learn the atmospheric light coefficient and the transmission map, respectively. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed approach performs better than several state-of-the-art methods quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Predicting depth from a single image is an attractive research topic since it provides one more dimension of information to enable machines to better perceive the world. Recently, deep learning has emerged as an effective approach to monocular depth estimation. As obtaining labeled data is costly, there is a recent trend to move from supervised learning to unsupervised learning to obtain monocular depth. However, most unsupervised learning methods capable of achieving high depth prediction accuracy will require a deep network architecture which will be too heavy and complex to run on embedded devices with limited storage and memory spaces. To address this issue, we propose a new powerful network with a recurrent module to achieve the capability of a deep network while at the same time maintaining an extremely lightweight size for real-time high performance unsupervised monocular depth prediction from video sequences. Besides, a novel efficient upsample block is proposed to fuse the features from the associated encoder layer and recover the spatial size of features with the small number of model parameters. We validate the effectiveness of our approach via extensive experiments on the KITTI dataset. Our new model can run at a speed of about 110 frames per second (fps) on a single GPU, 37 fps on a single CPU, and 2 fps on a Raspberry Pi 3. Moreover, it achieves higher depth accuracy with nearly 33 times fewer model parameters than state-of-the-art models. To the best of our knowledge, this work is the first extremely lightweight neural network trained on monocular video sequences for real-time unsupervised monocular depth estimation, which opens up the possibility of implementing deep learning-based real-time unsupervised monocular depth prediction on low-cost embedded devices. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have been extensively applied in image processing, including visual saliency map prediction of images. A major difficulty in using a DNN for visual saliency prediction is the lack of labeled ground truth of visual saliency. A powerful DNN usually contains a large number of trainable parameters. This condition can easily lead to model over-fitting. In this study, we develop a novel method that overcomes such difficulty by embedding hierarchical knowledge of existing visual saliency models in a DNN. We achieve the objective of exploiting the knowledge contained in the existing visual saliency models by using saliency maps generated by local, global, and semantic models to tune and fix about 92.5% of the parameters in our network in a hierarchical manner. As a result, the number of trainable parameters that need to be tuned by the ground truth is considerably reduced. This reduction enables us to fully utilize the power of a large DNN and overcome the issue of over-fitting at the same time. Furthermore, we introduce a simple but very effective center prior in designing the learning cost function of the DNN by attaching high importance to the errors around the image center. We also present extensive experimental results on four commonly used public databases to demonstrate the superiority of the proposed method over classical and state-of-the-art methods on various evaluation metrics. [ABSTRACT FROM AUTHOR]

Single image defogging is a classical and challenging problem in computer vision. Existing methods towards this problem mainly include handcrafted priors based methods that rely on the use of the atmospheric degradation model and learning-based approaches that require paired fog-fogfree training example images. In practice, however, prior-based methods are prone to failure due to their own limitations and paired training data are extremely difficult to acquire. Moreover, there are few studies on the unpaired trainable defogging network in this field. Thus, inspired by the principle of CycleGAN network, we have developed an end-to-end learning system that uses unpaired fog and fogfree training images, adversarial discriminators and cycle consistency losses to automatically construct a fog removal system. Similar to CycleGAN, our system has two transformation paths; one maps fog images to a fogfree image domain and the other maps fogfree images to a fog image domain. Instead of one stage mapping, our system uses a two stage mapping strategy in each transformation path to enhance the effectiveness of fog removal. Furthermore, we make explicit use of prior knowledge in the networks by embedding the atmospheric degradation principle and a sky prior for mapping fogfree images to the fog images domain. In addition, we also contribute the first real world nature fog-fogfree image dataset for defogging research. Our multiple real fog images dataset (MRFID) contains images of 200 natural outdoor scenes. For each scene, there is one clear image and corresponding four foggy images of different fog densities manually selected from a sequence of images taken by a fixed camera over the course of one year. Qualitative and quantitative comparison against several state-of-the-art methods on both synthetic and real world images demonstrate that our approach is effective and performs favorably for recovering a clear image from a foggy image. [ABSTRACT FROM AUTHOR]

Structure-texture image decomposition is a fundamental but challenging topic in computational graphics and image processing. In this paper, we introduce a structure-aware and a texture-aware measures to facilitate the structure-texture decomposition (STD) of images. Edge strengths and spatial scales that have been widely-used in previous STD researches cannot describe the structures and textures of images well. The proposed two measures differentiate image textures from image structures based on their distinctive characteristics. Specifically, the first one aims to measure the anisotropy of local gradients, and the second one is designed to measure the repeatability degree of signal patterns in a neighboring region. Since these two measures describe different properties of image structures and textures, they are complementary to each other. The STD is achieved by optimizing an objective function based on the two new measures. As using traditional optimization methods to solve the optimization problem will require designing different optimizers for different functional spaces, we employ an architecture of deep neural network to optimize the STD cost function in a unified manner. The experimental results demonstrate that, as compared with some state-of-the-art methods, our method can better separate image structure and texture and result in shaper edges in the structural component. Furthermore, to demonstrate the usefulness of the proposed STD method, we have successfully applied it to several applications including detail enhancement, edge detection, and visual quality assessment of super-resolved images. [ABSTRACT FROM AUTHOR]

Global positioning system (GPS) data generated from taxi trips is a valuable source of information that offers an insight into travel behaviours of urban populations with high spatio-temporal resolution. However, in its raw form, GPS taxi data does not offer information on the purpose (or intended activity) of travel. In this context, to enhance the utility of taxi GPS data sets, we propose a two-layer framework to identify the related activities of each taxi trip automatically and estimate the return trips and successive activities after the trip, by using geographic point-of-interest (POI) data and a combination of spatio-temporal clustering, Bayesian inference and Monte Carlo simulation. Two million taxi trips in New York, the United States of America, and ten million taxi trips in Shenzhen, China, are used as inputs for the two-layer framework. To validate each layer of the framework, we collect 6,003 trip diaries in New York and 712 questionnaire surveys in Shenzhen. The results show that the first layer of the framework performs better than comparable methods published in the literature, while the second layer has high accuracy when inferring return trips. [ABSTRACT FROM AUTHOR]

Deep learning approaches are widely applied to histopathological image analysis due to the impressive levels of performance achieved. However, when dealing with high-resolution histopathological images, utilizing the original image as input to the deep learning model is computationally expensive, while resizing the original image to achieve low resolution incurs information loss. Some hard-attention based approaches have emerged to select possible lesion regions from images to avoid processing the original image. However, these hard-attention based approaches usually take a long time to converge with weak guidance, and valueless patches may be trained by the classifier. To overcome this problem, we propose a deep selective attention approach that aims to select valuable regions in the original images for classification. In our approach, a decision network is developed to decide where to crop and whether the cropped patch is necessary for classification. These selected patches are then trained by the classification network, which then provides feedback to the decision network to update its selection policy. With such a co-evolution training strategy, we show that our approach can achieve a fast convergence rate and high classification accuracy. Our approach is evaluated on a public breast cancer histopathological image database, where it demonstrates superior performance compared to state-of-the-art deep learning approaches, achieving approximately 98% classification accuracy while only taking 50% of the training time of the previous hard-attention approach. [ABSTRACT FROM AUTHOR]

Motivation For the diagnosis of cancer, manually counting nuclei on massive histopathological images is tedious and the counting results might vary due to the subjective nature of the operation. Results This paper presents a new segmentation and counting method for nuclei, which can automatically provide nucleus counting results. This method segments nuclei with detected nuclei seed markers through a modified simple one-pass superpixel segmentation method. Rather than using a single pixel as a seed, we created a superseed for each nucleus to involve more information for improved segmentation results. Nucleus pixels are extracted by a newly proposed fusing method to reduce stain variations and preserve nucleus contour information. By evaluating segmentation results, the proposed method was compared to five existing methods on a dataset with 52 immunohistochemically (IHC) stained images. Our proposed method produced the highest mean F1-score of 0.668. By evaluating the counting results, another dataset with more than 30 000 IHC stained nuclei in 88 images were prepared. The correlation between automatically generated nucleus counting results and manual nucleus counting results was up to R 2 = 0.901 (P  < 0.001). By evaluating segmentation results of proposed method-based tool, we tested on a 2018 Data Science Bowl (DSB) competition dataset, three users obtained DSB score of 0.331 ± 0.006. Availability and implementation The proposed method has been implemented as a plugin tool in ImageJ and the source code can be freely downloaded. Supplementary information Supplementary data are available at Bioinformatics online. [ABSTRACT FROM AUTHOR]

Urban region function recognition is key to rational urban planning and management. Due to the complex socioeconomic nature of functional land use, recognizing urban region function in high-density cities using remote sensing images alone is difficult. The inclusion of social sensing has the potential to improve the function classification performance. However, effectively integrating the multi-source and multi-modal remote and social sensing data remains technically challenging. In this paper, we have proposed a novel end-to-end deep learning-based remote and social sensing data fusion model to address this issue. Two neural network based methods, one based on a 1-dimensional convolutional neural network (CNN) and the other based on a long short-term memory (LSTM) network, have been developed to automatically extract discriminative time-dependent social sensing signature features, which are fused with remote sensing image features extracted via a residual neural network. One of the major difficulties in exploiting social and remote sensing data is that the two data sources are asynchronous. We have developed a deep learning-based strategy to address this missing modality problem by enforcing cross-modal feature consistency (CMFC) and cross-modal triplet (CMT) constraints. We train the model in an end-to-end manner by simultaneously optimizing three costs, including the classification cost, the CMFC cost and the CMT cost. Extensive experiments have been conducted on publicly available datasets to demonstrate the effectiveness of the proposed method in fusing remote and social sensing data for urban region function recognition. The results show that the seemingly unrelated physically sensed image data and social activities sensed signatures can indeed complement each other to help enhance the accuracy of urban region function recognition. [ABSTRACT FROM AUTHOR]