Search

Background: Microvascular invasion (MVI) is an independent prognostic factor that is associated with early recurrence and poor survival after resection of hepatocellular carcinoma (HCC). However, the traditional pathology approach is relatively subjective, time‐consuming, and heterogeneous in the diagnosis of MVI. The aim of this study was to develop a deep‐learning model that could significantly improve the efficiency and accuracy of MVI diagnosis. Materials and Methods: We collected H&E‐stained slides from 753 patients with HCC at the First Affiliated Hospital of Zhejiang University. An external validation set with 358 patients was selected from The Cancer Genome Atlas database. The deep‐learning model was trained by simulating the method used by pathologists to diagnose MVI. Model performance was evaluated with accuracy, precision, recall, F1 score, and the area under the receiver operating characteristic curve. Results: We successfully developed a MVI artificial intelligence diagnostic model (MVI‐AIDM) which achieved an accuracy of 94.25% in the independent external validation set. The MVI positive detection rate of MVI‐AIDM was significantly higher than the results of pathologists. Visualization results demonstrated the recognition of micro MVIs that were difficult to differentiate by the traditional pathology. Additionally, the model provided automatic quantification of the number of cancer cells and spatial information regarding MVI. Conclusions: We developed a deep learning diagnostic model, which performed well and improved the efficiency and accuracy of MVI diagnosis. The model provided spatial information of MVI that was essential to accurately predict HCC recurrence after surgery. [ABSTRACT FROM AUTHOR]

Retrosynthesis is the cornerstone of organic chemistry, providing chemists in material and drug manufacturing access to poorly available and brand‐new molecules. Conventional rule‐based or expert‐based computer‐aided synthesis has obvious limitations, such as high labor costs and limited search space. In recent years, dramatic breakthroughs driven by deep learning have revolutionized retrosynthesis. Here we aim to present a comprehensive review of recent advances in AI‐based retrosynthesis. For single‐step and multi‐step retrosynthesis both, we first introduce their goal and provide a thorough taxonomy of existing methods. Afterwards, we analyze these methods in terms of their mechanism and performance, and introduce popular evaluation metrics for them, in which we also provide a detailed comparison among representative methods on several public datasets. In the next part, we introduce popular databases and established platforms for retrosynthesis. Finally, this review concludes with a discussion about promising research directions in this field. This article is categorized under:Data Science > Artificial Intelligence/Machine LearningData Science > Computer Algorithms and ProgrammingData Science > Chemoinformatics [ABSTRACT FROM AUTHOR]

"AI & Drug Discovery" mode has significantly promoted drug development and achieved excellent performance, especially with the rapid development of deep learning, making remarkable contributions to protecting human physiological health. However, due to the "black‐box" characteristic of the deep learning model, the decision route and predicted results in different research stages assisted by deep models are usually unexplainable, limiting their application in practice and more in‐depth research of drug discovery. Focusing on the drug molecules, an explainable fragment‐based molecular property attribution technique is proposed for analyzing the influence of particular molecule fragments on properties and the relationship between the molecular properties herein. Quantitative experiments on 42 benchmark property tasks demonstrate that 325 attribution fragments, which account for 90% of the overall attribution results obtained by the proposed method, have positive relevance to the corresponding property tasks. More impressively, most of the attribution results randomly selected are consistent with the existing mechanism explanations. The discovery mentioned above provides a reference standard for assisting researchers in developing more specific and practical drug molecule studies, such as synthesizing molecule with the targeted property using a fragment obtained from the attribution method. An interactive preprint version of the article can be found at: https://www.authorea.com/doi/full/10.22541/au.165279262.29589148. [ABSTRACT FROM AUTHOR]

Video highlight detection (VHD) has emerged as a pressing task due to the unprecedentedly increasing amount of video data, such as those from e-commerce live-broadcasting platforms. Many approaches focus on exploiting text data, in the form of video description or time-sync comments, to facilitate the VHD task. Despite the promising results, they have largely overlooked the noises inherent in the text data and have mostly relied on isolating the feature of text and video. In this paper, we introduce a novel model to handle VHD, termed Coevolution Network (CoEvo-Net), that allows us to account for joint learning of the language and video features explicitly via a coevolution paradigm, in which features from the two data modalities progressively refine each other. This is achieved by a dedicated CoEvo-Cell that takes language and video together as inputs, extracts cross-modality, and filters the undesired parts of the input, such as words in a sentence. Furthermore, we release a large-scale dataset of e-commerce for VHD, in which each video is coupled with a sentence for description, to benchmark the sentence-based VHD approaches. Extensive experiments on the released dataset demonstrate that CoEvo-Net achieves state-of-the-art performance. Our dataset and code will be made publicly available. [ABSTRACT FROM AUTHOR]

Background: An accurate pathological diagnosis of hepatocellular carcinoma (HCC), one of the malignant tumors with the highest mortality rate, is time-consuming and heavily reliant on the experience of a pathologist. In this report, we proposed a deep learning model that required minimal noise reduction or manual annotation by an experienced pathologist for HCC diagnosis and classification. Methods: We collected a whole-slide image of hematoxylin and eosin-stained pathological slides from 592 HCC patients at the First Affiliated Hospital, College of Medicine, Zhejiang University between 2015 and 2020. We propose a noise-specific deep learning model. The model was trained initially with 137 cases cropped into multiple-scaled datasets. Patch screening and dynamic label smoothing strategies are adopted to handle the histopathological liver image with noise annotation from the perspective of input and output. The model was then tested in an independent cohort of 455 cases with comparable tumor types and differentiations. Results: Exhaustive experiments demonstrated that our two-step method achieved 87.81% pixel-level accuracy and 98.77% slide-level accuracy in the test dataset. Furthermore, the generalization performance of our model was also verified using The Cancer Genome Atlas dataset, which contains 157 HCC pathological slides, and achieved an accuracy of 87.90%. Conclusions: The noise-specific histopathological classification model of HCC based on deep learning is effective for the dataset with noisy annotation, and it significantly improved the pixel-level accuracy of the regular convolutional neural network (CNN) model. Moreover, the model also has an advantage in detecting well-differentiated HCC and microvascular invasion. [ABSTRACT FROM AUTHOR]

The seminal work of Gatys et al. demonstrated the power of Convolutional Neural Networks (CNNs) in creating artistic imagery by separating and recombining image content and style. This process of using CNNs to render a content image in different styles is referred to as Neural Style Transfer (NST). Since then, NST has become a trending topic both in academic literature and industrial applications. It is receiving increasing attention and a variety of approaches are proposed to either improve or extend the original NST algorithm. In this paper, we aim to provide a comprehensive overview of the current progress towards NST. We first propose a taxonomy of current algorithms in the field of NST. Then, we present several evaluation methods and compare different NST algorithms both qualitatively and quantitatively. The review concludes with a discussion of various applications of NST and open problems for future research. A list of papers discussed in this review, corresponding codes, pre-trained models and more comparison results are publicly available at: https://osf.io/f8tu4/. [ABSTRACT FROM AUTHOR]

Extracting color themes from an image is to get a color palette that consists of dominant colors of the image. In this article, we construct a color network to build the intrinsic connections of color information of pixels. By applying improved linear iterative clustering (SLIC) algorithm [1,2] , we obtain initial color themes. In the following stage, with learning from human-extract color themes, we can get the final sorted color themes result. Experimental results demonstrate that our model outperforms previous approaches in terms of the number of themes, span and accuracy. [ABSTRACT FROM AUTHOR]

With the wide-spread of smartphones, mobile phone screen has become an important IO device in HCI and its quality is of great matter in interaction. Traditional defect detection process involves heavy labor cost or relies on unstable low-level features and suffers from both scale and model sensitive problems. Screen defect varies in size, shape, intensity and is hard to be described. Efficient and accurate detection system remains an urgent need in mobile phone screen manufacturing. In this paper, we propose an end-to-end screen defect detection framework. A defect detection network with merging and splitting strategies (MSDDN) to deal with multiple size and shape variations of defect image patches is firstly designed. After training, feature maps of the last layer before the output of MSDDN can be regarded as good representations of a screen image patch. These feature maps are concatenated into a unified feature vector. We then train a recurrent neural network (SCN) to decide which input screen image patch in a sequence is the most likely to contain defects, where the patches are cropped from the same image, and the feature maps are used as input. As SCN emphasizes on the comparison of image patches from one image, it is less sensitive to different screen batches. The patch with the highest probability of containing defects, or called focus area, is further processed with a sliding window to fill in the MSDDN to produce the final results. Finally, to improve the efficiency of the calculation process to fulfill the real industrial demand, we perform both filter selection and weight quantization on the weights in MSDDN under the purpose of building a low-precision version network without great loss in accuracy (MSDDN-l). Experimental results show MSDDN can better handle the defect variations than traditional models and general purpose convolutional neural networks. Meanwhile, SCN can accurately predict the focus area, and MSDDN-l can greatly improve the efficiency. [ABSTRACT FROM AUTHOR]

In this work, we propose a stroke-based hairstyle editing network, dubbed HairstyleNet, allowing users to conveniently change the hairstyles of an image in an interactive fashion. Different from previous works, we simplify the hairstyle editing process where users can manipulate local or entire hairstyles by adjusting the parameterized hair regions. Our HairstyleNet consists of two stages: a stroke parameterization stage and a stroke-to-hair generation stage. In the stroke parameterization stage, we first introduce parametric strokes to approximate the hair wisps, where the stroke shape is controlled by a quadratic Bézier curve and a thickness parameter. Since rendering strokes with thickness to an image is not differentiable, we opt to leverage a neural renderer to construct the mapping from stroke parameters to a stroke image. Thus, the stroke parameters can be directly estimated from hair regions in a differentiable way, enabling us to flexibly edit the hairstyles of input images. In the stroke-to-hair generation stage, we design a hairstyle refinement network that first encodes coarsely composed images of hair strokes, face, and background into latent representations and then generates high-fidelity face images with desirable new hairstyles from the latent codes. Extensive experiments demonstrate that our HairstyleNet achieves state-of-the-art performance and allows flexible hairstyle manipulation.

Deep cooperative multi-agent reinforcement learning has demonstrated its remarkable success over a wide spectrum of complex control tasks. However, recent advances in multi-agent learning mainly focus on value decomposition while leaving entity interactions still intertwined, which easily leads to over-fitting on noisy interactions between entities. In this work, we introduce a novel interactiOn Pattern disenTangling (OPT) method, to disentangle the entity interactions into interaction prototypes, each of which represents an underlying interaction pattern within a subgroup of the entities. OPT facilitates filtering the noisy interactions between irrelevant entities and thus significantly improves generalizability as well as interpretability. Specifically, OPT introduces a sparse disagreement mechanism to encourage sparsity and diversity among discovered interaction prototypes. Then the model selectively restructures these prototypes into a compact interaction pattern by an aggregator with learnable weights. To alleviate the training instability issue caused by partial observability, we propose to maximize the mutual information between the aggregation weights and the history behaviors of each agent. Experiments on single-task, multi-task and zero-shot benchmarks demonstrate that the proposed method yields results superior to the state-of-the-art counterparts. Our code is available at https://github.com/liushunyu/OPT.

Researchers of temporal networks (e.g., social networks and transaction networks) have been interested in mining dynamic patterns of nodes from their diverse interactions. Inspired by recently powerful graph mining methods like skip-gram models and graph neural networks (GNNs), existing approaches focus on generating temporal node embeddings sequentially with nodes' sequential interactions. However, the sequential modeling of previous approaches cannot handles the transition structure between nodes' neighbors with limited memorization capacity. In detail, an effective method for the transition structures is required to both model nodes' personalized patterns adaptively and capture node dynamics accordingly. In this article, we propose a method, namely t ransition p ropagation g raph n eural n etworks (TIP-GNN), to tackle the challenges of encoding nodes' transition structures. The proposed TIP-GNN focuses on the bilevel graph structure in temporal networks: besides the explicit interaction graph, a node's sequential interactions can also be constructed as a transition graph. Based on the bilevel graph, TIP-GNN further encodes transition structures by multistep transition propagation and distills information from neighborhoods by a bilevel graph convolution. Experimental results over various temporal networks reveal the efficiency of our TIP-GNN, with at most 7.2% improvements of accuracy on temporal link prediction. Extensive ablation studies further verify the effectiveness and limitations of the transition propagation module. Our code is available at https://github.com/doujiang-zheng/TIP-GNN.

Pseudo supervision is regarded as the core idea in semi-supervised learning for semantic segmentation, and there is always a tradeoff between utilizing only the high-quality pseudo labels and leveraging all the pseudo labels. Addressing that, we propose a novel learning approach, called Conservative-Progressive Collaborative Learning (CPCL), among which two predictive networks are trained in parallel, and the pseudo supervision is implemented based on both the agreement and disagreement of the two predictions. One network seeks common ground via intersection supervision and is supervised by the high-quality labels to ensure a more reliable supervision, while the other network reserves differences via union supervision and is supervised by all the pseudo labels to keep exploring with curiosity. Thus, the collaboration of conservative evolution and progressive exploration can be achieved. To reduce the influences of the suspicious pseudo labels, the loss is dynamic re-weighted according to the prediction confidence. Extensive experiments demonstrate that CPCL achieves state-of-the-art performance for semi-supervised semantic segmentation.

Knowledge amalgamation (KA) is a novel deep model reusing task aiming to transfer knowledge from several well-trained teachers to a multi-talented and compact student. Currently, most of these approaches are tailored for convolutional neural networks (CNNs). However, there is a tendency that Transformers, with a completely different architecture, are starting to challenge the domination of CNNs in many computer vision tasks. Nevertheless, directly applying the previous KA methods to Transformers leads to severe performance degradation. In this work, we explore a more effective KA scheme for Transformer-based object detection models. Specifically, considering the architecture characteristics of Transformers, we propose to dissolve the KA into two aspects: sequence-level amalgamation (SA) and task-level amalgamation (TA). In particular, a hint is generated within the sequence-level amalgamation by concatenating teacher sequences instead of redundantly aggregating them to a fixed-size one as previous KA approaches. Besides, the student learns heterogeneous detection tasks through soft targets with efficiency in the task-level amalgamation. Extensive experiments on PASCAL VOC and COCO have unfolded that the sequence-level amalgamation significantly boosts the performance of students, while the previous methods impair the students. Moreover, the Transformer-based students excel in learning amalgamated knowledge, as they have mastered heterogeneous detection tasks rapidly and achieved superior or at least comparable performance to those of the teachers in their specializations.