Search

Tiantian Tang,1,2,* Yujiao Wang,1,2,* Taotao Li,1,2,* Ding Liu,1,2 Kai Yang,1,2 Jing Sun,1,2 Yajun Shi,1,2 Dongyan Guo,1,2 Junbo Zou,1,2 Fengyun Bai,3 Ying Sun,3 Mei Wang,1,2 Xiaofei Zhang1,2 1Key Laboratory of Basic and New Drug Research in Chinese Medicine, Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, People’s Republic of China; 2Shaanxi Provincial University Engineering Research Center of Chinese Medicine Aromatic Industry, Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, People’s Republic of China; 3Shaanxi Dongtai Pharmaceutical Co., Ltd., Xianyang, Shaanxi, People’s Republic of China*These authors contributed equally to this workCorrespondence: Mei Wang; Xiaofei Zhang, Key Laboratory of Basic and New Drug Research in Chinese Medicine; Shaanxi Provincial University Engineering Research Center of Chinese Medicine Aromatic Industry, Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, 712046, People’s Republic of China, Email wangmei0708@163.com; 2051028@sntcm.edu.cnObjective: To explore the mechanism and active components of the anti-colitis effects of myrrh essential oil (MEO).Methods: In this study, we investigated the anti-inflammatory effects and molecular mechanisms of MEO on dextran sulfate sodium (DSS)-induced colitis with in vitro cell experiments, RNA-seq (RNA Sequencing), Weighted gene co-expression network analysis (WGCNA), combined with “weighting coefficient” network pharmacology, as and in vivo pharmacodynamic experiments. A 3% DSS solution was used to induce colitis in BALB/c mice and MEO was administered orally. We performed gas chromatography-mass spectrometry (GC-MS) analysis of the MEO components. The disease activity index (DAI) was evaluated by observing body weight, fecal characteristics, and blood in the stool of mice. The levels of inflammatory cytokines (TNF-α and IL-1β) in mouse serum were measured using ELISA (Enzyme-linked immunosorbent assay) kits. Additional

The atmosphere of Triton was probed directly by observing a ground-based stellar occultation on 6 October 2022. This rare event yielded 23 positive light curves collected from 13 separate observation stations contributing to our campaign. The significance of this event lies in its potential to directly validate the modest pressure fluctuation on Triton, a phenomenon not definitively verified by previous observations, including only five stellar occultations, and the Voyager 2 radio occultation in 1989. Using an approach consistent with a comparable study, we precisely determined a surface pressure of $14.07_{-0.13}^{+0.21}~\mathrm{\mu bar}$ in 2022. This new pressure rules out any significant monotonic variation in pressure between 2017 and 2022 through direct observations, as it is in alignment with the 2017 value. Additionally, both the pressures in 2017 and 2022 align with the 1989 value. This provides further support for the conclusion drawn from the previous volatile transport model simulation, which is consistent with the observed alignment between the pressures in 1989 and 2017; that is to say, the pressure fluctuation is modest. Moreover, this conclusion suggests the existence of a northern polar cap extended down to at least $45^\circ$N$-60^\circ$N and the presence of nitrogen between $30^\circ$S and $0^\circ$., Comment: Astronomy & Astrophysics, in press. 9 pages, 2 figures, 3 tables

Motivation: Single-cell RNA sequencing (scRNA-seq) is a groundbreaking technology extensively utilized in biological research, facilitating the examination of gene expression at the individual cell level within a given tissue sample. While numerous tools have been developed for scRNA-seq data analysis, the challenge persists in capturing the distinct features of such data and replicating virtual datasets that share analogous statistical properties. Results: Our study introduces a generative approach termed scRNA-seq Diffusion Transformer (scRDiT). This method generates virtual scRNA-seq data by leveraging a real dataset. The method is a neural network constructed based on Denoising Diffusion Probabilistic Models (DDPMs) and Diffusion Transformers (DiTs). This involves subjecting Gaussian noises to the real dataset through iterative noise-adding steps and ultimately restoring the noises to form scRNA-seq samples. This scheme allows us to learn data features from actual scRNA-seq samples during model training. Our experiments, conducted on two distinct scRNA-seq datasets, demonstrate superior performance. Additionally, the model sampling process is expedited by incorporating Denoising Diffusion Implicit Models (DDIM). scRDiT presents a unified methodology empowering users to train neural network models with their unique scRNA-seq datasets, enabling the generation of numerous high-quality scRNA-seq samples. Availability and implementation: https://github.com/DongShengze/scRDiT, Comment: 11 pages, 4 figures

In 1985 the National Gallery of Australian acquired 278 woodblock prints collected by British-born Peter Townsend during his period of residence in China between 1942 and 1951. The acquisition of this group of works, known as the Peter Townsend Collection, was made possible by the vision and generosity of the Australia-China Council. The collection consists of woodcuts and wood-engravings created during a period of great political, social and economic upheaval. The prints represent the modernisation of an ancient Chinese art form. Artists drew inspiration from European and American expressionism and Soviet-style Socialist Realism as well as traditional Chinese folk styles, resulting in striking innovations, as many formally trained artists revitalised familiar modes of visual expression for revolutionary ends. The project Artists Look at Art was a two-day workshop initiated by the University of Melbourne and the National Gallery of Australia which generated artists’ and academics’ responses to the collection, to create an information resource that will make the Peter Townsend Collection more accessible to members of the public and enrich the information about the collection currently available via the Gallery’s website. The workshop brought together participants in Australia and China and was simultaneously hosted onsite and online. Artists and art historians were selected for their connections to the Townsend collection, through their practice, training or the insights they were likely to bring to the workshop viewing and discussions. Several of the artists are represented in the National Collection. High resolution images of the Townsend collection were sent to all participants and each participant selected five works to discuss. The selected works were displayed at the workshop.The two-day workshop was video-recorded in the Learning Studio at the National Gallery. The outcome of the workshop is this edited video.

Recent deep learning methods have achieved promising results in image shadow removal. However, most of the existing approaches focus on working locally within shadow and non-shadow regions, resulting in severe artifacts around the shadow boundaries as well as inconsistent illumination between shadow and non-shadow regions. It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. In this work, we first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer, to exploit non-shadow regions to help shadow region restoration. A multi-scale channel attention framework is employed to hierarchically capture the global information. Based on that, we propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD, to evaluate the proposed method. Our method achieves state-of-the-art performance by using up to 150X fewer model parameters., Comment: Accepted by AAAI2023

Sihua Wen,1,* Kai Yang,1,* Yunfeng Bai,2 Yanan Wu,1 Ding Liu,1 Xu Wu,1 Xiaofei Zhang,1 Jing Sun1 1Department of Pharmaceutics, The Key Laboratory of Basic and New Drug Research of Traditional Chinese Medicine, Shaanxi University of Chinese Medicine, Xianyang, People’s Republic of China; 2Shaanxi Dongtai Pharmaceutical Co., Ltd, Shaanxi, People’s Republic of China*These authors contributed equally to this workCorrespondence: Xiaofei Zhang; Jing Sun, Department of Pharmaceutics, College of Pharmacy, Shaanxi University of Chinese Medicine, Xianyang, People’s Republic of China, Tel +86 177 7003 7322, Fax +86 029-38185333, Email 2051028@sntcm.edu.cn; sunjing202203@126.comPurpose: To study the active components, drug targets and mechanism of Schisandra chinensis (S.chinensis) combined with coenzyme Q10 (CQ10) in the treatment of heart failure (HF).Methods: Network pharmacology combined with the gene expression omnibus chip method to analyze the main pathways by which S.chinensis combined with CQ10 functioned to treat heart failure. Subsequently, the biological activities of the major pathway key proteins and their corresponding compounds were verified by molecular docking techniques. Finally, the molecular mechanism of S. chinensis combined with CQ10 for the treatment of heart failure was verified using a rat heart failure model induced by isoproterenol hydrochloride and using hematoxylin-eosin staining, TUNEL, immunohistochemistry and Western blot.Results: Network pharmacology combined with experimental validation suggests that the mechanism of action of S.chinensis combined with CQ10 in the treatment of heart failure may involve CQ10, Citral, Schisandrone, Schisanhenol B, Gomisin O, Schisandrin C and other components, which may synergistically inhibit the PI3K-AKT signaling pathway and affect the expression of AKT1, PIK3CG and other targets on this pathway. In addition, S. chinensis combined with CQ10 could effectively improve the cardiac c

The article introduces a new method for applying Quantum Clustering to graph structures. Quantum Clustering (QC) is a novel density-based unsupervised learning method that determines cluster centers by constructing a potential function. In this method, we use the Graph Gradient Descent algorithm to find the centers of clusters. GPU parallelization is utilized for computing potential values. We also conducted experiments on five widely used datasets and evaluated using four indicators. The results show superior performance of the method. Finally, we discuss the influence of $\sigma$ on the experimental results.

Diffusion models (DMs) have recently been introduced in image deblurring and exhibited promising performance, particularly in terms of details reconstruction. However, the diffusion model requires a large number of inference iterations to recover the clean image from pure Gaussian noise, which consumes massive computational resources. Moreover, the distribution synthesized by the diffusion model is often misaligned with the target results, leading to restrictions in distortion-based metrics. To address the above issues, we propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Specifically, we perform the DM in a highly compacted latent space to generate the prior feature for the deblurring process. The deblurring process is implemented by a regression-based method to obtain better distortion accuracy. Meanwhile, the highly compact latent space ensures the efficiency of the DM. Furthermore, we design the hierarchical integration module to fuse the prior into the regression-based model from multiple scales, enabling better generalization in complex blurry scenarios. Comprehensive experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods. Code and trained models are available at https://github.com/zhengchen1999/HI-Diff., Comment: Accepted to NeurIPS 2023 (Spotlight). Code is available at https://github.com/zhengchen1999/HI-Diff

Bisected N-glycans represent a unique class of protein N-glycans that play critical roles in many biological processes. Herein, we describe the systematic synthesis of these structures. A bisected N-glycan hexasaccharide was chemically assembled with two orthogonal protecting groups attached at the C2 of the branching mannose residues, followed by sequential installation of GlcNAc and LacNAc building blocks to afford two asymmetric bisecting "cores". Subsequent enzymatic modular extension of the "cores" yielded a comprehensive library of biantennary N-glycans containing the bisecting GlcNAc and presenting 6 common glycan determinants in a combinatorial fashion. These bisected N-glycans and their non-bisected counterparts were used to construct a distinctive glycan microarray to study their recognition by a wide variety of glycan-binding proteins (GBPs), including plant lectins, animal lectins, and influenza A virus hemagglutinins. Significantly, the bisecting GlcNAc could bestow (PHA-L, rDCIR2), enhance (PHA-E), or abolish (ConA, GNL, anti-CD15s antibody, etc.) N-glycan recognition of specific GBPs, and is tolerated by many others. In summary, synthesized compounds and the unique glycan microarray provide ideal standards and tools for glycoanalysis and functional glycomic studies. The microarray data provide new information regarding the fine details of N-glycan recognition by GBPs, and in turn improve their applications.

Deep reinforcement learning (DRL) has proven to be an effective framework for solving various complex control problems. In manufacturing, industrial robots can be trained to learn dexterous manipulation skills from raw pixels with DRL. However, training robots in the real world is a time-consuming, high-cost and of safety concerns process. A frequently adopted approach for easing this is to train robots through simulations first and then deploy algorithms (or policies) on physical robots. How to transfer policies of robot learning from simulation to the real world is a challenging issue. Digital twin that is able to create a dynamic, up-to-date representation of a physical robotic grasping system provides an effective approach for addressing this issue. In this paper, we focus on the scenario of DRL-based assembly-oriented industrial grasping and propose a digital twin-enabled approach for achieving effective transfer of DRL algorithms to a physical robot. Two parallel training systems, i.e., the physical robotic system and corresponding digital twin system, respectively, are established, which take virtual and real images as inputs. The output of the digital twin system is used to correct the real grasping point so that accurate grasping can be achieved. Experimental results verify the effectiveness of the intelligent grasping algorithm and the digital twin-enabled sim-to-real transfer approach and mechanism., QC 20220627

[This corrects the article DOI: 10.1039/D1SC05435J.].

This paper reviews the Challenge on Super-Resolution of Compressed Image and Video at AIM 2022. This challenge includes two tracks. Track 1 aims at the super-resolution of compressed image, and Track~2 targets the super-resolution of compressed video. In Track 1, we use the popular dataset DIV2K as the training, validation and test sets. In Track 2, we propose the LDV 3.0 dataset, which contains 365 videos, including the LDV 2.0 dataset (335 videos) and 30 additional videos. In this challenge, there are 12 teams and 2 teams that submitted the final results to Track 1 and Track 2, respectively. The proposed methods and solutions gauge the state-of-the-art of super-resolution on compressed image and video. The proposed LDV 3.0 dataset is available at https://github.com/RenYang-home/LDV_dataset. The homepage of this challenge is at https://github.com/RenYang-home/AIM22_CompressSR., Comment: Camera-ready version

The success of existing video super-resolution (VSR) algorithms stems mainly exploiting the temporal information from the neighboring frames. However, none of these methods have discussed the influence of the temporal redundancy in the patches with stationary objects and background and usually use all the information in the adjacent frames without any discrimination. In this paper, we observe that the temporal redundancy will bring adverse effect to the information propagation,which limits the performance of the most existing VSR methods. Motivated by this observation, we aim to improve existing VSR algorithms by handling the temporal redundancy patches in an optimized manner. We develop two simple yet effective plug and play methods to improve the performance of existing local and non-local propagation-based VSR algorithms on widely-used public videos. For more comprehensive evaluating the robustness and performance of existing VSR algorithms, we also collect a new dataset which contains a variety of public videos as testing set. Extensive evaluations show that the proposed methods can significantly improve the performance of existing VSR methods on the collected videos from wild scenarios while maintain their performance on existing commonly used datasets. The code is available at https://github.com/HYHsimon/Boosted-VSR.

Object detection is a basic computer vision task to loccalize and categorize objects in a given image. Most state-of-the-art detection methods utilize a fixed number of proposals as an intermediate representation of object candidates, which is unable to adapt to different computational constraints during inference. In this paper, we propose a simple yet effective method which is adaptive to different computational resources by generating dynamic proposals for object detection. We first design a module to make a single query-based model to be able to inference with different numbers of proposals. Further, we extend it to a dynamic model to choose the number of proposals according to the input image, greatly reducing computational costs. Our method achieves significant speed-up across a wide range of detection models including two-stage and query-based models while obtaining similar or even better accuracy.

Deep learning based approaches has achieved great performance in single image super-resolution (SISR). However, recent advances in efficient super-resolution focus on reducing the number of parameters and FLOPs, and they aggregate more powerful features by improving feature utilization through complex layer connection strategies. These structures may not be necessary to achieve higher running speed, which makes them difficult to be deployed to resource-constrained devices. In this work, we propose a novel Residual Local Feature Network (RLFN). The main idea is using three convolutional layers for residual local feature learning to simplify feature aggregation, which achieves a good trade-off between model performance and inference time. Moreover, we revisit the popular contrastive loss and observe that the selection of intermediate features of its feature extractor has great influence on the performance. Besides, we propose a novel multi-stage warm-start training strategy. In each stage, the pre-trained weights from previous stages are utilized to improve the model performance. Combined with the improved contrastive loss and training strategy, the proposed RLFN outperforms all the state-of-the-art efficient image SR models in terms of runtime while maintaining both PSNR and SSIM for SR. In addition, we won the first place in the runtime track of the NTIRE 2022 efficient super-resolution challenge. Code will be available at https://github.com/fyan111/RLFN.

This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29.00dB on DIV2K validation set. IMDN is set as the baseline for efficiency measurement. The challenge had 3 tracks including the main track (runtime), sub-track one (model complexity), and sub-track two (overall performance). In the main track, the practical runtime performance of the submissions was evaluated. The rank of the teams were determined directly by the absolute value of the average runtime on the validation set and test set. In sub-track one, the number of parameters and FLOPs were considered. And the individual rankings of the two metrics were summed up to determine a final ranking in this track. In sub-track two, all of the five metrics mentioned in the description of the challenge including runtime, parameter count, FLOPs, activations, and memory consumption were considered. Similar to sub-track one, the rankings of five metrics were summed up to determine a final ranking. The challenge had 303 registered participants, and 43 teams made valid submissions. They gauge the state-of-the-art in efficient single image super-resolution., Comment: Validation code of the baseline model is available at https://github.com/ofsoundof/IMDN. Validation of all submitted models is available at https://github.com/ofsoundof/NTIRE2022_ESR

Runtime and memory consumption are two important aspects for efficient image super-resolution (EISR) models to be deployed on resource-constrained devices. Recent advances in EISR exploit distillation and aggregation strategies with plenty of channel split and concatenation operations to make full use of limited hierarchical features. In contrast, sequential network operations avoid frequently accessing preceding states and extra nodes, and thus are beneficial to reducing the memory consumption and runtime overhead. Following this idea, we design our lightweight network backbone by mainly stacking multiple highly optimized convolution and activation layers and decreasing the usage of feature fusion. We propose a novel sequential attention branch, where every pixel is assigned an important factor according to local and global contexts, to enhance high-frequency details. In addition, we tailor the residual block for EISR and propose an enhanced residual block (ERB) to further accelerate the network inference. Finally, combining all the above techniques, we construct a fast and memory-efficient network (FMEN) and its small version FMEN-S, which runs 33% faster and reduces 74% memory consumption compared with the state-of-the-art EISR model: E-RFDN, the champion in AIM 2020 efficient super-resolution challenge. Besides, FMEN-S achieves the lowest memory consumption and the second shortest runtime in NTIRE 2022 challenge on efficient super-resolution. Code is available at https://github.com/NJU-Jet/FMEN., Comment: Accepted by NTIRE 2022 (CVPR Workshop)

O-GalNAc glycans (or mucin O-glycans) play pivotal roles in diverse biological and pathological processes, including tumor growth and progression. Structurally defined O-GalNAc glycans are essential for functional studies but synthetic challenges and their inherent structural diversity and complexity have limited access to these compounds. Herein, we report an efficient and robust chemoenzymatic modular assembly (CEMA) strategy to construct structurally diverse O-GalNAc glycans. The key to this strategy is the convergent assembly of O-GalNAc cores 1-4 and 6 from three chemical building blocks, followed by enzymatic diversification of the cores by 13 well-tailored enzyme modules. A total of 83 O-GalNAc glycans presenting various natural glycan epitopes are obtained and used to generate a unique synthetic mucin O-glycan microarray. Binding specificities of glycan-binding proteins (GBPs) including plant lectins and selected anti-glycan antibodies towards these O-GalNAc glycans are revealed by this microarray, promoting their applicability in functional O-glycomics. Serum samples from colorectal cancer patients and healthy controls are assayed using the array reveal higher bindings towards less common cores 3, 4, and 6 than abundant cores 1 and 2, providing insights into O-GalNAc glycan structure-activity relationships.

China as an Issue is an ongoing lecture series organized by the Beijing Inside-Out Art Museum since 2018. Chinese scholars are invited to discuss topics related to China or the world, as well as foreign scholars to speak about China or international questions involving the subject of China. Through rigorous scrutinization of a specific issue we try to avoid making generalizations as well as the parochial tendency to reject extraterritorial or foreign theories in the study of domestic issues. The attempt made here is not only to see the world from a local Chinese perspective, but also to observe China ..., https://www.librarystack.org/china-as-an-issue-volume-1/?ref=unknown

O-GalNAc glycans (or mucin O-glycans) play pivotal roles in diverse biological and pathological processes, including tumor growth and progression. Structurally defined O-GalNAc glycans are essential for functional studies but synthetic challenges and their inherent structural diversity and complexity have limited access to these compounds. Herein, we report an efficient and robust chemoenzymatic modular assembly (CEMA) strategy to construct structurally diverse O-GalNAc glycans. The key to this strategy is the convergent assembly of O-GalNAc cores 1-4 and 6 from three chemical building blocks, followed by enzymatic diversification of the cores by 13 well-tailored enzyme modules. A total of 83 O-GalNAc glycans presenting various natural glycan epitopes are obtained and used to generate a unique synthetic mucin O-glycan microarray. Binding specificities of glycan-binding proteins (GBPs) including plant lectins and selected anti-glycan antibodies towards these O-GalNAc glycans are revealed by this microarray, promoting their applicability in functional O-glycomics. Serum samples from colorectal cancer patients and healthy controls are assayed using the array reveal higher bindings towards less common cores 3, 4, and 6 than abundant cores 1 and 2, providing insights into O-GalNAc glycan structure-activity relationships.

Accessing large numbers of structurally diverse glycans and derivatives is essential to functional glycomics. We showed a general tolerance of galactosyltransferases toward uridine-diphosphate-galactosamine (UDP-GalN), which is not a commonly used sugar nucleotide donor. The property was harnessed to develop a two-step chemoenzymatic strategy for facile synthesis of novel and divergent N-acetylgalactosamine (GalNAc)-glycosides and derivatives in preparative scales. The discovery and the application of the new property of existing glycosyltransferases expand their catalytic capabilities in generating novel carbohydrate linkages, thus prompting the synthesis of diverse glycans and glycoconjugates for biological studies.

China as an Issue is an ongoing lecture series organized by the Beijing Inside-Out Art Museum since 2018. Chinese scholars are invited to discuss topics related to China or the world, as well as foreign scholars to speak about China or international questions involving the subject of China. Through rigorous scrutinization of a specific issue we try to avoid making generalizations as well as the parochial tendency to reject extraterritorial or foreign theories in the study of domestic issues. The attempt made here is not only to see the world from a local Chinese perspective, but also to observe China from a global perspective. By calling into question the underlying typology of the inside and the outside we consider China as an issue requiring discussion, rather than already having an established premise. By inviting fellow thinkers from a wide range of disciplines to discuss these topics we were able to negotiate and push the parameters of art and stimulate a discourse that intersects the arts with other discursive fields…, https://www.librarystack.org/china-as-an-issue-volume-1/?ref=unknown

China as an Issue is an ongoing lecture series organized by the Beijing Inside-Out Art Museum since 2018. Chinese scholars are invited to discuss topics related to China or the world, as well as foreign scholars to speak about China or international questions involving the subject of China. Through rigorous scrutinization of a specific issue we try to avoid making generalizations as well as the parochial tendency to reject extraterritorial or foreign theories in the study of domestic issues. The attempt made here is not only to see the world from a local Chinese perspective, but also to observe China from a global perspective. By calling into question the underlying typology of the inside and the outside we consider China as an issue requiring discussion, rather than already having an established premise. By inviting fellow thinkers from a wide range of disciplines to discuss these topics we were able to negotiate and push the parameters of art and stimulate a discourse that intersects the arts with other discursive fields…, https://www.librarystack.org/china-as-an-issue-volume-1/?ref=unknown

Transformer-based models have demonstrated excellent capabilities of capturing patterns and structures in natural language generation and achieved state-of-the-art results in many tasks. In this paper we present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism, which leverages the memorized information in the training data via k-Nearest Neighbor search. Our system is evaluated on two datasets made by customer/assistant dialogs: the Taskmaster-1, released by Google and holding high quality, goal-oriented conversational data and a proprietary dataset collected from a real customer service call center. Both achieve better BLEU scores over strong baselines., Comment: The International FLAIRS Conference Proceedings volume 34 issue 1

Understanding the informative behaviour of deep neural networks is challenged by misused estimators and the complexity of network structure, which leads to inconsistent observations and diversified interpretation. Here we propose the LogDet estimator -- a reliable matrix-based entropy estimator that approximates Shannon differential entropy. We construct informative measurements based on LogDet estimator, verify our method with comparable experiments and utilize it to analyse neural network behaviour. Our results demonstrate the LogDet estimator overcomes the drawbacks that emerge from highly diverse and degenerated distribution thus is reliable to estimate entropy in neural networks. The Network analysis results also find a functional distinction between shallow and deeper layers, which can help understand the compression phenomenon in the Information bottleneck theory of neural networks., Comment: 15pages,22 figures

In this paper, we explore open-domain sketch-to-photo translation, which aims to synthesize a realistic photo from a freehand sketch with its class label, even if the sketches of that class are missing in the training data. It is challenging due to the lack of training supervision and the large geometric distortion between the freehand sketch and photo domains. To synthesize the absent freehand sketches from photos, we propose a framework that jointly learns sketch-to-photo and photo-to-sketch generation. However, the generator trained from fake sketches might lead to unsatisfying results when dealing with sketches of missing classes, due to the domain gap between synthesized sketches and real ones. To alleviate this issue, we further propose a simple yet effective open-domain sampling and optimization strategy to "fool" the generator into treating fake sketches as real ones. Our method takes advantage of the learned sketch-to-photo and photo-to-sketch mapping of in-domain data and generalizes it to the open-domain classes. We validate our method on the Scribble and SketchyCOCO datasets. Compared with the recent competing methods, our approach shows impressive results in synthesizing realistic color, texture, and maintaining the geometric composition for various categories of open-domain sketches. Our code is available at https://github.com/Mukosame/AODA, Comment: Accepted by WACV 2022

Video inpainting aims to fill spatio-temporal "corrupted" regions with plausible content. To achieve this goal, it is necessary to find correspondences from neighbouring frames to faithfully hallucinate the unknown content. Current methods achieve this goal through attention, flow-based warping, or 3D temporal convolution. However, flow-based warping can create artifacts when optical flow is not accurate, while temporal convolution may suffer from spatial misalignment. We propose 'Progressive Temporal Feature Alignment Network', which progressively enriches features extracted from the current frame with the feature warped from neighbouring frames using optical flow. Our approach corrects the spatial misalignment in the temporal feature propagation stage, greatly improving visual quality and temporal consistency of the inpainted videos. Using the proposed architecture, we achieve state-of-the-art performance on the DAVIS and FVI datasets compared to existing deep learning approaches. Code is available at https://github.com/MaureenZOU/TSAM., Comment: Accepted in CVPR2021

Xu Lin,1,* Pei Wang,1 Ding-Wei Liu,2 Yu-Wei Guo,1 Chun-Hui Xie,3 Bin Wang,1,* Rui Dong,4 Li-Xin Sun,1 Ming-Shan Wang,1 Yan-Lin Bi1 1Department of Anesthesiology, Qingdao Municipal Hospital Affiliated to Qingdao University, Qingdao, People’s Republic of China; 2Department of Laboratory, Qingdao Municipal Hospital Affiliated to Qingdao University, Qingdao, People’s Republic of China; 3Department of Anesthesiology, Weifang Medical University, Weifang, People’s Republic of China; 4Department of Anesthesiology, Drum Tower Hospital Affiliated to Nanjing University Medical School, Nanjing, People’s Republic of China*These authors contributed equally to this workCorrespondence: Yan-Lin Bi; Ming-Shan WangDepartment of Anesthesiology, Qingdao Municipal Hospital Affiliated to Qingdao University, Qingdao, People’s Republic of ChinaTel +86 186 6167 5610; +86 186 61675608Email yanlinbi68@sina.cn; wmsqingdao@163.comPurpose: Postoperative delirium (POD) is common in elderly patients undergoing laparoscopic surgery for gastric and colorectal malignancies. POD may be affected by different fraction of inspired oxygen (FiO2). The purpose of this study was to compare the effects of different FiO2 on POD.Patients and Methods: A randomized, double-blind controlled trial was performed in Qingdao Municipal Hospital Affiliated to Qingdao University. A total of 662 patients aged 65 to 85 years old underwent isolated laparoscopic radical gastrectomy, radical resection of colon cancer, or radical resection of rectal cancer only. A random number table method was used to divide the patients into two groups: 40% FiO2 (group A) and 80% FiO2 (group B). The primary endpoint was the incidence of POD, which was assessed by the Confusion Assessment Method (CAM) twice daily during the first 7 postoperative days, and POD severity was measured by the Memorial Delirium Assessment Scale (MDAS). The secondary endpoints were the intraoperative regional cerebral

In their 2018 paper Preference for realistic art predicts support for Brexit sociologists Noah Carl, Lindsay Richard and Anthony Heath presented evidence that research participants who voted Leave in the 2016 Brexit referendum were 15 – 20% more likely to exhibit a preference for ‘realistic’ painting. On the face of things this makes perfect sense; many Brexit voters are ‘small-c’ conservatives, committed to ‘traditional’ conservative values and therefore, one would assume, ‘traditional’ painting. Historically or traditionally, what has been celebrated as ‘Art’ until the 20th century was broadly realistic, figurative or representational art. The development of Western culture has historically been tied into the development of a ‘realist’ art. From the three-point perspective of the enlightenment to the invention of the camera in the industrial revolution, ‘Art History’ (euro- phallo-centric art history) has been, like the society from which it emerges, a slow progression towards more realistic, productive and exploitative modes of capture. As Carol Yinghua Lu and Liu Ding’s text on Socialist Realism and the text ‘Notes on Abstraction and Figuration’ outline, there is no ignoring the link between art and politics. Because of this inexorable tie between exploitation and representation, the exploiters (those with the means of representation) have always been in charge of what and who is represented. The result of this relationship is made grossly clear in artworks such as Rex Whistler’s The Expedition in Pursuit of Rare Meat, painted in 1927, which surrounds diners in the Tate Britain’s private restaurant and depicts images of enslaved black children. Unless you are Queen Victoria or a Dutch merchant, the idea of acquiring agency through figurative or representational artwork has historically, it seems, been out of the question…, https://www.librarystack.org/representation/?ref=unknown

In their 2018 paper Preference for realistic art predicts support for Brexit sociologists Noah Carl, Lindsay Richard and Anthony Heath presented evidence that research participants who voted Leave in the 2016 Brexit referendum were 15 – 20% more likely to exhibit a preference for ‘realistic’ painting. On the face of things this makes perfect sense; many Brexit voters are ‘small-c’ conservatives, committed to ‘traditional’ conservative values and therefore, one would assume, ‘traditional’ painting. Historically or traditionally, what has been celebrated as ‘Art’ until the 20th century was broadly realistic, figurative or representational art. The development of Western culture has historically been tied into the development of a ‘realist’ art. From the three-point perspective of the enlightenment to the invention of the camera in the industrial revolution, ‘Art History’ (euro- phallo-centric art history) has been, like the society from which it emerges, a slow progression towards more realistic, productive and exploitative modes of capture. As Carol Yinghua Lu and Liu Ding’s text on Socialist Realism and the text ‘Notes on Abstraction and Figuration’ outline, there is no ignoring the link between art and politics. Because of this inexorable tie between exploitation and representation, the exploiters (those with the means of representation) have always been in charge of what and who is represented. The result of this relationship is made grossly clear in artworks such as Rex Whistler’s The Expedition in Pursuit of Rare Meat, painted in 1927, which surrounds diners in the Tate Britain’s private restaurant and depicts images of enslaved black children. Unless you are Queen Victoria or a Dutch merchant, the idea of acquiring agency through figurative or representational artwork has historically, it seems, been out of the question…, https://www.librarystack.org/representation/?ref=unknown

We propose a new assumption in outlier detection: Normal data instances are commonly located in the area that there is hardly any fluctuation on data density, while outliers are often appeared in the area that there is violent fluctuation on data density. And based on this hypothesis, we apply a novel density-based approach to unsupervised outlier detection. This approach, called Quantum Clustering (QC), deals with unlabeled data processing and constructs a potential function to find the centroids of clusters and the outliers. The experiments show that the potential function could clearly find the hidden outliers in data points effectively. Besides, by using QC, we could find more subtle outliers by adjusting the parameter $\sigma$. Moreover, our approach is also evaluated on two datasets (Air Quality Detection and Darwin Correspondence Project) from two different research areas, and the results show the wide applicability of our method., Comment: 9 pages, 18 figures

Inspired by the robustness and efficiency of sparse representation in sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks. Our method structurally enforces sparsity constraints upon hidden neurons. The sparsity constraints are favorable for gradient-based learning algorithms and attachable to convolution layers in various networks. Sparsity in neurons enables computation saving by only operating on non-zero components without hurting accuracy. Meanwhile, our method can magnify representation dimensionality and model capacity with negligible additional computation cost. Experiments show that sparse representation is crucial in deep neural networks for multiple image restoration tasks, including image super-resolution, image denoising, and image compression artifacts removal. Code is available at https://github.com/ychfan/nsr

Tensor networks (TN) have found a wide use in machine learning, and in particular, TN and deep learning bear striking similarities. In this work, we propose the quantum-classical hybrid tensor networks (HTN) which combine tensor networks with classical neural networks in a uniform deep learning framework to overcome the limitations of regular tensor networks in machine learning. We first analyze the limitations of regular tensor networks in the applications of machine learning involving the representation power and architecture scalability. We conclude that in fact the regular tensor networks are not competent to be the basic building blocks of deep learning. Then, we discuss the performance of HTN which overcome all the deficiency of regular tensor networks for machine learning. In this sense, we are able to train HTN in the deep learning way which is the standard combination of algorithms such as Back Propagation and Stochastic Gradient Descent. We finally provide two applicable cases to show the potential applications of HTN, including quantum states classification and quantum-classical autoencoder. These cases also demonstrate the great potentiality to design various HTN in deep learning way.

In this paper, we tackle the problem of enhancing real-world low-light images with significant noise in an unsupervised fashion. Conventional unsupervised learning-based approaches usually tackle the low-light image enhancement problem using an image-to-image translation model. They focus primarily on illumination or contrast enhancement but fail to suppress the noise that ubiquitously exists in images taken under real-world low-light conditions. To address this issue, we explicitly decouple this task into two sub-tasks: illumination enhancement and noise suppression. We propose to learn a two-stage GAN-based framework to enhance the real-world low-light images in a fully unsupervised fashion. To facilitate the unsupervised training of our model, we construct samples with pseudo labels. Furthermore, we propose an adaptive content loss to suppress real image noise in different regions based on illumination intensity. In addition to conventional benchmark datasets, a new unpaired low-light image enhancement dataset is built and used to thoroughly evaluate the performance of our model. Extensive experiments show that our proposed method outperforms the state-of-the-art unsupervised image enhancement methods in terms of both illumination enhancement and noise reduction., Comment: 12 pages

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their "clean" correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code will be available at https://github.com/SHI-Labs/Pyramid-Attention-Networks

Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video. Previous approaches only utilize single-frame features for the detection, segmentation, and tracking of objects and they suffer in the video scenario due to several distinct challenges such as motion blur and drastic appearance change. To eliminate ambiguities introduced by only using single-frame features, we propose a novel comprehensive feature aggregation approach (CompFeat) to refine features at both frame-level and object-level with temporal and spatial context information. The aggregation process is carefully designed with a new attention mechanism which significantly increases the discriminative power of the learned features. We further improve the tracking capability of our model through a siamese design by incorporating both feature similarities and spatial similarities. Experiments conducted on the YouTube-VIS dataset validate the effectiveness of proposed CompFeat. Our code will be available at https://github.com/SHI-Labs/CompFeat-for-Video-Instance-Segmentation., Comment: Accepted to AAAI 2021

In this paper, we present a simple and parameter-efficient drop-in module for one-stage object detectors like SSD [25] when learning from scratch (i.e., without pre-trained models). We call our module GFR (Gated Feature Reuse), which exhibits two main advantages. First, we introduce a novel gate-controlled prediction strategy enabled by Squeeze-and-Excitation [14] to adaptively enhance or attenuate supervision at different scales based on the input object size. As a result, our model is more effective in detecting diverse sizes of objects. Second, we propose a feature-pyramids structure to squeeze rich spatial and semantic features into a single prediction layer, which strengthens feature representation and reduces the number of parameters to learn. We apply the proposed structure on DSOD [31, 32] and SSD [25] detection frameworks, and evaluate the performance on PASCAL VOC 2007, 2012, 2012 Comp3 and COCO datasets. With fewer model parameters, GFR-DSOD outperforms the baseline DSOD by 1.4%, 1.1%, 1.7% and 0.6%, respectively. GFR-SSD also outperforms the original SSD and SSD with dense prediction by 3.6% and 2.8% on VOC 2007 dataset.

In this paper, we present a simple and parameter-efficient drop-in module for one-stage object detectors like SSD [25] when learning from scratch (i.e., without pre-trained models). We call our module GFR (Gated Feature Reuse), which exhibits two main advantages. First, we introduce a novel gate-controlled prediction strategy enabled by Squeeze-and-Excitation [14] to adaptively enhance or attenuate supervision at different scales based on the input object size. As a result, our model is more effective in detecting diverse sizes of objects. Second, we propose a feature-pyramids structure to squeeze rich spatial and semantic features into a single prediction layer, which strengthens feature representation and reduces the number of parameters to learn. We apply the proposed structure on DSOD [31, 32] and SSD [25] detection frameworks, and evaluate the performance on PASCAL VOC 2007, 2012, 2012 Comp3 and COCO datasets. With fewer model parameters, GFR-DSOD outperforms the baseline DSOD by 1.4%, 1.1%, 1.7% and 0.6%, respectively. GFR-SSD also outperforms the original SSD and SSD with dense prediction by 3.6% and 2.8% on VOC 2007 dataset.

In this paper, we present a simple and parameter-efficient drop-in module for one-stage object detectors like SSD [25] when learning from scratch (i.e., without pre-trained models). We call our module GFR (Gated Feature Reuse), which exhibits two main advantages. First, we introduce a novel gate-controlled prediction strategy enabled by Squeeze-and-Excitation [14] to adaptively enhance or attenuate supervision at different scales based on the input object size. As a result, our model is more effective in detecting diverse sizes of objects. Second, we propose a feature-pyramids structure to squeeze rich spatial and semantic features into a single prediction layer, which strengthens feature representation and reduces the number of parameters to learn. We apply the proposed structure on DSOD [31, 32] and SSD [25] detection frameworks, and evaluate the performance on PASCAL VOC 2007, 2012, 2012 Comp3 and COCO datasets. With fewer model parameters, GFR-DSOD outperforms the baseline DSOD by 1.4%, 1.1%, 1.7% and 0.6%, respectively. GFR-SSD also outperforms the original SSD and SSD with dense prediction by 3.6% and 2.8% on VOC 2007 dataset.

While scale-invariant modeling has substantially boosted the performance of visual recognition tasks, it remains largely under-explored in deep networks based image restoration. Naively applying those scale-invariant techniques (e.g. multi-scale testing, random-scale data augmentation) to image restoration tasks usually leads to inferior performance. In this paper, we show that properly modeling scale-invariance into neural networks can bring significant benefits to image restoration performance. Inspired from spatial-wise convolution for shift-invariance, "scale-wise convolution" is proposed to convolve across multiple scales for scale-invariance. In our scale-wise convolutional network (SCN), we first map the input image to the feature space and then build a feature pyramid representation via bi-linear down-scaling progressively. The feature pyramid is then passed to a residual network with scale-wise convolutions. The proposed scale-wise convolution learns to dynamically activate and aggregate features from different input scales in each residual building block, in order to exploit contextual information on multiple scales. In experiments, we compare the restoration accuracy and parameter efficiency among our model and many different variants of multi-scale neural networks. The proposed network with scale-wise convolution achieves superior performance in multiple image restoration tasks including image super-resolution, image denoising and image compression artifacts removal. Code and models are available at: https://github.com/ychfan/scn_sr, Comment: AAAI 2020

Blind video deblurring restores sharp frames from a blurry sequence without any prior. It is a challenging task because the blur due to camera shake, object movement and defocusing is heterogeneous in both temporal and spatial dimensions. Traditional methods train on datasets synthesized with a single level of blur, and thus do not generalize well across levels of blurriness. To address this challenge, we propose a dual attention mechanism to dynamically aggregate temporal cues for deblurring with an end-to-end trainable network structure. Specifically, an internal attention module adaptively selects the optimal temporal scales for restoring the sharp center frame. An external attention module adaptively aggregates and refines multiple sharp frame estimates, from several internal attention modules designed for different blur levels. To train and evaluate on more diverse blur severity levels, we propose a Challenging DVD dataset generated from the raw DVD video set by pooling frames with different temporal windows. Our framework achieves consistently better performance on this more challenging dataset while obtaining strongly competitive results on the original DVD benchmark. Extensive ablative studies and qualitative visualizations further demonstrate the advantage of our method in handling real video blur., Comment: Accepted in WACV 2020

Recent years, many researches attempt to open the black box of deep neural networks and propose a various of theories to understand it. Among them, Information Bottleneck (IB) theory claims that there are two distinct phases consisting of fitting phase and compression phase in the course of training. This statement attracts many attentions since its success in explaining the inner behavior of feedforward neural networks. In this paper, we employ IB theory to understand the dynamic behavior of convolutional neural networks (CNNs) and investigate how the fundamental features such as convolutional layer width, kernel size, network depth, pooling layers and multi-fully connected layer have impact on the performance of CNNs. In particular, through a series of experimental analysis on benchmark of MNIST and Fashion-MNIST, we demonstrate that the compression phase is not observed in all these cases. This shows us the CNNs have a rather complicated behavior than feedforward neural networks., Comment: 7 pages,28 figures

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. The code is available at \url{https://github.com/yueruchen/EnlightenGAN}

Tensor network (TN) has recently triggered extensive interests in developing machine-learning models in quantum many-body Hilbert space. Here we purpose a generative TN classification (GTNC) approach for supervised learning. The strategy is to train the generative TN for each class of the samples to construct the classifiers. The classification is implemented by comparing the distance in the many-body Hilbert space. The numerical experiments by GTNC show impressive performance on the MNIST and Fashion-MNIST dataset. The testing accuracy is competitive to the state-of-the-art convolutional neural network while higher than the naive Bayes classifier (a generative classifier) and support vector machine. Moreover, GTNC is more efficient than the existing TN models that are in general discriminative. By investigating the distances in the many-body Hilbert space, we find that (a) the samples are naturally clustering in such a space; and (b) bounding the bond dimensions of the TN's to finite values corresponds to removing redundant information in the image recognition. These two characters make GTNC an adaptive and universal model of excellent performance., Comment: 7 pages, 5 figures

This paper studies the challenging problem of fingerprint image denoising and inpainting. To tackle the challenge of suppressing complicated artifacts (blur, brightness, contrast, elastic transformation, occlusion, scratch, resolution, rotation, and so on) while preserving fine textures, we develop a multi-scale convolutional network, termed U- Finger. Based on the domain expertise, we show that the usage of dilated convolutions as well as the removal of padding have important positive impacts on the final restoration performance, in addition to multi-scale cascaded feature modules. Our model achieves the overall ranking of No.2 in the ECCV 2018 Chalearn LAP Inpainting Competition Track 3 (Fingerprint Denoising and Inpainting). Among all participating teams, we obtain the MSE of 0.0231 (rank 2), PSNR 16.9688 dB (rank 2), and SSIM 0.8093 (rank 3) on the hold-out testing set., Comment: ECCV 2018 Track-3 Challenge Inpainting to denoise fingerprint

Image denoising and high-level vision tasks are usually handled independently in the conventional practice of computer vision, and their connection is fragile. In this paper, we cope with the two jointly and explore the mutual influence between them with the focus on two questions, namely (1) how image denoising can help improving high-level vision tasks, and (2) how the semantic information from high-level vision tasks can be used to guide image denoising. First for image denoising we propose a convolutional neural network in which convolutions are conducted in various spatial resolutions via downsampling and upsampling operations in order to fuse and exploit contextual information on different scales. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We experimentally show that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network produces more visually appealing results. Extensive experiments demonstrate the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online: https://github.com/Ding-Liu/DeepDenoising, Comment: arXiv admin note: text overlap with arXiv:1706.04284

Advances in image super-resolution (SR) have recently benefited significantly from rapid developments in deep neural networks. Inspired by these recent discoveries, we note that many state-of-the-art deep SR architectures can be reformulated as a single-state recurrent neural network (RNN) with finite unfoldings. In this paper, we explore new structures for SR based on this compact RNN view, leading us to a dual-state design, the Dual-State Recurrent Network (DSRN). Compared to its single state counterparts that operate at a fixed spatial resolution, DSRN exploits both low-resolution (LR) and high-resolution (HR) signals jointly. Recurrent signals are exchanged between these states in both directions (both LR to HR and HR to LR) via delayed feedback. Extensive quantitative and qualitative evaluations on benchmark datasets and on a recent challenge demonstrate that the proposed DSRN performs favorably against state-of-the-art algorithms in terms of both memory consumption and predictive accuracy.

Face detection is a well-explored problem. Many challenges on face detectors like extreme pose, illumination, low resolution and small scales are studied in the previous work. However, previous proposed models are mostly trained and tested on good-quality images which are not always the case for practical applications like surveillance systems. In this paper, we first review the current state-of-the-art face detectors and their performance on benchmark dataset FDDB, and compare the design protocols of the algorithms. Secondly, we investigate their performance degradation while testing on low-quality images with different levels of blur, noise, and contrast. Our results demonstrate that both hand-crafted and deep-learning based face detectors are not robust enough for low-quality images. It inspires researchers to produce more robust design for face detection in the wild.

The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process. Such an approximation has not yet been rigorously examined, and standard dictionaries often lead to non-optimal performance when used for computing thresholded features. In this paper, we first present two theoretical recovery guarantees for the thresholded feature to exactly recover the nonzero support of the sparse code. Motivated by them, we then formulate the Dictionary Learning for Thresholded Features (DLTF) model, which learns an optimized dictionary for applying the thresholded feature. In particular, for the $(k, 2)$ norm involved, a novel proximal operator with log-linear time complexity $O(m\log m)$ is derived. We evaluate the performance of DLTF on a vast range of synthetic and real-data tasks, where DLTF demonstrates remarkable efficiency, effectiveness and robustness in all experiments. In addition, we briefly discuss the potential link between DLTF and deep learning building blocks., Comment: Accepted by IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)