Search

The decay of sunspot plays a key role in magnetic flux transportation in solar active regions (ARs). To better understand the physical mechanism of the entire decay process of a sunspot, an {\alpha}-configuration sunspot in AR NOAA 12411 was studied. Based on the continuum intensity images and vector magnetic field data with stray light correction from Solar Dynamics Observatory/Helioseismic and Magnetic Imager, the area, vector magnetic field and magnetic flux in the umbra and penumbra are calculated with time, respectively. Our main results are as follows: (1) The decay curves of the sunspot area in its umbra, penumbra, and whole sunspot take the appearance of Gaussian profiles. The area decay rates of the umbra, penumbra and whole sunspot are -1.56 MSH/day, -12.61 MSH/day and -14.04 MSH/day, respectively; (2) With the decay of the sunspot, the total magnetic field strength and the vertical component of the penumbra increase, and the magnetic field of the penumbra becomes more vertical. Meanwhile, the total magnetic field strength and vertical magnetic field strength for the umbra decrease, and the inclination angle changes slightly with an average value of about 20{\deg}; (3) The magnetic flux decay curves of the sunspot in its umbra, penumbra, and whole sunspot exhibit quadratic patterns, their magnetic flux decay rates of the umbra, penumbra and whole sunspot are -9.84 * 10^19 Mx/day, -1.59 * 10^20 Mx/day and -2.60 * 10^20 Mx/day , respectively. The observation suggests that the penumbra may be transformed into the umbra, resulting in the increase of the average vertical magnetic field strength and the reduction of the inclination angle in the penumbra during the decay of the sunspot.

Scene text image super-resolution has significantly improved the accuracy of scene text recognition. However, many existing methods emphasize performance over efficiency and ignore the practical need for lightweight solutions in deployment scenarios. Faced with the issues, our work proposes an efficient framework called SGENet to facilitate deployment on resource-limited platforms. SGENet contains two branches: super-resolution branch and semantic guidance branch. We apply a lightweight pre-trained recognizer as a semantic extractor to enhance the understanding of text information. Meanwhile, we design the visual-semantic alignment module to achieve bidirectional alignment between image features and semantics, resulting in the generation of highquality prior guidance. We conduct extensive experiments on benchmark dataset, and the proposed SGENet achieves excellent performance with fewer computational costs. Code is available at https://github.com/SijieLiu518/SGENet

Image composition involves seamlessly integrating given objects into a specific visual context. The current training-free methods rely on composing attention weights from several samplers to guide the generator. However, since these weights are derived from disparate contexts, their combination leads to coherence confusion in synthesis and loss of appearance information. These issues worsen with their excessive focus on background generation, even when unnecessary in this task. This not only slows down inference but also compromises foreground generation quality. Moreover, these methods introduce unwanted artifacts in the transition area. In this paper, we formulate image composition as a subject-based local editing task, solely focusing on foreground generation. At each step, the edited foreground is combined with the noisy background to maintain scene consistency. To address the remaining issues, we propose PrimeComposer, a faster training-free diffuser that composites the images by well-designed attention steering across different noise levels. This steering is predominantly achieved by our Correlation Diffuser, utilizing its self-attention layers at each step. Within these layers, the synthesized subject interacts with both the referenced object and background, capturing intricate details and coherent relationships. This prior information is encoded into the attention weights, which are then integrated into the self-attention layers of the generator to guide the synthesis process. Besides, we introduce a Region-constrained Cross-Attention to confine the impact of specific subject-related words to desired regions, addressing the unwanted artifacts shown in the prior method thereby further improving the coherence in the transition area. Our method exhibits the fastest inference efficiency and extensive experiments demonstrate our superiority both qualitatively and quantitatively.

Out-of-distribution detection (OOD) is a crucial technique for deploying machine learning models in the real world to handle the unseen scenarios. In this paper, we first propose a simple yet effective Neural Activation Prior (NAP) for OOD detection. Our neural activation prior is based on a key observation that, for a channel before the global pooling layer of a fully trained neural network, the probability of a few neurons being activated with a large response by an in-distribution (ID) sample is significantly higher than that by an OOD sample. An intuitive explanation is that for a model fully trained on ID dataset, each channel would play a role in detecting a certain pattern in the ID dataset, and a few neurons can be activated with a large response when the pattern is detected in an input sample. Then, a new scoring function based on this prior is proposed to highlight the role of these strongly activated neurons in OOD detection. Our approach is plug-and-play and does not lead to any performance degradation on ID data classification and requires no extra training or statistics from training or external datasets. Notice that previous methods primarily rely on post-global-pooling features of the neural networks, while the within-channel distribution information we leverage would be discarded by the global pooling operator. Consequently, our method is orthogonal to existing approaches and can be effectively combined with them in various applications. Experimental results show that our method achieves the state-of-the-art performance on CIFAR benchmark and ImageNet dataset, which demonstrates the power of the proposed prior. Finally, we extend our method to Transformers and the experimental findings indicate that NAP can also significantly enhance the performance of OOD detection on Transformers, thereby demonstrating the broad applicability of this prior knowledge.

Out-of-distribution (OOD) detection plays a crucial role in ensuring the security of neural networks. Existing works have leveraged the fact that In-distribution (ID) samples form a subspace in the feature space, achieving state-of-the-art (SOTA) performance. However, the comprehensive characteristics of the ID subspace still leave under-explored. Recently, the discovery of Neural Collapse ($\mathcal{NC}$) sheds light on novel properties of the ID subspace. Leveraging insight from $\mathcal{NC}$, we observe that the Principal Angle between the features and the ID feature subspace forms a superior representation for measuring the likelihood of OOD. Building upon this observation, we propose a novel $\mathcal{NC}$-inspired OOD scoring function, named Entropy-enhanced Principal Angle (EPA), which integrates both the global characteristic of the ID subspace and its inner property. We experimentally compare EPA with various SOTA approaches, validating its superior performance and robustness across different network architectures and OOD datasets., Comment: Accepted by ICASSP 2024

Scene text synthesis involves rendering specified texts onto arbitrary images. Current methods typically formulate this task in an end-to-end manner but lack effective character-level guidance during training. Besides, their text encoders, pre-trained on a single font type, struggle to adapt to the diverse font styles encountered in practical applications. Consequently, these methods suffer from character distortion, repetition, and absence, particularly in polystylistic scenarios. To this end, this paper proposes DreamText for high-fidelity scene text synthesis. Our key idea is to reconstruct the diffusion training process, introducing more refined guidance tailored to this task, to expose and rectify the model's attention at the character level and strengthen its learning of text regions. This transformation poses a hybrid optimization challenge, involving both discrete and continuous variables. To effectively tackle this challenge, we employ a heuristic alternate optimization strategy. Meanwhile, we jointly train the text encoder and generator to comprehensively learn and utilize the diverse font present in the training dataset. This joint training is seamlessly integrated into the alternate optimization process, fostering a synergistic relationship between learning character embedding and re-estimating character attention. Specifically, in each step, we first encode potential character-generated position information from cross-attention maps into latent character masks. These masks are then utilized to update the representation of specific characters in the current step, which, in turn, enables the generator to correct the character's attention in the subsequent steps. Both qualitative and quantitative results demonstrate the superiority of our method to the state of the art.

Inherent ambiguity in layout annotations poses significant challenges to developing accurate 360{\deg} room layout estimation models. To address this issue, we propose a novel Bi-Layout model capable of predicting two distinct layout types. One stops at ambiguous regions, while the other extends to encompass all visible areas. Our model employs two global context embeddings, where each embedding is designed to capture specific contextual information for each layout type. With our novel feature guidance module, the image feature retrieves relevant context from these embeddings, generating layout-aware features for precise bi-layout predictions. A unique property of our Bi-Layout model is its ability to inherently detect ambiguous regions by comparing the two predictions. To circumvent the need for manual correction of ambiguous annotations during testing, we also introduce a new metric for disambiguating ground truth layouts. Our method demonstrates superior performance on benchmark datasets, notably outperforming leading approaches. Specifically, on the MatterportLayout dataset, it improves 3DIoU from 81.70% to 82.57% across the full test set and notably from 54.80% to 59.97% in subsets with significant ambiguity. Project page: https://liagm.github.io/Bi_Layout, Comment: CVPR 2024, Project page: https://liagm.github.io/Bi_Layout

Cross-modal knowledge transfer enhances point cloud representation learning in LiDAR semantic segmentation. Despite its potential, the \textit{weak teacher challenge} arises due to repetitive and non-diverse car camera images and sparse, inaccurate ground truth labels. To address this, we propose the Efficient Image-to-LiDAR Knowledge Transfer (ELiTe) paradigm. ELiTe introduces Patch-to-Point Multi-Stage Knowledge Distillation, transferring comprehensive knowledge from the Vision Foundation Model (VFM), extensively trained on diverse open-world images. This enables effective knowledge transfer to a lightweight student model across modalities. ELiTe employs Parameter-Efficient Fine-Tuning to strengthen the VFM teacher and expedite large-scale model training with minimal costs. Additionally, we introduce the Segment Anything Model based Pseudo-Label Generation approach to enhance low-quality image labels, facilitating robust semantic representations. Efficient knowledge transfer in ELiTe yields state-of-the-art results on the SemanticKITTI benchmark, outperforming real-time inference models. Our approach achieves this with significantly fewer parameters, confirming its effectiveness and efficiency., Comment: 9 pages, 6 figures, ICME 2024 oral

This paper presents a robust fine-tuning method designed for pre-trained 3D point cloud models, to enhance feature robustness in downstream fine-tuned models. We highlight the limitations of current fine-tuning methods and the challenges of learning robust models. The proposed method, named Weight-Space Ensembles for Fine-Tuning then Linear Probing (WiSE-FT-LP), integrates the original pre-training and fine-tuning models through weight space integration followed by Linear Probing. This approach significantly enhances the performance of downstream fine-tuned models under distribution shifts, improving feature robustness while maintaining high performance on the target distribution. We apply this robust fine-tuning method to mainstream 3D point cloud pre-trained models and evaluate the quality of model parameters and the degradation of downstream task performance. Experimental results demonstrate the effectiveness of WiSE-FT-LP in enhancing model robustness, effectively balancing downstream task performance and model feature robustness without altering the model structures., Comment: 9 pages, 5 figures

The naringin extraction process was optimised using response surface methodology (RSM). A central component design was adopted, which included four parameters: extraction temperature (X1), material-liquid ratio (X2), extraction time (X3), and ultrasonic frequency (X4) of 74.79 °C, 1.58 h, 1:56.51 g/mL, and 28.05 KHz, respectively. Based on these optimal extraction conditions, naringin was tested to verify the model's accuracy. Naringin yield was 36.2502 mg/g, which was equivalent to the predicted yield of 36.0124 mg/g. DM101 macroporous adsorption resin was used to purify naringin. The effects of loading concentration, loading flow rate, and sample pH on the adsorption rate of naringin and the effect of ethanol concentration on the desorption rate of naringin were investigated. The optimum conditions for naringin purification using macroporous resins were determined. The optimal loading concentration, sample solution pH, and loading flow rate were 0.075 mg/mL, 3.5, and 1.5 mL/min, respectively. Three parallel tests were conducted under these conditions, and the average naringin yield was 77.5643%. Naringin's structure was identified using infrared spectroscopy and nuclear magnetic resonance. In vitro determination of the lipid-lowering activity of naringin was also conducted. These results showed that naringin has potential applications as a functional food for lowering blood lipid levels.

Zeolitic imidazolate frameworks (ZIFs) are acknowledged as one of the most perfect precursors to synthesize metal–nitrogen/carbon materials in H2-air and Zn-air fuel cells to boost their cathodic sluggish oxygen reactions. However, abundant atomically dispersed metal–nitrogen sites are useless because of being buried in the derived carbon nanoparticles. Here, a molten-salt-assisted pyrolysis strategy is proposed to prepare two-dimensional porous metal–nitrogen/carbon materials with an attempt to expose metal–nitrogen sites, which involves thermal intercalation/exfoliation and space-confined pyrolysis of layer-structured ZIFs from the molten-salt mixture of KCl/LiCl. Specifically, atomically dispersed Co–N/C porous nanosheets were prepared, showing a high special surface area of 1100 m2 g–1 to host abundant exposed Co–Nx sites. When the Co–N/C electrocatalyst was assembled into a liquid-state Zn-air fuel cell, its peak power density and specific capacity reached up to 239.7 mW cm–2 and 784.9 mAh g–1, respectively, and excellent discharge–charge cycling stability was demonstrated. Furthermore, a Co–N/C-based solid-state Zn-air fuel cell was fabricated, also showing outstanding cell performance, such as a peak power density of 147.2 mW cm–2. This work provides a feasible method to synthesize porous two-dimensional functional carbon materials for electrocatalysis and energy devices.

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example-based chart retrieval methods are built upon non-decoupled and low-level visual features that are hard to interpret, while definition-based ones are constrained to pre-defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What-You-Think-Is-What-You-Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language-image pre-training (CLIP) model to achieve zero-shot classification as well as multi-modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.

We present XFormer, a novel human mesh and motion capture method that achieves real-time performance on consumer CPUs given only monocular images as input. The proposed network architecture contains two branches: a keypoint branch that estimates 3D human mesh vertices given 2D keypoints, and an image branch that makes predictions directly from the RGB image features. At the core of our method is a cross-modal transformer block that allows information to flow across these two branches by modeling the attention between 2D keypoint coordinates and image spatial features. Our architecture is smartly designed, which enables us to train on various types of datasets including images with 2D/3D annotations, images with 3D pseudo labels, and motion capture datasets that do not have associated images. This effectively improves the accuracy and generalization ability of our system. Built on a lightweight backbone (MobileNetV3), our method runs blazing fast (over 30fps on a single CPU core) and still yields competitive accuracy. Furthermore, with an HRNet backbone, XFormer delivers state-of-the-art performance on Huamn3.6 and 3DPW datasets. © 2023 International Joint Conferences on Artificial Intelligence. All rights reserved.

This research explores the influence of renewable fuels, including three kinds of biodiesel along with ethanol on the physical properties and structural characteristics of particulate matter (PM) emitted from a diesel engine in comparison with pure diesel. After adding 10 vol% of grape seed biodiesel, coffee biodiesel and eucalyptus oil into diesel, three biodiesel blended fuels (10% grape seed biodiesel (DGs10), 10% spent coffee ground biodiesel (DC10) and eucalyptus oil biodiesel (DEu10)) were produced and tested in this study. Besides, one ethanol blend containing 9 vol% of ethanol and 1 vol% of biodiesel (blend stabilizer) was also tested to do the comparison. In the present study, scanning transmission electron microscope (STEM) and scanning electron microscope (SEM) were employed for analyzing the microstructure, nanostructure and electron diffraction pattern of PM. Raman spectrometer (RS) was also used for the analysis of structural characterization of PM. In addition, several experimental instruments like microbalance, measuring cup, viscometer, oxygen bomb calorimeter and Gas Chromatography-Mass Spectrometer (GC-MS) were employed to detect the fuel properties, including density, heating value, viscosity, composition and cetane number. A conclusion can be drawn that both biodiesel blends and ethanol blend have a changing effect on the PM properties compared to pure diesel, where biodiesel blends have a slightly weaker influence than ethanol blend. Regarding the biodiesel blends, DGs10 has more impact than DC10 and DEu10 in changes of PM properties, particularly in the reduction of PM mass, making it a good candidate for renewable fuel for diesel engines.

Transformer is beneficial for image denoising tasks since it can model long-range dependencies to overcome the limitations presented by inductive convolutional biases. However, directly applying the transformer structure to remove noise is challenging because its complexity grows quadratically with the spatial resolution. In this paper, we propose an efficient Dual-branch Deformable Transformer (DDT) denoising network which captures both local and global interactions in parallel. We divide features with a fixed patch size and a fixed number of patches in local and global branches, respectively. In addition, we apply deformable attention operation in both branches, which helps the network focus on more important regions and further reduces computational complexity. We conduct extensive experiments on real-world and synthetic denoising tasks, and the proposed DDT achieves state-of-the-art performance with significantly fewer computational costs., Comment: The code is avaliable at: https://github.com/Merenguelkl/DDT

Deep learning has seen rapid growth in recent years and achieved state-of-the-art performance in a wide range of applications. However, training models typically requires expensive and time-consuming collection of large quantities of labeled data. This is particularly true within the scope of medical imaging analysis (MIA), where data are limited and labels are expensive to be acquired. Thus, label-efficient deep learning methods are developed to make comprehensive use of the labeled data as well as the abundance of unlabeled and weak-labeled data. In this survey, we extensively investigated over 300 recent papers to provide a comprehensive overview of recent progress on label-efficient learning strategies in MIA. We first present the background of label-efficient learning and categorize the approaches into different schemes. Next, we examine the current state-of-the-art methods in detail through each scheme. Specifically, we provide an in-depth investigation, covering not only canonical semi-supervised, self-supervised, and multi-instance learning schemes, but also recently emerged active and annotation-efficient learning strategies. Moreover, as a comprehensive contribution to the field, this survey not only elucidates the commonalities and unique features of the surveyed methods but also presents a detailed analysis of the current challenges in the field and suggests potential avenues for future research., Comment: Update Few-shot Methods

Ideal part editing should guarantee the diversity of edited parts, the fidelity to the remaining parts, and the quality of the results. However, previous methods do not disentangle each part completely, which means the edited parts will affect the others, resulting in poor diversity and fidelity. In addition, some methods lack constraints between parts, which need manual selections of edited results to ensure quality. Therefore, we propose a four-stage process for point cloud part editing: Segmentation, Generation, Assembly, and Selection. Based on this process, we introduce SGAS, a model for part editing that employs two strategies: feature disentanglement and constraint. By independently fitting part-level feature distributions, we realize the feature disentanglement. By explicitly modeling the transformation from object-level distribution to part-level distributions, we realize the feature constraint. Considerable experiments on different datasets demonstrate the efficiency and effectiveness of SGAS on point cloud part editing. In addition, SGAS can be pruned to realize unsupervised part-aware point cloud generation and achieves state-of-the-art results., Comment: 9 pages, 7 figures, AAAI 2024

In computational pathology, whole slide image (WSI) classification presents a formidable challenge due to its gigapixel resolution and limited fine-grained annotations. Multiple instance learning (MIL) offers a weakly supervised solution, yet refining instance-level information from bag-level labels remains complex. While most of the conventional MIL methods use attention scores to estimate instance importance scores (IIS) which contribute to the prediction of the slide labels, these often lead to skewed attention distributions and inaccuracies in identifying crucial instances. To address these issues, we propose a new approach inspired by cooperative game theory: employing Shapley values to assess each instance's contribution, thereby improving IIS estimation. The computation of the Shapley value is then accelerated using attention, meanwhile retaining the enhanced instance identification and prioritization. We further introduce a framework for the progressive assignment of pseudo bags based on estimated IIS, encouraging more balanced attention distributions in MIL models. Our extensive experiments on CAMELYON-16, BRACS, and TCGA-LUNG datasets show our method's superiority over existing state-of-the-art approaches, offering enhanced interpretability and class-wise insights. We will release the code upon acceptance., Comment: submitted to IEEE TRANSACTIONS ON MEDICAL IMAGING

With the increasing prevalence of smartphones and websites, Image Aesthetic Assessment (IAA) has become increasingly crucial. While the significance of attributes in IAA is widely recognized, many attribute-based methods lack consideration for the selection and utilization of aesthetic attributes. Our initial step involves the acquisition of aesthetic attributes from both intra- and inter-perspectives. Within the intra-perspective, we extract the direct visual attributes of images, constituting the absolute attribute. In the inter-perspective, our focus lies in modeling the relative score relationships between images within the same sequence, forming the relative attribute. Then, to better utilize image attributes in aesthetic assessment, we propose the Unified Multi-attribute Aesthetic Assessment Framework (UMAAF) to model both absolute and relative attributes of images. For absolute attributes, we leverage multiple absolute-attribute perception modules and an absolute-attribute interacting network. The absolute-attribute perception modules are first pre-trained on several absolute-attribute learning tasks and then used to extract corresponding absolute attribute features. The absolute-attribute interacting network adaptively learns the weight of diverse absolute-attribute features, effectively integrating them with generic aesthetic features from various absolute-attribute perspectives and generating the aesthetic prediction. To model the relative attribute of images, we consider the relative ranking and relative distance relationships between images in a Relative-Relation Loss function, which boosts the robustness of the UMAAF. Furthermore, UMAAF achieves state-of-the-art performance on TAD66K and AVA datasets, and multiple experiments demonstrate the effectiveness of each module and the model's alignment with human preference.

Layout-to-image synthesis is an emerging technique in conditional image generation. It aims to generate complex scenes, where users require fine control over the layout of the objects in a scene. However, it remains challenging to control the object coherence, including semantic coherence (e.g., the cat looks at the flowers or not) and physical coherence (e.g., the hand and the racket should not be misaligned). In this paper, we propose a novel diffusion model with effective global semantic fusion (GSF) and self-similarity feature enhancement modules to guide the object coherence for this task. For semantic coherence, we argue that the image caption contains rich information for defining the semantic relationship within the objects in the images. Instead of simply employing cross-attention between captions and generated images, which addresses the highly relevant layout restriction and semantic coherence separately and thus leads to unsatisfying results shown in our experiments, we develop GSF to fuse the supervision from the layout restriction and semantic coherence requirement and exploit it to guide the image synthesis process. Moreover, to improve the physical coherence, we develop a Self-similarity Coherence Attention (SCA) module to explicitly integrate local contextual physical coherence into each pixel's generation process. Specifically, we adopt a self-similarity map to encode the coherence restrictions and employ it to extract coherent features from text embedding. Through visualization of our self-similarity map, we explore the essence of SCA, revealing that its effectiveness is not only in capturing reliable physical coherence patterns but also in enhancing complex texture generation. Extensive experiments demonstrate the superiority of our proposed method in both image generation quality and controllability., Comment: GitHub: https://github.com/CodeGoat24/EOCNet

Current subject-driven image generation methods encounter significant challenges in person-centric image generation. The reason is that they learn the semantic scene and person generation by fine-tuning a common pre-trained diffusion, which involves an irreconcilable training imbalance. Precisely, to generate realistic persons, they need to sufficiently tune the pre-trained model, which inevitably causes the model to forget the rich semantic scene prior and makes scene generation over-fit to the training data. Moreover, even with sufficient fine-tuning, these methods can still not generate high-fidelity persons since joint learning of the scene and person generation also lead to quality compromise. In this paper, we propose Face-diffuser, an effective collaborative generation pipeline to eliminate the above training imbalance and quality compromise. Specifically, we first develop two specialized pre-trained diffusion models, i.e., Text-driven Diffusion Model (TDM) and Subject-augmented Diffusion Model (SDM), for scene and person generation, respectively. The sampling process is divided into three sequential stages, i.e., semantic scene construction, subject-scene fusion, and subject enhancement. The first and last stages are performed by TDM and SDM respectively. The subject-scene fusion stage, that is the collaboration achieved through a novel and highly effective mechanism, Saliency-adaptive Noise Fusion (SNF). Specifically, it is based on our key observation that there exists a robust link between classifier-free guidance responses and the saliency of generated images. In each time step, SNF leverages the unique strengths of each model and allows for the spatial blending of predicted noises from both models automatically in a saliency-aware manner. Extensive experiments confirm the impressive effectiveness and robustness of the Face-diffuser., Comment: Accepted by CVPR2024. Code: https://github.com/CodeGoat24/Face-diffuser

Visually-situated languages such as charts and plots are omnipresent in real-world documents. These graphical depictions are human-readable and are often analyzed in visually-rich documents to address a variety of questions that necessitate complex reasoning and common-sense responses. Despite the growing number of datasets that aim to answer questions over charts, most only address this task in isolation, without considering the broader context of document-level question answering. Moreover, such datasets lack adequate common-sense reasoning information in their questions. In this work, we introduce a novel task named document-level chart question answering (DCQA). The goal of this task is to conduct document-level question answering, extracting charts or plots in the document via document layout analysis (DLA) first and subsequently performing chart question answering (CQA). The newly developed benchmark dataset comprises 50,010 synthetic documents integrating charts in a wide range of styles (6 styles in contrast to 3 for PlotQA and ChartQA) and includes 699,051 questions that demand a high degree of reasoning ability and common-sense understanding. Besides, we present the development of a potent question-answer generation engine that employs table data, a rich color set, and basic question templates to produce a vast array of reasoning question-answer pairs automatically. Based on DCQA, we devise an OCR-free transformer for document-level chart-oriented understanding, capable of DLA and answering complex reasoning and common-sense questions over charts in an OCR-free manner. Our DCQA dataset is expected to foster research on understanding visualizations in documents, especially for scenarios that require complex reasoning for charts in the visually-rich document. We implement and evaluate a set of baselines, and our proposed method achieves comparable results.

Pre-trained multimodal models have achieved significant success in retrieval-based question answering. However, current multimodal retrieval question-answering models face two main challenges. Firstly, utilizing compressed evidence features as input to the model results in the loss of fine-grained information within the evidence. Secondly, a gap exists between the feature extraction of evidence and the question, which hinders the model from effectively extracting critical features from the evidence based on the given question. We propose a two-stage framework for evidence retrieval and question-answering to alleviate these issues. First and foremost, we propose a progressive evidence refinement strategy for selecting crucial evidence. This strategy employs an iterative evidence retrieval approach to uncover the logical sequence among the evidence pieces. It incorporates two rounds of filtering to optimize the solution space, thus further ensuring temporal efficiency. Subsequently, we introduce a semi-supervised contrastive learning training strategy based on negative samples to expand the scope of the question domain, allowing for a more thorough exploration of latent knowledge within known samples. Finally, in order to mitigate the loss of fine-grained information, we devise a multi-turn retrieval and question-answering strategy to handle multimodal inputs. This strategy involves incorporating multimodal evidence directly into the model as part of the historical dialogue and question. Meanwhile, we leverage a cross-modal attention mechanism to capture the underlying connections between the evidence and the question, and the answer is generated through a decoding generation approach. We validate the model's effectiveness through extensive experiments, achieving outstanding performance on WebQA and MultimodelQA benchmark tests.

Event Relation Extraction (ERE) aims to extract multiple kinds of relations among events in texts. However, existing methods singly categorize event relations as different classes, which are inadequately capturing the intrinsic semantics of these relations. To comprehensively understand their intrinsic semantics, in this paper, we obtain prototype representations for each type of event relation and propose a Prototype-Enhanced Matching (ProtoEM) framework for the joint extraction of multiple kinds of event relations. Specifically, ProtoEM extracts event relations in a two-step manner, i.e., prototype representing and prototype matching. In the first step, to capture the connotations of different event relations, ProtoEM utilizes examples to represent the prototypes corresponding to these relations. Subsequently, to capture the interdependence among event relations, it constructs a dependency graph for the prototypes corresponding to these relations and utilized a Graph Neural Network (GNN)-based module for modeling. In the second step, it obtains the representations of new event pairs and calculates their similarity with those prototypes obtained in the first step to evaluate which types of event relations they belong to. Experimental results on the MAVEN-ERE dataset demonstrate that the proposed ProtoEM framework can effectively represent the prototypes of event relations and further obtain a significant improvement over baseline models., Comment: Work in progress

This paper considers a type of incremental aggregated gradient (IAG) method for large-scale distributed optimization. The IAG method is well suited for the parameter server architecture as the latter can easily aggregate potentially staled gradients contributed by workers. Although the convergence of IAG in the case of deterministic gradient is well known, there are only a few results for the case of its stochastic variant based on streaming data. Considering strongly convex optimization, this paper shows that the streaming IAG method achieves linear speedup when the workers are updating frequently enough, even if the data sample distribution across workers are heterogeneous. We show that the expected squared distance to optimal solution decays at O((1+T)/(nt)), where $n$ is the number of workers, t is the iteration number, and T/n is the update frequency of workers. Our analysis involves careful treatments of the conditional expectations with staled gradients and a recursive system with both delayed and noise terms, which are new to the analysis of IAG-type algorithms. Numerical results are presented to verify our findings., Comment: 8 pages, 3 figures

Accurate depth estimation under out-of-distribution (OoD) scenarios, such as adverse weather conditions, sensor failure, and noise contamination, is desirable for safety-critical applications. Existing depth estimation systems, however, suffer inevitably from real-world corruptions and perturbations and are struggled to provide reliable depth predictions under such cases. In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation. This challenge was developed based on the newly established KITTI-C and NYUDepth2-C benchmarks. We hosted two stand-alone tracks, with an emphasis on robust self-supervised and robust fully-supervised depth estimation, respectively. Out of more than two hundred participants, nine unique and top-performing solutions have appeared, with novel designs ranging from the following aspects: spatial- and frequency-domain augmentations, masked image modeling, image restoration and super-resolution, adversarial training, diffusion-based noise suppression, vision-language pre-training, learned model ensembling, and hierarchical feature enhancement. Extensive experimental analyses along with insightful observations are drawn to better understand the rationale behind each design. We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation and beyond. The datasets, competition toolkit, workshop recordings, and source code from the winning teams are publicly available on the challenge website., Comment: Technical Report; 65 pages, 34 figures, 24 tables; Code at https://github.com/ldkong1205/RoboDepth

Polymer syntheses in a high throughput format are still challenging due to the tedious procedures for prior deoxygenation and catalyst removal. 2D metal-organic framework (MOF) nanosheets are advantageous for elevating the catalytic efficiency and catalyst recyclability. Polymerization of a wide variety of monomers, including hydrophilic acrylamides and hydrophobic acrylates, is attempted directly in a multi-well plate by employing Zn-ZnPPF-2D nanosheets (PPF = porphyrin paddlewheel framework) as a heterogeneous photocatalyst. Various parameters such as monomer concentration, catalyst concentration, and light wavelength are investigated with respect to their effects on polymerization rate and the degree of control over the molecular weight and molecular weight distribution. Due to the larger surface area and more accessible catalytic sites, the top-performing Zn-ZnPPF-2D exhibits fast polymerization kinetics over the Zn-ZnPPF-3D bulk crystals. In addition, the synthesis of triblock copolymers with a single loading of catalysts confirms the outstanding catalytic performance of these 2D MOF catalysts. Finally, photopolymerization is demonstrated to be achievable entirely in a microliter-scale human cell culture medium. As such, this strategy provides high levels of control and precision over macromolecular synthesis outcomes that best align with the requirements of high throughput approaches toward biological applications. © 2022 Wiley-VCH GmbH.

The literature shows that information about the physical, chemical, and cell toxicity properties of particulate matter (PM) from diesel vehicles is not rich as the existence of a remarkable number of studies about the combustion, performance, and emissions of diesel vehicles using renewable liquid fuels, particularly biodiesels and alcohols. Also, the PM analyses from combustion of spent coffee ground biodiesel have not been comprehensively explored. Therefore, this research is presented. Pure diesel, 90% diesel + 10% biodiesel, and 90% diesel + 9% ethanol + 1% biodiesel, volume bases, were tested under a fast idle condition. STEM, SEM, EDS, Organic Carbon Analyzer, TGA/DSC, and Raman Spectrometer were employed for investigating the PM physical and chemical properties, and assays of cell viability, cellular reactive oxygen species, interleukin-6, and tumor necrosis factor-alpha were examined for investigating the PM cell toxicity properties. It is found that the application of both biodiesel and ethanol has the potential to change the PM properties, while the impact of ethanol is more than biodiesel on the changes. Regarding the important aspects, biodiesel can be effective for better human health (due to a decrease in cell death (−60.8%)) as well as good diesel particulate filter efficiency (due to lower activation energy (−7.6%) and frequency factor (−83.2%)). However, despite a higher impact of ethanol on the reductions in activation energy (−24.8%) and frequency factor (−99.0%), this fuel causes an increase in cell death (84.1%). Therefore, biodiesel can be an appropriate fuel to have a positive impact on human health, the environment, and emissions catalysts performance, simultaneously. © 2022 Elsevier B.V.

Identification of catalytic active sites is pivotal in the design of highly effective heterogeneous metal catalysts, especially for structure-sensitive reactions. Downsizing the dimension of the metal species on the catalyst increases the dispersion, which is maximized when the metal exists as single atoms, namely, single-atom catalysts (SACs). SACs have been reported to be efficient for various catalytic reactions. We show here that the Pt SACs, although with the highest metal atom utilization efficiency, are totally inactive in the cyclohexane (C6H12) dehydrogenation reaction, an important reaction that could enable efficient hydrogen transportation. Instead, catalysts enriched with fully exposed few-atom Pt ensembles, with a Pt–Pt coordination number of around 2, achieve the optimal catalytic performance. The superior performance of a fully exposed few-atom ensemble catalyst is attributed to its high d-band center, multiple neighboring metal sites, and weak binding of the product. © 2022 American Chemical Society

The present study investigates the dynamic and asymmetric impacts of eco-innovation and human capital development on ambient pollution by validating the Environment Kuznets Curve (EKC) hypothesis in China from 1988Q1 to 2018Q4. The findings confirm non-normality and structural breaks in data. Thus, Quantile Autoregressive Distributive Lag (QARDL) model and Granger Causality-in-Quantiles are applied to address non-linearity and structural breaks. The longrun results exhibit that eco-innovation and human capital have a significant negative relationship with carbon emissions, mainly from lower (0.05) to medium (0.5) quantiles and medium (0.50) to higher (0.95) emissions quantile. Moreover, economic growth contributes to higher emissions across all quantiles. In contrast, the square of economic growth has a significant negative association with emissions, confirming the validity of EKC from medium (0.40) to higher (0.95) quantiles. Lastly, Granger causality confirms a two-way causality between eco-innovation, human capital, and carbon emissions, and a one-way causality from human capital, economic growth to carbon emissions. These findings offer valuable policy recommendations.

Japan has 312,214 international students in May 2019, successfully achieving the 300,000 international students target. This study investigates the factors pulling international students into Japan for higher education. Data for literature review was collected from mainly Japanese sources including research reports, review papers, statistics, and online materials, which addressing the issues of international students’ mobility into Japan from 2008 until present. Data was reviewed qualitatively by comparative analysis, comparing the literature with the elements of 300,000 international students plan and the pull factors from the push-pull model. The results showed specific measures in strengthening campus internationalization, expanding international cooperation and alumni network, improving supporting services, diversifying scholarship schemes, job hunting support etc. could be considered as effective pulling factors. Moreover, while these factors could be applied effectively by some universities, it is hard for other universities to follow the same approach. The results imply obstacles to the mid-ranking public universities to attract more international students and the need to improve this situation.

Since 2020, university students in Japan are exposed to the impacts of Covid-19 pandemic. Data from a survey had shown that over 50% of international students had been experiencing some kind of stress (Tran, unpublished). For supporting international students in the prolonged pandemic period, we conducted a series of Stress Prevention Seminars during 2021. These seminars were designed for students to learn simple skills that could help them to cope with various stresses. Records of the seminars, including students’ interactions, Q&A, feedback, emotion score (Ogawa, 2000) and final evaluation were analyzed. This presentation is aiming to show: (1) preliminary results of the seminars conducted; and (2) potential of these events for strengthening mental health status of international students during pandemic. The results showed high understanding level and satisfaction rate, as well as inreased positive emotion score. The results suggested that providing these stress prevention seminars to international students is potentially to be an effective strategy, in addition to providing individual counseling and other activities, to improving stress resilience of international students for long term.

Japanese universities are popular destinations for international students. A lot of efforts have been made to provide supporting services and activities to international students such as Japanese classes, counseling and consultation, job hunting and career education, exchange events, summer school, culture experience tours etc. These activities bring satisfaction to international students, which is contributed indirectly as a pulling factor for attracting international students into Japan. However, little has been known about the approaches used by different universities for supporting international students as not many reports are published in English regarding this topic. This study investigates the variety of on campus supporting services and activities provided to international students in Japanese universities and satisfaction of international students as recipients of these services. The literature for review was mainly collected from Japanese sources using Google Scholar. Data was reviewed qualitatively by comparative analysis, comparing approaches applied by different universities with regards to the elements of 300,000 international students plan and other pull factors. The results showed specific approaches, results in providing campus supporting services by different institutions. The results also imply obstacles and potential solutions to the Japanese universities to attract international students.

Japan is an attractive study abroad destination for international students worldwide. However, there are still challenges in attracting international students and internationalization, especially for the universities not enlisted among the top-tier universities, and universities which locate outside metropolitan areas. Despite relative advantages of quality of education, affordable cost, student-friendly environment, and exotic region-specific culture, Japanese universities still face certain challenges in attracting international students. We conducted a literature review to investigate challenges that may hinder attracting international students to Japan's universities. The data showed that there are challenges in maintaining academic ranking and internationalization level, diversifying the student body, securing revenues for operation, regional popularity and disparity, strengthening the supporting system, maintaining cross-cultural exchange and communication, mobilizing scholarship, supporting employment and a part-time job, online transformation, etc. that may influence the efforts for effectively attracting international students.

The adoption of green technology is imperative to realise sustainable development. Considering the same, this study explores the drivers of Green Innovation (GI) based on the theoretical foundation of the Triple Bottom Line (environmental, social, and economic factors) with the integration of information and communication technologies (ICT) and institutional governance (INST) in Pakistan. This study employs a nonlinear autoregressive distributed lag (NARDL) framework on quarterly data from Q1-1996 to Q4- 2019. The results reveal that positive shocks in human capital (HCI) instigate GI by 1.05%, while negative shock undermines GI by 0.93%. Similarly, positive shocks in carbon dioxide (CO2) emissions increase GI by 0.63%, while any negative shock undermines GI by 0.01%. On the other hand, positive shock in ICT leads to 0.55% advanced GI; however, this effect turned stronger in negative shocks, which leads to reduced GI by 0.78% in the long-run. These results confirm the asymmetricity because positive and negative shocks in HCI, CO2 emissions, and ICT instigated GI differently. Finally, INST and GDP contribute to enhancing GI by 0.12% and 1.69%, respectively. The results indicate that the Pakistan government should improve institutional governance, adapt, and focus on sustainable practices with ICT integration to promote green technologies.

This paper explores the multi-scale aggregation strategy for scene text detection in natural images. We present the Aggregated Text TRansformer(ATTR), which is designed to represent texts in scene images with a multi-scale self-attention mechanism. Starting from the image pyramid with multiple resolutions, the features are first extracted at different scales with shared weight and then fed into an encoder-decoder architecture of Transformer. The multi-scale image representations are robust and contain rich information on text contents of various sizes. The text Transformer aggregates these features to learn the interaction across different scales and improve text representation. The proposed method detects scene texts by representing each text instance as an individual binary mask, which is tolerant of curve texts and regions with dense instances. Extensive experiments on public scene text detection datasets demonstrate the effectiveness of the proposed framework.

Intelligent fault diagnosis is essential to safe operation of machinery. However, due to scarce fault samples and data heterogeneity in field machinery, deep learning based diagnosis methods are prone to over-fitting with poor generalization ability. To solve the problem, this paper proposes a personalized federated learning framework, enabling multi-task fault diagnosis method across multiple factories in a privacypreserving manner. Firstly, rotating machines from different factories with similar vibration feature data are categorized into machine groups using a federated clustering method. Then, a multi-task deep learning model based on convolutional neural network is constructed to diagnose the multiple faults of machinery with heterogeneous information fusion. Finally, a personalized federated learning framework is proposed to solve data heterogeneity across different machines using adaptive hierarchical aggregation strategy. The case study on collected data from real machines verifies the effectiveness of the proposed framework. The result shows that the diagnosis accuracy could be improved significantly using the proposed personalized federated learning, especially for those machines with scarce fault samples.

Scene text erasing seeks to erase text contents from scene images and current state-of-the-art text erasing models are trained on large-scale synthetic data. Although data synthetic engines can provide vast amounts of annotated training samples, there are differences between synthetic and real-world data. In this paper, we employ self-supervision for feature representation on unlabeled real-world scene text images. A novel pretext task is designed to keep consistent among text stroke masks of image variants. We design the Progressive Erasing Network in order to remove residual texts. The scene text is erased progressively by leveraging the intermediate generated results which provide the foundation for subsequent higher quality results. Experiments show that our method significantly improves the generalization of the text erasing task and achieves state-of-the-art performance on public benchmarks.

Spiking Neural Networks (SNNs) have piqued researchers' interest because of their capacity to process temporal information and low power consumption. However, current state-of-the-art methods limited their biological plausibility and performance because their neurons are generally built on the simple Leaky-Integrate-and-Fire (LIF) model. Due to the high level of dynamic complexity, modern neuron models have seldom been implemented in SNN practice. In this study, we adopt the Phase Plane Analysis (PPA) technique, a technique often utilized in neurodynamics field, to integrate a recent neuron model, namely, the Izhikevich neuron. Based on the findings in the advancement of neuroscience, the Izhikevich neuron model can be biologically plausible while maintaining comparable computational cost with LIF neurons. By utilizing the adopted PPA, we have accomplished putting neurons built with the modified Izhikevich model into SNN practice, dubbed as the Standardized Izhikevich Tonic (SIT) neuron. For performance, we evaluate the suggested technique for image classification tasks in self-built LIF-and-SIT-consisted SNNs, named Hybrid Neural Network (HNN) on static MNIST, Fashion-MNIST, CIFAR-10 datasets and neuromorphic N-MNIST, CIFAR10-DVS, and DVS128 Gesture datasets. The experimental results indicate that the suggested method achieves comparable accuracy while exhibiting more biologically realistic behaviors on nearly all test datasets, demonstrating the efficiency of this novel strategy in bridging the gap between neurodynamics and SNN practice.

Given partial objects and some complete ones as references, point cloud completion aims to recover authentic shapes. However, existing methods pay little attention to general shapes, which leads to the poor authenticity of completion results. Besides, the missing patterns are diverse in reality, but existing methods can only handle fixed ones, which means a poor generalization ability. Considering that a partial point cloud is a subset of the corresponding complete one, we regard them as different samples of the same distribution and propose Structure Retrieval based Point Completion Network (SRPCN). It first uses k-means clustering to extract structure points and disperses them into distributions, and then KL Divergence is used as a metric to find the complete structure point cloud that best matches the input in a database. Finally, a PCN-like decoder network is adopted to generate the final results based on the retrieved structure point clouds. As structure plays an important role in describing the general shape of an object and the proposed structure retrieval method is robust to missing patterns, experiments show that our method can generate more authentic results and has a stronger generalization ability., Comment: I think the proposed method has some defects

As a major approach to looking for life beyond the Earth, the search for extraterrestrial intelligence (SETI) is committed to detecting technosignatures such as engineered radio signals that are indicative of technologically capable life. In this paper, we report a targeted SETI campaign employing an observation strategy named multi-beam coincidence matching (MBCM) at the Five-hundred-meter Aperture Spherical radio Telescope (FAST) towards 33 known exoplanet systems, searching for ETI narrow-band drifting signals across 1.05-1.45 GHz in two orthogonal linear polarization directions separately. A signal at 1140.604 MHz detected from the observation towards Kepler-438 originally peaked our interest because its features are roughly consistent with assumed ETI technosignatures. However, evidences such as its polarization characteristics are almost able to eliminate the possibility of an extraterrestrial origin. Our observations achieve an unprecedented sensitivity since the minimum equivalent isotropic radiated power (EIRP) we are able to detect reaches 1.48 x10^9 W., Comment: 17 pages, 12 figures, matches the published version

The great interest in rechargeable Zn–air batteries (ZABs) arouses extensive research on low-cost, high-active, and durable bifunctional electrocatalysts to boost the sluggish oxygen reduction reaction (ORR) and oxygen evolution reaction (OER). It remains a great challenge to simultaneously host high-active and independent ORR and OER sites in a single catalyst. Herein a dual-phasic carbon nanoarchitecture consisting of a single-atom phase for the ORR and nanosized phase for the OER is proposed. Specifically, single Co atoms supported on carbon nanotubes (single-atom phase) and nanosized Co encapsulated in zeolitic-imidazole-framework-derived carbon polyhedron (nanosized phase) are integrated together via carbon nanotube bridges. The obtained dual-phasic carbon catalyst shows a small overpotential difference of 0.74 V between OER potential at 10 mA cm−2 and ORR half-wave potential. The ZAB based on the bifunctional catalyst demonstrates a large power density of 172 mW cm−2. Furthermore, it shows a small charge-discharge potential gap of 0.51 V at 5 mA cm−2 and outstanding discharge-charge cycling durability. This study provides a feasible design concept to achieve multifunctional catalysts and promotes the development of rechargeable ZABs. © 2021 Wiley-VCH GmbH

Rationale: Coronavirus disease 2019 (COVID-19) has caused a global pandemic. A classifier combining chest X-ray (CXR) with clinical features may serve as a rapid screening approach. Methods: The study included 512 patients with COVID-19 and 106 with influenza A/B pneumonia. A deep neural network (DNN) was applied, and deep features derived from CXR and clinical findings formed fused features for diagnosis prediction. Results: The clinical features of COVID-19 and influenza showed different patterns. Patients with COVID-19 experienced less fever, more diarrhea, and more salient hypercoagulability. Classifiers constructed using the clinical features or CXR had an area under the receiver operating curve (AUC) of 0.909 and 0.919, respectively. The diagnostic efficacy of the classifier combining the clinical features and CXR was dramatically improved and the AUC was 0.952 with 91.5% sensitivity and 81.2% specificity. Moreover, combined classifier was functional in both severe and non-serve COVID-19, with an AUC of 0.971 with 96.9% sensitivity in non-severe cases, which was on par with the computed tomography (CT)-based classifier, but had relatively inferior efficacy in severe cases compared to CT. In extension, we performed a reader study involving three experienced pulmonary physicians, artificial intelligence (AI) system demonstrated superiority in turn-around time and diagnostic accuracy compared with experienced pulmonary physicians. Conclusions: The classifier constructed using clinical and CXR features is efficient, economical, and radiation safe for distinguishing COVID-19 from influenza A/B pneumonia, serving as an ideal rapid screening tool during the COVID-19 pandemic.

With constant increasing in numbers of students studying abroad, Vietnam is on the way to become one of the largest contributors to the world’s international student mobility. The overall picture was changed from several thousand students sent to the Eastern bloc during the cold war to about 200,000 students currently studying in Japan, Australia, US, Europe etc. This paper reviews the factors that affect the transition of trends and mobility of Vietnamese international students in the modern time. By examining related studies and literature, we demonstrated that transition of studying abroad in Vietnam is not subjectively resulted by decision of the students themselves or of their families, but it is strongly influenced objectively by external determinants including cultural, social, political, foreign relation, economic, educational factors in the country. The results found five macro factors that influence the transition trends of Vietnamese international students over time, including number of students, study destinations, career after graduation and impact on society.

As details are missing in most representations of structures, the lack of controllability to more information is one of the major weaknesses in structure-based controllable point cloud generation. It is observable that definitions of details and structures are subjective. Details can be treated as structures on small scales. To represent structures in different scales at the same time, we present a graph-based representation of structures called the Multiscale Structure Graph (MSG). Given structures in multiple scales, similar patterns of local structures can be found at different scales, positions, and angles. The knowledge learned from a regional structure pattern shall be transferred to other similar patterns. An encoding and generation mechanism, namely the Multiscale Structure-based Point Cloud Generator (MSPCG) is proposed, which can simultaneously learn point cloud generation from local patterns with miscellaneous spatial properties. The proposed method supports multiscale editions on point clouds by editing the MSG. By generating point clouds from local structures and learning simultaneously in multiple scales, our MSPCG has better generalization ability and scalability. Trained on the ShapeNet, our MSPCG can generate point clouds from a given structure for unseen categories and indoor scenes. The experimental results show that our method significantly outperforms baseline methods., Comment: 9 pages, 6 figures

In point cloud generation and completion, previous methods for transforming latent features to point clouds are generally based on fully connected layers (FC-based) or folding operations (Folding-based). However, point clouds generated by FC-based methods are usually troubled by outliers and rough surfaces. For folding-based methods, their data flow is large, convergence speed is slow, and they are also hard to handle the generation of non-smooth surfaces. In this work, we propose AXform, an attention-based method to transform latent features to point clouds. AXform first generates points in an interim space, using a fully connected layer. These interim points are then aggregated to generate the target point cloud. AXform takes both parameter sharing and data flow into account, which makes it has fewer outliers, fewer network parameters, and a faster convergence speed. The points generated by AXform do not have the strong 2-manifold constraint, which improves the generation of non-smooth surfaces. When AXform is expanded to multiple branches for local generations, the centripetal constraint makes it has properties of self-clustering and space consistency, which further enables unsupervised semantic segmentation. We also adopt this scheme and design AXformNet for point cloud completion. Considerable experiments on different datasets show that our methods achieve state-of-the-art results., Comment: 9 pages, 7 figures, AAAI 2022

Knowledge distillation (KD) has recently emerged as a powerful strategy to transfer knowledge from a pre-trained teacher model to a lightweight student, and has demonstrated its unprecedented success over a wide spectrum of applications. In spite of the encouraging results, the KD process per se poses a potential threat to network ownership protection, since the knowledge contained in network can be effortlessly distilled and hence exposed to a malicious user. In this paper, we propose a novel framework, termed as Safe Distillation Box (SDB), that allows us to wrap a pre-trained model in a virtual box for intellectual property protection. Specifically, SDB preserves the inference capability of the wrapped model to all users, but precludes KD from unauthorized users. For authorized users, on the other hand, SDB carries out a knowledge augmentation scheme to strengthen the KD performances and the results of the student model. In other words, all users may employ a model in SDB for inference, but only authorized users get access to KD from the model. The proposed SDB imposes no constraints over the model architecture, and may readily serve as a plug-and-play solution to protect the ownership of a pre-trained network. Experiments across various datasets and architectures demonstrate that, with SDB, the performance of an unauthorized KD drops significantly while that of an authorized gets enhanced, demonstrating the effectiveness of SDB., Comment: Accepted by AAAI2022

Document layout analysis (DLA) plays an important role in information extraction and document understanding. At present, document layout analysis has reached a milestone achievement, however, document layout analysis of non-Manhattan is still a challenge. In this paper, we propose an image layer modeling method to tackle this challenge. To measure the proposed image layer modeling method, we propose a manually-labeled non-Manhattan layout fine-grained segmentation dataset named FPD. As far as we know, FPD is the first manually-labeled non-Manhattan layout fine-grained segmentation dataset. To effectively extract fine-grained features of documents, we propose an edge embedding network named L-E^3Net. Experimental results prove that our proposed image layer modeling method can better deal with the fine-grained segmented document of the non-Manhattan layout.