Search

Heterogeneous graph neural networks (HGNNs) are essential for capturing the structure and semantic information in heterogeneous graphs. However, existing GPU-based solutions, such as PyTorch Geometric, suffer from low GPU utilization due to numerous short-execution-time and memory-bound CUDA kernels during HGNN training. To address this issue, we introduce HiFuse, an enhancement for PyTorch Geometric designed to accelerate mini-batch HGNN training on CPU-GPU systems. From the data perspective, we reorganize and merge multiple smaller vertex feature matrices into larger ones, enabling a single kernel to process larger data chunks. This efficiently exploits data locality, reduces the kernel launch overhead, and improves overall GPU utilization. From the workflow perspective, we sophisticatedly offload the construction of semantic graphs from GPU to CPU to reduce the number of CUDA kernels. To meet the parallelism requirements on CPU and ensure seamless execution between CPU and GPU, we employ parallelization techniques including multi-threading and asynchronous pipeline. This allows different stages of the process to overlap, enhancing GPU utilization and reducing end-to-end execution latency, leading to a more efficient and balanced use of computational resources. Through extensive experiments, HiFuse demonstrates an average 2.38 times speedup compared to a state-of-the-art solution.

In this paper, we propose and analyze both conforming and nonconforming virtual element methods (VEMs) for the fully nonlinear second-order elliptic Hamilton-Jacobi-Bellman (HJB) equations with Cordes coefficients. By incorporating stabilization terms, we establish the well-posedness of the proposed methods, thus avoiding the need to construct a discrete Miranda-Talenti estimate. We derive the optimal error estimate in the discrete $H^2$ norm for both numerical formulations. Furthermore, a semismooth Newton's method is employed to linearize the discrete problems. Several numerical experiments using the lowest-order VEMs are provided to demonstrate the efficacy of the proposed methods and to validate our theoretical results.

CDR (Cross-Domain Recommendation), i.e., leveraging information from multiple domains, is a critical solution to data sparsity problem in recommendation system. The majority of previous research either focused on single-target CDR (STCDR) by utilizing data from the source domains to improve the model's performance on the target domain, or applied dual-target CDR (DTCDR) by integrating data from the source and target domains. In addition, multi-target CDR (MTCDR) is a generalization of DTCDR, which is able to capture the link among different domains. In this paper we present HGDR (Heterogeneous Graph-based Framework with Disentangled Representations Learning), an end-to-end heterogeneous network architecture where graph convolutional layers are applied to model relations among different domains, meanwhile utilizes the idea of disentangling representation for domain-shared and domain-specifc information. First, a shared heterogeneous graph is generated by gathering users and items from several domains without any further side information. Second, we use HGDR to compute disentangled representations for users and items in all domains.Experiments on real-world datasets and online A/B tests prove that our proposed model can transmit information among domains effectively and reach the SOTA performance.

The key of the text-to-video retrieval (TVR) task lies in learning the unique similarity between each pair of text (consisting of words) and video (consisting of audio and image frames) representations. However, some problems exist in the representation alignment of video and text, such as a text, and further each word, are of different importance for video frames. Besides, audio usually carries additional or critical information for TVR in the case that frames carry little valid information. Therefore, in TVR task, multi-granularity representation of text, including whole sentence and every word, and the modal of audio are salutary which are underutilized in most existing works. To address this, we propose a novel multi-granularity feature interaction module called MGFI, consisting of text-frame and word-frame, for video-text representations alignment. Moreover, we introduce a cross-modal feature interaction module of audio and text called CMFI to solve the problem of insufficient expression of frames in the video. Experiments on benchmark datasets such as MSR-VTT, MSVD, DiDeMo show that the proposed method outperforms the existing state-of-the-art methods.

This paper presents an innovative continuous linear finite element approach to effectively solve biharmonic problems on surfaces. The key idea behind this method lies in the strategic utilization of a surface gradient recovery operator to compute the second-order surface derivative of a piecewise continuous linear function defined on the approximate surface, as conventional notions of second-order derivatives are not directly applicable in this context. By incorporating appropriate stabilizations, we rigorously establish the stability of the proposed formulation. Despite the presence of geometric error, we provide optimal error estimates in both the energy norm and $L^2$ norm. Theoretical results are supported by numerical experiments.

Autonomous driving demands high-quality LiDAR data, yet the cost of physical LiDAR sensors presents a significant scaling-up challenge. While recent efforts have explored deep generative models to address this issue, they often consume substantial computational resources with slow generation speeds while suffering from a lack of realism. To address these limitations, we introduce RangeLDM, a novel approach for rapidly generating high-quality range-view LiDAR point clouds via latent diffusion models. We achieve this by correcting range-view data distribution for accurate projection from point clouds to range images via Hough voting, which has a critical impact on generative learning. We then compress the range images into a latent space with a variational autoencoder, and leverage a diffusion model to enhance expressivity. Additionally, we instruct the model to preserve 3D structural fidelity by devising a range-guided discriminator. Experimental results on KITTI-360 and nuScenes datasets demonstrate both the robust expressiveness and fast speed of our LiDAR point cloud generation.

Solutions exhibiting weak initial singularities arise in various equations, including diffusion and subdiffusion equations. When employing the well-known L1 scheme to solve subdiffusion equations with weak singularities, numerical simulations reveal that this scheme exhibits varying convergence rates for different choices of model parameters (i.e., domain size, final time $T$, and reaction coefficient $\kappa$). This elusive phenomenon is not unique to the L1 scheme but is also observed in other numerical methods for reaction-diffusion equations such as the backward Euler (IE) scheme, Crank-Nicolson (C-N) scheme, and two-step backward differentiation formula (BDF2) scheme. The existing literature lacks an explanation for the existence of two different convergence regimes, which has puzzled us for a long while and motivated us to study this inconsistency between the standard convergence theory and numerical experiences. In this paper, we provide a general methodology to systematically obtain error estimates that incorporate the exponential decaying feature of the solution. We term this novel error estimate the `decay-preserving error estimate' and apply it to the aforementioned IE, C-N, and BDF2 schemes. Our decay-preserving error estimate consists of a low-order term with an exponential coefficient and a high-order term with an algebraic coefficient, both of which depend on the model parameters. Our estimates reveal that the varying convergence rates are caused by a trade-off between these two components in different model parameter regimes. By considering the model parameters, we capture different states of the convergence rate that traditional error estimates fail to explain. This approach retains more properties of the continuous solution. We validate our analysis with numerical results., Comment: 25 pages, 4 figures

In the realm of numerical weather forecasting, achieving higher resolution demands increased computational resources and time investment, and leveraging deep learning networks trained solely on data significantly reduces the time expenditure during forecasting. Recently, several global forecasting artificial-intelligence-based models are developed, which are mainly trained on reanalysis dataset with a spatial resolution of approximately 25km. However, regional forecasting prefers a higher spatial resolution, and boundary information for the region also plays an important role in regional forecasting, which turns out to be a major difference from global forecasting. Here we introduce a high resolution, short-term regional weather forecasting, artificial-intelligence-based model called 'YingLong', which is capable of hourly predicting weather fields including wind speed, temperature, and specific humidity at a 3km resolution. YingLong utilizes a parallel structure of global and local blocks to capture multiscale meteorological features and is trained on analysis dataset. Additionally, the necessary information around the regional boundary is introduced to YingLong through the boundary smoothing strategy, which significantly improves the regional forecasting results. By comparing forecast results with those from WRF-ARW, one of the best numerical prediction models, YingLong demonstrates superior forecasting performances in most cases, especially on surface variables.

This paper focuses on the superconvergence analysis of the Hessian recovery technique for the $C^0$ Interior Penalty Method (C0IP) in solving the biharmonic equation. We establish interior error estimates for C0IP method that serve as the superconvergent analysis tool. Using the argument of superconvergence by difference quotient, we prove superconvergent results of the recovered Hessian matrix on translation-invariant meshes. The Hessian recovery technique enables us to construct an asymptotically exact ${\it a\, posteriori}$ error estimator for the C0IP method. Numerical experiments are provided to support our theoretical results.

The convenience of 3D sensors has led to an increase in the use of 3D point clouds in various applications. However, the differences in acquisition devices or scenarios lead to divergence in the data distribution of point clouds, which requires good generalization of point cloud representation learning methods. While most previous methods rely on domain adaptation, which involves fine-tuning pre-trained models on target domain data, this may not always be feasible in real-world scenarios where target domain data may be unavailable. To address this issue, we propose InvariantOODG, which learns invariability between point clouds with different distributions using a two-branch network to extract local-to-global features from original and augmented point clouds. Specifically, to enhance local feature learning of point clouds, we define a set of learnable anchor points that locate the most useful local regions and two types of transformations to augment the input point clouds. The experimental results demonstrate the effectiveness of the proposed model on 3D domain generalization benchmarks.

The parallel alternating direction method of multipliers (ADMM) algorithm is widely recognized for its effectiveness in handling large-scale datasets stored in a distributed manner, making it a popular choice for solving statistical learning models. However, there is currently limited research on parallel algorithms specifically designed for high-dimensional regression with combined (composite) regularization terms. These terms, such as elastic-net, sparse group lasso, sparse fused lasso, and their nonconvex variants, have gained significant attention in various fields due to their ability to incorporate prior information and promote sparsity within specific groups or fused variables. The scarcity of parallel algorithms for combined regularizations can be attributed to the inherent nonsmoothness and complexity of these terms, as well as the absence of closed-form solutions for certain proximal operators associated with them. In this paper, we propose a unified constrained optimization formulation based on the consensus problem for these types of convex and nonconvex regression problems and derive the corresponding parallel ADMM algorithms. Furthermore, we prove that the proposed algorithm not only has global convergence but also exhibits linear convergence rate. Extensive simulation experiments, along with a financial example, serve to demonstrate the reliability, stability, and scalability of our algorithm. The R package for implementing the proposed algorithms can be obtained at https://github.com/xfwu1016/CPADMM.

In many statistical modeling problems, such as classification and regression, it is common to encounter sparse and blocky coefficients. Sparse fused Lasso is specifically designed to recover these sparse and blocky structured features, especially in cases where the design matrix has ultrahigh dimensions, meaning that the number of features significantly surpasses the number of samples. Quantile loss is a well-known robust loss function that is widely used in statistical modeling. In this paper, we propose a new sparse fused lasso classification model, and develop a unified multi-block linearized alternating direction method of multipliers algorithm that effectively selects sparse and blocky features for regression and classification. Our algorithm has been proven to converge with a derived linear convergence rate. Additionally, our algorithm has a significant advantage over existing methods for solving ultrahigh dimensional sparse fused Lasso regression and classification models due to its lower time complexity. Note that the algorithm can be easily extended to solve various existing fused Lasso models. Finally, we present numerical results for several synthetic and real-world examples, which demonstrate the robustness, scalability, and accuracy of the proposed classification model and algorithm

The nonlocality of the fractional operator causes numerical difficulties for long time computation of the time-fractional evolution equations. This paper develops a high-order fast time-stepping discontinuous Galerkin finite element method for the time-fractional diffusion equations, which saves storage and computational time. The optimal error estimate $O(N^{-p-1} + h^{m+1} + \varepsilon N^{r\alpha})$ of the current time-stepping discontinuous Galerkin method is rigorous proved, where $N$ denotes the number of time intervals, $p$ is the degree of polynomial approximation on each time subinterval, $h$ is the maximum space step, $r\ge1$, $m$ is the order of finite element space, and $\varepsilon>0$ can be arbitrarily small. Numerical simulations verify the theoretical analysis., Comment: 21 pages, 1 figure,4 tables

The parallel alternating direction method of multipliers (ADMM) algorithms have gained popularity in statistics and machine learning due to their efficient handling of large sample data problems. However, the parallel structure of these algorithms, based on the consensus problem, can lead to an excessive number of auxiliary variables when applied to highdimensional data, resulting in large computational burden. In this paper, we propose a partition-insensitive parallel framework based on the linearized ADMM (LADMM) algorithm and apply it to solve nonconvex penalized high-dimensional regression problems. Compared to existing parallel ADMM algorithms, our algorithm does not rely on the consensus problem, resulting in a significant reduction in the number of variables that need to be updated at each iteration. It is worth noting that the solution of our algorithm remains largely unchanged regardless of how the total sample is divided, which is known as partition-insensitivity. Furthermore, under some mild assumptions, we prove the convergence of the iterative sequence generated by our parallel algorithm. Numerical experiments on synthetic and real datasets demonstrate the feasibility and validity of the proposed algorithm. We provide a publicly available R software package to facilitate the implementation of the proposed algorithm.

Numerous error estimates have been carried out on various numerical schemes for subdiffusion equations. Unfortunately most error bounds suffer from a factor $1/(1-\alpha)$ or $\Gamma(1-\alpha)$, which blows up as the fractional order $\alpha\to 1^-$, a phenomenon not consistent with regularity of the continuous problem and numerical simulations in practice. Although efforts have been made to avoid the factor blow-up phenomenon, a robust analysis of error estimates still remains incomplete for numerical schemes with general nonuniform time steps. In this paper, we will consider the $\alpha$-robust error analysis of convolution-type schemes for subdiffusion equations with general nonuniform time-steps, and provide explicit factors in error bounds with dependence information on $\alpha$ and temporal mesh sizes. As illustration, we apply our abstract framework to two widely used schemes, i.e., the L1 scheme and Alikhanov's scheme. Our rigorous proofs reveal that the stability and convergence of a class of convolution-type schemes is $\alpha$-robust, i.e., the factor will not blowup while $\alpha\to 1^-$ with general nonuniform time steps even when rather general initial regularity condition is considered., Comment: 21 pages

A rotation-two-component Camassa-Holm (R2CH) system was proposed recently to describe the motion of shallow water waves under the influence of gravity. This is a highly nonlinear and strongly coupled system of partial differential equations. A crucial issue in designing numerical schemes is to preserve invariants as many as possible at the discrete level. In this paper, we present a provable implicit nonlinear difference scheme which preserves at least three discrete conservation invariants: energy, mass, and momentum, and prove the existence of the difference solution via the Browder theorem. The error analysis is based on novel and refined estimates of the bilinear operator in the difference scheme. By skillfully using the energy method, we prove that the difference scheme not only converges unconditionally when the rotational parameter diminishes, but also converges without any step-ratio restriction for the small energy case when the rotational parameter is nonzero. The convergence orders in both settings (zero or nonzero rotation parameter) are $O(\tau^2 + h^2)$ for the velocity in the $L^\infty$-norm and the surface elevation in the $L^2$-norm, where $\tau$ denotes the temporal stepsize and $h$ the spatial stepsize, respectively. The theoretical predictions are confirmed by a properly designed two-level iteration scheme. Comparing with existing numerical methods in the literature, the proposed method demonstrates its effectiveness for long-time simulation over larger domains and superior resolution for both smooth and non-smooth initial values.

The proposal of Six Degrees of Separation makes people realize that the world is not as big as we imagined. Even if the world's population now exceeds 7 billion, two strangers can still get in touch through a limited intermediary. When online social networks have taken the world by storm, can people connect through shorter distances? This issue is worth thinking about. This paper describes Six Degrees of Separation and the limitations of this theory in practical applications. Combined with online social networks, the paper analyzes the development and change of the degrees of connection between people. Finally, the paper considers the actual coverage of online social networks and other issues, and rationalizes the possible degrees of the future Six Degrees of Separation., Comment: 7 pages

We introduce and analyze a robust nonconforming finite element method for a three dimensional singularly perturbed quad-curl model problem. For the solution of the model problem, we derive proper a priori bounds, based on which, we prove that the proposed finite element method is robust with respect to the singular perturbation parameter $\varepsilon$ and the numerical solution is uniformly convergent with order $h^{1/2}$. In addition, we investigate the effect of treating the second boundary condition weakly by Nitsche's method. We show that such a treatment leads to sharper error estimates than imposing the boundary condition strongly when the parameter $\varepsilon< h$. Finally, numerical experiments are provided to illustrate the good performance of the method and confirm our theoretical predictions.

We propose and analyze a new family of nonconforming finite elements for the three-dimensional quad-curl problem. The proposed finite element spaces are subspaces of $\pmb{H}(\mathrm{curl})$, but not of $\pmb{H}(\mathrm{grad}~\mathrm{curl})$, which are different from the existing nonconforming ones. The well-posedness of the discrete problem is proved and optimal error estimates in discrete $\pmb{H}(\mathrm{grad}~\mathrm{curl})$ norm, $\pmb{H}(\mathrm{curl})$ norm and $\pmb{L}^2$ norm are derived. Numerical experiments are provided to illustrate the good performance of the method and confirm our theoretical predictions.

We propose two families of nonconforming elements on cubical meshes: one for the $-\text{curl}\Delta\text{curl}$ problem and the other for the Brinkman problem. The element for the $-\text{curl}\Delta\text{curl}$ problem is the first nonconforming element on cubical meshes. The element for the Brinkman problem can yield a uniformly stable finite element method with respect to the parameter $\nu$. The lowest-order elements for the $-\text{curl}\Delta\text{curl}$ and the Brinkman problems have 48 and 30 degrees of freedom, respectively. The two families of elements are subspaces of $H(\text{curl};\Omega)$ and $H(\text{div};\Omega)$, and they, as nonconforming approximation to $H(\text{gradcurl};\Omega)$ and $[H^1(\Omega)]^3$, can form a discrete Stokes complex together with the Lagrange element and the $L^2$ element.

Using newly developed ${\bf H}(\mathrm{curl}^2)$ conforming elements, we solve the Maxwell's transmission eigenvalue problem. Both real and complex eigenvalues are considered. Based on the fixed-point weak formulation with reasonable assumptions, the optimal error estimates for numerical eigenvalues and eigenfunctions (in the ${\bf H}(\mathrm{curl}^2)$-norm and ${\bf H}(\mathrm{curl})$-semi-norm) are established.

An hp-version interior penalty discontinuous Galerkin (IPDG) method under nonconforming meshes is proposed to solve the quad-curl eigenvalue problem. We prove well-posedness of the numerical scheme for the quad-curl equation and then derive an error estimate in a mesh-dependent norm, which is optimal with respect to h but has different p-version error bounds under conforming and nonconforming tetrahedron meshes. The hp-version discrete compactness of the DG space is established for the convergence proof. The performance of the method is demonstrated by numerical experiments using conforming/nonconforming meshes and h-version/p-version refinement. The optimal h-version convergence rate and the exponential p-version convergence rate are observed.

In the past few years, there has been a dramatic growth in e-manga (electronic Japanese-style comics). Faced with the booming demand for manga research and the large amount of unlabeled manga data, we raised a new task, called unsupervised manga character re-identification. However, the artistic expression and stylistic limitations of manga pose many challenges to the re-identification problem. Inspired by the idea that some content-related features may help clustering, we propose a Face-body and Spatial-temporal Associated Clustering method (FSAC). In the face-body combination module, a face-body graph is constructed to solve problems such as exaggeration and deformation in artistic creation by using the integrity of the image. In the spatial-temporal relationship correction module, we analyze the appearance features of characters and design a temporal-spatial-related triplet loss to fine-tune the clustering. Extensive experiments on a manga book dataset with 109 volumes validate the superiority of our method in unsupervised manga character re-identification.

Anderson localization is a famous wave phenomenon that describes the absence of diffusion of waves in a disordered medium. Here we generalize the landscape theory of Anderson localization to general elliptic operators and complex boundary conditions using a probabilistic approach, and further investigate some mathematical aspects of Anderson localization that are rarely discussed before. First, we observe that under the Neumann boundary condition, the low energy quantum states are localized on the boundary of the domain with high probability. We provide a detailed explanation of this phenomenon using the concept of extended subregions and obtain an analytical expression of this probability in the one-dimensional case. Second, we find that the quantum states may be localized in multiple different subregions with high probability in the one-dimensional case and we derive an explicit expression of this probability for various boundary conditions. Finally, we examine a bifurcation phenomenon of the localization subregion as the strength of disorder varies. The critical threshold of bifurcation is analytically computed based on a toy model and the dependence of the critical threshold on model parameters is analyzed., Comment: 31 pages, 7 figures

In this work, we analyze the three-step backward differentiation formula (BDF3) method for solving the Allen-Cahn equation on variable grids. For BDF2 method, the discrete orthogonal convolution (DOC) kernels are positive, the stability and convergence analysis are well established in [Liao and Zhang, \newblock Math. Comp., \textbf{90} (2021) 1207--1226; Chen, Yu, and Zhang, \newblock SIAM J. Numer. Anal., Major Revised]. However, the numerical analysis for BDF3 method with variable steps seems to be highly nontrivial, since the DOC kernels are not always positive. By developing a novel spectral norm inequality, the unconditional stability and convergence are rigorously proved under the updated step ratio restriction $r_k:=\tau_k/\tau_{k-1}\leq 1.405$ (compared with $r_k\leq 1.199$ in [Calvo and Grigorieff, \newblock BIT. \textbf{42} (2002) 689--701]) for BDF3 method. Finally, numerical experiments are performed to illustrate the theoretical results. To the best of our knowledge, this is the first theoretical analysis of variable steps BDF3 method for the Allen-Cahn equation.

We investigate numerical solutions of high order curl problems with various formulations and finite elements. We show that several classical conforming finite elements lead to spurious solutions, while mixed formulations with finite elements in complexes solve the problems correctly. To explain the numerical results, we clarify the cohomological structures in high order curl problems by relating the partial differential equations to the Hodge-Laplacian boundary problems of the gradcurl-complexes., Comment: 23 pages, 7 figures

Context: Stack Overflow is very helpful for software developers who are seeking answers to programming problems. Previous studies have shown that a growing number of questions are of low quality and thus obtain less attention from potential answerers. Gao et al. proposed an LSTM-based model (i.e., BiLSTM-CC) to automatically generate question titles from the code snippets to improve the question quality. However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens. Objective: This paper proposes CCBERT, a deep learning based novel model to enhance the performance of question title generation by making full use of the bi-modal information of the entire question body. Method: CCBERT follows the encoder-decoder paradigm and uses CodeBERT to encode the question body into hidden representations, a stacked Transformer decoder to generate predicted tokens, and an additional copy attention layer to refine the output distribution. Both the encoder and decoder perform the multi-head self-attention operation to better capture the long-range dependencies. This paper builds a dataset containing around 200,000 high-quality questions filtered from the data officially published by Stack Overflow to verify the effectiveness of the CCBERT model. Results: CCBERT outperforms all the baseline models on the dataset. Experiments on both code-only and low-resource datasets show the superiority of CCBERT with less performance degradation. The human evaluation also shows the excellent performance of CCBERT concerning both readability and correlation criteria., Comment: This is the accepted version by Information and Software Technology

Hierarchical Text Classification (HTC), which aims to predict text labels organized in hierarchical space, is a significant task lacking in investigation in natural language processing. Existing methods usually encode the entire hierarchical structure and fail to construct a robust label-dependent model, making it hard to make accurate predictions on sparse lower-level labels and achieving low Macro-F1. In this paper, we propose a novel PAMM-HiA-T5 model for HTC: a hierarchy-aware T5 model with path-adaptive mask mechanism that not only builds the knowledge of upper-level labels into low-level ones but also introduces path dependency information in label prediction. Specifically, we generate a multi-level sequential label structure to exploit hierarchical dependency across different levels with Breadth-First Search (BFS) and T5 model. To further improve label dependency prediction within each path, we then propose an original path-adaptive mask mechanism (PAMM) to identify the label's path information, eliminating sources of noises from other paths. Comprehensive experiments on three benchmark datasets show that our novel PAMM-HiA-T5 model greatly outperforms all state-of-the-art HTC approaches especially in Macro-F1. The ablation studies show that the improvements mainly come from our innovative approach instead of T5.

Vibrations of structures subjected to concentrated point loads have many applications in mechanical engineering. Experiments are expensive and numerical methods are often used for simulations. In this paper, we consider the plate vibration with nonlinear dependence on the eigen-parameter. The problem is formulated as the eigenvalue problem of a holomorphic Fredholm operator function. The Bogner-Fox-Schmit element is used for the discretization and the spectral indicator method is employed to compute the eigenvalues. The convergence is proved using the abstract approximation theory of Karma. Numerical examples are presented for validations.