Search

In the field of high-dimensional data analysis, modeling methods based on quantile loss function are highly regarded due to their ability to provide a comprehensive statistical perspective and effective handling of heterogeneous data. In recent years, many studies have focused on using the parallel alternating direction method of multipliers (P-ADMM) to solve high-dimensional quantile regression and classification problems. One efficient strategy is to reformulate the quantile loss function by introducing slack variables. However, this reformulation introduces a theoretical challenge: even when the regularization term is convex, the convergence of the algorithm cannot be guaranteed. To address this challenge, this paper proposes the Gaussian Back-Substitution strategy, which requires only a simple and effective correction step that can be easily integrated into existing parallel algorithm frameworks, achieving a linear convergence rate. Furthermore, this paper extends the parallel algorithm to handle some novel quantile loss classification models. Numerical simulations demonstrate that the proposed modified P-ADMM algorithm exhibits excellent performance in terms of reliability and efficiency.

Post-processing techniques are essential tools for enhancing the accuracy of finite element approximations and achieving superconvergence. Among these, recovery techniques stand out as vital methods, playing significant roles in both post-processing and pre-processing. This paper provides an overview of recent developments in recovery techniques and their applications in adaptive computations. The discussion encompasses both gradient recovery and Hessian recovery methods. To establish the superconvergence properties of these techniques, two theoretical frameworks are introduced. Applications of these methods are demonstrated in constructing asymptotically exact {\it a posteriori} error estimators for second-order elliptic equations, fourth-order elliptic equations, and interface problems. Numerical experiments are performed to evaluate the asymptotic exactness of recovery type a posteriori error estimators., Comment: To appear in Advances in Applied Mechanics

The Fredholm-Hammerstein integral equations (FHIEs) with weakly singular kernels exhibit multi-point singularity at the endpoints or boundaries. The dense discretized matrices result in high computational complexity when employing numerical methods. To address this, we propose a novel class of mapped Hermite functions, which are constructed by applying a mapping to Hermite polynomials.We establish fundamental approximation theory for the orthogonal functions. We propose MHFs-spectral collocation method and MHFs-smoothing transformation method to solve the two-point weakly singular FHIEs, respectively. Error analysis and numerical results demonstrate that our methods, based on the new orthogonal functions, are particularly effective for handling problems with weak singularities at two endpoints, yielding exponential convergence rate. We position this work as the first to directly study the mapped spectral method for multi-point singularity problems, to the best of our knowledge.

Monge-Amp\`{e}re equation is a prototype second-order fully nonlinear partial differential equation. In this paper, we propose a new idea to design and analyze the $C^0$ interior penalty method to approximation the viscosity solution of the Monge-Amp\`{e}re equation. The new methods is inspired from the discrete Miranda-Talenti estimate. Based on the vanishing moment representation, we approximate the Monge-Amp\`{e}re equation by the fourth order semi-linear equation with some additional boundary conditions. We will use the discrete Miranda-Talenti estimates to ensure the well-posedness of the numerical scheme and derive the error estimates.

Heterogeneous graph neural networks (HGNNs) are essential for capturing the structure and semantic information in heterogeneous graphs. However, existing GPU-based solutions, such as PyTorch Geometric, suffer from low GPU utilization due to numerous short-execution-time and memory-bound CUDA kernels during HGNN training. To address this issue, we introduce HiFuse, an enhancement for PyTorch Geometric designed to accelerate mini-batch HGNN training on CPU-GPU systems. From the data perspective, we reorganize and merge multiple smaller vertex feature matrices into larger ones, enabling a single kernel to process larger data chunks. This efficiently exploits data locality, reduces the kernel launch overhead, and improves overall GPU utilization. From the workflow perspective, we sophisticatedly offload the construction of semantic graphs from GPU to CPU to reduce the number of CUDA kernels. To meet the parallelism requirements on CPU and ensure seamless execution between CPU and GPU, we employ parallelization techniques including multi-threading and asynchronous pipeline. This allows different stages of the process to overlap, enhancing GPU utilization and reducing end-to-end execution latency, leading to a more efficient and balanced use of computational resources. Through extensive experiments, HiFuse demonstrates an average 2.38 times speedup compared to a state-of-the-art solution.

In this paper, we propose and analyze both conforming and nonconforming virtual element methods (VEMs) for the fully nonlinear second-order elliptic Hamilton-Jacobi-Bellman (HJB) equations with Cordes coefficients. By incorporating stabilization terms, we establish the well-posedness of the proposed methods, thus avoiding the need to construct a discrete Miranda-Talenti estimate. We derive the optimal error estimate in the discrete $H^2$ norm for both numerical formulations. Furthermore, a semismooth Newton's method is employed to linearize the discrete problems. Several numerical experiments using the lowest-order VEMs are provided to demonstrate the efficacy of the proposed methods and to validate our theoretical results.

CDR (Cross-Domain Recommendation), i.e., leveraging information from multiple domains, is a critical solution to data sparsity problem in recommendation system. The majority of previous research either focused on single-target CDR (STCDR) by utilizing data from the source domains to improve the model's performance on the target domain, or applied dual-target CDR (DTCDR) by integrating data from the source and target domains. In addition, multi-target CDR (MTCDR) is a generalization of DTCDR, which is able to capture the link among different domains. In this paper we present HGDR (Heterogeneous Graph-based Framework with Disentangled Representations Learning), an end-to-end heterogeneous network architecture where graph convolutional layers are applied to model relations among different domains, meanwhile utilizes the idea of disentangling representation for domain-shared and domain-specifc information. First, a shared heterogeneous graph is generated by gathering users and items from several domains without any further side information. Second, we use HGDR to compute disentangled representations for users and items in all domains. Experiments on real-world datasets and online A/B tests prove that our proposed model can transmit information among domains effectively and reach the SOTA performance. The code can be found here: https://github.com/NetEase-Media/HGCDR.

The key of the text-to-video retrieval (TVR) task lies in learning the unique similarity between each pair of text (consisting of words) and video (consisting of audio and image frames) representations. However, some problems exist in the representation alignment of video and text, such as a text, and further each word, are of different importance for video frames. Besides, audio usually carries additional or critical information for TVR in the case that frames carry little valid information. Therefore, in TVR task, multi-granularity representation of text, including whole sentence and every word, and the modal of audio are salutary which are underutilized in most existing works. To address this, we propose a novel multi-granularity feature interaction module called MGFI, consisting of text-frame and word-frame, for video-text representations alignment. Moreover, we introduce a cross-modal feature interaction module of audio and text called CMFI to solve the problem of insufficient expression of frames in the video. Experiments on benchmark datasets such as MSR-VTT, MSVD, DiDeMo show that the proposed method outperforms the existing state-of-the-art methods.

This paper presents an innovative continuous linear finite element approach to effectively solve biharmonic problems on surfaces. The key idea behind this method lies in the strategic utilization of a surface gradient recovery operator to compute the second-order surface derivative of a piecewise continuous linear function defined on the approximate surface, as conventional notions of second-order derivatives are not directly applicable in this context. By incorporating appropriate stabilizations, we rigorously establish the stability of the proposed formulation. Despite the presence of geometric error, we provide optimal error estimates in both the energy norm and $L^2$ norm. Theoretical results are supported by numerical experiments.

Autonomous driving demands high-quality LiDAR data, yet the cost of physical LiDAR sensors presents a significant scaling-up challenge. While recent efforts have explored deep generative models to address this issue, they often consume substantial computational resources with slow generation speeds while suffering from a lack of realism. To address these limitations, we introduce RangeLDM, a novel approach for rapidly generating high-quality range-view LiDAR point clouds via latent diffusion models. We achieve this by correcting range-view data distribution for accurate projection from point clouds to range images via Hough voting, which has a critical impact on generative learning. We then compress the range images into a latent space with a variational autoencoder, and leverage a diffusion model to enhance expressivity. Additionally, we instruct the model to preserve 3D structural fidelity by devising a range-guided discriminator. Experimental results on KITTI-360 and nuScenes datasets demonstrate both the robust expressiveness and fast speed of our LiDAR point cloud generation.

Solutions exhibiting weak initial singularities arise in various equations, including diffusion and subdiffusion equations. When employing the well-known L1 scheme to solve subdiffusion equations with weak singularities, numerical simulations reveal that this scheme exhibits varying convergence rates for different choices of model parameters (i.e., domain size, final time $T$, and reaction coefficient $\kappa$). This elusive phenomenon is not unique to the L1 scheme but is also observed in other numerical methods for reaction-diffusion equations such as the backward Euler (IE) scheme, Crank-Nicolson (C-N) scheme, and two-step backward differentiation formula (BDF2) scheme. The existing literature lacks an explanation for the existence of two different convergence regimes, which has puzzled us for a long while and motivated us to study this inconsistency between the standard convergence theory and numerical experiences. In this paper, we provide a general methodology to systematically obtain error estimates that incorporate the exponential decaying feature of the solution. We term this novel error estimate the `decay-preserving error estimate' and apply it to the aforementioned IE, C-N, and BDF2 schemes. Our decay-preserving error estimate consists of a low-order term with an exponential coefficient and a high-order term with an algebraic coefficient, both of which depend on the model parameters. Our estimates reveal that the varying convergence rates are caused by a trade-off between these two components in different model parameter regimes. By considering the model parameters, we capture different states of the convergence rate that traditional error estimates fail to explain. This approach retains more properties of the continuous solution. We validate our analysis with numerical results., Comment: 25 pages, 4 figures

Recently, artificial intelligence-based (AI-based) models for forecasting of global weather have been rapidly developed. Most of the global models are trained on reanalysis datasets with a spatial resolution of 0.25{\deg}*0.25{\deg}. However, research on AI-based high spatial resolution limited area weather forecasting models remains limited. In this study, YingLong, an AI-based limited area weather forecasting model with a spatial resolution of 3 km * 3 km is developed. YingLong employs a parallel structure of global and local blocks to capture multiscale meteorological features and operates much faster than the dynamical limited area model WRF-ARW. In two selected limited areas (one relatively flat and the other featuring significant mountain ranges), YingLong (with lateral boundary condition imposed by the global AI-based model Pangu-weather) demonstrates superior skill in forecasting surface wind speed compared to WRF-ARW. Additionally, it shows comparable skill in forecasting surface temperature and pressure. The accuracy of surface temperature and humidity forecasts can be further improved by applying better boundary conditions. YingLong also addresses issues related to the lateral boundary conditions of AI-based limited area models, such as selecting the width of the lateral boundary region and combining finer and coarser resolution predictions in this region. Therefore, YingLong has a great potential to generate cost-effective multiyear high-resolution synthetic wind speed that maintain meteorological realism both spatially and temporally, aiding in the planning and operations for wind power generation companies.

This paper focuses on the superconvergence analysis of the Hessian recovery technique for the $C^0$ Interior Penalty Method (C0IP) in solving the biharmonic equation. We establish interior error estimates for C0IP method that serve as the superconvergent analysis tool. Using the argument of superconvergence by difference quotient, we prove superconvergent results of the recovered Hessian matrix on translation-invariant meshes. The Hessian recovery technique enables us to construct an asymptotically exact ${\it a\, posteriori}$ error estimator for the C0IP method. Numerical experiments are provided to support our theoretical results.

The convenience of 3D sensors has led to an increase in the use of 3D point clouds in various applications. However, the differences in acquisition devices or scenarios lead to divergence in the data distribution of point clouds, which requires good generalization of point cloud representation learning methods. While most previous methods rely on domain adaptation, which involves fine-tuning pre-trained models on target domain data, this may not always be feasible in real-world scenarios where target domain data may be unavailable. To address this issue, we propose InvariantOODG, which learns invariability between point clouds with different distributions using a two-branch network to extract local-to-global features from original and augmented point clouds. Specifically, to enhance local feature learning of point clouds, we define a set of learnable anchor points that locate the most useful local regions and two types of transformations to augment the input point clouds. The experimental results demonstrate the effectiveness of the proposed model on 3D domain generalization benchmarks.

The parallel alternating direction method of multipliers (ADMM) algorithm is widely recognized for its effectiveness in handling large-scale datasets stored in a distributed manner, making it a popular choice for solving statistical learning models. However, there is currently limited research on parallel algorithms specifically designed for high-dimensional regression with combined (composite) regularization terms. These terms, such as elastic-net, sparse group lasso, sparse fused lasso, and their nonconvex variants, have gained significant attention in various fields due to their ability to incorporate prior information and promote sparsity within specific groups or fused variables. The scarcity of parallel algorithms for combined regularizations can be attributed to the inherent nonsmoothness and complexity of these terms, as well as the absence of closed-form solutions for certain proximal operators associated with them. In this paper, we propose a unified constrained optimization formulation based on the consensus problem for these types of convex and nonconvex regression problems and derive the corresponding parallel ADMM algorithms. Furthermore, we prove that the proposed algorithm not only has global convergence but also exhibits linear convergence rate. Extensive simulation experiments, along with a financial example, serve to demonstrate the reliability, stability, and scalability of our algorithm. The R package for implementing the proposed algorithms can be obtained at https://github.com/xfwu1016/CPADMM.

In many statistical modeling problems, such as classification and regression, it is common to encounter sparse and blocky coefficients. Sparse fused Lasso is specifically designed to recover these sparse and blocky structured features, especially in cases where the design matrix has ultrahigh dimensions, meaning that the number of features significantly surpasses the number of samples. Quantile loss is a well-known robust loss function that is widely used in statistical modeling. In this paper, we propose a new sparse fused lasso classification model, and develop a unified multi-block linearized alternating direction method of multipliers algorithm that effectively selects sparse and blocky features for regression and classification. Our algorithm has been proven to converge with a derived linear convergence rate. Additionally, our algorithm has a significant advantage over existing methods for solving ultrahigh dimensional sparse fused Lasso regression and classification models due to its lower time complexity. Note that the algorithm can be easily extended to solve various existing fused Lasso models. Finally, we present numerical results for several synthetic and real-world examples, which demonstrate the robustness, scalability, and accuracy of the proposed classification model and algorithm

The nonlocality of the fractional operator causes numerical difficulties for long time computation of the time-fractional evolution equations. This paper develops a high-order fast time-stepping discontinuous Galerkin finite element method for the time-fractional diffusion equations, which saves storage and computational time. The optimal error estimate $O(N^{-p-1} + h^{m+1} + \varepsilon N^{r\alpha})$ of the current time-stepping discontinuous Galerkin method is rigorous proved, where $N$ denotes the number of time intervals, $p$ is the degree of polynomial approximation on each time subinterval, $h$ is the maximum space step, $r\ge1$, $m$ is the order of finite element space, and $\varepsilon>0$ can be arbitrarily small. Numerical simulations verify the theoretical analysis., Comment: 21 pages, 1 figure,4 tables

The parallel alternating direction method of multipliers (ADMM) algorithms have gained popularity in statistics and machine learning due to their efficient handling of large sample data problems. However, the parallel structure of these algorithms, based on the consensus problem, can lead to an excessive number of auxiliary variables when applied to highdimensional data, resulting in large computational burden. In this paper, we propose a partition-insensitive parallel framework based on the linearized ADMM (LADMM) algorithm and apply it to solve nonconvex penalized high-dimensional regression problems. Compared to existing parallel ADMM algorithms, our algorithm does not rely on the consensus problem, resulting in a significant reduction in the number of variables that need to be updated at each iteration. It is worth noting that the solution of our algorithm remains largely unchanged regardless of how the total sample is divided, which is known as partition-insensitivity. Furthermore, under some mild assumptions, we prove the convergence of the iterative sequence generated by our parallel algorithm. Numerical experiments on synthetic and real datasets demonstrate the feasibility and validity of the proposed algorithm. We provide a publicly available R software package to facilitate the implementation of the proposed algorithm.

Numerous error estimates have been carried out on various numerical schemes for subdiffusion equations. Unfortunately most error bounds suffer from a factor $1/(1-\alpha)$ or $\Gamma(1-\alpha)$, which blows up as the fractional order $\alpha\to 1^-$, a phenomenon not consistent with regularity of the continuous problem and numerical simulations in practice. Although efforts have been made to avoid the factor blow-up phenomenon, a robust analysis of error estimates still remains incomplete for numerical schemes with general nonuniform time steps. In this paper, we will consider the $\alpha$-robust error analysis of convolution-type schemes for subdiffusion equations with general nonuniform time-steps, and provide explicit factors in error bounds with dependence information on $\alpha$ and temporal mesh sizes. As illustration, we apply our abstract framework to two widely used schemes, i.e., the L1 scheme and Alikhanov's scheme. Our rigorous proofs reveal that the stability and convergence of a class of convolution-type schemes is $\alpha$-robust, i.e., the factor will not blowup while $\alpha\to 1^-$ with general nonuniform time steps even when rather general initial regularity condition is considered., Comment: 21 pages