Search

Weight averaging has become a standard technique for enhancing model performance. However, methods such as Stochastic Weight Averaging (SWA) and Latest Weight Averaging (LAWA) often require manually designed procedures to sample from the training trajectory, and the results depend heavily on hyperparameter tuning. To minimize human effort, this paper proposes a simple yet efficient algorithm called Selective Weight Averaging (SeWA), which adaptively selects checkpoints during the final stages of training for averaging. Based on SeWA, we show that only a few points are needed to achieve better generalization and faster convergence. Theoretically, solving the discrete subset selection problem is inherently challenging. To address this, we transform it into a continuous probabilistic optimization framework and employ the Gumbel-Softmax estimator to learn the non-differentiable mask for each checkpoint. Further, we theoretically derive the SeWA's stability-based generalization bounds, which are sharper than that of SGD under both convex and non-convex assumptions. Finally, solid extended experiments in various domains, including behavior cloning, image classification, and text classification, further validate the effectiveness of our approach.

Manufacturability is vital for product design and production, with accessibility being a key element, especially in subtractive manufacturing. Traditional methods for geometric accessibility analysis are time-consuming and struggle with scalability, while existing deep learning approaches in manufacturability analysis often neglect geometric challenges in accessibility and are limited to specific model types. In this paper, we introduce DeepMill, the first neural framework designed to accurately and efficiently predict inaccessible and occlusion regions under varying machining tool parameters, applicable to both CAD and freeform models. To address the challenges posed by cutter collisions and the lack of extensive training datasets, we construct a cutter-aware dual-head octree-based convolutional neural network (O-CNN) and generate an inaccessible and occlusion regions analysis dataset with a variety of cutter sizes for network training. Experiments demonstrate that DeepMill achieves 94.7% accuracy in predicting inaccessible regions and 88.7% accuracy in identifying occlusion regions, with an average processing time of 0.04 seconds for complex geometries. Based on the outcomes, DeepMill implicitly captures both local and global geometric features, as well as the complex interactions between cutters and intricate 3D models.

This work addresses the critical question of why and when diffusion models, despite being designed for generative tasks, can excel at learning high-quality representations in a self-supervised manner. To address this, we develop a mathematical framework based on a low-dimensional data model and posterior estimation, revealing a fundamental trade-off between generation and representation quality near the final stage of image generation. Our analysis explains the unimodal representation dynamics across noise scales, mainly driven by the interplay between data denoising and class specification. Building on these insights, we propose an ensemble method that aggregates features across noise levels, significantly improving both clean performance and robustness under label noise. Extensive experiments on both synthetic and real-world datasets validate our findings., Comment: First two authors contributed equally

Catastrophic forgetting is a critical chanllenge for incremental object detection (IOD). Most existing methods treat the detector monolithically, relying on instance replay or knowledge distillation without analyzing component-specific forgetting. Through dissection of Faster R-CNN, we reveal a key insight: Catastrophic forgetting is predominantly localized to the RoI Head classifier, while regressors retain robustness across incremental stages. This finding challenges conventional assumptions, motivating us to develop a framework termed NSGP-RePRE. Regional Prototype Replay (RePRE) mitigates classifier forgetting via replay of two types of prototypes: coarse prototypes represent class-wise semantic centers of RoI features, while fine-grained prototypes model intra-class variations. Null Space Gradient Projection (NSGP) is further introduced to eliminate prototype-feature misalignment by updating the feature extractor in directions orthogonal to subspace of old inputs via gradient projection, aligning RePRE with incremental learning dynamics. Our simple yet effective design allows NSGP-RePRE to achieve state-of-the-art performance on the Pascal VOC and MS COCO datasets under various settings. Our work not only advances IOD methodology but also provide pivotal insights for catastrophic forgetting mitigation in IOD. Code will be available soon., Comment: 14 pages, 7 figures, 9 tables

3D Gaussian Splatting has emerged as an efficient photorealistic novel view synthesis method. However, its reliance on sparse Structure-from-Motion (SfM) point clouds consistently compromises the scene reconstruction quality. To address these limitations, this paper proposes a novel 3D reconstruction framework Gaussian Processes Gaussian Splatting (GP-GS), where a multi-output Gaussian Process model is developed to achieve adaptive and uncertainty-guided densification of sparse SfM point clouds. Specifically, we propose a dynamic sampling and filtering pipeline that adaptively expands the SfM point clouds by leveraging GP-based predictions to infer new candidate points from the input 2D pixels and depth maps. The pipeline utilizes uncertainty estimates to guide the pruning of high-variance predictions, ensuring geometric consistency and enabling the generation of dense point clouds. The densified point clouds provide high-quality initial 3D Gaussians to enhance reconstruction performance. Extensive experiments conducted on synthetic and real-world datasets across various scales validate the effectiveness and practicality of the proposed framework., Comment: 14 pages,11 figures

Bipartite graphs, representing two-mode networks, arise in many research fields. These networks have two disjoint node sets representing distinct entity types, for example persons and groups, with edges representing associations between the two entity types. In bipartite graphs, the smallest possible cycle is a cycle of length four, and hence four-cycles are the smallest structure to model closure in such networks. Exponential-family random graph models (ERGMs) are a widely used model for social, and other, networks, including specifically bipartite networks. Existing ERGM terms to model four-cycles in bipartite networks, however, are relatively rarely used. In this work we demonstrate some problems with these existing terms to model four-cycles, and define new ERGM terms to help overcome these problems. The position of the new terms in the ERGM dependence hierarchy, and their interpretation, is discussed. The new terms are demonstrated in simulation experiments, and their application illustrated with ERGM models of empirical networks ranging in size from hundreds of nodes to hundreds of thousands of nodes., Comment: 54 pages including appendices

How to calculate the entanglement entropy between two subsystems for a mixed state has remained an important problem. In this paper, we provide a straightforward method, namely the subtraction approach, to solve this problem for generic covariant bipartite mixed states, where time dependence is explicitly included. We further demonstrate that the mixed state entanglement entropy $S_\text{vN}$ can be calculated in a more abstract yet powerful way. Within the context of the AdS$_3$/CFT$_2$ and AdS$_5$/CFT$_4$ correspondences, we show that $S_\text{vN}$ exactly matches the corresponding entanglement wedge cross section in the AdS bulk, respectively., Comment: 17 pages, 8 figures

The increasing penetration of distributed energy resources (DER) has sparked interest in promoting their participation in the power market. Here we consider a setting in which different virtual power plants (VPPs) with certain flexible resources take part in electricity trading, either by direct participation in the wholesale power market, or interfaced by the Distribution System Operator (DSO). Our goal is to examine the role and influence of the DSO as a stakeholder, for which we formulate a Stackelberg game via a bilevel optimization model: the DSO maximizes profits at the upper level, while VPPs minimize operating costs at the lower level. To solve this problem, we use the Karush-Kuhn-Tucke optimality conditions of the convex lower-level problems to achieve a single-level mixed-integer nonlinear program. The results show that the role of the DSO as an intermediary agent leads to a decrease in operating costs for the VPPs, while guaranteeing a profit for the DSO.

We report the X-ray spectral and timing analysis of the high mass X-ray binary EXO 2030+375 during the 2021 type-II outburst based on the Insight-HXMT observations. Pulsations can be detected in the energy band of 1-150 keV. The pulse profile shows energy and luminosity dependence and variability. We observed transitions in the pulse profile shape during the rising and the decaying phase of the outburst. The pulse fraction exhibits an anti-correlation with luminosity and a non-monotonic energy dependence, with a possible dip near 30 keV during the outburst peak. The hardness-intensity diagrams (7-10 keV/4-7 keV) suggest state transitions during the early and late phases of the outburst. These transitions are consistent with the luminosity at which the pulse profile shape changes occur, revealing the source reaching the critical luminosity and transitioning between super-critical and sub-critical accretion regimes. We performed the average and phase-resolved spectral analysis, where the flux-resolved average spectra show a stable spectral evolution with luminosity. The phase-resolved spectral analysis reveals that the dependence of spectral parameters on the pulse phase varies with different luminosities.

In recent years, the interaction between dark matter (DM) and dark energy has become a topic of interest in cosmology. Interacting dark matter-dark energy (IDE) models have a substantial impact on the formation of cosmological large-scale structures, which serve as the background for DM halo evolution. This impact can be examined through the shape and spin orientation of halos in numerical simulations incorporating IDE effects. In our work, we use the N-body simulation pipeline ME-GADGET to simulate and study the halo spin and orientation in IDE models. We found that in models where DM transfers into DE (IDE I), the alignment of halo shapes with the surrounding tidal field is enhanced, while the alignment of halo spins with the tidal field is decreased compared to {\Lambda}CDM. Conversely, in models where DE transfers into DM (IDE II), the opposite occurs. We have provided fitted functions to describe these alignment signals. Our study provides the foundation for more accurate modeling of observations in the future such as China Space Station Telescope.

Generating visual text in natural scene images is a challenging task with many unsolved problems. Different from generating text on artificially designed images (such as posters, covers, cartoons, etc.), the text in natural scene images needs to meet the following four key criteria: (1) Fidelity: the generated text should appear as realistic as a photograph and be completely accurate, with no errors in any of the strokes. (2) Reasonability: the text should be generated on reasonable carrier areas (such as boards, signs, walls, etc.), and the generated text content should also be relevant to the scene. (3) Utility: the generated text can facilitate to the training of natural scene OCR (Optical Character Recognition) tasks. (4) Controllability: The attribute of the text (such as font and color) should be controllable as needed. In this paper, we propose a two stage method, SceneVTG++, which simultaneously satisfies the four aspects mentioned above. SceneVTG++ consists of a Text Layout and Content Generator (TLCG) and a Controllable Local Text Diffusion (CLTD). The former utilizes the world knowledge of multi modal large language models to find reasonable text areas and recommend text content according to the nature scene background images, while the latter generates controllable multilingual text based on the diffusion model. Through extensive experiments, we respectively verified the effectiveness of TLCG and CLTD, and demonstrated the state-of-the-art text generation performance of SceneVTG++. In addition, the generated images have superior utility in OCR tasks like text detection and text recognition. Codes and datasets will be available.

This letter proposes an alternative underdetermined framework for fault location that utilizes current measurements along with the branch-bus matrix, providing another option besides the traditional voltage-based methods. To enhance fault location accuracy in the presence of multiple outliers, the robust YALL1 algorithm is used to resist outlier interference and accurately recover the sparse vector, thereby pinpointing the fault precisely. The results on the IEEE 39-bus test system demonstrate the effectiveness and robustness of the proposed method.

Deep neural networks have attained remarkable success across diverse classification tasks. Recent empirical studies have shown that deep networks learn features that are linearly separable across classes. However, these findings often lack rigorous justifications, even under relatively simple settings. In this work, we address this gap by examining the linear separation capabilities of shallow nonlinear networks. Specifically, inspired by the low intrinsic dimensionality of image data, we model inputs as a union of low-dimensional subspaces (UoS) and demonstrate that a single nonlinear layer can transform such data into linearly separable sets. Theoretically, we show that this transformation occurs with high probability when using random weights and quadratic activations. Notably, we prove this can be achieved when the network width scales polynomially with the intrinsic dimension of the data rather than the ambient dimension. Experimental results corroborate these theoretical findings and demonstrate that similar linear separation properties hold in practical scenarios beyond our analytical scope. This work bridges the gap between empirical observations and theoretical understanding of the separation capacity of nonlinear networks, offering deeper insights into model interpretability and generalization., Comment: 32 pages, 9 figures

We report on the potential occurrence of a numerical instability in the long-time simulation of black holes using the Baumgarte-Shapiro-Shibata-Nakamura formulation of numerical relativity, even in the simple set-up of a Schwarzschild black hole. Through extensive numerical experiments, we identify that this ``late-time instability'' arises from accumulated violations of the momentum constraint. To address this issue, we propose two modified versions of the so-called conformal covariant Z4 scheme, designed to propagate momentum constraint violations without damping. Our results demonstrate that these alternative formulations, which we refer to as CCZ4' and CCZ3, effectively resolve the late-time numerical instability not only in Schwarzschild spacetimes but also in black hole spacetimes with matter fields. Notably, by preventing damping of the momentum constraint violation, the Hamiltonian constraint damping can be significantly increased, which plays a crucial role in stabilizing long-term evolution in our proposed schemes., Comment: 23 pages