Search

Geometry problem solving (GPS) requires capacities of multi-modal understanding, multi-hop reasoning and theorem knowledge application. In this paper, we propose a neural-symbolic model for plane geometry problem solving (PGPS), named PGPSNet-v2, with three key steps: modal fusion, reasoning process and knowledge verification. In modal fusion, we leverage textual clauses to express fine-grained structural and semantic content of geometry diagram, and fuse diagram with textual problem efficiently through structural-semantic pre-training. For reasoning, we design an explicable solution program to describe the geometric reasoning process, and employ a self-limited decoder to generate solution program autoregressively. To reduce solution errors, a multi-level theorem verifier is proposed to eliminate solutions that do not match geometric principles, alleviating the hallucination of the neural model. We also construct a large-scale geometry problem dataset called PGPS9K, containing fine-grained annotations of textual clauses, solution program and involved knowledge tuples. Extensive experiments on datasets Geometry3K and PGPS9K show that our PGPSNet solver outperforms existing symbolic and neural solvers in GPS performance, while maintaining good explainability and reliability, and the solver components (fusion, reasoning, verification) are all justified effective., Comment: under review by journal

The integration of renewable energy sources (RES) into microgrids poses challenges to reliable and economic operation due to the inherent uncertainty and volatility of RES. Contrary to the previous dispatch methods that require precise predictions of RES, this paper proposes a novel prediction-free and data-driven coordinated dispatch framework for reliable microgrid operations. In the offline stage, ex-post optimal dispatch sequences are generated based on historical dispatch from "God' s-eye view". The sequences offer a global reference and are sequentially updated based on the newly observed data. Subsequently, we propose an adaptive virtual-queue-based online convex optimization (OCO) method to generate the real-time control policy of microgrid, which aim to minimize the instant operation cost while tracking the offline reference. We provide theoretical proof that the proposed method outperforms the existing OCO methods and admits sublinear dynamic regret bound and sublinear hard cumulative constraint violation bound for OCO with time-varying constraints. Case study illustrates that the proposed method outperforms state-of-the-art methods in terms of economic optimality, computational efficiency, and security.

Two new proofs are provided, offering two new perspectives on Godbersen's conjecture. One of the proofs utilizes Helly's theorem to provide a concise and elegant proof of the inequality in Godbersen's conjecture. The other proof utilizes the Brunn-Minkowski inequality to provide a completely new proof of the inclusion $-K\subset nK$ for convex bodies $K$ with centroid at the origin, thereby proving Godbersen's conjecture.

Although Vision Transformers (ViTs) have recently advanced computer vision tasks significantly, an important real-world problem was overlooked: adapting to variable input resolutions. Typically, images are resized to a fixed resolution, such as 224x224, for efficiency during training and inference. However, uniform input size conflicts with real-world scenarios where images naturally vary in resolution. Modifying the preset resolution of a model may severely degrade the performance. In this work, we propose to enhance the model adaptability to resolution variation by optimizing the patch embedding. The proposed method, called Multi-Scale Patch Embedding (MSPE), substitutes the standard patch embedding with multiple variable-sized patch kernels and selects the best parameters for different resolutions, eliminating the need to resize the original image. Our method does not require high-cost training or modifications to other parts, making it easy to apply to most ViT models. Experiments in image classification, segmentation, and detection tasks demonstrate the effectiveness of MSPE, yielding superior performance on low-resolution inputs and performing comparably on high-resolution inputs with existing methods.

Graph matching is a fundamental tool in computer vision and pattern recognition. In this paper, we introduce an algorithm for graph matching based on the proximal operator, referred to as differentiable proximal graph matching (DPGM). Specifically, we relax and decompose the quadratic assignment problem for the graph matching into a sequence of convex optimization problems. The whole algorithm can be considered as a differentiable map from the graph affinity matrix to the prediction of node correspondence. Therefore, the proposed method can be organically integrated into an end-to-end deep learning framework to jointly learn both the deep feature representation and the graph affinity matrix. In addition, we provide a theoretical guarantee to ensure the proposed method converges to a stable point with a reasonable number of iterations. Numerical experiments show that PGM outperforms existing graph matching algorithms on diverse datasets such as synthetic data, and CMU House. Meanwhile, PGM can fully harness the capability of deep feature extractors and achieve state-of-art performance on PASCAL VOC keypoints.

In the circuit quantum electrodynamics architectures, to realize a long-range quantum network mediated by flying photon, it is necessary to shape the temporal profile of emitted photons to achieve high transfer efficiency between two quantum nodes. In this work, we demonstrate a new single-rail and dual-rail time-bin shaped photon generator without additional flux-tunable elements, which can act as a quantum interface of a point-to-point quantum network. In our approach, we adopt a qubit-resonator-transmission line configuration, and the effective coupling strength between the qubit and the resonator can be varied by parametrically modulating the qubit frequency. In this way, the coupling is directly proportional to the parametric modulation amplitude and covers a broad tunable range beyond 20 MHz for the sample we used. Additionally, when emitting shaped photons, we find that the spurious frequency shift (-0.4 MHz) due to parametric modulation is small and can be readily calibrated through chirping. We develop an efficient photon field measurement setup based on the data stream processing of GPU. Utilizing this system, we perform photon temporal profile measurement, quantum state tomography of photon field, and quantum process tomography of single-rail quantum state transfer based on a heterodyne measurement scheme. The single-rail encoding state transfer fidelity of shaped photon emission is 90.32%, and that for unshaped photon is 97.20%, respectively. We believe that the fidelity of shaped photon emission is mainly limited by the qubit coherence time. The results demonstrate that our method is hardware efficient, simple to implement, and scalable. It could become a viable tool in a high-quality quantum network utilizing both single-rail and dual-rail time-bin encoding.

Scenario reduction (SR) aims to identify a small yet representative scenario set to depict the underlying uncertainty, which is critical to scenario-based stochastic optimization (SBSO) of power systems. Existing SR techniques commonly aim to achieve statistical approximation to the original scenario set. However, SR and SBSO are commonly considered into two distinct and decoupled processes, which cannot guarantee a superior approximation of the original optimality. Instead, this paper incorporates the SBSO problem structure into the SR process and introduces a novel problem-driven scenario reduction framework. Specifically, we transform the original scenario set in distribution space into the decision applicability between scenarios in problem space. Subsequently, the SR process, embedded by a distinctive problem-driven distance metric, is rendered as a mixed-integer linear programming formulation to obtain the representative scenario set while minimizing the optimality gap. Furthermore, ex-ante and ex-post problem-driven evaluation indices are proposed to evaluate the performance of SR. A two-stage stochastic economic dispatch problem with renewable generation and energy storage validates the effectiveness of the proposed framework. Numerical experiments demonstrate that the proposed framework significantly outperforms existing SR methods by identifying salient (e.g., worst-case) scenarios, and achieving an optimality gap of less than 0.1% within acceptable computation time., Comment: This is a manuscript submitted to IEEE Transactions on Power Systems. This manuscript contains 10 pages, 5 figures

Test-time adaptation (TTA) aims at adapting a model pre-trained on the labeled source domain to the unlabeled target domain. Existing methods usually focus on improving TTA performance under covariate shifts, while neglecting semantic shifts. In this paper, we delve into a realistic open-set TTA setting where the target domain may contain samples from unknown classes. Many state-of-the-art closed-set TTA methods perform poorly when applied to open-set scenarios, which can be attributed to the inaccurate estimation of data distribution and model confidence. To address these issues, we propose a simple but effective framework called unified entropy optimization (UniEnt), which is capable of simultaneously adapting to covariate-shifted in-distribution (csID) data and detecting covariate-shifted out-of-distribution (csOOD) data. Specifically, UniEnt first mines pseudo-csID and pseudo-csOOD samples from test data, followed by entropy minimization on the pseudo-csID data and entropy maximization on the pseudo-csOOD data. Furthermore, we introduce UniEnt+ to alleviate the noise caused by hard data partition leveraging sample-level confidence. Extensive experiments on CIFAR benchmarks and Tiny-ImageNet-C show the superiority of our framework. The code is available at https://github.com/gaozhengqing/UniEnt, Comment: CVPR 2024

The CO(1--0) and [\ion{C}{1}](1--0) emission lines are well-established tracers of cold molecular gas mass in local galaxies. At high redshift, where the interstellar medium (ISM) is likely to be denser, there have been limited direct comparisons of both ground state transitions. Here we present a study of CO(1--0) and [\ion{C}{1}](1--0) emission in a sample of 20 unlensed dusty, star-forming galaxies at $z=2-5$. The CO(1--0)/[\ion{C}{1}](1--0) ratio is constant up to at least $z=5$, supporting the use of [CI](1-0) as a gas mass tracer. PDR modelling of the available data indicates a median H$_2$ density of log$(n~[$cm$^{-3}])=4.7\pm0.2$, and UV radiation field log$(G_{\mathrm{UV}} [G$_0$])=3.2\pm0.2$. We use the CO(1--0), [\ion{C}{1}](1--0) and 3mm dust continuum measurements to cross--calibrate the respective gas mass conversion factors, finding no dependence of these factors on either redshift or infrared luminosity. Assuming a variable CO conversion factor then implies [\ion{C}{1}] and dust conversion factors that differ from canonically assumed values but are consistent with the solar/super-solar metallicities expected for our sources. Radiative transfer modelling shows that the warmer CMB at high redshift can significantly affect the [\ion{C}{1}] as well as CO emission, which can change the derived molecular gas masses by up to 70\% for the coldest kinetic gas temperatures expected. Nevertheless, we show that the magnitude of the effect on the ratio of the tracers is within the known scatter of the $L'_\mathrm{CO}-L'_\mathrm{[CI]}$ relation. Further determining the absolute decrease of individual line intensities will require well-sampled spectral line energy distributions (SLEDs) to model the gas excitation conditions in more detail.

Convolutional Neural Networks (CNNs) have advanced significantly in visual representation learning and recognition. However, they face notable challenges in performance and computational efficiency when dealing with real-world, multi-scale image inputs. Conventional methods rescale all input images into a fixed size, wherein a larger fixed size favors performance but rescaling small size images to a larger size incurs digitization noise and increased computation cost. In this work, we carry out a comprehensive, layer-wise investigation of CNN models in response to scale variation, based on Centered Kernel Alignment (CKA) analysis. The observations reveal lower layers are more sensitive to input image scale variations than high-level layers. Inspired by this insight, we propose Multi-scale Unified Network (MUSN) consisting of multi-scale subnets, a unified network, and scale-invariant constraint. Our method divides the shallow layers into multi-scale subnets to enable feature extraction from multi-scale inputs, and the low-level features are unified in deep layers for extracting high-level semantic features. A scale-invariant constraint is posed to maintain feature consistency across different scales. Extensive experiments on ImageNet and other scale-diverse datasets, demonstrate that MSUN achieves significant improvements in both model performance and computational efficiency. Particularly, MSUN yields an accuracy increase up to 44.53% and diminishes FLOPs by 7.01-16.13% in multi-scale scenarios.

Deep neural networks perform remarkably well in close-world scenarios. However, novel classes emerged continually in real applications, making it necessary to learn incrementally. Class-incremental learning (CIL) aims to gradually recognize new classes while maintaining the discriminability of old ones. Existing CIL methods have two limitations: a heavy reliance on preserving old data for forgetting mitigation and the need for vast labeled data for knowledge adaptation. To overcome these issues, we propose a non-exemplar semi-supervised CIL framework with contrastive learning and semi-supervised incremental prototype classifier (Semi-IPC). On the one hand, contrastive learning helps the model learn rich representations, easing the trade-off between learning representations of new classes and forgetting that of old classes. On the other hand, Semi-IPC learns a prototype for each class with unsupervised regularization, enabling the model to incrementally learn from partially labeled new data while maintaining the knowledge of old classes. Experiments on benchmark datasets demonstrate the strong performance of our method: without storing any old samples and only using less than 1% of labels, Semi-IPC outperforms advanced exemplar-based methods. We hope our work offers new insights for future CIL research. The code will be made publicly available.

Self-supervised learning (SSL) has emerged as an effective paradigm for deriving general representations from vast amounts of unlabeled data. However, as real-world applications continually integrate new content, the high computational and resource demands of SSL necessitate continual learning rather than complete retraining. This poses a challenge in striking a balance between stability and plasticity when adapting to new information. In this paper, we employ Centered Kernel Alignment for quantitatively analyzing model stability and plasticity, revealing the critical roles of batch normalization layers for stability and convolutional layers for plasticity. Motivated by this, we propose Branch-tuning, an efficient and straightforward method that achieves a balance between stability and plasticity in continual SSL. Branch-tuning consists of branch expansion and compression, and can be easily applied to various SSL methods without the need of modifying the original methods, retaining old data or models. We validate our method through incremental experiments on various benchmark datasets, demonstrating its effectiveness and practical value in real-world scenarios. We hope our work offers new insights for future continual self-supervised learning research. The code will be made publicly available.

Isolated interacting quantum systems generally thermalize, yet there are several counterexamples for the breakdown of ergodicity, such as many-body localization and quantum scars. Recently, ergodicity breaking has been observed in systems subjected to linear potentials, termed Stark many-body localization. This phenomenon is closely associated with Hilbert-space fragmentation, characterized by a strong dependence of dynamics on initial conditions. Here, we experimentally explore initial-state dependent dynamics using a ladder-type superconducting processor with up to 24 qubits, which enables precise control of the qubit frequency and initial state preparation. In systems with linear potentials, we observe distinct non-equilibrium dynamics for initial states with the same quantum numbers and energy, but with varying domain wall numbers. This distinction becomes increasingly pronounced as the system size grows, in contrast with disordered interacting systems. Our results provide convincing experimental evidence of the fragmentation in Stark systems, enriching our understanding of the weak breakdown of ergodicity., Comment: main text: 7 pages, 4 figures; supplementary: 13 pages, 14 figures

Graph matching is a commonly used technique in computer vision and pattern recognition. Recent data-driven approaches have improved the graph matching accuracy remarkably, whereas some traditional algorithm-based methods are more robust to feature noises, outlier nodes, and global transformation (e.g.~rotation). In this paper, we propose a graph neural network (GNN) based approach to combine the advantages of data-driven and traditional methods. In the GNN framework, we transform traditional graph-matching solvers as single-channel GNNs on the association graph and extend the single-channel architecture to the multi-channel network. The proposed model can be seen as an ensemble method that fuses multiple algorithms at every iteration. Instead of averaging the estimates at the end of the ensemble, in our approach, the independent iterations of the ensembled algorithms exchange their information after each iteration via a 1x1 channel-wise convolution layer. Experiments show that our model improves the performance of traditional algorithms significantly. In addition, we propose a random sampling strategy to reduce the computational complexity and GPU memory usage, so the model applies to matching graphs with thousands of nodes. We evaluate the performance of our method on three tasks: geometric graph matching, semantic feature matching, and few-shot 3D shape classification. The proposed model performs comparably or outperforms the best existing GNN-based methods., Comment: Accepted by IJCV in 2024

Generalized Category Discovery (GCD) is a pragmatic and challenging open-world task, which endeavors to cluster unlabeled samples from both novel and old classes, leveraging some labeled data of old classes. Given that knowledge learned from old classes is not fully transferable to new classes, and that novel categories are fully unlabeled, GCD inherently faces intractable problems, including imbalanced classification performance and inconsistent confidence between old and new classes, especially in the low-labeling regime. Hence, some annotations of new classes are deemed necessary. However, labeling new classes is extremely costly. To address this issue, we take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD). The goal is to improve the performance of GCD by actively selecting a limited amount of valuable samples for labeling from the oracle. To solve this problem, we devise an adaptive sampling strategy, which jointly considers novelty, informativeness and diversity to adaptively select novel samples with proper uncertainty. However, owing to the varied orderings of label indices caused by the clustering of novel classes, the queried labels are not directly applicable to subsequent training. To overcome this issue, we further propose a stable label mapping algorithm that transforms ground truth labels to the label space of the classifier, thereby ensuring consistent training across different active selection stages. Our method achieves state-of-the-art performance on both generic and fine-grained datasets. Our code is available at https://github.com/mashijie1028/ActiveGCD, Comment: Accepted to CVPR 2024

Reliable confidence estimation is a challenging yet fundamental requirement in many risk-sensitive applications. However, modern deep neural networks are often overconfident for their incorrect predictions, i.e., misclassified samples from known classes, and out-of-distribution (OOD) samples from unknown classes. In recent years, many confidence calibration and OOD detection methods have been developed. In this paper, we find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We investigate this problem and reveal that popular calibration and OOD detection methods often lead to worse confidence separation between correctly classified and misclassified examples, making it difficult to decide whether to trust a prediction or not. Finally, we propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance under various settings including balanced, long-tailed, and covariate-shift classification scenarios. Our study not only provides a strong baseline for reliable confidence estimation but also acts as a bridge between understanding calibration, OOD detection, and failure prediction. The code is available at \url{https://github.com/Impression2805/FMFP}., Comment: Accepted by IEEE TPAMI. arXiv admin note: text overlap with arXiv:2303.02970; text overlap with arXiv:2007.01458 by other authors

Machine learning has achieved remarkable success in many applications. However, existing studies are largely based on the closed-world assumption, which assumes that the environment is stationary, and the model is fixed once deployed. In many real-world applications, this fundamental and rather naive assumption may not hold because an open environment is complex, dynamic, and full of unknowns. In such cases, rejecting unknowns, discovering novelties, and then incrementally learning them, could enable models to be safe and evolve continually as biological systems do. This paper provides a holistic view of open-world machine learning by investigating unknown rejection, novel class discovery, and class-incremental learning in a unified paradigm. The challenges, principles, and limitations of current methodologies are discussed in detail. Finally, we discuss several potential directions for future research. This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm, to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.

High-order topological phases of matter refer to the systems of $n$-dimensional bulk with the topology of $m$-th order, exhibiting $(n-m)$-dimensional boundary modes and can be characterized by topological pumping. Here, we experimentally demonstrate two types of second-order topological pumps, forming four 0-dimensional corner localized states on a 4$\times$4 square lattice array of 16 superconducting qubits. The initial ground state of the system for half-filling, as a product of four identical entangled 4-qubit states, is prepared using an adiabatic scheme. During the pumping procedure, we adiabatically modulate the superlattice Bose-Hubbard Hamiltonian by precisely controlling both the hopping strengths and on-site potentials. At the half pumping period, the system evolves to a corner-localized state in a quadrupole configuration. The robustness of the second-order topological pump is also investigated by introducing different on-site disorder. Our work studies the topological properties of high-order topological phases from the dynamical transport picture using superconducting qubits, which would inspire further research on high-order topological phases.

The extension of the Rayleigh-Ritz variational principle to ensemble states $\rho_{\mathbf{w}}\equiv\sum_k w_k |\Psi_k\rangle \langle\Psi_k|$ with fixed weights $w_k$ lies ultimately at the heart of several recent methodological developments for targeting excitation energies by variational means. Prominent examples are density and density matrix functional theory, Monte Carlo sampling, state-average complete active space self-consistent field methods and variational quantum eigensolvers. In order to provide a sound basis for all these methods and to improve their current implementations, we prove the validity of the underlying critical hypothesis: Whenever the ensemble energy is well-converged, the same holds true for the ensemble state $\rho_{\mathbf{w}}$ as well as the individual eigenstates $|\Psi_k\rangle$ and eigenenergies $E_k$. To be more specific, we derive linear bounds $d_-\Delta{E}_{\mathbf{w}} \leq \Delta Q \leq d_+ \Delta\Delta{E}_{\mathbf{w}}$ on the errors $\Delta Q $ of these sought-after quantities. A subsequent analytical analysis and numerical illustration proves the tightness of our universal inequalities. Our results and particularly the explicit form of $d_{\pm}\equiv d_{\pm}^{(Q)}(\mathbf{w},\mathbf{E})$ provide valuable insights into the optimal choice of the auxiliary weights $w_k$ in practical applications., Comment: 22 pages, 9 figures

Existing federated learning methods have effectively addressed decentralized learning in scenarios involving data privacy and non-IID data. However, in real-world situations, each client dynamically learns new classes, requiring the global model to maintain discriminative capabilities for both new and old classes. To effectively mitigate the effects of catastrophic forgetting and data heterogeneity under low communication costs, we designed a simple and effective method named PLoRA. On the one hand, we adopt prototype learning to learn better feature representations and leverage the heuristic information between prototypes and class features to design a prototype re-weight module to solve the classifier bias caused by data heterogeneity without retraining the classification layer. On the other hand, our approach utilizes a pre-trained model as the backbone and utilizes LoRA to fine-tune with a tiny amount of parameters when learning new classes. Moreover, PLoRA does not rely on similarity-based module selection strategies, thereby further reducing communication overhead. Experimental results on standard datasets indicate that our method outperforms the state-of-the-art approaches significantly. More importantly, our method exhibits strong robustness and superiority in various scenarios and degrees of data heterogeneity. Our code will be publicly available., Comment: 11 pages, 4 figures, conference

Thouless pumping, a dynamical version of the integer quantum Hall effect, represents the quantized charge pumped during an adiabatic cyclic evolution. Here we report experimental observations of nontrivial topological pumping that is induced by disorder even during a topologically trivial pumping trajectory. With a 41-qubit superconducting quantum processor, we develop a Floquet engineering technique to realize cycles of adiabatic pumping by simultaneously varying the on-site potentials and the hopping couplings. We demonstrate Thouless pumping in the presence of disorder and show its breakdown as the strength of disorder increases. Moreover, we observe two types of topological pumping that are induced by on-site potential disorder and hopping disorder, respectively. Especially, an intrinsic topological pump that is induced by quasi-periodic hopping disorder has never been experimentally realized before. Our highly controllable system provides a valuable quantum simulating platform for studying various aspects of topological physics in the presence of disorder.