Search

In this paper, we study the properties of nondegenerate cylindrical singularities of mean curvature flow. We prove they are isolated in spacetime and provide a complete description of the geometry and topology change of the flow passing through the singularities. Particularly, the topology change agrees with the level sets change near a critical point of a Morse function, which is the same as performing surgery. The proof is based on a new $L^2$-distance monotonicity formula, which allows us to derive a discrete almost monotonicity of the ``decay order", a discrete mean curvature flow analog to Almgren's frequency function., Comment: 52 pages, comments are welcome!

In this work, we introduce CHAIR (Classifier of Hallucination As ImproveR), a supervised framework for detecting hallucinations by analyzing internal logits from each layer of every token. Our method extracts a compact set of features such as maximum, minimum, mean, standard deviation, and slope-from the token logits across all layers, enabling effective hallucination detection without overfitting. Experiments on TruthfulQA and MMLU datasets demonstrate that CHAIR significantly improves detection accuracy, particularly in zero-shot scenarios, showcasing its robustness and generalizability. Beyond hallucination detection, CHAIR highlights the potential of using internal representations for designing advanced decoding strategies. By leveraging patterns in logits, we suggest that more sophisticated models and adaptive decoding methods could further reduce hallucinations and enhance text completion quality. CHAIR not only offers a practical solution for detecting hallucinations but also lays the groundwork for exploring richer representations in LLMs to improve their factuality and coherence.

In clinical practice, multiple biomarkers are often measured on the same subject for disease diagnosis, and combining them can improve diagnostic accuracy. Existing studies typically combine multiple biomarkers by maximizing the Area Under the ROC Curve (AUC), assuming a gold standard exists or that biomarkers follow a multivariate normal distribution. However, practical diagnostic settings require both optimal combination coefficients and an effective cutoff value, and the reference test may be imperfect. In this paper, we propose a two-stage method for identifying the optimal linear combination and cutoff value based on the Youden index. First, it maximizes an approximation of the empirical AUC to estimate the optimal linear coefficients for combining multiple biomarkers. Then, it maximizes the empirical Youden index to determine the optimal cutoff point for disease classification. Under the semiparametric single index model and regularity conditions, the estimators for the linear coefficients, cutoff point, and Youden index are consistent. This method is also applicable when the reference standard is imperfect. We demonstrate the performance of our method through simulations and apply it to construct a diagnostic scale for Chinese medicine., Comment: This manuscript has been submitted to Statistics in Medicine and is currently under review

In this article we show that generally almost regular flows, introduced by Bamler and Kleiner, in closed 3-manifolds will either go extinct in finite time or flow to a collection of smooth embedded minimal surfaces, possibly with multiplicity. Using a perturbative argument then we construct piecewise almost regular flows which either go extinct in finite time or flow to a stable minimal surface, possibly with multiplicity. We apply these results to construct minimal surfaces in 3-manifolds in a variety of circumstances, mainly novel from the point of the view that the arguments are via parabolic methods., Comment: Comments welcome!

Currently, most low-light image enhancement methods only consider information from a single view, neglecting the correlation between cross-view information. Therefore, the enhancement results produced by these methods are often unsatisfactory. In this context, there have been efforts to develop methods specifically for low-light stereo image enhancement. These methods take into account the cross-view disparities and enable interaction between the left and right views, leading to improved performance. However, these methods still do not fully exploit the interaction between left and right view information. To address this issue, we propose a model called Toward Sufficient Dual-View Interaction for Low-light Stereo Image Enhancement (SDI-Net). The backbone structure of SDI-Net is two encoder-decoder pairs, which are used to learn the mapping function from low-light images to normal-light images. Among the encoders and the decoders, we design a module named Cross-View Sufficient Interaction Module (CSIM), aiming to fully exploit the correlations between the binocular views via the attention mechanism. The quantitative and visual results on public datasets validate the superiority of our method over other related methods. Ablation studies also demonstrate the effectiveness of the key elements in our model.

An integrated error-correction scheme with high throughput, low frame errors rate (FER) and high reconciliation efficiency under low signal to noise ratio (SNR) is one of the major bottlenecks to realize high-performance and low-cost continuous variable quantum key distribution (CV-QKD). To solve this long-standing problem, a novel two-stage error correction method with limited precision that is suitable for integration given limited on-chip hardware resource while maintaining excellent decoding performance is proposed, and experimentally verified on a commercial FPGA. Compared to state-of-art results, the error-correction throughput can be improved more than one order of magnitude given FER<0.1 based on the proposed method, where 544.03 Mbps and 393.33 Mbps real-time error correction is achieved for typical 0.2 and 0.1 code rate, respectively. Besides, compared with traditional decoding method, the secure key rate (SKR) for CV-QKD under composable security framework can be improved by 140.09% and 122.03% by using the proposed two-stage decoding method for codes rate 0.2 and 0.1, which can support 32.70 Mbps and 5.66 Mbps real-time SKR under typical transmission distances of 25 km and 50 km, correspondingly. The record-breaking results paves the way for large-scale deployment of high-rate integrated CV-QKD systems in metropolitan quantum secure network.

Secure precision time synchronization is important for applications of Cyber-Physical Systems. However, several attacks, especially the Time Delay Attack (TDA), deteriorates the performance of time synchronization system seriously. Multiple paths scheme is thought as an effective security countermeasure to decrease the influence of TDA. However, the effective secure combination algorithm is still missed for precision time synchronization. In this paper, a secure combination algorithm based on Dempster-Shafer theory is proposed for multiple paths method. Special optimizations are done for the combination algorithm to solve the potential problems due to untrusted evidence. Theoretical simulation shows that the proposed algorithm works much better than Fault Tolerant Algorithm (FTA) and the attack detection method based on single path. And experimental demonstration proves the feasibility and superiority of the proposed algorithm, where the time stability with 27.97 ps, 1.57 ps, and 1.12 ps at average time 1s, 10s, 100s is achieved under TDA and local clock jump. The proposed algorithm can be used to improve the security and resilience of many importance synchronization protocol, such as NTP, PTP, and TWFTT.

The emergence of large language models (LLMs) relies heavily on distributed training strategies, among which pipeline parallelism plays a crucial role. As LLMs' training sequence length extends to 32k or even 128k, the current pipeline parallel methods face severe bottlenecks, including high memory footprints and substantial pipeline bubbles, greatly hindering model scalability and training throughput. To enhance memory efficiency and training throughput, in this work, we introduce an efficient sequence-level one-forward-one-backward (1F1B) pipeline scheduling method tailored for training LLMs on long sequences named Seq1F1B. Seq1F1B decomposes batch-level schedulable units into finer sequence-level units, reducing bubble size and memory footprint. Considering that Seq1F1B may produce slight extra bubbles if sequences are split evenly, we design a computation-wise strategy to partition input sequences and mitigate this side effect. Compared to competitive pipeline baseline methods such as Megatron 1F1B pipeline parallelism, our method achieves higher training throughput with less memory footprint. Notably, Seq1F1B efficiently trains a LLM with 30B parameters on sequences up to 64k using 64 NVIDIA A100 GPUs without recomputation strategies, a feat unachievable with existing methods. Our source code is based on Megatron-LM, and now is avaiable at: https://github.com/MayDomine/Seq1F1B.git., Comment: 12 pages, 4 figures, 6 tables

Visual deep learning (VDL) systems have shown significant success in real-world applications like image recognition, object detection, and autonomous driving. To evaluate the reliability of VDL, a mainstream approach is software testing, which requires diverse mutations over image semantics. The rapid development of multi-modal large language models (MLLMs) has introduced revolutionary image mutation potentials through instruction-driven methods. Users can now freely describe desired mutations and let MLLMs generate the mutated images. Hence, parallel to large language models' (LLMs) recent success in traditional software fuzzing, one may also expect MLLMs to be promising for VDL testing in terms of offering unified, diverse, and complex image mutations. However, the quality and applicability of MLLM-based mutations in VDL testing remain largely unexplored. We present the first study, aiming to assess MLLMs' adequacy from 1) the semantic validity of MLLM mutated images, 2) the alignment of MLLM mutated images with their text instructions (prompts), and 3) the faithfulness of how different mutations preserve semantics that are ought to remain unchanged. With large-scale human studies and quantitative evaluations, we identify MLLM's promising potentials in expanding the covered semantics of image mutations. Notably, while SoTA MLLMs (e.g., GPT-4V) fail to support or perform worse in editing existing semantics in images (as in traditional mutations like rotation), they generate high-quality test inputs using "semantic-replacement" mutations (e.g., "dress a dog with clothes"), which bring extra semantics to images; these were infeasible for past approaches. Hence, we view MLLM-based mutations as a vital complement to traditional mutations, and advocate future VDL testing tasks to combine MLLM-based methods and traditional image mutations for comprehensive and reliable testing.

Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long sequences. One potential solution for the long sequence problem is to utilize distributed clusters to parallelize the computation of attention modules across multiple devices (e.g., GPUs). However, adopting a distributed approach inevitably introduces extra memory overheads to store local attention results and incurs additional communication costs to aggregate local results into global ones. In this paper, we propose a distributed attention framework named ``BurstAttention'' to optimize memory access and communication operations at both the global cluster and local device levels. In our experiments, we compare BurstAttention with other competitive distributed attention solutions for long sequence processing. The experimental results under different length settings demonstrate that BurstAttention offers significant advantages for processing long sequences compared with these competitive baselines, reducing 40% communication overheads and achieving 1.37 X speedup during training 128K sequence length on 32 X A100., Comment: 13 pages, 7 figures

For $d\ge2$ and iid $d$-dimensional observations $X^{(1)},X^{(2)},\dots$ with independent Exponential$(1)$ coordinates, we revisit the study by Fill and Naiman (Electron. J. Probab., 2020) of the boundary (relative to the closed positive orthant), or "frontier", $F_n$ of the closed Pareto record-setting (RS) region \[ \mbox{RS}_n:=\{0\le x\in{\mathbb R}^d:x\not\prec X^{(i)}\mbox{\ for all $1\le i\le n$}\} \] at time $n$, where $0\le x$ means that $0\le x_j$ for $1\le j\le d$ and $x\prec y$ means that $x_j0$ and $c_n\to\infty$ we have \[ {\mathbb P}(F_n^- -\ln n\in (-(2+\varepsilon)\ln\ln\ln n,c_n))\to 1 \] (describing typical behavior) and almost surely \[ \limsup \frac{F_n^- - \ln n}{\ln \ln n} \le 0 \quad \mbox{and} \quad \liminf \frac{F_n^- - \ln n}{\ln \ln \ln n} \in [-2, -1]. \] In this paper we use the theory of generators (minima of $F_n$) together with the first- and second-moment methods to improve considerably the trailing-point location results to \[ F_n^- - (\ln n - \ln \ln \ln n) \overset{\mathrm{P}}{\longrightarrow} - \ln(d - 1) \] (describing typical behavior) and, for $d \ge 3$, almost surely \begin{align*} &\limsup [F_n^- - (\ln n - \ln \ln \ln n)] \leq -\ln(d - 2) + \ln 2 \\ \mbox{and }&\liminf [F_n^- - (\ln n - \ln \ln \ln n)] \ge - \ln d - \ln 2. \end{align*}, Comment: 32 pages, 2 figures. arXiv admin note: text overlap with arXiv:1901.05621

Given a sequence of independent random vectors taking values in ${\mathbb R}^d$ and having common continuous distribution function $F$, say that the $n^{\rm \scriptsize th}$ observation sets a (Pareto) record if it is not dominated (in every coordinate) by any preceding observation. Let $p_n(F) \equiv p_{n, d}(F)$ denote the probability that the $n^{\rm \scriptsize th}$ observation sets a record. There are many interesting questions to address concerning $p_n$ and multivariate records more generally, but this short paper focuses on how $p_n$ varies with $F$, particularly if, under $F$, the coordinates exhibit negative dependence or positive dependence (rather than independence, a more-studied case). We introduce new notions of negative and positive dependence ideally suited for such a study, called negative record-setting probability dependence (NRPD) and positive record-setting probability dependence (PRPD), relate these notions to existing notions of dependence, and for fixed $d \geq 2$ and $n \geq 1$ prove that the image of the mapping $p_n$ on the domain of NRPD (respectively, PRPD) distributions is $[p^*_n, 1]$ (resp., $[n^{-1}, p^*_n]$), where $p^*_n$ is the record-setting probability for any continuous $F$ governing independent coordinates., Comment: 16 pages, 1 figure; this revision responds to three anonymous reviews; paper accepted to Probability in the Engineering and Informational Sciences

We construct new examples of immortal mean curvature flow of smooth embedded connected hypersurfaces in closed manifolds, which converge to minimal hypersurfaces with multiplicity $2$ as time approaches infinity., Comment: 18 pages, 3 figures, all comments welcome!

Concept Bottleneck Models (CBMs) aim to deliver interpretable and interventionable predictions by bridging features and labels with human-understandable concepts. While recent CBMs show promising potential, they suffer from information leakage, where unintended information beyond the concepts (either when concepts are represented with probabilities or binary states) are leaked to the subsequent label prediction. Consequently, distinct classes are falsely classified via indistinguishable concepts, undermining the interpretation and intervention of CBMs. This paper alleviates the information leakage issue by introducing label supervision in concept predication and constructing a hierarchical concept set. Accordingly, we propose a new paradigm of CBMs, namely SupCBM, which achieves label predication via predicted concepts and a deliberately-designed intervention matrix. SupCBM focuses on concepts that are mostly relevant to the predicted label and only distinguishes classes when different concepts are presented. Our evaluations show that SupCBM outperforms SOTA CBMs over diverse datasets. It also manifests better generality across different backbone models. With proper quantification of information leakage in different CBMs, we demonstrate that SupCBM significantly reduces the information leakage.

We construct a new example of an immortal mean curvature flow of smooth embedded connected surfaces in $\mathbb R^3$, which converges to a plane with multiplicity $2$ as time approaches infinity., Comment: v2: minor corrections. 29 pages, 6 figures, all comments welcome!

Video Moment Retrieval (VMR) aims to retrieve temporal segments in untrimmed videos corresponding to a given language query by constructing cross-modal alignment strategies. However, these existing strategies are often sub-optimal since they ignore the modality imbalance problem, \textit{i.e.}, the semantic richness inherent in videos far exceeds that of a given limited-length sentence. Therefore, in pursuit of better alignment, a natural idea is enhancing the video modality to filter out query-irrelevant semantics, and enhancing the text modality to capture more segment-relevant knowledge. In this paper, we introduce Modal-Enhanced Semantic Modeling (MESM), a novel framework for more balanced alignment through enhancing features at two levels. First, we enhance the video modality at the frame-word level through word reconstruction. This strategy emphasizes the portions associated with query words in frame-level features while suppressing irrelevant parts. Therefore, the enhanced video contains less redundant semantics and is more balanced with the textual modality. Second, we enhance the textual modality at the segment-sentence level by learning complementary knowledge from context sentences and ground-truth segments. With the knowledge added to the query, the textual modality thus maintains more meaningful semantics and is more balanced with the video modality. By implementing two levels of MESM, the semantic information from both modalities is more balanced to align, thereby bridging the modality gap. Experiments on three widely used benchmarks, including the out-of-distribution settings, show that the proposed framework achieves a new start-of-the-art performance with notable generalization ability (e.g., 4.42% and 7.69% average gains of R1@0.7 on Charades-STA and Charades-CG). The code will be available at https://github.com/lntzm/MESM., Comment: Accepted to AAAI 2024

We show that for certain one-parameter families of initial conditions in $\mathbb R^3$, when we run mean curvature flow, a genus one singularity must appear in one of the flows. Moreover, such a singularity is robust under perturbation of the family of initial conditions. This contrasts sharply with the case of just a single flow. As an application, we construct an embedded, genus one self-shrinker with entropy lower than a shrinking doughnut., Comment: 37 pages. Revised version according to the referee's suggestions and comments. Accepted by Geometry & Topology

EXplainable AI (XAI) is an essential topic to improve human understanding of deep neural networks (DNNs) given their black-box internals. For computer vision tasks, mainstream pixel-based XAI methods explain DNN decisions by identifying important pixels, and emerging concept-based XAI explore forming explanations with concepts (e.g., a head in an image). However, pixels are generally hard to interpret and sensitive to the imprecision of XAI methods, whereas "concepts" in prior works require human annotation or are limited to pre-defined concept sets. On the other hand, driven by large-scale pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promotable framework for performing precise and comprehensive instance segmentation, enabling automatic preparation of concept sets from a given image. This paper for the first time explores using SAM to augment concept-based XAI. We offer an effective and flexible concept-based explanation method, namely Explain Any Concept (EAC), which explains DNN decisions with any concept. While SAM is highly effective and offers an "out-of-the-box" instance segmentation, it is costly when being integrated into defacto XAI pipelines. We thus propose a lightweight per-input equivalent (PIE) scheme, enabling efficient explanation with a surrogate model. Our evaluation over two popular datasets (ImageNet and COCO) illustrate the highly encouraging performance of EAC over commonly-used XAI methods.

In this paper, we present an implementation of JSON-diff framework JYCM, extending the existing framework by introducing the concept of "unordered" comparisons and allowing users to customize their comparison scenarios flexibly. Furthermore, we provide a diff-result renderer to visualize better and understand the differences between JSON objects. Our work enables more adaptable and comprehensive comparisons to accommodate a wider range of use cases and requirements., Comment: Updated 2023-6-18: add related works: [JEDI: These aren't the JSON documents you're looking for...]

As the popularity of large language models (LLMs) soars across various applications, ensuring their alignment with human values has become a paramount concern. In particular, given that LLMs have great potential to serve as general-purpose AI assistants in daily life, their subtly unethical suggestions become a serious and real concern. Tackling the challenge of automatically testing and repairing unethical suggestions is thus demanding. This paper introduces the first framework for testing and repairing unethical suggestions made by LLMs. We first propose ETHICSSUITE, a test suite that presents complex, contextualized, and realistic moral scenarios to test LLMs. We then propose a novel suggest-critic-reflect (SCR) process, serving as an automated test oracle to detect unethical suggestions. We recast deciding if LLMs yield unethical suggestions (a hard problem; often requiring human expertise and costly to decide) into a PCR task that can be automatically checked for violation. Moreover, we propose a novel on-the-fly (OTF) repairing scheme that repairs unethical suggestions made by LLMs in real-time. The OTF scheme is applicable to LLMs in a black-box API setting with moderate cost. With ETHICSSUITE, our study on seven popular LLMs (e.g., ChatGPT, GPT-4) uncovers in total 109,824 unethical suggestions. We apply our OTF scheme on two LLMs (Llama-13B and ChatGPT), which generates valid repair to a considerable amount of unethical ones, paving the way for more ethically conscious LLMs.

Given a smooth closed embedded self-shrinker $S$ with index $I$ in $\mathbb R^{n}$, we construct an $I$-dimensional family of complete translators polynomially asymptotic to $S\times\mathbb R$ at infinity. We further prove that $\mathbb R^{n+1}$ can be decomposed in many ways into a one-parameter family of closed sets $\coprod_{a\in \mathbb R} T_a$, and each closed set $T_a$ contains a complete translator asymptotic to $S\times\mathbb R$ at infinity. If the closed set $T_a$ fattens, namely it has nonempty interior, then there are at least two translators asymptotic to each other at an exponential rate, which can be viewed as a kind of nonuniqueness. We show that this fattening phenomenon is non-generic but indeed happens., Comment: 44 pages, comments are welcome

This paper examines the dynamics of mean curvature flow as it approaches a cylindrical singularity. We reveal the mechanism for the isolatedness of cylindrical singularities in terms of the normal form of the asymptotic expansion of the rescaled mean curvature flow and formulate the notion of nondegeneracy of a cylindrical singularity. Our findings show that, generically, such a non-degenerate singularity is robust and isolated. In the presence of more complicated singularity sets such as in the example of a marriage ring, we demonstrate how to make the first-time singularity set a singleton by an arbitrarily small initial perturbation. These discoveries have implications for the level set flows' generic low regularity and the type-I nature of generic rotational MCFs. Additionally, the paper introduces the concept of a ``firecracker'' type singularity and its significance in studying the isolatedness of cylindrical singularities. The research combines techniques and concepts from geometric flows, dynamical systems, and semilinear heat equations., Comment: 53 pages, comments are welcome!

Many works have shown the overfitting hazard of selecting a trading strategy based only on good IS (in sample) performance. But most of them have merely shown such phenomena exist without offering ways to avoid them. We propose an approach to avoid overfitting: A good (meaning non-overfitting) trading strategy should still work well on paths generated in accordance with the distribution of the historical data. We use GAN with LSTM to learn or fit the distribution of the historical time series . Then trading strategies are backtested by the paths generated by GAN to avoid overfitting.(This paper is an tanslated English version of a thesis (10.6342/NTU201801645) which was originally written in Chinese in 2018, where some statements and claims are outdated in 2022)