Search

Asian Americans as a group are overrepresented among STEM college graduates and have the highest average college enrollment rate of any racial or ethnic category. Thus, Asian Americans are typically excluded from educational interventions directed at improving STEM education for Students of Color because they are not considered to be underrepresented minorities. However, statistics obscure the individual needs of the more than 20 ethnic subgroups that fall under the umbrella term Asian Americans. Using a participatory action research approach, this paper documents the institutional and sociocultural factors that push out HMoob (or Hmong) American college students from STEM programs at one large, predominantly White university; and the coordinate processes of gatekeeping and transactional advising that either redirect those students toward non-STEM programs or force them out of the university completely.

This article clarifies historical and sociocultural factors that impact the role of STEM in the racialization of Asian Americans. Drawing on critical race and other theories of Asian American racialization, and a review of empirical research on the experiences of Asian American college students in STEM, we develop a conceptual framework called "STEM Asianization" that highlights the role of STEM ideology in the model minority racialization of Asian Americans. Consequences for Asian American students include (1) erasure of the intersectional experiences of minoritized Asian American students; (2) dehumanization of Asian Americans and establishment of a bamboo ceiling; (3) representation of Asian Americans as a perpetual foreigner/Yellow Peril during times of cultural and political crisis; and (4) representation of Asian Americans who cannot or do not conform to the STEM achievement narrative as a failed minority. We argue that STEM Asianization reproduces White supremacy by ideologically reinforcing the colorblind meritocracy of STEM institutions in the United States. [Additional funding provided by the Wisconsin Louis Stokes Alliance for Minority Participation.]

Lean Methodology for Garment Modernization. This article presents the lean methodology for modernizing garment manufacturing, focusing on lean thinking, lean practices, automation development, VSM, and CRP, and how to integrate them effectively. While isolated automation of specific operations can improve efficiency and reduce cycle time, it does not necessarily enhance overall garment output and efficiency. To achieve these broader improvements, it is essential to consider the entire production line and process using VSM and CRP to optimize production and center balance. This approach can increase efficiency, and reduce manufacturing costs, labor time, and lead time, ultimately adding value to the company and factory., Comment: 11 pages,7 Figures

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, distributed quantum-assisted probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability., Comment: 50 pages, 42 figures

Neoadjuvant chemotherapy (NAC) response prediction for triple negative breast cancer (TNBC) patients is a challenging task clinically as it requires understanding complex histology interactions within the tumor microenvironment (TME). Digital whole slide images (WSIs) capture detailed tissue information, but their giga-pixel size necessitates computational methods based on multiple instance learning, which typically analyze small, isolated image tiles without the spatial context of the TME. To address this limitation and incorporate TME spatial histology interactions in predicting NAC response for TNBC patients, we developed a histology context-aware transformer graph convolution network (NACNet). Our deep learning method identifies the histopathological labels on individual image tiles from WSIs, constructs a spatial TME graph, and represents each node with features derived from tissue texture and social network analysis. It predicts NAC response using a transformer graph convolution network model enhanced with graph isomorphism network layers. We evaluate our method with WSIs of a cohort of TNBC patient (N=105) and compared its performance with multiple state-of-the-art machine learning and deep learning models, including both graph and non-graph approaches. Our NACNet achieves 90.0% accuracy, 96.0% sensitivity, 88.0% specificity, and an AUC of 0.82, through eight-fold cross-validation, outperforming baseline models. These comprehensive experimental results suggest that NACNet holds strong potential for stratifying TNBC patients by NAC response, thereby helping to prevent overtreatment, improve patient quality of life, reduce treatment cost, and enhance clinical outcomes, marking an important advancement toward personalized breast cancer treatment., Comment: This paper is accepted by Computerized Medical Imaging and Graphics (Nov 07 2024)

Since the initial discovery of two-dimensional van der Waals (vdW) materials, significant effort has been made to incorporate the three properties of magnetism, band structure topology, and strong electron correlations $-$ to leverage emergent quantum phenomena and expand their potential applications. However, the discovery of a single vdW material that intrinsically hosts all three ingredients has remained an outstanding challenge. Here we report the discovery of a Kondo-interacting topological antiferromagnet in the vdW 5$f$ electron system UOTe. It has a high antiferromagnetic (AFM) transition temperature of 150 K, with a unique AFM configuration that breaks the combined parity and time reversal ($PT$) symmetry in an even number of layers while maintaining zero net magnetic moment. Our angle-resolved photoemission spectroscopy (ARPES) measurements reveal Dirac bands near the Fermi level, which combined with our theoretical calculations demonstrate UOTe as an AFM Dirac semimetal. Within the AFM order, we observed the presence of the Kondo interaction, as evidenced by the emergence of a 5$f$ flat band near the Fermi level below 100 K and hybridization between the Kondo band and the Dirac band. Our density functional theory calculations in its bilayer form predict UOTe as a rare example of a fully-compensated AFM Chern insulator., Comment: Accepted to Advanced Materials

In this paper, we developed a spectral emulator based on the Mapping Nearby Galaxies at Apache Point Observatory Stellar Library (MaStar) and a grouping optimization strategy to estimate effective temperature (T_eff), surface gravity (log g), metallicity ([Fe/H]) and the abundance of alpha elements with respect to iron ([alpha/Fe]) for O-M-type stars within the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) low-resolution spectra. The primary aim is to use a rapid spectral-fitting method, specifically the spectral emulator with the grouping optimization strategy, to create a comprehensive catalog for stars of all types within LAMOST, addressing the shortcomings in parameter estimations for both cold and hot stars present in the official LAMOST AFGKM-type catalog. This effort is part of our series of studies dedicated to establishing an empirical spectral library for LAMOST. Experimental results demonstrate that our method is effectively applicable to parameter prediction for LAMOST, with the single-machine processing time within $70$ hr. We observed that the internal error dispersions for T_eff, log g, [Fe/H], and [alpha/Fe] across different spectral types lie within the ranges of $15-594$ K, $0.03-0.27$ dex, $0.02-0.10$ dex, and $0.01-0.04$ dex, respectively, indicating a good consistency. A comparative analysis with external data highlighted deficiencies in the official LAMOST catalog and issues with MaStar parameters, as well as potential limitations of our method in processing spectra with strong emission lines and bad pixels. The derived atmospheric parameters as a part of this work are available at https://nadc.china-vo.org/res/r101402/ ., Comment: 34 pages, 28 Figures, published in ApJS

Subwavelength resonance is a vital acoustic phenomenon in contrasting media. The narrow bandgap width of single-layer resonator has prompted the exploration of multi-layer metamaterials as an effective alternative, which consist of alternating nests of high-contrast materials, called ``resonators'', and a background media. In this paper, we develop a general mathematical framework for studying acoustics within multi-layer high-contrast structures. Firstly, by using layer potential techniques, we establish the representation formula in terms of a matrix type operator with a block tridiagonal form for multi-layer structures within general geometry. Then we prove the existence of subwavelength resonances via Gohberg-Sigal theory, which generalizes the celebrated Minnaert resonances in single-layer structures. Intriguingly, we find that the primary contribution to mode splitting lies in the fact that as the number of nested resonators increases, the degree of the corresponding characteristic polynomial also increases, while the type of resonance (consists solely of monopolar resonances) remains unchanged. Furthermore, we derive original formulas for the subwavelength resonance frequencies of concentric dual-resonator. Numerical results associated with different nested resonators are presented to corroborate the theoretical findings.

Assessment of the sympathetic nervous system (SNS) is one of the major approaches for studying affective states. Skin nerve activity (SKNA) derived from high-frequency components of electrocardiogram (ECG) signals has been a promising surrogate for assessing the SNS. However, current SKNA analysis tools have shown high variability across study protocols and experiments. Hence, we propose a time-varying spectral approach based on SKNA to assess the SNS with higher sensitivity and reliability. We collected ECG signals at a sampling frequency of 10 KHz from sixteen subjects who underwent various SNS stimulations. Our spectral analysis revealed that frequency bands between 150 - 1,000 Hz showed significant increases in power during SNS stimulations. Using this information, we developed a time-varying index of sympathetic function measurement based on SKNA, termed, Time-Varying Skin Nerve Activity (TVSKNA). TVSKNA is calculated in three steps: time-frequency decomposition, reconstruction using selected frequency bands, and smoothing. TVSKNA indices exhibited generally higher Youden's J, balanced accuracy, and area under the receiver operating characteristic curve, indicating higher sensitivity. The coefficient of variance was lower with TVSKNA indices for most SNS tasks. TVSKNA can serve as a highly sensitive and reliable marker of quantitative assessment of sympathetic function, especially during emotion and stress., Comment: Submitted to IEEE Transactions on Affective Computing for publication consideration

The advanced role-playing capabilities of Large Language Models (LLMs) have paved the way for developing Role-Playing Agents (RPAs). However, existing benchmarks, such as HPD, which incorporates manually scored character relationships into the context for LLMs to sort coherence, and SocialBench, which uses specific profiles generated by LLMs in the context of multiple-choice tasks to assess character preferences, face limitations like poor generalizability, implicit and inaccurate judgments, and excessive context length. To address the above issues, we propose an automatic, scalable, and generalizable paradigm. Specifically, we construct a benchmark by extracting relations from a general knowledge graph and leverage RPA's inherent hallucination properties to prompt it to interact across roles, employing ChatGPT for stance detection and defining relationship hallucination along with three related metrics. Extensive experiments validate the effectiveness and stability of our metrics. Our findings further explore factors influencing these metrics and discuss the trade-off between relationship hallucination and factuality.

The erratic nature of chaotic behavior is thought to erode the stability of periodic behavior, including topological oscillations. However, we discover that in the presence of chaos, non-trivial topology not only endures but also provides robust protection to chaotic dynamics within a topological lattice hosting non-linear oscillators. Despite the difficulty in defining topological invariants in non-linear settings, non-trivial topological robustness still persists in the parametric state of chaotic boundary oscillations. We demonstrate this interplay between chaos and topology by incorporating chaotic Chua's circuits into a topological Su-Schrieffer-Heeger (SSH) circuit. By extrapolating from the linear limit to deep into the non-linear regime, we find that distinctive correlations in the bulk and edge scroll dynamics effectively capture the topological origin of the protected chaos. Our findings suggest that topologically protected chaos can be robustly achieved across a broad spectrum of periodically-driven systems, thereby offering new avenues for the design of resilient and adaptable non-linear networks., Comment: 13 figures, 23 pages

We investigate size variation with rest-frame wavelength for star-forming galaxies based on the second JWST Advanced Deep Extragalactic Survey data release. Star-forming galaxies are typically smaller at longer wavelength from UV-to-NIR at $z<3.5$, especially for more massive galaxies, indicating the inside-out assembly with in-situ star formation if ignoring dust attenuation. The size variation with wavelength shows strong dependence on stellar mass, and shows little or no dependence on redshift, specific star formation rate and galaxy environment. This suggests that the size growth of star-forming galaxies is a self-regulated process primarily governed by stellar mass. We model size as a function of both mass and redshift simultaneously, obtaining $R_{\rm e} \propto M_*^{0.23} (1+z)^{-1.04}$ at a wavelength of 0.45 ${\mu \mathrm{m}}$, and $R_{\rm e} \propto M_*^{0.20} (1+z)^{-1.08}$ at 1.0 ${\mu \mathrm{m}}$. Based on this size evolution and the star formation main sequence from the literature, we obtain the locus of typical size growth for individual galaxies of different masses on the mass-size plane. The moving trend of galaxies on the mass-size plane, which indicates the slopes of their locus, strongly correlates with the size ratio between 0.45 ${\mu \mathrm{m}}$ and 1.0 ${\mu \mathrm{m}}$, supporting the idea that the size variation with wavelength provides important information on size growth of galaxies on short timescales., Comment: Accepted for publication in ApJ, 19 pages, 11 figures

With rapid evolution of mobile core network (MCN) architectures, large-scale control-plane traffic (CPT) traces are critical to studying MCN design and performance optimization by the R&D community. The prior-art control-plane traffic generator SMM heavily relies on domain knowledge which requires re-design as the domain evolves. In this work, we study the feasibility of developing a high-fidelity MCN control plane traffic generator by leveraging generative ML models. We identify key challenges in synthesizing high-fidelity CPT including generic (to data-plane) requirements such as multimodality feature relationships and unique requirements such as stateful semantics and long-term (time-of-day) data variations. We show state-of-the-art, generative adversarial network (GAN)-based approaches shown to work well for data-plane traffic cannot meet these fidelity requirements of CPT, and develop a transformer-based model, CPT-GPT, that accurately captures complex dependencies among the samples in each traffic stream (control events by the same UE) without the need for GAN. Our evaluation of CPT-GPT on a large-scale control-plane traffic trace shows that (1) it does not rely on domain knowledge yet synthesizes control-plane traffic with comparable fidelity as SMM; (2) compared to the prior-art GAN-based approach, it reduces the fraction of streams that violate stateful semantics by two orders of magnitude, the max y-distance of sojourn time distributions of streams by 16.0%, and the transfer learning time in deriving new hourly models by 3.36x.

We introduce the GEM (Generative Estimator for Mutual Information), an evaluation metric for assessing language generation by Large Language Models (LLMs), particularly in generating informative judgments, without the need for a gold standard reference. GEM broadens the scenarios where we can benchmark LLM generation performance-from traditional ones, like machine translation and summarization, where gold standard references are readily available, to subjective tasks without clear gold standards, such as academic peer review. GEM uses a generative model to estimate mutual information between candidate and reference responses, without requiring the reference to be a gold standard. In experiments on a human-annotated dataset, GEM demonstrates competitive correlations with human scores compared to the state-of-the-art GPT-4o Examiner, and outperforms all other baselines. Additionally, GEM is more robust against strategic manipulations, such as rephrasing or elongation, which can artificially inflate scores under a GPT-4o Examiner. We also present GRE-bench (Generating Review Evaluation Benchmark) which evaluates LLMs based on how well they can generate high-quality peer reviews for academic research papers. Because GRE-bench is based upon GEM, it inherits its robustness properties. Additionally, GRE-bench circumvents data contamination problems (or data leakage) by using the continuous influx of new open-access research papers and peer reviews each year. We show GRE-bench results of various popular LLMs on their peer review capabilities using the ICLR2023 dataset.

This paper presents DeepCRF, a new framework that harnesses deep learning to extract subtle micro-signals from channel state information (CSI) measurements, enabling robust and resilient radio-frequency fingerprinting (RFF) of commercial-off-the-shelf (COTS) WiFi devices across diverse channel conditions. Building on our previous research, which demonstrated that micro-signals in CSI, termed micro-CSI, most likely originate from RF circuitry imperfections and can serve as unique RF fingerprints, we develop a new approach to overcome the limitations of our prior signal space-based method. While the signal space-based method is effective in strong line-of-sight (LoS) conditions, we show that it struggles with the complexities of non-line-of-sight (NLoS) scenarios, compromising the robustness of CSI-based RFF. To address this challenge, DeepCRF incorporates a carefully trained convolutional neural network (CNN) with model-inspired data augmentation, supervised contrastive learning, and decision fusion techniques, enhancing its generalization capabilities across unseen channel conditions and resilience against noise. Our evaluations demonstrate that DeepCRF significantly improves device identification accuracy across diverse channels, outperforming both the signal space-based baseline and state-of-the-art neural network-based benchmarks. Notably, it achieves an average identification accuracy of 99.53% among 19 COTS WiFi network interface cards in real-world unseen scenarios using 4 CSI measurements per identification procedure.

Prevalent in biological applications (e.g., human phenotype measurements), multimodal datasets can provide valuable insights into the underlying biological mechanisms. However, current machine learning models designed to analyze such datasets still lack interpretability and theoretical guarantees, which are essential to biological applications. Recent advances in causal representation learning have shown promise in uncovering the interpretable latent causal variables with formal theoretical certificates. Unfortunately, existing works for multimodal distributions either rely on restrictive parametric assumptions or provide rather coarse identification results, limiting their applicability to biological research which favors a detailed understanding of the mechanisms. In this work, we aim to develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biological datasets. Theoretically, we consider a flexible nonparametric latent distribution (c.f., parametric assumptions in prior work) permitting causal relationships across potentially different modalities. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities, which, as we will discuss, is natural for a large collection of biological systems. Empirically, we propose a practical framework to instantiate our theoretical insights. We demonstrate the effectiveness of our approach through extensive experiments on both numerical and synthetic datasets. Results on a real-world human phenotype dataset are consistent with established medical research, validating our theoretical and methodological framework.

Immense effort has been dedicated to minimizing the presence of harmful or biased generative content and better aligning AI output to human intention; however, research investigating the cultural values of LLMs is still in very early stages. Cultural values underpin how societies operate, providing profound insights into the norms, priorities, and decision making of their members. In recognition of this need for further research, we draw upon cultural psychology theory and the empirically-validated GLOBE framework to propose the LLM-GLOBE benchmark for evaluating the cultural value systems of LLMs, and we then leverage the benchmark to compare the values of Chinese and US LLMs. Our methodology includes a novel "LLMs-as-a-Jury" pipeline which automates the evaluation of open-ended content to enable large-scale analysis at a conceptual level. Results clarify similarities and differences that exist between Eastern and Western cultural value systems and suggest that open-generation tasks represent a more promising direction for evaluation of cultural values. We interpret the implications of this research for subsequent model development, evaluation, and deployment efforts as they relate to LLMs, AI cultural alignment more broadly, and the influence of AI cultural value systems on human-AI collaboration outcomes.

Autoregressive modeling has been a huge success in the field of natural language processing (NLP). Recently, autoregressive models have emerged as a significant area of focus in computer vision, where they excel in producing high-quality visual content. Autoregressive models in NLP typically operate on subword tokens. However, the representation strategy in computer vision can vary in different levels, \textit{i.e.}, pixel-level, token-level, or scale-level, reflecting the diverse and hierarchical nature of visual data compared to the sequential structure of language. This survey comprehensively examines the literature on autoregressive models applied to vision. To improve readability for researchers from diverse research backgrounds, we start with preliminary sequence representation and modeling in vision. Next, we divide the fundamental frameworks of visual autoregressive models into three general sub-categories, including pixel-based, token-based, and scale-based models based on the strategy of representation. We then explore the interconnections between autoregressive models and other generative models. Furthermore, we present a multi-faceted categorization of autoregressive models in computer vision, including image generation, video generation, 3D generation, and multi-modal generation. We also elaborate on their applications in diverse domains, including emerging domains such as embodied AI and 3D medical AI, with about 250 related references. Finally, we highlight the current challenges to autoregressive models in vision with suggestions about potential research directions. We have also set up a Github repository to organize the papers included in this survey at: \url{https://github.com/ChaofanTao/Autoregressive-Models-in-Vision-Survey}.

We consider quantum circuit models where the gates are drawn from arbitrary gate ensembles given by probabilistic distributions over certain gate sets and circuit architectures, which we call stochastic quantum circuits. Of main interest in this work is the speed of convergence of stochastic circuits with different gate ensembles and circuit architectures to unitary t-designs. A key motivation for this theory is the varying preference for different gates and circuit architectures in different practical scenarios. In particular, it provides a versatile framework for devising efficient circuits for implementing $t$-designs and relevant applications including random circuit and scrambling experiments, as well as benchmarking the performance of gates and circuit architectures. We examine various important settings in depth. A key aspect of our study is an "ironed gadget" model, which allows us to systematically evaluate and compare the convergence efficiency of entangling gates and circuit architectures. Particularly notable results include i) gadgets of two-qubit gates with KAK coefficients $\left(\frac{\pi}{4}-\frac{1}{8}\arccos(\frac{1}{5}),\frac{\pi}{8},\frac{1}{8}\arccos(\frac{1}{5})\right)$ (which we call $\chi$ gates) directly form exact 2- and 3-designs; ii) the iSWAP gate family achieves the best efficiency for convergence to 2-designs under mild conjectures with numerical evidence, even outperforming the Haar-random gate, for generic many-body circuits; iii) iSWAP + complete graph achieve the best efficiency for convergence to 2-designs among all graph circuits. A variety of numerical results are provided to complement our analysis. We also derive robustness guarantees for our analysis against gate perturbations. Additionally, we provide cursory analysis on gates with higher locality and found that the Margolus gate outperforms various other well-known gates., Comment: 50 pages + 8 tables + 6 figures

Let $S$ be a finite set of pairwise coprime positive integers and $Ax^2+Bx$ be an integer valued polynomial with $A> B\ge 0$. For integers $k\ge 1$ and $n\ge 0$, the coefficients $\gamma_{S,A,B}^k (n)$ are defined as \begin{align*} \prod_{s\in S}\frac{1}{1-q^s}\sum_{j\not\in [-k,k-1]} (-1)^{j+k}q^{Aj^2+Bj}=\sum_{n= 0}^{\infty}\gamma_{S,A,B}^k (n)q^n. \end{align*} In this paper, we investigate the positivity of $\gamma_{S,A,B}^k (n)$ for $|S|=4,5$., Comment: 20 pages

We propose a novel technique for optimizing a modular fault-tolerant quantum computing architecture, taking into account any desired space-time trade--offs between the number of physical qubits and the fault-tolerant execution time of a quantum algorithm. We consider a concept architecture comprising a dedicated zone as a multi-level magic state factory and a core processor for efficient logical operations, forming a supply chain network for production and consumption of magic states. Using a heuristic algorithm, we solve the multi-objective optimization problem of minimizing space and time subject to a user-defined error budget for the success of the computation, taking the performance of various fault-tolerant protocols such as quantum memory, state preparation, magic state distillation, code growth, and logical operations into account. As an application, we show that physical quantum resource estimation reduces to a simple model involving a small number of key parameters, namely, the circuit volume, the error prefactors ($\mu$) and error suppression rates ($\Lambda$) of the fault-tolerant protocols, and an allowed slowdown factor ($\beta$). We show that, in the proposed architecture, $10^5$--$10^8$ physical qubits are required for quantum algorithms with $T$-counts in the range $10^6$--$10^{15}$ and logical qubit counts in the range $10^2$--$10^4$, when run on quantum computers with quantum memory $\Lambda$ in the range 3--10, for all slowdown factors $\beta \geq 0.2$., Comment: 21 pages, 6 figures

Traditional interactive environments limit agents' intelligence growth with fixed tasks. Recently, single-agent environments address this by generating new tasks based on agent actions, enhancing task diversity. We consider the decision-making problem in multi-agent settings, where tasks are further influenced by social connections, affecting rewards and information access. However, existing multi-agent environments lack a combination of adaptive physical surroundings and social connections, hindering the learning of intelligent behaviors. To address this, we introduce AdaSociety, a customizable multi-agent environment featuring expanding state and action spaces, alongside explicit and alterable social structures. As agents progress, the environment adaptively generates new tasks with social structures for agents to undertake. In AdaSociety, we develop three mini-games showcasing distinct social structures and tasks. Initial results demonstrate that specific social structures can promote both individual and collective benefits, though current reinforcement learning and LLM-based algorithms show limited effectiveness in leveraging social structures to enhance performance. Overall, AdaSociety serves as a valuable research platform for exploring intelligence in diverse physical and social settings. The code is available at https://github.com/bigai-ai/AdaSociety., Comment: Accepted at NeurIPS D&B 2024

Given $\beta>1$, let $T_\beta$ be the $\beta$-transformation on the unit circle $[0,1)$, defined by $T_\beta(x)=\beta x\pmod 1$. For each $t\in[0,1)$ let $K_\beta(t)$ be the survivor set consisting of all $x\in[0,1)$ whose orbit $\{T^n_\beta(x): n\ge 0\}$ never hits the interval $[0,t)$. Kalle et al.~[{\em Ergodic Theory Dynam. Systems} {\bf 40} (2020), no.~9, 2482--2514] considered the case $\beta\in(1,2]$. They studied the set-valued bifurcation set $\mathscr{E}_\beta:=\{t\in[0,1): K_\beta(t')\ne K_\beta(t)~\forall t'>t\}$ and proved that the Hausdorff dimension function $t\mapsto\dim_H K_\beta(t)$ is a non-increasing Devil's staircase. In a previous paper [{\em Ergodic Theory Dynam. Systems} {\bf 43} (2023), no.~6, 1785--1828] we determined, for all $\beta\in(1,2]$, the critical value $\tau(\beta):=\min\{t>0: \eta_\beta(t)=0\}$. The purpose of the present article is to extend these results to all $\beta>1$. In addition to calculating $\tau(\beta)$, we show that (i) the function $\tau: \beta\mapsto\tau(\beta)$ is left continuous on $(1,\infty)$ with right-hand limits everywhere, but has countably infinitely many discontinuities; (ii) $\tau$ has no downward jumps; and (iii) there exists an open set $O\subset(1,\infty)$, whose complement $(1,\infty)\backslash O$ has zero Hausdorff dimension, such that $\tau$ is real-analytic, strictly convex and strictly decreasing on each connected component of $O$. We also prove several topological properties of the bifurcation set $\mathscr{E}_\beta$. The key to extending the results from $\beta\in(1,2]$ to all $\beta>1$ is an appropriate generalization of the Farey words that are used to parametrize the connected components of the set $O$. Some of the original proofs from the above-mentioned papers are simplified., Comment: 22 pages, 2 figures

The enumeration of polymer coverings on two-dimensional rectangular lattices is considered as "intractable". We prove that the number of coverings of $s$ polymer satisfies a simple recurrence relation $ \sum_{i=0}^{2s} (-1)^i \binom{2s}{i} a_{n-i, m-i} = 2^s {(2s)!} / {s!} $ on a $n \times m$ rectangular lattice with open boundary conditions in both directions.

We report the design-technology co-optimization (DTCO) scheme to develop a 28-nm cryogenic CMOS (Cryo-CMOS) technology for high-performance computing (HPC). The precise adjustment of halo implants manages to compensate the threshold voltage (VTH) shift at low temperatures. The optimized NMOS and PMOS transistors, featured by VTH<0.2V, sub-threshold swing (SS)<30 mV/dec, and on-state current (Ion)>1.2mA/um at 77K, warrant a reliable sub-0.6V operation. Moreover, the enhanced driving strength of Cryo-CMOS inherited from a higher transconductance leads to marked improvements in elevating the ring oscillator frequency by 20%, while reducing the power consumption of the compute-intensive cryogenic IC system by 37% at 77K.

This document is comprised of a collection of updated preliminary parameters for the key parts of the muon collider. The updated preliminary parameters follow on from the October 2023 Tentative Parameters Report. Particular attention has been given to regions of the facility that are believed to hold greater technical uncertainty in their design and that have a strong impact on the cost and power consumption of the facility. The data is collected from a collaborative spreadsheet and transferred to overleaf.

Deep neural networks (DNNs) have recently achieved impressive success across a wide range of real-world vision and language processing tasks, spanning from image classification to many other downstream vision tasks, such as object detection, tracking, and segmentation. However, previous well-established DNNs, despite being able to maintain superior accuracy, have also been evolving to be deeper and wider and thus inevitably necessitate prohibitive computational resources for both training and inference. This trend further enlarges the computational gap between computation-intensive DNNs and resource-constrained embedded computing systems, making it challenging to deploy powerful DNNs upon real-world embedded computing systems towards ubiquitous embedded intelligence. To alleviate the above computational gap and enable ubiquitous embedded intelligence, we, in this survey, focus on discussing recent efficient deep learning infrastructures for embedded computing systems, spanning from training to inference, from manual to automated, from convolutional neural networks to transformers, from transformers to vision transformers, from vision models to large language models, from software to hardware, and from algorithms to applications. Specifically, we discuss recent efficient deep learning infrastructures for embedded computing systems from the lens of (1) efficient manual network design for embedded computing systems, (2) efficient automated network design for embedded computing systems, (3) efficient network compression for embedded computing systems, (4) efficient on-device learning for embedded computing systems, (5) efficient large language models for embedded computing systems, (6) efficient deep learning software and hardware for embedded computing systems, and (7) efficient intelligent applications for embedded computing systems., Comment: ACM Transactions on Embedded Computing Systems (TECS) 2024

Recent advancements in computational chemistry have leveraged the power of trans-former-based language models, such as MoLFormer, pre-trained using a vast amount of simplified molecular-input line-entry system (SMILES) sequences, to understand and predict molecular properties and activities, a critical step in fields like drug discovery and materials science. To further improve performance, researchers have introduced graph neural networks with graph-based molecular representations, such as GEM, incorporating the topology, geometry, 2D or even 3D structures of molecules into pre-training. While most of molecular graphs in existing studies were automatically converted from SMILES sequences, it is to assume that transformer-based language models might be able to implicitly learn structure-aware representations from SMILES sequences. In this paper, we propose \ours{} -- a SMILES-based \underline{\em M}olecular \underline{\em L}anguage \underline{\em M}odel, which randomly masking SMILES subsequences corresponding to specific molecular \underline{\em F}unctional \underline{\em G}roups to incorporate structure information of atoms during the pre-training phase. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities. Extensive experimental evaluations across 11 benchmark classification and regression tasks in the chemical domain demonstrate the robustness and superiority of \ours{}. Our findings reveal that \ours{} outperforms existing pre-training models, either based on SMILES or graphs, in 9 out of the 11 downstream tasks, ranking as a close second in the remaining ones., Comment: Under review

Generating the vortex-state superposition in a matter wave is demanded in many quantum processes such as quantum memory and quantum metrology. Here we report the experimental generation of macroscopic superposition of vortex states in ultracold quantum gases. By transferring an optical vortex-state superposition to the center-of-mass rotational state of ultracold atoms using the Raman coupling technique, we realize two-vortex and three-vortex superposition states in quantum gases, demonstrating the high dimensionality of the vortex state. We show the controllability of the superposition states on the Bloch sphere. The lifetime of the vortex superposition state in quantum gases is as large as 25 ms, about two orders of magnitude longer than the storage time in atomic ensembles. This work paves the way for high dimensional quantum processing in matter waves., Comment: 17 pages, 12 figures

The rapid progress in artificial intelligence-generated content (AIGC), especially with diffusion models, has significantly advanced development of high-quality video generation. However, current video diffusion models exhibit demanding computational requirements and high peak memory usage, especially for generating longer and higher-resolution videos. These limitations greatly hinder the practical application of video diffusion models on standard hardware platforms. To tackle this issue, we present a novel, training-free framework named Streamlined Inference, which leverages the temporal and spatial properties of video diffusion models. Our approach integrates three core components: Feature Slicer, Operator Grouping, and Step Rehash. Specifically, Feature Slicer effectively partitions input features into sub-features and Operator Grouping processes each sub-feature with a group of consecutive operators, resulting in significant memory reduction without sacrificing the quality or speed. Step Rehash further exploits the similarity between adjacent steps in diffusion, and accelerates inference through skipping unnecessary steps. Extensive experiments demonstrate that our approach significantly reduces peak memory and computational overhead, making it feasible to generate high-quality videos on a single consumer GPU (e.g., reducing peak memory of AnimateDiff from 42GB to 11GB, featuring faster inference on 2080Ti)., Comment: Accepted to NeurIPS 2024

Large Multimodal Models (LMMs) have shown significant progress in various complex vision tasks with the solid linguistic and reasoning capacity inherited from large language models (LMMs). Low-rank adaptation (LoRA) offers a promising method to integrate external knowledge into LMMs, compensating for their limitations on domain-specific tasks. However, the existing LoRA model serving is excessively computationally expensive and causes extremely high latency. In this paper, we present an end-to-end solution that empowers diverse vision tasks and enriches vision applications with LoRA LMMs. Our system, VaLoRA, enables accurate and efficient vision tasks by 1) an accuracy-aware LoRA adapter generation approach that generates LoRA adapters rich in domain-specific knowledge to meet application-specific accuracy requirements, 2) an adaptive-tiling LoRA adapters batching operator that efficiently computes concurrent heterogeneous LoRA adapters, and 3) a flexible LoRA adapter orchestration mechanism that manages application requests and LoRA adapters to achieve the lowest average response latency. We prototype VaLoRA on five popular vision tasks on three LMMs. Experiment results reveal that VaLoRA improves 24-62% of the accuracy compared to the original LMMs and reduces 20-89% of the latency compared to the state-of-the-art LoRA model serving systems.

Accurate remaining useful life (RUL) predictions are critical to the safe operation of aero-engines. Currently, the RUL prediction task is mainly a regression paradigm with only mean square error as the loss function and lacks research on feature space structure, the latter of which has shown excellent performance in a large number of studies. This paper develops a multi-granularity supervised contrastive (MGSC) framework from plain intuition that samples with the same RUL label should be aligned in the feature space, and address the problems of too large minibatch size and unbalanced samples in the implementation. The RUL prediction with MGSC is implemented on using the proposed multi-phase training strategy. This paper also demonstrates a simple and scalable basic network structure and validates the proposed MGSC strategy on the CMPASS dataset using a convolutional long short-term memory network as a baseline, which effectively improves the accuracy of RUL prediction.

Nonreciprocal thermophotonics, by breaking Lorentz reciprocity, exceeds current theoretical efficiency limits, unlocking opportunities to energy devices and thermal management. However, energy transfer in current systems is highly defect-sensitive. This sensitivity is further amplified at deep subwavelength scales by inevitable multi-source interactions, interface wrinkles, and manufacturing tolerances, making precise control of thermal photons increasingly challenging. Here, we demonstrate a topological one-way heat transport in a deep-subwavelength thermophotonic lattice. This one-way heat flow, driven by global resonances, is strongly localized at the geometric boundaries and exhibits exceptional robustness against imperfections and disorder, achieving nearly five orders of radiative enhancement. Our findings offer a blueprint for developing robust thermal systems capable of withstanding strong perturbations., Comment: 15 pages, 4 figures

We report a detailed timing and spectral analysis of the X-ray pulsar 2S~1417-624 using the data from Insight-HXMT during the 2018 outburst. The pulse profiles are highly variable with respect to both unabsorbed flux and energy. A double-peaked pulse profile from the low flux evolved to a multi-peaked shape in the high-flux state. The pulse fraction is negatively correlated to the source flux in the range of $\sim$(1--6)$\ \times \ 10^{-9}$ erg cm$^{-2}$ s$^{-1}$, consistent with \textit{Rossi} X-ray Timing Explorer (RXTE) studies during the 2009 giant outburst. The energy-resolved pulse profiles around the peak outburst showed a four-peak shape in the low-energy bands and gradually evolved to triple peaks at higher energies. The continuum spectrum is well described by typical phenomenological models, such as the cut-off power law and the power law with high-energy cut-off models. Notably, we discovered high-energy cyclotron resonant scattering features (CRSFs) for the first time, which are around 100 keV with a statistical significance of $\sim$7$\sigma$ near the peak luminosity of the outburst. This CRSF line is significantly detected with different continuum models and provides very robust evidence for its presence. Furthermore, pulse-phase-resolved spectroscopy confirmed the presence of the line, whose energy varied from 97 to 107 keV over the pulse phase and appeared to have a maximum value at the narrow peak phase of the profiles., Comment: Accepted for publication in Astronomy & Astrophysics, 10 pages, 10 figures and 2 tables

In recent years, quantum computing has drawn significant interest within the field of high-energy physics. We explore the potential of quantum algorithms to resolve the combinatorial problems in particle physics experiments. As a concrete example, we consider top quark pair production in the fully hadronic channel at the Large Hadron Collider. We investigate the performance of various quantum algorithms such as the Quantum Approximation Optimization Algorithm (QAOA) and a feedback-based algorithm (FALQON). We demonstrate that the efficiency for selecting the correct pairing is greatly improved by utilizing quantum algorithms over conventional kinematic methods. Furthermore, we observe that gate-based universal quantum algorithms perform on par with machine learning techniques and either surpass or match the effectiveness of quantum annealers. Our findings reveal that quantum algorithms not only provide a substantial increase in matching efficiency but also exhibit scalability and adaptability, making them suitable for a variety of high-energy physics applications. Moreover, quantum algorithms eliminate the extensive training processes needed by classical machine learning methods, enabling real-time adjustments based on individual event data., Comment: 19 pages, 18 figures, 1 table

We report a time-resolved analysis of the accreting X-ray pulsar Cen X-3 using observations carried out by NuSTAR, which cover approximately two binary orbits in different intensity states. The pulse profile is relatively stable over the orbital phase and shows energy dependence. It has an obvious double-peaked shape in the energy band below 15 keV -- with the second pulse peak decreasing as energy increases -- and is gradually dominated by a single peak in higher energy bands. We find that the pulse profile in the energy band of 3-5 keV at high-intensity states shows a subtle triple-peaked shape, with the main peak divided into two subpeaks. We also find a positive correlation between the pulse fraction and both energy and flux. Our spectral analysis reveals that the spectra can be well described by the continuum of Fermi-Dirac cutoff and NPEX models, and the cyclotron line is detected with the centroid energies varying from 26 keV to 29 keV, along with the iron emission line around 6.4 keV. We investigated the dependence between the cyclotron resonant scattering feature (CRSF) centroid energy and luminosity and discuss the theoretical critical luminosity. Although the variation of $E_{\rm cyc}- L_X$ is not distinct, there is a possibility that the critical luminosity lies within the range of $\sim (0.5-4)\times 10^{37}$ erg s$^{-1}$ in the band of $4-78$ keV. The photon index shows a strong positive correlation with luminosity. Our orbital-phase analysis reveals that the spectral parameters show orbital variability, and the highly variable photoelectric absorption may indicate the existence of clumpy stellar winds., Comment: 10 pages, 10 figures

Spike train classification has recently become an important topic in the machine learning community, where each spike train is a binary event sequence with \emph{temporal-sparsity of signals of interest} and \emph{temporal-noise} properties. A promising model for it should follow the design principle of performing intensive computation only when signals of interest appear. So such tasks use mainly Spiking Neural Networks (SNNs) due to their consideration of temporal-sparsity of spike trains. However, the basic mechanism of SNNs ignore the temporal-noise issue, which makes them computationally expensive and thus high power consumption for analyzing spike trains on resource-constrained platforms. As an event-driven model, an SNN neuron makes a reaction given any input signals, making it difficult to quickly find signals of interest. In this paper, we introduce an event-attention mechanism that enables SNNs to dynamically highlight useful signals of the original spike trains. To this end, we propose SkipSNN, which extends existing SNN models by learning to mask out noise by skipping membrane potential updates and shortening the effective size of the computational graph. This process is analogous to how people choose to open and close their eyes to filter the information they see. We evaluate SkipSNN on various neuromorphic tasks and demonstrate that it achieves significantly better computational efficiency and classification accuracy than other state-of-the-art SNNs., Comment: Published as a research paper at IEEE BigData 2024

Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.

LLMs have emerged as a promising tool for assisting individuals in diverse text-generation tasks, including job-related texts. However, LLM-generated answers have been increasingly found to exhibit gender bias. This study evaluates three LLMs (GPT-3.5, GPT-4, Claude) to conduct a multifaceted audit of LLM-generated interview responses across models, question types, and jobs, and their alignment with two gender stereotypes. Our findings reveal that gender bias is consistent, and closely aligned with gender stereotypes and the dominance of jobs. Overall, this study contributes to the systematic examination of gender bias in LLM-generated interview responses, highlighting the need for a mindful approach to mitigate such biases in related applications., Comment: Accepted to NeurlIPS 2024, SoLaR workshop

We report the discovery of a peculiar X-ray transient, EP240408a, by Einstein Probe (EP) and follow-up studies made with EP, Swift, NICER, GROND, ATCA and other ground-based multi-wavelength telescopes. The new transient was first detected with Wide-field X-ray Telescope (WXT) on board EP on April 8th, 2024, manifested in an intense yet brief X-ray flare lasting for 12 seconds. The flare reached a peak flux of 3.9x10^(-9) erg/cm2/s in 0.5-4 keV, about 300 times brighter than the underlying X-ray emission detected throughout the observation. Rapid and more precise follow-up observations by EP/FXT, Swift and NICER confirmed the finding of this new transient. Its X-ray spectrum is non-thermal in 0.5-10 keV, with a power-law photon index varying within 1.8-2.5. The X-ray light curve shows a plateau lasting for about 4 days, followed by a steep decay till becoming undetectable about 10 days after the initial detection. Based on its temporal property and constraints from previous EP observations, an unusual timescale in the range of 7-23 days is found for EP240408a, which is intermediate between the commonly found fast and long-term transients. No counterparts have been found in optical and near-infrared, with the earliest observation at 17 hours after the initial X-ray detection, suggestive of intrinsically weak emission in these bands. We demonstrate that the remarkable properties of EP240408a are inconsistent with any of the transient types known so far, by comparison with, in particular, jetted tidal disruption events, gamma-ray bursts, X-ray binaries and fast blue optical transients. The nature of EP240408a thus remains an enigma. We suggest that EP240408a may represent a new type of transients with intermediate timescales of the order of about 10 days. The detection and follow-ups of more of such objects are essential for revealing their origin., Comment: 25 pages, 11 figures

Diffusion bridge models effectively facilitate image-to-image (I2I) translation by connecting two distributions. However, existing methods overlook the impact of noise in sampling SDEs, transition kernel, and the base distribution on sampling efficiency, image quality and diversity. To address this gap, we propose the Stochasticity-controlled Diffusion Bridge (SDB), a novel theoretical framework that extends the design space of diffusion bridges, and provides strategies to mitigate singularities during both training and sampling. By controlling stochasticity in the sampling SDEs, our sampler achieves speeds up to 5 times faster than the baseline, while also producing lower FID scores. After training, SDB sets new benchmarks in image quality and sampling efficiency via managing stochasticity within the transition kernel. Furthermore, introducing stochasticity into the base distribution significantly improves image diversity, as quantified by a newly introduced metric., Comment: 23 pages, 9 figures

Stochastic differential equations are commonly used to describe the evolution of stochastic processes. The uncertainty of such processes is best represented by the probability density function (PDF), whose evolution is governed by the Fokker-Planck partial differential equation (FP-PDE). However, it is generally infeasible to solve the FP-PDE in closed form. In this work, we show that physics-informed neural networks (PINNs) can be trained to approximate the solution PDF using existing methods. The main contribution is the analysis of the approximation error: we develop a theory to construct an arbitrary tight error bound with PINNs. In addition, we derive a practical error bound that can be efficiently constructed with existing training methods. Finally, we explain that this error-bound theory generalizes to approximate solutions of other linear PDEs. Several numerical experiments are conducted to demonstrate and validate the proposed methods., Comment: pre-print under review

Computational protein design (CPD) refers to the use of computational methods to design proteins. Traditional methods relying on energy functions and heuristic algorithms for sequence design are inefficient and do not meet the demands of the big data era in biomolecules, with their accuracy limited by the energy functions and search algorithms. Existing deep learning methods are constrained by the learning capabilities of the networks, failing to extract effective information from sparse protein structures, which limits the accuracy of protein design. To address these shortcomings, we developed an Efficient attention-based Models for Computational Protein Design using amino acid microenvironment (EMOCPD). It aims to predict the category of each amino acid in a protein by analyzing the three-dimensional atomic environment surrounding the amino acids, and optimize the protein based on the predicted high-probability potential amino acid categories. EMOCPD employs a multi-head attention mechanism to focus on important features in the sparse protein microenvironment and utilizes an inverse residual structure to optimize the network architecture. The proposed EMOCPD achieves over 80% accuracy on the training set and 68.33% and 62.32% accuracy on two independent test sets, respectively, surpassing the best comparative methods by over 10%. In protein design, the thermal stability and protein expression of the predicted mutants from EMOCPD show significant improvements compared to the wild type, effectively validating EMOCPD's potential in designing superior proteins. Furthermore, the predictions of EMOCPD are influenced positively, negatively, or have minimal impact based on the content of the 20 amino acids, categorizing amino acids as positive, negative, or neutral. Research findings indicate that EMOCPD is more suitable for designing proteins with lower contents of negative amino acids.

We investigate the effectiveness of a momentum trading signal based on the coverage network of financial analysts. This signal builds on the key information-brokerage role financial sell-side analysts play in modern stock markets. The baskets of stocks covered by each analyst can be used to construct a network between firms whose edge weights represent the number of analysts jointly covering both firms. Although the link between financial analysts coverage and co-movement of firms' stock prices has been investigated in the literature, little effort has been made to systematically learn the most effective combination of signals from firms covered jointly by analysts in order to benefit from any spillover effect. To fill this gap, we build a trading strategy which leverages the analyst coverage network using a graph attention network. More specifically, our model learns to aggregate information from individual firm features and signals from neighbouring firms in a node-level forecasting task. We develop a portfolio based on those predictions which we demonstrate to exhibit an annualized returns of 29.44% and a Sharpe ratio of 4.06 substantially outperforming market baselines and existing graph machine learning based frameworks. We further investigate the performance and robustness of this strategy through extensive empirical analysis. Our paper represents one of the first attempts in using graph machine learning to extract actionable knowledge from the analyst coverage network for practical financial applications.

In the search for the pentaquark $\Theta^+$ baryon, the world data on $K^+n\to K^+n$ and $K^+n\to K^0\,p$ reactions on the deuteron target and $K^0\,p\to K^+n$ and $K^0_L\,p\to K^0_S\,p$ reactions on the hydrogen target are revisited to study the isoscalar component of the scattering amplitudes. The determination of the $s$- and $p$-wave phase shifts for the two former channels on the deuteron target is presented in the low momentum region with the uncertainty due to the deuteron form factors at forward angles discussed. For the reactions on hydrogen targets, a similarity expected from time reversal between the charge exchange processes $K^+n\to K^0\,p$ and $K^0\,p\to K^+n$ is exploited, while we present the analysis of the $K_L^0\,p\to K_S^0\,p$ scattering based on the $s$-wave phase shift for low momenta combined with the $t$-channel $\rho^0(775)+\omega(782)+\phi(1020)$ vector meson exchange at high momenta. With the mass 1535 MeV and decay width 5 MeV the possibility of the $\Theta^+$ in the Breit-Wigner (BW) form is tested in the differential cross section measured by C. J. S. Damerell {\it et\ al.} at ${\rm P_{Lab}}=434$ MeV/$c$ and polarization. Our analysis suggests measuring the polarization observable rather than the differential cross section, which has sufficient sensitivity to distinguish the $\Theta^+$ baryon among all spin and parity states $J^P = 1/2^\pm$ and $3/2^\pm$., Comment: 20 pages, 14 figures, Acta Physica Polonica B memorial volume dedicated to Mitya Diakonov, Victor Petrov and Maxim Polyakov

Multi-dimensional time series data, such as matrix and tensor-variate time series, are increasingly prevalent in fields such as economics, finance, and climate science. Traditional Transformer models, though adept with sequential data, do not effectively preserve these multi-dimensional structures, as their internal operations in effect flatten multi-dimensional observations into vectors, thereby losing critical multi-dimensional relationships and patterns. To address this, we introduce the Tensor-Augmented Transformer (TEAFormer), a novel method that incorporates tensor expansion and compression within the Transformer framework to maintain and leverage the inherent multi-dimensional structures, thus reducing computational costs and improving prediction accuracy. The core feature of the TEAFormer, the Tensor-Augmentation (TEA) module, utilizes tensor expansion to enhance multi-view feature learning and tensor compression for efficient information aggregation and reduced computational load. The TEA module is not just a specific model architecture but a versatile component that is highly compatible with the attention mechanism and the encoder-decoder structure of Transformers, making it adaptable to existing Transformer architectures. Our comprehensive experiments, which integrate the TEA module into three popular time series Transformer models across three real-world benchmarks, show significant performance enhancements, highlighting the potential of TEAFormers for cutting-edge time series forecasting.

Continuous monitoring of non-invasive skin sympathetic nerve activity (SKNA) holds promise for understanding the sympathetic nervous system (SNS) dynamics in various physiological and pathological conditions. However, muscle noise artifacts present a challenge in accurate SKNA analysis, particularly in real-life scenarios. This study proposes a deep convolutional neural network (CNN) approach to detect and remove muscle noise from SKNA recordings obtained via ECG electrodes. Twelve healthy participants underwent controlled experimental protocols involving cognitive stress induction and voluntary muscle movements, while collecting SKNA data. Power spectral analysis revealed significant muscle noise interference within the SKNA frequency band (500-1000 Hz). A 2D CNN model was trained on the spectrograms of the data segments to classify them into baseline, stress-induced SKNA, and muscle noise-contaminated periods, achieving an average accuracy of 89.85% across all subjects. Our findings underscore the importance of addressing muscle noise for accurate SKNA monitoring, advancing towards wearable SKNA sensors for real-world applications., Comment: 4 pages, 5 figures, 1 table, IEEE-EMBS International Conference on Body Sensor Networks: NextGen Health: Sensor Innovation, AI, and Social Responsibility (IEEE BSN 2024)

In this paper we present Hybrid iterative Linear Quadratic Estimation (HiLQE), an optimization based offline state estimation algorithm for hybrid dynamical systems. We utilize the saltation matrix, a first order approximation of the variational update through an event driven hybrid transition, to calculate gradient information through hybrid events in the backward pass of an iterative linear quadratic optimization over state estimates. This enables accurate computation of the value function approximation at each timestep. Additionally, the forward pass in the iterative algorithm is augmented with hybrid dynamics in the rollout. A reference extension method is used to account for varying impact times when comparing states for the feedback gain in noise calculation. The proposed method is demonstrated on an ASLIP hopper system with position measurements. In comparison to the Salted Kalman Filter (SKF), the algorithm presented here achieves a maximum of 63.55% reduction in estimation error magnitude over all state dimensions near impact events.

Current image watermarking methods are vulnerable to advanced image editing techniques enabled by large-scale text-to-image models. These models can distort embedded watermarks during editing, posing significant challenges to copyright protection. In this work, we introduce W-Bench, the first comprehensive benchmark designed to evaluate the robustness of watermarking methods against a wide range of image editing techniques, including image regeneration, global editing, local editing, and image-to-video generation. Through extensive evaluations of eleven representative watermarking methods against prevalent editing techniques, we demonstrate that most methods fail to detect watermarks after such edits. To address this limitation, we propose VINE, a watermarking method that significantly enhances robustness against various image editing techniques while maintaining high image quality. Our approach involves two key innovations: (1) we analyze the frequency characteristics of image editing and identify that blurring distortions exhibit similar frequency properties, which allows us to use them as surrogate attacks during training to bolster watermark robustness; (2) we leverage a large-scale pretrained diffusion model SDXL-Turbo, adapting it for the watermarking task to achieve more imperceptible and robust watermark embedding. Experimental results show that our method achieves outstanding watermarking performance under various image editing techniques, outperforming existing methods in both image quality and robustness. Code is available at https://github.com/Shilin-LU/VINE.

Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs). However, recent work reveals that the effective context lengths of open-source LLMs often fall short, typically not exceeding half of their training lengths. In this work, we attribute this limitation to the left-skewed frequency distribution of relative positions formed in LLMs pretraining and post-training stages, which impedes their ability to effectively gather distant information. To address this challenge, we introduce ShifTed Rotray position embeddING (STRING). STRING shifts well-trained positions to overwrite the original ineffective positions during inference, enhancing performance within their existing training lengths. Experimental results show that without additional training, STRING dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B and Qwen2 72B, by over 10 points on popular long-context benchmarks RULER and InfiniteBench, establishing new state-of-the-art results for open-source LLMs. Compared to commercial models, Llama 3.1 70B with \method even achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.