Search

The Large Hadron Collider (LHC), the world's largest and most powerful particle accelerator, has been a pivotal tool in advancing our understanding of fundamental physics. By colliding heavy ions (such as lead ions), the LHC recreates conditions similar to those just after the Big Bang. This allows scientists to study the Quark-Gluon Plasma (QGP), a state of matter where quarks and gluons are not confined within protons and neutrons. These studies provide insights into the strong force and the early universe's behavior. In this paper, we provide a comprehensive overview of recent significant findings from A Large Ion Collider Experiment (ALICE) at LHC. The topics encompass measurements regarding to properties of the QGP, particle production, flow and correlations, dileptons, quarkonia and electromagnetic probes, heavy flavor, and jets. Additionally, we introduce future plans for detector upgrades of the ALICE experiment., Comment: 29 pages, 32 figures. This review is dedicated to Professor Wenqing Shen in honor of his leadership and significant impact on the Chinese heavy-ion physics community. All authors contributed equally to this work

Current speech-based LLMs are predominantly trained on extensive ASR and TTS datasets, excelling in tasks related to these domains. However, their ability to handle direct speech-to-speech conversations remains notably constrained. These models often rely on an ASR-to-TTS chain-of-thought pipeline, converting speech into text for processing before generating audio responses, which introduces latency and loses audio features. We propose a method that implicitly internalizes ASR chain of thought into a speech LLM, enhancing its native speech understanding capabilities. Our approach reduces latency and improves the model's native understanding of speech, paving the way for more efficient and natural real-time audio interactions. We also release a large-scale synthetic conversational dataset to facilitate further research., Comment: Corrected style from final to preprint

Sparse-view tomographic reconstruction is a pivotal direction for reducing radiation dose and augmenting clinical applicability. While many research works have proposed the reconstruction of tomographic images from sparse 2D projections, existing models tend to excessively focus on high-frequency information while overlooking low-frequency components within the sparse input images. This bias towards high-frequency information often leads to overfitting, particularly intense at edges and boundaries in the reconstructed slices. In this paper, we introduce the Frequency Regularized Neural Attenuation/Activity Field (Freq-NAF) for self-supervised sparse-view tomographic reconstruction. Freq-NAF mitigates overfitting by incorporating frequency regularization, directly controlling the visible frequency bands in the neural network input. This approach effectively balances high-frequency and low-frequency information. We conducted numerical experiments on CBCT and SPECT datasets, and our method demonstrates state-of-the-art accuracy., Comment: 6 pages,5 figures,Accepted to ICME 2024

The third-order anomalous Hall effect (TOAHE) driven by Berry connection polarizability in Dirac materials offers a promising avenue for exploring quantum geometric phenomena. We investigate the role of impurity scattering on TOAHE using the semiclassical Boltzmann framework, via a comparison of the intrinsic contributions (stemming from the Berry connection polarizability effect) with the extrinsic contributions caused by the disorder. To validate our theoretical findings, we employ a generalized two-dimensional low-energy Dirac model to analytically assess the intrinsic and extrinsic contributions to the TOAHE. Our analysis reveals distinct disorder-mediated effects, including skew scattering and side jump contributions. We also elucidate their intriguing dependencies on Fermi surface anisotropy and discuss opportunities for experimental exploration., Comment: 6 Pages, 1 Figure

By utilizing a multi-orbital periodic Anderson model with parameters obtained from \textit{ab initio} band structure calculations, combined with degenerate perturbation theory, we derive effective Kondo-Heisenberg and spin Hamiltonians that capture the interaction among the effective magnetic moments. This derivation encompasses fluctuations via both non-magnetic $4f^0$ and magnetic $4f^2$ virtual states, and its accuracy is confirmed through comparison with experimental data obtained from CeIn$_3$. The significant agreement observed between experimental results and theoretical predictions underscores the potential of deriving minimal models from first-principles calculations for achieving a quantitative description of $4f$ materials. Moreover, our microscopic derivation unveils the underlying origin of anisotropy in the exchange interaction between Kramers doublets, shedding light on the conditions under which this anisotropy may be weak compared to the isotropic contribution., Comment: 30 pages, 16 figures

Understanding how people behave in strategic settings--where they make decisions based on their expectations about the behavior of others--is a long-standing problem in the behavioral sciences. We conduct the largest study to date of strategic decision-making in the context of initial play in two-player matrix games, analyzing over 90,000 human decisions across more than 2,400 procedurally generated games that span a much wider space than previous datasets. We show that a deep neural network trained on these data predicts people's choices better than leading theories of strategic behavior, indicating that there is systematic variation that is not explained by those theories. We then modify the network to produce a new, interpretable behavioral model, revealing what the original network learned about people: their ability to optimally respond and their capacity to reason about others are dependent on the complexity of individual games. This context-dependence is critical in explaining deviations from the rational Nash equilibrium, response times, and uncertainty in strategic decisions. More broadly, our results demonstrate how machine learning can be applied beyond prediction to further help generate novel explanations of complex human behavior.

The compacton, peakon, and Burgers-Hopf equations regularized by the Galerkin truncation preserving finite Fourier modes are found to support new travelling waves and interacting solitonic structures amidst weaker less-ordered components (`longons'). Different perspectives focusing on the zero-Hamiltonian solitonic, chaotic-looking, and stationary longons are also offered., Comment: arXiv admin note: text overlap with arXiv:2404.08583

The era characterized by an exponential increase in data has led to the widespread adoption of data intelligence as a crucial task. Within the field of data mining, frequent episode mining has emerged as an effective tool for extracting valuable and essential information from event sequences. Various algorithms have been developed to discover frequent episodes and subsequently derive episode rules using the frequency function and anti-monotonicity principles. However, currently, there is a lack of algorithms specifically designed for mining episode rules that encompass user-specified query episodes. To address this challenge and enable the mining of target episode rules, we introduce the definition of targeted precise-positioning episode rules and formulate the problem of targeted mining precise-positioning episode rules. Most importantly, we develop an algorithm called Targeted Mining Precision Episode Rules (TaMIPER) to address the problem and optimize it using four proposed strategies, leading to significant reductions in both time and space resource requirements. As a result, TaMIPER offers high accuracy and efficiency in mining episode rules of user interest and holds promising potential for prediction tasks in various domains, such as weather observation, network intrusion, and e-commerce. Experimental results on six real datasets demonstrate the exceptional performance of TaMIPER., Comment: IEEE TETCI, 14 pages

The optical conductivity $\sigma(\omega,T)$ of CeRhSn was studied by broadband infrared spectroscopy. Temperature-dependent spectral weight transfer occurs over high energy ($0.8\,$eV) and temperature (${\sim}500\,$K) scales, classifying CeRhSn as a mixed valent compound. The optical conductivity reveals a substantial anisotropy in the electronic structure. Renormalization of $\sigma(\omega,T)$ occurs as a function of temperature to a coherent Kondo state with concomitant effective mass generation. Associated spectroscopic signatures were reproduced remarkably well by the combination of density functional theory and dynamical mean field theory using a momentum-independent self energy. The theory shows that the anisotropy for energies $>10\,$meV is mainly driven by the bare three-dimensional electronic structure that is renormalized by local electronic correlations. The possible influence of magnetic frustration and quantum criticality is restricted to lower energies., Comment: 8 pages, 8 figures

As Large Language Models (LLMs) are increasingly deployed in real-world settings, understanding the knowledge they implicitly use when making decisions is critical. One way to capture this knowledge is in the form of Bayesian prior distributions. We develop a prompt-based workflow for eliciting prior distributions from LLMs. Our approach is based on iterated learning, a Markov chain Monte Carlo method in which successive inferences are chained in a way that supports sampling from the prior distribution. We validated our method in settings where iterated learning has previously been used to estimate the priors of human participants -- causal learning, proportion estimation, and predicting everyday quantities. We found that priors elicited from GPT-4 qualitatively align with human priors in these settings. We then used the same method to elicit priors from GPT-4 for a variety of speculative events, such as the timing of the development of superhuman AI.

The observed similarities in the behavior of humans and Large Language Models (LLMs) have prompted researchers to consider the potential of using LLMs as models of human cognition. However, several significant challenges must be addressed before LLMs can be legitimately regarded as cognitive models. For instance, LLMs are trained on far more data than humans typically encounter, and may have been directly trained on human data in specific cognitive tasks or aligned with human preferences. Consequently, the origins of these behavioral similarities are not well understood. In this paper, we propose a novel way to enhance the utility of LLMs as cognitive models. This approach involves (i) leveraging computationally equivalent tasks that both an LLM and a rational agent need to master for solving a cognitive problem and (ii) examining the specific task distributions required for an LLM to exhibit human-like behaviors. We apply this approach to decision-making -- specifically risky and intertemporal choice -- where the key computationally equivalent task is the arithmetic of expected value calculations. We show that an LLM pretrained on an ecologically valid arithmetic dataset, which we call Arithmetic-GPT, predicts human behavior better than many traditional cognitive models. Pretraining LLMs on ecologically valid arithmetic datasets is sufficient to produce a strong correspondence between these models and human decision-making. Our results also suggest that LLMs used as cognitive models should be carefully investigated via ablation studies of the pretraining data.

In addition to a plethora of emergent phenomena, the spatial topology of optical vortices enables an array of applications spanning communications to quantum photonics. Nonlinear optics is essential in this context, providing access to an infinitely large set of quantum states associated with the orbital angular momentum of light. Nevertheless, the realization of such processes have failed to keep pace with the ever-growing need to shrink the fundamental length-scale of photonic technologies to the nanometer regime6. Here, we push the boundaries of vortex nonlinear optics to the ultimate limits of material dimensionality. By exploiting second and third-order frequency-mixing processes in semiconducting monolayers, we demonstrate the independent manipulation of the wavelength, orbital angular momentum, and spatial distribution of vortex light-fields. Due to the atomically-thin nature of the host quantum material, this control spans a broad spectral bandwidth in a highly-integrable platform, unconstrained by the traditional limits of bulk nonlinear optical materials. Our work heralds a new avenue for ultra-compact and scalable hybrid nanotechnologies empowered by twisted nonlinear light-matter interactions in van der Waals quantum nanomaterials.

Irregular compactons and peakons from some nonlinear dispersions can be regularized by another type of nonlinear dispersion, defined by a pseudo-differential operator in physical space for the Galerkin truncation preserving finite Fourier modes of wavenumbers no larger than $K$. This resembles yet differs from the Korteweg-de Vries (KdV) regularization of the Burgers-Hopf (BH) equation. The Galerkin-regularized compacton, peakon, KdV, and BH dynamics exhibit novel traveling waves and interacting solitonic structures amidst weaker, less-ordered components (`longons'), potentially yielding nontrivial implications for effective field theories and phenomenologies in various domains, including particle and condensed matter physics. Appropriate linear dispersion models can infinitely approximate the longon states. Time-dependent and stationary behaviors in the large-$K$ limit are addressed with numerical results., Comment: GrCompactonPeakon, GrKdV longons join the family

Inspired by the excellent performance of Mamba networks, we propose a novel Deep Mamba Multi-modal Learning (DMML). It can be used to achieve the fusion of multi-modal features. We apply DMML to the field of multimedia retrieval and propose an innovative Deep Mamba Multi-modal Hashing (DMMH) method. It combines the advantages of algorithm accuracy and inference speed. We validated the effectiveness of DMMH on three public datasets and achieved state-of-the-art results., Comment: Deep Mamba Multi-modal Learning; Deep Mamba Multi-modal Hashing

We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture. Our architectural design advancements include multi-headed matrix-valued states and a dynamic recurrence mechanism that improve expressivity while maintaining the inference efficiency characteristics of RNNs. We introduce a new multilingual corpus with 1.12 trillion tokens and a fast tokenizer based on greedy matching for enhanced multilinguality. We trained four Eagle models, ranging from 0.46 to 7.5 billion parameters, and two Finch models with 1.6 and 3.1 billion parameters and find that they achieve competitive performance across a wide variety of benchmarks. We release all our models on HuggingFace under the Apache 2.0 license. Models at: https://huggingface.co/RWKV Training code at: https://github.com/RWKV/RWKV-LM Inference code at: https://github.com/RWKV/ChatRWKV Time-parallel training code at: https://github.com/RWKV/RWKV-infctx-trainer

Photoinduced effects are now reckoned to be important tools to reveal a rich gamut of entrancing physics in topological materials, which are normally inaccessible via conventional probes. Here we investigate one of these intriguing effects, namely, {\it optical spin injection} (OSI) beyond the regular Weyl semimetal (WSM), specifically in multi- and higher-order WSMs. Remarkably, we demonstrate that OSI in multi-WSMs can emerge as a frequency-independent quantized response engendered by band linearity and topological charge associated with the node. Interestingly, the frequency independence can be destroyed by going beyond the first-order topological phase where we explicitly show the OSI response is accompanied by peaks for the first-order Weyl nodes while dips for the second-order Weyl nodes. In addition, we elucidate the behavior of OSI in type-II Weyl phases. The predicted signatures of OSI beyond the regular WSM, i.e., multi- and higher-order WSM, could be experimentally exploited, leading us to effectively access as well as distinguish different-order nontrivial Weyl topology., Comment: 6 pages, 3 figures, Comments are most welcome

The detection of abusive language remains a long-standing challenge with the extensive use of social networks. The detection task of abusive language suffers from limited accuracy. We argue that the existing detection methods utilize the fine-tuning technique of the pre-trained language models (PLMs) to handle downstream tasks. Hence, these methods fail to stimulate the general knowledge of the PLMs. To address the problem, we propose a novel Deep Prompt Multi-task Network (DPMN) for abuse language detection. Specifically, DPMN first attempts to design two forms of deep prompt tuning and light prompt tuning for the PLMs. The effects of different prompt lengths, tuning strategies, and prompt initialization methods on detecting abusive language are studied. In addition, we propose a Task Head based on Bi-LSTM and FFN, which can be used as a short text classifier. Eventually, DPMN utilizes multi-task learning to improve detection metrics further. The multi-task network has the function of transferring effective knowledge. The proposed DPMN is evaluated against eight typical methods on three public datasets: OLID, SOLID, and AbuseAnalyzer. The experimental results show that our DPMN outperforms the state-of-the-art methods., Comment: Accepted by the International Conference on Pattern Recognition (ICPR) 2024

The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership with Software Heritage (SWH), we build The Stack v2 on top of the digital commons of their source code archive. Alongside the SWH repositories spanning 619 programming languages, we carefully select other high-quality data sources, such as GitHub pull requests, Kaggle notebooks, and code documentation. This results in a training set that is 4x larger than the first StarCoder dataset. We train StarCoder2 models with 3B, 7B, and 15B parameters on 3.3 to 4.3 trillion tokens and thoroughly evaluate them on a comprehensive set of Code LLM benchmarks. We find that our small model, StarCoder2-3B, outperforms other Code LLMs of similar size on most benchmarks, and also outperforms StarCoderBase-15B. Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size. In addition, it matches or outperforms CodeLlama-34B, a model more than twice its size. Although DeepSeekCoder- 33B is the best-performing model at code completion for high-resource languages, we find that StarCoder2-15B outperforms it on math and code reasoning benchmarks, as well as several low-resource languages. We make the model weights available under an OpenRAIL license and ensure full transparency regarding the training data by releasing the SoftWare Heritage persistent IDentifiers (SWHIDs) of the source code data.

The Variational Quantum Eigensolver (VQE) is widely considered to be a promising candidate for a quantum-classical algorithm which could achieve near-term quantum advantage. However, current levels of hardware noise can require extensive application of error-mitigation techniques to achieve reliable computations. In this work, we use several IBM devices to explore a finite-size spin model with multiple `phase-like' regions characterized by distinct ground-state configurations. Using pre-optimized VQE solutions, we demonstrate that in contrast to calculating the energy, where zero-noise extrapolation is required in order to obtain qualitatively accurate yet still unreliable results, calculations of the energy derivative, two-site spin correlation functions, and the fidelity susceptibility yield accurate behavior across multiple regions, even with minimal or no application of error-mitigation approaches. Taken together, these sets of observables could be used to identify level crossings in VQE solutions in a simple and noise-robust manner, with potential near-term application to identifying quantum phase transitions, avoided crossings and non-adiabatic conical intersections in electronic structure calculations., Comment: 11 pages, 10 figures

We show that nonzero drag current in a double quantum-dot system, consisting of a biased drive dot and an unbiased passive dot coupled capacitively, can be generated by a Majorana bound state located at one of the leads connected to the passive dot. Importantly, the drag current induced by Majorana bound states, either an isolated Majorana bound state or two weakly coupled but spatially separated Majorana modes, shows qualitative differences from that induced by a near-zero-energy Andereev bound state. Thus, other than the tunneling spectroscopy, the proposed setup can serve as a complementary tool to detect Majorana fermions in proximitized Rashba nanowires., Comment: 7 pages, 4 figures

The wave equation governing the wave propagation in chiral phononic crystals, established through force equilibrium law, conceals the underlying physical information. This has led to a controversy over the bandgap mechanism. In this letter, we theoretically unveil the reason of this controversy, and put forward an alternative approach from wave behavior to formulate the wave equation, offering a new pathway to articulate the bandgap physics directly. We identify the obstacles in coupled acoustic and optic branches to widen and lower the bandgap, and introduce an approach based on spherical hinges to decrease the barriers, for customizing the bandgap frequency and width. Finally, we validate our proposal through numerical simulation and experimental demonstration., Comment: 7 pages, 4 figures

The blazar sequence, including negative correlations between radiative luminosity $L_{\rm rad}$ and synchrotron peak frequency $\nu$, and between Compton dominance $Y$ and $\nu$, is widely adopted as a phenomenological description of spectral energy distributions (SEDs) of blazars, although its underlying cause is hotly debated. In particular, these correlations turn positive after correcting Doppler boosting effect. In this work, we revisit the phenomenological and intrinsic blazar sequence with three samples, which are historical sample (SEDs are built with historical data), quasi-simultaneous sample (SEDs are built with quasi-simultaneous data) and Doppler factor corrected sample (a sample with available Doppler factors), selected from literature. We find that phenomenological blazar sequence holds in historical sample, but does not exist in quasi-simultaneous sample, and intrinsic correlation between $L_{\rm rad}$ and $\nu$ becomes positive in Doppler factor corrected sample. We also analyze if the blazar sequence still exists in subclasses of blazars, i.e., flat-spectrum radio quasars and BL Lacertae objects, with different values of $Y$. To interpret these correlations, we apply a simple scaling model, in which physical parameters of the dissipation region are connected to the location of the dissipation region. We find that the model generated results are highly sensitive to the chosen ranges and distributions of physical parameters. Therefore, we suggest that even though the simple scaling model can reproduce the blazar sequence under specific conditions that have been fine-tuned, such results may not have universal applicability. Further consideration of a more realistic emission model is expected., Comment: Accepted for publication in MNRAS (14 pages, 6 figures, 3 tables)

Simulating sampling algorithms with people has proven a useful method for efficiently probing and understanding their mental representations. We propose that the same methods can be used to study the representations of Large Language Models (LLMs). While one can always directly prompt either humans or LLMs to disclose their mental representations introspectively, we show that increased efficiency can be achieved by using LLMs as elements of a sampling algorithm. We explore the extent to which we recover human-like representations when LLMs are interrogated with Direct Sampling and Markov chain Monte Carlo (MCMC). We found a significant increase in efficiency and performance using adaptive sampling algorithms based on MCMC. We also highlight the potential of our method to yield a more general method of conducting Bayesian inference \textit{with} LLMs.

Autoregressive Large Language Models (LLMs) trained for next-word prediction have demonstrated remarkable proficiency at producing coherent text. But are they equally adept at forming coherent probability judgments? We use probabilistic identities and repeated judgments to assess the coherence of probability judgments made by LLMs. Our results show that the judgments produced by these models are often incoherent, displaying human-like systematic deviations from the rules of probability theory. Moreover, when prompted to judge the same event, the mean-variance relationship of probability judgments produced by LLMs shows an inverted-U-shaped like that seen in humans. We propose that these deviations from rationality can be explained by linking autoregressive LLMs to implicit Bayesian inference and drawing parallels with the Bayesian Sampler model of human probability judgments.

We propose a novel optimization-based human mesh recovery method from a single image. Given a test exemplar, previous approaches optimize the pre-trained regression network to minimize the 2D re-projection loss, which however suffer from over-/under-fitting problems. This is because the ``exemplar optimization'' at testing time has too weak relation to the pre-training process, and the exemplar optimization loss function is different from the training loss function. (1) We incorporate exemplar optimization into the training stage. During training, our method first executes exemplar optimization and subsequently proceeds with training-time optimization. The exemplar optimization may run into a wrong direction, while the subsequent training optimization serves to correct the deviation. Involved in training, the exemplar optimization learns to adapt its behavior to training data, thereby acquires generalibility to test exemplars. (2) We devise a dual-network architecture to convey the novel training paradigm, which is composed of a main regression network and an auxiliary network, in which we can formulate the exemplar optimization loss function in the same form as the training loss function. This further enhances the compatibility between the exemplar and training optimizations. Experiments demonstrate that our exemplar optimization after the novel training scheme significantly outperforms state-of-the-art approaches.

We study the magnetic heat capacity of a series of magnetically ordered Ce-based heavy fermion materials, which show an anomalous $T^3$ heat capacity in excess of the phonon contribution in many materials. For compounds for which magnon models have been worked out, we show that the local-moment magnon heat capacity derived from the measured magnon spectra underestimates the experimental specific heat. The excess heat capacity reveals increasing density of states with increasing energy, akin to a pseudogap. We show that this anomalous temperature-dependent term is not associated with proximity to a quantum critical point (QCP), but is strongly correlated with $T_N$, indicating the anomalous excitations are governed by the magnetic exchange interaction. This insight may hold key information for understanding magnetically ordered heavy fermions., Comment: 5 pages, 4 figures

Large language models (LLMs) can produce long, coherent passages of text, suggesting that LLMs, although trained on next-word prediction, must represent the latent structure that characterizes a document. Prior work has found that internal representations of LLMs encode one aspect of latent structure, namely syntax; here we investigate a complementary aspect, namely the document's topic structure. We motivate the hypothesis that LLMs capture topic structure by connecting LLM optimization to implicit Bayesian inference. De Finetti's theorem shows that exchangeable probability distributions can be represented as a mixture with respect to a latent generating distribution. Although text is not exchangeable at the level of syntax, exchangeability is a reasonable starting assumption for topic structure. We thus hypothesize that predicting the next token in text will lead LLMs to recover latent topic distributions. We examine this hypothesis using Latent Dirichlet Allocation (LDA), an exchangeable probabilistic topic model, as a target, and we show that the representations formed by LLMs encode both the topics used to generate synthetic data and those used to explain natural corpus data., Comment: 13 pages, 4 figures

The two-dimensional (2D) material Cr$_2$Ge$_2$Te$_6$ is a member of the class of insulating van der Waals magnets. Here, using high resolution angle-resolved photoemission spectroscopy in a detailed temperature dependence study, we identify a clear response of the electronic structure to a dimensional crossover in the form of two distinct temperature scales marking onsets of modifications in the electronic structure. Specifically, we observe Te $p$-orbital-dominated bands to undergo changes at the Curie transition temperature T$_C$ while the Cr $d$-orbital-dominated bands begin evolving at a higher temperature scale. Combined with neutron scattering, density functional theory calculations, and Monte Carlo simulations, we find that the electronic system can be consistently understood to respond sequentially to the distinct temperatures at which in-plane and out-of-plane spin correlations exceed a characteristic length scale. Our findings reveal the sensitivity of the orbital-selective electronic structure for probing the dynamical evolution of local moment correlations in vdW insulating magnets., Comment: PRB, in press

The multi-view hash method converts heterogeneous data from multiple views into binary hash codes, which is one of the critical technologies in multimedia retrieval. However, the current methods mainly explore the complementarity among multiple views while lacking confidence learning and fusion. Moreover, in practical application scenarios, the single-view data contain redundant noise. To conduct the confidence learning and eliminate unnecessary noise, we propose a novel Adaptive Confidence Multi-View Hashing (ACMVH) method. First, a confidence network is developed to extract useful information from various single-view features and remove noise information. Furthermore, an adaptive confidence multi-view network is employed to measure the confidence of each view and then fuse multi-view features through a weighted summation. Lastly, a dilation network is designed to further enhance the feature representation of the fused features. To the best of our knowledge, we pioneer the application of confidence learning into the field of multimedia retrieval. Extensive experiments on two public datasets show that the proposed ACMVH performs better than state-of-the-art methods (maximum increase of 3.24%). The source code is available at https://github.com/HackerHyper/ACMVH., Comment: accepted by International Conference on Acoustics, Speech and Signal Processing 2024(ICASSP2024)

The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case, and that in fact these systems offer new opportunities for Bayesian modeling. Specifically, we argue that Bayesian models of cognition and artificial neural networks lie at different levels of analysis and are complementary modeling approaches, together offering a way to understand human cognition that spans these levels. We also argue that the same perspective can be applied to intelligent machines, where a Bayesian approach may be uniquely valuable in understanding the behavior of large, opaque artificial neural networks that are trained on proprietary data.

In this project, we demonstrate that phoneme-based models for speech processing can achieve strong crosslinguistic generalizability to unseen languages. We curated the IPAPACK, a massively multilingual speech corpora with phonemic transcriptions, encompassing more than 115 languages from diverse language families, selectively checked by linguists. Based on the IPAPACK, we propose CLAP-IPA, a multi-lingual phoneme-speech contrastive embedding model capable of open-vocabulary matching between arbitrary speech signals and phonemic sequences. The proposed model was tested on 95 unseen languages, showing strong generalizability across languages. Temporal alignments between phonemes and speech signals also emerged from contrastive training, enabling zeroshot forced alignment in unseen languages. We further introduced a neural forced aligner IPA-ALIGNER by finetuning CLAP-IPA with the Forward-Sum loss to learn better phone-to-audio alignment. Evaluation results suggest that IPA-ALIGNER can generalize to unseen languages without adaptation., Comment: NAACL 2024 Main Conference