Search

This work proposes FireRedTTS, a foundation text-to-speech framework, to meet the growing demands for personalized and diverse generative speech applications. The framework comprises three parts: data processing, foundation system, and downstream applications. First, we comprehensively present our data processing pipeline, which transforms massive raw audio into a large-scale high-quality TTS dataset with rich annotations and a wide coverage of content, speaking style, and timbre. Then, we propose a language-model-based foundation TTS system. The speech signal is compressed into discrete semantic tokens via a semantic-aware speech tokenizer, and can be generated by a language model from the prompt text and audio. Then, a two-stage waveform generator is proposed to decode them to the high-fidelity waveform. We present two applications of this system: voice cloning for dubbing and human-like speech generation for chatbots. The experimental results demonstrate the solid in-context learning capability of FireRedTTS, which can stably synthesize high-quality speech consistent with the prompt text and audio. For dubbing, FireRedTTS can clone target voices in a zero-shot way for the UGC scenario and adapt to studio-level expressive voice characters in the PUGC scenario via few-shot fine-tuning with 1-hour recording. Moreover, FireRedTTS achieves controllable human-like speech generation in a casual style with paralinguistic behaviors and emotions via instruction tuning, to better serve spoken chatbots.

WalledEval is a comprehensive AI safety testing toolkit designed to evaluate large language models (LLMs). It accommodates a diverse range of models, including both open-weight and API-based ones, and features over 35 safety benchmarks covering areas such as multilingual safety, exaggerated safety, and prompt injections. The framework supports both LLM and judge benchmarking and incorporates custom mutators to test safety against various text-style mutations, such as future tense and paraphrasing. Additionally, WalledEval introduces WalledGuard, a new, small, and performant content moderation tool, and two datasets: SGXSTest and HIXSTest, which serve as benchmarks for assessing the exaggerated safety of LLMs and judges in cultural contexts. We make WalledEval publicly available at https://github.com/walledai/walledeval., Comment: Under review

Manufacturing complexities and uncertainties have impeded the transition from material prototypes to commercial batteries, making prototype verification critical to quality assessment. A fundamental challenge involves deciphering intertwined chemical processes to characterize degradation patterns and their quantitative relationship with battery performance. Here we show that a physics-informed machine learning approach can quantify and visualize temporally resolved losses concerning thermodynamics and kinetics only using electric signals. Our method enables non-destructive degradation pattern characterization, expediting temperature-adaptable predictions of entire lifetime trajectories, rather than end-of-life points. The verification speed is 25 times faster yet maintaining 95.1% accuracy across temperatures. Such advances facilitate more sustainable management of defective prototypes before massive production, establishing a 19.76 billion USD scrap material recycling market by 2060 in China. By incorporating stepwise charge acceptance as a measure of the initial manufacturing variability of normally identical batteries, we can immediately identify long-term degradation variations. We attribute the predictive power to interpreting machine learning insights using material-agnostic featurization taxonomy for degradation pattern decoupling. Our findings offer new possibilities for dynamic system analysis, such as battery prototype degradation, demonstrating that complex pattern evolutions can be accurately predicted in a non-destructive and data-driven fashion by integrating physics-informed machine learning.

Building a theoretical understanding of the capabilities of large language models (LLMs) is vital for our ability to predict and explain the behavior of these systems. Here, we investigate the structure of LLM capabilities by extracting latent capabilities from patterns of individual differences across a varied population of LLMs. Using a combination of Bayesian and frequentist factor analysis, we analyzed data from 29 different LLMs across 27 cognitive tasks. We found evidence that LLM capabilities are not monolithic. Instead, they are better explained by three well-delineated factors that represent reasoning, comprehension and core language modeling. Moreover, we found that these three factors can explain a high proportion of the variance in model performance. These results reveal a consistent structure in the capabilities of different LLMs and demonstrate the multifaceted nature of these capabilities. We also found that the three abilities show different relationships to model properties such as model size and instruction tuning. These patterns help refine our understanding of scaling laws and indicate that changes to a model that improve one ability might simultaneously impair others. Based on these findings, we suggest that benchmarks could be streamlined by focusing on tasks that tap into each broad model ability., Comment: 10 pages, 3 figures + references and appendices, for data and analysis code see https://github.com/RyanBurnell/revealing-LLM-capabilities

Visual recognition of materials and their states is essential for understanding most aspects of the world, from determining whether food is cooked, metal is rusted, or a chemical reaction has occurred. However, current image recognition methods are limited to specific classes and properties and can't handle the vast number of material states in the world. To address this, we present MatSim: the first dataset and benchmark for computer vision-based recognition of similarities and transitions between materials and textures, focusing on identifying any material under any conditions using one or a few examples. The dataset contains synthetic and natural images. The synthetic images were rendered using giant collections of textures, objects, and environments generated by computer graphics artists. We use mixtures and gradual transitions between materials to allow the system to learn cases with smooth transitions between states (like gradually cooked food). We also render images with materials inside transparent containers to support beverage and chemistry lab use cases. We use this dataset to train a siamese net that identifies the same material in different objects, mixtures, and environments. The descriptor generated by this net can be used to identify the states of materials and their subclasses using a single image. We also present the first few-shot material recognition benchmark with images from a wide range of fields, including the state of foods and drinks, types of grounds, and many other use cases. We show that a net trained on the MatSim synthetic dataset outperforms state-of-the-art models like Clip on the benchmark and also achieves good results on other unsupervised material classification tasks., Comment: for associated code and dataset, see https://zenodo.org/record/7390166#.Y5ku6mHMJH4 or https://e1.pcloud.link/publink/show?code=kZIiSQZCYU5M4HOvnQykql9jxF4h0KiC5MX and https://icedrive.net/s/A13FWzZ8V2aP9T4ufGQ1N3fBZxDF

Capacity is one of the most important performance metrics for wireless communication networks. It describes the maximum rate at which the information can be transmitted of a wireless communication system. To support the growing demand for wireless traffic, wireless networks are becoming more dense and complicated, leading to a higher difficulty to derive the capacity. Unfortunately, most existing methods for the capacity calculation take a polynomial time complexity. This will become unaffordable for future ultra-dense networks, where both the number of base stations (BSs) and the number of users are extremely large. In this paper, we propose a fast algorithm TOSE to estimate the capacity for ultra-dense wireless networks. Based on the spiked model of random matrix theory (RMT), our algorithm can avoid the exact eigenvalue derivations of large dimensional matrices, which are complicated and inevitable in conventional capacity calculation methods. Instead, fast eigenvalue estimations can be realized based on the spike approximations in our TOSE algorithm. Our simulation results show that TOSE is an accurate and fast capacity approximation algorithm. Its estimation error is below 5%, and it runs in linear time, which is much lower than the polynomial time complexity of existing methods. In addition, TOSE has superior generality, since it is independent of the distributions of BSs and users, and the shape of network areas., Comment: 6 pages, 4 figures. arXiv admin note: text overlap with arXiv:2204.03393

Wireless network capacity can be regarded as the most important performance metric for wireless communication systems. With the fast development of wireless communication technology, future wireless systems will become more and more complicated. As a result, the channel gain matrix will become a large-dimensional random matrix, leading to an extremely high computational cost to obtain the capacity. In this paper, we propose a moment passing method (MPM) to realize the fast and accurate capacity estimation for future ultra-dense wireless systems. It can determine the capacity with quadratic complexity, which is optimal considering that the cost of a single matrix operation is not less than quadratic complexity. Moreover, it has high accuracy. The simulation results show that the estimation error of this method is below 2 percent. Finally, our method is highly general, as it is independent of the distributions of BSs and users, and the shape of network areas. More importantly, it can be applied not only to the conventional multi-user multiple input and multiple output (MU-MIMO) networks, but also to the capacity-centric networks designed for B5G/6G.

Wireless network capacity is one of the most important performance metrics for wireless communication networks. Future wireless networks will be composed of extremely large number of base stations (BSs) and users, and organized in the form of multiple clusters. Unfortunately, the determination of average cluster capacity for such future wireless networks is difficult, and lacks of both analytical expressions and fast algorithms. In this paper, we propose a fast algorithm TOSE to estimate the average cluster capacity based on the random matrix theory (RMT). It can avoid the exact eigenvalue derivations of large dimensional matrices, which are complicated and inevitable in conventional capacity determination methods. Instead, fast eigenvalue estimations can be realized based on RMT in our TOSE algorithm. In addition, we derive the analytical upper and lower bounds of the average cluster capacity. Our numerical experiments show that TOSE is faster than the conventional Cholesky decomposition method, by at least three orders of magnitude. Besides, TOSE has superior generality, since it is independent of the distributions of BSs and users, and the shape of network areas., Comment: Another version of this paper has been uploaded by another author

Pattern forming with externally imposed symmetry is ubiquitous in nature but lightly studied.We present experimental studies of pattern formation and selection by spatial periodic forcing in rapidly rotating convection. We observe symmetric convection patterns in form of regular vortex lattice near the instability onset, when the periodicity of the external forcing is set close to the intrinsic vortex spacing. We show that the new patterns arise as a dynamical process of imperfect bifurcation which can be well described by a Ginzburg-Landau-like model. With increasing buoyancy strength the effect of external forcing weakens, and the convective vortices evolve from a stationary state to exhibit restricted and finally stochastic motions., Comment: 5 pages, 4 figures

Modern microelectronic processors have migrated towards parallel computing architectures with many-core processors. However, such expansion comes with diminishing returns exacted by the high cost of data movement between individual processors. The use of optical interconnects has burgeoned as a promising technology that can address the limits of this data transfer. While recent pushes to enhance optical communication have focused on developing wavelength-division multiplexing technology, this approach will eventually saturate the usable bandwidth, and new dimensions of data transfer will be paramount to fulfill the ever-growing need for speed. Here we demonstrate an integrated intra- and inter-chip multi-dimensional communication scheme enabled by photonic inverse design. Using inverse-designed mode-division multiplexers, we combine wavelength- and mode- multiplexing and send massively parallel data through nano-photonic waveguides and optical fibres. Crucially, as we take advantage of an orthogonal optical basis, our approach is inherently scalable to a multiplicative enhancement over the current state of the art., Comment: Fig.2-4 present new experimental results -- (i) demonstration of a broadband, low cross-talk multiplexer, (ii) a silicon photonic mode-division multiplexing with a chip-scale soliton microcomb source, and (iii) a chip-to-chip optical interconnect using a multimode-matched fibre and inverse-designed beam couplers

The research on Wi-Fi sensing has been thriving over the past decade but the process has not been smooth. Three barriers always hamper the research: unknown baseband design and its influence, inadequate hardware, and the lack of versatile and flexible measurement software. This paper tries to eliminate these barriers through the following work. First, we present an in-depth study of the baseband design of the Qualcomm Atheros AR9300 (QCA9300) NIC. We identify a missing item of the existing CSI model, namely, the CSI distortion, and identify the baseband filter as its origin. We also propose a distortion removal method. Second, we reintroduce both the QCA9300 and software-defined radio (SDR) as powerful hardware for research. For the QCA9300, we unlock the arbitrary tuning of both the carrier frequency and bandwidth. For SDR, we develop a high?performance software implementation of the 802.11a/g/n/ac/ax baseband, allowing users to fully control the baseband and access the complete physical-layer information. Third, we release the PicoScenes software, which supports concurrent CSI measure?ment from multiple QCA9300, Intel Wireless Link (IWL5300) and SDR hardware. PicoScenes features rich low-level controls, packet injection and software baseband implementation. It also allows users to develop their own measurement plugins. Finally, we report state-of-the-art results in the extensive evaluations of the PicoScenes system, such as the >2 GHz available spectrum on the QCA9300, concurrent CSI measurement, and up to 40 kHz and 1 kHz CSI measurement rates achieved by the QCA9300 and SDR. PicoScenes is available at https://ps.zpj.io.