Search

Binarization, which converts weight parameters to binary values, has emerged as an effective strategy to reduce the size of large language models (LLMs). However, typical binarization techniques significantly diminish linguistic effectiveness of LLMs. To address this issue, we introduce a novel binarization technique called Mixture of Scales (BinaryMoS). Unlike conventional methods, BinaryMoS employs multiple scaling experts for binary weights, dynamically merging these experts for each token to adaptively generate scaling factors. This token-adaptive approach boosts the representational power of binarized LLMs by enabling contextual adjustments to the values of binary weights. Moreover, because this adaptive process only involves the scaling factors rather than the entire weight matrix, BinaryMoS maintains compression efficiency similar to traditional static binarization methods. Our experimental results reveal that BinaryMoS surpasses conventional binarization techniques in various natural language processing tasks and even outperforms 2-bit quantization methods, all while maintaining similar model size to static binarization techniques.

We introduce QUICK, a group of novel optimized CUDA kernels for the efficient inference of quantized Large Language Models (LLMs). QUICK addresses the shared memory bank-conflict problem of state-of-the-art mixed precision matrix multiplication kernels. Our method interleaves the quantized weight matrices of LLMs offline to skip the shared memory write-back after the dequantization. We demonstrate up to 1.91x speedup over existing kernels of AutoAWQ on larger batches and up to 1.94x throughput gain on representative LLM models on various NVIDIA GPU devices., Comment: 9 pages, 8 figures

Large language models (LLMs) have proven to be highly effective across various natural language processing tasks. However, their large number of parameters poses significant challenges for practical deployment. Pruning, a technique aimed at reducing the size and complexity of LLMs, offers a potential solution by removing redundant components from the network. Despite the promise of pruning, existing methods often struggle to achieve substantial end-to-end LLM inference speedup. In this paper, we introduce SLEB, a novel approach designed to streamline LLMs by eliminating redundant transformer blocks. We choose the transformer block as the fundamental unit for pruning, because LLMs exhibit block-level redundancy with high similarity between the outputs of neighboring blocks. This choice allows us to effectively enhance the processing speed of LLMs. Our experimental results demonstrate that SLEB outperforms previous LLM pruning methods in accelerating LLM inference while also maintaining superior perplexity and accuracy, making SLEB as a promising technique for enhancing the efficiency of LLMs. The code is available at: https://github.com/jiwonsong-dev/SLEB.

The emergence of diffusion models has greatly broadened the scope of high-fidelity image synthesis, resulting in notable advancements in both practical implementation and academic research. With the active adoption of the model in various real-world applications, the need for on-device deployment has grown considerably. However, deploying large diffusion models such as Stable Diffusion with more than one billion parameters to mobile devices poses distinctive challenges due to the limited computational and memory resources, which may vary according to the device. In this paper, we present the challenges and solutions for deploying Stable Diffusion on mobile devices with TensorFlow Lite framework, which supports both iOS and Android devices. The resulting Mobile Stable Diffusion achieves the inference latency of smaller than 7 seconds for a 512x512 image generation on Android devices with mobile GPUs., Comment: 7 pages, 8 figures, ICML 2023 Workshop on Challenges in Deployable Generative AI

Large language models (LLMs) with hundreds of billions of parameters require powerful server-grade GPUs for inference, limiting their practical deployment. To address this challenge, we introduce the outlier-aware weight quantization (OWQ) method, which aims to minimize LLM's footprint through low-precision representation. OWQ prioritizes a small subset of structured weights sensitive to quantization, storing them in high-precision, while applying highly tuned quantization to the remaining dense weights. This sensitivity-aware mixed-precision scheme reduces the quantization error notably, and extensive experiments demonstrate that 3.1-bit models using OWQ perform comparably to 4-bit models optimized by OPTQ. Furthermore, OWQ incorporates a parameter-efficient fine-tuning for task-specific adaptation, called weak column tuning (WCT), enabling accurate task-specific LLM adaptation with minimal memory overhead in the optimized format. OWQ represents a notable advancement in the flexibility, efficiency, and practicality of LLM optimization literature. The source code is available at https://github.com/xvyaward/owq, Comment: Accepted at AAAI 2024 (oral presentation)

In recent years, emotional text-to-speech has shown considerable progress. However, it requires a large amount of labeled data, which is not easily accessible. Even if it is possible to acquire an emotional speech dataset, there is still a limitation in controlling emotion intensity. In this work, we propose a novel method for cross-speaker emotion transfer and manipulation using vector arithmetic in latent style space. By leveraging only a few labeled samples, we generate emotional speech from reading-style speech without losing the speaker identity. Furthermore, emotion strength is readily controllable using a scalar value, providing an intuitive way for users to manipulate speech. Experimental results show the proposed method affords superior performance in terms of expressiveness, naturalness, and controllability, preserving speaker identity., Comment: accepted to ICASSP 2023

In this paper, we first provide precise tensorial formulae for the asymptotic operators of contact instantons $w:\dot \Sigma \to Q$ and of pseudoholomorphic curves $u:(\dot\Sigma,j) \to (Q \times \mathbb R, \widetilde J)$ on the symplectization of contact manifold $(Q,\lambda)$. The formula exhibits explicit dependence on the compatible pair $(\lambda,J)$ of the given contact triad $(Q,\lambda, J)$. Then based on this, we present a perturbation theory of the asymptotic operator under the change of compatible CR almost complex structure $J$ for given contact form $\lambda$. This perturbation theory has been missing in the literature on the study of pseudoholomorphic curves on symplectization. Through this systematic tensorial approach, we also provide the study of finer asymptotic behavior at the punctures, such as convergence of tangent plane of contact instantons, in terms of the eigenvalues and eigenvectors of the asymptotic operator, which also simplifies the corresponding study of pseudoholomorphic curves on symplectization in the literature., Comment: 44 pages, Comments welcome. arXiv admin note: substantial text overlap with arXiv:2302.06122; v2) Many typos corrected, some imprecise calculations corrected in Appendix; v3) 37 pages, re-oranization of sections, typos corrected, final accepted version in the journal, Advances in Geometry

In this survey article, we present the analysis of pseudoholomorphic curves $u:(\dot \Sigma,j) \to (Q \times \mathbb{R}, \widetilde J)$ on the symplectization of contact manifold $(Q,\lambda)$ as a subcase of the analysis of contact instantons $w:\dot \Sigma \to Q$, i.e., of the maps $w$ satisfying the equation $$ {\bar{\partial}}^\pi w = 0, \, d(w^*\lambda \circ j) = 0 $$ on the contact manifold $(Q,\lambda)$, which has been carried out by a coordinate-free covariant tensorial calculus. When the analysis is applied to that of pseudoholomorphic curves $u = (w,f)$ with $w = \pi_Q \circ u$, $f = s\circ u$ on symplectization, the outcome is generally stronger and more accurate than the common results on the regularity presented in the literature in that all of our a priori estimates can be written purely in terms $w$ not involving $f$. The a priori elliptic estimates for $w$ are largely consequences of various Weitzenb\"ock-type formulae with respect to the contact triad connection introduced by Wang and the first author in [OW14], and the estimate for $f$ is a consequence thereof by simple integration of the equation $df = w^*\lambda \circ j$. We also derive a simple precise tensorial formulae for the linearized operator and for the asymptotic operator that admit a perturbation theory of the operators with respect to (adapted) almost complex structures: The latter has been missing in the analysis of pseudoholomorphic curves on symplectization in the existing literature., Comment: 75 pages; Comments welcome; v2) typos corrected, one reference added; v3) The section on asymptotic analysis rewritten by correcting several misstatements thereon based on the results from our new arXiv posting arXiv:2303.01011 and a few new references added

Wake-up words (WUW) is a short sentence used to activate a speech recognition system to receive the user's speech input. WUW utterances include not only the lexical information for waking up the system but also non-lexical information such as speaker identity or emotion. In particular, recognizing the user's emotional state may elaborate the voice communication. However, there is few dataset where the emotional state of the WUW utterances is labeled. In this paper, we introduce Hi, KIA, a new WUW dataset which consists of 488 Korean accent emotional utterances collected from four male and four female speakers and each of utterances is labeled with four emotional states including anger, happy, sad, or neutral. We present the step-by-step procedure to build the dataset, covering scenario selection, post-processing, and human validation for label agreement. Also, we provide two classification models for WUW speech emotion recognition using the dataset. One is based on traditional hand-craft features and the other is a transfer-learning approach using a pre-trained neural network. These classification models could be used as benchmarks in further research., Comment: Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2022

Recent advancements in autonomous technology allow for new opportunities in vehicle interior design. Such a shift in in-vehicle activity suggests vehicle interior spaces should provide an adequate manner by considering users' affective desires. Therefore, this study aims to investigate the affective role of future vehicle interiors. Thirty one participants in ten focus groups were interviewed about challenges they face regarding their current vehicle interior and expectations they have for future vehicles. Results from content analyses revealed the affective role of future vehicle interiors. Advanced exclusiveness and advanced convenience were two primary aspects identified. The identified affective roles of each aspect are a total of eight visceral levels, four visceral levels each, including focused, stimulating, amused, pleasant, safe, comfortable, accommodated, and organized. We expect the results from this study to lead to the development of affective vehicle interiors by providing the fundamental knowledge for developing conceptual direction and evaluating its impact on user experiences., Comment: 15 pages, 4 figures, 2 tables

This study investigates the emotional responses to the color of vehicle interior lighting using self-assessment and electroencephalography (EEG). The study was divided into two sessions: the first session investigated the potential of ambient lighting colors, and the second session was used to develop in-vehicle lighting color guidelines. Every session included thirty subjects. In the first session, four lighting colors were assessed using seventeen adjectives. As a result, 'Preference, Softness, Brightness, and Uniqueness were found to be the four factors that best characterize the atmospheric properties of interior lighting in vehicles. Ambient illumination, according to EEG data, increased people's arousal and lowered their alpha waves. The following session investigated a wider spectrum of colors using four factors extracted from the previous session. As a result, bluish and purplish lighting colors had the highest preference and uniqueness among ten lighting colors. Green received an intermediate preference and a high uniqueness score. With its great brightness and softness, Neutral White also achieved an intermediate preference rating. Despite receiving a low preference rating, warm colors were considered to be soft. Red was the least preferred color, but its uniqueness and roughness were highly rated. This study is expected to provide a basic theory on emotional lighting guidelines in the vehicle context, providing manufacturers with objective rationale.

Expressive text-to-speech has shown improved performance in recent years. However, the style control of synthetic speech is often restricted to discrete emotion categories and requires training data recorded by the target speaker in the target style. In many practical situations, users may not have reference speech recorded in target emotion but still be interested in controlling speech style just by typing text description of desired emotional style. In this paper, we propose a text-based interface for emotional style control and cross-speaker style transfer in multi-speaker TTS. We propose the bi-modal style encoder which models the semantic relationship between text description embedding and speech style embedding with a pretrained language model. To further improve cross-speaker style transfer on disjoint, multi-style datasets, we propose the novel style loss. The experimental results show that our model can generate high-quality expressive speech even in unseen style., Comment: Accepted to Interspeech 2022

An automated design data archiving could reduce the time wasted by designers from working creatively and effectively. Though many datasets on classifying, detecting, and instance segmenting on car exterior exist, these large datasets are not relevant for design practices as the primary purpose lies in autonomous driving or vehicle verification. Therefore, we release GP22, composed of car styling features defined by automotive designers. The dataset contains 1480 car side profile images from 37 brands and ten car segments. It also contains annotations of design features that follow the taxonomy of the car exterior design features defined in the eye of the automotive designer. We trained the baseline model using YOLO v5 as the design feature detection model with the dataset. The presented model resulted in an mAP score of 0.995 and a recall of 0.984. Furthermore, exploration of the model performance on sketches and rendering images of the car side profile implies the scalability of the dataset for design purposes., Comment: 5th CVFAD workshop, CVPR2022

We present EdiTTS, an off-the-shelf speech editing methodology based on score-based generative modeling for text-to-speech synthesis. EdiTTS allows for targeted, granular editing of audio, both in terms of content and pitch, without the need for any additional training, task-specific optimization, or architectural modifications to the score-based model backbone. Specifically, we apply coarse yet deliberate perturbations in the Gaussian prior space to induce desired behavior from the diffusion model while applying masks and softening kernels to ensure that iterative edits are applied only to the target region. Through listening tests and speech-to-text back transcription, we show that EdiTTS outperforms existing baselines and produces robust samples that satisfy user-imposed requirements., Comment: 4 pages, 3 figures, 3 tables, INTERSPEECH 2022

We study Morse theory on noncompact manifolds equipped with exhaustions by compact pieces, defining the Morse homology of a pair which consists of the manifold and related geometric/homotopy data. We construct a collection of Morse data parametrized by cubes of arbitrary dimensions. From this collection, we obtain a family of linear maps subject to some coherency conditions, which can be packaged into a homotopy coherent diagram. We introduce a chain complex which is a colimit for the diagram and show that it computes the Morse homology., Comment: 71 pages, 9 figures

This paper describes a fast speaker search system to retrieve segments of the same voice identity in the large-scale data. A recent study shows that Locality Sensitive Hashing (LSH) enables quick retrieval of a relevant voice in the large-scale data in conjunction with i-vector while maintaining accuracy. In this paper, we proposed Random Speaker-variability Subspace (RSS) projection to map a data into LSH based hash tables. We hypothesized that rather than projecting on completely random subspace without considering data, projecting on randomly generated speaker variability space would give more chance to put the same speaker representation into the same hash bins, so we can use less number of hash tables. Multiple RSS can be generated by randomly selecting a subset of speakers from a large speaker cohort. From the experimental result, the proposed approach shows 100 times and 7 times faster than the linear search and LSH, respectively, Comment: Interspeech 2019

Although there are more than 6,500 languages in the world, the pronunciations of many phonemes sound similar across the languages. When people learn a foreign language, their pronunciation often reflects their native language's characteristics. This motivates us to investigate how the speech synthesis network learns the pronunciation from datasets from different languages. In this study, we are interested in analyzing and taking advantage of multilingual speech synthesis network. First, we train the speech synthesis network bilingually in English and Korean and analyze how the network learns the relations of phoneme pronunciation between the languages. Our experimental result shows that the learned phoneme embedding vectors are located closer if their pronunciations are similar across the languages. Consequently, the trained networks can synthesize the English speakers' Korean speech and vice versa. Using this result, we propose a training framework to utilize information from a different language. To be specific, we pre-train a speech synthesis network using datasets from both high-resource language and low-resource language, then we fine-tune the network using the low-resource language dataset. Finally, we conducted more simulations on 10 different languages to show it is generally extendable to other languages.

We propose prosody embeddings for emotional and expressive speech synthesis networks. The proposed methods introduce temporal structures in the embedding networks, thus enabling fine-grained control of the speaking style of the synthesized speech. The temporal structures can be designed either on the speech side or the text side, leading to different control resolutions in time. The prosody embedding networks are plugged into end-to-end speech synthesis networks and trained without any other supervision except for the target speech for synthesizing. It is demonstrated that the prosody embedding networks learned to extract prosodic features. By adjusting the learned prosody features, we could change the pitch and amplitude of the synthesized speech both at the frame level and the phoneme level. We also introduce the temporal normalization of prosody embeddings, which shows better robustness against speaker perturbations during prosody transfer tasks., Comment: ICASSP 2019, best viewed in color

We propose a neural text-to-speech (TTS) model that can imitate a new speaker's voice using only a small amount of speech sample. We demonstrate voice imitation using only a 6-seconds long speech sample without any other information such as transcripts. Our model also enables voice imitation instantly without additional training of the model. We implemented the voice imitating TTS model by combining a speaker embedder network with a state-of-the-art TTS model, Tacotron. The speaker embedder network takes a new speaker's speech sample and returns a speaker embedding. The speaker embedding with a target sentence are fed to Tacotron, and speech is generated with the new speaker's voice. We show that the speaker embeddings extracted by the speaker embedder network can represent the latent structure in different voices. The generated speech samples from our model have comparable voice quality to the ones from existing multi-speaker TTS models.

Artificial Neural Network computation relies on intensive vector-matrix multiplications. Recently, the emerging nonvolatile memory (NVM) crossbar array showed a feasibility of implementing such operations with high energy efficiency, thus there are many works on efficiently utilizing emerging NVM crossbar array as analog vector-matrix multiplier. However, its nonlinear I-V characteristics restrain critical design parameters, such as the read voltage and weight range, resulting in substantial accuracy loss. In this paper, instead of optimizing hardware parameters to a given neural network, we propose a methodology of reconstructing a neural network itself optimized to resistive memory crossbar arrays. To verify the validity of the proposed method, we simulated various neural network with MNIST and CIFAR-10 dataset using two different specific Resistive Random Access Memory (RRAM) model. Simulation results show that our proposed neural network produces significantly higher inference accuracies than conventional neural network when the synapse devices have nonlinear I-V characteristics., Comment: 14 pages