Search

KV cache compression methods have mainly relied on scalar quantization techniques to reduce the memory requirements during decoding. In this work, we apply residual vector quantization, which has been widely used for high fidelity audio compression, to compress KV cache in large language models (LLM). We adapt the standard recipe with minimal changes to compress the output of any key or value projection matrix in a pretrained LLM: we scale the vector by its standard deviation, divide channels into groups and then quantize each group with the same residual vector quantizer. We learn the codebook using exponential moving average and there are no other learnable parameters including the input and output projections normally used in a vector quantization set up. We find that a residual depth of 8 recovers most of the performance of the unquantized model. We also find that grouping non-contiguous channels together works better than grouping contiguous channels for compressing key matrix and the method further benefits from a light weight finetuning of LLM together with the quantization. Overall, the proposed technique is competitive with existing quantization methods while being much simpler and results in 5.5x compression compared to half precision.

Audio deepfake detection is crucial to combat the malicious use of AI-synthesized speech. Among many efforts undertaken by the community, the ASVspoof challenge has become one of the benchmarks to evaluate the generalizability and robustness of detection models. In this paper, we present Reality Defender's submission to the ASVspoof5 challenge, highlighting a novel pretraining strategy which significantly improves generalizability while maintaining low computational cost during training. Our system SLIM learns the style-linguistics dependency embeddings from various types of bonafide speech using self-supervised contrastive learning. The learned embeddings help to discriminate spoof from bonafide speech by focusing on the relationship between the style and linguistics aspects. We evaluated our system on ASVspoof5, ASV2019, and In-the-wild. Our submission achieved minDCF of 0.1499 and EER of 5.5% on ASVspoof5 Track 1, and EER of 7.4% and 10.8% on ASV2019 and In-the-wild respectively., Comment: Accepted into ASVspoof5 workshop

The parametric optimization for the ultrasound computed tomography system is introduced. It is hypothesized that the pulse characteristic directly affects the information present in the reconstructed profile. The ultrasound excitation modes based on pulse-width modifications are studied to estimate the effect on reconstruction quality. Studies show that the pulse width affects the response of the transducer and, thus, the reconstruction. The ultrasound scanning parameters, mainly pulse width, are assessed and optimally set by an Artificial Intelligence driven process, according to the object without the requirement of a-priori information. The optimization study uses a novel intelligent object placement procedure to ensure repeatability of the same region of interest, a key requirement to minimize the error. Further, Kanpur Theorem 1 is implemented to evaluate the quality of the acquired projection data and discard inferior quality data. Scanning results corresponding to homogeneous and heterogeneous phantoms are presented. The image processing step involves deep learning model evaluating the dice coefficient for estimating the reconstruction quality if prior information about the inner profile is known or a classical error estimate otherwise. The models segmentation accuracy is 95.72 percentage and intersection over union score is 0.8842 on the validation dataset. The article also provides valuable insights about the development and low-level control of the system., Comment: 20 Pages, 13 figures

Image-based or morphological profiling is a rapidly expanding field wherein cells are "profiled" by extracting hundreds to thousands of unbiased, quantitative features from images of cells that have been perturbed by genetic or chemical perturbations. The Cell Painting assay is the most popular imaged-based profiling assay wherein six small-molecule dyes label eight cellular compartments and thousands of measurements are made, describing quantitative traits such as size, shape, intensity, and texture within the nucleus, cytoplasm, and whole cell (Cimini et al., 2023). We have created the Cell Painting Gallery, a publicly available collection of Cell Painting datasets, with granular dataset descriptions and access instructions. It is hosted by AWS on the Registry of Open Data (RODA). As of January 2024, the Cell Painting Gallery holds 656 terabytes (TB) of image and associated numerical data. It includes the largest publicly available Cell Painting dataset, in terms of perturbations tested (Joint Undertaking for Morphological Profiling or JUMP (Chandrasekaran et al., 2023)), along with many other canonical datasets using Cell Painting, close derivatives of Cell Painting (such as LipocyteProfiler (Laber et al., 2023) and Pooled Cell Painting (Ramezani et al., 2023))., Comment: 9 pages, 1 table

In this paper, we propose an efficient and accurate streaming speech recognition model based on the FastConformer architecture. We adapted the FastConformer architecture for streaming applications through: (1) constraining both the look-ahead and past contexts in the encoder, and (2) introducing an activation caching mechanism to enable the non-autoregressive encoder to operate autoregressively during inference. The proposed model is thoughtfully designed in a way to eliminate the accuracy disparity between the train and inference time which is common for many streaming models. Furthermore, our proposed encoder works with various decoder configurations including Connectionist Temporal Classification (CTC) and RNN-Transducer (RNNT) decoders. Additionally, we introduced a hybrid CTC/RNNT architecture which utilizes a shared encoder with both a CTC and RNNT decoder to boost the accuracy and save computation. We evaluate the proposed model on LibriSpeech dataset and a multi-domain large scale dataset and demonstrate that it can achieve better accuracy with lower latency and inference time compared to a conventional buffered streaming model baseline. We also showed that training a model with multiple latencies can achieve better accuracy than single latency models while it enables us to support multiple latencies with a single model. Our experiments also showed the hybrid architecture would not only speedup the convergence of the CTC decoder but also improves the accuracy of streaming models compared to single decoder models., Comment: Shorter version accepted to ICASSP 2024

Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient training and inference, we carefully redesigned Conformer with a novel downsampling schema. The proposed model, named Fast Conformer(FC), is 2.8x faster than the original Conformer, supports scaling to Billion parameters without any changes to the core architecture and also achieves state-of-the-art accuracy on Automatic Speech Recognition benchmarks. To enable transcription of long-form speech up to 11 hours, we replaced global attention with limited context attention post-training, while also improving accuracy through fine-tuning with the addition of a global token. Fast Conformer, when combined with a Transformer decoder also outperforms the original Conformer in accuracy and in speed for Speech Translation and Spoken Language Understanding., Comment: Accepted at ASRU 2023

An in-house developed 2D ultrasound computerized Tomography system is fully automated. Performance analysis of instrument and software interfacing soft tools, namely the LabVIEW, MATLAB, C, and Python, is presented. The instrument interfacing algorithms, hardware control algorithms, signal processing, and analysis codes are written using above mentioned soft tool platforms. Total of eight performance indices are used to compare the ease of (a) realtime control of electromechanical assembly, (b) sensors, instruments integration, (c) synchronized data acquisition, and (d) simultaneous raw data processing. It is found that C utilizes the least processing power and performs a lower number of processes to perform the same task. In runtime analysis (data acquisition and realtime control), LabVIEW performs best, taking 365.69s in comparison to MATLAB (623.83s), Python ( 1505.54s), and C (1252.03s) to complete the experiment. Python performs better in establishing faster interfacing and minimum RAM usage. LabVIEW is recommended for its fast process execution. C is recommended for the most economical implementation. Python is recommended for complex system automation having a very large number of components involved. This article provides a methodology to select optimal soft tools for instrument automation-related aspects., Comment: 15 pages, 9 figures, 2 tables, 23 references

Transformer architecture has gained popularity due to its ability to scale with large dataset. Consequently, there is a need to reduce the model size and latency, especially for on-device deployment. We focus on vision transformer proposed for image recognition task (Dosovitskiy et al., 2021), and explore the application of different compression techniques such as low rank approximation and pruning for this purpose. Specifically, we investigate a structured pruning method proposed recently in Zhu et al. (2021) and find that mostly feedforward blocks are pruned with this approach, that too, with severe degradation in accuracy. We propose a hybrid compression approach to mitigate this where we compress the attention blocks using low rank approximation and use the previously mentioned pruning with a lower rate for feedforward blocks in each transformer layer. Our technique results in 50% compression with 14% relative increase in classification error whereas we obtain 44% compression with 20% relative increase in error when only pruning is applied. We propose further enhancements to bridge the accuracy gap but leave it as a future work.

A 2D Compact ultrasound computerized tomography (UCT) system is developed. Fully automatic post processing tools involving signal and image processing are developed as well. Square of the amplitude values are used in transmission mode with natural 1.5 MHz frequency and rise time 10.4 ns and fall time 8.4 ns and duty cycle of 4.32%. Highest peak to corresponding trough values are considered as transmitting wave between transducers in direct line talk. Sensitivity analysis of methods to extract peak to corresponding trough per transducer are discussed in this paper. Total five methods are tested. These methods are taken from broad categories: (a) Conventional and (b) Artificial Intelligence (AI) based methods. Conventional methods, namely: (a) simple gradient based peak detection, (b) Fourier based, (c) wavelet transform are compared with AI based methods: (a) support vector machine (SVM), (b) artificial neural network (ANN). Classification step was performed as well to discard the signal which does not has contribution of transmission wave. It is found that conventional methods have better performance. Reconstruction error, accuracy, F-Score, recall, precision, specificity and MCC for 40 x 40 data 1600 data files are measured. Each data file contains 50,002 data point. Ten such data files are used for training the Neural Network. Each data file has 7/8 wave packets and each packet corresponds to one transmission amplitude data. Reconstruction error is found to be minimum for ANN method. Other performance indices show that FFT method is processing the UCT signal with best recovery., Comment: 10 Pages, 5 Figures

In this paper, we review various end-to-end automatic speech recognition algorithms and their optimization techniques for on-device applications. Conventional speech recognition systems comprise a large number of discrete components such as an acoustic model, a language model, a pronunciation model, a text-normalizer, an inverse-text normalizer, a decoder based on a Weighted Finite State Transducer (WFST), and so on. To obtain sufficiently high speech recognition accuracy with such conventional speech recognition systems, a very large language model (up to 100 GB) is usually needed. Hence, the corresponding WFST size becomes enormous, which prohibits their on-device implementation. Recently, fully neural network end-to-end speech recognition algorithms have been proposed. Examples include speech recognition systems based on Connectionist Temporal Classification (CTC), Recurrent Neural Network Transducer (RNN-T), Attention-based Encoder-Decoder models (AED), Monotonic Chunk-wise Attention (MoChA), transformer-based speech recognition systems, and so on. These fully neural network-based systems require much smaller memory footprints compared to conventional algorithms, therefore their on-device implementation has become feasible. In this paper, we review such end-to-end speech recognition models. We extensively discuss their structures, performance, and advantages compared to conventional algorithms., Comment: Accepted as an invited paper to the Asilomar Conference on Signals, Systems, and Computers 2020. Figures are slightly updated in Aug. 2021