Search

Large language models (LLMs) have emerged due to their capability to generate high-quality content across diverse contexts. To reduce their explosively increasing demands for computing resources, a mixture of experts (MoE) has emerged. The MoE layer enables exploiting a huge number of parameters with less computation. Applying state-of-the-art continuous batching increases throughput; however, it leads to frequent DRAM access in the MoE and attention layers. We observe that conventional computing devices have limitations when processing the MoE and attention layers, which dominate the total execution time and exhibit low arithmetic intensity (Op/B). Processing MoE layers only with devices targeting low-Op/B such as processing-in-memory (PIM) architectures is challenging due to the fluctuating Op/B in the MoE layer caused by continuous batching. To address these challenges, we propose Duplex, which comprises xPU tailored for high-Op/B and Logic-PIM to effectively perform low-Op/B operation within a single device. Duplex selects the most suitable processor based on the Op/B of each layer within LLMs. As the Op/B of the MoE layer is at least 1 and that of the attention layer has a value of 4-8 for grouped query attention, prior PIM architectures are not efficient, which place processing units inside DRAM dies and only target extremely low-Op/B (under one) operations. Based on recent trends, Logic-PIM adds more through-silicon vias (TSVs) to enable high-bandwidth communication between the DRAM die and the logic die and place powerful processing units on the logic die, which is best suited for handling low-Op/B operations ranging from few to a few dozens. To maximally utilize the xPU and Logic-PIM, we propose expert and attention co-processing., Comment: 15 pages, 16 figures, accepted at MICRO 2024

Fully homomorphic encryption (FHE) is a cryptographic technology capable of resolving security and privacy problems in cloud computing by encrypting data in use. However, FHE introduces tremendous computational overhead for processing encrypted data, causing FHE workloads to become 2-6 orders of magnitude slower than their unencrypted counterparts. To mitigate the overhead, we propose Cheddar, an FHE library for CUDA GPUs, which demonstrates significantly faster performance compared to prior GPU implementations. We develop optimized functionalities at various implementation levels ranging from efficient low-level primitives to streamlined high-level operational sequences. Especially, we improve major FHE operations, including number-theoretic transform and base conversion, based on efficient kernel designs using a small word size of 32 bits. By these means, Cheddar demonstrates 2.9 to 25.6 times higher performance for representative FHE workloads compared to prior GPU implementations., Comment: 12 pages, 5 figures

We present a method for producing a flat, large-area Fermi gas of $^6$Li with a uniform area density. The method uses a programmable optical potential within a feedback loop to flatten the in-plane trapping potential for atoms. The optical potential is generated using a laser beam, whose intensity profile is adjusted by a spatial light modulator and optimized through measurements of the density distribution of the sample. The resulting planar sample exhibits a uniform area density within a region of about 480 $\mu$m in diameter and the standard deviation of the trap bottom potential is estimated to be $\approx k_B \times$ 6.1 nK, which is less than 20$\%$ of the transverse confinement energy. We discuss a dimensional crossover toward 2D regime by reducing the number of atoms in the planar trap, including the effect of the spatial variation of the transverse trapping frequency in the large-area sample., Comment: 8 pages, 6 figures

We present a turbulence-sustaining mechanism in a spinor Bose-Einstein condensate, which is based on the chaotic nature of internal spin dynamics. Magnetic driving induces a complete chaotic evolution of the local spin state, thereby continuously randomizing the spin texture of the condensate to maintain the turbulent state. We experimentally demonstrate the onset of turbulence in the driven condensate as the driving frequency changes and show that it is consistent with the regular-to-chaotic transition of the local spin dynamics. This chaos-assisted turbulence establishes the spin-driven spinor condensate as an intriguing platform for exploring quantum chaos and related superfluid turbulence phenomena., Comment: 10 pages, 8 figures

Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critical problem of the enormous computational cost for the FHE evaluation of convolutional layers (conv2d), mainly due to the high cost of data reordering and bootstrapping. We first propose an encoding method introducing nested structures inside encoded vectors for FHE, which enables us to develop efficient conv2d algorithms with reduced data reordering costs. However, the new encoding method also introduces additional computations for conversion between encoding methods, which could negate its advantages. We discover that fusing conv2d with bootstrapping eliminates such computations while reducing the cost of bootstrapping. Then, we devise optimized execution flows for various types of conv2d and apply them to end-to-end implementation of CNNs. NeuJeans accelerates the performance of conv2d by up to 5.68 times compared to state-of-the-art FHE-based PI work and performs the PI of a CNN at the scale of ImageNet (ResNet18) within a mere few seconds, Comment: 16 pages, 9 figures

In electrochemistry, reactions and charge-transfer are to a large extent determined by the atomistic structure of the solid-liquid interface. Yet due to the presence of the liquid electrolyte, many surface-science methods cannot be applied here. Hence, the exact microscopic structure that is present under operating conditions often remains unknown. Reflection anisotropy spectroscopy (RAS) is one of the few techniques that allow for an in operando investigation of the structure of solid-liquid interfaces. However, an interpretation of RAS data on the atomistic scale can only be obtained by comparison to computational spectroscopy. While the number of computational RAS studies related to electrochemical systems is currently still limited, those studies so far have not taken into account the dynamic nature of the solid-liquid interface. In this work, we investigate the temporal evolution of the spectroscopic response of the Au(110) missing row reconstruction in contact with water by combining ab initio molecular dynamics with computational spectroscopy. Our results show significant changes in the time evolution of the RA spectra, in particular providing an explanation for the typically observed differences in intensity when comparing theory and experiment. Moreover, these findings point to the importance of structural surface/interface variability while at the same time emphasising the potential of RAS for probing these dynamic interfaces., Comment: 15 pages, 5 figures

Incorporating fully homomorphic encryption (FHE) into the inference process of a convolutional neural network (CNN) draws enormous attention as a viable approach for achieving private inference (PI). FHE allows delegating the entire computation process to the server while ensuring the confidentiality of sensitive client-side data. However, practical FHE implementation of a CNN faces significant hurdles, primarily due to FHE's substantial computational and memory overhead. To address these challenges, we propose a set of optimizations, which includes GPU/ASIC acceleration, an efficient activation function, and an optimized packing scheme. We evaluate our method using the ResNet models on the CIFAR-10 and ImageNet datasets, achieving several orders of magnitude improvement compared to prior work and reducing the latency of the encrypted CNN inference to 1.4 seconds on an NVIDIA A100 GPU. We also show that the latency drops to a mere 0.03 seconds with a custom hardware design., Comment: 3 pages, 1 figure, appears at DISCC 2023 (2nd Workshop on Data Integrity and Secure Cloud Computing, in conjunction with the 56th International Symposium on Microarchitecture (MICRO 2023))

Fully homomorphic encryption (FHE) is in the spotlight as a definitive solution for privacy, but the high computational overhead of FHE poses a challenge to its practical adoption. Although prior studies have attempted to design ASIC accelerators to mitigate the overhead, their designs require excessive chip resources (e.g., areas) to contain and process massive data for FHE operations. We propose CiFHER, a chiplet-based FHE accelerator with a resizable structure, to tackle the challenge with a cost-effective multi-chip module (MCM) design. First, we devise a flexible core architecture whose configuration is adjustable to conform to the global organization of chiplets and design constraints. Its distinctive feature is a composable functional unit providing varying computational throughput for the number-theoretic transform, the most dominant function in FHE. Then, we establish generalized data mapping methodologies to minimize the interconnect overhead when organizing the chips into the MCM package in a tiled manner, which becomes a significant bottleneck due to the packaging constraints. This study demonstrates that a CiFHER package composed of a number of compact chiplets provides performance comparable to state-of-the-art monolithic ASIC accelerators while significantly reducing the package-wide power consumption and manufacturing cost., Comment: 12 pages, 10 figures, to appear in 2024 International Symposium on Secure and Private Execution Environment Design (SEED)

We numerically investigate the stationary turbulent states of spin-1 Bose-Einstein condensates under continuous spin driving. We analyze the entanglement entropy and magnetization correlation function to demonstrate the isotropic nature of the intricate spin texture that is generated in the nonequilibrium steady state. We observe a $-7/3$ power-law behavior in the spin-dependent interaction energy spectrum. To gain further insight into the statistical properties of the spin texture, we introduce a spin state ensemble obtained through position projection, revealing its close resemblance to the Haar random ensemble for spin-1 systems. We also present the probability distribution of the spin vector magnitude in the turbulent condensate, which can be tested in experiments. Our numerical study highlights the characteristics of stationary turbulence in the spinor BEC system and confirms previous experimental findings by Hong et al. [Phys. Rev. A 108, 013318 (2023)]., Comment: 11 pages, 7 figures

While generative modeling on multimodal image-text data has been actively developed with large-scale paired datasets, there have been limited attempts to generate both image and text data by a single model rather than a generation of one fixed modality conditioned on the other modality. In this paper, we explore a unified generative vision-and-language (VL) model that can produce both images and text sequences. Especially, we propose a generative VL transformer based on the non-autoregressive mask prediction, named MAGVLT, and compare it with an autoregressive generative VL transformer (ARGVLT). In comparison to ARGVLT, the proposed MAGVLT enables bidirectional context encoding, fast decoding by parallel token predictions in an iterative refinement, and extended editing capabilities such as image and text infilling. For rigorous training of our MAGVLT with image-text pairs from scratch, we combine the image-to-text, text-to-image, and joint image-and-text mask prediction tasks. Moreover, we devise two additional tasks based on the step-unrolled mask prediction and the selective prediction on the mixture of two image-text pairs. Experimental results on various downstream generation tasks of VL benchmarks show that our MAGVLT outperforms ARGVLT by a large margin even with significant inference speedup. Particularly, MAGVLT achieves competitive results on both zero-shot image-to-text and text-to-image generation tasks from MS-COCO by one moderate-sized model (fewer than 500M parameters) even without the use of monomodal data and networks., Comment: CVPR 2023

Convolutional neural network (CNN) inference using fully homomorphic encryption (FHE) is a promising private inference (PI) solution due to the capability of FHE that enables offloading the whole computation process to the server while protecting the privacy of sensitive user data. Prior FHE-based CNN (HCNN) work has demonstrated the feasibility of constructing deep neural network architectures such as ResNet using FHE. Despite these advancements, HCNN still faces significant challenges in practicality due to the high computational and memory overhead. To overcome these limitations, we present HyPHEN, a deep HCNN construction that incorporates novel convolution algorithms (RAConv and CAConv), data packing methods (2D gap packing and PRCR scheme), and optimization techniques tailored to HCNN construction. Such enhancements enable HyPHEN to substantially reduce the memory footprint and the number of expensive homomorphic operations, such as ciphertext rotation and bootstrapping. As a result, HyPHEN brings the latency of HCNN CIFAR-10 inference down to a practical level at 1.4 seconds (ResNet-20) and demonstrates HCNN ImageNet inference for the first time at 14.7 seconds (ResNet-18)., Comment: 15 pages, 12 figures

We report the observation of stationary turbulence in antiferromagnetic spin-1 Bose-Einstein condensates driven by a radio-frequency magnetic field. The magnetic driving injects energy into the system by spin rotation and the energy is dissipated via dynamic instability, resulting in the emergence of an irregular spin texture in the condensate. Under continuous driving, the spinor condensate evolves into a nonequilibrium steady state with characteristic spin turbulence, while the low energy scale of spin excitations ensures that the sample's lifetime is minimally affected. When the driving strength is on par with the system's spin interaction energy and the quadratic Zeeman energy, remarkably, the stationary turbulent state exhibits spin-isotropic features in spin composition and spatial spin texture. We numerically show that ambient field fluctuations play a crucial role in sustaining the turbulent state within the system. These results open up new avenues for exploring quantum turbulence in spinor superfluid systems., Comment: 9 pages, 9 figures

Episodic count has been widely used to design a simple yet effective intrinsic motivation for reinforcement learning with a sparse reward. However, the use of episodic count in a high-dimensional state space as well as over a long episode time requires a thorough state compression and fast hashing, which hinders rigorous exploitation of it in such hard and complex exploration environments. Moreover, the interference from task-irrelevant observations in the episodic count may cause its intrinsic motivation to overlook task-related important changes of states, and the novelty in an episodic manner can lead to repeatedly revisit the familiar states across episodes. In order to resolve these issues, in this paper, we propose a learnable hash-based episodic count, which we name LECO, that efficiently performs as a task-specific intrinsic reward in hard exploration problems. In particular, the proposed intrinsic reward consists of the episodic novelty and the task-specific modulation where the former employs a vector quantized variational autoencoder to automatically obtain the discrete state codes for fast counting while the latter regulates the episodic novelty by learning a modulator to optimize the task-specific extrinsic reward. The proposed LECO specifically enables the automatic transition from exploration to exploitation during reinforcement learning. We experimentally show that in contrast to the previous exploration methods LECO successfully solves hard exploration problems and also scales to large state spaces through the most difficult tasks in MiniGrid and DMLab environments., Comment: Accepted to NeurIPS 2022

Indium phosphide and derived compound semiconductors are materials often involved in high-efficiency solar water splitting due to their versatile opto-electronic properties. Surface corrosion, however, typically deteriorates the performance of photoelectrochemical solar cells based on this material class. It has been reported that (photo)electrochemical surface functionalisation protects the surface by combining etching and controlled corrosion. Nevertheless, the overall involved process is not fully understood. Therefore, access to the electrochemical interface structure under operando conditions is crucial for a more detailed understanding. One approach for gaining structural insight is the use of operando reflection anisotropy spectroscopy. This technique allows the time-resolved investigation of the interfacial structure while applying potentials in the electrolyte. In this study, p-doped InP(100) surfaces are cycled between anodic and cathodic potentials in two different electrolytes, hydrochloric acid and sulphuric acid. For low, 10 mM electrolyte concentrations, we observe a reversible processes related to the reduction of a surface oxide phase in the cathodic potential range which is reformed near open-circuit potentials. Higher concentrations of 0.5 N, however, already lead to initial surface corrosion., Comment: 10 pages, 6 figures

Convergence in self-consistent-field cycles can be a major computational bottleneck of density-functional theory calculations. We propose a Resta-like preconditioning method for full-potential all-electron calculations in the linearized augmented planewave (LAPW) method to smoothly converge to self-consistency. We implemented this preconditioner in the exciting code and apply it to the two semiconducting systems of MoS$_2$ slabs and P-rich GaP(100) surfaces as well as the metallic system Au(111), containing a sufficiently large amount of vacuum. Our calculations demonstrate that the implemented scheme performs reliably as well as more efficiently regardless of system size, suppressing long-range charge sloshing. While the suitability of this preconditioning higher for semiconducting systems, the convergence for metals is only slightly decreased and thus still trustworthy to apply. Furthermore, a mixing algorithm with the preconditioner shows an improvement over that with the Kerker preconditioner for the investigated semiconducting systems.

Homomorphic Encryption (HE) is one of the most promising post-quantum cryptographic schemes that enable privacy-preserving computation on servers. However, noise accumulates as we perform operations on HE-encrypted data, restricting the number of possible operations. Fully HE (FHE) removes this restriction by introducing the bootstrapping operation, which refreshes the data; however, FHE schemes are highly memory-bound. Bootstrapping, in particular, requires loading GBs of evaluation keys and plaintexts from off-chip memory, which makes FHE acceleration fundamentally bottlenecked by the off-chip memory bandwidth. In this paper, we propose ARK, an Accelerator for FHE with Runtime data generation and inter-operation Key reuse. ARK enables practical FHE workloads with a novel algorithm-architecture co-design to accelerate bootstrapping. We first eliminate the off-chip memory bandwidth bottleneck through runtime data generation and inter-operation key reuse. This approach enables ARK to fully exploit on-chip memory by substantially reducing the size of the working set. On top of such algorithmic enhancements, we build ARK microarchitecture that minimizes on-chip data movement through an efficient, alternating data distribution policy based on the data access patterns and a streamlined dataflow organization of the tailored functional units -- including base conversion, number-theoretic transform, and automorphism units. Overall, our co-design effectively handles the heavy computation and data movement overheads of FHE, drastically reducing the cost of HE operations, including bootstrapping., Comment: 18 pages, 9 figures

In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge. Participants were tasked with developing a program or agent that can win (i.e., 'ascend' in) the popular dungeon-crawler game of NetHack by interacting with the NetHack Learning Environment (NLE), a scalable, procedurally generated, and challenging Gym environment for reinforcement learning (RL). The challenge showcased community-driven progress in AI with many diverse approaches significantly beating the previously best results on NetHack. Furthermore, it served as a direct comparison between neural (e.g., deep RL) and symbolic AI, as well as hybrid systems, demonstrating that on NetHack symbolic bots currently outperform deep RL by a large margin. Lastly, no agent got close to winning the game, illustrating NetHack's suitability as a long-term benchmark for AI research., Comment: Under review at PMLR for the NeuRIPS 2021 Competition Workshop Track, 10 pages + 10 in appendices

Homomorphic encryption (HE) enables the secure offloading of computations to the cloud by providing computation on encrypted data (ciphertexts). HE is based on noisy encryption schemes in which noise accumulates as more computations are applied to the data. The limited number of operations applicable to the data prevents practical applications from exploiting HE. Bootstrapping enables an unlimited number of operations or fully HE (FHE) by refreshing the ciphertext. Unfortunately, bootstrapping requires a significant amount of additional computation and memory bandwidth as well. Prior works have proposed hardware accelerators for computation primitives of FHE. However, to the best of our knowledge, this is the first to propose a hardware FHE accelerator that supports bootstrapping as a first-class citizen. In particular, we propose BTS - Bootstrappable, Technologydriven, Secure accelerator architecture for FHE. We identify the challenges of supporting bootstrapping in the accelerator and analyze the off-chip memory bandwidth and computation required. In particular, given the limitations of modern memory technology, we identify the HE parameter sets that are efficient for FHE acceleration. Based on the insights gained from our analysis, we propose BTS, which effectively exploits the parallelism innate in HE operations by arranging a massive number of processing elements in a grid. We present the design and microarchitecture of BTS, including a network-on-chip design that exploits a deterministic communication pattern. BTS shows 5,556x and 1,306x improved execution time on ResNet-20 and logistic regression over a CPU, with a chip area of 373.6mm^2 and up to 163.2W of power., Comment: 15 pages, 10 figures

Large-batch training has been essential in leveraging large-scale datasets and models in deep learning. While it is computationally beneficial to use large batch sizes, it often requires a specially designed learning rate (LR) schedule to achieve a comparable level of performance as in smaller batch training. Especially, when the number of training epochs is constrained, the use of a large LR and a warmup strategy is critical in the final performance of large-batch training due to the reduced number of updating steps. In this work, we propose an automated LR scheduling algorithm which is effective for neural network training with a large batch size under the given epoch budget. In specific, the whole schedule consists of two phases: adaptive warmup and predefined decay, where the LR is increased until the training loss no longer decreases and decreased to zero until the end of training. Here, whether the training loss has reached the minimum value is robustly checked with Gaussian process smoothing in an online manner with a low computational burden. Coupled with adaptive stochastic optimizers such as AdamP and LAMB, the proposed scheduler successfully adjusts the LRs without cumbersome hyperparameter tuning and achieves comparable or better performances than tuned baselines on various image classification benchmarks and architectures with a wide range of batch sizes., Comment: 15 pages, 7 figures, 4 tables, 8th ICML Workshop on Automated Machine Learning (2021)

We devise a mixing algorithm for full-potential (FP) all-electron calculations in the linearized augmented planewave (LAPW) method. Pulay's direct inversion in the iterative subspace is complemented with the Kerker preconditioner and further improvements to achieve smooth convergence, avoiding charge sloshing and noise in the exchange-correlation potential. As the Kerker preconditioner was originally designed for the planewave basis, we have adapted it to the FP-LAPW method and implemented in the exciting code. Applications to the $2\times 2$ Au(111) surface with a vacancy and to the Pd(111) surface demonstrate that this approach and our implementation work reliably with both density and potential mixing.

The possibility of modifying the work function of electrodes is important for optimizing the energy barriers for charge-injection (extraction) at the interface to an organic material. In this study, we perform density-functional-theory calculations to investigate the impact of dithiol-terminated polyethylene glycol (PEG(thiol)) based self-assembled monolayers (SAMs) with different numbers of PEG repeat units on the work function of the Au(111) surface. We find that a monolayer of PEG(thiol) decreases the work function of the Au(111) surface, where the magnitude of this reduction strongly depends on the length of the PEG backbone. The main contribution arises from the dipole due to the adsorption-induced charge rearrangement at the interface. Our work reveals a pronounced odd-even effect, which can be traced back to the dipole moment of the PEG(thiol) layer.

In this paper, we propose a novel edge-labeling graph neural network (EGNN), which adapts a deep neural network on the edge-labeling graph, for few-shot learning. The previous graph neural network (GNN) approaches in few-shot learning have been based on the node-labeling framework, which implicitly models the intra-cluster similarity and the inter-cluster dissimilarity. In contrast, the proposed EGNN learns to predict the edge-labels rather than the node-labels on the graph that enables the evolution of an explicit clustering by iteratively updating the edge-labels with direct exploitation of both intra-cluster similarity and the inter-cluster dissimilarity. It is also well suited for performing on various numbers of classes without retraining, and can be easily extended to perform a transductive inference. The parameters of the EGNN are learned by episodic training with an edge-labeling loss to obtain a well-generalizable model for unseen low-data problem. On both of the supervised and semi-supervised few-shot image classification tasks with two benchmark datasets, the proposed EGNN significantly improves the performances over the existing GNNs., Comment: accepted to CVPR 2019

Recent work has shown how chemical reaction network theory may be used to design dynamical systems that can be implemented biologically in nucleic acid-based chemistry. While this has allowed the construction of advanced open-loop circuitry based on cascaded DNA strand displacement (DSD) reactions, little progress has so far been made in developing the requisite theoretical machinery to inform the systematic design of feedback controllers in this context. Here, we develop a number of foundational theoretical results on the equilibria, stability, and dynamics of nucleic acid controllers. In particular, we show that the implementation of feedback controllers using DSD reactions introduces additional nonlinear dynamics, even in the case of purely linear designs, e.g. PI controllers. By decomposing the effects of these non-observable nonlinear dynamics, we show that, in general, the stability of the linear system design does not necessarily imply the stability of the underlying chemical network, which can be lost under experimental variability when feedback interconnections are introduced. We provide an in-depth theoretical analysis of an example illustrating this phenomenon, whereby the linear design does not capture the instability of the full nonlinear system implemented as a DSD reaction network, and we further confirm these results using VisualDSD, a bespoke software tool for simulating nucleic acid-based circuits. Our analysis highlight the many interesting and unique characteristics of this important new class of feedback control systems., Comment: Considerable extension with respect to the previous version: additional results on the equilibria and stability for this class of systems, and the simulation with DSD reactions are now carried the VisualDSD tool