Search

Large Language Models (LLMs) have showcased exceptional capabilities in various domains, attracting significant interest from both academia and industry. Despite their impressive performance, the substantial size and computational demands of LLMs pose considerable challenges for practical deployment, particularly in environments with limited resources. The endeavor to compress language models while maintaining their accuracy has become a focal point of research. Among the various methods, knowledge distillation has emerged as an effective technique to enhance inference speed without greatly compromising performance. This paper presents a thorough survey from three aspects: method, evaluation, and application, exploring knowledge distillation techniques tailored specifically for LLMs. Specifically, we divide the methods into white-box KD and black-box KD to better illustrate their differences. Furthermore, we also explored the evaluation tasks and distillation effects between different distillation methods, and proposed directions for future research. Through in-depth understanding of the latest advancements and practical applications, this survey provides valuable resources for researchers, paving the way for sustained progress in this field., Comment: 28 pages

Physical layer security (PLS) is a promising technology to secure wireless communications by exploiting the physical properties of the wireless channel. However, the passive nature of PLS creates a significant imbalance between the effort required by eavesdroppers and legitimate users to secure data. To address this imbalance, in this article, we propose a novel framework of physical layer deception (PLD), which combines PLS with deception technologies to actively counteract wiretapping attempts. Combining a two-stage encoder with randomized ciphering and non-orthogonal multiplexing, the PLD approach enables the wireless communication system to proactively counter eavesdroppers with deceptive messages. Relying solely on the superiority of the legitimate channel over the eavesdropping channel, the PLD framework can effectively protect the confidentiality of the transmitted messages, even against eavesdroppers who possess knowledge equivalent to that of the legitimate receiver. We prove the validity of the PLD framework with in-depth analyses and demonstrate its superiority over conventional PLS approaches with comprehensive numerical benchmarks., Comment: Submitted to IEEE Transactions on Wireless Communications

A stable and fast path linking arbitrary pair of quantum states is commonly desired for the engineering protocols inspired by stimulated Raman adiabatic passage, such as shortcut to adiabaticity and holonomic transformation. It has immediate applications in quantum control and quantum computation and is fundamental about the exact solution of a time-dependent Schr\"odinger equation. We construct a universal control framework based on the system dynamics within an ancillary picture, in which the system Hamiltonian is diagonal so that no transition exists among the ancillary bases during the time evolution. Practically the desired evolution path can be obtained by the von Neumann equation for the parametric ancillary bases. We demonstrate that our control framework can be reduced to the nonadiabatic holonomic quantum transformation, the Lewis-Riesenfeld theory for invariant, and the counterdiabatic driving method under distinct scenarios and conditions. Also it can be used to achieve the cyclic transfer of system population that could be a hard or complex problem for the existing methods. For the state engineering over a finite-dimensional quantum system, our work can provide a full-rank nonadiabatic time-evolution operator.

We introduce GreenCOD, a green method for detecting camouflaged objects, distinct in its avoidance of backpropagation techniques. GreenCOD leverages gradient boosting and deep features extracted from pre-trained Deep Neural Networks (DNNs). Traditional camouflaged object detection (COD) approaches often rely on complex deep neural network architectures, seeking performance improvements through backpropagation-based fine-tuning. However, such methods are typically computationally demanding and exhibit only marginal performance variations across different models. This raises the question of whether effective training can be achieved without backpropagation. Addressing this, our work proposes a new paradigm that utilizes gradient boosting for COD. This approach significantly simplifies the model design, resulting in a system that requires fewer parameters and operations and maintains high performance compared to state-of-the-art deep learning models. Remarkably, our models are trained without backpropagation and achieve the best performance with fewer than 20G Multiply-Accumulate Operations (MACs). This new, more efficient paradigm opens avenues for further exploration in green, backpropagation-free model training.

In the dynamic realms of machine learning and deep learning, the robustness and reliability of models are paramount, especially in critical real-world applications. A fundamental challenge in this sphere is managing Out-of-Distribution (OOD) samples, significantly increasing the risks of model misclassification and uncertainty. Our work addresses this challenge by enhancing the detection and management of OOD samples in neural networks. We introduce OOD-R (Out-of-Distribution-Rectified), a meticulously curated collection of open-source datasets with enhanced noise reduction properties. In-Distribution (ID) noise in existing OOD datasets can lead to inaccurate evaluation of detection algorithms. Recognizing this, OOD-R incorporates noise filtering technologies to refine the datasets, ensuring a more accurate and reliable evaluation of OOD detection algorithms. This approach not only improves the overall quality of data but also aids in better distinguishing between OOD and ID samples, resulting in up to a 2.5\% improvement in model accuracy and a minimum 3.2\% reduction in false positives. Furthermore, we present ActFun, an innovative method that fine-tunes the model's response to diverse inputs, thereby improving the stability of feature extraction and minimizing specificity issues. ActFun addresses the common problem of model overconfidence in OOD detection by strategically reducing the influence of hidden units, which enhances the model's capability to estimate OOD uncertainty more accurately. Implementing ActFun in the OOD-R dataset has led to significant performance enhancements, including an 18.42\% increase in AUROC of the GradNorm method and a 16.93\% decrease in FPR95 of the Energy method. Overall, our research not only advances the methodologies in OOD detection but also emphasizes the importance of dataset integrity for accurate algorithm evaluation.

The technological landscape is swiftly advancing towards large-scale systems, creating significant opportunities, particularly in the domain of Terahertz (THz) communications. Networks designed for massive connectivity, comprising numerous Internet of Things (IoT) devices, are at the forefront of this advancement. In this paper, we consider Wireless Power Transfer (WPT)-enabled networks that support these IoT devices with massive Ultra-Reliable and Low-Latency Communication (mURLLC) services.The focus of such networks is information freshness, with the Age-of-Information (AoI) serving as the pivotal performance metric. In particular, we aim to minimize the maximum AoI among IoT devices by optimizing the scheduling policy. Our analytical findings establish the convexity property of the problem, which can be solved efficiently. Furthermore, we introduce the concept of AoI-oriented cluster capacity, examining the relationship between the number of supported devices and the AoI performance in the network. Numerical simulations validate the advantage of our proposed approach in enhancing AoI performance, indicating its potential to guide the design of future THz communication systems for IoT applications requiring mURLLC services.

A stable and high-quality single-magnon state is desired by the single-magnon source for quantum information application with a macroscopic spin system. We consider a hybrid system where a magnon mode is directly coupled to a nonresonant superconducting qubit via the exchange interaction. The magnon and qubit are under the driving and probing fields with the same frequency, respectively. We find that the single-magnon probability $P_1$ can be maximized when the product of the magnon-driving field detuning and the qubit-probing field detuning is equivalent to the square of the magnon-qubit coupling strength, $\Delta_q\Delta_m=J^2$. Then, the double-magnon probability $P_2$ can be minimized by tuning the ratio of the probing intensity to the driving intensity and the relative phase between the two fields. Under these optimized conditions with accessible strong driving intensity and low decay rate, strong magnon blockade gives rise to a stable single-magnon state with a high quality. It features a large brightness (the single-magnon probability) $P_1\approx0.40$ and a high purity (the equal-time second-order correlation function) $g^{(2)}(0)\sim10^{-5}$. The two indicators as a whole prevail over the existing results for photon, phonon, and magnon modes with respect to a stable single-quantum state. The optimized conditions with a scalable modification $\Delta_q\Delta_m\approx NJ^2$ apply to the situation when one focus on only one of the $N$ magnon modes that are simultaneously coupled to a common qubit., Comment: 10 pages, 7 figures

Learning generalized representations from limited training samples is crucial for applying deep neural networks in low-resource scenarios. Recently, methods based on Contrastive Language-Image Pre-training (CLIP) have exhibited promising performance in few-shot adaptation tasks. To avoid catastrophic forgetting and overfitting caused by few-shot fine-tuning, existing works usually freeze the parameters of CLIP pre-trained on large-scale datasets, overlooking the possibility that some parameters might not be suitable for downstream tasks. To this end, we revisit CLIP's visual encoder with a specific focus on its distinctive attention pooling layer, which performs a spatial weighted-sum of the dense feature maps. Given that dense feature maps contain meaningful semantic information, and different semantics hold varying importance for diverse downstream tasks (such as prioritizing semantics like ears and eyes in pet classification tasks rather than side mirrors), using the same weighted-sum operation for dense features across different few-shot tasks might not be appropriate. Hence, we propose fine-tuning the parameters of the attention pooling layer during the training process to encourage the model to focus on task-specific semantics. In the inference process, we perform residual blending between the features pooled by the fine-tuned and the original attention pooling layers to incorporate both the few-shot knowledge and the pre-trained CLIP's prior knowledge. We term this method as Semantic-Aware FinE-tuning (SAFE). SAFE is effective in enhancing the conventional few-shot CLIP and is compatible with the existing adapter approach (termed SAFE-A).

Practical object detection application can lose its effectiveness on image inputs with natural distribution shifts. This problem leads the research community to pay more attention on the robustness of detectors under Out-Of-Distribution (OOD) inputs. Existing works construct datasets to benchmark the detector's OOD robustness for a specific application scenario, e.g., Autonomous Driving. However, these datasets lack universality and are hard to benchmark general detectors built on common tasks such as COCO. To give a more comprehensive robustness assessment, we introduce COCO-O(ut-of-distribution), a test dataset based on COCO with 6 types of natural distribution shifts. COCO-O has a large distribution gap with training data and results in a significant 55.7% relative performance drop on a Faster R-CNN detector. We leverage COCO-O to conduct experiments on more than 100 modern object detectors to investigate if their improvements are credible or just over-fitting to the COCO test set. Unfortunately, most classic detectors in early years do not exhibit strong OOD generalization. We further study the robustness effect on recent breakthroughs of detector's architecture design, augmentation and pre-training techniques. Some empirical findings are revealed: 1) Compared with detection head or neck, backbone is the most important part for robustness; 2) An end-to-end detection transformer design brings no enhancement, and may even reduce robustness; 3) Large-scale foundation models have made a great leap on robust object detection. We hope our COCO-O could provide a rich testbed for robustness study of object detection. The dataset will be available at https://github.com/alibaba/easyrobust/tree/main/benchmarks/coco_o., Comment: Accepted in ICCV2023, https://github.com/alibaba/easyrobust/tree/main/benchmarks/coco_o

Nonadiabatic holonomic quantum gates are high-speed and robust. Nevertheless, they were found to be more fragile than the adiabatic gates when systematic errors become dominant. Inspired by the dark-path scheme that was used to partially relieve the systematic error in the absence of external noise, we construct a universal set of nonadiabatic holonomic $N$-qubit gates using the Rydberg-Rydberg interaction between atoms under off-resonant driving. Based on an effective four-level configuration in the Rydberg-atom system, the modified nonadiabatic holonomic geometric gates present a clear resilience to both systematic error in the whole parametric range and external noise. In our scheme, the conventional ultrastrong interaction between control atoms and the target atom for the nonadiabatic holonomic quantum computation is compensated by the detuning of the driving fields on the target atom. That idea yields a deeper understanding about the holonomic transformation. Moreover, our scheme is compact and scale-free with respect to $N$. It is interesting to find that the three-qubit gate is less susceptible to errors than the double-qubit one.

A novel learning solution to image steganalysis based on the green learning paradigm, called Green Steganalyzer (GS), is proposed in this work. GS consists of three modules: 1) pixel-based anomaly prediction, 2) embedding location detection, and 3) decision fusion for image-level detection. In the first module, GS decomposes an image into patches, adopts Saab transforms for feature extraction, and conducts self-supervised learning to predict an anomaly score of their center pixel. In the second module, GS analyzes the anomaly scores of a pixel and its neighborhood to find pixels of higher embedding probabilities. In the third module, GS focuses on pixels of higher embedding probabilities and fuses their anomaly scores to make final image-level classification. Compared with state-of-the-art deep-learning models, GS achieves comparable detection performance against S-UNIWARD, WOW and HILL steganography schemes with significantly lower computational complexity and a smaller model size, making it attractive for mobile/edge applications. Furthermore, GS is mathematically transparent because of its modular design.

It is widely reported that deep generative models can classify out-of-distribution (OOD) samples as in-distribution with high confidence. In this work, we propose a hypothesis that this phenomenon is due to the reconstruction task, which can cause the generative model to focus too much on low-level features and not enough on semantic information. To address this issue, we introduce SR-OOD, an OOD detection framework that utilizes sample repairing to encourage the generative model to learn more than just an identity map. By focusing on semantics, our framework improves OOD detection performance without external data and label information. Our experimental results demonstrate the competitiveness of our approach in detecting OOD samples.

We propose a new security framework for physical layer security (PLS) in the finite blocklength (FBL) regime that incorporates deception technology, allowing for active countermeasures against potential eavesdroppers. Using a symmetric block cipher and power-domain non-orthogonal multiplexing (NOM), our approach is able to achieve high secured reliability while effectively deceiving the eavesdropper, and can benefit from increased transmission power. This work represents a promising direction for future research in PLS with deception technology., Comment: Accepted for presentation at SPAWC 2023. Appendices are omitted in the submitted version due to length limit. Minor corrections made in the latest arXiv version

Recent studies have shown that higher accuracy on ImageNet usually leads to better robustness against different corruptions. Therefore, in this paper, instead of following the traditional research paradigm that investigates new out-of-distribution corruptions or perturbations deep models may encounter, we conduct model debugging in in-distribution data to explore which object attributes a model may be sensitive to. To achieve this goal, we create a toolkit for object editing with controls of backgrounds, sizes, positions, and directions, and create a rigorous benchmark named ImageNet-E(diting) for evaluating the image classifier robustness in terms of object attributes. With our ImageNet-E, we evaluate the performance of current deep learning models, including both convolutional neural networks and vision transformers. We find that most models are quite sensitive to attribute changes. A small change in the background can lead to an average of 9.23\% drop on top-1 accuracy. We also evaluate some robust models including both adversarially trained models and other robust trained models and find that some models show worse robustness against attribute changes than vanilla models. Based on these findings, we discover ways to enhance attribute robustness with preprocessing, architecture designs, and training strategies. We hope this work can provide some insights to the community and open up a new avenue for research in robust computer vision. The code and dataset are available at https://github.com/alibaba/easyrobust., Comment: Accepted by CVPR2023

In a transfer-based attack against Automatic Speech Recognition (ASR) systems, attacks are unable to access the architecture and parameters of the target model. Existing attack methods are mostly investigated in voice assistant scenarios with restricted voice commands, prohibiting their applicability to more general ASR related applications. To tackle this challenge, we propose a novel contextualized attack with deletion, insertion, and substitution adversarial behaviors, namely TransAudio, which achieves arbitrary word-level attacks based on the proposed two-stage framework. To strengthen the attack transferability, we further introduce an audio score-matching optimization strategy to regularize the training process, which mitigates adversarial example over-fitting to the surrogate model. Extensive experiments and analysis demonstrate the effectiveness of TransAudio against open-source ASR models and commercial APIs.

A hybrid system established by the direct interaction between a magnon mode and a superconducting transmon qubit is used to realize a high-degree blockade for magnon. It is a fundamental way toward quantum manipulation at the level of a single magnon and preparation of single magnon sources. Through weakly driving the magnon and probing the qubit, our magnon-blockade proposal can be optimized when the transversal coupling strength between the magnon and qubit is equivalent to the detuning of the qubit and the probing field or that of the magnon and the driving field. Under this condition, the equal-time second-order correlation function $g^{(2)}(0)$ can be analytically minimized when the probing intensity is about three times the driving intensity. Moreover, the magnon blockade could be further enhanced by proper driving intensity and system decay rate, whose magnitudes outrange the current systems of cavity QED and cavity optomechanics. In particular, the correlation function achieves $g^{(2)}(0)\sim10^{-7}$, about two orders lower than that for the photon blockade in cavity optomechanics. Also, we discuss the effects on $g^{(2)}(0)$ from thermal noise and the extra longitudinal interaction between the magnon and qubit. Our optimized conditions for blockade are found to persist in these nonideal situations.

With the development of deep learning technology, the facial manipulation system has become powerful and easy to use. Such systems can modify the attributes of the given facial images, such as hair color, gender, and age. Malicious applications of such systems pose a serious threat to individuals' privacy and reputation. Existing studies have proposed various approaches to protect images against facial manipulations. Passive defense methods aim to detect whether the face is real or fake, which works for posterior forensics but can not prevent malicious manipulation. Initiative defense methods protect images upfront by injecting adversarial perturbations into images to disrupt facial manipulation systems but can not identify whether the image is fake. To address the limitation of existing methods, we propose a novel two-tier protection method named Information-containing Adversarial Perturbation (IAP), which provides more comprehensive protection for {facial images}. We use an encoder to map a facial image and its identity message to a cross-model adversarial example which can disrupt multiple facial manipulation systems to achieve initiative protection. Recovering the message in adversarial examples with a decoder serves passive protection, contributing to provenance tracking and fake image detection. We introduce a feature-level correlation measurement that is more suitable to measure the difference between the facial images than the commonly used mean squared error. Moreover, we propose a spectral diffusion method to spread messages to different frequency channels, thereby improving the robustness of the message against facial manipulation. Extensive experimental results demonstrate that our proposed IAP can recover the messages from the adversarial examples with high average accuracy and effectively disrupt the facial manipulation systems., Comment: \copyright 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

How to provide information security while fulfilling ultra reliability and low-latency requirements is one of the major concerns for enabling the next generation of ultra-reliable and low-latency communications service (xURLLC), specially in machine-type communications. In this work, we investigate the reliability-security tradeoff via defining the leakage-failure probability, which is a metric that jointly characterizes both reliability and security performances for short-packet transmissions. We discover that the system performance can be enhanced by counter-intuitively allocating fewer resources for the transmission with finite blocklength (FBL) codes. In order to solve the corresponding optimization problem for the joint resource allocation, we propose an optimization framework, that leverages lower-bounded approximations for the decoding error probability in the FBL regime. We characterize the convexity of the reformulated problem and establish an efficient iterative searching method, the convergence of which is guaranteed. To show the extendability of the framework, we further discuss the blocklength allocation schemes with practical requirements of reliable-secure performance, as well as the transmissions with the statistical channel state information (CSI). Numerical results verify the accuracy of the proposed approach and demonstrate the reliability-security tradeoff under various setups., Comment: To appear in IEEE Journal on Selected Areas of Communications

In the search for high-temperature superconductivity in hydrides, a plethora of multi-hydrogen superconductors have been theoretically predicted, and some have been synthesized experimentally under ultrahigh pressures of several hundred GPa. However, the impracticality of these high-pressure methods has been a persistent issue. In response, we propose a new approach to achieve high-temperature superconductivity under atmospheric pressure by implanting hydrogen into lead to create a stable few-hydrogen metal-bonded perovskite, Pb$_4$H. This approach diverges from the popular design methodology of multi-hydrogen covalent high critical temperature ($T_c$) superconductors under ultrahigh pressure. By solving the anisotropic Migdal-Eliashberg (ME) equations, we demonstrate that perovskite Pb$_4$H is a typical phonon-mediated superconductor with a $T_c$ of 46 K, which is six times higher than that of bulk Pb (7.22 K) and higher than that of MgB$_2$ (39 K). The high $T_c$ can be attributed to the strong electron-phonon coupling (EPC) strength of 2.45, which arises from hydrogen implantation in lead that induces several high-frequency optical phonon modes with a relatively large phonon linewidth resulting from H atom vibration. The metallic-bonding in perovskite Pb$_4$H not only improves the structural stability but also guarantees better ductility than the widely investigated multi-hydrogen, iron-based, and cuprate superconductors. These results suggest that there is potential for the exploration of new high-temperature superconductors under atmospheric pressure and may reignite interest in their experimental synthesis soon., Comment: 6 pages, 4 figures

In the "Beyond Moore's Law" era, with increasing edge intelligence, domain-specific computing embracing unconventional approaches will become increasingly prevalent. At the same time, adopting a variety of nanotechnologies will offer benefits in energy cost, computational speed, reduced footprint, cyber resilience, and processing power. The time is ripe for a roadmap for unconventional computing with nanotechnologies to guide future research, and this collection aims to fill that need. The authors provide a comprehensive roadmap for neuromorphic computing using electron spins, memristive devices, two-dimensional nanomaterials, nanomagnets, and various dynamical systems. They also address other paradigms such as Ising machines, Bayesian inference engines, probabilistic computing with p-bits, processing in memory, quantum memories and algorithms, computing with skyrmions and spin waves, and brain-inspired computing for incremental learning and problem-solving in severely resource-constrained environments. These approaches have advantages over traditional Boolean computing based on von Neumann architecture. As the computational requirements for artificial intelligence grow 50 times faster than Moore's Law for electronics, more unconventional approaches to computing and signal processing will appear on the horizon, and this roadmap will help identify future needs and challenges. In a very fertile field, experts in the field aim to present some of the dominant and most promising technologies for unconventional computing that will be around for some time to come. Within a holistic approach, the goal is to provide pathways for solidifying the field and guiding future impactful discoveries., Comment: 80 pages accepted in Nano Futures

The extreme requirements for high reliability and low latency in the upcoming Sixth Generation (6G) wireless networks are challenging the design of multi-hop wireless transport networks. Inspired by the advent of the virtualization concept in the wireless networks design and openness paradigm as fostered by the O-RAN Alliance, we target a revolutionary resource allocation scheme to improve the overall transmission efficiency. In this paper, we investigate the problem of multi-hop decode-and-forward (DF) relaying in the finite blocklength (FBL) regime, and propose a DMH-HARQ scheme, which maximizes the end-to-end (E2E) communication reliability in the wireless transport network. We also propose an integer dynamic programming (DP) algorithm to efficiently solve the optimal DMH-HARQ strategy. Constrained within a certain time frame to accomplish E2E transmission, our proposed approach is proven to outperform the conventional listening-based cooperative ARQ, as well as any static HARQ strategy, regarding the E2E reliability. It is applicable without dependence on special delay constraint, and is particularly competitive for long-distance transport network with many hops., Comment: Submitted to IEEE Transactions on Mobile Computing

Hidden sectors are ubiquitous in supergravity theories, in strings and in branes. Well motivated models such as the Stueckelberg hidden sector model could provide a candidate for dark matter. In such models, the hidden sector communicates with the visible sector via the exchange of a dark photon (dark $Z'$) while dark matter is constituted of Dirac fermions in the hidden sector. Using data from collider searches and precision measurements of SM processes as well as the most recent limits from dark matter direct and indirect detection experiments, we perform a comprehensive scan over a wide range of the $Z'$ mass and set exclusion bounds on the parameter space from sub-GeV to several TeV. We then discuss the discovery potential of an $\mathcal{O}$(TeV) scale $Z'$ at HL-LHC and the ability of future forward detectors to probe very weakly interacting sub-GeV $Z'$ bosons. Our analysis shows that the parameter space in which a $Z'$ can decay to hidden sector dark matter is severely constrained whereas limits become much weaker for a $Z'$ with no dark decays. The analysis also favors a self-thermalized dark sector which is necessary to satisfy the dark matter relic density., Comment: 35 pages, 15 figures. Accepted for publication in JHEP

Out-Of-Distribution (OOD) detection has received broad attention over the years, aiming to ensure the reliability and safety of deep neural networks (DNNs) in real-world scenarios by rejecting incorrect predictions. However, we notice a discrepancy between the conventional evaluation vs. the essential purpose of OOD detection. On the one hand, the conventional evaluation exclusively considers risks caused by label-space distribution shifts while ignoring the risks from input-space distribution shifts. On the other hand, the conventional evaluation reward detection methods for not rejecting the misclassified image in the validation dataset. However, the misclassified image can also cause risks and should be rejected. We appeal to rethink OOD detection from a human-centric perspective, that a proper detection method should reject the case that the deep model's prediction mismatches the human expectations and adopt the case that the deep model's prediction meets the human expectations. We propose a human-centric evaluation and conduct extensive experiments on 45 classifiers and 8 test datasets. We find that the simple baseline OOD detection method can achieve comparable and even better performance than the recently proposed methods, which means that the development in OOD detection in the past years may be overestimated. Additionally, our experiments demonstrate that model selection is non-trivial for OOD detection and should be considered as an integral of the proposed method, which differs from the claim in existing works that proposed methods are universal across different models., Comment: Preprint version. Submitted to International Journal of Computer Vision

Enabling ultra-reliable and low-latency communication services while providing massive connectivity is one of the major goals to be accomplished in future wireless communication networks. In this paper, we investigate the performance of a hybrid multi-access scheme in the finite blocklength (FBL) regime that combines the advantages of both non-orthogonal multiple access (NOMA) and time-division multiple access (TDMA) schemes. Two latency-sensitive application scenarios are studied, distinguished by whether the queuing behaviour has an influence on the transmission performance or not. In particular, for the latency-critical case with one-shot transmission, we aim at a certain physical-layer quality-of-service (QoS) performance, namely the optimization of the reliability. And for the case in which queuing behaviour plays a role, we focus on the link-layer QoS performance and provide a design that maximizes the effective capacity. For both designs, we leverage the characterizations in the FBL regime to provide the optimal framework by jointly allocating the blocklength and transmit power of each user. In particular, for the reliability-oriented design, the original problem is decomposed and the joint convexity of sub-problems is shown via a variable substitution method. For the effective-capacity-oriented design, we exploit the method of Lagrange multipliers to formulate a solvable dual problem with strong duality to the original problem. Via simulations, we validate our analytical results of convexity/concavity and show the advantage of our proposed approaches compared to other existing schemes., Comment: 32pages, 9 figures. Accepted by IEEE Transactions on Wireless Communications

Transferable adversarial attacks against Deep neural networks (DNNs) have received broad attention in recent years. An adversarial example can be crafted by a surrogate model and then attack the unknown target model successfully, which brings a severe threat to DNNs. The exact underlying reasons for the transferability are still not completely understood. Previous work mostly explores the causes from the model perspective, e.g., decision boundary, model architecture, and model capacity. adversarial attacks against Deep neural networks (DNNs) have received broad attention in recent years. An adversarial example can be crafted by a surrogate model and then attack the unknown target model successfully, which brings a severe threat to DNNs. The exact underlying reasons for the transferability are still not completely understood. Previous work mostly explores the causes from the model perspective. Here, we investigate the transferability from the data distribution perspective and hypothesize that pushing the image away from its original distribution can enhance the adversarial transferability. To be specific, moving the image out of its original distribution makes different models hardly classify the image correctly, which benefits the untargeted attack, and dragging the image into the target distribution misleads the models to classify the image as the target class, which benefits the targeted attack. Towards this end, we propose a novel method that crafts adversarial examples by manipulating the distribution of the image. We conduct comprehensive transferable attacks against multiple DNNs to demonstrate the effectiveness of the proposed method. Our method can significantly improve the transferability of the crafted attacks and achieves state-of-the-art performance in both untargeted and targeted scenarios, surpassing the previous best method by up to 40$\%$ in some cases., Comment: \copyright 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Out-of-distribution (OOD) detection is a critical task for ensuring the reliability and safety of deep neural networks in real-world scenarios. Different from most previous OOD detection methods that focus on designing OOD scores or introducing diverse outlier examples to retrain the model, we delve into the obstacle factors in OOD detection from the perspective of typicality and regard the feature's high-probability region of the deep model as the feature's typical set. We propose to rectify the feature into its typical set and calculate the OOD score with the typical features to achieve reliable uncertainty estimation. The feature rectification can be conducted as a {plug-and-play} module with various OOD scores. We evaluate the superiority of our method on both the commonly used benchmark (CIFAR) and the more challenging high-resolution benchmark with large label space (ImageNet). Notably, our approach outperforms state-of-the-art methods by up to 5.11$\%$ in the average FPR95 on the ImageNet benchmark.

We summarize the state of Beyond the Standard Model (BSM) model building in particle physics for Snowmass 2021, focusing mainly on several whitepaper contributions to BSM model building (TF08) and closely related areas., Comment: 44 pages, 1 figure, Submitted to the Proceedings of the US Community Study on the Future of Particle Physics (Snowmass 2021). Topical Group Report for TF08 - BSM Model Building

To support ultra-reliable and low-latency services for mission-critical applications, transmissions are usually carried via short blocklength codes, i.e., in the so-called finite blocklength (FBL) regime. Different from the infinite blocklength regime where transmissions are assumed to be arbitrarily reliable at the Shannon's capacity, the reliability and capacity performances of an FBL transmission are impacted by the coding blocklength. The relationship among reliability, coding rate, blocklength and channel quality has recently been characterized in the literature, considering the FBL performance model. In this paper, we follow this model, and prove the joint convexity of the FBL error probability with respect to blocklength and transmit power within a region of interest, as a key enabler for designing systems to achieve globally optimal performance levels. Moreover, we apply the joint convexity to general use cases and efficiently solve the joint optimization problem in the setting with multiple users. We also extend the applicability of the proposed approach by proving that the joint convexity still holds in fading channels, as well as in relaying networks. Via simulations, we validate our analytical results and demonstrate the advantage of leveraging the joint convexity compared to other commonly-applied approaches., Comment: 33 pages, 7 figures, to be published in IEEE Transactions on Wireless Communications

Adversarial Training (AT), which is commonly accepted as one of the most effective approaches defending against adversarial examples, can largely harm the standard performance, thus has limited usefulness on industrial-scale production and applications. Surprisingly, this phenomenon is totally opposite in Natural Language Processing (NLP) task, where AT can even benefit for generalization. We notice the merit of AT in NLP tasks could derive from the discrete and symbolic input space. For borrowing the advantage from NLP-style AT, we propose Discrete Adversarial Training (DAT). DAT leverages VQGAN to reform the image data to discrete text-like inputs, i.e. visual words. Then it minimizes the maximal risk on such discrete images with symbolic adversarial perturbations. We further give an explanation from the perspective of distribution to demonstrate the effectiveness of DAT. As a plug-and-play technique for enhancing the visual representation, DAT achieves significant improvement on multiple tasks including image classification, object detection and self-supervised learning. Especially, the model pre-trained with Masked Auto-Encoding (MAE) and fine-tuned by our DAT without extra data can get 31.40 mCE on ImageNet-C and 32.77% top-1 accuracy on Stylized-ImageNet, building the new state-of-the-art. The code will be available at https://github.com/alibaba/easyrobust., Comment: Accepted to NeurIPS 2022, https://github.com/alibaba/easyrobust

We propose a nuclear-spin-polarization protocol in a general evolution-and-measurement framework. The protocol works in a spin-star configuration, where the central spin is coupled to the surrounding bath (nuclear) spins by flip-flop interaction of equal strength and is subject to a sequence of projective measurements on its ground state. Then a nondeterministic nuclear spin polarization could be implemented by entropy reduction through measurement. The optimized measurement-interval $\tau_{\rm opt}$ is analytically obtained in the near-resonant condition, which is relevant to the nuclear spins' polarization degree of the last-round measurement, the number of nuclear spins, and the coupling strength between the central spin and nuclear spins. Hundreds and even thousands of randomly aligned nuclear spins at the thermal state could be almost fully polarized with an optimized sequence of less than $20$ unequal-time-spacing measurements. In comparison to the conventional approaches, our protocol is not sensitive to the magnetic-field intensity, and it is robust against the extra counterrotating interaction in the near-resonant situation.