Search

World models are receiving increasing attention in autonomous driving for their ability to predict potential future scenarios. In this paper, we present BEVWorld, a novel approach that tokenizes multimodal sensor inputs into a unified and compact Bird's Eye View (BEV) latent space for environment modeling. The world model consists of two parts: the multi-modal tokenizer and the latent BEV sequence diffusion model. The multi-modal tokenizer first encodes multi-modality information and the decoder is able to reconstruct the latent BEV tokens into LiDAR and image observations by ray-casting rendering in a self-supervised manner. Then the latent BEV sequence diffusion model predicts future scenarios given action tokens as conditions. Experiments demonstrate the effectiveness of BEVWorld in autonomous driving tasks, showcasing its capability in generating future scenes and benefiting downstream tasks such as perception and motion prediction. Code will be available at https://github.com/zympsyche/BevWorld., Comment: 10 pages

In the rapidly evolving field of natural language processing, dialogue systems primarily employ a single-step dialogue paradigm. Although this paradigm is efficient, it lacks the depth and fluidity of human interactions and does not appear natural. We introduce a novel \textbf{Step}-by-Step Dialogue Paradigm (Stephanie), designed to mimic the ongoing dynamic nature of human conversations. By employing a dual learning strategy and a further-split post-editing method, we generated and utilized a high-quality step-by-step dialogue dataset to fine-tune existing large language models, enabling them to perform step-by-step dialogues. We thoroughly present Stephanie. Tailored automatic and human evaluations are conducted to assess its effectiveness compared to the traditional single-step dialogue paradigm. We will release code, Stephanie datasets, and Stephanie LLMs to facilitate the future of chatbot eras.

The multi-sector intelligent surface (IS), benefiting from a smarter wave manipulation capability, has been shown to enhance channel gain and offer full-space coverage in communications. However, the benefits of multi-sector IS in wireless sensing remain unexplored. This paper introduces the application of multi-sector IS for wireless sensing/localization. Specifically, we propose a new self-sensing system, where an active source controller uses the multi-sector IS geometry to reflect/scatter the emitted signals towards the entire space, thereby achieving full-space coverage for wireless sensing. Additionally, dedicated sensors are installed aligned with the IS elements at each sector, which collect echo signals from the target and cooperate to sense the target angle. In this context, we develop a maximum likelihood estimator of the target angle for the proposed multi-sector IS self-sensing system, along with the corresponding theoretical limits defined by the Cram\'er-Rao Bound. The analysis reveals that the advantages of the multi-sector IS self-sensing system stem from two aspects: enhancing the probing power on targets (thereby improving power efficiency) and increasing the rate of target angle (thereby enhancing the transceiver's sensitivity to target angles). Finally, our analysis and simulations confirm that the multi-sector IS self-sensing system, particularly the 4-sector architecture, achieves full-space sensing capability beyond the single-sector IS configuration. Furthermore, similarly to communications, employing directive antenna patterns on each sector's IS elements and sensors significantly enhances sensing capabilities. This enhancement originates from both aspects of improved power efficiency and target angle sensitivity, with the former also being observed in communications while the latter being unique in sensing., Comment: 13 pages, 9 figures

Beyond diagonal reconfigurable intelligent surface (BD-RIS) is a new advance and generalization of the RIS technique. BD-RIS breaks through the isolation between RIS elements by creatively introducing inter-element connections, thereby enabling smarter wave manipulation and enlarging coverage. However, exploring proper channel estimation schemes suitable for BD-RIS aided communication systems still remains an open problem. In this paper, we study channel estimation and beamforming design for BD-RIS aided multi-antenna systems. We first describe the channel estimation strategy based on the least square (LS) method, derive the mean square error (MSE) of the LS estimation, and formulate the joint pilot sequence and BD-RIS design problem with unique constraints induced by BD-RIS architectures. Specifically, we propose an efficient pilot sequence and BD-RIS design which theoretically guarantees to achieve the minimum MSE. With the estimated channel, we then consider two BD-RIS scenarios and propose beamforming design algorithms. Finally, we provide simulation results to verify the effectiveness of the proposed channel estimation scheme and beamforming design algorithms. We also show that more interelement connections in BD-RIS improves the performance while increasing the training overhead for channel estimation., Comment: 14 pages, 10 figures, submitted to IEEE journal

Histo-genomic multi-modal methods have recently emerged as a powerful paradigm, demonstrating significant potential for improving cancer prognosis. However, genome sequencing, unlike histopathology imaging, is still not widely accessible in underdeveloped regions, limiting the application of these multi-modal approaches in clinical settings. To address this, we propose a novel Genome-informed Hyper-Attention Network, termed G-HANet, which is capable of effectively distilling the histo-genomic knowledge during training to elevate uni-modal whole slide image (WSI)-based inference for the first time. Compared with traditional knowledge distillation methods (i.e., teacher-student architecture) in other tasks, our end-to-end model is superior in terms of training efficiency and learning cross-modal interactions. Specifically, the network comprises the cross-modal associating branch (CAB) and hyper-attention survival branch (HSB). Through the genomic data reconstruction from WSIs, CAB effectively distills the associations between functional genotypes and morphological phenotypes and offers insights into the gene expression profiles in the feature space. Subsequently, HSB leverages the distilled histo-genomic associations as well as the generated morphology-based weights to achieve the hyper-attention modeling of the patients from both histopathology and genomic perspectives to improve cancer prognosis. Extensive experiments are conducted on five TCGA benchmarking datasets and the results demonstrate that G-HANet significantly outperforms the state-of-the-art WSI-based methods and achieves competitive performance with genome-based and multi-modal methods. G-HANet is expected to be explored as a useful tool by the research community to address the current bottleneck of insufficient histo-genomic data pairing in the context of cancer prognosis and precision oncology.

While Large Language Models (LLMs) have demonstrated exceptional multitasking abilities, fine-tuning these models on downstream, domain-specific datasets is often necessary to yield superior performance on test sets compared to their counterparts without fine-tuning. However, the comprehensive effects of fine-tuning on the LLMs' generalization ability are not fully understood. This paper delves into the differences between original, unmodified LLMs and their fine-tuned variants. Our primary investigation centers on whether fine-tuning affects the generalization ability intrinsic to LLMs. To elaborate on this, we conduct extensive experiments across five distinct language tasks on various datasets. Our main findings reveal that models fine-tuned on generation and classification tasks exhibit dissimilar behaviors in generalizing to different domains and tasks. Intriguingly, we observe that integrating the in-context learning strategy during fine-tuning on generation tasks can enhance the model's generalization ability. Through this systematic investigation, we aim to contribute valuable insights into the evolving landscape of fine-tuning practices for LLMs., Comment: NAACL 2024

We are currently in an era of fierce competition among various large language models (LLMs) continuously pushing the boundaries of benchmark performance. However, genuinely assessing the capabilities of these LLMs has become a challenging and critical issue due to potential data contamination, and it wastes dozens of time and effort for researchers and engineers to download and try those contaminated models. To save our precious time, we propose a novel and useful method, Clean-Eval, which mitigates the issue of data contamination and evaluates the LLMs in a cleaner manner. Clean-Eval employs an LLM to paraphrase and back-translate the contaminated data into a candidate set, generating expressions with the same meaning but in different surface forms. A semantic detector is then used to filter the generated low-quality samples to narrow down this candidate set. The best candidate is finally selected from this set based on the BLEURT score. According to human assessment, this best candidate is semantically similar to the original contamination data but expressed differently. All candidates can form a new benchmark to evaluate the model. Our experiments illustrate that Clean-Eval substantially restores the actual evaluation results on contaminated LLMs under both few-shot learning and fine-tuning scenarios., Comment: NAACL2024(findings)

The wireless domain is witnessing a flourishing of integrated systems, e.g. (a) integrated sensing and communications, and (b) simultaneous wireless information and power transfer, due to their potential to use resources (spectrum, power) judiciously. Inspired by this trend, we investigate integrated sensing, communications and powering (ISCAP), through the design of a wideband OFDM signal to power a sensor while simultaneously performing target-sensing and communication. To characterize the ISCAP performance region, we assume symbols with non-zero mean asymmetric Gaussian distribution (i.e., the input distribution), and optimize its mean and variance at each subcarrier to maximize the harvested power, subject to constraints on the achievable rate (communications) and the average side-to-peak-lobe difference (sensing). The resulting input distribution, through simulations, achieves a larger performance region than that of (i) a symmetric complex Gaussian input distribution with identical mean and variance for the real and imaginary parts, (ii) a zero-mean symmetric complex Gaussian input distribution, and (iii) the superposed power-splitting communication and sensing signal (the coexisting solution). In particular, the optimized input distribution balances the three functions by exhibiting the following features: (a) symbols in subcarriers with strong communication channels have high variance to satisfy the rate constraint, while the other symbols are dominated by the mean, forming a relatively uniform sum of mean and variance across subcarriers for sensing; (b) with looser communication and sensing constraints, large absolute means appear on subcarriers with stronger powering channels for higher harvested power. As a final note, the results highlight the great potential of the co-designed ISCAP system for further efficiency enhancement., Comment: 13 pages, 5 figures

This paper proposes a neural radiance field (NeRF) approach for novel view synthesis of dynamic scenes using forward warping. Existing methods often adopt a static NeRF to represent the canonical space, and render dynamic images at other time steps by mapping the sampled 3D points back to the canonical space with the learned backward flow field. However, this backward flow field is non-smooth and discontinuous, which is difficult to be fitted by commonly used smooth motion models. To address this problem, we propose to estimate the forward flow field and directly warp the canonical radiance field to other time steps. Such forward flow field is smooth and continuous within the object region, which benefits the motion model learning. To achieve this goal, we represent the canonical radiance field with voxel grids to enable efficient forward warping, and propose a differentiable warping process, including an average splatting operation and an inpaint network, to resolve the many-to-one and one-to-many mapping issues. Thorough experiments show that our method outperforms existing methods in both novel view rendering and motion modeling, demonstrating the effectiveness of our forward flow motion modeling. Project page: https://npucvr.github.io/ForwardFlowDNeRF, Comment: Accepted by ICCV2023 as oral. Project page: https://npucvr.github.io/ForwardFlowDNeRF

We study channel estimation for a beyond diagonal reconfigurable intelligent surface (BD-RIS) aided multiple input single output system. We first describe the channel estimation strategy based on the least square (LS) method, derive the mean square error (MSE) of the LS estimator, and formulate the BD-RIS design problem that minimizes the estimation MSE with unique constraints induced by group-connected architectures of BD-RIS. Then, we propose an efficient BD-RIS design which theoretically guarantees to achieve the MSE lower bound. Finally, we provide simulation results to verify the effectiveness of the proposed channel estimation scheme., Comment: 5 pages, 2 figures, accepted by CAMSAP 2023

Orthogonal frequency division multiplexing (OFDM) has been widely adopted in dual-function radar-communication (DFRC) systems, where radar and communications are performed simultaneously with a common signal. However, with random communication symbols (CS) in DFRC, the transmit signal has a random ambiguity function that affects the radar's range-velocity estimation performance, whose influence is remained uncovered. Hence, this paper focuses on minimizing the outlier probability (OP) -- the probability of incorrectly estimating a target's range-velocity bin -- in OFDM DFRC w.r.t the CS probability distribution (i.e., the \emph{input distribution}). Conditioned on the CSs, the OP only depends on the CS magnitudes. Hence, we consider the following two schemes for the above optimization: CSs with (1) constant magnitude (phase shift keying input), and (2) random magnitude (Gaussian input). For (1), the problem reduces to the familiar power allocation design across OFDM's subcarriers and symbols, with uniform power allocation across subcarriers and a \emph{windowed} power allocation across symbols being near-optimal. For (2), the mean and variance of the Gaussian distribution at each subcarrier is optimized, with an additional communication constraint to avoid the zero-variance solution where no CSs are carried. We observe that subcarriers with strong communication channels feature strong variance (i.e., favour communications) while the others are characterized by a strong mean (favouring radar). However, the overall power allocation (i.e., the sum of mean and variance) across the OFDM subcarriers and symbols is similar to (1). Simulations show that CSs with random magnitudes degrade the sensing performance, but can be compensated significantly with the proposed input distribution optimization., Comment: 13 pages, 8 figures

Waveform optimization has shown its great potential to boost the performance of far-field wireless power transfer (WPT). Current research has optimized transmit waveform, adaptive to channel state information (CSI), to maximize the harvested power in WPT while accounting for energy harvester (EH)'s non-linearity. However, the existing transmit waveform design disregards the non-linear high power amplifiers (HPA) at the transmitter. Driven by this, this paper optimizes the multi-carrier waveform at the input of HPA to maximize the harvested DC power considering both HPA's and EH's non-linearities. Two optimization models are formulated based on whether the frequencies of the multi-carrier waveform are concentrated within the transmit pass band or not. Analysis and simulations show that, while EH's non-linearity boosts the power harvesting performance, HPA's non-linearity degrades the harvested power. Hence, the optimal waveform shifts from multi-carrier that exploits EH's non-linearity to single-carrier that reduces HPA's detrimental non-linear distortion as the operational regime of WPT becomes more sensitive to HPA's non-linearity and less sensitive to EH's non-linearity (and inversely). Simultaneously, operating towards HPA's non-linear regime by increasing the input signal power benefits the harvested power since HPA's DC power supply is better exploited, whereas the end-to-end power transfer efficiency (PTE) might decrease because of the increasing non-linear degradation. Throughout the simulations, the proposed waveforms show significant gain over those not accounting for HPA's non-linearity, especially in frequency-flat channels. We also compare the two proposed waveforms and show that the severity of HPA's non-linearity dictates which of the two proposed waveforms is more beneficial., Comment: 13 pages, 11 figures

Waveform optimization has recently been shown to be a key technique to boost the efficiency and range of far-field wireless power transfer (WPT). Current research has optimized transmit waveform adaptive to channel state information (CSI) and accounting for energy harvester (EH)'s non-linearity but under the assumption of linear high power amplifiers (HPA) at the transmitter. This paper proposes a channel-adaptive waveform design strategy that optimizes the transmitter's input waveform considering both HPA and EH non-linearities. Simulations demonstrate that HPA's non-linearity degrades the energy harvesting efficiency of WPT significantly, while the performance loss can be compensated by using the proposed optimal input waveform., Comment: Submitted to ICASSP 2022