Search

Augmenting Large Language Models (LLMs) with retrieved external knowledge has proven effective for improving the factual accuracy of generated responses. Despite their success, retrieval-augmented LLMs still face the distractibility issue, where the generated responses are negatively influenced by noise from both external and internal knowledge sources. In this paper, we introduce a novel, training-free decoding method guided by entropy considerations to mitigate this issue. Our approach utilizes entropy-based document-parallel ensemble decoding to prioritize low-entropy distributions from retrieved documents, thereby enhancing the extraction of relevant information of context. Additionally, it incorporates a contrastive decoding mechanism that contrasts the obtained low-entropy ensemble distribution with the high-entropy distribution derived from the model's internal knowledge across layers, which ensures a greater emphasis on reliable external information. Extensive experiments on open-domain question answering datasets demonstrate the superiority of our method.

This paper studies the rate-distortion-perception (RDP) tradeoff for a Gaussian vector source coding problem where the goal is to compress the multi-component source subject to distortion and perception constraints. The purpose of imposing a perception constraint is to ensure visually pleasing reconstructions. This paper studies this RDP setting with either the Kullback-Leibler (KL) divergence or Wasserstein-2 metric as the perception loss function, and shows that for Gaussian vector sources, jointly Gaussian reconstructions are optimal. We further demonstrate that the optimal tradeoff can be expressed as an optimization problem, which can be explicitly solved. An interesting property of the optimal solution is as follows. Without the perception constraint, the traditional reverse water-filling solution for characterizing the rate-distortion (RD) tradeoff of a Gaussian vector source states that the optimal rate allocated to each component depends on a constant, called the water-level. If the variance of a specific component is below the water-level, it is assigned a {zero} compression rate. However, with active distortion and perception constraints, we show that the optimal rates allocated to the different components are always {positive}. Moreover, the water-levels that determine the optimal rate allocation for different components are unequal. We further treat the special case of perceptually perfect reconstruction and study its RDP function in the high-distortion and low-distortion regimes to obtain insight to the structure of the optimal solution.

While fine-tuned large language models (LLMs) excel in generating grammatically valid SQL in Text-to-SQL parsing, they often struggle to ensure semantic accuracy in queries, leading to user confusion and diminished system usability. To tackle this challenge, we introduce SQLFixAgent, an innovative multi-agent collaborative framework designed for detecting and repairing erroneous SQL. Our framework comprises a core agent, SQLRefiner, alongside two auxiliary agents: SQLReviewer and QueryCrafter. The SQLReviewer agent employs the rubber duck debugging method to identify potential semantic mismatches between SQL statement and user query. If the error is detected, the QueryCrafter agent generates multiple SQL statements as candidate repairs using a fine-tuned SQLTool. Subsequently, leveraging similar repair retrieval and failure memory reflexion, the SQLRefiner agent selects the most fitting SQL statement from the candidates as the final repair. We evaluated our proposed framework on five Text-to-SQL benchmarks. The experimental results show that our method consistently enhances the performance of the baseline model, specifically achieving an execution accuracy improvement of over 3\% on the Bird benchmark. Our framework also has a higher token efficiency compared to other advanced methods, making it more competitive.

User studies are crucial for meeting user needs. In user studies, real experimental scenarios and participants are constructed and recruited. However, emerging and unfamiliar studies face limitations, including safety concerns and iterative efficiency. To address these challenges, this study utilizes a large language model (LLM) to create generative AI virtual scenarios for user experience. By recruiting real users to evaluate this experience, we can collect feedback that enables rapid iteration in the early design phase. The air taxi is particularly representative of these challenges and has been chosen as the case study for this research. The key contribution was designing a virtual ATJ using OpenAI's GPT-4 model and AI image and video generators. Based on the LLM-generated scripts, key visuals were created for the air taxi, and the ATJ was evaluated by 72 participants. Furthermore, the LLM demonstrated the ability to identify and suggest environments that significantly improve participants' attitudes toward air taxis. Education level and gender significantly influenced participants' attitudes and their satisfaction with the ATJ. Our study confirms the capability of generative AI to support user studies, providing a feasible approach and valuable insights for designing air taxi user experiences in the early design phase., Comment: 39 pages, 6 main figures, 10 appendix figures

Data generation-based zero-shot learning, although effective in training Small Task-specific Models (STMs) via synthetic datasets generated by Pre-trained Language Models (PLMs), is often limited by the low quality of such synthetic datasets. Previous solutions have primarily focused on single PLM settings, where synthetic datasets are typically restricted to specific sub-spaces and often deviate from real-world distributions, leading to severe distribution bias. To mitigate such bias, we propose FuseGen, a novel data generation-based zero-shot learning framework that introduces a new criteria for subset selection from synthetic datasets via utilizing multiple PLMs and trained STMs. The chosen subset provides in-context feedback to each PLM, enhancing dataset quality through iterative data generation. Trained STMs are then used for sample re-weighting as well, further improving data quality. Extensive experiments across diverse tasks demonstrate that FuseGen substantially outperforms existing methods, highly effective in boosting STM performance in a PLM-agnostic way. Code is provided in https://github.com/LindaLydia/FuseGen., Comment: 17 pages, 8 figures, 12 tabels

Large Language Models (LLMs) have shown great potential in the biomedical domain with the advancement of retrieval-augmented generation (RAG). However, existing retrieval-augmented approaches face challenges in addressing diverse queries and documents, particularly for medical knowledge queries, resulting in sub-optimal performance. To address these limitations, we propose a novel plug-and-play LLM-based retrieval method called Self-Rewarding Tree Search (SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm. By combining the reasoning capabilities of LLMs with the effectiveness of tree search, SeRTS boosts the zero-shot performance of retrieving high-quality and informative results for RAG. We further enhance retrieval performance by fine-tuning LLMs with Proximal Policy Optimization (PPO) objectives using the trajectories collected by SeRTS as feedback. Controlled experiments using the BioASQ-QA dataset with GPT-3.5-Turbo and LLama2-7b demonstrate that our method significantly improves the performance of the BM25 retriever and surpasses the strong baseline of self-reflection in both efficiency and scalability. Moreover, SeRTS generates higher-quality feedback for PPO training than self-reflection. Our proposed method effectively adapts LLMs to document retrieval tasks, enhancing their ability to retrieve highly relevant documents for RAG in the context of medical knowledge queries. This work presents a significant step forward in leveraging LLMs for accurate and comprehensive biomedical question answering.

Random fluctuations can lead to cooperative effects in complex systems. We here report the experimental observation of noise-induced quantum synchronization in a chain of superconducting transmon qubits with nearest-neighbor interactions. The application of Gaussian white noise to a single site leads to synchronous oscillations in the entire chain. We show that the two synchronized end qubits are entangled, with nonzero concurrence, and that they belong to a class of generalized Bell states known as maximally entangled mixed states, whose entanglement cannot be increased by any global unitary. We further demonstrate the stability against frequency detuning of both synchronization and entanglement by determining the corresponding generalized Arnold tongue diagrams. Our results highlight the constructive influence of noise in a quantum many-body system and uncover the potential role of synchronization for mixed-state quantum information science.

Existing image super-resolution (SR) techniques often fail to generalize effectively in complex real-world settings due to the significant divergence between training data and practical scenarios. To address this challenge, previous efforts have either manually simulated intricate physical-based degradations or utilized learning-based techniques, yet these approaches remain inadequate for producing large-scale, realistic, and diverse data simultaneously. In this paper, we introduce a novel Realistic Decoupled Data Generator (RealDGen), an unsupervised learning data generation framework designed for real-world super-resolution. We meticulously develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model to create realistic low-resolution images from unpaired real LR and HR images. Extensive experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations, significantly advancing the performance of popular SR models on various real-world benchmarks.

The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems. However, the scarcity of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). Building on the achievements of the previous MIPI Workshops held at ECCV 2022 and CVPR 2023, we introduce our third MIPI challenge including three tracks focusing on novel image sensors and imaging algorithms. In this paper, we summarize and review the Few-shot RAW Image Denoising track on MIPI 2024. In total, 165 participants were successfully registered, and 7 teams submitted results in the final testing phase. The developed solutions in this challenge achieved state-of-the-art erformance on Few-shot RAW Image Denoising. More details of this challenge and the link to the dataset can be found at https://mipichallenge.org/MIPI2024., Comment: CVPR 2024 Mobile Intelligent Photography and Imaging (MIPI) Workshop--Few-shot RAWImage Denoising Challenge Report. Website: https://mipi-challenge.org/MIPI2024/

In this paper, we consider a class of difference-of-convex (DC) optimization problems, where the global Lipschitz gradient continuity assumption on the smooth part of the objective function is not required. Such problems are prevalent in many contemporary applications such as compressed sensing, statistical regression, and machine learning, and can be solved by a general Bregman proximal DC algorithm (BPDCA). However, the existing BPDCA is developed based on the stringent requirement that the involved subproblems must be solved exactly, which is often impractical and limits the applicability of the BPDCA. To facilitate the practical implementations and wider applications of the BPDCA, we develop an inexact Bregman proximal difference-of-convex algorithm (iBPDCA) by incorporating two types of relative-type stopping criteria for solving the subproblems. The proposed inexact framework has considerable flexibility to encompass many existing exact and inexact methods, and can accommodate different types of errors that may occur when solving the subproblem. This enables the potential application of our inexact framework across different DC decompositions to facilitate the design of a more efficient DCA scheme in practice. The global subsequential convergence and the global sequential convergence of our iBPDCA are established under suitable conditions including the Kurdyka-{\L}ojasiewicz property. Some numerical experiments on the $\ell_{1-2}$ regularized least squares problem and the constrained $\ell_{1-2}$ sparse optimization problem are conducted to show the superior performance of our iBPDCA in comparison to existing algorithms. These results also empirically verify the necessity of developing different types of stopping criteria to facilitate the efficient computation of the subproblem in each iteration of our iBPDCA.

Buildings produce one-third of carbon emissions globally, however, data absence regarding global floorspace poses challenges in advancing building carbon neutrality. We compile the measured building stocks for 14 major economies and apply our global building stock model, GLOBUS, to evaluate future trends in stock turnover. Based on a scenario not considering renovation, by 2070 the building stock in developed economies will be ~1.4 times that of 2020 (100 billion m2); in developing economies it is expected to be 2.2 times that of 2020 (313 billion m2). Based on a techno-economic potential scenario, however, stocks in developed economies will decline to approximately 0.8 times the 2020 level, while stocks in developing economies will increase to nearly twice the 2020 level due to their fewer buildings currently. Overall, GLOBUS provides a way of calculating the global building stock, helping scientists, engineers, and policymakers conduct a range of investigation across various future scenarios., Comment: 25 pages, 4 figures

Recent attacks on federated learning (FL) can introduce malicious model updates that circumvent widely adopted Euclidean distance-based detection methods. This paper proposes a novel defense strategy, referred to as LayerCAM-AE, designed to counteract model poisoning in federated learning. The LayerCAM-AE puts forth a new Layer Class Activation Mapping (LayerCAM) integrated with an autoencoder (AE), significantly enhancing detection capabilities. Specifically, LayerCAM-AE generates a heat map for each local model update, which is then transformed into a more compact visual format. The autoencoder is designed to process the LayerCAM heat maps from the local model updates, improving their distinctiveness and thereby increasing the accuracy in spotting anomalous maps and malicious local models. To address the risk of misclassifications with LayerCAM-AE, a voting algorithm is developed, where a local model update is flagged as malicious if its heat maps are consistently suspicious over several rounds of communication. Extensive tests of LayerCAM-AE on the SVHN and CIFAR-100 datasets are performed under both Independent and Identically Distributed (IID) and non-IID settings in comparison with existing ResNet-50 and REGNETY-800MF defense models. Experimental results show that LayerCAM-AE increases detection rates (Recall: 1.0, Precision: 1.0, FPR: 0.0, Accuracy: 1.0, F1 score: 1.0, AUC: 1.0) and test accuracy in FL, surpassing the performance of both the ResNet-50 and REGNETY-800MF. Our code is available at: https://github.com/jjzgeeks/LayerCAM-AE

Previous work has demonstrated that, in the Variance Preserving (VP) scenario, the nascent Directly Denoising Diffusion Models (DDDM) can generate high-quality images in one step while achieving even better performance in multistep sampling. However, the Pseudo-LPIPS loss used in DDDM leads to concerns about the bias in assessment. Here, we propose a unified DDDM (uDDDM) framework that generates images in one-step/multiple steps for both Variance Preserving (VP) and Variance Exploding (VE) cases. We provide theoretical proofs of the existence and uniqueness of the model's solution paths, as well as the non-intersecting property of the sampling paths. Additionally, we propose an adaptive Pseudo-Huber loss function to balance the convergence to the true solution and the stability of convergence process.Through a comprehensive evaluation, we demonstrate that uDDDMs achieve FID scores comparable to the best-performing methods available for CIFAR-10 in both VP and VE. Specifically, uDDDM achieves one-step generation on CIFAR10 with FID of 2.63 and 2.53 for VE and VP respectively. By extending the sampling to 1000 steps, we further reduce FID score to 1.71 and 1.65 for VE and VP respectively, setting state-of-the-art performance in both cases.

Monitoring sleep states is essential for evaluating sleep quality and diagnosing sleep disorders. Traditional manual staging is time-consuming and prone to subjective bias, often resulting in inconsistent outcomes. Here, we developed an automated model for sleep staging and disorder classification to enhance diagnostic accuracy and efficiency. Considering the characteristics of polysomnography (PSG) multi-lead sleep monitoring, we designed a multimodal sleep state classification model, MSSC-BiMamba, that combines an Efficient Channel Attention (ECA) mechanism with a Bidirectional State Space Model (BSSM). The ECA module allows for weighting data from different sensor channels, thereby amplifying the influence of diverse sensor inputs. Additionally, the implementation of bidirectional Mamba (BiMamba) enables the model to effectively capture the multidimensional features and long-range dependencies of PSG data. The developed model demonstrated impressive performance on sleep stage classification tasks on both the ISRUC-S3 and ISRUC-S1 datasets, respectively containing data with healthy and unhealthy sleep patterns. Also, the model exhibited a high accuracy for sleep health prediction when evaluated on a combined dataset consisting of ISRUC and Sleep-EDF. Our model, which can effectively handle diverse sleep conditions, is the first to apply BiMamba to sleep staging with multimodal PSG data, showing substantial gains in computational and memory efficiency over traditional Transformer-style models. This method enhances sleep health management by making monitoring more accessible and extending advanced healthcare through innovative technology., Comment: 10 pages

Instruction following is crucial in contemporary LLM. However, when extended to multimodal setting, it often suffers from misalignment between specific textual instruction and targeted local region of an image. To achieve more accurate and nuanced multimodal instruction following, we introduce Instruction-guided Visual Masking (IVM), a new versatile visual grounding model that is compatible with diverse multimodal models, such as LMM and robot model. By constructing visual masks for instruction-irrelevant regions, IVM-enhanced multimodal models can effectively focus on task-relevant image regions to better align with complex instructions. Specifically, we design a visual masking data generation pipeline and create an IVM-Mix-1M dataset with 1 million image-instruction pairs. We further introduce a new learning technique, Discriminator Weighted Supervised Learning (DWSL) for preferential IVM training that prioritizes high-quality data samples. Experimental results on generic multimodal tasks such as VQA and embodied robotic control demonstrate the versatility of IVM, which as a plug-and-play tool, significantly boosts the performance of diverse multimodal models, yielding new state-of-the-art results across challenging multimodal benchmarks. Code is available at https://github.com/2toinf/IVM., Comment: preprint, 21 pages

Post-training Sparsity (PTS) is a recently emerged avenue that chases efficient network sparsity with limited data in need. Existing PTS methods, however, undergo significant performance degradation compared with traditional methods that retrain the sparse networks via the whole dataset, especially at high sparsity ratios. In this paper, we attempt to reconcile this disparity by transposing three cardinal factors that profoundly alter the performance of conventional sparsity into the context of PTS. Our endeavors particularly comprise (1) A base-decayed sparsity objective that promotes efficient knowledge transferring from dense network to the sparse counterpart. (2) A reducing-regrowing search algorithm designed to ascertain the optimal sparsity distribution while circumventing overfitting to the small calibration set in PTS. (3) The employment of dynamic sparse training predicated on the preceding aspects, aimed at comprehensively optimizing the sparsity structure while ensuring training stability. Our proposed framework, termed UniPTS, is validated to be much superior to existing PTS methods across extensive benchmarks. As an illustration, it amplifies the performance of POT, a recently proposed recipe, from 3.9% to 68.6% when pruning ResNet-50 at 90% sparsity ratio on ImageNet. We release the code of our paper at https://github.com/xjjxmu/UniPTS., Comment: Accepted by CVPR2024

Extra large-scale multiple-input multiple-output (XL-MIMO) is a key technology for future wireless communication systems. This paper considers the effects of visibility region (VR) at the base station (BS) in a non-stationary multi-user XL-MIMO scenario, where only partial antennas can receive users' signal. In time division duplexing (TDD) mode, we first estimate the VR at the BS by detecting the energy of the received signal during uplink training phase. The probabilities of two detection errors are derived and the uplink channel on the detected VR is estimated. In downlink data transmission, to avoid cumbersome Monte-Carlo trials, we derive a deterministic approximate expression for ergodic {average energy efficiency (EE)} with the regularized zero-forcing (RZF) precoding. In frequency division duplexing (FDD) mode, the VR is estimated in uplink training and then the channel information of detected VR is acquired from the feedback channel. In downlink data transmission, the approximation of ergodic average {EE} is also derived with the RZF precoding. Invoking approximate results, we propose an alternate optimization algorithm to design the detection threshold and the pilot length in both TDD and FDD modes. The numerical results reveal the impacts of VR estimation error on ergodic average {EE} and demonstrate the effectiveness of our proposed algorithm., Comment: 13 pages, 11 figures

Acoustic waves in fluid with spin-0 nature have been long believed not to support spin Hall effect and strong orbital Hall effect that enables experimental observation. Here we report the first theoretical explication and experimental demonstration of giant acoustic geometric spin and orbital Hall effect characterized by a large transverse shift. We reveal that this effect occurs when a vortex beam is observed from a tilted reference frame free of wave-interface interactions or gradient-index media needed for observing conventional ones, and can be amplified by simply binding the beam tightly. Thanks to this mechanism, large transverse shifts proportional to angular momentum are observed in a compact system. Our work provides deeper insights into the physics of angular momentum of classic waves.

In this paper, we present the Directly Denoising Diffusion Model (DDDM): a simple and generic approach for generating realistic images with few-step sampling, while multistep sampling is still preserved for better performance. DDDMs require no delicately designed samplers nor distillation on pre-trained distillation models. DDDMs train the diffusion model conditioned on an estimated target that was generated from previous training iterations of its own. To generate images, samples generated from the previous time step are also taken into consideration, guiding the generation process iteratively. We further propose Pseudo-LPIPS, a novel metric loss that is more robust to various values of hyperparameter. Despite its simplicity, the proposed approach can achieve strong performance in benchmark datasets. Our model achieves FID scores of 2.57 and 2.33 on CIFAR-10 in one-step and two-step sampling respectively, surpassing those obtained from GANs and distillation-based models. By extending the sampling to 1000 steps, we further reduce FID score to 1.79, aligning with state-of-the-art methods in the literature. For ImageNet 64x64, our approach stands as a competitive contender against leading models.

Thrombotic and chronic occlusions of large blood vessels are a major cause of mortality and morbidity, and so there is a need for improved treatments in many clinical circumstances. Endovascular ultrasound approaches have been shown to hold considerable potential to treat large vessel thrombotic occlusions. Here, we report the development of a novel forward-looking therapeutic ultrasound catheter approach. The design concept centers on the use of a radially polarized hollow cylindrical transducer situated at the distal tip. This approach enables a compact configuration where the central lumen can accommodate a guidewire during navigation as well as provide a route to release cavitation seeds and therapeutic agents adjacent to the site of occlusion. PZT-5H transducers with outer/inner diameters of 1.35/0.73 mm were evaluated using simulations and experiments for their capacity to project forward-looking ultrasound. A length of 2.5 mm operating in the 3rd harmonic of the length mode resonance (1.85 MHz) was selected. Catheters (1.55 mm outer diameter) were designed and fabricated with a liner, outer jacket, and braiding, where the transducers were incorporated with air-backing and a front-face matching layer. Forward-looking pressures of 2.5 MPa (peak negative) at 0.5 mm were achieved. Proof of principle vessel phantom experiments were performed demonstrating the ability to eject microbubbles in proximity to an occlusion and stimulate inertial cavitation. This approach holds potential for treating thrombotic and chronic total occlusions., Comment: 14 pages, 12 figures, 5 supplementary figures. This work will be submitted to the IEEE for possible publication. Copyright may be transferred without notice

Thermal quantities, including the the entropy density and gluon spectrum, of quark matter within a box that is finite in the longitudinal direction are calculated using a bag model. Under the assumption of entropy conservation, the corresponding gluon dissociation rate of J/psi is studied. It reaches a maximum at a certain longitudinal size L_m, below which the suppression is weak even if the temperature becomes higher than that without the finite size effect, and above which the dissociation rate approaches to the thermodynamic limit gradually with increasing longitudinal size of the fireball., Comment: 8 pages, 4 figures, 2 tables

Advancements in the fabrication of superconducting quantum devices have highlighted tantalum as a promising material, owing to its low surface oxidation microwave loss at low temperatures. However, tantalum films exhibit significantly larger kinetic inductances compared to materials such as aluminum or niobium. Given the inevitable variations in film thickness, this increased kinetic inductance leads to considerable, uncontrolled frequency variances and shifts in components like superconducting coplanar waveguide (SCPW) resonators. Achieving high precision in resonator frequencies is crucial, particularly when multiple resonators share a common Purcell filter with limited bandwidth in superconducting quantum information processors. Here, we tackle this challenge from both fabrication and design perspectives, achieving a reduction in resonator frequency fluctuation by a factor of more than 100. Concurrently, the internal quality factor of the SCPW resonator remains at high level. Our findings open up new avenues for the enhanced utilization of tantalum in large-scale superconducting chips.

Advanced generative model (e.g., diffusion model) derived from simplified continuity assumptions of data distribution, though showing promising progress, has been difficult to apply directly to geometry generation applications due to the multi-modality and noise-sensitive nature of molecule geometry. This work introduces Geometric Bayesian Flow Networks (GeoBFN), which naturally fits molecule geometry by modeling diverse modalities in the differentiable parameter space of distributions. GeoBFN maintains the SE-(3) invariant density modeling property by incorporating equivariant inter-dependency modeling on parameters of distributions and unifying the probabilistic modeling of different modalities. Through optimized training and sampling techniques, we demonstrate that GeoBFN achieves state-of-the-art performance on multiple 3D molecule generation benchmarks in terms of generation quality (90.87% molecule stability in QM9 and 85.6% atom stability in GEOM-DRUG. GeoBFN can also conduct sampling with any number of steps to reach an optimal trade-off between efficiency and quality (e.g., 20-times speedup without sacrificing performance)., Comment: ICLR 2024

Relational triple extraction is crucial work for the automatic construction of knowledge graphs. Existing methods only construct shallow representations from a token or token pair-level. However, previous works ignore local spatial dependencies of relational triples, resulting in a weakness of entity pair boundary detection. To tackle this problem, we propose a novel Region-based Table Filling method (RTF). We devise a novel region-based tagging scheme and bi-directional decoding strategy, which regard each relational triple as a region on the relation-specific table, and identifies triples by determining two endpoints of each region. We also introduce convolution to construct region-level table representations from a spatial perspective which makes triples easier to be captured. In addition, we share partial tagging scores among different relations to improve learning efficiency of relation classifier. Experimental results show that our method achieves state-of-the-art with better generalization capability on three variants of two widely used benchmark datasets., Comment: Rejected by EMNLP 2023

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that enables distributed clients to collaboratively learn a shared global model while keeping the data on the original device, thereby preserving privacy. A central challenge in FL is the effective aggregation of local model weights from disparate and potentially unbalanced participating clients. Existing methods often treat each client indiscriminately, applying a single proportion to the entire local model. However, it is empirically advantageous for each weight to be assigned a specific proportion. This paper introduces an innovative Element-Wise Weights Aggregation Method for Federated Learning (EWWA-FL) aimed at optimizing learning performance and accelerating convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local weights to the global model at the level of individual elements, thereby allowing each participating client to make element-wise contributions to the learning process. By taking into account the unique dataset characteristics of each client, EWWA-FL enhances the robustness of the global model to different datasets while also achieving rapid convergence. The method is flexible enough to employ various weighting strategies. Through comprehensive experiments, we demonstrate the advanced capabilities of EWWA-FL, showing significant improvements in both accuracy and convergence speed across a range of backbones and benchmarks., Comment: 2023 IEEE International Conference on Data Mining Workshops (ICDMW)

This paper puts forth a new training data-untethered model poisoning (MP) attack on federated learning (FL). The new MP attack extends an adversarial variational graph autoencoder (VGAE) to create malicious local models based solely on the benign local models overheard without any access to the training data of FL. Such an advancement leads to the VGAE-MP attack that is not only efficacious but also remains elusive to detection. VGAE-MP attack extracts graph structural correlations among the benign local models and the training data features, adversarially regenerates the graph structure, and generates malicious local models using the adversarial graph structure and benign models' features. Moreover, a new attacking algorithm is presented to train the malicious local models using VGAE and sub-gradient descent, while enabling an optimal selection of the benign local models for training the VGAE. Experiments demonstrate a gradual drop in FL accuracy under the proposed VGAE-MP attack and the ineffectiveness of existing defense mechanisms in detecting the attack, posing a severe threat to FL., Comment: 12 pages, 8 figures, 2 tables

Wireless federated learning (WFL) suffers from heterogeneity prevailing in the data distributions, computing powers, and channel conditions of participating devices. This paper presents a new Federated Learning with Adjusted leaRning ratE (FLARE) framework to mitigate the impact of the heterogeneity. The key idea is to allow the participating devices to adjust their individual learning rates and local training iterations, adapting to their instantaneous computing powers. The convergence upper bound of FLARE is established rigorously under a general setting with non-convex models in the presence of non-i.i.d. datasets and imbalanced computing powers. By minimizing the upper bound, we further optimize the scheduling of FLARE to exploit the channel heterogeneity. A nested problem structure is revealed to facilitate iteratively allocating the bandwidth with binary search and selecting devices with a new greedy method. A linear problem structure is also identified and a low-complexity linear programming scheduling policy is designed when training models have large Lipschitz constants. Experiments demonstrate that FLARE consistently outperforms the baselines in test accuracy, and converges much faster with the proposed scheduling policy.

The role-play ability of Large Language Models (LLMs) has emerged as a popular research direction. However, existing studies focus on imitating well-known public figures or fictional characters, overlooking the potential for simulating ordinary individuals. Such an oversight limits the potential for advancements in digital human clones and non-player characters in video games. To bridge this gap, we introduce ECHO, an evaluative framework inspired by the Turing test. This framework engages the acquaintances of the target individuals to distinguish between human and machine-generated responses. Notably, our framework focuses on emulating average individuals rather than historical or fictional figures, presenting a unique advantage to apply the Turing Test. We evaluated three role-playing LLMs using ECHO, with GPT-3.5 and GPT-4 serving as foundational models, alongside the online application GPTs from OpenAI. Our results demonstrate that GPT-4 more effectively deceives human evaluators, and GPTs achieves a leading success rate of 48.3%. Furthermore, we investigated whether LLMs could discern between human-generated and machine-generated texts. While GPT-4 can identify differences, it could not determine which texts were human-produced. Our code and results of reproducing the role-playing LLMs are made publicly available via https://github.com/CUHK-ARISE/ECHO., Comment: 9 pages

The next generation of very long baseline interferometry (VLBI) is stepping into the era of microarcsecond ($\mu$as) astronomy, and pushing astronomy, especially astrometry, to new heights. VLBI with the Square Kilometre Array (SKA), SKA-VLBI, will increase current sensitivity by an order of magnitude, and reach astrometric precision routinely below 10 $\mu$as, even challenging 1 $\mu$as. This advancement allows precise parallax and proper motion measurements of various celestial objects. Such improvements can be used to study objects (including isolated objects, and binary or multiple systems) in different stellar stages (such as star formation, main-sequence stars, asymptotic giant branch stars, pulsars, black holes, white dwarfs, etc.), unveil the structure and evolution of complex systems (such as the Milky Way), benchmark the international celestial reference frame, and reveal cosmic expansion. Furthermore, the theory of general relativity can also be tested with SKA-VLBI using precise measurements of light deflection under the gravitational fields of different solar system objects and the perihelion precession of solar system objects., Comment: 41 pages, 12 figures, 4 tables. Accepted to RAA (Review)

Counterfactual reasoning, as a crucial manifestation of human intelligence, refers to making presuppositions based on established facts and extrapolating potential outcomes. Existing multimodal large language models (MLLMs) have exhibited impressive cognitive and reasoning capabilities, which have been examined across a wide range of Visual Question Answering (VQA) benchmarks. Nevertheless, how will existing MLLMs perform when faced with counterfactual questions? To answer this question, we first curate a novel \textbf{C}ounter\textbf{F}actual \textbf{M}ulti\textbf{M}odal reasoning benchmark, abbreviated as \textbf{CFMM}, to systematically assess the counterfactual reasoning capabilities of MLLMs. Our CFMM comprises six challenging tasks, each including hundreds of carefully human-labeled counterfactual questions, to evaluate MLLM's counterfactual reasoning capabilities across diverse aspects. Through experiments, interestingly, we find that existing MLLMs prefer to believe what they see, but ignore the counterfactual presuppositions presented in the question, thereby leading to inaccurate responses. Furthermore, we evaluate a wide range of prevalent MLLMs on our proposed CFMM. The significant gap between their performance on our CFMM and that on several VQA benchmarks indicates that there is still considerable room for improvement in existing MLLMs toward approaching human-level intelligence. On the other hand, through boosting MLLMs performances on our CFMM in the future, potential avenues toward developing MLLMs with advanced intelligence can be explored.

Physics-Informed Neural Networks (PINNs) integrate physical principles-typically mathematical models expressed as differential equations-into the machine learning (ML) processes to guarantee the physical validity of ML model solutions. This approach has gained traction in science and engineering for modeling a wide range of physical phenomena. Nonetheless, the effectiveness of PINNs in solving nonlinear hyperbolic partial differential equations (PDEs), is found challenging due to discontinuities inherent in such PDE solutions. While previous research has focused on advancing training algorithms, our study highlights that encoding precise physical laws into PINN framework suffices to address the challenge. By coupling well-constructed governing equations into the most basic, simply structured PINNs, this research tackles both data-independent solution and data-driven discovery of the Buckley-Leverett equation, a typical hyperbolic PDE central to modeling multi-phase fluid flow in porous media. Our results reveal that vanilla PINNs are adequate to solve the Buckley-Leverett equation with superior precision and even handling more complex scenarios including variations in fluid mobility ratios, the addition of a gravity term to the original governing equation, and the presence of multiple discontinuities in the solution. This capability of PINNs enables accurate, efficient modeling and prediction of practical engineering problems, such as water flooding, polymer flooding, inclined flooding, and carbon dioxide injection into saline aquifers. Furthermore, the forward PINN framework with slight modifications can be adapted for inverse problems, allowing the estimation of PDE parameters in the Buckley-Leverett equation from observed data. Sensitivity analysis demonstrate that PINNs remain effective under conditions of slight data scarcity and up to a 5% data impurity.

Generative models for structure-based drug design (SBDD) have shown promising results in recent years. Existing works mainly focus on how to generate molecules with higher binding affinity, ignoring the feasibility prerequisites for generated 3D poses and resulting in false positives. We conduct thorough studies on key factors of ill-conformational problems when applying autoregressive methods and diffusion to SBDD, including mode collapse and hybrid continuous-discrete space. In this paper, we introduce MolCRAFT, the first SBDD model that operates in the continuous parameter space, together with a novel noise reduced sampling strategy. Empirical results show that our model consistently achieves superior performance in binding affinity with more stable 3D structure, demonstrating our ability to accurately model interatomic interactions. To our best knowledge, MolCRAFT is the first to achieve reference-level Vina Scores (-6.59 kcal/mol) with comparable molecular size, outperforming other strong baselines by a wide margin (-0.84 kcal/mol). Code is available at https://github.com/AlgoMole/MolCRAFT., Comment: Accepted to ICML 2024

In this report, we present the 1st place solution for ICCV 2023 OmniObject3D Challenge: Sparse-View Reconstruction. The challenge aims to evaluate approaches for novel view synthesis and surface reconstruction using only a few posed images of each object. We utilize Pixel-NeRF as the basic model, and apply depth supervision as well as coarse-to-fine positional encoding. The experiments demonstrate the effectiveness of our approach in improving sparse-view reconstruction quality. We ranked first in the final test with a PSNR of 25.44614.

Inherent ambiguity in layout annotations poses significant challenges to developing accurate 360{\deg} room layout estimation models. To address this issue, we propose a novel Bi-Layout model capable of predicting two distinct layout types. One stops at ambiguous regions, while the other extends to encompass all visible areas. Our model employs two global context embeddings, where each embedding is designed to capture specific contextual information for each layout type. With our novel feature guidance module, the image feature retrieves relevant context from these embeddings, generating layout-aware features for precise bi-layout predictions. A unique property of our Bi-Layout model is its ability to inherently detect ambiguous regions by comparing the two predictions. To circumvent the need for manual correction of ambiguous annotations during testing, we also introduce a new metric for disambiguating ground truth layouts. Our method demonstrates superior performance on benchmark datasets, notably outperforming leading approaches. Specifically, on the MatterportLayout dataset, it improves 3DIoU from 81.70% to 82.57% across the full test set and notably from 54.80% to 59.97% in subsets with significant ambiguity. Project page: https://liagm.github.io/Bi_Layout/, Comment: CVPR 2024, Project page: https://liagm.github.io/Bi_Layout/

Mixing the fundamental ($\omega$) and the second harmonic (2$\omega$) waves in gas phase is a widely employed technique for emitting terahertz (THz) pulses. The THz generation driven by bi-chromatic fields can be described by the photocurrent model, where the THz generation is attributed to free electrons ionized by the $\omega$ field, and the 2$\omega$ field provides a perturbation to break the symmetry of the asymptotic momentum of free electrons. However, we find that the THz radiation is amplified by one order of magnitude when driven by bi-focal bi-chromatic fields, contradicting the common understanding of the photocurrent model. Meanwhile, present measurements demonstrate that the THz radiation mainly originates from the plasma created by the 2$\omega$ pulses instead of the $\omega$ pulses. Energy transfer from the 2$\omega$ beam to the THz beam during the THz generation has been observed, validating the major contribution of the 2$\omega$ beam. Furthermore, the THz bandwidth has been observed to extensively exceed the bandwidth of the pump pulse, not be explained by the photocurrent model as well. These counterintuitive results indicate that undiscovered physical mechanisms are involved in bi-chromatic THz generation in plasma, presenting a significant challenge for understanding strong-field nonlinear optics and simultaneously expanding various applications.

Surrounding perceptions are quintessential for safe driving for connected and autonomous vehicles (CAVs), where the Bird's Eye View has been employed to accurately capture spatial relationships among vehicles. However, severe inherent limitations of BEV, like blind spots, have been identified. Collaborative perception has emerged as an effective solution to overcoming these limitations through data fusion from multiple views of surrounding vehicles. While most existing collaborative perception strategies adopt a fully connected graph predicated on fairness in transmissions, they often neglect the varying importance of individual vehicles due to channel variations and perception redundancy. To address these challenges, we propose a novel Priority-Aware Collaborative Perception (PACP) framework to employ a BEV-match mechanism to determine the priority levels based on the correlation between nearby CAVs and the ego vehicle for perception. By leveraging submodular optimization, we find near-optimal transmission rates, link connectivity, and compression metrics. Moreover, we deploy a deep learning-based adaptive autoencoder to modulate the image reconstruction quality under dynamic channel conditions. Finally, we conduct extensive studies and demonstrate that our scheme significantly outperforms the state-of-the-art schemes by 8.27% and 13.60%, respectively, in terms of utility and precision of the Intersection over Union.

Numerous roadside perception datasets have been introduced to propel advancements in autonomous driving and intelligent transportation systems research and development. However, it has been observed that the majority of their concentrates is on urban arterial roads, inadvertently overlooking residential areas such as parks and campuses that exhibit entirely distinct characteristics. In light of this gap, we propose CORP, which stands as the first public benchmark dataset tailored for multi-modal roadside perception tasks under campus scenarios. Collected in a university campus, CORP consists of over 205k images plus 102k point clouds captured from 18 cameras and 9 LiDAR sensors. These sensors with different configurations are mounted on roadside utility poles to provide diverse viewpoints within the campus region. The annotations of CORP encompass multi-dimensional information beyond 2D and 3D bounding boxes, providing extra support for 3D seamless tracking and instance segmentation with unique IDs and pixel masks for identifying targets, to enhance the understanding of objects and their behaviors distributed across the campus premises. Unlike other roadside datasets about urban traffic, CORP extends the spectrum to highlight the challenges for multi-modal perception in campuses and other residential areas.

The existing federated learning (FL) methods for spatio-temporal forecasting fail to capture the inherent spatio-temporal heterogeneity, which calls for personalized FL (PFL) methods to model the spatio-temporally variant patterns. While contrastive learning approach is promising in addressing spatio-temporal heterogeneity, the existing methods are noneffective in determining negative pairs and can hardly apply to PFL paradigm. To tackle this limitation, we propose a novel PFL method, named Federated dUal sEmantic aLignment-based contraStive learning (FUELS), which can adaptively align positive and negative pairs based on semantic similarity, thereby injecting precise spatio-temporal heterogeneity into the latent representation space by auxiliary contrastive tasks. From temporal perspective, a hard negative filtering module is introduced to dynamically align heterogeneous temporal representations for the supplemented intra-client contrastive task. From spatial perspective, we design lightweight-but-efficient prototypes as client-level semantic representations, based on which the server evaluates spatial similarity and yields client-customized global prototypes for the supplemented inter-client contrastive task. Extensive experiments demonstrate that FUELS outperforms state-of-the-art methods, with communication cost decreasing by around 94%.

Achieving high-performance perovskite photovoltaics, especially in ambient air relies heavily on optimizing process parameters. However, traditional manual methods often struggle to effectively control the key variables. This inherent challenge requires a paradigm shift toward automated platforms capable of precise and reproducible experiments. Herein, we use a fully automated device acceleration platform (DAP) to optimize the process parameters for preparing full perovskite devices using a two-step method in ambient air. Eight process parameters that have the potential to significantly influence device performance are systematically optimized. Specifically, we delve into the impact of the dispense speed of organic ammonium halide, a parameter that is difficult to control manually, on both perovskite film and device performance. Through the targeted design of experiments, we reveal that the dispense speed significantly affects device performance primarily by adjusting the residual PbI2 content in the films. We find that moderate dispense speeds, e.g., 50 {\mu}l/s, contribute to top-performance devices. Conversely, too fast or too slow speeds result in devices with relatively poorer performance and lower reproducibility. The optimized parameter set enables us to establish a Standard Operation Procedure (SOP) for additive-free perovskite processing under ambient conditions, which yield devices with efficiencies surpassing 23%, satisfactory reproducibility, and state-of-the-art photo-thermal stability. This research underscores the importance of understanding the causality of process parameters in enhancing perovskite photovoltaic performance. Furthermore, our study highlights the pivotal role of automated platforms in discovering innovative workflows and accelerating the development of high-performing perovskite photovoltaic technologies.

Assembly of dissimilar metals can be achieved by different methods, for example, casting, welding, and additive manufacturing (AM). However, undesired phases formed in liquid-phase assembling processes due to solute segregation during solidification diminish mechanical and other properties of the processed parts. In the present work, an open-source software named MaterialsMap, has been developed based on the CALculation of Phase Diagrams (CALPHAD) approach. The primary objective of MaterialsMap is to facilitate the design of an optimal composition pathway for assembling dissimilar alloys with liquid-phases based on the formation of desired and undesired phases along the pathway. In MaterialsMap, equilibrium thermodynamic calculations are used to predict equilibrium phases formed at slow cooling rate, while Scheil-Gulliver simulations are employed to predict non-equilibrium phases formed during rapid cooling. By combining these two simulations, MaterialsMap offers a thorough guide for understanding phase formation in various manufacturing processes, assisting users in making informed decisions during material selection and production. As a demonstration of this approach, a compositional pathway was designed from pure Al to pure Cu through Ag using MaterialsMap. The design was experimentally verified using resistance spot welding (RSW).

We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs. Our framework represents expert LLMs as special expert tokens within the vocabulary of a meta LLM. The meta LLM can route to an expert LLM like generating new tokens. Expert-Token-Routing not only supports learning the implicit expertise of expert LLMs from existing instruction dataset but also allows for dynamic extension of new expert LLMs in a plug-and-play manner. It also conceals the detailed collaboration process from the user's perspective, facilitating interaction as though it were a singular LLM. Our framework outperforms various existing multi-LLM collaboration paradigms across benchmarks that incorporate six diverse expert domains, demonstrating effectiveness and robustness in building generalist LLM system via synergizing multiple expert LLMs.

In recent years, the rapid development of artificial intelligence technology, especially the emergence of large language models (LLMs) such as ChatGPT, has presented significant prospects for application in the field of education. LLMs possess the capability to interpret knowledge, answer questions, and consider context, thus providing support for dialogic teaching to students. Therefore, an examination of the capacity of LLMs to effectively fulfill instructional roles, thereby facilitating student learning akin to human educators within dialogic teaching scenarios, is an exceptionally valuable research topic. This research recruited 34 undergraduate students as participants, who were randomly divided into two groups. The experimental group engaged in dialogic teaching using ChatGPT, while the control group interacted with human teachers. Both groups learned the histogram equalization unit in the information-related course "Digital Image Processing". The research findings show comparable scores between the two groups on the retention test. However, students who engaged in dialogue with ChatGPT exhibited lower performance on the transfer test. Electroencephalography data revealed that students who interacted with ChatGPT exhibited higher levels of cognitive activity, suggesting that ChatGPT could help students establish a knowledge foundation and stimulate cognitive activity. However, its strengths on promoting students. knowledge application and creativity were insignificant. Based upon the research findings, it is evident that ChatGPT cannot fully excel in fulfilling teaching tasks in the dialogue teaching in information related courses. Combining ChatGPT with traditional human teachers might be a more ideal approach. The synergistic use of both can provide students with more comprehensive learning support, thus contributing to enhancing the quality of teaching.

Two algorithms for computing the rational univariate representation of zero-dimensional ideals with parameters are presented in the paper. Different from the rational univariate representation of zero-dimensional ideals without parameters, the number of zeros of zero-dimensional ideals with parameters under various specializations is different, which leads to choosing and checking the separating element, the key to computing the rational univariate representation, is difficult. In order to pick out the separating element, by partitioning the parameter space we can ensure that under each branch the ideal has the same number of zeros. Subsequently with the help of the extended subresultant theorem for parametric cases, two ideas are given to conduct the further partition of parameter space for choosing and checking the separating element. Based on these, we give two algorithms for computing rational univariate representations of zero-dimensional ideals with parameters. Furthermore, the two algorithms have been implemented on the computer algebra system Singular. Experimental data show that the second algorithm has the better performance in contrast to the first one.

Superconducting qubits are a promising platform for building fault-tolerant quantum computers, with recent achievement showing the suppression of logical error with increasing code size. However, leakage into non-computational states, a common issue in practical quantum systems including superconducting circuits, introduces correlated errors that undermine QEC scalability. Here, we propose and demonstrate a leakage reduction scheme utilizing tunable couplers, a widely adopted ingredient in large-scale superconducting quantum processors. Leveraging the strong frequency tunability of the couplers and stray interaction between the couplers and readout resonators, we eliminate state leakage on the couplers, thus suppressing space-correlated errors caused by population propagation among the couplers. Assisted by the couplers, we further reduce leakage to higher qubit levels with high efficiency (98.1%) and low error rate on the computational subspace (0.58%), suppressing time-correlated errors during QEC cycles. The performance of our scheme demonstrates its potential as an indispensable building block for scalable QEC with superconducting qubits., Comment: 25 pages, 15 figures