Search

LiDAR-based outdoor 3D object detection has received widespread attention. However, training 3D detectors from the LiDAR point cloud typically relies on expensive bounding box annotations. This paper presents OC3D, an innovative weakly supervised method requiring only coarse clicks on the bird's eye view of the 3D point cloud. A key challenge here is the absence of complete geometric descriptions of the target objects from such simple click annotations. To address this problem, our proposed OC3D adopts a two-stage strategy. In the first stage, we initially design a novel dynamic and static classification strategy and then propose the Click2Box and Click2Mask modules to generate box-level and mask-level pseudo-labels for static and dynamic instances, respectively. In the second stage, we design a Mask2Box module, leveraging the learning capabilities of neural networks to update mask-level pseudo-labels, which contain less information, to box-level pseudo-labels. Experimental results on the widely used KITTI and nuScenes datasets demonstrate that our OC3D with only coarse clicks achieves state-of-the-art performance compared to weakly-supervised 3D detection methods. Combining OC3D with a missing click mining strategy, we propose an OC3D++ pipeline, which requires only 0.2% annotation cost in the KITTI dataset to achieve performance comparable to fully supervised methods. The code will be made publicly available.

LiDAR-based vision systems are integral for 3D object detection, which is crucial for autonomous navigation. However, they suffer from performance degradation in adverse weather conditions due to the quality deterioration of LiDAR point clouds. Fusing LiDAR with the weather-robust 4D radar sensor is expected to solve this problem. However, the fusion of LiDAR and 4D radar is challenging because they differ significantly in terms of data quality and the degree of degradation in adverse weather. To address these issues, we introduce L4DR, a weather-robust 3D object detection method that effectively achieves LiDAR and 4D Radar fusion. Our L4DR includes Multi-Modal Encoding (MME) and Foreground-Aware Denoising (FAD) technique to reconcile sensor gaps, which is the first exploration of the complementarity of early fusion between LiDAR and 4D radar. Additionally, we design an Inter-Modal and Intra-Modal ({IM}2 ) parallel feature extraction backbone coupled with a Multi-Scale Gated Fusion (MSGF) module to counteract the varying degrees of sensor degradation under adverse weather conditions. Experimental evaluation on a VoD dataset with simulated fog proves that L4DR is more adaptable to changing weather conditions. It delivers a significant performance increase under different fog levels, improving the 3D mAP by up to 20.0% over the traditional LiDAR-only approach. Moreover, the results on the K-Radar dataset validate the consistent performance improvement of L4DR in real-world adverse weather conditions.

The advent of sixth-generation (6G) mobile networks introduces two groundbreaking capabilities: sensing and artificial intelligence (AI). Sensing leverages multi-modal sensors to capture real-time environmental data, while AI brings powerful models to the network edge, enabling intelligent Internet-of-Things (IoT) applications. These features converge in the Integrated Sensing and Edge AI (ISEA) paradigm, where edge devices collect and locally process sensor data before aggregating it centrally for AI tasks. Point clouds (PtClouds), generated by depth sensors, are crucial in this setup, supporting applications such as autonomous driving and mixed reality. However, the heavy computational load and communication demands of PtCloud fusion pose challenges. To address these, the FlyCom$^2$ framework is proposed, optimizing distributed PtCloud fusion through on-the-fly communication and computing, namely streaming on-sensor processing, progressive data uploading integrated communication-efficient AirComp, and the progressive output of a global PtCloud representation. FlyCom$^2$ distinguishes itself by aligning PtCloud fusion with Gaussian process regression (GPR), ensuring that global PtCloud representation progressively improves as more observations are received. Joint optimization of local observation synthesis and AirComp receiver settings is based on minimizing prediction error, balancing communication distortions, data heterogeneity, and temporal correlation. This framework enhances PtCloud fusion by balancing local processing demands with efficient central aggregation, paving the way for advanced 6G applications. Validation on real-world datasets demonstrates the efficacy of FlyCom$^2$, highlighting its potential in next-generation mobile networks., Comment: This is an ongoing work under revision

Let $q=p^n$ be an odd prime power and let $\mathbb{F}_q$ be the finite field with $q$ elements. Let $\widehat{\mathbb{F}_q^{\times}}$ be the group of all multiplicative characters of $\mathbb{F}_q$ and let $\chi$ be a generator of $\widehat{\mathbb{F}_q^{\times}}$. In this paper, we investigate arithmetic properties of certain cyclotomic matrices involving nonzero squares over $\mathbb{F}_q$. For example, let $s_1,s_2,\cdots,s_{(q-1)/2}$ be all nonzero squares over $\mathbb{F}_q$. For any integer $1\le r\le q-2$, define the matrix $$B_{q,2}(\chi^r):=\left[\chi^r(s_i+s_j)+\chi^r(s_i-s_j)\right]_{1\le i,j\le (q-1)/2}.$$ We prove that if $q\equiv 3\pmod 4$, then $$\det (B_{q,2}(\chi^r))=\prod_{0\le k\le (q-3)/2}J_q(\chi^r,\chi^{2k})= \begin{cases} (-1)^{\frac{q-3}{4}}{\bf i}^nG_q(\chi^r)^{\frac{q-1}{2}}/\sqrt{q} & \mbox{if}\ r\equiv 1\pmod 2,\\ G_q(\chi^r)^{\frac{q-1}{2}}/q & \mbox{if}\ r\equiv 0\pmod 2, \end{cases}$$ where $J_q(\chi^r,\chi^{2k})$ and $G_q(\chi^r)$ are the Jacobi sum and the Gauss sum over $\mathbb{F}_q$ respectively., Comment: 15 pages

The emergence of large-scale foundation models (FoMo's) that can perform human-like intelligence motivates their deployment at the network edge for devices to access state-of-the-art artificial intelligence. For better user experiences, the pre-trained FoMo's need to be adapted to specialized downstream tasks through fine-tuning techniques. To transcend a single device's memory and computation limitations, we advocate multi-device cooperation within the device-edge cooperative fine-tuning (DEFT) paradigm, where edge devices cooperate to simultaneously optimize different parts of fine-tuning parameters within a FoMo. However, the parameter blocks reside at different depths within a FoMo architecture, leading to varied computation latency-and-memory cost due to gradient backpropagation-based calculations. The heterogeneous on-device computation and memory capacities and channel conditions necessitate an integrated communication-and-computation allocation of local computation loads and communication resources to achieve low-latency (LoLa) DEFT. To this end, we consider the depth-ware DEFT block allocation problem. The involved optimal block-device matching is tackled by the proposed low-complexity Cutting-RecoUNting-CHecking (CRUNCH) algorithm, which is designed by exploiting the monotone-increasing property between block depth and computation latency-and-memory cost. Next, the joint bandwidth-and-block allocation makes the problem more sophisticated. We observe a splittable Lagrangian expression through the transformation and analysis of the original problem, where the variables indicating device involvement are introduced. Then, the dual ascent method is employed to tackle this problem iteratively. Through extensive experiments conducted on the GLUE benchmark, our results demonstrate significant latency reduction achievable by LoLa DEFT for fine-tuning a RoBERTa model., Comment: This work has been submitted to the IEEE for possible publication

Motivated by the recent work of Zhi-Wei Sun on determinants involving the Legendre symbol, in this paper, we study some matrices concerning subgroups of finite fields. For example, let $q\equiv 3\pmod 4$ be an odd prime power and let $\phi$ be the unique quadratic multiplicative character of the finite field $\mathbb{F}_q$. If set $\{s_1,\cdots,s_{(q-1)/2}\}=\{x^2:\ x\in\mathbb{F}_q\setminus\{0\}\}$, then we prove that $$\det\left[t+\phi(s_i+s_j)+\phi(s_i-s_j)\right]_{1\le i,j\le (q-1)/2}=\left(\frac{q-1}{2}t-1\right)q^{\frac{q-3}{4}}.$$ This confirms a conjecture of Zhi-Wei Sun.

Let $p$ be an odd prime and $x$ be an indeterminate. Recently, Z.-W. Sun proposed the following conjecture: $$\det\left[x+\left(\frac{j-i}{p}\right)\right]_{0\le i,j\le \frac{p-1}{2}}=\begin{cases} (\frac{2}{p})pb_px-a_p & \mbox{if}\ p\equiv 1\pmod4, 1 & \mbox{if}\ p\equiv 3\pmod4, \end{cases}$$ where $a_p$ and $b_p$ are rational numbers related to the fundamental unit and class number of the real quadratic field $\mathbb{Q}(\sqrt{p})$. In this paper, we confirm the above conjecture of Sun based on Vsemirnov's decomposition of Chapman's "evil determinant".

The prevalent approaches of unsupervised 3D object detection follow cluster-based pseudo-label generation and iterative self-training processes. However, the challenge arises due to the sparsity of LiDAR scans, which leads to pseudo-labels with erroneous size and position, resulting in subpar detection performance. To tackle this problem, this paper introduces a Commonsense Prototype-based Detector, termed CPD, for unsupervised 3D object detection. CPD first constructs Commonsense Prototype (CProto) characterized by high-quality bounding box and dense points, based on commonsense intuition. Subsequently, CPD refines the low-quality pseudo-labels by leveraging the size prior from CProto. Furthermore, CPD enhances the detection accuracy of sparsely scanned objects by the geometric knowledge from CProto. CPD outperforms state-of-the-art unsupervised 3D detectors on Waymo Open Dataset (WOD), PandaSet, and KITTI datasets by a large margin. Besides, by training CPD on WOD and testing on KITTI, CPD attains 90.85% and 81.01% 3D Average Precision on easy and moderate car classes, respectively. These achievements position CPD in close proximity to fully supervised detectors, highlighting the significance of our method. The code will be available at https://github.com/hailanyi/CPD., Comment: Accepted by CVPR 2024

Inspired by the works of L. Carlitz and Z.-W. Sun on cyclotomic matrices, in this paper, we investigate certain cyclotomic matrices involving Gauss sums over finite fields, which can be viewed as finite field analogues of certain matrices related to the Gamma function. For example, let $q=p^n$ be an odd prime power with $p$ prime and $n\in\mathbb{Z}^+$. Let $\zeta_p=e^{2\pi{\bf i}/p}$ and let $\chi$ be a generator of the group of all mutiplicative characters of the finite field $\mathbb{F}_q$. For the Gauss sum $$G_q(\chi^{r})=\sum_{x\in\mathbb{F}_q}\chi^{r}(x)\zeta_p^{{\rm Tr}_{\mathbb{F}_q/\mathbb{F}_p}(x)},$$ we prove that $$\det \left[G_q(\chi^{2i+2j})\right]_{0\le i,j\le (q-3)/2}=(-1)^{\alpha_p}\left(\frac{q-1}{2}\right)^{\frac{q-1}{2}}2^{\frac{p^{n-1}-1}{2}},$$ where $$\alpha_p= \begin{cases} 1 & \mbox{if}\ n\equiv 1\pmod 2, (p^2+7)/8 & \mbox{if}\ n\equiv 0\pmod 2. \end{cases}$$, Comment: 15 pages. Comments are very welcome

LiDAR-based 3D object detection models have traditionally struggled under rainy conditions due to the degraded and noisy scanning signals. Previous research has attempted to address this by simulating the noise from rain to improve the robustness of detection models. However, significant disparities exist between simulated and actual rain-impacted data points. In this work, we propose a novel rain simulation method, termed DRET, that unifies Dynamics and Rainy Environment Theory to provide a cost-effective means of expanding the available realistic rain data for 3D detection training. Furthermore, we present a Sunny-to-Rainy Knowledge Distillation (SRKD) approach to enhance 3D detection under rainy conditions. Extensive experiments on the WaymoOpenDataset large-scale dataset show that, when combined with the state-of-the-art DSVT model and other classical 3D detectors, our proposed framework demonstrates significant detection accuracy improvements, without losing efficiency. Remarkably, our framework also improves detection capabilities under sunny conditions, therefore offering a robust solution for 3D detection regardless of whether the weather is rainy or sunny, Comment: Accepted by AAAI2024

Large Language Models (LLMs) exhibit robust problem-solving capabilities for diverse tasks. However, most LLM-based agents are designed as specific task solvers with sophisticated prompt engineering, rather than agents capable of learning and evolving through interactions. These task solvers necessitate manually crafted prompts to inform task rules and regulate LLM behaviors, inherently incapacitating to address complex dynamic scenarios e.g., large interactive games. In light of this, we propose Agent-Pro: an LLM-based Agent with Policy-level Reflection and Optimization that can learn a wealth of expertise from interactive experiences and progressively elevate its behavioral policy. Specifically, it involves a dynamic belief generation and reflection process for policy evolution. Rather than action-level reflection, Agent-Pro iteratively reflects on past trajectories and beliefs, fine-tuning its irrational beliefs for a better policy. Moreover, a depth-first search is employed for policy optimization, ensuring continual enhancement in policy payoffs. Agent-Pro is evaluated across two games: Blackjack and Texas Hold'em, outperforming vanilla LLM and specialized models. Our results show Agent-Pro can learn and evolve in complex and dynamic scenes, which also benefits numerous LLM-based applications., Comment: Accepted to ACL-2024 Main, camera-ready version

Encoding is the carrier of information. AI models possess basic capabilities in syntax, semantics, and reasoning, but these capabilities are sensitive to specific inputs. In this study, we introduce an encoding algorithm, TSIS (Simplified TSID), to the t-SMILES family as a fragment-based linear molecular representation. TSID has been demonstrated to significantly outperform classical SMILES, DeepSMILES, and SELFIES in previous work. A further comparative analysis in this study reveals that the tree structure used by TSID is more easily learned than anticipated, regardless of whether Transformer or LSTM models are used. Furthermore, TSIS demonstrates comparable performance to TSID and significantly outperforms SMILES, SELFIES, and SAFE. While SEFLIES and SAFE present significant challenges in semantic and syntactic analysis, respectively, due to their inherent complexity.

In 2019, Zhi-Wei Sun posed an interesting conjecture on certain determinants with Legendre symbol entries. In this paper, by using the arithmetic properties of $p$-th cyclotomic field and the finite field $\mathbb{F}_p$, we confirm this conjecture., Comment: 13 pages

This paper introduces HPC-Net, a high-precision and rapidly convergent object detection network., Comment: 10 pages,5 figures, conference

Recent advances in planar optics with geometric-phase superstructures have brought a new paradigm in the control of structured light and, in particular, has substantially enhanced the capabilities of generating and detecting orbital angular momentum (OAM) states of light and associated spatial modes. However, the structured modal interface that can reciprocally link OAM states via adiabatic control and access-associated higher-order geometric phase remains absent in planar optics. In this work, we propose and experimentally demonstrate a planar optical astigmatic retarder fabricated with liquid-crystal (LC) geometric phase. The LC superstructure was designed with the principle of fractional Fourier transformation and is capable of reciprocal conversion between all possible OAM states on the same modal sphere. Such a planar device paves the way towards an easily deployed modal interface of paraxial OAM states, unlocks the resource of higher-order geometric phase, and has promising applications in high-dimensional classical/quantum information.

Since the diffraction behavior of a light field is fully determined by its spatial wavefunction, i.e., its spatial complex amplitude (SCA), full characterization of spatial wavefunction, plays a vital role in modern optics from both the fundamental and applied aspects. In this work, we present a novel complex-amplitude profiler based on spatial Stokes tomography with the capability to fully determine the SCA of a light field in a single shot with high precision and resolution. The SCA slice observed at any propagation plane provides complete information about the light field, thus allowing us to further retrieve the complete beam structure in 3 dimensions space, as well as the exact modal constitution in terms of spatial degrees of freedom. The principle demonstrated here provides an important advancement for the full characterization of light beams with a broad spectrum of potential applications in various areas of optics, especially for the growing field of structured light.

Foundation models (FoMos), referring to large-scale AI models, possess human-like capabilities and are able to perform competitively in the domain of human intelligence. The breakthrough in FoMos has inspired researchers to deploy such models in the sixth-generation (6G) mobile networks for automating a broad range of tasks in next-generation mobile applications. While the sizes of FoMos are reaching their peaks, their next phase is expected to focus on fine-tuning the models to specific downstream tasks. This inspires us to propose the vision of FoMo fine-tuning as a 6G service. Its key feature is the exploitation of existing parameter-efficient fine-tuning (PEFT) techniques to tweak only a small fraction of model weights for a FoMo to become customized for a specific task. To materialize the said vision, we survey the state-of-the-art PEFT and then present a novel device-edge fine-tuning (DEFT) framework for providing efficient and privacy-preserving fine-tuning services at the 6G network edge. The framework consists of the following comprehensive set of techniques: 1) Control of fine-tuning parameter sizes in different transformer blocks of a FoMo; 2) Over-the-air computation for realizing neural connections in DEFT; 3) Federated DEFT in a multi-device system by downloading a FoMo emulator or gradients; 4) On-the-fly prompt-ensemble tuning; 5) Device-to-device prompt transfer among devices. Experiments are conducted using pre-trained FoMos with up to 11 billion parameters to demonstrate the effectiveness of DEFT techniques. The article is concluded by presenting future research opportunities., Comment: 13 pages, 6 figures

For the 6G mobile networks, in-situ model downloading has emerged as an important use case to enable real-time adaptive artificial intelligence on edge devices. However, the simultaneous downloading of diverse and high-dimensional models to multiple devices over wireless links presents a significant communication bottleneck. To overcome the bottleneck, we propose the framework of model broadcasting and assembling (MBA), which represents the first attempt on leveraging reusable knowledge, referring to shared parameters among tasks, to enable parameter broadcasting to reduce communication overhead. The MBA framework comprises two key components. The first, the MBA protocol, defines the system operations including parameter selection from a model library, power control for broadcasting, and model assembling at devices. The second component is the joint design of parameter-selection-and-power-control (PS-PC), which provides guarantees on devices' model performance and minimizes the downloading latency. The corresponding optimization problem is simplified by decomposition into the sequential PS and PC sub-problems without compromising its optimality. The PS sub-problem is solved efficiently by designing two efficient algorithms. On one hand, the low-complexity algorithm of greedy parameter selection features the construction of candidate model sets and a selection metric, both of which are designed under the criterion of maximum reusable knowledge among tasks. On the other hand, the optimal tree-search algorithm gains its efficiency via the proposed construction of a compact binary tree pruned using model architecture constraints and an intelligent branch-and-bound search. Given optimal PS, the optimal PC policy is derived in closed form. Extensive experiments demonstrate the substantial reduction in downloading latency achieved by the proposed MBA compared to traditional model downloading., Comment: Submitted to IEEE for possible publication

Motivated by the works of L. Carlitz, R. Chapman and Z.-W. Sun on cyclotomic matrices, in this paper, we investigate certain cyclotomic matrices concerning the Jacobi sums over finite fields, which can be viewed as finite field analogues of certain matrices involving the Beta function. For example, let $q>1$ be a prime power and let $\chi$ be a generator of the group of all multiplicative characters of $\mathbb{F}_q$. Then we prove that $$\det\left[J_q(\chi^i,\chi^j)\right]_{1\le i,j\le q-2}=(q-1)^{q-3},$$ where $J_q(\chi^i,\chi^j)$ is the Jacobi sum over $\mathbb{F}_q$. This is a finite analogue of $$\det [B(i,j)]_{1\le i,j\le n}=(-1)^{\frac{n(n-1)}{2}}\prod_{r=0}^{n-1}\frac{(r!)^3}{(n+r)!},$$ where $B$ is the Beta function. Also, if $q=p\ge5$ is an odd prime, then we show that $$\det \left[J_p(\chi^{2i},\chi^{2j})\right]_{1\le i,j\le (p-3)/2}=\frac{1+(-1)^{\frac{p+1}{2}}p}{4}\left(\frac{p-1}{2}\right)^{\frac{p-5}{2}}.$$, Comment: 16 pages. Comments are welcome

In this paper, motivated by the work of Chapman, Vsemirnov and Sun et al., we investigate some arithmetic properties of the generalized Legendre matrices over finite fields. For example, letting $a_1,\cdots,a_{(q-1)/2}$ be all non-zero squares in the finite field $\mathbb{F}_q$ which contains $q$ elements with $2\nmid q$, we give the explicit value of $D_{(q-1)/2}=\det[(a_i+a_j)^{(q-3)/2}]_{1\le i,j\le (q-1)/2}$. In particular, if $q=p$ is a prime greater than $3$, then $$\left(\frac{\det D_{(p-1)/2}}{p}\right)= \begin{cases} 1 & \mbox{if}\ p\equiv1\pmod4, (-1)^{(h(-p)+1)/2} & \mbox{if}\ p\equiv 3\pmod4\ \text{and}\ p>3, \end{cases}$$ where $(\cdot/p)$ is the Legendre symbol and $h(-p)$ is the class number of $\mathbb{Q}(\sqrt{-p})$., Comment: 12 pages

In this paper, we study some topics concerning the additive decompositions of the set $D_k$ of all $k$th power residues modulo a prime $p$. For example, given a positive integer $k\ge2$, we prove that $$\lim_{x\rightarrow+\infty}\frac{B(x)}{\pi(x)}=0,$$ where $\pi(x)$ is the number of primes $p\le x$ and $B(x)$ denotes the cardinality of the set $$\{p\le x: p\equiv1\pmod k; D_k\ \text{has a non-trivial 2-additive decomposition}\}.$$

Recently, virtual/pseudo-point-based 3D object detection that seamlessly fuses RGB images and LiDAR data by depth completion has gained great attention. However, virtual points generated from an image are very dense, introducing a huge amount of redundant computation during detection. Meanwhile, noises brought by inaccurate depth completion significantly degrade detection precision. This paper proposes a fast yet effective backbone, termed VirConvNet, based on a new operator VirConv (Virtual Sparse Convolution), for virtual-point-based 3D object detection. VirConv consists of two key designs: (1) StVD (Stochastic Voxel Discard) and (2) NRConv (Noise-Resistant Submanifold Convolution). StVD alleviates the computation problem by discarding large amounts of nearby redundant voxels. NRConv tackles the noise problem by encoding voxel features in both 2D image and 3D LiDAR space. By integrating VirConv, we first develop an efficient pipeline VirConv-L based on an early fusion design. Then, we build a high-precision pipeline VirConv-T based on a transformed refinement scheme. Finally, we develop a semi-supervised pipeline VirConv-S based on a pseudo-label framework. On the KITTI car 3D detection test leaderboard, our VirConv-L achieves 85% AP with a fast running speed of 56ms. Our VirConv-T and VirConv-S attains a high-precision of 86.3% and 87.2% AP, and currently rank 2nd and 1st, respectively. The code is available at https://github.com/hailanyi/VirConv., Comment: Accepted by CVPR 2023

Reciprocal spin-orbit coupling (SOC) via geometric phase with flat optics provides a promising platform for shaping and controlling paraxial structured light. Current devices, from the pioneering q-plates to the recent J-plates, provide only spin-dependent wavefront modulation without amplitude control. However, achieving control over all the spatial dimensions of paraxial SOC states requires spin-dependent control of corresponding complex amplitude, which remains challenging for flat optics. Here, to address this issue, we present a new type of flat-optics elements termed structured geometric phase gratings that is capable of conjugated complex-amplitude control for orthogonal input circular polarizations. By using a microstructured liquid crystal photoalignment technique, we engineered a series of flat-optics elements and experimentally showed their excellent precision in arbitrary SOC control. This principle unlocks the full-field control of paraxial structured light via flat optics, providing a promising way to develop an information exchange and processing units for general photonic SOC states, as well as extra-/intracavity mode convertors for high-precision laser beam shaping.

Effective representation of molecules is a crucial factor affecting the performance of artificial intelligence models. This study introduces a flexible, fragment-based, multiscale molecular representation framework called t-SMILES (tree-based SMILES) with three code algorithms: TSSA, TSDY and TSID. It describes molecules using SMILES-type strings obtained by performing a breadth-first search on a full binary tree formed from a fragmented molecular graph. Systematic evaluations using JTVAE, BRICS, MMPA, and Scaffold show the feasibility of constructing a multi-code molecular description system, where various descriptions complement each other, enhancing the overall performance. In addition, it can avoid overfitting and achieve higher novelty scores while maintaining reasonable similarity on labeled low-resource datasets, regardless of whether the model is original, data-augmented, or pre-trained then fine-tuned. Furthermore, it significantly outperforms classical SMILES, DeepSMILES, SELFIES and baseline models in goal-directed tasks. And it surpasses state-of-the-art fragment, graph and SMILES based approaches on ChEMBL, Zinc, and QM9.