Search

Quantization has emerged as an essential technique for deploying deep neural networks (DNNs) on devices with limited resources. However, quantized models exhibit vulnerabilities when exposed to various noises in real-world applications. Despite the importance of evaluating the impact of quantization on robustness, existing research on this topic is limited and often disregards established principles of robustness evaluation, resulting in incomplete and inconclusive findings. To address this gap, we thoroughly evaluated the robustness of quantized models against various noises (adversarial attacks, natural corruptions, and systematic noises) on ImageNet. Extensive experiments demonstrate that lower-bit quantization is more resilient to adversarial attacks but is more susceptible to natural corruptions and systematic noises. Notably, our investigation reveals that impulse noise (in natural corruptions) and the nearest neighbor interpolation (in systematic noises) have the most significant impact on quantized models. Our research contributes to advancing the robust quantization of models and their deployment in real-world scenarios., Comment: Workshop at IEEE Conference on Computer Vision and Pattern Recognition 2023

Adversarial attacks in the physical world can harm the robustness of detection models. Evaluating the robustness of detection models in the physical world can be challenging due to the time-consuming and labor-intensive nature of many experiments. Thus, virtual simulation experiments can provide a solution to this challenge. However, there is no unified detection benchmark based on virtual simulation environment. To address this challenge, we proposed an instant-level data generation pipeline based on the CARLA simulator. Using this pipeline, we generated the DCI dataset and conducted extensive experiments on three detection models and three physical adversarial attacks. The dataset covers 7 continuous and 1 discrete scenes, with over 40 angles, 20 distances, and 20,000 positions. The results indicate that Yolo v6 had strongest resistance, with only a 6.59% average AP drop, and ASA was the most effective attack algorithm with a 14.51% average AP reduction, twice that of other algorithms. Static scenes had higher recognition AP, and results under different weather conditions were similar. Adversarial attack algorithm improvement may be approaching its 'limitation'., Comment: CVPR 2023 workshop

Simulations of laser-induced electron dynamics in a molecular system are performed using time-dependent (TD) equation-of-motion (EOM) coupled-cluster (CC) theory. The target system has been chosen to highlight potential shortcomings of truncated TD-EOM-CC methods [represented in this work by TD-EOM-CC with single and double excitations (TD-EOM-CCSD)], where unphysical spectroscopic features can emerge. Specifically, we explore driven resonant electronic excitations in magnesium fluoride in the proximity of an avoided crossing. Near the avoided crossing, the CCSD similarity-transformed Hamiltonian is defective, meaning that it has complex eigenvalues, and oscillator strengths may take on negative values. When an external field is applied to drive transitions to states exhibiting these traits, unphysical dynamics are observed. For example, the stationary states that make up the time-dependent state acquire populations that can be negative, exceed one, or even be complex-valued.

Accurate and consistent 3D tracking from multiple cameras is a key component in a vision-based autonomous driving system. It involves modeling 3D dynamic objects in complex scenes across multiple cameras. This problem is inherently challenging due to depth estimation, visual occlusions, appearance ambiguity, etc. Moreover, objects are not consistently associated across time and cameras. To address that, we propose an end-to-end \textbf{MU}lti-camera \textbf{TR}acking framework called MUTR3D. In contrast to prior works, MUTR3D does not explicitly rely on the spatial and appearance similarity of objects. Instead, our method introduces \textit{3D track query} to model spatial and appearance coherent track for each object that appears in multiple cameras and multiple frames. We use camera transformations to link 3D trackers with their observations in 2D images. Each tracker is further refined according to the features that are obtained from camera images. MUTR3D uses a set-to-set loss to measure the difference between the predicted tracking results and the ground truths. Therefore, it does not require any post-processing such as non-maximum suppression and/or bounding box association. MUTR3D outperforms state-of-the-art methods by 5.3 AMOTA on the nuScenes dataset. Code is available at: \url{https://github.com/a1600012888/MUTR3D}., Comment: Appear on CVPR 2022 Workshop on Autonomous Driving

Perception and prediction are two separate modules in the existing autonomous driving systems. They interact with each other via hand-picked features such as agent bounding boxes and trajectories. Due to this separation, prediction, as a downstream module, only receives limited information from the perception module. To make matters worse, errors from the perception modules can propagate and accumulate, adversely affecting the prediction results. In this work, we propose ViP3D, a query-based visual trajectory prediction pipeline that exploits rich information from raw videos to directly predict future trajectories of agents in a scene. ViP3D employs sparse agent queries to detect, track, and predict throughout the pipeline, making it the first fully differentiable vision-based trajectory prediction approach. Instead of using historical feature maps and trajectories, useful information from previous timestamps is encoded in agent queries, which makes ViP3D a concise streaming prediction method. Furthermore, extensive experimental results on the nuScenes dataset show the strong vision-based prediction performance of ViP3D over traditional pipelines and previous end-to-end models., Comment: CVPR 2023

We introduce a framework for multi-camera 3D object detection. In contrast to existing works, which estimate 3D bounding boxes directly from monocular images or use depth prediction networks to generate input for 3D object detection from 2D information, our method manipulates predictions directly in 3D space. Our architecture extracts 2D features from multiple camera images and then uses a sparse set of 3D object queries to index into these 2D features, linking 3D positions to multi-view images using camera transformation matrices. Finally, our model makes a bounding box prediction per object query, using a set-to-set loss to measure the discrepancy between the ground-truth and the prediction. This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model. Moreover, our method does not require post-processing such as non-maximum suppression, dramatically improving inference speed. We achieve state-of-the-art performance on the nuScenes autonomous driving benchmark., Comment: Accepted to CORL 2021

Real-time video frame interpolation (VFI) is very useful in video processing, media players, and display devices. We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for VFI. To realize a high-quality flow-based VFI method, RIFE uses a neural network named IFNet that can estimate the intermediate flows end-to-end with much faster speed. A privileged distillation scheme is designed for stable IFNet training and improve the overall performance. RIFE does not rely on pre-trained optical flow models and can support arbitrary-timestep frame interpolation with the temporal encoding input. Experiments demonstrate that RIFE achieves state-of-the-art performance on several public benchmarks. Compared with the popular SuperSlomo and DAIN methods, RIFE is 4--27 times faster and produces better results. Furthermore, RIFE can be extended to wider applications thanks to temporal encoding. The code is available at https://github.com/megvii-research/ECCV2022-RIFE., Comment: Accepted to ECCV 2022

Currently, there are two major bottleneck problems which seriously affect prevention of the Newcastle disease (ND): interference of maternal antibody on NDV vaccination and slow production of neutralization antibody. To overcome these problems, in present study, four rNDV vaccine strains expressing bio-adjuvants chIL2, chIL15, chGM-CSF or fliC gene were constructed and rescued using reverse genetics approach. The HI antibody titers of SPF birds immunized with rNDV reached to 5.5log2, 4.7log2, 6.5log2 and 5.8log2, respectively at the 8th day post immunization, while the antibody titers of the parental virus and control were 3.3log2 and 1log2, respectively. The immunized chickens were challenged by 104ELD50 dose of the virulent NDV BJ strains at the 7th day post immunization. The protection rate of the four rNDVs bio-adjuvant groups was 100%, while the protection rate of the parental group was 80%. We also examined the anti-maternal antibody activity of these adjuvant vaccines by detection HI titer after vaccination of chickens with high (8.4log2) or low (5log2) maternal antibody levels. In chicken flock with higher maternal antibody, parental strain could not resist the influence of the maternal antibody and induce any notable change of HI antibody kinetics. However, both rClon30-chGM-CSF and rClon30-flic were able to resist the influence of the maternal antibody and maintained the HI antibody above the protection level during the 14day's experiment. In chicken flock with lower maternal antibody, the parental rclone30 strain could not induce HI titer to the protection level until the 14th day, but both rClon30-GM-CSF and clone30-fliC raised the HI antibody to above the protection level at the 7th day post vaccination.

Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnerable to adversarial perturbations, which raised a serious robustness issue of deep networks. Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks. A major drawback of existing adversarial training algorithms is the computational overhead of the generation of adversarial examples, typically far greater than that of the network training. This leads to the unbearable overall computational cost of adversarial training. In this paper, we show that adversarial training can be cast as a discrete time differential game. Through analyzing the Pontryagin's Maximal Principle (PMP) of the problem, we observe that the adversary update is only coupled with the parameters of the first layer of the network. This inspires us to restrict most of the forward and back propagation within the first layer of the network during adversary updates. This effectively reduces the total number of full forward and backward propagation to only one for each group of adversary updates. Therefore, we refer to this algorithm YOPO (You Only Propagate Once). Numerical experiments demonstrate that YOPO can achieve comparable defense accuracy with approximately 1/5 ~ 1/4 GPU time of the projected gradient descent (PGD) algorithm. Our codes are available at https://https://github.com/a1600012888/YOPO-You-Only-Propagate-Once., Comment: Accepted as a conference paper at NeurIPS 2019

Deep neural networks with more parameters and FLOPs have higher capacity and generalize better to diverse domains. But to be deployed on edge devices, the model's complexity has to be constrained due to limited compute resource. In this work, we propose a method to improve the model capacity without increasing inference-time complexity. Our method is based on an assumption of data locality: for an edge device, within a short period of time, the input data to the device are sampled from a single domain with relatively low diversity. Therefore, it is possible to utilize a specialized, low-complexity model to achieve good performance in that input domain. To leverage this, we propose Domain-aware Dynamic Network (DDN), which is a high-capacity dynamic network in which each layer contains multiple weights. During inference, based on the input domain, DDN dynamically combines those weights into one single weight that specializes in the given domain. This way, DDN can keep the inference-time complexity low but still maintain a high capacity. Experiments show that without increasing the parameters, FLOPs, and actual latency, DDN achieves up to 2.6\% higher AP50 than a static network on the BDD100K object-detection benchmark.

This study was conducted to investigate enhancement of anti-tumor effects of the lentogenic Newcastle disease virus Clone30 strain (NDV rClone30) expressing cytosine deaminase (CD) gene against tumor cells and in murine groin tumor-bearing models. Cytotoxic effects of the rClone30-CD/5-FC on the HepG2 cell line were examined by an MTT method. Anti-tumor activity of rClone30-CD/5-FC was examined in H22 tumor-bearing mice. Compared to the rClone30-CD virus treatment alone, NDV rClone30-CD/5-FC at 0.1 and 1 MOIs exerted significant cytotoxic effects (P

Newcastle disease virus (NDV) is an intrinsically tumor-specific virus, several clinical trials have reported that mesogenic NDV is a safe and effective agent for human cancer therapy. Interleukin 2 (IL2) is a cytokine that stimulates T cell propagation to trigger innate and adaptive immunity. IL2 has been used for cancer therapy and has achieved curative effects. In this study, a recombinant NDV LaSota strain expressing human interleukin 2 (rLaSota/IL2) was generated. The ability of rLaSota/IL2 to express human IL2 was detected in the infected tumor cells. In addition, the activity of IL2 was analyzed. The antitumor potential of rLaSota/IL2 was studied by xenograph mice carrying H22 and B16-F10 cells. Tumor-specific CD4+ and CD8+ T cells and MHC II were also analyzed in the two tumor-bearing models. Our study showed that rLaSota/IL2 significantly stimulated tumor-specific cytotoxic T-lymphocyte (CTL) responses and increased regulatory CD4+ and cytotoxic CD8+ T cells proliferation. The treatment with rLaSota/IL2 led to tumor regression in tumor-bearing mice and prolonged the survival of tumor-bearing mice. Furthermore, tumor challenging experiments demonstrated that rLaSota/IL2 invoked mice a unique capacity to remember a pathogen through the generation of memory T cells, which protect the host in the event of reinfection and form adaptive immune system. The result indicates that tumor-infiltrating CD4+ T regulatory cells may denote the effective regression of tumors. Taken together, rLaSota/IL2 has potential for immunotherapy and oncolytic therapy of cancers and may be an ideal candidate for clinical application in future cancer therapy.