Search

Most existing RGB-based trackers target low frame rate benchmarks of around 30 frames per second. This setting restricts the tracker's functionality in the real world, especially for fast motion. Event-based cameras as bioinspired sensors provide considerable potential for high frame rate tracking due to their high temporal resolution. However, event-based cameras cannot offer fine-grained texture information like conventional cameras. This unique complementarity motivates us to combine conventional frames and events for high frame rate object tracking under various challenging conditions. Inthispaper, we propose an end-to-end network consisting of multi-modality alignment and fusion modules to effectively combine meaningful information from both modalities at different measurement rates. The alignment module is responsible for cross-style and cross-frame-rate alignment between frame and event modalities under the guidance of the moving cues furnished by events. While the fusion module is accountable for emphasizing valuable features and suppressing noise information by the mutual complement between the two modalities. Extensive experiments show that the proposed approach outperforms state-of-the-art trackers by a significant margin in high frame rate tracking. With the FE240hz dataset, our approach achieves high frame rate tracking up to 240Hz.

HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to expensive sensors and time-consuming computation. Camera-based methods usually need to perform road segmentation and view transformation separately, which often causes distortion and missing content. To push the limits of the technology, we present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. We propose a front-to-top view projection (FTVP) module, which takes the constraint of cycle consistency between views into account and makes full use of their correlation to strengthen the view transformation and scene understanding. In addition, we also apply multi-scale FTVP modules to propagate the rich spatial information of low-level features to mitigate spatial deviation of the predicted object location. Experiments on public benchmarks show that our method achieves the state-of-the-art performance in the tasks of road layout estimation, vehicle occupancy estimation, and multi-class semantic estimation. For multi-class semantic estimation, in particular, our model outperforms all competitors by a large margin. Furthermore, our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction., Comment: Extension to CVPR'21 paper "Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-View Transformation"

Deep reinforcement learning has achieved great success in laser-based collision avoidance work because the laser can sense accurate depth information without too much redundant data, which can maintain the robustness of the algorithm when it is migrated from the simulation environment to the real world. However, high-cost laser devices are not only difficult to apply on a large scale but also have poor robustness to irregular objects, e.g., tables, chairs, shelves, etc. In this paper, we propose a vision-based collision avoidance framework to solve the challenging problem. Our method attempts to estimate the depth and incorporate the semantic information from RGB data to obtain a new form of data, pseudo-laser data, which combines the advantages of visual information and laser information. Compared to traditional laser data that only contains the one-dimensional distance information captured at a certain height, our proposed pseudo-laser data encodes the depth information and semantic information within the image, which makes our method more effective for irregular obstacles. Besides, we adaptively add noise to the laser data during the training stage to increase the robustness of our model in the real world, due to the estimated depth information is not accurate. Experimental results show that our framework achieves state-of-the-art performance in several unseen virtual and real-world scenarios., IROS 2021

Domain Generalization (DG) aims to generalize a model trained on multiple source domains to an unseen target domain. The source domains always require precise annotations, which can be cumbersome or even infeasible to obtain in practice due to the vast amount of data involved. Web data, however, offers an opportunity to access large amounts of unlabeled data with rich style information, which can be leveraged to improve DG. From this perspective, we introduce a novel paradigm of DG, termed as Semi-Supervised Domain Generalization (SSDG), to explore how the labeled and unlabeled source domains can interact, and establish two settings, including the close-set and open-set SSDG. The close-set SSDG is based on existing public DG datasets, while the open-set SSDG, built on the newly-collected web-crawled datasets, presents a novel yet realistic challenge that pushes the limits of current technologies. A natural approach of SSDG is to transfer knowledge from labeled data to unlabeled data via pseudo labeling, and train the model on both labeled and pseudo-labeled data for generalization. Since there are conflicting goals between domain-oriented pseudo labeling and out-of-domain generalization, we develop a pseudo labeling phase and a generalization phase independently for SSDG. Unfortunately, due to the large domain gap, the pseudo labels provided in the pseudo labeling phase inevitably contain noise, which has negative affect on the subsequent generalization phase. Therefore, to improve the quality of pseudo labels and further enhance generalizability, we propose a cyclic learning framework to encourage a positive feedback between these two phases, utilizing an evolving intermediate domain that bridges the labeled and unlabeled domains in a curriculum learning manner..., Comment: 13 pages, 13 figures

In recent years, vision-based crowd analysis has been studied extensively due to its practical applications in real world. In this paper, we formulate a novel crowd analysis problem, in which we aim to predict the crowd distribution in the near future given sequential frames of a crowd video without any identity annotations. Studying this research problem will benefit applications concerned with forecasting crowd dynamics. To solve this problem, we propose a global-residual two-stream recurrent network, which leverages the consecutive crowd video frames as inputs and their corresponding density maps as auxiliary information to predict the future crowd distribution. Moreover, to strengthen the capability of our network, we synthesize scene-specific crowd density maps using simulated data for pretraining. Finally, we demonstrate that our framework is able to predict the crowd distribution for different crowd scenarios and we delve into applications including predicting future crowd count, forecasting high-density region, etc.

Navigation in dense crowds is a well-known open problem in robotics with many challenges in mapping, localization, and planning. Traditional solutions consider dense pedestrians as passive/active moving obstacles that are the cause of all troubles: they negatively affect the sensing of static scene landmarks and must be actively avoided for safety. In this paper, we provide a new perspective: the crowd flow locally observed can be treated as a sensory measurement about the surrounding scenario, encoding not only the scene's traversability but also its social navigation preference. We demonstrate that even using the crowd-flow measurement alone without any sensing about static obstacles, our method still accomplishes good results for mapping, localization, and social-aware planning in dense crowds. Videos of the experiments are available at https://sites.google.com/view/crowdmapping., Comment: Accepted to ISER 2020

Pentobarbital infusion can be a treatment intervention to manage acute brain injury patients with refractory increases in intracranial pressure (ICP). In fact, acute brain injury patients often experience metabolic changes, which result in different nutritional requirements. These considerations are critical since inappropriate nutrition can result in complications such as refeeding syndrome or underfeeding. The purpose of this study was to assess calories received through nutrition feeding for patients with acute brain injury who underwent pentobarbital infusion, compared against predicted nutrition requirements, and evaluate the need for more accurate methods to optimize nutrition regimens. This is a retrospective cohort study of acute brain injury adult patients admitted to University of North Carolina (UNC) hospital neuroscience Intensive Care Unit (ICU) from May 1, 2010 to August 31, 2013 who underwent pentobarbital infusion for longer than 48 hours and at least 16 years old. Data were collected 48 hours before, during, and 48 hours after pentobarbital infusion. Eighteen patients were included in the study. The median age was 57 years (Interquartile range, IQR 54 – 63). Patients spent an average of 6.4 days in the pentobarbital phase. Mean total nutrition calories from tube feed for pre-pentobarbital was 1385.4 kcal ± 542.4; during pentobarbital phase was 510.3 kcal ± 317.1; and post-pentobarbital phase was 1021 kcal ± 509. In addition, both Harris-Benedict equation and 25 kcal/kg predicted higher total nutritional calories required for these patients with acute brain injury receiving pentobarbital infusion.

A new concept on strengthening-plasticity is presented in this paper, along with the design rules of the alloys with high expandable performance. The plasticity of expandable alloys induced by the γ→ɛ martensitic transformation, can be obviously improved. In order to enhance transformation-induced plasticity, alloys with low stacking-fault energy and high starting temperature of ɛ martensite M s(ɛ) should be looked for. Therefore, a formula for the calculation of M s(ɛ) of Fe-based alloys is also given in this paper in order to facilitate the design of expandable tubular materials.

Fine-grained (Y, Mg)-PSZ/MgAl 2 O 4 ceramics are prepared by coating-coprecipitation technology. This kind of material exhibits very good thermal stability at 1100°C for it possesses strength values over 900 MPa and toughness over 15 MPam 1/2 even as the ageing time reaches 800h. The microstructures and phase compositions are examined with XRD, SEM and TEM. The results show that the material derives from a new sub-eutectoid decomposition reaction: c-ZrO 2 →c′-ZrO 2 +t-ZrO 2 +MgO, which is different from the thermal decomposition of conventional Mg-PSZ without Y 2 O 3 addition. Furthermore, two kinds of precipitations can be detected according to their stabilizer. It is found that the tweed-like and colony-like t-ZrO 2 phases in the ceramics, which are stabilized by Y 2 O 3 , can grow up and lead to an increase in volume of the transformable t-ZrO 2 with heat treatment. Only the transformable t-ZrO 2 , originating from the eutectoid decomposition of Css, plays an important role in toughening the matrix. 1998 Elsevier Science Limited and Techna S.r.l.