Search

Decentralised supply chain systems have become more widely used to meet the modern supply chain systems, which also offer better transparency, trust, and efficiency. This thesis explored four main challenges associated with traditional supply chain systems, including identity management, service discovery, deal management, and contract execution. The research attempts to overcome these challenges by designing a framework, and this thesis is focused on the first two challenges including identity management and service discovery in decentralised supply chains. Later challenges would be resolved in the future research. Moreover, by employing W3C Decentralised Identifier (DID), this framework fulfils varied business and technical needs for supply chain identity management with transparent sharing and connecting rich attributes. In the framework, blockchain is also utilized to register DIDs in a graphic-database-based service repository system, which can organise DIDs in a way that resembles a graph, making service discovery more efficient. Besides, this framework is further validated with the implementation of an Ethereum-based prototype as a proof of concept to demonstrate its feasibility. Overall, this research brings cutting-edge technologies together with practical applications to provide a step toward revolutionising decentralised supply chains.

Eleocharis vivipara, an amphibious sedge in the Cyperaceae family, has several remarkable properties, most notably its alternate use of C3 photosynthesis underwater and C4 photosynthesis on land. However, the absence of genomic data has hindered its utility for evolutionary and genetic research. Here, we present a high-quality genome for E. vivipara, representing the first chromosome-level genome for the Eleocharis genus, with an approximate size of 965.22 Mb mainly distributed across 10 chromosomes. Its Hi–C pattern, chromosome clustering results, and one-to-one genome synteny across two subgroups indicates a tetraploid structure with chromosome count 2n = 4x = 20. Phylogenetic analysis suggests that E. vivipara diverged from Cyperus esculentus approximately 32.96 million years ago (Mya), and underwent a whole-genome duplication (WGD) about 3.5 Mya. Numerous fusion and fission events were identified between the chromosomes of E. vivipara and its close relatives. We demonstrate that E. vivipara has holocentromeres, a chromosomal feature which can maintain the stability of such chromosomal rearrangements. Experimental transplantation and cross-section studies showed its terrestrial culms developed C4 Kranz anatomy with increased number of chloroplasts in the bundle sheath (BS) cells. Gene expression and weighted gene co-expression network analysis (WGCNA) showed overall elevated expression of core genes associated with the C4 pathway, and significant enrichment of genes related to modified culm anatomy and photosynthesis efficiency. We found evidence of mixed nicotinamide adenine dinucleotide - malic enzyme and phosphoenolpyruvate carboxykinase type C4 photosynthesis in E. vivipara, and hypothesize that the evolution of C4 photosynthesis predates the WGD event. The mixed type is dominated by subgenome A and supplemented by subgenome B. Collectively, our findings not only shed light on the evolution of E. vivipara and karyotype within the Cyperaceae family, but also p

In the backdrop of global energy transformation, power systems integrating high proportions of renewable energy sources are facing unprecedented challenges in operational stability and dispatch efficiency. To address these challenges, this study introduces a generation-storage coordination real -time dispatch strategy based on Causal Power System Dynamic Reinforcement Learning (CPSDRL). Diverging from traditional reinforcement learning approaches, CPSDRL innovatively incorporates causal inference within the state prediction model - the crux of model -based reinforcement learning - thereby establishing the Power Causal Dynamic Model (PCDM). Assisted by the prior knowledge of power systems, the model significantly enhances prediction accuracy and reliability through a two-stage training process. Utilizing PCDM, this study further applies a direct policy search algorithm to optimize the real -time dispatch strategy. Experimental results indicate that the proposed method improves the stability of generation-storage coordination real -time dispatch and exhibits competitive advantages in sample efficiency and computational speed, compared to traditional model -based and modelfree reinforcement learning algorithms. This method is expected to enhance the practicality and adaptability of causal reinforcement learning techniques in power system scheduling and control.

To address the issues of magnetic parameter dependence and insufficient dynamic performance in high-speed sensorless control of switched reluctance motors (SRMs), this article proposes a sensorless control scheme based on the delayed signal cancellation flux observer (DSCFO) for high-speed SRM drives. This scheme improves the dynamic performance while achieving sensorless control, without the need for magnetic characteristics. Due to the superior filtering efficiency of the DSCFO, the proposed observer eliminates the non-linear terms in the calculated single-phase flux linkage and extracts a pair of orthogonal signals with positional information. Following this, the rotor position and speed information are estimated through the phase locked loop, enabling sensorless control. Finally, experiments are conducted on a three-phase 5.5 kW 12/8 SRM experimental setup to verify the effectiveness of the proposed scheme. Experimental results indicate that compared to other methods, the proposed approach can achieve sensorless control with only single-phase while possessing desirable accuracy and stronger robustness.

In this paper, a new axial-type magnetic gear with an auxiliary flux-enhancing structure (AFS-AMG) is proposed. Compared to conventional AMGs, it has a higher torque density and higher permanent magnet (PM) utilization factor. Firstly, the design rules and operating principles of the proposed AFS-AMG are elaborated. Then, the mapping relation between the radial-type magnetic gears (RMGs) and AMGs are elucidated. Compared to its counterparts in RMGs, the AFS-AMG achieves a small size. Then, the geometrical parameters of the AFS-AMG are optimized to obtain better electromagnetic performance, where the torque density per volume and per PM volume is adopted as the evaluation standard. Finally, three different AMG topologies are constructed in finite element analysis (FEA) software for comparison. It is proven that the AFS-AMG has the largest torque density per volume and per PM volume.

Existing model-free predictive control (MFPC) methods are primarily based on mature controller design formulas, with various observers added to these controllers. Currently, there is a lack of modifications to the mechanism of modelfree predictive control itself. This letter starts with the original formulation of the ultra-local model and redesigns a model-free predictive control method from the fundamental formula level. First, a term based on an exponential-like function is proposed, modifying the proportion of the controller's error term, called exponential-MFPC (EMFPC). Subsequently, a switching control concept is introduced into the controller design, facilitating better integration of the existing method with the new one, ensuring system performance across different error intervals. Ultimately, the effectiveness of the proposed switched ultra-local model-free predictive controller (SMPFC) is demonstrated by the experimental result of the superiority in the loading aspect with smaller speed overshoot and speed oscillations. IEEE

Parameter estimation of permanent magnet synchronous motors (PMSMs) is broadly researched, while the parameter observability has not been well-discussed. This article employs the local weak observability theory to obtain unified observable conditions and give a definite reference for parameter estimation. Based on the results, a novel machine parameter estimation algorithm based on convex optimization is proposed to verify these conditions. This scheme establishes an integrated cost function and solves it by the numerical method to estimate parameters. Then, with the proposed approach, several parameters are obtained together by one estimator, which avoids multiple observer designs and reduces the system's complexity. Moreover, convergency analyses of the cost function demonstrate that the states of PMSMs are locally weakly observable at the unified conditions. Finally, the stability of the observable conditions and the feasibility of the proposed method are verified on a 4-kW PMSM experimental setup. © 1986-2012 IEEE.

Lujie Wei,1,* Pingyang Li,2,* Xiaofeng Liu,1 Yuxia Wang,1 Zhengping Tang,1 Hang Zhao,1 Lu Yu,3 Kaixiu Li,1 Jianping Li,1 Min Du,4 Xinzhu Chen,2 Xin Zheng,2 Yixiong Zheng,5 Yao Luo,2 Jing Chen,5 Xiamin Jiang,6 Xiaobing Chen,7 Huaicong Long1 1Geriatric Intensive Care Unit, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, Sichuan, People’s Republic of China; 2Medical College, University of Electronic Science and Technology of China, Chengdu, Sichuan, People’s Republic of China; 3Department of Respiratory and Critical Care Medicine, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, Sichuan, People’s Republic of China; 4Department of Respiratory and Critical Care Medicine, Nanbu County People’s Hospital, Nanbu, Sichuan, People’s Republic of China; 5College of Medicine and Life Sciences, Chengdu University of Traditional Chinese Medicine, Chengdu, Sichuan, People’s Republic of China; 6Clinical College, North Sichuan Medical College, Nanchong, Sichuan, People’s Republic of China; 7Department of Respiratory and Critical Care Medicine, Affiliated Hospital of North Sichuan Medical College, Nanchong, Sichuan, People’s Republic of China*These authors contributed equally to this workCorrespondence: Huaicong Long, Geriatric Intensive Care Unit, Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, Sichuan, 610000, People’s Republic of China, Tel +86 18981838105, Fax +86-28-8779 5585, Email longhc69@163.comBackground: Chronic obstructive pulmonary disease (COPD) is closely associated with frailty, and prevention of acute exacerbations is important for disease management. Moreover, COPD patients with frailty experience a higher risk of acute exacerbations. However, the frailty instruments that can better predict acute exacerbations remain unclear.Purpose: (1) To explore the f

Autonomous vehicles are gradually entering city roads today, with the help of high-definition maps (HDMaps). However, the reliance on HDMaps prevents autonomous vehicles from stepping into regions without this expensive digital infrastructure. This fact drives many researchers to study online HDMap generation algorithms, but the performance of these algorithms at far regions is still unsatisfying. We present P-MapNet, in which the letter P highlights the fact that we focus on incorporating map priors to improve model performance. Specifically, we exploit priors in both SDMap and HDMap. On one hand, we extract weakly aligned SDMap from OpenStreetMap, and encode it as an additional conditioning branch. Despite the misalignment challenge, our attention-based architecture adaptively attends to relevant SDMap skeletons and significantly improves performance. On the other hand, we exploit a masked autoencoder to capture the prior distribution of HDMap, which can serve as a refinement module to mitigate occlusions and artifacts. We benchmark on the nuScenes and Argoverse2 datasets. Through comprehensive experiments, we show that: (1) our SDMap prior can improve online map generation performance, using both rasterized (by up to $+18.73$ $\rm mIoU$) and vectorized (by up to $+8.50$ $\rm mAP$) output representations. (2) our HDMap prior can improve map perceptual metrics by up to $6.34\%$. (3) P-MapNet can be switched into different inference modes that covers different regions of the accuracy-efficiency trade-off landscape. (4) P-MapNet is a far-seeing solution that brings larger improvements on longer ranges. Codes and models are publicly available at https://jike5.github.io/P-MapNet., Comment: Code: https://jike5.github.io/P-MapNet

Autonomous vehicles rely extensively on perception systems to navigate and interpret their surroundings. Despite significant advancements in these systems recently, challenges persist under conditions like occlusion, extreme lighting, or in unfamiliar urban areas. Unlike these systems, humans do not solely depend on immediate observations to perceive the environment. In navigating new cities, humans gradually develop a preliminary mental map to supplement real-time perception during subsequent visits. Inspired by this human approach, we introduce a novel framework, Pre-Sight, that leverages past traversals to construct static prior memories, enhancing online perception in later navigations. Our method involves optimizing a city-scale neural radiance field with data from previous journeys to generate neural priors. These priors, rich in semantic and geometric details, are derived without manual annotations and can seamlessly augment various state-of-the-art perception models, improving their efficacy with minimal additional computational cost. Experimental results on the nuScenes dataset demonstrate the framework's high compatibility with diverse online perception models. Specifically, it shows remarkable improvements in HD-map construction and occupancy prediction tasks, highlighting its potential as a new perception framework for autonomous driving systems. Our code will be released at https://github.com/yuantianyuan01/PreSight.

This paper proposes an analytical model of the double-stator spoke-type permanent magnet vernier machine (DSSTVM) using the subdomain method (SDM), which can be used to calculate the magnetic field distribution and corresponding electromagnetic parameters of the DSSTVM. The whole field domain is divided into several subdomains according to the magnetic characteristics of each region, within which Laplace's and Poisson's equations are solved accordingly in terms of magnetic vector potential (MVP). Then, the corresponding magnetic flux density distribution, back electromotive force (EMF), and electromagnetic torque of the DSSTVM can be obtained. Ultimately, finite element analysis (FEA) is adopted to validate the proposed analytical model's effectiveness for quickly predicting the no-load and on-load performances of the DSSTVM.

A primary hurdle of autonomous driving in urban environments is understanding complex and long-tail scenarios, such as challenging road conditions and delicate human behaviors. We introduce DriveVLM, an autonomous driving system leveraging Vision-Language Models (VLMs) for enhanced scene understanding and planning capabilities. DriveVLM integrates a unique combination of chain-of-thought (CoT) modules for scene description, scene analysis, and hierarchical planning. Furthermore, recognizing the limitations of VLMs in spatial reasoning and heavy computational requirements, we propose DriveVLM-Dual, a hybrid system that synergizes the strengths of DriveVLM with the traditional autonomous driving pipeline. DriveVLM-Dual achieves robust spatial understanding and real-time inference speed. Extensive experiments on both the nuScenes dataset and our SUP-AD dataset demonstrate the effectiveness of DriveVLM and the enhanced performance of DriveVLM-Dual, surpassing existing methods in complex and unpredictable driving conditions., Comment: Project Page: https://tsinghua-mars-lab.github.io/DriveVLM

In the realm of audio-language pre-training (ALP), the challenge of achieving cross-modal alignment is significant. Moreover, the integration of audio inputs with diverse distributions and task variations poses challenges in developing generic audio-language models. In this study, we introduce MINT, a novel ALP framework boosting audio-language models through multi-target pre-training and instruction tuning. MINT leverages the strength of frozen pre-trained audio encoders and large language models (LLMs) to improve audio-language pre-training, enabling effective transferablility to both audio-text understanding and generation tasks. To address the modality gap, we propose Bridge-Net, a lightweight trainable module that enhances cross-modality alignment and the model's ability to follow instructions for a variety of audio-text tasks. Bridge-Net is pivotal within MINT, initially enhancing audio-language representation learning through a multi-target pre-training approach. Subsequently, Bridge-Net further boosts audio-to-language generative learning by integrating a frozen language model with instruction tuning. This integration empowers MINT to extract features in a flexible and effective manner, specifically tailored to the provided instructions for diverse tasks. Experimental results demonstrate that MINT attains superior performance across various audio-language understanding and generation tasks, highlighting its robust generalization capabilities even in zero-shot scenarios.

This technical report introduces PIXART-{\delta}, a text-to-image synthesis framework that integrates the Latent Consistency Model (LCM) and ControlNet into the advanced PIXART-{\alpha} model. PIXART-{\alpha} is recognized for its ability to generate high-quality images of 1024px resolution through a remarkably efficient training process. The integration of LCM in PIXART-{\delta} significantly accelerates the inference speed, enabling the production of high-quality images in just 2-4 steps. Notably, PIXART-{\delta} achieves a breakthrough 0.5 seconds for generating 1024x1024 pixel images, marking a 7x improvement over the PIXART-{\alpha}. Additionally, PIXART-{\delta} is designed to be efficiently trainable on 32GB V100 GPUs within a single day. With its 8-bit inference capability (von Platen et al., 2023), PIXART-{\delta} can synthesize 1024px images within 8GB GPU memory constraints, greatly enhancing its usability and accessibility. Furthermore, incorporating a ControlNet-like module enables fine-grained control over text-to-image diffusion models. We introduce a novel ControlNet-Transformer architecture, specifically tailored for Transformers, achieving explicit controllability alongside high-quality image generation. As a state-of-the-art, open-source image generation model, PIXART-{\delta} offers a promising alternative to the Stable Diffusion family of models, contributing significantly to text-to-image synthesis., Comment: Technical Report

The field of time series forecasting has garnered significant attention in recent years, prompting the development of advanced models like TimeSieve, which demonstrates impressive performance. However, an analysis reveals certain unfaithfulness issues, including high sensitivity to random seeds and minute input noise perturbations. Recognizing these challenges, we embark on a quest to define the concept of \textbf{\underline{F}aithful \underline{T}ime\underline{S}ieve \underline{(FTS)}}, a model that consistently delivers reliable and robust predictions. To address these issues, we propose a novel framework aimed at identifying and rectifying unfaithfulness in TimeSieve. Our framework is designed to enhance the model's stability and resilience, ensuring that its outputs are less susceptible to the aforementioned factors. Experimentation validates the effectiveness of our proposed framework, demonstrating improved faithfulness in the model's behavior. Looking forward, we plan to expand our experimental scope to further validate and optimize our algorithm, ensuring comprehensive faithfulness across a wide range of scenarios. Ultimately, we aspire to make this framework can be applied to enhance the faithfulness of not just TimeSieve but also other state-of-the-art temporal methods, thereby contributing to the reliability and robustness of temporal modeling as a whole.

In optimizing performance and extending the lifespan of lithium batteries, accurate state prediction is pivotal. Traditional regression and classification methods have achieved some success in battery state prediction. However, the efficacy of these data-driven approaches heavily relies on the availability and quality of public datasets. Additionally, generating electrochemical data predominantly through battery experiments is a lengthy and costly process, making it challenging to acquire high-quality electrochemical data. This difficulty, coupled with data incompleteness, significantly impacts prediction accuracy. Addressing these challenges, this study introduces the End of Life (EOL) and Equivalent Cycle Life (ECL) as conditions for generative AI models. By integrating an embedding layer into the CVAE model, we developed the Refined Conditional Variational Autoencoder (RCVAE). Through preprocessing data into a quasi-video format, our study achieves an integrated synthesis of electrochemical data, including voltage, current, temperature, and charging capacity, which is then processed by the RCVAE model. Coupled with customized training and inference algorithms, this model can generate specific electrochemical data for EOL and ECL under supervised conditions. This method provides users with a comprehensive electrochemical dataset, pioneering a new research domain for the artificial synthesis of lithium battery data. Furthermore, based on the detailed synthetic data, various battery state indicators can be calculated, offering new perspectives and possibilities for lithium battery performance prediction.

Deploying large arrays of antennas for wireless communications, an idea referred to as massive MIMO, is seen as a critical technology for achieving the huge gains in spectral efficiency (SE) needed to meet the ever-increasing demand for wireless services. The main challenges of this technique are its high cost (in terms of both power consumption and hardware complexity) and achieving good performance of the transceiver elements, especially at mmWave frequencies.In order to reduce the cost of the mmWave receiver, a 28-GHz one-bit receiver element deploying wireless LO distribution is studied and fabricated using the TowerSemi 0.18um BiCMOS process. Unlike conventional MIMO structures, in this work both the RF and the broadcast single-ended local oscillator (LO) signal are received through the RF input port and directly converted to an IF signal by using a simple low-power square-law detector. Moreover, since the composite LO and RF signals share the same amplifying chain, additional LO buffers are not needed. The analysis of the noise performance and challenges for designing the receiver element are also presented.For the second work, a four-channel mmWave MIMO transceiver array, which operates at 28GHz for the 5G New Radio (n257). Each channel includes a transmitter (Tx) a receiver (Rx), and a LO phase shifter that combines the passive and active phase-shifting techniques. To achieve a flatter and wider bandwidth for both Rx and Tx, a novel passive matching network, based on a transformer, is analyzed and implemented on chip. The chip was fabricated in TowerSemi 0.18um BiCMOS process.The transimpedance amplifier (TIA) is another critical block following the frequency converter, which can be the bottleneck of the system’s linearity. With the increasing of communication data rates, the TIA is required to obtain a wider bandwidth with good linearity. As a result, it is important to model the high-frequency nonlinear behavior of the TIA. In the third work, the filtering

LiDAR localization and mapping systems are widely studied for practical applications such as building management, emergency response and unknown place exploration. Such systems usually use LiDAR odometry or simultaneous localization and mapping (SLAM) algorithms to continuously estimate the pose of the equipped devices such as the LiDAR scanner. These algorithms face several challenges: pose initialization, drift of the trajectory, and recovery from failure. To overcome these challenges, this thesis makes four important contributions. The first contribution of this thesis is MoLi-PoseNet, a novel LiDAR relocalization method using a surface-based 3D model. Unlike conventional LiDAR relocalization methods, where the LiDAR pose is estimated with respect to a map created by the same LiDAR, which may be inaccurate due to possible localization failures, the proposed approach takes advantage of a 3D model, which can be easily extracted from a BIM or created from floor plans. In MoLi-PoseNet, synthetic LiDAR scans are generated in the 3D model and then used to train a pose regression network. The trained network performs pose estimation on real LiDAR scans. Experimental evaluation of MoLi-PoseNet shows that MoLi-PoseNet can achieve meter-level relocalization accuracy in large indoor environments with low-level-of-detail models. The second contribution of this thesis is MoLO: a novel drift-free LiDAR odometry method. Current LiDAR odometry approaches suffer from drift in the estimated trajectory and these methods rely on loop closure and optimization to eliminate the drift. However, loops are not practical for the navigation tasks. In MoLO, the acquired LiDAR scans are registered with a 3D model to perform relocalization at a certain frequency and pose graph optimization is then implemented to optimize the trajectory. Experimental evaluation of MoLO shows that MoLO can eliminate drift and achieve real-time localization while providing an accuracy equivalent to loop closure o

Variable reluctance (VR) resolvers are universally employed as the absolute position sensor in motor control systems. In this article, a 4-pole 16-tooth VR resolver is proposed with the novel sinusoidal-reluctance rotor contour, and its operation feasibility is evaluated via the analytical investigation and finite element (FE) analysis. Considering the slot and air-gap flux leakage, the original third-order harmonic from the sinusoidal-reluctance rotor contour can be compensated to suppress the total harmonic distortion (THD) in the envelope. And the magnitude of the compensation third-order harmonic can be regulated by adjusting the tooth width. Under 4 mm tooth width, the THD of the cosine envelope with the proposed rotor contour can be limited to only 0.54%, and the angle error can also be restricted to only 0.012 degrees (0.72 arc min).

Huan Li, Chang Wang, Qing Wang, Xuehua Liu, Juanjuan Zhang, He Zhang, Wenjie Fei, Hang Zhao, Luping Ren Department of Endocrinology, Hebei General Hospital, Shijiazhuang, Hebei, People’s Republic of ChinaCorrespondence: Luping Ren, Email renluping1122@163.comObjective: Previous studies have shown that oxymatrine (OMT) can improve high-fat-high-fructose-diet-induced non-alcoholic fatty liver disease (NAFLD), and our study aimed to explore its possible metabolic potential mechanisms.Methods: Wistar rats were fed a high-fat-high-fructose diet for 8 weeks and treated with oxymatrine by gavage for the last 4 weeks. We measured biochemical indicators and pathological changes in each group and used liquid chromatography-mass spectrometry (LC-MS) to analyze changes in metabolites in the serum and liver of the rats.Results: The results showed that OMT can alleviate the high-fat-high-fructose-induced weight gain and hepatic lipid deposition in rats. Metabolomic analysis showed that the level of eicosapentaenoic acid (EPA) was downregulated and levels of desmosterol and d-galactose were upregulated in livers fed with HFDHFr. The levels of L-isoleucine, L-valine, arachidonic acid (AA), taurocholic acid (TCA), chenodeoxycholyltaurine (TDCA), isocitrate, and glutathione (GSH) were downregulated in the liver, whereas those of linoleic acid (LA), phosphocholine (PC), glycerophosphocholine (GPC), and oxidized glutathione (GSSG) were upregulated in the serum treated with OMT.Conclusion: In summary, OMT can improve HFDHFr-induced NAFLD, and metabolomic analysis can provide an early warning for the development of NAFLD as well as provide a rationale for therapeutic targets.Keywords: high-fat-high, fructose, fatty liver, oxymatrine, metabonomics

This paper proposes a new topology of a high-temperature superconducting linear flux switching motor (HTS-LFSM). The electromagnetic characteristics of this novel HTS-LFSM, including flux density, back-EMF, thrust force, and ripple, are analyzed. Key topology parameters of the HTS-LFSM are optimized, and its performance is compared with that of the traditional LFSPM. The comparison shows that the proposed HTS-LFSM can generate higher thrust force and smaller thrust ripple. The results indicate that the proposed machine offers more performance advantages for modern traffic. © 2023 IEEE.

With the rapid development of computer science, neural network (NN) has received widespread attention. However, it is mainly used as a black box in identification and prediction, which is highly complicated. Its explainability should be mentioned when it is used in real applications. So, this paper derives the explainable control scheme to make sure its theoretical rigorousness. Then a straightforward explanation of the neural network controller using a block diagram is given to help researchers to reproduce the method. Since the detailed design of the neural network should concern the principle of permanent magnet synchronous motor (PMSM), this analyses the real PMSM system and finds the control performance characteristic, especially in spontaneous disturbances. Unlike the black-box neural network, the recurrent radial basis function neural network (RRBF) controller method is proposed, guided by overshoot. Because various PMSMs have various performances of speed changing, three key functions of RRBF are proposed to check which variant is the most suitable for reducing the overshoot. Finally, the RRBF method is implemented in STM32, and its effectiveness in the experiment is checked. © 2023 IEEE.

Tiantian Sun,1,&ast; Chang Wang,1,2,&ast; Lijing Huo,3 Yichao Wang,1 Ke Liu,1 Changmei Wei,3 Hang Zhao,1 Shuchun Chen,1 Luping Ren1 1Department of Endocrinology, Hebei General Hospital, Shijiazhuang, People’s Republic of China; 2North China University of Science and Technology, Tangshan, People’s Republic of China; 3Department of Clinical Laboratory, Hebei General Hospital, Shijiazhuang, People’s Republic of China&ast;These authors contributed equally to this workCorrespondence: Luping Ren, No. 348 Heping West Road, Shijiazhuang, Hebei Province, People’s Republic of China, Tel +86 0311-85988975, Email renluping1122@163.comPurpose: To evaluate serum cortistatin (CST) levels in type 2 diabetes mellitus (T2DM) patients with or without non-alcoholic fatty liver disease (NAFLD) and to examine the relationship between CST and NAFLD.Methods: A total of 90 T2DM patients, which included 56 NAFLD patients (referred to as DM+NAFLD group) and 34 patients without NAFLD (DM-only group), and 83 non-diabetes individuals that included 39 NAFLD patients (NAFLD-only group) and 44 without NAFLD that acted as the normal-control group (NC group). The differences in the serum CST levels between the groups were compared, and the correlations between CST and other variables were calculated by applying both correlational analysis and multiple linear regression analysis.Results: The mean serum CST levels were significantly lower in the DM+NAFLD and DM groups than in the NC group (P < 0.05). In addition, the CST levels were lower in the DM group relative to that in the NAFLD group (P < 0.05). However, no statistical difference was noted in the serum CST between diabetic patients with and without NAFLD (P > 0.05). Similarly, in the non-diabetic group, the serum CST level was not significantly different between individuals with and without NAFLD (P > 0.05). Furthermore, the serum CST levels were negatively associated with the levels of total cholesterol (TC), low

Large language models (LLMs) with memory are computationally universal. However, mainstream LLMs are not taking full advantage of memory, and the designs are heavily influenced by biological brains. Due to their approximate nature and proneness to the accumulation of errors, conventional neural memory mechanisms cannot support LLMs to simulate complex reasoning. In this paper, we seek inspiration from modern computer architectures to augment LLMs with symbolic memory for complex multi-hop reasoning. Such a symbolic memory framework is instantiated as an LLM and a set of SQL databases, where the LLM generates SQL instructions to manipulate the SQL databases. We validate the effectiveness of the proposed memory framework on a synthetic dataset requiring complex reasoning. The project website is available at https://chatdatabase.github.io/ .

This paper tries to address a fundamental question in point cloud self-supervised learning: what is a good signal we should leverage to learn features from point clouds without annotations? To answer that, we introduce a point cloud representation learning framework, based on geometric feature reconstruction. In contrast to recent papers that directly adopt masked autoencoder (MAE) and only predict original coordinates or occupancy from masked point clouds, our method revisits differences between images and point clouds and identifies three self-supervised learning objectives peculiar to point clouds, namely centroid prediction, normal estimation, and curvature prediction. Combined with occupancy prediction, these four objectives yield an nontrivial self-supervised learning task and mutually facilitate models to better reason fine-grained geometry of point clouds. Our pipeline is conceptually simple and it consists of two major steps: first, it randomly masks out groups of points, followed by a Transformer-based point cloud encoder; second, a lightweight Transformer decoder predicts centroid, normal, and curvature for points in each voxel. We transfer the pre-trained Transformer encoder to a downstream peception model. On the nuScene Datset, our model achieves 3.38 mAP improvment for object detection, 2.1 mIoU gain for segmentation, and 1.7 AMOTA gain for multi-object tracking. We also conduct experiments on the Waymo Open Dataset and achieve significant performance improvements over baselines as well., Comment: Accepted to CVPR 2023

We abstract the features (i.e. learned representations) of multi-modal data into 1) uni-modal features, which can be learned from uni-modal training, and 2) paired features, which can only be learned from cross-modal interactions. Multi-modal models are expected to benefit from cross-modal interactions on the basis of ensuring uni-modal feature learning. However, recent supervised multi-modal late-fusion training approaches still suffer from insufficient learning of uni-modal features on each modality. We prove that this phenomenon does hurt the model's generalization ability. To this end, we propose to choose a targeted late-fusion learning method for the given supervised multi-modal task from Uni-Modal Ensemble(UME) and the proposed Uni-Modal Teacher(UMT), according to the distribution of uni-modal and paired features. We demonstrate that, under a simple guiding strategy, we can achieve comparable results to other complex late-fusion or intermediate-fusion methods on various multi-modal datasets, including VGG-Sound, Kinetics-400, UCF101, and ModelNet40.

Inferring past human motion from RGB images is challenging due to the inherent uncertainty of the prediction problem. Thermal images, on the other hand, encode traces of past human-object interactions left in the environment via thermal radiation measurement. Based on this observation, we collect the first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. Then we develop a three-stage neural network model for accurate past human pose estimation. Comprehensive experiments show that thermal cues significantly reduce the ambiguities of this task, and the proposed model achieves remarkable performance. The dataset is available at https://github.com/ZitianTang/Thermal-IM.

Robots operating in the real world require both rich manipulation skills as well as the ability to semantically reason about when to apply those skills. Towards this goal, recent works have integrated semantic representations from large-scale pretrained vision-language (VL) models into manipulation models, imparting them with more general reasoning capabilities. However, we show that the conventional pretraining-finetuning pipeline for integrating such representations entangles the learning of domain-specific action information and domain-general visual information, leading to less data-efficient training and poor generalization to unseen objects and tasks. To this end, we propose ProgramPort, a modular approach to better leverage pretrained VL models by exploiting the syntactic and semantic structures of language instructions. Our framework uses a semantic parser to recover an executable program, composed of functional modules grounded on vision and action across different modalities. Each functional module is realized as a combination of deterministic computation and learnable neural networks. Program execution produces parameters to general manipulation primitives for a robotic end-effector. The entire modular network can be trained with end-to-end imitation learning objectives. Experiments show that our model successfully disentangles action and perception, translating to improved zero-shot and compositional generalization in a variety of manipulation behaviors. Project webpage at: \url{https://progport.github.io}., Comment: ICLR 2023 camera-ready

Robotic perception requires the modeling of both 3D geometry and semantics. Existing methods typically focus on estimating 3D bounding boxes, neglecting finer geometric details and struggling to handle general, out-of-vocabulary objects. 3D occupancy prediction, which estimates the detailed occupancy states and semantics of a scene, is an emerging task to overcome these limitations. To support 3D occupancy prediction, we develop a label generation pipeline that produces dense, visibility-aware labels for any given scene. This pipeline comprises three stages: voxel densification, occlusion reasoning, and image-guided voxel refinement. We establish two benchmarks, derived from the Waymo Open Dataset and the nuScenes Dataset, namely Occ3D-Waymo and Occ3D-nuScenes benchmarks. Furthermore, we provide an extensive analysis of the proposed dataset with various baseline models. Lastly, we propose a new model, dubbed Coarse-to-Fine Occupancy (CTF-Occ) network, which demonstrates superior performance on the Occ3D benchmarks. The code, data, and benchmarks are released at https://tsinghua-mars-lab.github.io/Occ3D/., Comment: Accepted to NeurIPS 2023

High-definition (HD) semantic maps are crucial in enabling autonomous vehicles to navigate urban environments. The traditional method of creating offline HD maps involves labor-intensive manual annotation processes, which are not only costly but also insufficient for timely updates. Recent studies have proposed an alternative approach that generates local maps using online sensor observations. However, this approach is limited by the sensor's perception range and its susceptibility to occlusions. In this study, we propose Neural Map Prior (NMP), a neural representation of global maps. This representation automatically updates itself and improves the performance of local map inference. Specifically, we utilize two approaches to achieve this. Firstly, to integrate a strong map prior into local map inference, we apply cross-attention, a mechanism that dynamically identifies correlations between current and prior features. Secondly, to update the global neural map prior, we utilize a learning-based fusion module that guides the network in fusing features from previous traversals. Our experimental results, based on the nuScenes dataset, demonstrate that our framework is highly compatible with various map segmentation and detection architectures. It significantly improves map prediction performance, even in challenging weather conditions and situations with a longer perception range. To the best of our knowledge, this is the first learning-based system for creating a global map prior., Comment: CVPR 2023 Camera Ready

High-resolution images enable neural networks to learn richer visual representations. However, this improved performance comes at the cost of growing computational complexity, hindering their usage in latency-sensitive applications. As not all pixels are equal, skipping computations for less-important regions offers a simple and effective measure to reduce the computation. This, however, is hard to be translated into actual speedup for CNNs since it breaks the regularity of the dense convolution workload. In this paper, we introduce SparseViT that revisits activation sparsity for recent window-based vision transformers (ViTs). As window attentions are naturally batched over blocks, actual speedup with window activation pruning becomes possible: i.e., ~50% latency reduction with 60% sparsity. Different layers should be assigned with different pruning ratios due to their diverse sensitivities and computational costs. We introduce sparsity-aware adaptation and apply the evolutionary search to efficiently find the optimal layerwise sparsity configuration within the vast search space. SparseViT achieves speedups of 1.5x, 1.4x, and 1.3x compared to its dense counterpart in monocular 3D object detection, 2D instance segmentation, and 2D semantic segmentation, respectively, with negligible to no loss of accuracy., Comment: CVPR 2023. The first two authors contributed equally to this work. Project page: https://sparsevit.mit.edu

Yu Luo,1 Cuiyu Wang,1,2 Tian Zhang,1,3 Xiaoyu He,1,4 Jianan Hao,1,4 Andong Shen,3,5 Hang Zhao,1 Shuchun Chen,1 Luping Ren1 1Endocrinology Department, Hebei General Hospital, Shijiazhuang, People’s Republic of China; 2Graduate School, Hebei North University, Zhangjiakou, People’s Republic of China; 3Graduate School, North China University of Science and Technology, Tangshan, People’s Republic of China; 4Graduate School, Hebei Medical University, Shijiazhuang, People’s Republic of China; 5Gastroenterology Department, Hebei General Hospital, Shijiazhuang, People’s Republic of ChinaCorrespondence: Luping Ren, Email renluping1122@163.comPurpose: Non-alcoholic fatty liver disease (NAFLD) and type 2 diabetes mellitus (T2DM) are frequently co-occurring diseases. Liver fibrosis (LF), with increasing incidence, has a prognostic value for NAFLD mortality. Our study aimed to investigate the relevant factors for FL in T2DM individuals with NAFLD.Patients and Methods: A total of 565 T2DM patients with NAFLD from Hebei General Hospital participated in the study. Patients underwent an abdominal ultrasound, a questionnaire and laboratory tests. The fibrosis-4 index (FIB-4) was used to evaluate LF, with FIB ≥ 1.3 indicating LF and FIB ≥ 2.67 indicating F3-4 fibrosis.Results: Compared with NLF group, LF group had higher levels of systolic blood pressure (SBP), alanine aminotransferase (ALT), aspartate aminotransferase (AST), and γ-glutamyl transpeptidase (GGT). The glomerular filtration rate (GFR), low-density lipoprotein cholesterol (LDL), glycated hemoglobin (HbA1c), and platelets (PLT) in LF patients were lower than those without LF. Patients with LF were older than those without LF. ALT, AST, and GGT in patients with severe LF were higher than those with mild LF, while platelet was lower. Age, SBP, duration of diabetes, ALT, AST, and GGT were positively correlated with FIB-4, while eGFR, TC, LDL, and HbA1c were negatively correlated with FIB-4. Logistic regression s

Existing sensitivity analysis methods suffer from issues such as small differentiation in parameter sensitivity and slow computational speed. To solve these problems, three machine learning methods, namely Ridge regression, Lasso regression, and Elastic Net, are investigated in this paper. First, a PMSM's performance is analyzed using Finite Element Method (FEM) to identify the variation range of the structural parameters that require optimization. Following that, a finite element sample database is established based on this range using the Design of Experiment (DOE). Next, the variable selection results of PMSM optimization are compared using different machine learning methods and traditional machine sensitivity analysis method. Finally, the performances of the PMSM optimized based on the three machine learning methods are evaluated. The results indicates that Lasso Regression proves to be the most effective method. It achieves rapid convergence and effectively identifies the most sensitive parameter, enhancing the optimization efficiency of PMSMs while retaining the global optimum. © 2023 IEEE.

The derivative of the control increment concerning the output of the neural network (NN) stands as a pivotal factor within the NN-assisted control tuning approach for permanent magnet synchronous motors (PMSMs). However, the significance of selecting an appropriate formula lacks explanation. Through an analysis of gradient-based optimization methods, the underlying principle behind the update of control increments becomes evident. Consequently, the design of the control tuning scheme incorporates gradient descent with and without momentum, as well as accelerated momentum. These techniques are applied to formulate the update law for the backpropagation neural network (BPNN) assisted control. Subsequently, these devised methodologies are integrated into the BPNN framework for the purpose of comparison. Finally, the results are validated in experiments. © 2023 IEEE.

The growing demand for accurate control in varying and unknown environments has sparked a corresponding increase in the requirements for power supply components, including permanent magnet synchronous motors (PMSMs). To infer the unknown part of the system, machine learning techniques are widely employed, especially Gaussian process regression (GPR) due to its flexibility of continuous system modeling and its guaranteed performance. For practical implementation, distributed GPR is adopted to alleviate the high computational complexity. However, the study of distributed GPR from a control perspective remains an open problem. In this paper, a control-aware optimal aggregation strategy of distributed GPR for PMSMs is proposed based on the Lyapunov stability theory. This strategy exclusively leverages the posterior mean, thereby obviating the need for computationally intensive calculations associated with posterior variance in alternative approaches. Moreover, the straightforward calculation process of our proposed strategy lends itself to seamless implementation in high-frequency PMSM control. The effectiveness of the proposed strategy is demonstrated in the simulations. © 2023 IEEE.

This paper focuses on the calculation of different kinds of mechanical loss and put forward a complete method to calculate the mechanical loss for high-speed PMSM accurately. Windage loss, bearing loss, and cooling fan loss are calculated concretely, and the flow regime of the airgap is analyzed for windage loss calculation. Then, the equivalent thermal conductivity of the airgap in the turbulent regime is calculated, and it turned out to be much larger than that of stationary air, which cannot be ignored in high-fidelity modeling. Later, the two-way electromagnetic-thermal coupling method is applied in the multiphysics design. The results show the flow regime of the airgap cannot be neglected in windage loss calculation, and the mechanical loss calculation method put forward in this paper is more complete than previous research. Therefore, it can predict the electromagnetic-thermal performance of high-speed PMSM more accurately. © 2023 IEEE.

The speed difference among underwater motors can induce either strong or weak coupling effects among different control loops within six-degree-of-freedom (6-DOF) autonomous underwater vehicles (AUVs). However, this issue is rarely accorded specialized attention. This paper utilizes the impact of the yaw dimension on roll angle control as an example, and proposes a novel coupling control method. This method hinges on collaborative action among multiple motors to mitigate the amplitude of speed difference fluctuations while introducing a coupling compensator in the feedforward loop. The efficacy of this method in enhancing the stability and robustness of motion control, as well as its superiority over conventional decoupling control methods, are substantiated through both simulation and experimentation. © 2023 IEEE.

Latent Consistency Models (LCMs) have achieved impressive performance in accelerating text-to-image generative tasks, producing high-quality images with minimal inference steps. LCMs are distilled from pre-trained latent diffusion models (LDMs), requiring only ~32 A100 GPU training hours. This report further extends LCMs' potential in two aspects: First, by applying LoRA distillation to Stable-Diffusion models including SD-V1.5, SSD-1B, and SDXL, we have expanded LCM's scope to larger models with significantly less memory consumption, achieving superior image generation quality. Second, we identify the LoRA parameters obtained through LCM distillation as a universal Stable-Diffusion acceleration module, named LCM-LoRA. LCM-LoRA can be directly plugged into various Stable-Diffusion fine-tuned models or LoRAs without training, thus representing a universally applicable accelerator for diverse image generation tasks. Compared with previous numerical PF-ODE solvers such as DDIM, DPM-Solver, LCM-LoRA can be viewed as a plug-in neural PF-ODE solver that possesses strong generalization abilities. Project page: https://github.com/luosiallen/latent-consistency-model., Comment: Technical Report

Motion prediction and planning are vital tasks in autonomous driving, and recent efforts have shifted to machine learning-based approaches. The challenges include understanding diverse road topologies, reasoning traffic dynamics over a long time horizon, interpreting heterogeneous behaviors, and generating policies in a large continuous state space. Inspired by the success of large language models in addressing similar complexities through model scaling, we introduce a scalable trajectory model called State Transformer (STR). STR reformulates the motion prediction and motion planning problems by arranging observations, states, and actions into one unified sequence modeling task. Our approach unites trajectory generation problems with other sequence modeling problems, powering rapid iterations with breakthroughs in neighbor domains such as language modeling. Remarkably, experimental results reveal that large trajectory models (LTMs), such as STR, adhere to the scaling laws by presenting outstanding adaptability and learning efficiency. Qualitative results further demonstrate that LTMs are capable of making plausible predictions in scenarios that diverge significantly from the training data distribution. LTMs also learn to make complex reasonings for long-term planning, without explicit loss designs or costly high-level annotations.

Scene completion and forecasting are two popular perception problems in research for mobile agents like autonomous vehicles. Existing approaches treat the two problems in isolation, resulting in a separate perception of the two aspects. In this paper, we introduce a novel LiDAR perception task of Occupancy Completion and Forecasting (OCF) in the context of autonomous driving to unify these aspects into a cohesive framework. This task requires new algorithms to address three challenges altogether: (1) sparse-to-dense reconstruction, (2) partial-to-complete hallucination, and (3) 3D-to-4D prediction. To enable supervision and evaluation, we curate a large-scale dataset termed OCFBench from public autonomous driving datasets. We analyze the performance of closely related existing baseline models and our own ones on our dataset. We envision that this research will inspire and call for further investigation in this evolving and crucial area of 4D perception. Our code for data curation and baseline implementation is available at https://github.com/ai4ce/Occ4cast.

This paper investigates how to better leverage large-scale pre-trained uni-modal models to further enhance discriminative multi-modal learning. Even when fine-tuned with only uni-modal data, these models can outperform previous multi-modal models in certain tasks. It's clear that their incorporation into multi-modal learning would significantly improve performance. However, multi-modal learning with these models still suffers from insufficient learning of uni-modal features, which weakens the resulting multi-modal model's generalization ability. While fine-tuning uni-modal models separately and then aggregating their predictions is straightforward, it doesn't allow for adequate adaptation between modalities, also leading to sub-optimal results. To this end, we introduce Multi-Modal Low-Rank Adaptation learning (MMLoRA). By freezing the weights of uni-modal fine-tuned models, adding extra trainable rank decomposition matrices to them, and subsequently performing multi-modal joint training, our method enhances adaptation between modalities and boosts overall performance. We demonstrate the effectiveness of MMLoRA on three dataset categories: audio-visual (e.g., AVE, Kinetics-Sound, CREMA-D), vision-language (e.g., MM-IMDB, UPMC Food101), and RGB-Optical Flow (UCF101).

Imitation learning (IL) enables agents to mimic expert behaviors. Most previous IL techniques focus on precisely imitating one policy through mass demonstrations. However, in many applications, what humans require is the ability to perform various tasks directly through a few demonstrations of corresponding tasks, where the agent would meet many unexpected changes when deployed. In this scenario, the agent is expected to not only imitate the demonstration but also adapt to unforeseen environmental changes. This motivates us to propose a new topic called imitator learning (ItorL), which aims to derive an imitator module that can on-the-fly reconstruct the imitation policies based on very limited expert demonstrations for different unseen tasks, without any extra adjustment. In this work, we focus on imitator learning based on only one expert demonstration. To solve ItorL, we propose Demo-Attention Actor-Critic (DAAC), which integrates IL into a reinforcement-learning paradigm that can regularize policies' behaviors in unexpected situations. Besides, for autonomous imitation policy building, we design a demonstration-based attention architecture for imitator policy that can effectively output imitated actions by adaptively tracing the suitable states in demonstrations. We develop a new navigation benchmark and a robot environment for \topic~and show that DAAC~outperforms previous imitation methods \textit{with large margins} both on seen and unseen tasks.

With the growing success of multi-modal learning, research on the robustness of multi-modal models, especially when facing situations with missing modalities, is receiving increased attention. Nevertheless, previous studies in this domain exhibit certain limitations, as they often lack theoretical insights or their methodologies are tied to specific network architectures or modalities. We model the scenarios of multi-modal models encountering missing modalities from an information-theoretic perspective and illustrate that the performance ceiling in such scenarios can be approached by efficiently utilizing the information inherent in non-missing modalities. In practice, there are two key aspects: (1) The encoder should be able to extract sufficiently good features from the non-missing modality; (2) The extracted features should be robust enough not to be influenced by noise during the fusion process across modalities. To this end, we introduce Uni-Modal Ensemble with Missing Modality Adaptation (UME-MMA). UME-MMA employs uni-modal pre-trained weights for the multi-modal model to enhance feature extraction and utilizes missing modality data augmentation techniques to better adapt to situations with missing modalities. Apart from that, UME-MMA, built on a late-fusion learning framework, allows for the plug-and-play use of various encoders, making it suitable for a wide range of modalities and enabling seamless integration of large-scale pre-trained encoders to further enhance performance. And we demonstrate UME-MMA's effectiveness in audio-visual datasets~(e.g., AV-MNIST, Kinetics-Sound, AVE) and vision-language datasets~(e.g., MM-IMDB, UPMC Food101).

Latent Diffusion models (LDMs) have achieved remarkable results in synthesizing high-resolution images. However, the iterative sampling process is computationally intensive and leads to slow generation. Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al). Viewing the guided reverse diffusion process as solving an augmented probability flow ODE (PF-ODE), LCMs are designed to directly predict the solution of such ODE in latent space, mitigating the need for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently distilled from pre-trained classifier-free guided diffusion models, a high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training. Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets. Evaluation on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve state-of-the-art text-to-image generation performance with few-step inference. Project Page: https://latent-consistency-models.github.io

We present a simple yet effective approach that can transform the OpenAI GPT-3.5 model into a reliable motion planner for autonomous vehicles. Motion planning is a core challenge in autonomous driving, aiming to plan a driving trajectory that is safe and comfortable. Existing motion planners predominantly leverage heuristic methods to forecast driving trajectories, yet these approaches demonstrate insufficient generalization capabilities in the face of novel and unseen driving scenarios. In this paper, we propose a novel approach to motion planning that capitalizes on the strong reasoning capabilities and generalization potential inherent to Large Language Models (LLMs). The fundamental insight of our approach is the reformulation of motion planning as a language modeling problem, a perspective not previously explored. Specifically, we represent the planner inputs and outputs as language tokens, and leverage the LLM to generate driving trajectories through a language description of coordinate positions. Furthermore, we propose a novel prompting-reasoning-finetuning strategy to stimulate the numerical reasoning potential of the LLM. With this strategy, the LLM can describe highly precise trajectory coordinates and also its internal decision-making process in natural language. We evaluate our approach on the large-scale nuScenes dataset, and extensive experiments substantiate the effectiveness, generalization ability, and interpretability of our GPT-based motion planner. Code is now available at https://github.com/PointsCoder/GPT-Driver., Comment: NeurIPS 2023 Foundation Models for Decision Making Workshop

Offline Reinforcement Learning (RL) has emerged as a promising framework for learning policies without active interactions, making it especially appealing for autonomous driving tasks. Recent successes of Transformers inspire casting offline RL as sequence modeling, which performs well in long-horizon tasks. However, they are overly optimistic in stochastic environments with incorrect assumptions that the same goal can be consistently achieved by identical actions. In this paper, we introduce an UNcertainty-awaRE deciSion Transformer (UNREST) for planning in stochastic driving environments without introducing additional transition or complex generative models. Specifically, UNREST estimates state uncertainties by the conditional mutual information between transitions and returns, and segments sequences accordingly. Discovering the `uncertainty accumulation' and `temporal locality' properties of driving environments, UNREST replaces the global returns in decision transformers with less uncertain truncated returns, to learn from true outcomes of agent actions rather than environment transitions. We also dynamically evaluate environmental uncertainty during inference for cautious planning. Extensive experimental results demonstrate UNREST's superior performance in various driving scenarios and the power of our uncertainty estimation strategy.

City modeling and generation have attracted an increased interest in various applications, including gaming, urban planning, and autonomous driving. Unlike previous works focused on the generation of single objects or indoor scenes, the huge volumes of spatial data in cities pose a challenge to the generative models. Furthermore, few publicly available 3D real-world city datasets also hinder the development of methods for city generation. In this paper, we first collect over 3,000,000 geo-referenced objects for the city of New York, Zurich, Tokyo, Berlin, Boston and several other large cities. Based on this dataset, we propose AETree, a tree-structured auto-encoder neural network, for city generation. Specifically, we first propose a novel Spatial-Geometric Distance (SGD) metric to measure the similarity between building layouts and then construct a binary tree over the raw geometric data of building based on the SGD metric. Next, we present a tree-structured network whose encoder learns to extract and merge spatial information from bottom-up iteratively. The resulting global representation is reversely decoded for reconstruction or generation. To address the issue of long-dependency as the level of the tree increases, a Long Short-Term Memory (LSTM) Cell is employed as a basic network element of the proposed AETree. Moreover, we introduce a novel metric, Overlapping Area Ratio (OAR), to quantitatively evaluate the generation results. Experiments on the collected dataset demonstrate the effectiveness of the proposed model on 2D and 3D city generation. Furthermore, the latent features learned by AETree can serve downstream urban planning applications.

Learning-based vehicle planning is receiving increasing attention with the emergence of diverse driving simulators and large-scale driving datasets. While offline reinforcement learning (RL) is well suited for these safety-critical tasks, it still struggles to plan over extended periods. In this work, we present a skill-based framework that enhances offline RL to overcome the long-horizon vehicle planning challenge. Specifically, we design a variational autoencoder (VAE) to learn skills from offline demonstrations. To mitigate posterior collapse of common VAEs, we introduce a two-branch sequence encoder to capture both discrete options and continuous variations of the complex driving skills. The final policy treats learned skills as actions and can be trained by any off-the-shelf offline RL algorithms. This facilitates a shift in focus from per-step actions to temporally extended skills, thereby enabling long-term reasoning into the future. Extensive results on CARLA prove that our model consistently outperforms strong baselines at both training and new scenarios. Additional visualizations and experiments demonstrate the interpretability and transferability of extracted skills.

Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly in complex environments. Existing methods can generate either diverse but blind locomotion skills or vision-based but specialized skills by using reference animal data or complex rewards. However, autonomous parkour requires robots to learn generalizable skills that are both vision-based and diverse to perceive and react to various scenarios. In this work, we propose a system for learning a single end-to-end vision-based parkour policy of diverse parkour skills using a simple reward without any reference motion data. We develop a reinforcement learning method inspired by direct collocation to generate parkour skills, including climbing over high obstacles, leaping over large gaps, crawling beneath low barriers, squeezing through thin slits, and running. We distill these skills into a single vision-based parkour policy and transfer it to a quadrupedal robot using its egocentric depth camera. We demonstrate that our system can empower two different low-cost robots to autonomously select and execute appropriate parkour skills to traverse challenging real-world environments., Comment: CoRL 2023 (Oral). Project website at https://robot-parkour.github.io

High-Definition (HD) maps are essential for the safety of autonomous driving systems. While existing techniques employ camera images and onboard sensors to generate vectorized high-precision maps, they are constrained by their reliance on single-frame input. This approach limits their stability and performance in complex scenarios such as occlusions, largely due to the absence of temporal information. Moreover, their performance diminishes when applied to broader perception ranges. In this paper, we present StreamMapNet, a novel online mapping pipeline adept at long-sequence temporal modeling of videos. StreamMapNet employs multi-point attention and temporal information which empowers the construction of large-range local HD maps with high stability and further addresses the limitations of existing methods. Furthermore, we critically examine widely used online HD Map construction benchmark and datasets, Argoverse2 and nuScenes, revealing significant bias in the existing evaluation protocols. We propose to resplit the benchmarks according to geographical spans, promoting fair and precise evaluations. Experimental results validate that StreamMapNet significantly outperforms existing methods across all settings while maintaining an online inference speed of $14.2$ FPS. Our code is available at https://github.com/yuantianyuan01/StreamMapNet.