Search

Autism spectrum disorder (ASD) is a developmental disorder characterized by symptoms: intellectual difficulties, motor, and language impairment, altered sensory processing, social impediments, and repetitive behaviors. Only 10-20% ASD have clear genetic causes, and fragile X syndrome (FXS) is one of them. FXS is caused by a lack of fragile X messenger ribonucleoprotein (FMRP) whose expression is transcriptionally silenced due to Frm1 gene hypermethylation. Altered sensory processing across all domains has been widely described in humans with FXS. Most noticeable alterations in auditory processing are auditory hyperactivity and impaired temporal processing. The Fmr1 knockout (KO) mouse is a well characterized animal model for FXS studies which also shows signs of auditory hypersensitivity and impaired temporal processing: increased evoked and induced auditory responses, susceptible to audiogenic seizures (AGS), and reduced phase-locking to temporally modulated sound. The consistent manifestations of auditory hyperactivity and impaired temporal processing between human FXS individuals and Frm1 KO make it possible to evaluate potential FXS treatment with Frm1 KO mice. Previous studies by others implied enhancing serotonin signaling ameliorated auditory hyperactivity, yet they failed to pinpoint the specific serotonin receptor subtypes involved. With a more specific and highly selective post-synaptic serotonin-1A (5-HT1A) receptor agonist, NLX-101, we found the AGS susceptibility in developing Frm1 KO mice is significantly reduced, suggesting enhancing 5-HT1A signaling is effective in reducing auditory hyperactivity at the behavioral level in Frm1 KO mice. Electroencephalography (EEG) data acquired from Frm1 KO mice following NLX-101 treatment showed that enhancing 5-HT1A signaling not only reduced auditory hyperactivity at the network level but also improved temporal processing by increasing consistency of the auditory responses to temporally modulated stimuli. RNAscop

Herein, a series of water-soluble supramolecular inclusion complexes (ICs) probes were prepared using cyclodextrins (CDs) and fraxetin (FRA) to detect nicotine (NT) with high selectivity in vitro and in vivo. The FRA/CD ICs prepared through the saturated solution method exhibited excellent water solubility, stability, and biocompatibility. A clear host-guest inclusion model was provided by the theoretical calculations. The investigation revealed that NT was able to enter into the cavities of FRA/beta-CD IC and FRA/gamma-CD IC, and further formed charge transfer complexes with FRA in the CD cavities, resulting in a rapid and highly selective fluorescenceenhanced response with the lowest detection limits of 1.9 x 10-6 M and 9.7 x 10- 7 M, and the linear response ranged from 0.02 to 0.3 mM and 0.01-0.05 mM, respectively. The IC probes showed good antiinterference performance to common interferents or different pH environments, with satisfactory reproducibility and repeatability of response to NT. Furthermore, the potentiality of the probes was confirmed through fluorescence imaging experiments using human lung cancer cells and the lung tissue of mice. This study offers a fresh perspective for detecting NT in environmental and biomedical analysis.

Diffusion models have demonstrated great success in text-to-video (T2V) generation. However, existing methods may face challenges when handling complex (long) video generation scenarios that involve multiple objects or dynamic changes in object numbers. To address these limitations, we propose VideoTetris, a novel framework that enables compositional T2V generation. Specifically, we propose spatio-temporal compositional diffusion to precisely follow complex textual semantics by manipulating and composing the attention maps of denoising networks spatially and temporally. Moreover, we propose an enhanced video data preprocessing to enhance the training data regarding motion dynamics and prompt understanding, equipped with a new reference frame attention mechanism to improve the consistency of auto-regressive video generation. Extensive experiments demonstrate that our VideoTetris achieves impressive qualitative and quantitative results in compositional T2V generation. Code is available at: https://github.com/YangLing0818/VideoTetris, Comment: Code: https://github.com/YangLing0818/VideoTetris

Recent advancements in text-to-image generation have been propelled by the development of diffusion models and multi-modality learning. However, since text is typically represented sequentially in these models, it often falls short in providing accurate contextualization and structural control. So the generated images do not consistently align with human expectations, especially in complex scenarios involving multiple objects and relationships. In this paper, we introduce the Scene Graph Adapter(SG-Adapter), leveraging the structured representation of scene graphs to rectify inaccuracies in the original text embeddings. The SG-Adapter's explicit and non-fully connected graph representation greatly improves the fully connected, transformer-based text representations. This enhancement is particularly notable in maintaining precise correspondence in scenarios involving multiple relationships. To address the challenges posed by low-quality annotated datasets like Visual Genome, we have manually curated a highly clean, multi-relational scene graph-image paired dataset MultiRels. Furthermore, we design three metrics derived from GPT-4V to effectively and thoroughly measure the correspondence between images and scene graphs. Both qualitative and quantitative results validate the efficacy of our approach in controlling the correspondence in multiple relationships.

As an alternative to expensive expert evaluation, Image Aesthetic Assessment (IAA) stands out as a crucial task in computer vision. However, traditional IAA methods are typically constrained to a single data source or task, restricting the universality and broader application. In this work, to better align with human aesthetics, we propose a Unified Multi-modal Image Aesthetic Assessment (UNIAA) framework, including a Multi-modal Large Language Model (MLLM) named UNIAA-LLaVA and a comprehensive benchmark named UNIAA-Bench. We choose MLLMs with both visual perception and language ability for IAA and establish a low-cost paradigm for transforming the existing datasets into unified and high-quality visual instruction tuning data, from which the UNIAA-LLaVA is trained. To further evaluate the IAA capability of MLLMs, we construct the UNIAA-Bench, which consists of three aesthetic levels: Perception, Description, and Assessment. Extensive experiments validate the effectiveness and rationality of UNIAA. UNIAA-LLaVA achieves competitive performance on all levels of UNIAA-Bench, compared with existing MLLMs. Specifically, our model performs better than GPT-4V in aesthetic perception and even approaches the junior-level human. We find MLLMs have great potential in IAA, yet there remains plenty of room for further improvement. The UNIAA-LLaVA and UNIAA-Bench will be released.

Previous methods for Video Frame Interpolation (VFI) have encountered challenges, notably the manifestation of blur and ghosting effects. These issues can be traced back to two pivotal factors: unavoidable motion errors and misalignment in supervision. In practice, motion estimates often prove to be error-prone, resulting in misaligned features. Furthermore, the reconstruction loss tends to bring blurry results, particularly in misaligned regions. To mitigate these challenges, we propose a new paradigm called PerVFI (Perception-oriented Video Frame Interpolation). Our approach incorporates an Asymmetric Synergistic Blending module (ASB) that utilizes features from both sides to synergistically blend intermediate features. One reference frame emphasizes primary content, while the other contributes complementary information. To impose a stringent constraint on the blending process, we introduce a self-learned sparse quasi-binary mask which effectively mitigates ghosting and blur artifacts in the output. Additionally, we employ a normalizing flow-based generator and utilize the negative log-likelihood loss to learn the conditional distribution of the output, which further facilitates the generation of clear and fine details. Experimental results validate the superiority of PerVFI, demonstrating significant improvements in perceptual quality compared to existing methods. Codes are available at \url{https://github.com/mulns/PerVFI}, Comment: Accepted by CVPR 2024

In this research, we present a novel approach to motion customization in video generation, addressing the widespread gap in the thorough exploration of motion representation within video generative models. Recognizing the unique challenges posed by video's spatiotemporal nature, our method introduces Motion Embeddings, a set of explicit, temporally coherent one-dimensional embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer modules of video diffusion models, modulating self-attention computations across frames without compromising spatial integrity. Our approach offers a compact and efficient solution to motion representation and enables complex manipulations of motion characteristics through vector arithmetic in the embedding space. Furthermore, we identify the Temporal Discrepancy in video generative models, which refers to variations in how different motion modules process temporal relationships between frames. We leverage this understanding to optimize the integration of our motion embeddings. Our contributions include the introduction of a tailored motion embedding for customization tasks, insights into the temporal processing differences in video models, and a demonstration of the practical advantages and effectiveness of our method through extensive experiments., Comment: Project Page: https://wileewang.github.io/MotionInversion

This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated Content Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2024. This challenge is to address a major challenge in the field of image and video processing, namely, Image Quality Assessment (IQA) and Video Quality Assessment (VQA) for AI-Generated Content (AIGC). The challenge is divided into the image track and the video track. The image track uses the AIGIQA-20K, which contains 20,000 AI-Generated Images (AIGIs) generated by 15 popular generative models. The image track has a total of 318 registered participants. A total of 1,646 submissions are received in the development phase, and 221 submissions are received in the test phase. Finally, 16 participating teams submitted their models and fact sheets. The video track uses the T2VQA-DB, which contains 10,000 AI-Generated Videos (AIGVs) generated by 9 popular Text-to-Video (T2V) models. A total of 196 participants have registered in the video track. A total of 991 submissions are received in the development phase, and 185 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. Some methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on AIGC.

Beyond 5G (B5G) communication networks face the challenge of meeting the demanding requirements of various service types, including uRLLC, mIoT, eMBB, and emerging technologies like Extended Reality (XR). Edge computing can address these demands effectively because of the ability to bring computational power and resources closer to the source of data. Nevertheless, the realization of this potential necessitates an open, flexible, and automated architectural framework capable of supporting disaggregated applications and network designs. In this context, this paper introduces a novel architecture designed to advance the evolution of edge computing in B5G, developed within the EU-funded project VERGE. The proposed architecture is modular and scalable, guided by artificial intelligence (AI), and founded on three essential pillars: "edge for AI," "AI for edge," and "security, privacy, and trustworthiness for AI." After presenting this architecture, the paper showcases its applicability through the examination of two vertical use cases within the industrial and transportation domains., Project VERGE has received funding from the Smart Networks and Services Joint Undertaking (SNS JU) under the European Union’s Horizon Europe research and innovation programme under Grant Agreement No 101096034. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. UK participants in VERGE are supported by UKRI grant numbers 10071211 (Samsung Electronics (UK) Limited) and 10061781 (King’s College London), Peer Reviewed, Postprint (author's final draft)

Chorus waves are naturally occurring electromagnetic emissions in space and are known to produce highly energetic electrons in the hazardous radiation belt. The characteristic feature of chorus is its fast frequency chirping, whose mechanism remains a long-standing problem. While many theories agree on its nonlinear nature, they differ on whether or how the background magnetic field inhomogeneity plays a key role. Here, using observations of chorus at Mars and Earth, we report direct evidence showing that the chorus chirping rate is consistently related to the background magnetic field inhomogeneity, despite orders of magnitude difference in a key parameter quantifying the inhomogeneity at the two planets. Our results show an extreme test of a recently proposed chorus generation model and confirm the connection between the chirping rate and magnetic field inhomogeneity, opening the door to controlled plasma wave excitation in the laboratory and space.

Vehicles are becoming increasingly complex and are prone to faults and failures, which threaten the dependability of vehicles in terms of availability, reliability, safety, and security. When vehicles are detected with certain types of faults and get into alarm situations, human drivers play a vital role in deciding what strategies and actions to take. Once driverless vehicles are introduced, human drivers' roles in decision-making will no longer exist, which urges new solutions on both technological and managerial levels. This thesis depicts the current human decision-making process by analyzing field study data in the truck industry, which contributes to gaining domain knowledge and identifying research gaps. An integrated vehicle health management scheme is applied to automate this decision-making process by integrating vehicle health state estimation and prediction, resource utilization, and self-adaptive management. To implement this scheme, fault diagnosis and decision-making methods are proposed, and a decision support system is designed. Fault diagnosis is a critical functional module for providing reliable vehicle health state information for decision-making. To address the influence of uncertainties in fault diagnosis, we propose an uncertainty analysis framework and a fault diagnosis method using Bayesian inference.Simulation experiments validate that the proposed method could effectively diagnose the root cause of fault symptoms under environmental uncertainty. A risk-based automated decision-making method is presented, which imitates the human decision-making process.On this basis, a collaborative decision-making method is proposed by considering traffic congestion, which is a currently neglected public concern.Experiment results show that the proposed methods could effectively reduce the economic risk and the risk of traffic congestion. In the end, a decision support system is designed to provide decision information to its human users. Besides, rev, Fordon blir allt mer komplexa och är benägna att få fler felkällor, vilket hotar fordonens pålitlighet när det gäller tillgänglighet, tillförlitlighet och säkerhet.När fordon upptäcks med vissa typer av fel och hamnar i larmsituationer spelar mänskliga förare en avgörande roll när det gäller att bestämma vilka åtgärder som ska vidtas.När förarlösa fordon väl har introducerats kommer mänskliga förares roller i beslutsfattandet inte längre vara tillgängligt, vilket kräver nya lösningar på både teknisk nivå och på ledningsnivå. Denna avhandling skildrar den nuvarande mänskliga beslutsprocessen genom att analysera fältstudiedata i lastbilsindustrin, vilket bidrar till mer kunskap om området och att identifiera forskningsluckor.Ett integrerat system för fordonshälsa används för att automatisera denna beslutsprocess genom att integrera uppskattning och förutsägelse av fordonets hälsotillstånd, resursanvändning och fordonets möjlighet att själva anpassa sig för att hantera fel.För att implementera detta schema föreslås feldiagnostik och beslutsmetoder, och ett beslutsstödssystem utformas. Feldiagnos är en kritisk funktionsmodul för att tillhandahålla tillförlitlig information om fordonets hälsotillstånd för beslutsfattande.För att hantera osäkerheter i feldiagnostik, föreslår vi ett ramverk för osäkerhetsanalys och en feldiagnosmetod som använder Bayesiansk slutledning.Simuleringsexperiment bekräftar att den föreslagna metoden effektivt kan diagnostisera grundorsaken till felsymptom, även vid osäkerhet om fordonets kontext. En riskbaserad automatiserad beslutsmetod presenteras, som imiterar den mänskliga beslutsprocessen.På grundval av detta föreslås en samarbetsmetod för beslutsfattande genom att överväga trafikstockningar, som är ett stort allmänt problem.Experimentresultat visar att de föreslagna metoderna effektivt kan minska den ekonomiska risken och risken för trafikstockningar. Dessutom har ett system för beslutsstöd utformats för att ge information till mänskliga a

Edge computing is a key enabling technology that is expected to play a crucial role in beyond 5G (B5G) and 6G communication networks. By bringing computation closer to where the data is generated, and leveraging Artificial Intelligence (AI) capabilities for advanced automation and orchestration, edge computing can enable a wide range of emerging applications with extreme requirements in terms of latency and computation, across multiple vertical domains. In this context, this paper first discusses the key technological challenges for the seamless integration of edge computing within B5G/6G and then presents a roadmap for the edge computing evolution, proposing a novel design approach for an open, intelligent, trustworthy, and distributed edge architecture., VERGE has received funding from the Smart Networks and Services Joint Undertaking (SNS JU) under the European Union’s Horizon Europe research and innovation programme under Grant Agreement No 101096034., Peer Reviewed, Postprint (author's final draft)

Videos stored on mobile devices or delivered on the Internet are usually in compressed format and are of various unknown compression parameters, but most video super-resolution (VSR) methods often assume ideal inputs resulting in large performance gap between experimental settings and real-world applications. In spite of a few pioneering works being proposed recently to super-resolve the compressed videos, they are not specially designed to deal with videos of various levels of compression. In this paper, we propose a novel and practical compression-aware video super-resolution model, which could adapt its video enhancement process to the estimated compression level. A compression encoder is designed to model compression levels of input frames, and a base VSR model is then conditioned on the implicitly computed representation by inserting compression-aware modules. In addition, we propose to further strengthen the VSR model by taking full advantage of meta data that is embedded naturally in compressed video streams in the procedure of information fusion. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the proposed method on compressed VSR benchmarks. The codes will be available at https://github.com/aprBlue/CAVSR © 2023 IEEE.

Vehicles are becoming increasingly complex and are prone to faults and failures, which threaten the dependability of vehicles in terms of availability, reliability, safety, and security. When vehicles are detected with certain types of faults and get into alarm situations, human drivers play a vital role in deciding what strategies and actions to take. Once driverless vehicles are introduced, human drivers' roles in decision-making will no longer exist, which urges new solutions on both technological and managerial levels. This thesis depicts the current human decision-making process by analyzing field study data in the truck industry, which contributes to gaining domain knowledge and identifying research gaps. An integrated vehicle health management scheme is applied to automate this decision-making process by integrating vehicle health state estimation and prediction, resource utilization, and self-adaptive management. To implement this scheme, fault diagnosis and decision-making methods are proposed, and a decision support system is designed. Fault diagnosis is a critical functional module for providing reliable vehicle health state information for decision-making. To address the influence of uncertainties in fault diagnosis, we propose an uncertainty analysis framework and a fault diagnosis method using Bayesian inference.Simulation experiments validate that the proposed method could effectively diagnose the root cause of fault symptoms under environmental uncertainty. A risk-based automated decision-making method is presented, which imitates the human decision-making process.On this basis, a collaborative decision-making method is proposed by considering traffic congestion, which is a currently neglected public concern.Experiment results show that the proposed methods could effectively reduce the economic risk and the risk of traffic congestion. In the end, a decision support system is designed to provide decision information to its human users. Besides, rev, Fordon blir allt mer komplexa och är benägna att få fler felkällor, vilket hotar fordonens pålitlighet när det gäller tillgänglighet, tillförlitlighet och säkerhet.När fordon upptäcks med vissa typer av fel och hamnar i larmsituationer spelar mänskliga förare en avgörande roll när det gäller att bestämma vilka åtgärder som ska vidtas.När förarlösa fordon väl har introducerats kommer mänskliga förares roller i beslutsfattandet inte längre vara tillgängligt, vilket kräver nya lösningar på både teknisk nivå och på ledningsnivå. Denna avhandling skildrar den nuvarande mänskliga beslutsprocessen genom att analysera fältstudiedata i lastbilsindustrin, vilket bidrar till mer kunskap om området och att identifiera forskningsluckor.Ett integrerat system för fordonshälsa används för att automatisera denna beslutsprocess genom att integrera uppskattning och förutsägelse av fordonets hälsotillstånd, resursanvändning och fordonets möjlighet att själva anpassa sig för att hantera fel.För att implementera detta schema föreslås feldiagnostik och beslutsmetoder, och ett beslutsstödssystem utformas. Feldiagnos är en kritisk funktionsmodul för att tillhandahålla tillförlitlig information om fordonets hälsotillstånd för beslutsfattande.För att hantera osäkerheter i feldiagnostik, föreslår vi ett ramverk för osäkerhetsanalys och en feldiagnosmetod som använder Bayesiansk slutledning.Simuleringsexperiment bekräftar att den föreslagna metoden effektivt kan diagnostisera grundorsaken till felsymptom, även vid osäkerhet om fordonets kontext. En riskbaserad automatiserad beslutsmetod presenteras, som imiterar den mänskliga beslutsprocessen.På grundval av detta föreslås en samarbetsmetod för beslutsfattande genom att överväga trafikstockningar, som är ett stort allmänt problem.Experimentresultat visar att de föreslagna metoderna effektivt kan minska den ekonomiska risken och risken för trafikstockningar. Dessutom har ett system för beslutsstöd utformats för att ge information till mänskliga a

Vehicles are becoming increasingly complex and are prone to faults and failures, which threaten the dependability of vehicles in terms of availability, reliability, safety, and security. When vehicles are detected with certain types of faults and get into alarm situations, human drivers play a vital role in deciding what strategies and actions to take. Once driverless vehicles are introduced, human drivers' roles in decision-making will no longer exist, which urges new solutions on both technological and managerial levels. This thesis depicts the current human decision-making process by analyzing field study data in the truck industry, which contributes to gaining domain knowledge and identifying research gaps. An integrated vehicle health management scheme is applied to automate this decision-making process by integrating vehicle health state estimation and prediction, resource utilization, and self-adaptive management. To implement this scheme, fault diagnosis and decision-making methods are proposed, and a decision support system is designed. Fault diagnosis is a critical functional module for providing reliable vehicle health state information for decision-making. To address the influence of uncertainties in fault diagnosis, we propose an uncertainty analysis framework and a fault diagnosis method using Bayesian inference.Simulation experiments validate that the proposed method could effectively diagnose the root cause of fault symptoms under environmental uncertainty. A risk-based automated decision-making method is presented, which imitates the human decision-making process.On this basis, a collaborative decision-making method is proposed by considering traffic congestion, which is a currently neglected public concern.Experiment results show that the proposed methods could effectively reduce the economic risk and the risk of traffic congestion. In the end, a decision support system is designed to provide decision information to its human users. Besides, rev, Fordon blir allt mer komplexa och är benägna att få fler felkällor, vilket hotar fordonens pålitlighet när det gäller tillgänglighet, tillförlitlighet och säkerhet.När fordon upptäcks med vissa typer av fel och hamnar i larmsituationer spelar mänskliga förare en avgörande roll när det gäller att bestämma vilka åtgärder som ska vidtas.När förarlösa fordon väl har introducerats kommer mänskliga förares roller i beslutsfattandet inte längre vara tillgängligt, vilket kräver nya lösningar på både teknisk nivå och på ledningsnivå. Denna avhandling skildrar den nuvarande mänskliga beslutsprocessen genom att analysera fältstudiedata i lastbilsindustrin, vilket bidrar till mer kunskap om området och att identifiera forskningsluckor.Ett integrerat system för fordonshälsa används för att automatisera denna beslutsprocess genom att integrera uppskattning och förutsägelse av fordonets hälsotillstånd, resursanvändning och fordonets möjlighet att själva anpassa sig för att hantera fel.För att implementera detta schema föreslås feldiagnostik och beslutsmetoder, och ett beslutsstödssystem utformas. Feldiagnos är en kritisk funktionsmodul för att tillhandahålla tillförlitlig information om fordonets hälsotillstånd för beslutsfattande.För att hantera osäkerheter i feldiagnostik, föreslår vi ett ramverk för osäkerhetsanalys och en feldiagnosmetod som använder Bayesiansk slutledning.Simuleringsexperiment bekräftar att den föreslagna metoden effektivt kan diagnostisera grundorsaken till felsymptom, även vid osäkerhet om fordonets kontext. En riskbaserad automatiserad beslutsmetod presenteras, som imiterar den mänskliga beslutsprocessen.På grundval av detta föreslås en samarbetsmetod för beslutsfattande genom att överväga trafikstockningar, som är ett stort allmänt problem.Experimentresultat visar att de föreslagna metoderna effektivt kan minska den ekonomiska risken och risken för trafikstockningar. Dessutom har ett system för beslutsstöd utformats för att ge information till mänskliga a

Plasmaspheric hiss is an electromagnetic wave mode that occurs ubiquitously in the high-density plasmasphere and contributes crucially to the dynamic behavior of the Earth’s Van Allen radiation belts. While plasmaspheric hiss is commonly considered to be a broadband emission with frequencies from ∼100 Hz to several kHz, here we report Van Allen Probes measurements of unambiguous banded signatures of plasmaspheric hiss, uniquely characterized by an upper band above ∼200 Hz, a lower band below ∼100 Hz and a power gap in between. Banded plasmaspheric hiss occurs with the probability ∼8% in the postnoon sector within 2.5-5.0 Earth radii, showing strong dependence on geomagnetic and solar wind conditions. Observations also suggest that banded hiss waves result possibly from two combined sources, the upper band originating from the transformation of chorus waves propagating from outside the plasmasphere, and the lower band from localized excitation inside the plasmasphere, which however requires future investigation. The banded hiss waves shed new light on the evolution of the Earth’s radiation belts and have implications for understanding whistler-mode waves in planetary magnetospheres.

Vehicles are becoming increasingly complex and are prone to faults and failures, which threaten the dependability of vehicles in terms of availability, reliability, safety, and security. When vehicles are detected with certain types of faults and get into alarm situations, human drivers play a vital role in deciding what strategies and actions to take. Once driverless vehicles are introduced, human drivers' roles in decision-making will no longer exist, which urges new solutions on both technological and managerial levels. This thesis depicts the current human decision-making process by analyzing field study data in the truck industry, which contributes to gaining domain knowledge and identifying research gaps. An integrated vehicle health management scheme is applied to automate this decision-making process by integrating vehicle health state estimation and prediction, resource utilization, and self-adaptive management. To implement this scheme, fault diagnosis and decision-making methods are proposed, and a decision support system is designed. Fault diagnosis is a critical functional module for providing reliable vehicle health state information for decision-making. To address the influence of uncertainties in fault diagnosis, we propose an uncertainty analysis framework and a fault diagnosis method using Bayesian inference.Simulation experiments validate that the proposed method could effectively diagnose the root cause of fault symptoms under environmental uncertainty. A risk-based automated decision-making method is presented, which imitates the human decision-making process.On this basis, a collaborative decision-making method is proposed by considering traffic congestion, which is a currently neglected public concern.Experiment results show that the proposed methods could effectively reduce the economic risk and the risk of traffic congestion. In the end, a decision support system is designed to provide decision information to its human users. Besides, rev, Fordon blir allt mer komplexa och är benägna att få fler felkällor, vilket hotar fordonens pålitlighet när det gäller tillgänglighet, tillförlitlighet och säkerhet.När fordon upptäcks med vissa typer av fel och hamnar i larmsituationer spelar mänskliga förare en avgörande roll när det gäller att bestämma vilka åtgärder som ska vidtas.När förarlösa fordon väl har introducerats kommer mänskliga förares roller i beslutsfattandet inte längre vara tillgängligt, vilket kräver nya lösningar på både teknisk nivå och på ledningsnivå. Denna avhandling skildrar den nuvarande mänskliga beslutsprocessen genom att analysera fältstudiedata i lastbilsindustrin, vilket bidrar till mer kunskap om området och att identifiera forskningsluckor.Ett integrerat system för fordonshälsa används för att automatisera denna beslutsprocess genom att integrera uppskattning och förutsägelse av fordonets hälsotillstånd, resursanvändning och fordonets möjlighet att själva anpassa sig för att hantera fel.För att implementera detta schema föreslås feldiagnostik och beslutsmetoder, och ett beslutsstödssystem utformas. Feldiagnos är en kritisk funktionsmodul för att tillhandahålla tillförlitlig information om fordonets hälsotillstånd för beslutsfattande.För att hantera osäkerheter i feldiagnostik, föreslår vi ett ramverk för osäkerhetsanalys och en feldiagnosmetod som använder Bayesiansk slutledning.Simuleringsexperiment bekräftar att den föreslagna metoden effektivt kan diagnostisera grundorsaken till felsymptom, även vid osäkerhet om fordonets kontext. En riskbaserad automatiserad beslutsmetod presenteras, som imiterar den mänskliga beslutsprocessen.På grundval av detta föreslås en samarbetsmetod för beslutsfattande genom att överväga trafikstockningar, som är ett stort allmänt problem.Experimentresultat visar att de föreslagna metoderna effektivt kan minska den ekonomiska risken och risken för trafikstockningar. Dessutom har ett system för beslutsstöd utformats för att ge information till mänskliga a

Video panoptic segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. We believe that the decoupling strategy proposed by DVIS enables more effective utilization of temporal information for both "thing" and "stuff" objects. In this report, we successfully validated the effectiveness of the decoupling strategy in video panoptic segmentation. Finally, our method achieved a VPQ score of 51.4 and 53.7 in the development and test phases, respectively, and ultimately ranked 1st in the VPS track of the 2nd PVUW Challenge. The code is available at https://github.com/zhang-tao-whu/DVIS

We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS). Unlike previous methods that model video segmentation in an end-to-end manner, our approach decouples video segmentation into three cascaded sub-tasks: segmentation, tracking, and refinement. This decoupling design allows for simpler and more effective modeling of the spatio-temporal representations of objects, especially in complex scenes and long videos. Accordingly, we introduce two novel components: the referring tracker and the temporal refiner. These components track objects frame by frame and model spatio-temporal representations based on pre-aligned features. To improve the tracking capability of DVIS, we propose a denoising training strategy and introduce contrastive learning, resulting in a more robust framework named DVIS++. Furthermore, we evaluate DVIS++ in various settings, including open vocabulary and using a frozen pre-trained backbone. By integrating CLIP with DVIS++, we present OV-DVIS++, the first open-vocabulary universal video segmentation framework. We conduct extensive experiments on six mainstream benchmarks, including the VIS, VSS, and VPS datasets. Using a unified architecture, DVIS++ significantly outperforms state-of-the-art specialized methods on these benchmarks in both close- and open-vocabulary settings. Code:~\url{https://github.com/zhang-tao-whu/DVIS_Plus}.

The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts which, however, often require good skills to specify. To make SAM robust to casual prompts, this paper presents the first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities, notably imprecise bounding boxes and insufficient points. Our key finding reveals that given such low-quality prompts, SAM's mask decoder tends to activate image features that are biased towards the background or confined to specific object parts. To mitigate this issue, our key idea consists of calibrating solely SAM's mask attention by adjusting the sampling locations and amplitudes of image features, while the original SAM model architecture and weights remain unchanged. Consequently, our deformable sampling plugin (DSP) enables SAM to adaptively shift attention to the prompted target regions in a data-driven manner, facilitated by our effective robust training strategy (RTS). During inference, dynamic routing plugin (DRP) is proposed that toggles SAM between the deformable and regular grid sampling modes, conditioned on the input prompt quality. Thus, our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality, with 3) minimal learnable parameters (0.08 M) and fast adaptation (by 1 training epoch). Extensive experiments across multiple datasets validate the effectiveness and advantages of our approach, underscoring Stable-SAM as a more robust solution for segmenting anything. Codes will be released upon acceptance. https://github.com/fanq15/Stable-SAM, Comment: Smaller file size for the easy access. Codes will be released upon acceptance. https://github.com/fanq15/Stable-SAM

Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for the trainable tracker, allowing it to achieve more stable and accurate object tracking in complex and long videos. Additionally, we explore the role of visual foundation models in video instance segmentation. By utilizing a frozen VIT-L model pre-trained by DINO v2, DVIS demonstrates remarkable performance improvements. With these enhancements, our method achieves 57.9 AP and 56.0 AP in the development and test phases, respectively, and ultimately ranked 1st in the VIS track of the 5th LSVOS Challenge. The code will be available at https://github.com/zhang-tao-whu/DVIS.

Existing methods for interactive segmentation in radiance fields entail scene-specific optimization and thus cannot generalize across different scenes, which greatly limits their applicability. In this work we make the first attempt at Scene-Generalizable Interactive Segmentation in Radiance Fields (SGISRF) and propose a novel SGISRF method, which can perform 3D object segmentation for novel (unseen) scenes represented by radiance fields, guided by only a few interactive user clicks in a given set of multi-view 2D images. In particular, the proposed SGISRF focuses on addressing three crucial challenges with three specially designed techniques. First, we devise the Cross-Dimension Guidance Propagation to encode the scarce 2D user clicks into informative 3D guidance representations. Second, the Uncertainty-Eliminated 3D Segmentation module is designed to achieve efficient yet effective 3D segmentation. Third, Concealment-Revealed Supervised Learning scheme is proposed to reveal and correct the concealed 3D segmentation errors resulted from the supervision in 2D space with only 2D mask annotations. Extensive experiments on two real-world challenging benchmarks covering diverse scenes demonstrate 1) effectiveness and scene-generalizability of the proposed method, 2) favorable performance compared to classical method requiring scene-specific optimization.

Recent interactive segmentation methods iteratively take source image, user guidance and previously predicted mask as the input without considering the invariant nature of the source image. As a result, extracting features from the source image is repeated in each interaction, resulting in substantial computational redundancy. In this work, we propose the Feature Decoupling-Recycling Network (FDRN), which decouples the modeling components based on their intrinsic discrepancies and then recycles components for each user interaction. Thus, the efficiency of the whole interactive process can be significantly improved. To be specific, we apply the Decoupling-Recycling strategy from three perspectives to address three types of discrepancies, respectively. First, our model decouples the learning of source image semantics from the encoding of user guidance to process two types of input domains separately. Second, FDRN decouples high-level and low-level features from stratified semantic representations to enhance feature learning. Third, during the encoding of user guidance, current user guidance is decoupled from historical guidance to highlight the effect of current user guidance. We conduct extensive experiments on 6 datasets from different domains and modalities, which demonstrate the following merits of our model: 1) superior efficiency than other methods, particularly advantageous in challenging scenarios requiring long-term interactions (up to 4.25x faster), while achieving favorable segmentation performance; 2) strong applicability to various methods serving as a universal enhancement technique; 3) well cross-task generalizability, e.g., to medical image segmentation, and robustness against misleading user guidance., Comment: Accepted to ACM MM 2023

Capitalizing on the rapid development of neural networks, recent video frame interpolation (VFI) methods have achieved notable improvements. However, they still fall short for real-world videos containing large motions. Complex deformation and/or occlusion caused by large motions make it an extremely difficult problem in video frame interpolation. In this paper, we propose a simple yet effective solution, H-VFI, to deal with large motions in video frame interpolation. H-VFI contributes a hierarchical video interpolation transformer (HVIT) to learn a deformable kernel in a coarse-to-fine strategy in multiple scales. The learnt deformable kernel is then utilized in convolving the input frames for predicting the interpolated frame. Starting from the smallest scale, H-VFI updates the deformable kernel by a residual in succession based on former predicted kernels, intermediate interpolated results and hierarchical features from transformer. Bias and masks to refine the final outputs are then predicted by a transformer block based on interpolated results. The advantage of such a progressive approximation is that the large motion frame interpolation problem can be decomposed into several relatively simpler sub-tasks, which enables a very accurate prediction in the final results. Another noteworthy contribution of our paper consists of a large-scale high-quality dataset, YouTube200K, which contains videos depicting a great variety of scenarios captured at high resolution and high frame rate. Extensive experiments on multiple frame interpolation benchmarks validate that H-VFI outperforms existing state-of-the-art methods especially for videos with large motions.

This paper proposes a novel video inpainting method. We make three main contributions: First, we extended previous Transformers with patch alignment by introducing Deformed Patch-based Homography (DePtH), which improves patch-level feature alignments without additional supervision and benefits challenging scenes with various deformation. Second, we introduce Mask Pruning-based Patch Attention (MPPA) to improve patch-wised feature matching by pruning out less essential features and using saliency map. MPPA enhances matching accuracy between warped tokens with invalid pixels. Third, we introduce a Spatial-Temporal weighting Adaptor (STA) module to obtain accurate attention to spatial-temporal tokens under the guidance of the Deformation Factor learned from DePtH, especially for videos with agile motions. Experimental results demonstrate that our method outperforms recent methods qualitatively and quantitatively and achieves a new state-of-the-art.

Temporal modeling is crucial for video super-resolution. Most of the video super-resolution methods adopt the optical flow or deformable convolution for explicitly motion compensation. However, such temporal modeling techniques increase the model complexity and might fail in case of occlusion or complex motion, resulting in serious distortion and artifacts. In this paper, we propose to explore the role of explicit temporal difference modeling in both LR and HR space. Instead of directly feeding consecutive frames into a VSR model, we propose to compute the temporal difference between frames and divide those pixels into two subsets according to the level of difference. They are separately processed with two branches of different receptive fields in order to better extract complementary information. To further enhance the super-resolution result, not only spatial residual features are extracted, but the difference between consecutive frames in high-frequency domain is also computed. It allows the model to exploit intermediate SR results in both future and past to refine the current SR output. The difference at different time steps could be cached such that information from further distance in time could be propagated to the current frame for refinement. Experiments on several video super-resolution benchmark datasets demonstrate the effectiveness of the proposed method and its favorable performance against state-of-the-art methods., Comment: CVPR 2022

This paper addresses the challenges of decision-making for autonomous vehicles under faults during a transport mission. A real-time decision-making problem of vehicle routing planning considering maintenance management is formulated as an optimization problem. The goal is to minimize the total time to finish the transport mission by selecting the optimal workshop to conduct the maintenance and the corresponding routes. Two methods are proposed to solve the optimization problem based on two methods of fundamental solutions: (1) Mixed Integer Programming; (2) Dijkstra's algorithm. We adapt these methods to solve the optimization problem and consider improving the computation efficiency. Numerical studies of test cases of highway and urban scenarios are presented to demonstrate the proposed methods, which show the feasibility and high computational efficiency of both methods., Comment: Accepted by the IEEE 9th International Conference on Industrial Engineering and Applications (ICIEA 2022). Date: November 2021. Email: taoxin@kth.se, zhaoyuan@hi.is, zhaoyuan.epslab@gmail.com

With the development of vehicular technologies on automation, electrification, and digitalization, vehicles are becoming more intelligent while being exposed to more complex, uncertain, and frequently occurring faults. In this paper, we look into the maintenance planning of an operating vehicle under fault condition and formulate it as a multi-criteria decision-making problem. The maintenance decisions are generated by route searching in road networks and evaluated based on risk assessment considering the uncertainty of vehicle breakdowns. Particularly, we consider two criteria, namely the risk of public time loss and the risk of mission delay, representing the concerns of the public sector and the private sector, respectively. A public time loss model is developed to evaluate the traffic congestion caused by a vehicle breakdown and the corresponding towing process. The Pareto optimal set of non-dominated decisions is derived by evaluating the risk of the decisions. We demonstrate the relevance of the problem and the effectiveness of the proposed method by numerical experiments derived from real-world scenarios. The experiments show that neglecting the risk of vehicle breakdown on public roads can cause a high risk of public time loss in dense traffic flow. With the proposed method, alternate decisions can be derived to reduce the risks of public time loss significantly with a low increase in the risk of mission delay. This study aims at catalyzing public-private partnership through collaborative decision-making between the private sector and the public sector, thus archiving a more sustainable transportation system in the future.

Rice paddy soil-associated microbiota participate in biogeochemical processes that underpin rice yield and soil sustainability, yet continental-scale biogeographic patterns of paddy soil microbiota remain elusive. The soil bacteria of four typical Chinese rice-growing regions were characterized and compared to those of nonpaddy soils. The paddy soil bacteria were significantly less diverse, with unique taxonomic and functional composition, and harbored distinct cooccurrence network topology. Both stochastic and deterministic processes shaped soil bacteria assembly, but paddy samples exhibited a stronger deterministic signature than nonpaddy samples. Compared to other environmental factors, climatic factors such as mean monthly precipitation and mean annual temperature described most of the variance in soil bacterial community structure. Cooccurrence network analysis suggests that the continental biogeographic variance in bacterial community structure was described by the competition between two mutually exclusive bacterial modules in the community. Keystone taxa identified in network models (Anaerolineales, Ignavibacteriae, and Deltaproteobacteria) were more sensitive to changes in environmental factors, leading us to conclude that environmental factors may influence paddy soil bacterial communities via these keystone taxa. Characterizing the uniqueness of bacterial community patterns in paddy soil (compared to nonpaddy soils) at continental scales is central to improving crop productivity and resilience and to sustaining agricultural soils. IMPORTANCE Rice fields provide food for over half of the world's human population. The ecology of paddy soil microbiomes is shaped by human activities, which can have a profound impact on rice yield, greenhouse gas emissions, and soil health. Investigations of the soil bacteria in four typical Chinese rice-growing regions showed that (i) soil bacterial communities maintain highly modularized species-to-species network structures

Rice paddy soil-associated microbiota participate in biogeochemical processes that underpin rice yield and soil sustainability, yet continental-scale biogeographic patterns of paddy soil microbiota remain elusive. The soil bacteria of four typical Chinese rice-growing regions were characterized and compared to those of nonpaddy soils. The paddy soil bacteria were significantly less diverse, with unique taxonomic and functional composition, and harbored distinct cooccurrence network topology. Both stochastic and deterministic processes shaped soil bacteria assembly, but paddy samples exhibited a stronger deterministic signature than nonpaddy samples. Compared to other environmental factors, climatic factors such as mean monthly precipitation and mean annual temperature described most of the variance in soil bacterial community structure. Cooccurrence network analysis suggests that the continental biogeographic variance in bacterial community structure was described by the competition between two mutually exclusive bacterial modules in the community. Keystone taxa identified in network models (Anaerolineales, Ignavibacteriae, and Deltaproteobacteria) were more sensitive to changes in environmental factors, leading us to conclude that environmental factors may influence paddy soil bacterial communities via these keystone taxa. Characterizing the uniqueness of bacterial community patterns in paddy soil (compared to nonpaddy soils) at continental scales is central to improving crop productivity and resilience and to sustaining agricultural soils. IMPORTANCE Rice fields provide food for over half of the world's human population. The ecology of paddy soil microbiomes is shaped by human activities, which can have a profound impact on rice yield, greenhouse gas emissions, and soil health. Investigations of the soil bacteria in four typical Chinese rice-growing regions showed that (i) soil bacterial communities maintain highly modularized species-to-species network structures

Scenario-based approaches have been receiving a huge amount of attention in research and engineering of automated driving systems. Due to the complexity and uncertainty of the driving environment, and the complexity of the driving task itself, the number of possible driving scenarios that an ADS or ADAS may encounter is virtually infinite. Therefore it is essential to be able to reason about the identification of scenarios and in particular critical ones that may impose unacceptable risk if not considered. Critical scenarios are particularly important to support design, verification and validation efforts, and as a basis for a safety case. In this paper, we present the results of a systematic literature review in the context of autonomous driving. The main contributions are: (i) introducing a comprehensive taxonomy for critical scenario identification methods; (ii) giving an overview of the state-of-the-art research based on the taxonomy encompassing 86 papers between 2017 and 2020; and (iii) identifying open issues and directions for further research. The provided taxonomy comprises three main perspectives encompassing the problem definition (the why), the solution (the methods to derive scenarios), and the assessment of the established scenarios. In addition, we discuss open research issues considering the perspectives of coverage, practicability, and scenario space explosion., Comment: 37 pages, 24 figures

Rong-Zheng Ma,1 Ping-Ping Han,1 Xin-Cao Tao,2 Huan Li,1 Ling Wang,1 Zhen-Guo Zhai,2 Li-Ping Fu1 1Department of Nuclear Medicine, China-Japan Friendship Hospital, Beijing, 100029, People’s Republic of China; 2Department of Pulmonary and Critical Care Medicine, China-Japan Friendship Hospital, National Center for Respiratory Medicine, Institute of Respiratory Medicine, Chinese Academy of Medical Sciences, National Clinical Research Center for Respiratory Diseases, Beijing, 100029, People’s Republic of ChinaCorrespondence: Li-Ping FuDepartment of Nuclear Medicine, China-Japan Friendship Hospital, No. 2 of Yinghuayuan Road, Chaoyang District, Beijing, 100029, People’s Republic of ChinaTel +86 10 84205389Fax +86 10-84205389Email fulipjh@126.comZhen-Guo ZhaiDepartment of Pulmonary and Critical Care Medicine, China-Japan Friendship Hospital, No. 2 of Yinghuayuan Road, Chaoyang District, Beijing, 100029, People’s Republic of ChinaTel +86 10 84206269Fax +86 10 84206269Email zhaizhguo235@outlook.comObjective: The study is designed to evaluate the diagnostic ability of single-photon emission computed tomography (SPECT) pulmonary ventilation/perfusion (V/Q) imaging in patients with chronic thromboembolic pulmonary hypertension (CTEPH) and investigate its feasibility in assessing patient risk.Methods: A total of 83 patients suspected of having CTEPH who received V/Q tomography were retrospectively analyzed. The consistency between SPECT V/Q imaging and pulmonary angiography was compared to investigate the correlation between the percentage of pulmonary perfusion defect score (PPDs%) and the hemodynamic indices. Patients were grouped according to the pulmonary arterial hypertension risk stratification, and the V/Q imaging results were compared between different groups.Results: For the 1494 pulmonary segments of the 83 patients, the sensitivity, specificity, and accuracy of identifying pulmonary segments with defects using V/Q imaging was 87.05%, 82.78% (668/807), and 84.7

New autonomous driving technologies are emerging every day and some of them have been commercially applied in the real world. While benefiting from these technologies, autonomous trucks are facing new challenges in short-term maintenance planning, which directly influences the truck operator's profit. In this paper, we implement a vehicle health management system by addressing the maintenance planning issues of autonomous trucks on a transport mission. We also present a maintenance planning model using a risk-based decision-making method, which identifies the maintenance decision with minimal economic risk of the truck company. Both availability losses and maintenance costs are considered when evaluating the economic risk. We demonstrate the proposed model by numerical experiments illustrating real-world scenarios. In the experiments, compared to three baseline methods, the expected economic risk of the proposed method is reduced by up to $47\%$. We also conduct sensitivity analyses of different model parameters. The analyses show that the economic risk significantly decreases when the estimation accuracy of remaining useful life, the maximal allowed time of delivery delay before order cancellation, or the number of workshops increases. The experiment results contribute to identifying future research and development attentions of autonomous trucks from an economic perspective., Comment: 22 pages, 13 figures, journal, submitted to Reliability Engineering & System Safety, under review

Reference-based image super-resolution (RefSR) has shown promising success in recovering high-frequency details by utilizing an external reference image (Ref). In this task, texture details are transferred from the Ref image to the low-resolution (LR) image according to their point- or patch-wise correspondence. Therefore, high-quality correspondence matching is critical. It is also desired to be computationally efficient. Besides, existing RefSR methods tend to ignore the potential large disparity in distributions between the LR and Ref images, which hurts the effectiveness of the information utilization. In this paper, we propose the MASA network for RefSR, where two novel modules are designed to address these problems. The proposed Match & Extraction Module significantly reduces the computational cost by a coarse-to-fine correspondence matching scheme. The Spatial Adaptation Module learns the difference of distribution between the LR and Ref images, and remaps the distribution of Ref features to that of LR features in a spatially adaptive way. This scheme makes the network robust to handle different reference images. Extensive quantitative and qualitative experiments validate the effectiveness of our proposed model., Comment: Accepted by CVPR 2021

Electron Cyclotron Harmonic (ECH) waves driven by a loss cone distribution are studied in this work by self‐consistent particle‐in‐cell simulations. These waves have been suggested to play an important role in diffuse auroral precipitation in the outer magnetosphere. However, particle simulation of this instability is difficult because the saturation amplitude of the wave driven by a realistic size loss cone distribution is very small. In this work we use an extraordinarily large number of particles to reduce simulation noise so that the growth and saturation of ECH waves can be investigated. Our simulation results are consistent with linear theory in terms of growth rate, and with observation in terms of wave amplitude. We demonstrate that the heating of cold electrons is negligible and non‐resonant, different from previous conclusions, and suggest that the saturation of the wave is caused by the filling of the loss cone of hot electrons.

Electron Cyclotron Harmonic (ECH) waves driven by a loss cone distribution are studied in this work by self‐consistent particle‐in‐cell simulations. These waves have been suggested to play an important role in diffuse auroral precipitation in the outer magnetosphere. However, particle simulation of this instability is difficult because the saturation amplitude of the wave driven by a realistic size loss cone distribution is very small. In this work we use an extraordinarily large number of particles to reduce simulation noise so that the growth and saturation of ECH waves can be investigated. Our simulation results are consistent with linear theory in terms of growth rate, and with observation in terms of wave amplitude. We demonstrate that the heating of cold electrons is negligible and non‐resonant, different from previous conclusions, and suggest that the saturation of the wave is caused by the filling of the loss cone of hot electrons.

Video super-resolution (VSR) aims to utilize multiple low-resolution frames to generate a high-resolution prediction for each frame. In this process, inter- and intra-frames are the key sources for exploiting temporal and spatial information. However, there are a couple of limitations for existing VSR methods. First, optical flow is often used to establish temporal correspondence. But flow estimation itself is error-prone and affects recovery results. Second, similar patterns existing in natural images are rarely exploited for the VSR task. Motivated by these findings, we propose a temporal multi-correspondence aggregation strategy to leverage similar patches across frames, and a cross-scale nonlocal-correspondence aggregation scheme to explore self-similarity of images across scales. Based on these two new modules, we build an effective multi-correspondence aggregation network (MuCAN) for VSR. Our method achieves state-of-the-art results on multiple benchmark datasets. Extensive experiments justify the effectiveness of our method., Comment: Accepted By ECCV2020

Blind inpainting is a task to automatically complete visual contents without specifying masks for missing areas in an image. Previous works assume missing region patterns are known, limiting its application scope. In this paper, we relax the assumption by defining a new blind inpainting setting, making training a blind inpainting neural system robust against various unknown missing region patterns. Specifically, we propose a two-stage visual consistency network (VCN), meant to estimate where to fill (via masks) and generate what to fill. In this procedure, the unavoidable potential mask prediction errors lead to severe artifacts in the subsequent repairing. To address it, our VCN predicts semantically inconsistent regions first, making mask prediction more tractable. Then it repairs these estimated missing regions using a new spatial normalization, enabling VCN to be robust to the mask prediction errors. In this way, semantically convincing and visually compelling content is thus generated. Extensive experiments are conducted, showing our method is effective and robust in blind image inpainting. And our VCN allows for a wide spectrum of applications.

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work.

Conditional generative adversarial networks have shown exceptional generation performance over the past few years. However, they require large numbers of annotations. To address this problem, we propose a novel generative adversarial network utilizing weak supervision in the form of pairwise comparisons (PC-GAN) for image attribute editing. In the light of Bayesian uncertainty estimation and noise-tolerant adversarial training, PC-GAN can estimate attribute rating efficiently and demonstrate robust performance in noise resistance. Through extensive experiments, we show both qualitatively and quantitatively that PC-GAN performs comparably with fully-supervised methods and outperforms unsupervised baselines., Comment: Accepted for spotlight at AAAI-20

In this paper, we are interested in generating an cartoon face of a person by using unpaired training data between real faces and cartoon ones. A major challenge of this task is that the structures of real and cartoon faces are in two different domains, whose appearance differs greatly from each other. Without explicit correspondence, it is difficult to generate a high quality cartoon face that captures the essential facial features of a person. In order to solve this problem, we propose landmark assisted CycleGAN, which utilizes face landmarks to define landmark consistency loss and to guide the training of local discriminator in CycleGAN. To enforce structural consistency in landmarks, we utilize the conditional generator and discriminator. Our approach is capable to generate high-quality cartoon faces even indistinguishable from those drawn by artists and largely improves state-of-the-art.

Current image translation methods, albeit effective to produce high-quality results in various applications, still do not consider much geometric transform. We in this paper propose the spontaneous motion estimation module, along with a refinement part, to learn attribute-driven deformation between source and target domains. Extensive experiments and visualization demonstrate effectiveness of these modules. We achieve promising results in unpaired-image translation tasks, and enable interesting applications based on spontaneous motion., Comment: Accepted by ICCV 2019

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work., QC 20180920

Embedded Control Systems (ECS) are getting increasingly complex for the realization of Cyber-Physical Systems (CPS) with advanced autonomy (e.g. autonomous driving of cars). This compromises system dependability, especially when components developed separately are integrated. Under the circumstance, dynamic anomaly detection and risk management often become a necessary means for compensating the insufficiencies of conventional verification and validation, and architectural solutions (e.g. hardware redundancy). The aim of this work is to support the design of embedded software services for dynamic anomaly detection of components in ECS, through probabilistic inference methods (e.g. Hidden Markov Model - HMM). In particular, the work provides a method for classifying the signal features of operational sensors and thereby applies Monte-Carlo sensitivity analysis for eliciting the probabilistic properties for error estimation. Such approach, based upon a physical model, reduces the dependency on empirical data for bringing about confidence on newly developed components., QC 20181001

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work., QC 20180920

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work., QC 20180920

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work., QC 20180920

The complexity of automated driving poses challenges for providing safety assurance. Focusing on the architecting of an Autonomous Driving Intelligence (ADI), i.e. the computational intelligence, sensors and communication needed for high levels of automated driving, we investigate so called safety supervisors that complement the nominal functionality. We present a problem formulation and a functional architecture of a fault-tolerant ADI that encompasses a nominal and a safety supervisor channel. We then discuss the sources of hazardous events, the division of responsibilities among the channels, and when the supervisor should take over. We conclude with identified directions for further work., QC 20180920