Search

Face parsing aims to assign pixel-wise semantic labels to different facial components (e.g., hair, brows, and lips) in given face images. However, directly predicting pixel-level labels for each facial component over the whole face image would obtain limited accuracy, especially for tiny facial components. To address this problem, some recent works propose to first crop tiny patches from the whole face image and then predict masks for each facial component. However, such cropping-and-segmenting strategy consists of two independent stages, which cannot be jointly optimized. Besides, as one valuable piece of information for parsing the highly structured facial components, context cues are not elaborately explored by the existing works. To address these issues, we propose a component-level refinement network (CLRNet) for precisely segmenting out each facial component. Specifically, we introduce an attention mechanism to bridge the two independent stages together and form an end-to-end trainable pipeline for face parsing. Furthermore, we incorporate the global context information into the refining process for each cropped facial component patch, providing informative cues for accurate parsing. Extensive experiments are carried out on two benchmark datasets, LFW-PL and HELEN. The results demonstrate the superiority of the proposed CLRNet over other state-of-the-art methods, especially for tiny facial components.

Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections is limited to general observations using population analysis. In this paper, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject, for specified age-timepoints.To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (cycleGAN) that translates features of neuronal structures in a region, across age-timepoints, for large brain microscopy volumes. We improve the reconstruction quality of neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss.Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. We show that neuReGANerator has a reconstruction accuracy of 94% in predicting neuronal structures. Finally, to visualize the predicted change in projections, NeuRegenerate offers two modes: (1) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, across the age timepoints, and (2) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes in neuronal fibers within the cholinergic system of the mouse brain between a young and old specimen., Comment: Accepted for publication in IEEE Transactions on Visualization and Computer Graphics

Object detection is an essential task in computer vision and image processing. It has many applications in various domains like medical diagnosis, civil military, video surveillance, security, etc. In some vision-related approaches, object detection used as an integral part, such as semantic segmentation, instance segmentation, pose estimation, suspicious activity detection, etc. The first stage in the pipeline is to detect an object. The survey begins with significant highlights of deep learning for object detection. It provides a comprehensive study on object representation; Convolution Neural Network (CNN) and different Deep Convolution Neural Network architecture. It presents a concise review of renowned datasets and definitive measurement metrics, forming the primitive baseline to evaluate the detection framework. Then studies in detail on detection framework one-stage and two-stage detectors and evaluates each framework with standard datasets listing its vital significance. The study also explores different issues of object detection like multi-scale, intra-class variations, generalization & security. Moreover, lists the primitive steps for creating object detector for different conditions from the reviewed survey. Finally, it proposes promising research directions.

Six degrees of freedom (6D) pose estimation or measurement of target objects are important robotic vision techniques that are key aspects for achieving autonomously executed robot tasks. The existing techniques solve this problem well for ordinary textured or rough-surfaced objects; however, 6D pose estimation remains a challenge for textureless, shiny metal parts in industry. This article proposes both a new framework for performing 6D pose estimation and a practical method based on the proposed framework that outperforms the state-of-the-art methods. All the existing methods follow an image-to-feature framework: they first extract features from images and then use the features to infer object poses. In contrast, our idea is a novel generative feature-to-image framework based on generative models, whose pipeline is a reverse mapping from feature to image. In other words, given a feature representing a pose, our method generates an image of the object in the exact same pose. Based on this framework, we propose a specific algorithm to indirectly infer the pose, which enables an initially random answer to gradually be regressed to the correct result. The experimental results demonstrate that the proposed deep learning method possesses greater robustness and achieves higher accuracy than the traditional methods

With the development of sequencing technology, microbiological genome sequencing analysis has attracted extensive attention. For inexperienced users without sufficient bioinformatics skills, making sense of sequencing data for microbial identification, especially for bacterial identification, through reads analysis is still challenging. In order to address the challenge of effectively analyzing genomic information, in this paper, we develop an effective approach and automatic bioinformatics pipeline called PBGI for bacterial genome identification, performing automatedly and customized bioinformatics analysis using short-reads or long-reads sequencing data produced by multiple platforms such as Illumina, PacBio and Oxford Nanopore. An evaluation of the proposed approach on the practical data set is presented, showing that PBGI provides a user-friendly way to perform bacterial identification through short or long reads analysis, and could provide accurate analyzing results. The source code of the PBGI is freely available at https://github.com/lyotvincent/PBGI.

Knitting can efficiently fabricate stretchable and durable soft surfaces. These surfaces are often designed to be worn on solid objects as covers, garments, and accessories. Given a 3D model, we consider a knit for it wearable if the knit not only reproduces the shape of the 3D model but also can be put on and taken off from the model without deforming the model. This "wearability" places additional constraints on surface design and fabrication, which existing machine knitting approaches do not take into account. We introduce the first practical automatic pipeline to generate knit designs that are both wearable and machine knittable. Our pipeline handles knittability and wearability with two separate modules that run in parallel. Specifically, given a 3D object and its corresponding 3D garment surface, our approach first converts the garment surface into a topological disc by introducing a set of cuts. The resulting cut surface is then fed into a physically-based unclothing simulation module to ensure the garment's wearability over the object. The unclothing simulation determines which of the previously introduced cuts could be sewn permanently without impacting wearability. Concurrently, the cut surface is converted into an anisotropic stitch mesh. Then, our novel, stochastic, any-time flat-knitting scheduler generates fabrication instructions for an industrial knitting machine. Finally, we fabricate the garment and manually assemble it into one complete covering worn by the target object. We demonstrate our method's robustness and knitting efficiency by fabricating models with various topological and geometric complexities. Further, we show that our method can be incorporated into a knitting design tool for creating knitted garments with customized patterns.

Sand deposition in pipelines during crude oil transportation is a serious industrial problem; these problems often lead to extra expenses in the course of transportation, causing a threat to the environment and economic damage. The intent of this study is to model the minimum condition required to transport sand particles as dispersed phase (Suspended particles) in flowing crude in pipelines. A mechanistic approach based on force balance and particle trajectory of Lagrangian Multiphase modeling was used to develop a model for determining the minimum transport condition for sand flowing in petroleum to avoid sand trapping along the bed of the pipeline. The model developed is a quadratic function of slip velocity and the findings was compared with experimental data and other three models and found to compete favorably. Consistency exists between the predicted outcomes and the experimental results with a relative error of 1.1%. This model will be useful for oil and gas industries worldwide for accurate prediction of critical suspending velocities of petroleum containing sand and other tiny particles to avoid deposition of particles along the pipeline and eliminate corrosion-related danger in the pipelines caused by the particles.

Realistic 3D facial modeling and animation have been increasingly used in many graphics, animation, and virtual reality applications. However, generating realistic fine-scale wrinkles on 3D faces, in particular, on animated 3D faces, is still a challenging problem that is far away from being resolved. In this article we propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine-scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, we implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. We also introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. Through many experiments, we demonstrate our method can robustly synthesize plausible fine-scale wrinkles on a variety of coarse-scale 3D faces with different shapes and expressions.

With the advent of the next-generation traffic monitoring systems, there has been a significant increase in the spatial-temporal resolution of vehicle mobility data in many cities. Effective analysis and visualization of such data can provide transportation planners with data-driven insights, which can facilitate the understanding of multiscale traffic dynamics. In this paper, we present a web-based traffic emulator for emulating and visualizing near-real-time and historical traffic flows on highways using data from road-side sensors. To construct a continuous traffic flow, the emulator adopts an analytical pipeline that can (a) integrate traffic data collected from discrete road-side radar detection sensors, (b) interpolate traffic conditions (vehicle speed and volume) on unmeasured road segments based on traffic flow theory, and (c) generate lane-specific vehicle trajectories and movements using a mathematically optimized representation of the road network. Our app also provides an integrated visual workflow that allows users to explore the interconnected traffic dynamics using an appropriate traffic flow visualization selected based on the level of detail. We devise two innovative geo-visualization techniques that utilize an animated strips-network representation and a lane usage matrix to visualize lane performances. To ensure a smooth emulation of large-scale traffic flow in an easy-to-access web environment, we implement the emulator using client-side GPU-accelerated techniques. Finally, we close with a case study that visualizes traffic dynamics of two scenarios - an afternoon peak hour and a traffic accident - in Chattanooga, Tennessee. Our app visualizes the responses of traffic dynamics during different traffic conditions, and to the presence of the traffic accident at different spatial scales.

In terms of pipeline leak detection, the unavoidable fact is that existing data could not provide enough effective leak data to train a high accuracy model. To address this issue, this article proposes mixed generative adversarial networks (mixed-GANs) as a practical way to provide additional data, ensuring data reliability. First, multitype generative networks with heterogeneous parameter-updating mechanisms are designed to explore a variety of different solutions and eliminate the potential risks of instable training and scenario collapse. Then, based on expert experience, two data constraints are proposed to describe leak characteristics and further evaluate the quality of generated leak data in the training process. Through integrating the particle swarm optimization algorithm into generative model training, mixed-GAN has better generation performance than the conventional gradient descent algorithm. Based on the above-mentioned contents, the proposed model is able to provide satisfactory leak data with different scenarios, contributing to data quantity expansion, data credibility enhancement, and data variety enrichment. Finally, extensive experiments are given to illustrate the effectiveness of the proposed generative model for pipeline network leak detection.

Buried natural gas pipelines are vulnerable to external corrosion because they are encased in a soil environment for a long time. Identifying the causes of external corrosion and taking specific maintenance measures is essential. In this work, a risk analysis and maintenance decision-making model for natural gas pipelines with external corrosion is proposed based on a Bayesian network. A fault tree model is first employed to identify the causes of external corrosion. The Bayesian network for risk analysis is determined accordingly. The maintenance strategies are then inserted into the Bayesian network to show a reduction of the risk. The costs of maintenance strategies and the reduced risk after maintenance are combined in an optimization function to build a decision-making model. Because of the limitations of historical data, some of the parameters in the Bayesian network are obtained from a probabilistic estimation model, which combines expert experience and fuzzy set theory. Finally, a case study is carried out to verify the feasibility of the maintenance decision model. This indicates that the method proposed in this work can be used to provide effective maintenance schemes for different pipeline external corrosion scenarios and to reduce the possible losses caused by external corrosion.

Deep neural network (DNN) based video analysis has become one of the most essential and challenging tasks to capture implicit information from video streams. Although DNNs significantly improve the analysis quality, they introduce intensive compute and memory demands and require dedicated hardware for efficient processing. The customized heterogeneous system is one of the promising solutions with general-purpose processors (CPUs) and specialized processors (DNN Accelerators). Among various heterogeneous systems, the combination of CPU and FPGA has been intensively studied for DNN inference with improved latency and energy consumption compared to CPU + GPU schemes and with increased flexibility and reduced time-to-market cost compared to CPU + ASIC designs. However, deploying DNN-based video analysis on CPU + FPGA systems still presents challenges from the tedious RTL programming, the intricate design verification, and the time-consuming design space exploration. To address these challenges, we present a novel framework, called EcoSys, to explore co-design and optimization opportunities on CPU-FPGA heterogeneous systems for accelerating video analysis. Novel technologies include 1) a coherent memory space shared by the host and the customized accelerator to enable efficient task partitioning and online DNN model refinement with reduced data transfer latency; 2) an end-to-end design flow that supports high-level design abstraction and allows rapid development of customized hardware accelerators from Python-based DNN descriptions. 3) a design space exploration (DSE) engine that determines the design space and explores the optimized solutions by considering the targeted heterogeneous system and user-specific constraints; and 4) a complete set of co-optimization solutions, including a layer-based pipeline, a feature map partition scheme, and an efficient memory hierarchical design for the accelerator and multi-threading programming for the CPU. In this paper, we demonstrate our design framework to accelerate the long-term recurrent convolution network (LRCN), which analyzes the input video and output one semantic caption for each frame. EcoSys can deliver 314.7 and 58.1 frames per second (FPS) by targeting the LRCN model with AlexNet and VGG-16 backbone, respectively. Compared to the multithreaded CPU and pure FPGA design, EcoSys achieves 20.6× and 5.3× higher throughput performance.

There are situations in offshore energy development where potential impact forces between submarine slides and pipelines need to be estimated. The horizontal slide – pipeline impact force, parallel to the main travel direction of the sliding mass and normal to the pipeline axis, is generally dominant compared with other force components and hence of particular concern. In practice, pipelines may be suspended at varying distances above the seabed (gap) and existing methods do not consider how this will affect the horizontal slide – pipeline forces. This paper investigates the effects of pipeline–seabed gap and pipeline diameter on the horizontal slide – pipeline impact force via 181 computational fluid dynamics (CFD) simulations at Reynolds numbers of 0.36–287. Results show that variation in the pipeline–seabed gap and pipeline diameter alters the slide mass flow behaviour as it flows past the pipeline and hence the impact force when the pipeline–seabed gap is below a critical value. A modified hybrid geotechnical – fluid dynamics framework for estimating the horizontal impact force is proposed by considering the effects of the pipeline–seabed gap and pipeline diameter, which is validated with existing experimental datasets.

Data visualization and interactive data exploration are important aspects of illustrating complex concepts and results from analyses of omics data. A suitable visualization has to be intuitive and accessible. Web-based dashboards have become popular tools for the arrangement, consolidation, and display of such visualizations. However, the combination of automated data processing pipelines handling omics data and dynamically generated, interactive dashboards is poorly solved. Here, we present i2dash, an R package intended to encapsulate functionality for the programmatic creation of customized dashboards. It supports interactive and responsive (linked) visualizations across a set of predefined graphical layouts. i2dash addresses the needs of data analysts/ software developers for a tool that is compatible and attachable to any R-based analysis pipeline, thereby fostering the separation of data visualization on one hand and data analysis tasks on the other hand. In addition, the generic design of i2dash enables the development of modular extensions for specific needs. As a proof of principle, we provide an extension of i2dash optimized for single-cell RNA sequencing (scRNA-seq) analysis, supporting the creation of dashboards for the visualization needs of scRNA-seq experiments. Equipped with these features, i2dash is suitable for extensive use in large-scale sequencing/bioinformatics facilities. Along this line, we provide i2dash as a containerized solution, enabling a straightforward large-scale deployment and sharing of dashboards using cloud services. i2dash is freely available via the R package archive CRAN (https://CRAN.R-project.org/package=i2dash).

In-hand object localization and manipulation has always been a challenging task in robotic community. In this paper, we address this problem by vision-based tactile sensing with high spatial resolution. Specifically, we design a novel tactile sensor based on stereo vision, named GelStereo, which can perceive tactile point cloud with high spatial resolution (< 1 mm). A tactile-based in-hand object localization pipeline composed of saliency detection and probabilistic point-set registration algorithms of the perceived contact point cloud is presented. Furthermore, extensive qualitative and quantitative analyses of perceived tactile point cloud and in-hand localization and insertion experiments of small parts are performed on our robot platform. The experimental results verify the accuracy and robustness of the tactile point cloud sensed by the novel GelStereo tactile sensor and the proposed in-hand object localization pipeline. This novel high-resolution visuotactile sensing technology has predictable application potential in the field of dexterous robotic manipulation.

Currently, the world is in the midst of a severe global pandemic, which has affected all aspects of people's lives. As a result, there is a deluge of COVID-related digital media articles published in the United States, due to the disparate effects of the pandemic. This large volume of information is difficult to consume by the audience in a reasonable amount of time. In this paper, we develop a Natural Language Processing (NLP) pipeline that is capable of automatically distilling various digital articles into manageable pieces of information, while also modelling the progression topics discussed over time in order to aid readers in rapidly gaining holistic perspectives on pressing issues (i.e., the COVID-19 pandemic) from a diverse array of sources. We achieve these goals by first collecting a large corpus of COVID-related articles during the onset of the pandemic. After, we apply unsupervised and semi-supervised learning procedures to summarize articles, then cluster them based on their similarities using the community detection methods. Next, we identify the topic of each cluster of articles using the BART algorithm. Finally, we provide a detailed digital media analysis based on the NLP-pipeline outputs and show how the conversation surrounding COVID-19 evolved over time.

Wax sedimentation in pipelines is a severe crude oil production and transportation challenge. Pipeline surface roughness is experienced at the early stages of the problem; with time, the effective pipe cross-sectional area is reduced due to pipeline wax plugging, causing pumping pressure increase, equipment failures, and blockages, resulting in unnecessary downtime costs and pipeline abandonments in the worst situation. This paper reviews mathematical and experimental loops models used for pipeline solid wax predictions and calculations as functions of pressure, temperature, and fluid composition; by assessing model's Assumptions, strengths and weaknesses. It is found that most mathematical models applied molecular-diffusion mechanisms in modeling and neglected shear effects; which resulted in wax over-prediction. Experimental loop was time-consuming due to mounting and dismounting of test section during wax deposition measurements; our modification has included sensor-integration to detect, measure, and analyze wax deposition; Reliable wax predictions models are essential to properly design pipelines and adopt cost-effective strategies for wax deposition prevention, control, and removal.

Climate change is one of the greatest challenges for modern societies. Its consequences, often associated with extreme events, have dramatic results worldwide. New synergies between different disciplines including Artificial Intelligence (AI), Internet of Things (IoT), and edge computing can lead to radically new approaches for the real-time tracking of natural disasters that are also designed to reduce the environmental footprint. In this article, we propose an AI-based pipeline for processing natural disaster images taken from drones. The purpose of this pipeline is to reduce the number of images to be processed by the first responders of the natural disaster. It consists of three main stages, (1) a lightweight auto-encoder based on deep learning, (2) a dimensionality reduction using the t-SNE algorithm and (3) a fuzzy clustering procedure. This pipeline is evaluated on several edge computing platforms with low-power accelerators to assess the design of intelligent autonomous drones to provide this service in real time. Our experimental evaluation focuses on flooding, showing that the amount of information to be processed is substantially reduced whereas edge computing platforms with low-power GPUs are placed as a compelling alternative for processing these heavy computational workloads, obtaining a performance loss of only 2.3x compared to its cloud counterpart version, running both the training and inference steps.

We observe that data access and processing takes a significant amount of time in large-scale deep learning training tasks (DLTs) on image datasets. Three factors contribute to this problem: (1) the massive and recurrent accesses to large numbers of small files; (2) the repeated, expensive decoding computation on each image, and (3) the frequent communication between computation nodes and storage nodes. Existing work has addressed some aspects of these problems; however, no end-to-end solutions have been proposed. In this article, we propose DIESEL+, an all-in-one system which accelerates the entire I/O pipeline of deep learning training tasks. DIESEL+ contains several components: (1) local metadata snapshot; (2) per-task distributed caching; (3) chunk-wise shuffling; (4) GPU-assisted image decoding and (5) online region-of-interest (ROI) decoding. The metadata snapshot removes the bottleneck on metadata access in frequent reading of large numbers of files. The per-task distributed cache across the worker nodes of a DLT task to reduce the I/O pressure on the underlying storage. The chunk-based shuffle method converts small file reads into large chunk reads, so that the performance is improved without sacrificing the training accuracy. The GPU-assisted image decoding and the online ROI method minimize the image decoding workloads and reduce the cost of data movement between nodes. These techniques are seamlessly integrated into the system. In our experiments, DIESEL+ outperforms existing systems by a factor of two to three times on the overall training time.

Traditionally, reduced hair simulation methods are either restricted to heuristic approximations or bound to specific hairstyles. We introduce the first CNN-integrated framework for simulating various hairstyles. The approach produces visually realistic hairs with an interactive speed. To address the technical challenges, our hair simulation pipeline is designed as a two-stage process. First, we present a fully-convolutional neural interpolator as the backbone generator to compute dynamic weights for guide hair interpolation. Then, we adopt a second generator to produce fine-scale displacements to enhance the hair details. We train the neural interpolator with a dedicated loss function and the displacement generator with an adversarial discriminator. Experimental results demonstrate that our method is effective, efficient, and superior to the state-of-the-art on a wide variety of hairstyles. We further propose a performance-driven digital avatar system and an interactive hairstyle editing tool to illustrate the practical applications.

Water distribution and transmission lines are indispensable to urban infrastructure. The water pipelines are subject to both structural and functional deterioration due to various reasons including aging, negligence, and high demand for water supply. Hence, to ensure a safe and reliable water supply, the water utilities need to perform routine pipe condition assessments. The condition assessment is usually carried out by visual inspection with the machine vision system carried by a robotic platform. The inspection platforms will capture the internal condition of the water pipelines in a video stream. However, the robotic platform frequently experiences difficulties while traversing through the valves installed along the pipeline. This inhibits and disrupts the inspection process of the water pipelines. Therefore, this paper proposes a deep learning-based automatic valve detection framework to facilitate the robot's navigation and ensure continuous inspection without any interruptions. The valve detection model is developed by combining MobileNet-160 and Feature Pyramid Network (FPN) and is named as MFPN. The developed framework also employs a generative adversarial network to solve the sparse data set issues and improve the generality of the framework. The comparative study and ablation analyses demonstrate that the proposed framework can achieve a higher mAP value of 89.11% in comparison with the state-of-the-art. Hence, this light-weight and efficient solution can be deployed to the robotic platform for real-time valve detection and enable autonomous navigation of the robotic platform for condition assessment of water pipelines.

Measuring user engagement in interactive tasks can facilitate numerous applications toward optimizing user experience, ranging from eLearning to gaming. However, a significant challenge is the lack of non-contact methods that are robust in unconstrained environments. We present FaceEngage, a non-intrusive engagement estimator leveraging user facial recordings during actual gameplay in naturalistic conditions. Our contributions are three-fold. First, we show the potential of using front-facing videos as training data to build the engagement estimator. We compile FaceEngage Dataset with over 700 user-contributed YouTube picture-in-picture gaming videos (i.e., with full-screen game scenes and time-synchronized user facial recordings in subwindows). Second, we develop FaceEngage system which captures relevant gamer facial features from front-facing recordings to infer task engagement. We implement two pipelines: an estimator trained on user facial motion features inspired by prior psychological works, and a deep learning-enabled estimator. Lastly, we conduct extensive experiments and conclude: (i) certain user facial motion cues (e.g., blink rates) are engagement-indicative; (ii) our deep learning-enabled pipeline can automatically extract informative features, outperforming the facial motion feature-based pipeline; (iii) FaceEngage is robust to various video lengths and users/game genres. Despite the challenging nature of realistic videos, FaceEngage attains the accuracy of 83.8% and leave-one-user-out precision of 79.9%, both of which are superior to the face motion-based pipeline.

Pipeline systems in naval ships with special combat purposes (such as warships) are frequently exposed to the risk of damage. It is necessary to detect and isolate such unexpected damage quickly via an autonomous recovery system in the pipelines to ensure the ships survivability. This recovery system employs an autonomously controlled valve, which consists of a valve body, two pressure sensors, an actuator to open/close the valve, and a controller equipped with damage detection capability and a control algorithm. To enhance the reliability of the autonomous recovery system, this study first proposes a multistage control (MSC) algorithm that can comprehensively consider the various damage scenarios in a naval ship and then suggests the design procedure for the autonomous piping system based on support vector machine (SVM). To design the suggested algorithm, a simulation model for the one-dimensional real-time flow analysis is first established, and its results for possible damage scenarios are used as training data for the SVM with the valve operation information for recovery of the piping system and to minimize the damaged area. A performance comparison of the representative control algorithms for the autonomous piping system is conducted via flow simulations under three representative damage scenarios. Finally, a real-scale testbed of the firemain in a naval ship is designed and constructed, and the effectiveness of the algorithm is demonstrated through experiments.

Network intrusion detection systems (NIDS) are popular tools to defend local networks against attacks. These systems monitor the network traffic and flag suspicious behavior. Rule-based NIDS do that by checking the network traffic against a set of rules, which become obsolete as attackers learn new strategies to circumvent existing defenses. This article proposes synthesis of suricata rules (Syrius), a novel approach to synthesize rules for rule-based NIDS. Syrius leverages malicious (positive) and benign (negative) traffic to create rules for new attacks. Syrius is organized as a pipeline of three components to 1) create an overspecified seed rule, 2) derive plausible rules from the seed, and 3) rank plausible rules. We evaluated Syrius against a set of 21 network attacks with various characteristics. Syrius was capable of generating the correct rule among the top-3 and top-1 rules of the ranking, respectively, in 80.1% and 47.6% of the cases.

Deep neural network (DNN) is applied widely in many applications and achieves state-of-the-art performance. However, DNN lacks transparency and interpretability for users in structure. Attackers can use this feature to embed trojan horses in the DNN structure, such as inserting a backdoor into the DNN, so that DNN can learn both the normal main task and additional malicious tasks at the same time. Besides, DNN relies on data set for training. Attackers can tamper with training data to interfere with DNN training process, such as attaching a trigger on input data. Because of defects in DNN structure and data, the backdoor attack can be a serious threat to the security of DNN. The DNN attacked by backdoor performs well on benign inputs while it outputs an attacker-specified label on trigger attached inputs. Backdoor attack can be conducted in almost every stage of the machine learning pipeline. Although there are a few researches in the backdoor attack on image classification, a systematic review is still rare in this field. This paper is a comprehensive review of backdoor attacks. According to whether attackers have access to the training data, we divide various backdoor attacks into two types: poisoning-based attacks and non-poisoning-based attacks. We go through the details of each work in the timeline, discussing its contribution and deficiencies. We propose a detailed mathematical backdoor model to summary all kinds of backdoor attacks. In the end, we provide some insights about future studies.

The development of Science, Technology, Engineering and Math (STEM) professionals is a key focus for STEM education and outreach programs. These efforts are critical for our society to grow and resolve problems. STEM programs service kindergarten through graduate students in the hopes of bringing STEM professionals into the workforce. As youth move through these grade levels, different influencing factors play a role in guiding these students towards a STEM career. This establishes the career pipeline within the STEM system. STEM outreach has the advantage of targeting specific grades and demographics of students through this pipeline. This work looks to explore the efforts conducted through STEM outreach to fulfill this pipeline at critical grade levels. This is accomplished by utilizing currently active STEM professionals in the workforce. A methodology to identify and characterize critical STEM development stages is presented. Based on surveys conducted with current STEM professionals, findings are presented identifying key success indicators. These indicators are based on their personal pipeline experience in becoming a STEM professional. This work establishes stages within the STEM career pipeline. New and novel insights are gained related to changes in students mindsets towards a STEM career within the K-12 grades. Along with understanding of these mindsets the timing and rapidness of mindset shifts are detailed and provide a unique understanding of how students move through K-12 towards a STEM career. A significant finding is related to evidence that students become interested in science and engineering, can see science and engineering as an option and choose to become a scientist or engineer over a very short timeframe. Further this shift in mindset can occur as early as kindergarten. Based on these findings changes of how current STEM outreach activities are conducted is needed to meet the students needs.

Experimental hardware-research interfaces form a crucial role during developmental stages of any medical, signal-monitoring system as it allows researchers to test and optimize output results before perfecting the design for the actual FDA approved medical device and large-scale production. These testing platforms, intake raw signals through which performance of novel algorithms can be analyzed and modified to generate the desired data points for an optimized output, allowing the advancement of the medical device. With cochlear implants (CIs) and hearing aids (HAs) becoming a more common solution for varying degrees of hearing impairment, having modern signal processing strategies tested for such speech sensitive systems is a necessity. But the rigid design requirements of commercial CI and HA processors make it difficult to explore novel algorithms for research investigations and conducting longitudinal studies. This study presents the design, development, clinical evaluation, and applications of CCi-MOBILE, a computationally powerful signal processing testing platform built for researchers in the hearing-impaired field. The custom-made, portable research platform allows researchers to design and perform complex speech processing algorithm assessment offline and in real-time. It can be operated through user-friendly, open-source software and is compatible with implants manufactured by Cochlear Corporation. The FPGA design and hardware processing pipeline for CI stimulation is discussed followed by results from an acute study with implant users’ speech intelligibility in quiet and noisy conditions. The results show a consistent level of performance compared with CI users’ clinical processor, thus confirming the viability of the platform in chronic CI based studies.

Hearing loss is a serious condition affecting more than 1.5 billion people globally. Many affected people benefit from the use of devices, such as hearing aids, but these tools do not restore natural hearing, and many users still struggle to follow speech in the presence of background noise. Consequently, there is rapid growth in work to discover therapeutics to address this need. Our analysis of the therapeutic pipeline for inner ear and central processing disorders identified 23 assets in clinical trials and 56 in preclinical development, of which 25% have entered the pipeline in the past three years. The innovative potential of this pipeline is encouraging, but there are translational hurdles to be overcome. We highlight challenges for the pipeline and comment on opportunities to support and strengthen it.

Hydrogen is regarded as a key component of future low-carbon energy systems. Yet, for the implementation of hydrogen technologies on a large scale it is necessary to consider social acceptance. Studies on acceptance of hydrogen technologies have resulted in neutral to positive evaluations. This paper extends previous studies by focusing on large-scale hydrogen infrastructure. Within the project ELEGANCY, quantitative data on the acceptance was gathered among the German population. The results confirm the positive perception of hydrogen on a general level. However, the high level of acceptance is decreasing when it comes to infrastructure implementation in the own neighbourhood. In this paper, the gap between acceptance of hydrogen technology on a general level and the level of its infrastructural implementation is analysed, focusing on new pipeline infrastructure. The results show that sociodemographic factors, especially the variable age, as well as project-related factors, especially trust in stakeholders, have a high explanatory power.

This paper presents DistRDF2ML, the generic, scalable, and distributed framework for creating in-memory data preprocessing pipelines for Spark-based machine learning on RDF knowledge graphs. This framework introduces software modules that transform large-scale RDF data into ML-ready fixed-length numeric feature vectors. The developed modules are optimized to the multimodal nature of knowledge graphs. DistRDF2ML provides aligned software design and usage principles as common data science stacks that offer an easy-to-use package for creating machine learning pipelines. The modules used in the pipeline, the hyper-parameters and the results are exported as a semantic structure that can be used to enrich the original knowledge graph. The semantic representation of metadata and machine learning results offers the advantage of increasing the machine learning pipelines’ reusability, explainability, and reproducibility. The entire framework of DistRDF2ML is open source, integrated into the holistic SANSA stack, documented in scala-docs, and covered by unit tests. DistRDF2ML demonstrates its scalable design across different processing power configurations and (hyper-)parameter setups within various experiments. The framework brings the three worlds of knowledge graph engineers, distributed computation developers, and data scientists closer together and offers all of them the creation of explainable ML pipelines using a few lines of code.

Perception, localization, planning, and control, high-level functions often organized in a so-called pipeline, are amongst the core building blocks of modern autonomous (ground, air, and underwater) vehicle architectures. These functions are increasingly being implemented using learning-enabled components (LECs), i.e., (software) components leveraging knowledge acquisition and learning processes such as deep learning. Providing quantified component-level assurance as part of a wider (dynamic) assurance case can be useful in supporting both pre-operational approval of LECs (e.g., by regulators), and runtime hazard mitigation, e.g., using assurance-based failover configurations. This paper develops a notion of assurance for LECs based on i) identifying the relevant dependability attributes, and ii) quantifying those attributes and the associated uncertainty, using probabilistic techniques. We give a practical grounding for our work using an example from the aviation domain: an autonomous taxiing capability for an unmanned aircraft system (UAS), focusing on the application of LECs as sensors in the perception function. We identify the applicable quantitative measures of assurance, and characterize the associated uncertainty using a non-parametric Bayesian approach, namely Gaussian process regression. We additionally discuss the relevance and contribution of LEC assurance to system-level assurance, the generalizability of our approach, and the associated challenges., Comment: 8 pp, 4 figures, Appears in the proceedings of EDCC 2019

In this paper, we design novel interactive deep learning methods to improve semantic interactions in visual analytics applications. The ability of semantic interaction to infer analysts' precise intents during sensemaking is dependent on the quality of the underlying data representation. We propose the $\text{DeepSI}_{\text{finetune}}$ framework that integrates deep learning into the human-in-the-loop interactive sensemaking pipeline, with two important properties. First, deep learning extracts meaningful representations from raw data, which improves semantic interaction inference. Second, semantic interactions are exploited to fine-tune the deep learning representations, which then further improves semantic interaction inference. This feedback loop between human interaction and deep learning enables efficient learning of user- and task-specific representations. To evaluate the advantage of embedding the deep learning within the semantic interaction loop, we compare $\text{DeepSI}_{\text{finetune}}$ against a state-of-the-art but more basic use of deep learning as only a feature extractor pre-processed outside of the interactive loop. Results of two complementary studies, a human-centered qualitative case study and an algorithm-centered simulation-based quantitative experiment, show that $\text{DeepSI}_{\text{finetune}}$ more accurately captures users' complex mental models with fewer interactions.

How functional MRI (fMRI) data are analyzed depends on the researcher and the toolbox used. It is not uncommon that the processing pipeline is rewritten for each new dataset. Consequently, code transparency, quality control and objective analysis pipelines are important for improving reproducibility in neuroimaging studies. Toolboxes, such as Nipype and fMRIPrep, have documented the need for and interest in automated analysis pipelines. Here, we introduce fMRIflows: a consortium of fully automatic neuroimaging pipelines for fMRI analysis, which performs standard preprocessing, as well as 1st- and 2nd-level univariate and multivariate analysis. In addition to the standardized processing pipelines, fMRIflows also provides flexible temporal and spatial filtering to account for datasets with increasingly high temporal resolution and to help appropriately prepare data for multivariate analysis and improve signal decoding accuracy. This paper first describes fMRIflows’ structure and functionality, then explains its infrastructure and access, and lastly validates the toolbox by comparing it to other neuroimaging processing pipelines such as fMRIPrep, FSL and SPM. This validation was performed on three datasets with varying temporal resolution to ensure flexibility and robustness, as well as to showcase the improved capability of fMRIflows. The outcome of the validation analysis shows that fMRIflows preprocessing pipeline performs comparably to the ones obtained from other toolboxes. Importantly, fMRIflows’ flexible temporal filtering approach leads to improved signal-to-noise-ratio after preprocessing and increased statistical sensitivity, particularly in datasets with high temporal resolution. fMRIflows is a fully automatic fMRI processing pipeline which uniquely offers univariate and multivariate single-subject and group analyses as well as preprocessing. It offers flexible spatial and temporal filtering and thereby leads to more pronounced results for datasets with temporal resolutions at or below 1000ms.

Carbohydrate chains are ubiquitous in the complex molecular processes of life. These highly diverse chains are recognized by a variety of protein receptors, enabling glycans to regulate many biological functions. High-resolution structures of protein-glycoligand complexes reveal the atomic details necessary to understand this level of molecular recognition and inform application-focused scientific and engineering pursuits. When experimental challenges hinder high-throughput determination of quality structures, computational tools can, in principle, fill the gap. In this work, we introduce GlycanDock, a residue-centric protein-glycoligand docking refinement algorithm developed within the Rosetta macromolecular modeling and design software suite. We performed a benchmark docking assessment using a set of 109 experimentally determined protein-glycoligand complexes as well as 62 unbound protein structures. The GlycanDock algorithm can sample and discriminate among protein-glycoligand models of native-like structural accuracy with statistical reliability from starting structures of up to 7 A root-mean-square deviation in the glycoligand ring atoms. We show that GlycanDock-refined models qualitatively replicated the known binding specificity of a bacterial carbohydrate-binding module. Finally, we present a protein-glycoligand docking pipeline for generating putative protein-glycoligand complexes when only the glycoligand sequence and unbound protein structure are known. In combination with other carbohydrate modeling tools, the GlycanDock docking refinement algorithm will accelerate research in the glycosciences.

This paper presents a systematic approach to evaluate the time interval of optimal maintenance strategy for the subsea process system influenced by Microbiological Influenced Corrosion (MIC) within multiple defects. The proposed method incorporates the non-homogeneous Poisson, homogeneous gamma, and non-homogeneous Markov processes for modeling the generation of multiple defects, the average pit depth growth, and maximum pit depth, respectively. The maintenance strategy comprises industrial procedure, probability of failure detection, errors sizing in-line inspection tools, management actions costs, and failure cost. The developed framework simulates maintenance strategies considering time interval, cost, probability of detection, average pit depth, and maximum pit depth and identifies the optimal strategy. The practical application is demonstrated in a North Sea subsea pipeline system under MIC’s influence. This work assists decision-makers in selecting the optimal conditioned-based maintenance strategy for the processing system. While the application is demonstrated to subsea process systems under MIC influence, the developed approach is equally applicable to other process systems.

Many decisions in a machine learning pipeline involve non-differentiable and discontinuous objectives and search spaces. Examples include feature selection, model selection and hyper-parameter tuning, where candidate solutions in an outer optimization loop must be evaluated via a learning subsystem. Evolutionary algorithms (EAs) are prominent gradient-free methods to handle such tasks. However, EAs are known to pose steep computational challenges, especially when dealing with large-instance datasets. As opposed to prior works that often fall back on parallel computing hardware to resolve this big data problem of EAs, in this paper we propose a novel algorithm-centric solution based on evolutionary multitasking. Our approach involves the creation of a band of minions, i.e., small data proxies to the main target task, that are constructed by subsampling a fraction of the large dataset. We then combine the minions with the main task in a single multi-task optimization framework, boosting evolutionary search by using small data to quickly optimize for the large dataset. Our key algorithmic contribution in this setting is to allocate computational resources to each of the tasks in a principled manner. The paper considers wrapper-based feature selection as an illustrative case study of the broader idea of using multitasking to speedup outer-loop evolutionary configurations of any machine learning subsystem. The experiments reveal that multitasking can indeed speedup baseline EAs, by more than 40% on some datasets.

Water pipelines are one of the most vital structures to convey fresh water for consumption or irrigation over long distances. However, the major problem of water transportation pipeline is leak which can cause water resources loss, possible human injuries, and environmental pollution problems. For that, these pipelines must be carefully and real-time monitored. In this work, we describe a comprehensive framework for existing monitoring water pipeline techniques based on both wired and wireless networks. Furthermore, we elaborate a comparative overview in terms of keys parameters touching monitoring models. After the evaluation of all the existing pipeline monitoring methods, it is much more evident that techniques based on wireless sensors networks have variety and are the best selection for water pipeline monitoring purposes. This study is a guideline to select which model is the most suitable to design a new pipeline monitoring scheme.

Skeleton creation is an important phase in the character animation pipeline. However, handcrafting skeleton takes extensive labor time and domain knowledge. Automatic skeletonization provides a solution. However, most of the current approaches are far from real-time and lack the flexibility to control the skeleton complexity. In this paper, we present an efficient skeletonization method, which can be seamlessly integrated into the sketch-based modeling process in real-time. The method contains three steps: local sub-skeleton extraction; sub-skeleton connection; and global skeleton refinement. Firstly, the local skeleton is extracted from the processed polygon stroke and forms a subpart along with the sub-mesh. Then, local sub-skeletons are connected according to the intersecting relationships and the modeling sequence of subparts. Lastly, a global refinement method is proposed to give users coarse-to-fine control on the connected skeleton. We demonstrate the effectiveness of our method on a variety of examples created by both novices and professionals., Shape Modeling International 2021

The search for more realism in renderings led to increased interest in tabular bidirectional reflectance distribution function (BRDF) samples captured from real-world materials. This representation provides a high degree of realism at the cost of a long acquisition time and high storage space. Because of those drawbacks, the literature presents few BRDF databases captured from real materials, and the available ones present a limited number of samples. Aiming to create new and realistic-looking materials, this work proposes an approach to generate new BRDFs from a tabular BRDF database. Our goal is to provide a pipeline where the user expresses appearance necessities by selecting materials from a BRDF database. We generate an appearance-driven space shaped by the user’s interests, in which it is feasible to navigate and retrieve new tabular BRDFs. This new space should enable smooth navigation and, to provide visual richness and realism, not rely only on the materials selected by the user. To this end, we first preprocess a tabular BRDF database by keeping BRDF samples with relevant reflectance features. A low-dimensional representation of this preprocessed database is obtained by applying a dimensionality reduction method, and a clustering algorithm is applied to obtain clusters of materials with similar features. Based on the materials selected by the user, we define one or more clusters of materials to build the appearance-driven space. Any point in this lower-dimensional space can be mapped to the original BRDFs space. Therefore, new BRDFs that present similar properties to the ones which the user selected are created. We compare the performance of linear (Multidimensional Scaling - MDS) and nonlinear (Isometric Feature Mapping - ISOMAP) dimensionality reduction methods in this task and analyze the obtained materials.

The rapid development of Information-Centric Wireless Sensor Network (ICWSN) has solved the challenges of information transmission and processing caused by the accelerated growth of wearable devices and the wide deployment of the Internet of Things (IoT) recently. The privacy security is also a growing problem. The existing works use earphones, covert and user-friendly wearable devices, for user authentication. However, some of the earphone-based authentication solutions need to customize special earphones, which are not universal. Other solutions use microphones and speakers of earphones for authentication, which are susceptible to changes in the auricle’s internal environment, resulting in a decline in performance. To solve this problem, a new authentication solution based on the existing commercial earphones is proposed to authenticate a user by tapping on the earphone rhythmically. This rhythmic tap behaviour causes a change of the signal waveform of the built-in accelerometer in the earphone. Based on this, we design a pipeline to authenticate the user’s identity. We first design an event detection algorithm to segment the tap signal accurately. Then we use the global features calculated based on event detection algorithm and local features extracted from the Convolutional Neural Network (CNN) for building an authentication model using the Naive Bayes (NB) classifier. Finally, 20 users are recruited to evaluate the experiment, and the recognition accuracy reaches 98%. Moreover, we extend the experiment to prove that it has a good performance against the different attacks and is robust in different scenarios.

Visualization collections, accessed by platforms such as Tableau Online or Power Bl, are used by millions of people to share and access diverse analytical knowledge in the form of interactive visualization bundles. Result snippets, compact previews of these bundles, are presented to users to help them identify relevant content when browsing collections. Our engagement with Tableau product teams and review of existing snippet designs on five platforms showed us that current practices fail to help people judge the relevance of bundles because they include only the title and one image. Users frequently need to undertake the time-consuming endeavour of opening a bundle within its visualization system to examine its many views and dashboards. In response, we contribute the first systematic approach to visualization snippet design. We propose a framework for snippet design that addresses eight key challenges that we identify. We present a computational pipeline to compress the visual and textual content of bundles into representative previews that is adaptive to a provided pixel budget and provides high information density with multiple images and carefully chosen keywords. We also reflect on the method of visual inspection through random sampling to gain confidence in model and parameter choices.

Recent advances and applications of aerial image semantic segmentation have yielded an increase in their use in day-to-day tasks. However, state-of-art algorithms, composed mostly of deep semantic segmentation networks (DSSNs), may not be suitable for application domains in which labels (targeted output masks) are scarce. This work proposes a fully unsupervised semantic segmentation method able to find the appropriate number of semantics labels (yielded from clustering) without the need of a previously annotated dataset. Once the semantic labels are identified, a classifier trained from these labels can be used to assign semantics to new input images. In contrast to other clustering-based segmentation methods, our method does not require any input parameter to find the clustering partition that best accommodates data. Empirical results on two real datasets showed that the use of semantics in a template-matching-based unmanned aerial vehicle (UAV) localization pipeline reduced the overall position estimation error.

Over the past decades, the successful invention and employment of multiple sensors have marked the advent of a new era in multisensor remote-sensing (RS) images acquisition. To effectively utilize the massive multisensor images for RS scene understanding, we expect that a scene classification model learned with particular sensor data can generalize well to other sensor data. However, this is a very challenging task due to the cross-sensor data differences. In the deep learning (DL) pipeline, a common way to handle this challenging task is to fine-tune the models pretrained on source sensor data with limited labeled data from the target sensor. Unfortunately, fine-tune technique is usually applied between homogeneous networks, which may not be the best choice if the source and target data are largely different. To address these issues, we formulate the cross-sensor RS scene understanding problem as a heterogeneous network-oriented transfer learning problem, in which the source and the target networks are different and data-oriented selected. Afterward, the knowledge between heterogeneous networks is transferred using the pseudo-label recursive propagation mechanism inspired by the concept of knowledge distillation. To the best of our knowledge, this is the first time to investigate the cross-sensor scene classification problem by constructing such a heterogeneous networks' transfer scheme in RS fields. Our experiments using two cross-sensor RS datasets [aerial images → multispectral images (MSIs) and aerial images → hyper-spectral images (HSIs)] demonstrated that the proposed transfer learning strategy based on heterogeneous networks outperforms the supervised learning (SL) and fine-tune scheme for cross-sensor scene classification.

Many pipeline networks around the world do not have a failure history. Because of financial reasons or lack of technology, this is a common issue for asset managers in developing and under-developed countries. In the case of data scarcity, it is not straightforward to assess the risk and reliability of such systems. This research proposes a probabilistic framework for analysing the reliability of corroded pipeline networks with limited failure history availability. To achieve this aim through a comprehensive reliability assessment methodology, state-of-the-art corrosion modelling, component and system failure analysis techniques are utilised. A variety of spatio-temporal reliability assessment methods are employed along with the random-field representation of spatially variable parameters to estimate the failure in a large-scale corrosion affected pipeline networks. The applicability of the methodology is evaluated through a case-study of buried pipeline network, and the results are verified with failure history data reported by the industry. The results obtained from the proposed methodology, combined with the cost data provided by asset managers make accurate risk assessment of process/water pipeline possible.

Separating 3D point clouds into individual instances is an important task for 3D vision. It is challenging due to the unknown and varying number of instances in a scene. Existing deep learning based works focus on a two-step pipeline: first learn a feature embedding and then cluster the points. Such a two-step pipeline leads to disconnected intermediate objectives. In this paper, we propose an integrated reformulation of 3D instance segmentation as a per-point classification problem. We propose ICM-3D, a single-step method to segment 3D instances via instantiated categorization. The augmented category information is automatically constructed from 3D spatial positions. We conduct extensive experiments to verify the effectiveness of ICM-3D and show that it obtains inspiring performance across multiple frameworks, backbones and benchmarks., IEEE Robotics and Automation Letters (RA-L). Preprint Version. Accepted August, 2021

Keyboard acts as one of the most commonly used mediums for human-computer interaction. Today, massive Internet of Things (IoT) devices are designed without a physical keyboard as they go tiny, but are almost all equipped with a wireless module for networks. In this work, we aim to enable a universal virtual keyboard using wireless signals, which would allow a typing interface for tiny IoT devices or serve as a portable alternative to the unwieldy physical keyboards. To this end, we present mmKey, the first universal virtual keyboard system using a single millimeter-wave (mmWave) radio. By leveraging the unique advantages of mmWave signals, mmKey converts any flat surface, with a printed paper keyboard, into an effective typing medium. mmKey enables concurrent keystrokes and supports multiple keyboard layouts (e.g., computer keyboard, piano keyboard, or phone keypad). We design a novel signal processing pipeline to detect, segment and separate, and finally recognize keystrokes. mmKey does not need any training except for a minimal one-time effort of only three key-presses for keyboard calibration upon the initial setup. We prototype mmKey using a commodity 802.11ad/ay chipset, customized to support radar-like operations, and evaluate it with different keyboard layouts under various settings. Experimental results with 10 participants demonstrate a keystroke recognition accuracy of > 95% for single-key case and > 90% for multi-key scenario, which leads to a word recognition accuracy of > 97%.

The efficiency of warehouses is vital to e-commerce. Fast order processing at the warehouses ensures timely deliveries and improves customer satisfaction. However, monitoring, analyzing, and manipulating order processing in the warehouses in real time are challenging for traditional methods due to the sheer volume of incoming orders, the fuzzy definition of delayed order patterns, and the complex decision-making of order handling priorities. In this paper, we adopt a data-driven approach and propose OrderMonitor, a visual analytics system that assists warehouse managers in analyzing and improving order processing efficiency in real time based on streaming warehouse event data. Specifically, the order processing pipeline is visualized with a novel pipeline design based on the sedimentation metaphor to facilitate real-time order monitoring and suggest potentially abnormal orders. We also design a novel visualization that depicts order timelines based on the Gantt charts and Marey's graphs. Such a visualization helps the managers gain insights into the performance of order processing and find major blockers for delayed orders. Furthermore, an evaluating view is provided to assist users in inspecting order details and assigning priorities to improve the processing performance. The effectiveness of OrderMonitor is evaluated with two case studies on a real-world warehouse dataset.