Search

To manage the complex demands of modern high-performance computing (HPC), software applications increasingly depend on software developed by other teams, often at other institutions. An HPC software ecosystem approach is required to support dependencies on third-party scientific software. An ecosystem approach provides layers of activity above the individual software product level that promote interoperability, quality improvement, porting, testing, and deployment. The U.S. Exascale Computing Project (ECP) developed its HPC software ecosystem using a three-pronged approach. First, the ECP adopted and invested in Spack, a package manager designed to handle complex HPC package dependencies. Second, the ECP created the Extreme Scale Scientific Software Stack, an effort that supports developing, deploying, and running scientific applications on HPC platforms. Third, the ECP supported software product communities, or software development kits, to develop and promote best practices, improve software interoperability, and other collaborative efforts. This article describes ECP contributions to HPC software ecosystem challenges. [ABSTRACT FROM AUTHOR]

Computational and data-enabled science and engineering are revolutionizing advances throughout science and society, at all scales of computing. For example, teams in the U.S. Department of Energy's Exascale Computing Project have been tackling new frontiers in modeling, simulation, and analysis by exploiting unprecedented exascale computing capabilities—building an advanced software ecosystem that supports next-generation applications and addresses disruptive changes in computer architectures. However, concerns are growing about the productivity of the developers of scientific software. Members of the Interoperable Design of Extreme-scale Application Software project serve as catalysts to address these challenges through fostering software communities, incubating and curating methodologies and resources, and disseminating knowledge to advance developer productivity and software sustainability. This article discusses how these synergistic activities are advancing scientific discovery—mitigating technical risks by building a firmer foundation for reproducible, sustainable science at all scales of computing, from laptops to clusters to exascale and beyond. [ABSTRACT FROM AUTHOR]

Wildfire activity around the world has been increasing for the past several decades. This has led to increased frequency of wildfires and longer fire seasons. When a fire is detected, one of the most important considerations is where to allocate limited firefighting resources to best contain the fire and protect infrastructure and urban areas. This is where robust, high-resolution fire simulations can play a critical role. Many software applications currently exist that predict the propagation of wildfires, each using a variety of input data and techniques. In this case study, we examine and implement a discrete probabilistic fire model developed and validated for the 2012 Tavira wildfire in Portugal. We then simulate a hypothetical fire in western Wyoming, USA, analyze the probable burn scar, and determine where firefighting resources are best allocated. [ABSTRACT FROM AUTHOR]

The dynamic visualization of ocean dynamics data is essential for effectively presenting oceanic information data. However, with the increasing spatial resolution of ocean remote sensing data, performing global-scale visualization in a single pass has become challenging. Directly rendering ocean data with high spatial resolution can result in problems such as increased communication time, blocked resource loading, and page lag. To address this issue, this paper proposes a common ocean data standard based on the characteristics of massive ocean element data, constructs an efficient three-level LOD model for ocean dynamic field rendering, and achieves particle rendering of ocean dynamic data by converting ocean data into image slices to complete the rendering of high-volume ocean data on the Web side. Compared with traditional methods, the proposed method significantly improves rendering performance. The average window construction time (FT) is reduced to 51.77ms, enhancing the overall rendering speed by approximately 64%. Meanwhile, the average frames per second (FPS) increase to 57.45fps, augmenting rendering stability and smoothness by around 18%. The peak memory consumption of the highest resolution data used in this paper is about 90MB, which is only 1/4 of the original. The proposed method effectively compensates for the disadvantage of slow rendering of large-scale ocean data visualization in some systems, enabling fast rendering of dynamic ocean data on the Web.

This paper proposes an intelligent stethoscope system that synchronously displays the electrocardiogram (ECG) and heart sound. The instrument, which accelerates auscultation, can be used for the diagnosis of valvular heart disease (VHD) for clinical physicians. The whole system with ECG patch and stethoscope includes four parts, namely, an analog front-end circuit for bio-signal acquisition, a heart sound-classifying integrated circuit with convolution neural network (CNN), a user-friendly application that synchronously displays the heart sound and ECG signals, and a cloud server with heart murmur detection algorithm for human study. In this system, three algorithms are used in processing both ECG and heart sound signals. The first algorithm is a synchronized algorithm, which can align heart sound and ECG signals simultaneously. The second algorithm is a heart sound-classifying algorithm that can distinguish the first (S1) and the second (S2) heart sound in heart sound signals for identifying the systolic and diastolic phases. The accuracies of the algorithm applied to normal heart sound and heart murmur are 100% and 96.7%, respectively. The third algorithm is heart murmur identification, which can detect systolic murmur and has a macro f1 score of 92.5%. The three algorithms proposed are beneficial for physicians in the diagnosis of VHD. After the establishment of the whole system, a CNN-based classification algorithm is also implemented with a $0.18 \mu \text{m}$ standard CMOS process for the demonstration of the edge computing. The machine learning techniques are implemented on the chip to accelerate the classification process.

Transfer learning enables to re-use knowledge learned on a source task to help learning a target task. A simple form of transfer learning is common in current state-of-the-art computer vision models, i.e., pre-training a model for image classification on the ILSVRC dataset, and then fine-tune on any target task. However, previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains (consumer photos, autonomous driving, aerial imagery, underwater, indoor scenes, synthetic, close-ups) and task types (semantic segmentation, object detection, depth estimation, keypoint detection). Importantly, these are all complex, structured output tasks types relevant to modern computer vision applications. In total we carry out over 2000 transfer learning experiments, including many where the source and target come from different image domains, task types, or both. We systematically analyze these experiments to understand the impact of image domain, task type, and dataset size on transfer learning performance. Our study leads to several insights and concrete recommendations: (1) for most tasks there exists a source which significantly outperforms ILSVRC’12 pre-training; (2) the image domain is the most important factor for achieving positive transfer; (3) the source dataset should include the image domain of the target dataset to achieve best results; (4) at the same time, we observe only small negative effects when the image domain of the source task is much broader than that of the target; (5) transfer across task types can be beneficial, but its success is heavily dependent on both the source and target task types. [ABSTRACT FROM AUTHOR]

To improve the viewer’s Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality of distorted 3D models. However, most previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level in the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, a support vector regression (SVR) model is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now at https://github.com/zzc-1998/NR-3DQA. [ABSTRACT FROM AUTHOR]

Deep metric learning (DML) is a cornerstone of many computer vision applications. It aims at learning a mapping from the input domain to an embedding space, where semantically similar objects are located nearby and dissimilar objects far from another. The target similarity on the training data is defined by user in form of ground-truth class labels. However, while the embedding space learns to mimic the user-provided similarity on the training data, it should also generalize to novel categories not seen during training. Besides user-provided groundtruth training labels, a lot of additional visual factors (such as viewpoint changes or shape peculiarities) exist and imply different notions of similarity between objects, affecting the generalization on the images unseen during training. However, existing approaches usually directly learn a single embedding space on all available training data, struggling to encode all different types of relationships, and do not generalize well. We propose to build a more expressive representation by jointly splitting the embedding space and the data hierarchically into smaller sub-parts. We successively focus on smaller subsets of the training data, reducing its variance and learning a different embedding subspace for each data subset. Moreover, the subspaces are learned jointly to cover not only the intricacies, but the breadth of the data as well. Only after that, we build the final embedding from the subspaces in the conquering stage. The proposed algorithm acts as a transparent wrapper that can be placed around arbitrary existing DML methods. Our approach significantly improves upon the state-of-the-art on image retrieval, clustering, and re-identification tasks evaluated using CUB200-2011, CARS196, Stanford Online Products, In-shop Clothes, and PKU VehicleID datasets. [ABSTRACT FROM AUTHOR]

Image smoothing is a fundamental procedure in applications of both computer vision and graphics. The required smoothing properties can be different or even contradictive among different tasks. Nevertheless, the inherent smoothing nature of one smoothing operator is usually fixed and thus cannot meet the various requirements of different applications. In this paper, we first introduce the truncated Huber penalty function which shows strong flexibility under different parameter settings. A generalized framework is then proposed with the introduced truncated Huber penalty function. When combined with its strong flexibility, our framework is able to achieve diverse smoothing natures where contradictive smoothing behaviors can even be achieved. It can also yield the smoothing behavior that can seldom be achieved by previous methods, and superior performance is thus achieved in challenging cases. These together enable our framework capable of a range of applications and able to outperform the state-of-the-art approaches in several tasks, such as image detail enhancement, clip-art compression artifacts removal, guided depth map restoration, image texture removal, etc. In addition, an efficient numerical solution is provided and its convergence is theoretically guaranteed even the optimization framework is non-convex and non-smooth. A simple yet effective approach is further proposed to reduce the computational cost of our method while maintaining its performance. The effectiveness and superior performance of our approach are validated through comprehensive experiments in a range of applications. Our code is available at https://github.com/wliusjtu/Generalized-Smoothing-Framework . [ABSTRACT FROM AUTHOR]

This article reviews various conceptual views behind the digital twins paradigm, highlights software technologies to develop digital twins, analyzes the use of digital twins during the lifecycle of industrial assets, and discusses a generic scenario from the electricity domain. [ABSTRACT FROM AUTHOR]

Recognizing and fitting shape primitives from underlying 3D models are key components of many computer graphics and computer vision applications. Although a vast number of structural recovery methods are available, they usually fail to identify blending surfaces, which corresponds to small transitional regions among relatively large primary patches. To address this issue, we present a novel approach for automatic segmentation and surface fitting with accurate geometric parameters from 3D models, especially mechanical parts. Overall, we formulate the structural segmentation as a Markov random field (MRF) labeling problem. In contrast to existing techniques, we first propose a new clustering algorithm to build superfacets by incorporating 3D local geometric information. This algorithm extracts the general quadric and rolling-ball blending regions, and improves the robustness of further segmentation. Next, we apply a specially designed MRF framework to efficiently partition the original model into different meaningful patches of known surface types by defining the multilabel energy function on the superfacets. Furthermore, we present an iterative optimization algorithm based on skeleton extraction to fit rolling-ball blending patches by recovering the parameters of the rolling center trajectories and ball radius. Experiments on different complex models demonstrate the effectiveness and robustness of the proposed method, and the superiority of our method is also verified through comparisons with state-of-the-art approaches. We further apply our algorithm in applications such as mesh editing by changing the radius of the rolling balls. [ABSTRACT FROM AUTHOR]

Homography estimation is an important task in computer vision applications, such as image stitching, video stabilization, and camera calibration. Traditional homography estimation methods heavily depend on the quantity and distribution of feature correspondences, leading to poor robustness in low-texture scenes. The learning solutions, on the contrary, try to learn robust deep features but demonstrate unsatisfying performance in the scenes with low overlap rates. In this paper, we address these two problems simultaneously by designing a contextual correlation layer (CCL). The CCL can efficiently capture the long-range correlation within feature maps and can be flexibly used in a learning framework. In addition, considering that a single homography can not represent the complex spatial transformation in depth-varying images with parallax, we propose to predict multi-grid homography from global to local. Moreover, we equip our network with a depth perception capability, by introducing a novel depth-aware shape-preserved loss. Extensive experiments demonstrate the superiority of our method over state-of-the-art solutions in the synthetic benchmark dataset and real-world dataset. The codes and models will be available at https://github.com/nie-lang/Multi-Grid-Deep-Homography. [ABSTRACT FROM AUTHOR]

Many computer vision applications rely on feature detection and description, hence the need for computationally efficient and robust 4D light field (LF) feature detectors and descriptors. In this paper, we propose a novel light field feature descriptor based on the Fourier disparity layer representation, for light field imaging applications. After the Harris feature detection in a scale-disparity space, the proposed feature descriptor is then extracted using a circular neighborhood rather than a square neighborhood. It is shown to yield more accurate feature matching, compared with the LiFF LF feature, with a lower computational complexity. In order to evaluate the feature matching performance with the proposed descriptor, we generated a synthetic stereo LF dataset with ground truth matching points. Experimental results with synthetic and real-world dataset show that our solution outperforms existing methods in terms of both feature detection robustness and feature matching accuracy. [ABSTRACT FROM AUTHOR]

Intelligent video surveillance is an important computer vision application in natural environments. Since detected objects under surveillance are usually low-resolution and noisy, their accurate recognition represents a huge challenge. Knowledge distillation is an effective method to deal with it, but existing related work usually focuses on reducing the channel count of a student network, not feature map size. As a result, they cannot transfer “privilege information” hidden in feature maps of a wide and deep teacher network into a thin and shallow student one, leading to the latter’s poor performance. To address this issue, we propose a Feature Map Distillation (FMD) framework under which the feature map size of teacher and student networks is different. FMD consists of two main components: Feature Decoder Distillation (FDD) and Feature Map Consistency-enforcement (FMC). FDD reconstructs the shallow texture features of a thin student network to approximate the corresponding samples in a teacher network, which allows the high-resolution ones to directly guide the learning of the shallow features of the student network. FMC makes the size and direction of each deep feature map consistent between student and teacher networks, which constrains each pair of feature maps to produce the same feature distribution. FDD and FMC allow a thin student network to learn rich “privilege information” in feature maps of a wide teacher network. The overall performance of FMD is verified in multiple recognition tasks by comparing it with state-of-the-art knowledge distillation methods on low-resolution and noisy objects. [ABSTRACT FROM AUTHOR]

Object detectors are vital to many modern computer vision applications. However, even state-of-the-art object detectors are not perfect. On two images that look similar to human eyes, the same detector can make different predictions because of small image distortions like camera sensor noise and lighting changes. This problem is called inconsistency. Existing accuracy metrics do not properly account for inconsistency, and similar work in this area only targets improvements on artificial image distortions. Therefore, we propose a method to use nonartificial video frames to measure object detection consistency over time, across frames. Using this method, we show that the consistency of modern object detectors ranges from 83.2% to 97.1% on different video datasets from the multiple object tracking challenge. We conclude by showing that applying image distortion corrections such as WEBP Image Compression and Unsharp Masking can improve consistency by as much as 5.1%, with no loss in accuracy. [ABSTRACT FROM AUTHOR]

The successful integration of computer vision, robotic actuation, and photoacoustic imaging to find and follow targets of interest during surgical and interventional procedures requires accurate photoacoustic target detectability. This detectability has traditionally been assessed with image quality metrics, such as contrast, contrast-to-noise ratio, and signal-to-noise ratio (SNR). However, predicting target tracking performance expectations when using these traditional metrics is difficult due to unbounded values and sensitivity to image manipulation techniques like thresholding. The generalized contrast-to-noise ratio (gCNR) is a recently introduced alternative target detectability metric, with previous work dedicated to empirical demonstrations of applicability to photoacoustic images. In this article, we present theoretical approaches to model and predict the gCNR of photoacoustic images with an associated theoretical framework to analyze relationships between imaging system parameters and computer vision task performance. Our theoretical gCNR predictions are validated with histogram-based gCNR measurements from simulated, experimental phantom, ex vivo, and in vivo datasets. The mean absolute errors between predicted and measured gCNR values ranged from ${3.2} \times {10}^{-{3}}$ to ${2.3} \times {10}^{-{2}}$ for each dataset, with channel SNRs ranging −40 to 40 dB and laser energies ranging 0.07 $\mu \text{J}$ to 68 mJ. Relationships among gCNR, laser energy, target and background image parameters, target segmentation, and threshold levels were also investigated. Results provide a promising foundation to enable predictions of photoacoustic gCNR and visual servoing segmentation accuracy. The efficiency of precursory surgical and interventional tasks (e.g., energy selection for photoacoustic-guided surgeries) may also be improved with the proposed framework. [ABSTRACT FROM AUTHOR]

Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications. Existing methods for 3D shape segmentation suffer from complex geometry processing and heavy computation caused by using low-level features and fragmented segmentation results due to the lack of global consideration. We present an efficient method, called SEG-MAT, based on the medial axis transform (MAT) of the input shape. Specifically, with the rich geometrical and structural information encoded in the MAT, we are able to develop a simple and principled approach to effectively identify the various types of junctions between different parts of a 3D shape. Extensive evaluations and comparisons show that our method outperforms the state-of-the-art methods in terms of segmentation quality and is also one order of magnitude faster. [ABSTRACT FROM AUTHOR]

Image enhancement is a basic requirement for any computer vision application for further processing of an image. A common limitation with most of the existing methods, when applied to nearly invisible images, is the loss of color details during the enhancement process. So, a fuzzy c-means clustering-based method for image enhancement is proposed which enhances the perceptually invisible image along with preserving its color and naturalness. In this method, the image pixels are grouped into different clusters and are assigned membership values to those clusters. Based on this membership value, its intensity level is modified in the spatial domain. Modification of the gray levels proportional to the membership values leads to the stretching of the image histogram, similar in shape, to the original histogram. The process results in a very small shift in the mean intensity which preserves the color and brightness-related information of the image. The method enhances the image contrast and maintains the naturalness without introducing any artifacts. The simulation results on standard datasets reflect that the proposed algorithm is superior to many state-of-the-art and traditional methods for perceptually invisible images. [ABSTRACT FROM AUTHOR]

Approximate Nearest Neighbor(ANN) search is the core problem in many large-scale machine learning and computer vision applications such as multimodal retrieval. Hashing is becoming increasingly popular, since it can provide efficient similarity search and compact data representations suitable for handling such large-scale ANN search problems. Most hashing algorithms concentrate on learning more effective projection functions. However, the accuracy loss in the quantization step has been ignored and barely studied. In this paper, we analyse the importance of various projected dimensions, distribute them into several groups and quantize them with two types of values which can both better preserve the neighborhood structure among data. One is Variable Integer-based Quantization (VIQ) that quantizes each projected dimension with integer values. The other is Variable Codebook-based Quantization (VCQ) that quantizes each projected dimension with corresponding codebook values. We conduct experiments on five common public data sets containing up to one million vectors. The results show that the proposed VCQ and VIQ algorithms can both achieve much higher accuracy than state-of-the-art quantization methods. Furthermore, although VCQ performs better than VIQ, ANN search with VIQ provides much higher search efficiency. [ABSTRACT FROM AUTHOR]

This article proposes a new calibration method, called the mixed-binning (MB) method, to pursue high-linearity time-to-digital converters (TDCs) for light detection and ranging applications. The proposed TDCs were developed using tapped delay-line (TDL) cells in field-programmable gate arrays (FPGAs). With the MB method, we implemented a resolution-adjustable TDC showing excellent linearity in Xilinx UltraScale FPGAs. We demonstrate a 128-channel TDC to show that the proposed method is cost-effective in logic resources. We also developed a software tool to predict the performances of TDL-based TDCs robustly. Results from both software analysis and hardware implementations are in good agreement and show that the proposed design has great potential for multichannel applications; the averaged ${{\bf DN}}{{{\bf L}}_{{\boldsymbol{pk}} - {\boldsymbol{pk}}}}$ and ${{\bf IN}}{{{\bf L}}_{{\boldsymbol{pk}} - {\boldsymbol{pk}}}}$ are close to or even less than 0.05 LSB in multichannel designs. [ABSTRACT FROM AUTHOR]

Low-light images suffer from low contrast and unclear details, which not only reduces the available information for humans but limits the application of computer vision algorithms. Among the existing enhancement techniques, Retinex-based and learning-based methods are under the spotlight today. In this paper, we bridge the gap between the two methods. First, we propose a novel “generative” strategy for Retinex decomposition, by which the decomposition is cast as a generative problem. Second, based on the strategy, a unified deep framework is proposed to estimate the latent components and perform low-light image enhancement. Third, our method can weaken the coupling relationship between the two components while performing Retinex decomposition. Finally, the RetinexDIP performs Retinex decomposition without any external images, and the estimated illumination can be easily adjusted and is used to perform enhancement. The proposed method is compared with ten state-of-the-art algorithms on seven public datasets, and the experimental results demonstrate the superiority of our method. Code is available at: https://github.com/zhaozunjin/RetinexDIP. [ABSTRACT FROM AUTHOR]

Homing sequence derivation for nondeterministic finite state machines (NFSMs) has important applications in software/hardware system testing and verification. Unlike prior methods based on explicit tree-based search, in this article we formulate the derivation of a preset/adaptive homing sequence in terms of quantified Boolean formula (QBF) solving. This formulation exploits compact circuit representation of NFSMs and QBF encoding of the existence condition of homing sequence for effective computation. The implicit circuit representation effectively avoids explicit state enumeration, and can be more scalable. Different encoding schemes and QBF solvers are evaluated for their suitability for the homing sequence derivation. Experiments on various computation methods and benchmarks show the generality and feasibility of a proposed approach. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has demonstrated great performance in various applications on powerful computers and servers. Recently, with the advancement of more powerful mobile devices (e.g., smartphones and touch pads), researchers are seeking DL solutions that could be deployed on mobile devices. Compared to traditional DL solutions using cloud servers, deploying DL on mobile devices have unique advantages in data privacy, communication overhead, and system cost. This article provides a comprehensive survey for the current studies of adopting and deploying DL on mobile devices. Specifically, we summarize and compare the state-of-the-art DL techniques on mobile devices in various application domains involving vision, speech/speaker recognition, human activity recognition, transportation mode detection, and security. We generalize an optimization pipeline for bringing DL to mobile devices, including model-oriented optimization mechanisms (e.g., pruning and quantization) and nonmodel-oriented optimization mechanisms (e.g., software accelerator and hardware design). Moreover, we summarize popular DL libraries regarding their support to state-of-the-art models (software) and processors (hardware). Based on our summarization, we further provide insights into potential research opportunities for developing DL for mobile devices. [ABSTRACT FROM AUTHOR]

In recent years, several clustering algorithms have been proposed with the aim of mining knowledge from streams of data generated at a high speed by a variety of hardware platforms and software applications. Among these algorithms, density-based approaches have proved to be particularly attractive, thanks to their capability of handling outliers and capturing clusters with arbitrary shapes. The streaming setting poses additional challenges that need to be addressed as well: data streams are potentially unbounded and affected by concept drift, i.e., a modification over time in the underlying data generation process. In this article, we propose temporal streaming fuzzy density-based spatial clustering of applications with noise (TSF-DBSCAN), a novel fuzzy clustering algorithm for streaming data. TSF-DBSCAN is an extension of the well-known DBSCAN algorithm, one of the most popular density-based clustering approaches. Fuzziness is introduced in TSF-DBSCAN to model the uncertainty about the distance threshold that defines the neighborhood of an object. As a consequence, TSF-DBSCAN identifies clusters with fuzzy overlapping borders. A fading model, which makes objects less relevant as they become more remote in time, endows TSF-DBSCAN with the capability of adapting to evolving data streams. The integration of the model in a two-stage approach ensures computational and memory efficiency: during the online stage, continuously arriving objects are organized in proper data structures that are later exploited in the offline stage to determine a fine-grained partition. An extensive experimental analysis on synthetic and real-world datasets shows that TSF-DBSCAN yields competitive performance when compared to other clustering algorithms recently proposed for streaming data. [ABSTRACT FROM AUTHOR]

Universities around the world have been taking advantage of open educational resources (OERs) for publishing educational material. The teachers and students can access, the published material for teaching, learning and research through institutional repositories. This article proposes the design, construction and validation of the semantic platform (MIIDAS), considering two case studies of higher-level educational institutions (civil and military schools). The MIIDAS Platform integrates educational resources in institutional repositories related to the areas of Computing and Electronics engineering through the incorporation of Semantic Web technologies. The descriptions of resources consider metadata of the OAI-PMH protocol and also add educational metadata. The platform uses RDF resource descriptions, ontologies and the SPARQL query language to improve resource retrieval, in addition to proposing guidelines for the management of educational resources. The results of validation of the platform are presented through the development of evaluation instruments, which suggest possible ways to improve the platform. The article provides the basis for developing platforms that help in spreading open educational resources and allow the construction of semantically enrich data sets.

Grid structures are common in high-throughput assays to parallelize experiments in biochemical or biological experiments. Manual analysis of grid images is laborious, time-consuming, expensive, and critical in terms of reproducibility. However, it is still common to do such analysis manually, as there is no standardized software for automated analysis. In this paper, we introduce a generic method to automatically detect grid structures in images and to perform flexible spot-wise analysis after successful grid detection. The deep learning-based approach of the grid structure detection allows being flexible concerning different grid types. The combination with a robust parameter estimation algorithm lowers the requirements of the detection quality and thus enhances robustness. Further, the method conducts semi-automated grid detection if a fully automated processing fails. An open-source software tool Grid Screener that implements the proposed methods is provided as a ready-for-use tool for researchers. The usability is demonstrated by taking different criteria into account, which are important for a successful application. We present the benefits of our proposed tool Grid Screener utilizing three different grid types in the context of high-throughput screening to show our contribution towards further lab automation. Our tool performs much faster than manual analysis, while maintaining or even enhancing accuracy.

Managing random hardware faults requires the faults to be detected online, thus simplifying recovery. Algorithm-based fault tolerance has been proposed as a low-cost mechanism to check online the result of computations against random hardware failures. In this case, the checksum of the actual result is checked against a predicted checksum computed in parallel by a hardware checker. In this work, we target the design of such checkers for convolution engines that are currently the most critical building block in image processing and computer vision applications. The proposed convolution checksum checker, named ConvGuard, utilizes a newly introduced invariance condition of convolution to predict implicitly the output checksum using only the pixels at the border of the input image. In this way, ConvGuard reduces the power required for accumulating the input pixels without requiring large buffers to hold intermediate checksum results. The design of ConvGuard is generic and can be configured for different output sizes and strides. The experimental results show that ConvGuard utilizes only a small percentage of the area/power of an efficient convolution engine while being significantly smaller and more power efficient than a state-of-the-art checksum checker for various practical cases. [ABSTRACT FROM AUTHOR]

Nowadays, people's lifestyle is more and more dependent on mobile applications (Apps), such as shopping, financial management and surfing the internet. However, developers mainly focus on the implementation of Apps and the improvement of user experience while ignoring security issues. In this paper, we perform the comprehensive study on vulnerabilities caused by misuse of APIs and form a methodology for this type of vulnerability analysis. We investigate the security of three types of Android Apps including finance, shopping and browser which are closely related to human life. And we analyze four vulnerabilities including Improper certificate validation(CWE-295:ICV), WebView bypass certificate validation vulnerability(CVE-2014-5531:WBCVV), WebView remote code execution vulnerability(CVE-2014-1939:WRCEV) and Alibaba Cloud OSS credential disclosure vulnerability(CNVD-2017-09774:ACOCDV). In order to verify the effectiveness of our analysis method in large-scale Apps on the Internet, we propose a novel scalable tool - VulArcher, which is based on heuristic method and used to discover if the above vulnerabilities exist in Apps. We download a total of 6114 of the above three types of samples in App stores, and we use VulArcher to perform the above vulnerability detection for each App. We perform manual verification by randomly selecting 100 samples of each vulnerability. We find that the accuracy rate for ACOCDV can reach 100%, the accuracy rate for WBCVV can reach 95%, and the accuracy rate for the other two vulnerabilities can reach 87%. And one of vulnerabilities detected by VulArcher has been included in China National Vulnerability Database (CNVD) ID(CNVD-2017-23282). Experiments show that our tool is feasible and effective. For the convenience of researchers in related communities, We make our data and tool available at https://buptnsrclab.github.io/blog/2020/01/03/vularcher-site-launched.

The growing availability of self-contained and affordable augmented reality headsets such as the Microsoft HoloLens is encouraging the adoption of these devices also in the healthcare sector. However, technological and human-factor limitations still hinder their routine use in clinical practice. Among them, the major drawbacks are due to their general-purpose nature and to the lack of a standardized framework suited for medical applications and devoid of platform-dependent tracking techniques and/or complex calibration procedures. To overcome such limitations, in this paper we present a software framework that is designed to support the development of augmented reality applications for custom-made head-mounted displays designed to aid high-precision manual tasks. The software platform is highly configurable, computationally efficient, and it allows the deployment of augmented reality applications capable to support in situ visualization of medical imaging data. The framework can provide both optical and video see-through-based augmentations and it features a robust optical tracking algorithm. An experimental study was designed to assess the efficacy of the platform in guiding a simulated task of surgical incision. In the experiments, the user was asked to perform a digital incision task, with and without the aid of the augmented reality headset. The task accuracy was evaluated by measuring the similarity between the traced curve and the planned one. The average error in the augmented reality tests was

Patients suffering from Parkinson's Disease (PD) manifest relevant changes in their speech, consisting of specific landmarks in articulation, phonation, fluency, and prosody. Usually, phonation and articulation changes are estimated and evaluated using different methods and statistical frameworks. Speech is especially relevant as a vehicular mechanism to monitor neurological evolution using well-known features extracted from sustained phonations (mainly vowels), diadochokinetic exercises, or running speech. Recent studies have shown that acoustic neurostimulation using binaural beats influences the cognitive and neuromotor conditions of patients with PD, at least temporarily after stimulation. The aim of this study is to describe an added-value solution considering the cooperation of both previously mentioned methods: speech analysis-based monitoring called within the project, Monitoring Parkinson using Locution (MonParLoc), and acoustical neurostimulation, called within project neuro-Acoustic-stimulation Parkinson (AcousticPar). The applications designed in both projects are embedded into a global solution denominated Teca-Park which consists of four main activities: speech evaluation, neurostimulation, motor symptom longitudinally, and questionnaires. This framework is conceived to be a powerful tool for treating and monitoring longitudinally remotely and contact-free. MonParLoc was tested and validated in real scenarios involving patient associations. Validation results produced in these associations demonstrating the utility of this approach are given in the study, particularly in reference to protocol vulnerability and robustness. This paper proposes a complete framework (a mobile app and a scorecard solution) including different services for Parkinson's clinical monitoring and patient management using speech, movement, and acoustic stimulation.

Temperature rise simulation deals with different physical domains describing heat generation and heat dissipation effects. Cigré working group A3.36 has conducted a study on temperature rise simulation of switchgear for distribution and transmission level. The working group built up new, manufacturer independent devices. Members of the working group simulated the temperature rise in advance to the tests, which were performed at laboratories of universities, as independent parties. In order to assess the simulation accuracy, the simulation results are compared to each other and to the measured results. This paper summarizes the main findings from the medium voltage device. The study has shown that when simulating the temperature rise on a completely new design, with reliable input values of contact resistances and emissivity coefficients, an accuracy of about 25% could be expected. Primary reason seems to be based on the complexity of simulating the heat dissipation mechanisms. To get a more accurate result, verification and validation is needed in order to adjust the parameters determining the heat dissipation. This study has also illustrated that it requires good knowledge (especially of the radiation) in order to adjust these parameters properly. [ABSTRACT FROM AUTHOR]

In the past half of the decade, object detection approaches based on the convolutional neural network have been widely studied and successfully applied in many computer vision applications. However, detecting objects in inclement weather conditions remains a major challenge because of poor visibility. In this article, we address the object detection problem in the presence of fog by introducing a novel dual-subnet network (DSNet) that can be trained end-to-end and jointly learn three tasks: visibility enhancement, object classification, and object localization. DSNet attains complete performance improvement by including two subnetworks: detection subnet and restoration subnet. We employ RetinaNet as a backbone network (also called detection subnet), which is responsible for learning to classify and locate objects. The restoration subnet is designed by sharing feature extraction layers with the detection subnet and adopting a feature recovery (FR) module for visibility enhancement. Experimental results show that our DSNet achieved 50.84 percent mean average precision (mAP) on a synthetic foggy dataset that we composed and 41.91 percent mAP on a public natural foggy dataset (Foggy Driving dataset), outperforming many state-of-the-art object detectors and combination models between dehazing and detection methods while maintaining a high speed. [ABSTRACT FROM AUTHOR]

To maintain sensitivity to new physics in the coming years of Large Hadron Collider (LHC) operations, A Toroidal LHC ApparatuS (ATLAS) collaboration has been working on upgrading a portion of the front-end (FE) electronics and replacing some parts of the detector with new devices that can operate under the much harsher background conditions of future LHC runs. The legacy FE of the ATLAS detector sent data to the data acquisition (DAQ) system via the so-called Read Out Drivers (RODs) custom-made VMEbus boards devoted to data processing, configuration, and control. The data were then received by the Read Out System (ROS), which was responsible for buffering them during the High-Level Trigger (HLT) processing. From Run 3 onward, all new trigger and detector systems will be read out using new components, replacing the combination of the ROD and the ROS. This new path will feature an application called the Software Read Out Driver (SW ROD), which will run on a commodity server receiving FE data via the Front-End Link eXchange (FELIX) system. The SW ROD will perform event fragment building and buffering as well as serving the data on request to the HLT. The SW ROD application has been designed as a highly customizable high-performance framework providing support for detector-specific event building and data processing algorithms. The implementation that will be used for Run 3 of the LHC is capable of building event fragments at a rate of 100 kHz from an input stream consisting of up to 120 MHz of individual data packets. This document will cover the design and the implementation of the SW ROD application and will present the results of performance measurements executed on the server models selected to host SW ROD applications during Run 3. [ABSTRACT FROM AUTHOR]

Infrared target tracking is a key technology in many applications of computer vision, such as traffic control, pedestrian surveillance, intelligent detection, and so on. However, due to the lack of color and texture information, tracking drift is easy to happen when the tracker encounters objects that are similar to the target in terms of grayscale distribution or contour shape. To address this issue, we propose a hierarchical convolution fusion-based adaptive Siamese (HCFA-Siam) network for infrared target tracking. First, in order to improve the discriminative capability of the network, a new hierarchical convolution fusion network is proposed to fuse the shallow spatial information and deep semantic information. Then, a channel attention module is followed to selectively enhance the feature channels of the infrared target, making the feature more discriminative. Finally, to further distinguish the target from the above-mentioned interference of appearance similarity, we add an adaptive update mechanism-based negative template pool at the front end of the network as the second input of the template branch. The experimental results on infrared sequences show that our method outperforms other state-of-the-art trackers in both success rate and precision. [ABSTRACT FROM AUTHOR]

Superpixels are widely used in computer vision applications. Most of the existing superpixel methods use established criteria to indiscriminately process all pixels, resulting in superpixel boundary adherence and regularity being unnecessarily inter-inhibitive. This study builds upon a previous work by proposing a new segmentation strategy that classifies image content into meaningful areas containing object boundaries and meaningless parts that include color-homogeneous and texture-rich regions. Based on this classification, we design two distinct criteria to process the pixels in different environments to achieve highly accurate superpixels in content-meaningful areas and keep the regularity of the superpixels in content-meaningless regions. Additionally, we add a group of weights when adopting the color feature, successfully reducing the undersegmentation error. The superior accuracy and the moderate compactness achieved by the proposed method in comparative experiments with several state-of-the-art methods indicate that the content-adaptive criteria efficiently reduce the compromise between boundary adherence and compactness. [ABSTRACT FROM AUTHOR]

We have witnessed a growing interest in video salient object detection (VSOD) techniques in today’s computer vision applications. In contrast with temporal information (which is still considered a rather unstable source thus far), the spatial information is more stable and ubiquitous, thus it could influence our vision system more. As a result, the current main-stream VSOD approaches have inferred and obtained their saliency primarily from the spatial perspective, still treating temporal information as subordinate. Although the aforementioned methodology of focusing on the spatial aspect is effective in achieving a numeric performance gain, it still has two critical limitations. First, to ensure the dominance by the spatial information, its temporal counterpart remains inadequately used, though in some complex video scenes, the temporal information may represent the only reliable data source, which is critical to derive the correct VSOD. Second, both spatial and temporal saliency cues are often computed independently in advance and then integrated later on, while the interactions between them are omitted completely, resulting in saliency cues with limited quality. To combat these challenges, this paper advocates a novel spatiotemporal network, where the key innovation is the design of its temporal unit. Compared with other existing competitors (e.g., convLSTM), the proposed temporal unit exhibits an extremely lightweight design that does not degrade its strong ability to sense temporal information. Furthermore, it fully enables the computation of temporal saliency cues that interact with their spatial counterparts, ultimately boosting the overall VSOD performance and realizing its full potential towards mutual performance improvement for each. The proposed method is easy to implement yet still effective, achieving high-quality VSOD at 50 FPS in real-time applications. [ABSTRACT FROM AUTHOR]

The image contrast is a feature capturing the variation of the image signal across the space. Such a feature is very useful to describe the local image structure at different scales and thus it is relevant to many computer vision applications, like image/texture retrieval and object recognition. In this work, we present MiRCo, a novel measure of image contrast derived from the Retinex theory. MiRCo is robust against in-plane rotations and light changes at multiple scales. Thanks to these properties, MiRCo enables an accurate and robust description of the local image structure. Here we describe and discuss the mathematical insights of MiRCo also in comparison with other popular contrast measures. [ABSTRACT FROM AUTHOR]

We consider visual tracking in numerous applications of computer vision and seek to achieve optimal tracking accuracy and robustness based on various evaluation criteria for applications in intelligent monitoring during disaster recovery activities. We propose a novel framework to integrate a Kalman filter (KF) with spatial-temporal regularized correlation filters (STRCF) for visual tracking to overcome the instability problem due to large-scale application variation. To solve the problem of target loss caused by sudden acceleration and steering, we present a stride length control method to limit the maximum amplitude of the output state of the framework, which provides a reasonable constraint based on the laws of motion of objects in real-world scenarios. Moreover, we analyze the attributes influencing the performance of the proposed framework in large-scale experiments. The experimental results illustrate that the proposed framework outperforms STRCF on OTB-2013, OTB-2015 and Temple-Color datasets for some specific attributes and achieves optimal visual tracking for computer vision. Compared with STRCF, our framework achieves AUC gains of 2.8%, 2%, 1.8%, 1.3%, and 2.4% for the background clutter, illumination variation, occlusion, out-of-plane rotation, and out-of-view attributes on the OTB-2015 datasets, respectively. For sporting events, our framework presents much better performance and greater robustness than its competitors. [ABSTRACT FROM AUTHOR]

Shadow removal can significantly improve the image visual quality and has many applications in computer vision. Deep learning methods based on CNNs have become the most effective approach for shadow removal by training on either paired data, where both the shadow and underlying shadow-free versions of an image are known, or unpaired data, where shadow and shadow-free training images are totally different with no correspondence. In practice, CNN training on unpaired data is more preferred given the easiness of training data collection. In this paper, we present a new Lightness-Guided Shadow Removal Network (LG-ShadowNet) for shadow removal by training on unpaired data. In this method, we first train a CNN module to compensate for the lightness and then train a second CNN module with the guidance of lightness information from the first CNN module for final shadow removal. We also introduce a loss function to further utilise the colour prior of existing data. Extensive experiments on widely used ISTD, adjusted ISTD and USR datasets demonstrate that the proposed method outperforms the state-of-the-art methods with training on unpaired data. [ABSTRACT FROM AUTHOR]

Superpixel segmentation, as a central image processing task, has many applications in computer vision and computer graphics. Boundary alignment and shape compactness are leading indicators to evaluate a superpixel segmentation algorithm. Furthermore, convexity can make superpixels reflect more geometric structures in images and provide a more concise over-segmentation result. In this paper, we consider generating convex and compact superpixels while satisfying the constraints of adhering to the boundary as far as possible. We formulate the new superpixel segmentation into an edge-constrained centroidal power diagram (ECCPD) optimization problem. In the implementation, we optimize the superpixel configurations by repeatedly performing two alternative operations, which include site location updating and weight updating through a weight function defined by image features. Compared with existing superpixel methods, our method can partition an image into fully convex and compact superpixels with better boundary adherence. Extensive experimental results show that our approach outperforms existing superpixel segmentation methods in boundary alignment and compactness for generating convex superpixels. [ABSTRACT FROM AUTHOR]

Detection and analysis of informative keypoints is a fundamental problem in image analysis and computer vision. Keypoint detectors are omnipresent in visual automation tasks, and recent years have witnessed a significant surge in the number of such techniques. Evaluating the quality of keypoint detectors remains a challenging task owing to the inherent ambiguity over what constitutes a good keypoint. In this context, we introduce a reference based keypoint quality index which is based on the theory of spatial pattern analysis. Unlike traditional correspondence-based quality evaluation which counts the number of feature matches within a specified neighborhood, we present a rigorous mathematical framework to compute the statistical correspondence of the detections inside a set of salient zones (cluster cores) defined by the spatial distribution of a reference set of keypoints. We leverage the versatility of the level sets to handle hypersurfaces of arbitrary geometry, and develop a mathematical framework to estimate the model parameters analytically to reflect the robustness of a feature detection algorithm. Extensive experimental studies involving several keypoint detectors tested under different imaging scenarios demonstrate efficacy of our method to evaluate keypoint quality for generic applications in computer vision and image analysis. [ABSTRACT FROM AUTHOR]

Achieving an automatic trade-off between accuracy and efficiency for a single deep neural network is highly desired in time-sensitive computer vision applications. To achieve anytime prediction, existing methods only embed fixed exits to neural networks and make the predictions with the fixed exits for all the samples (refer to the “latest-all” strategy). However, it is observed that the latest exit within a time budget does not always provide a more accurate prediction than the earlier exits for testing samples of various difficulties, making the “latest-all” strategy a sub-optimal solution. Motivated by this, we propose to improve the anytime prediction accuracy by allowing each sample to adaptively select its own optimal exit within a specific time budget. Specifically, we propose a new Routing Convolutional Network (RCN). For any given time budget, it adaptively selects the optimal layer as exit for a specific testing sample. To learn an optimal policy for sample routing, a Q-network is embedded into the RCN at each exit, considering both potential information gain and time-cost. To further boost the anytime prediction accuracy, the exits and the Q-networks are optimized alternately to mutually boost each other under the cost-sensitive environment. Apart from applying to whole image classification, RCN can also be adapted to dense prediction tasks, e.g., scene parsing, to achieve the pixel-level anytime prediction. Extensive experimental results on CIFAR-10, CIFAR-100, and ImageNet classification benchmarks, and Cityscapes scene parsing benchmark demonstrate the efficacy of the proposed RCN for anytime recognition. [ABSTRACT FROM AUTHOR]

In graph theory, a tree is one of the more popular families of graphs with a wide range of applications in computer science as well as many other related fields. While there are several distance measures over the set of all trees, we consider here the one which defines the so-called tree distance, defined by the minimum number of edit operations, of removing and adding edges, in order to change one tree into another. From a coding theoretic perspective, codes over the tree distance are used for the correction of edge erasures and errors. However, studying this distance measure is important for many other applications that use trees and properties on their locality and the number of neighbor trees. Under this paradigm, the largest size of code over trees with a prescribed minimum tree distance is investigated. Upper bounds on these codes as well as code constructions are presented. A significant part of our study is dedicated to the problem of calculating the size of the ball of trees of a given radius. These balls are not regular and thus we show that while the star tree has asymptotically the smallest size of the ball, the maximum is achieved for the path tree. [ABSTRACT FROM AUTHOR]

Continuous colormaps are integral parts of many visualization techniques, such as heat-maps, surface plots, and flow visualization. Despite that the critiques of rainbow colormaps have been around and well-acknowledged for three decades, rainbow colormaps are still widely used today. One reason behind the resilience of rainbow colormaps is the lack of tools for users to create a continuous colormap that encodes semantics specific to the application concerned. In this paper, we present a web-based software system, CCC-Tool (short for Charting Continuous Colormaps) under the URL https://ccctool.com , for creating, editing, and analyzing such application-specific colormaps. We introduce the notion of “colormap specification (CMS)” that maintains the essential semantics required for defining a color mapping scheme. We provide users with a set of advanced utilities for constructing CMS’s with various levels of complexity, examining their quality attributes using different plots, and exporting them to external application software. We present two case studies, demonstrating that the CCC-Tool can help domain scientists as well as visualization experts in designing semantically-rich colormaps. [ABSTRACT FROM AUTHOR]

Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions. Inherently able to efficiently capture structured, latent semantic spaces and high-order interactions, tensors have a long history of applications in a wide span of computer vision problems. With the advent of the deep learning paradigm shift in computer vision, tensors have become even more fundamental. Indeed, essential ingredients in modern deep learning architectures, such as convolutions and attention mechanisms, can readily be considered as tensor mappings. In effect, tensor methods are increasingly finding significant applications in deep learning, including the design of memory and compute efficient network architectures, improving robustness to random noise and adversarial attacks, and aiding the theoretical understanding of deep networks. This article provides an in-depth and practical review of tensors and tensor methods in the context of representation learning and deep learning, with a particular focus on visual data analysis and computer vision applications. Concretely, besides fundamental work in tensor-based visual data analysis methods, we focus on recent developments that have brought on a gradual increase in tensor methods, especially in deep learning architectures and their implications in computer vision applications. To further enable the newcomer to grasp such concepts quickly, we provide companion Python notebooks, covering key aspects of this article and implementing them, step-by-step with TensorLy. [ABSTRACT FROM AUTHOR]

Visual semantic information comprises two important parts: the meaning of each visual semantic unit and the coherent visual semantic relation conveyed by these visual semantic units. Essentially, the former one is a visual perception task while the latter corresponds to visual context reasoning. Remarkable advances in visual perception have been achieved due to the success of deep learning. In contrast, visual semantic information pursuit, a visual scene semantic interpretation task combining visual perception and visual context reasoning, is still in its early stage. It is the core task of many different computer vision applications, such as object detection, visual semantic segmentation, visual relationship detection, or scene graph generation. Since it helps to enhance the accuracy and the consistency of the resulting interpretation, visual context reasoning is often incorporated with visual perception in current deep end-to-end visual semantic information pursuit methods. Surprisingly, a comprehensive review for this exciting area is still lacking. In this survey, we present a unified theoretical paradigm for all these methods, followed by an overview of the major developments and the future trends in each potential direction. The common benchmark datasets, the evaluation metrics and the comparisons of the corresponding methods are also introduced. [ABSTRACT FROM AUTHOR]

Modern software systems, either system or application software, are increasingly being developed on top of virtualized software platforms. They may simply intend to execute on virtual machines or they may be expected to port to physical machines eventually. In either case, the devices, virtual or silicon, in the target virtual or physical machines are expected to conform to the specifications based on which the software systems have been developed. Non-conformance of these devices to the specifications can cause catastrophic failures of the software systems. In this article, we propose a mutation-based framework for effective and efficient conformance checking between virtual/silicon device implementations and their specifications. Based on our defined mutation operators, device specifications can be automatically instrumented with weak mutant-killing constraints to model potential erroneous device behaviors. To kill all feasible mutants, our approach adopts a cooperative symbolic execution mechanism that can efficiently automate the test case generation and conformance checking for virtual/silicon devices. By symbolically executing the instrumented specifications with virtual/silicon device traces obtained from the cooperative execution, our method can accurately measure whether the designs have been sufficiently validated and report the inconsistencies between device specifications and implementations. Comprehensive experiments on two industrial network adapters and their virtual devices demonstrate the effectiveness of our proposed approach in conformance checking for both virtual and silicon devices. [ABSTRACT FROM AUTHOR]

Prof. Tosiyasu L. Kunii (1938–2020) passed away on November 3, 2020. He was a longstanding member of the Editorial Board of IEEE Computer Graphics & Applications, serving for over 21 years from 1982 into 2002 and over the span of six different Editors-in-Chief. His own work was published 18 times over the years in the pages of IEEE CG&A, and his contributions to multiple research areas within computer graphics and computing are highly regarded. Among his many accomplishments, he was an IEEE Life Fellow and recipient of the prestigious IEEE Booth Education Award. With more than 50 published books and 500 refereed articles across the major publications and conferences of our field, and with colleagues and former students distributed around the world, his work and legacy will live on. [ABSTRACT FROM AUTHOR]

Dependency analysis is typically limited to the static analysis of code structures. We applied this practice to safety-critical systems that were re-engineered to reduce safety testing and technology upgrade cost. We discuss the need for a well-defined description of architectural dependencies to address the observed gaps. [ABSTRACT FROM AUTHOR]

Recent artificial intelligence (AI) advances and the consequent gains in convenience signify that AI adoption is imperative. The major questions are how AI may be used appropriately and what the contexts for such use are. [ABSTRACT FROM AUTHOR]