Search

Hyperdimensional computing (HDC) has emerged as an alternative lightweight learning solution to deep neural networks. A key characteristic of HDC is the great extent of parallelism that can facilitate hardware acceleration. However, previous hardware implementations of HDC seldom focus on GPU designs, which were also inefficient partly due to the complexity of accelerating HDC on GPUs. In this paper, we present OpenHD, a flexible and high-performance GPU-powered framework for automating the mapping of general HDC applications including classification and clustering to GPUs. OpenHD takes advantage of memory optimization strategies specialized for HDC, minimizing the access time to different memory subsystems, and removing redundant operations. We also propose a novel training method to enable data parallelism in HDC training. Our evaluation result shows that the proposed training rapidly achieves the target accuracy, reducing the required training epochs by 4×. With OpenHD, users can deploy GPU-accelerated HDC applications without domain expert knowledge. Compared to the state-of-the-art GPU-powered HDC implementation, our evaluation on NVIDIA Jetson TX2 shows that OpenHD is up to 10.5× and 314× faster for HDC-based classification and clustering, respectively. Compared with non-HDC classification and clustering on GPUs, OpenHD-based HDC is 11.7× and 53× faster at comparable accuracy. OpenHD is available at: https://github.com/UCSD-SEELab/openhd. [ABSTRACT FROM AUTHOR]

Current edge computing systems are deployed in highly complex application scenarios with dynamically changing requirements. In order to provide the expected performance and energy efficiency values in these situations, the use of heterogeneous hardware/software platforms at the edge has become widespread. However, these computing platforms still suffer from the lack of unified software-driven programming models to efficiently deploy multi-purpose hardware-accelerated solutions. In parallel, edge computing systems also face another huge challenge: operating under multiple conditions that were not taken into account during any of the design stages. Moreover, these conditions may change over time, forcing self-adaptation mechanisms to become a must. This paper presents an integrated architecture to exploit hardware-accelerated data-parallel models and transparent hardware/software multithreading. In particular, the proposed architecture leverages the ARTICo3 framework and ReconOS to allow developers to select the most suitable programming model to deploy their edge computing applications onto run-time reconfigurable hardware devices. An evolvable hardware system is used as an additional architectural component during validation, providing support for continuous lifelong learning in smart edge computing scenarios. In particular, the proposed setup exhibits online learning capabilities that include learning by imitation from software-based reference algorithms. Experimental results show the benefits of the proposed approach, exposing different run-time tradeoffs (e.g., computing performance versus functional correctness of the evolved solutions), and highlighting the benefits of using scalable data-parallel models to perform circuit evolution under dynamically changing application scenarios. [ABSTRACT FROM AUTHOR]

Training jobs of deep neural networks (DNNs) can be accelerated through distributed variants of stochastic gradient descent (SGD), where multiple nodes process training examples and exchange updates. The total throughput of the nodes depends not only on their computing power, but also on their networking speeds and coordination mechanism (synchronous or asynchronous, centralized or decentralized), since communication bottlenecks and stragglers can result in sublinear scaling when additional nodes are provisioned. In this paper, we propose two classes of performance models to predict throughput of distributed SGD: fine-grained models, representing many elementary computation/communication operations and their dependencies; and coarse-grained models, where SGD steps at each node are represented as a sequence of high-level phases without parallelism between computation and communication. Using a PyTorch implementation, real-world DNN models and different cloud environments, our experimental evaluation illustrates that, while fine-grained models are more accurate and can be easily adapted to new variants of distributed SGD, coarse-grained models can provide similarly accurate predictions when augmented with ad hoc heuristics, and their parameters can be estimated with profiling information that is easier to collect. [ABSTRACT FROM AUTHOR]

With the integration of Artificial Intelligence (AI) and Internet of Things (IoT), the Federated Edge Learning (FEL), a promising computing framework is developing. However, there are still unsolved issues on communication efficiency and data security due to the huge models and unreliable transmission links. To address these issues, this paper proposes a novel federated edge learning system, called LightFed, where the edge nodes upload only vital partial local models, and successfully achieve lightweight communication and model aggregation. First, a novel model aggregation method Model Splitting and Splicing (MSS) and a Selective Parameter Transmission (SPT) scheme are proposed. By detecting the updating gradients of local parameters and filtering significant parameters, selective rotated transmission and efficient aggregation of local models are achieved. Second, a Training Filling Model (TFM) is proposed to infer the total data distribution of edge nodes, and train a filling model to mitigate the unbalanced training data without violating the data privacy of individual users. Moreover, a blockchain-powered confusion transmission mechanism is proposed for defending the attacks from external adversaries and protecting the model information. Finally, extensive experimental results demonstrate that our LightFed significantly outperforms the existing FEL systems in terms of communication efficiency and privacy security. [ABSTRACT FROM AUTHOR]

As an emulous alternative to deep neural networks, Deep Forest emerges with features like low complexity, fewer hyper-parameters, and good robustness, which are predominantly desired in distributed computing applications and ecosystems. Recently, an efficient distributed Deep Forest system, named ForestLayer, was proposed, designing a fine-grained sub-Forest-based task-parallel algorithm to improve the parallel computing efficiency of Deep Forest. However, the sub-Forest splitting of ForestLayer is static and one-off without adaptability to the computing environment, nevertheless, the size of splitting granularity has a significant impact on the system performance. To further improve the computing efficiency and scalability of the distributed Deep Forest, in this paper, we propose a novel distributed Deep Forest algorithm, named BLB-gcForest (Bag of Little Bootstraps-gcForest), which augments the gcForest (multi-Grained Cascade Forest) approach for constructing Deep Forest. BLB-gcForest carries out parallel computation for each tree in sub-Forests at a finer parallel granularity and integrates with the Bag of Little Bootstraps (BLB) mechanism to reduce massive transmitted feature instances for Cascade Forest Layers, utterly improving both computation efficiency and communication efficiency. Moreover, to solve the problem of the forest splitting granularity, we further design an adaptive sub-Forest splitting algorithm to ensure the maximum resource utilization for parallel computation of each sub-Forest. Experimental results on four well-known large-scale datasets, namely YEAST, LETTER, MNIST, CIFAR10, show that the training efficiency of BLB-gcForest achieves up to 20.3x and 1.64x speedups compared with the state-of-the-art gcForest and ForestLayer, respectively while guaranteeing higher accuracy and better robustness [ABSTRACT FROM AUTHOR]

Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre. Previous works rely on conventional aggregation modules (e.g., dilated convolution, convolutional LSTM), which only make use of the local context. In this paper, we propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance, by progressively capturing the global context. We firstly develop a hierarchy Transformer to capture intra-video relation that includes richer spatial and temporal cues from neighbor pixels and previous frames. A joint space-time window shift scheme is proposed to efficiently aggregate these two cues into each pixel embedding. Then, we explore inter-video relation via pixel-to-pixel contrastive learning, which well structures the global embedding space. A multi-source contrast training objective is developed to group the pixel embeddings across videos with the ground-truth guidance, which is crucial for learning the global property of the whole data. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches. Code is available at https://github.com/YuemingJin/STswinCL. [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

Video summarization (VS) is generally formulated as a subset selection problem where a set of representative keyframes or key segments is selected from an entire video frame set. Though many sparse subset selection based VS algorithms have been proposed in the past decade, most of them adopt linear sparse formulation in the explicit feature vector space of video frames, and don’t consider the local or global relationships among frames. In this paper, we first extend the conventional sparse subset selection for VS into kernel block sparse subset selection (KBS3) to utilize the advantage of kernel sparse coding and introduce a local inter-frame relationship through packing of frame blocks. Going a step further, we propose a similarity based block sparse subset selection (SB2S3) model by applying a specially designed transformation matrix on the KBS3 model in order to introduce a kind of global inter-frame relationship through the similarity. Finally, a greedy pursuit based algorithm is devised for the proposed NP-hard model optimization. The proposed SB2S3 has the following advantages: 1) through the similarity between each frame and any other frame, the global relationship among all frames can be considered; 2) through block sparse coding, the local relationship of adjacent frames is further considered; and 3) it has a wider application, since features can derive similarity, but not vice versa. It is believed that the effect of modeling such global and local relationships among frames in this paper, is similar to that of modeling the long-range and short-range dependencies among frames in deep learning based methods. Experimental results on three benchmark datasets have demonstrated that the proposed approach is superior to not only other sparse subset selection based VS methods but also most unsupervised deep-learning based VS methods. [ABSTRACT FROM AUTHOR]

Time-domain indirect methods based on Lyapunov stability theory and linear matrix inequality techniques (LMIs) have been applied for delay-dependent stability analysis of large-scale load frequency control (LFC) schemes. This paper aims to enhance the numerical tractability of large-scale LMIs by exploiting the special characteristics of the LFC loops. First, in the typical LFC model, only a few delayed states that are directly influenced by transmission delays are distinguished from other normal system states. Hence, an improved reconstruction model is formed, based on which the delay-dependent stability condition is established with the decreased order of the LMIs and decision variables. Then, to further improve the numerical tractability of the developed stability criterion, all weighting matrices required in the augmented Lyapunov functional are enforced to have structural restrictions by proposing an extended symmetry-exploiting technique. Case studies show that the method proposed in this paper significantly improves the calculation efficiency of stability criterion established for multi-area power systems at the cost of only a minor reduction in computational accuracy. [ABSTRACT FROM AUTHOR]

Interactive model steering helps people incrementally build machine learning models that are tailored to their domain and task. Existing visual analytic tools allow people to steer a single model (e.g., assignment attribute weights used by a dimension reduction model). However, the choice of model is critical in such situations. What if the model chosen is suboptimal for the task, dataset, or question being asked? What if instead of parameterizing and steering this model, a different model provides a better fit? This paper presents a technique to allow users to inspect and steer multiple machine learning models. The technique steers and samples models from a broader set of learning algorithms and model types. We incorporate this technique into a visual analytic prototype, BEAMES, that allows users to perform regression tasks via multimodel steering. This paper demonstrates the effectiveness of BEAMES via a use case, and discusses broader implications for multimodel steering. [ABSTRACT FROM AUTHOR]

Service restoration is one of the critical functions to enable the future self-healing distribution system. To restore the distribution system in a timely and reliable manner, the realistic system operating conditions need to be accurately characterized. In this paper, two main factors that have great impacts on distribution system restoration (DSR) in practice are investigated. First, cold load pickup (CLPU), generally caused by thermostatically controlled loads (TCLs), is a common phenomenon after an outage and shaped by the outage duration. However, the time-dependent behaviors of CLPU are rarely considered in literature. In this paper, the operating state evolution of TCLs after an outage is analyzed to characterize time-dependent CLPU. And the time-dependent CLPU is analytically embedded in DSR to accurately represent the actual behaviors of the restored loads. Second, it is difficult to predict loads that fluctuate during DSR due to the lack of real-time measurement data. Accordingly, a robust DSR based on the information gap decision theory (IGDT) is proposed to address this challenge, fully considering the uncertainty of CLPU. The proposed models are tested in IEEE 13-node and 123-node test feeders. Simulation results demonstrate that the time-dependent CLPU model and the uncertainty modeling of CLPU can accurately capture the actual behaviors of loads with TCLs after an outage, which greatly improves DSR decisions in practice. [ABSTRACT FROM AUTHOR]

This paper proposes a cloud-assisted joint charging scheduling and energy management framework for unmanned aerial vehicle (UAV) networks. For charging the UAVs those are extremely power hungry, charging towers are considered for plug-and-play charging during run-time operations. The charging towers should be cost-effective, thus it is equipped with photovoltaic power generation and energy storage systems functionalities. Furthermore, the towers should be cooperative for more cost-effectiveness by intelligent energy sharing. Based on the needs and setting, this paper proposes 1) charging scheduling between UAVs and towers and 2) cooperative energy managements among towers. For charging scheduling, the UAVs and towers should be scheduled for maximizing charging energy amounts and the scheduled pairs should determine charging energy allocation amounts. Here, two decisions are correlated, i.e., it is a non-convex problem. We re-formulate the non-convex to convex for guaranteeing optimal solutions. Lastly, the cooperative energy sharing among towers is designed and implemented with multi-agent deep reinforcement learning and then intelligent energy sharing can be realized. We can observe that the two methods are related and it should be managed, coordinated, and harmonized by a centralized orchestration manager under the consideration of fairness, energy-efficiency, and cost-effectiveness. Our data-intensive performance evaluation verifies that our proposed framework achieves desired performance. [ABSTRACT FROM AUTHOR]

Product lifecycle management (PLM) is an engineering management approach which helps organizations to manage product-related technical data for its complete lifecycle. The right product data enables better business functioning in organizations. PLM systems are recent and upcoming information technology tools in manufacturing firms. Although PLM systems were initially introduced as the on-premise tools, the current cloud technology is taking it from organizational boundaries to a global level through the Internet. Most enterprise applications have already started moving to clouds; PLM is also all set to be utilized through clouds. But it brings various challenges during cloud PLM adoption. Hence, this paper aims to empirically investigate the critical challenges that manufacturing firms face while adopting cloud PLM. There is rare academic research in the cloud PLM domain. This paper results in the three most significant challenges viz., “Data security on clouds,” “Lack of trust on services,” and “Lack of customization.” These critical challenges not only help the managers in manufacturing firms in planning to migrate on clouds but also the technical cloud providers to improve the cloud services and systems appropriately in order to increase the compatibility of cloud PLM in firms. The present paper uniquely identifies the critical challenges for cloud PLM adoption in large manufacturing firms. [ABSTRACT FROM AUTHOR]

Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. Existing solutions either rely on hand-crafted descriptors or supervised gait representation learning. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID. Specifically, we first create self-supervision by learning to reconstruct unlabeled skeleton sequences reversely, which involves richer high-level semantics to obtain better gait representations. Other pretext tasks are also explored to further improve self-supervised learning. Second, inspired by the fact that motion’s continuity endows adjacent skeletons in one skeleton sequence and temporally consecutive skeleton sequences with higher correlations (referred as locality in 3D skeleton data), we propose a locality-aware attention mechanism and a locality-aware contrastive learning scheme, which aim to preserve locality-awareness on intra-sequence level and inter-sequence level respectively during self-supervised learning. Last, with context vectors learned by our locality-aware attention mechanism and contrastive learning scheme, a novel feature named Constrastive Attention-based Gait Encodings (CAGEs) is designed to represent gait effectively. Empirical evaluations show that our approach significantly outperforms skeleton-based counterparts by 15-40 percent Rank-1 accuracy, and it even achieves superior performance to numerous multi-modal methods with extra RGB or depth information. Our codes are available at https://github.com/Kali-Hac/Locality-Awareness-SGE. [ABSTRACT FROM AUTHOR]

It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices, because even the powerful modern mobile devices are considered as “resource-constrained” when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to $14.08\times$ 14. 08 × speedup. [ABSTRACT FROM AUTHOR]

With substantial amount of time, resources and human (team) efforts invested to explore and develop successful deep neural networks (DNN), there emerges an urgent need to protect these inventions from being illegally copied, redistributed, or abused without respecting the intellectual properties of legitimate owners. Following recent progresses along this line, we investigate a number of watermark-based DNN ownership verification methods in the face of ambiguity attacks, which aim to cast doubts on the ownership verification by forging counterfeit watermarks. It is shown that ambiguity attacks pose serious threats to existing DNN watermarking methods. As remedies to the above-mentioned loophole, this paper proposes novel passport-based DNN ownership verification schemes which are both robust to network modifications and resilient to ambiguity attacks. The gist of embedding digital passports is to design and train DNN models in a way such that, the DNN inference performance of an original task will be significantly deteriorated due to forged passports. In other words, genuine passports are not only verified by looking for the predefined signatures, but also reasserted by the unyielding DNN model inference performances. Extensive experimental results justify the effectiveness of the proposed passport-based DNN ownership verification schemes. Code is available at https://github.com/kamwoh/DeepIPR. [ABSTRACT FROM AUTHOR]

$\ell _{1}$ ℓ 1 -norm quantile regression is a common choice if there exists outlier or heavy-tailed error in high-dimensional data sets. However, it is computationally expensive to solve this problem when the feature size of data is ultra high. As far as we know, existing screening rules can not speed up the computation of the $\ell _{1}$ ℓ 1 -norm quantile regression, which dues to the non-differentiability of the quantile function/pinball loss. In this paper, we introduce the dual circumscribed sphere technique and propose a novel $\ell _{1}$ ℓ 1 -norm quantile regression screening rule. Our rule is expressed as the closed-form function of given data and eliminates inactive features with a low computational cost. Numerical experiments on some simulation and real data sets show that this screening rule can be used to eliminate almost all inactive features. Moreover, this rule can help to reduce up to 23 times of computational time, compared with the computation without our screening rule. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep Efficient Relational Sentence Ordering Network (referred to as ERSON) by leveraging pre-trained language model in both encoder and decoder architectures to strengthen the coherence modeling of the entire model. Specifically, we first introduce a divide-and-fuse BERT (referred to as DF-BERT), a new refactor of BERT network, where lower layers in the improved model encode each sentence in the paragraph independently, which are shared by different sentence pairs, and the higher layers learn the cross-attention between sentence pairs jointly. It enables us to capture the semantic concepts and contextual information between the sentences of the paragraph, while significantly reducing the runtime and memory consumption without sacrificing the model performance. Besides, a Relational Pointer Decoder (referred to as RPD) is developed, which utilizes the pre-trained Next Sentence Prediction (NSP) task of BERT to capture the useful relative ordering information between sentences to enhance the order predictions. In addition, a variety of knowledge distillation based losses are added as auxiliary supervision to further improve the ordering performance. The extensive evaluations on Sentence Ordering, Order Discrimination, and Multi-Document Summarization tasks show the superiority of ERSON to the state-of-the-art ordering methods. [ABSTRACT FROM AUTHOR]

Dynamic routing networks, aimed at finding the best routing paths in the networks, have achieved significant improvements to neural networks in terms of accuracy and efficiency. In this paper, we see dynamic routing networks in a fresh light, formulating a routing method as a mapping from a sample space to a routing space. From the perspective of space mapping, prevalent methods of dynamic routing did not take into account how routing paths would be distributed in the routing space. Thus, we propose a novel method, termed CoDiNet, to model the relationship between a sample space and a routing space by regularizing the distribution of routing paths with the properties of consistency and diversity. In principle, the routing paths for the self-supervised similar samples should be closely distributed in the routing space.Moreover, we design a customizable dynamic routing module, which can strike a balance between accuracy and efficiency. When deployed upon ResNet models, our method achieves higher performance and effectively reduces average computational cost on four widely used datasets. [ABSTRACT FROM AUTHOR]

Many datasets including social media data and bibliographic data can be modeled as graphs. Clustering such graphs is able to provide useful insights into the structure of the data. To improve the quality of clustering, node attributes can be taken into account, resulting in attributed graphs. Existing attributed graph clustering methods generally consider attribute similarity and structural similarity separately. In this paper, we represent attributed graphs as star-schema heterogeneous graphs, where attributes are modeled as different types of graph nodes. This enables the use of personalized pagerank (PPR) as a unified distance measure that captures both structural and attribute similarities. We employ DBSCAN for clustering, and we update edge weights iteratively to balance the importance of different attributes. The rapidly growing volume of data nowadays challenges traditional clustering algorithms, and thus, a distributed method is required. Hence, we adopt a popular distributed graph computing system Blogel, based on which, we develop four exact and approximate approaches that enable efficient PPR score computation when edge weights are updated. To improve the effectiveness of the clustering, we propose a simple yet effective edge weight update strategy based on entropy. In addition, we present a game theory based method that enables trading efficiency for result quality. Extensive experiments on real-life datasets offer insights into the effectiveness and efficiency of our proposals. [ABSTRACT FROM AUTHOR]

Community detection is one of the most fundamental problems in social network analysis, while most existing research focuses on unsigned graphs. In real applications, social networks involve not only positive relationships but also negative ones. It is important to exploit the signed information to identify more stable communities. In this paper, we propose a novel model, named stable $k$ k -core, to measure the stability of a community in signed graphs. The stable $k$ k -core model not only emphasizes user engagement, but also eliminates unstable structures. We show that the problem of finding the maximum stable $k$ k -core is NP-hard. To scale for large graphs, novel pruning strategies and searching methods are proposed. We conduct extensive experiments on 6 real-world signed networks to verify the efficiency and effectiveness of proposed model and techniques. [ABSTRACT FROM AUTHOR]

Subgraph queries are routinely used to search for entities in richly attributed graphs e.g., social networks and knowledge graphs. With little knowledge of underlying data, users often need to rewrite queries multiple times to reach desirable answers. Why-questions are studied to clarify missing or unexpected query results. This paper makes a first step to answer Why-questions for entity search in attributed graphs. We consider three common types of Why-questions: Why-not, Why, and Why-rank, which suggest query manipulations that are responsible for user-specified missing, unexpected, and undesirably ranked entities, respectively. (1) We approach a general query rewriting paradigm that suggests to identify desired entities that are specified by Why-questions. We introduce measures that characterize good query rewrites by incorporating both query editing cost and answer closeness. (2) While computing optimal query rewrites is intractable, we develop feasible algorithms, from approximation to fast heuristics, and provide query rewrites with (near) optimality guarantees whenever possible, for Why, Why-not and Why-rank questions. We further show that our results remain intact for Why questions that (1) request a single query rewrite to clarify multiple types of entities, and (2) variants such as Why-empty and Why-so-many, by providing the matching algorithms. Using real-world graphs, we experimentally verify that our algorithms are effective and feasible for large graphs. Our case study also verifies their application in e.g., knowledge exploration. [ABSTRACT FROM AUTHOR]

This paper presents a passive lumped model for DC cables and overhead lines, taking into account the frequency dependency of the per-unit-length parameters due to the skin and proximity effects. In comparison to distributed parameter line models, such as the Frequency-Dependent Phase model of PSCAD, the proposed model does not require line propagation delay extraction and does not impose any restrictions on the simulation timestep. As a result, for electrically short lines, the introduced model is more computationally efficient than distributed parameter models as it enables simulation using larger timesteps. The performance of the proposed model is evaluated in comparison with the PI model and the Frequency-Dependent Phase model through time-domain simulations of a DC microgrid and a radial DC distribution feeder. The studies performed in PSCAD focus on fault-induced transients. The results indicate that for electrically short lines the developed model is significantly more accurate compared to the PI model and provides similar accuracy with more computational efficiency compared to the Frequency-Dependent Phase model. [ABSTRACT FROM AUTHOR]

This paper presents frequency domain (FD) Norton-type equivalents of single- and three-phase photovoltaic (PV) generators, aimed at transient calculations. The proposed equivalents can be interfaced with a general frequency-dependent network and/or nonlinear loads and include voltage/current sources representing initial conditions aimed at partitioned-time transient simulation. The transient solution is performed via the numerical Laplace transform (NLT) in a sequential scheme by using time sub-windows with corresponding initial conditions and using reduced number of samples. The proposed approach overcomes the issue of computationally heavy simulations as in classical implementations of NLT. Two case studies involving PV generator and nonlinear load are presented to illustrate the performance of the proposed FD PV models. [ABSTRACT FROM AUTHOR]

This papers presents a numerical approach based on computational fluid dynamics (CFD) that was developed to ensure that gravity oil-water separators at Hydro-Québec electrical substations comply with the environmental legislation in place. More precisely, multiphase simulations are performed for two operating conditions of an in service separator, and an experimental setup is used to produce an oil spill and to measure the time-varying oil concentration at the separator outlet. The results show that, for the tested conditions, the oil droplet size at a separator inlet generally follows a log-normal distribution and that the droplet diameter varies between 100 and 1000 µm with a mean value of 475 µm. Furthermore, both the experimental and numerical results show that the time required for the oil droplets to reach the separator outlet is quite shorter than the theoretical mean residence time. Finally, this study demonstrates that the numerical model predicts the separator efficiency within 1% of the measured value, and that it can be a valuable tool to investigate existing separators or to design new ones. [ABSTRACT FROM AUTHOR]

Point cloud generative models have aroused increasing concern for their realistic generation potentialities. However, most existing methods adopt deep-neural-network (DNN) models for continuous mapping. DNN usually induces mode collapse and mixture problems without clear interpretation. Consequently, in this paper, we design a geometry-inspired point cloud generative framework called PointOT. PointOT decouples the generative model into two separate sub-tasks: manifold learning of the point cloud and distribution transformation. Then, we propose corresponding instances according to the requirements in each sub-task, where they can be established by the point cloud auto-encoder (AE) and the semi-continuous optimal transportation (SCOT) mapping, respectively. In particular, the transportation map between the source and the target distributions is discrete rather than continuous in geometric view. The learned continuous shape model of the DNN point cloud does not conform with the discrete distribution transformation. Therefore, the proposed SCOT efficiently relieves these problems by connecting the continuous-to-discrete domain. Besides, we provide theoretical explanations from a geometric view and analyze the fundamental reason for mode collapse and mixture in point cloud generative models. The proposed SCOT algorithm without the DNN model is computationally efficient and makes the original black box semi-transparent. Final experiments validate the virtue of the proposed approach, including the designed decomposition framework and the rigorous theory. [ABSTRACT FROM AUTHOR]

In order to retain more feature information of local areas on a point cloud, local grouping and subsampling are the necessary data structuring steps in most hierarchical deep learning models. Due to the disorder nature of the points in a point cloud, the significant time cost may be consumed when grouping and subsampling the points, which consequently results in poor scalability. This paper proposes a fast data structuring method called PSNet (Point Structuring Net). PSNet transforms the spatial features of the points and matches them to the features of local areas in a point cloud. PSNet achieves grouping and sampling at the same time while the existing methods process sampling and grouping in two separate steps (such as using FPS plus kNN). PSNet performs feature transformation pointwise while the existing methods uses the spatial relationship among the points as the reference for grouping. Thanks to these features, PSNet has two important advantages: 1) the grouping and sampling results obtained by PSNet is stable and permutation invariant; and 2) PSNet can be easily parallelized. PSNet can replace the data structuring methods in the mainstream point cloud deep learning models in a plug-and-play manner. We have conducted extensive experiments. The results show that PSNet can improve the training and inference speed significantly while maintaining the model accuracy. [ABSTRACT FROM AUTHOR]

Attention mechanisms have significantly boosted the performance of video classification neural networks thanks to the utilization of perspective contexts. However, the current research on video attention generally focuses on adopting a specific aspect of contexts (e.g., channel, spatial/temporal, or global context) to refine the features and neglects their underlying correlation when computing attentions. This leads to incomplete context utilization and hence bears the weakness of limited performance improvement. To tackle the problem, this paper proposes an efficient attention-in-attention (AIA) method for element-wise feature refinement, which investigates the feasibility of inserting the channel context into the spatio-temporal attention learning module, referred to as CinST, and also its reverse variant, referred to as STinC. Specifically, we instantiate the video feature contexts as dynamics aggregated along a specific axis with global average and max pooling operations. The workflow of an AIA module is that the first attention block uses one kind of context information to guide the gating weights calculation of the second attention that targets at the other context. Moreover, all the computational operations in attention units act on the pooled dimension, which results in quite few computational cost increase (< 0.02%). To verify our method, we densely integrate it into two classical video network backbones and conduct extensive experiments on several standard video classification benchmarks. The source code of our AIA is available at https://github.com/haoyanbin918/Attention-in-Attention. [ABSTRACT FROM AUTHOR]

Intelligent metasurfaces to create smart electromagnetic environments have attracted a surge of interest. The multiscale property of metasurfaces results in a huge computational challenge, further hindering their engineering development. Our previous paper has developed a fast finite difference time domain (FDTD) algorithm for patchlike metasurfaces analysis using the surface susceptibility model (SSM). This study extends that to efficient fishnetlike metasurfaces analysis using the surface porosity model (SPM), a useful complement to SSM. We introduce the SPM to FDTD, replacing brute force simulations, to accelerate fishnetlike metasurfaces analysis. At last, we conduct a series of numerical experiments, demonstrating that the proposed FDTD-SPM algorithm greatly reduces the number of computational meshing lattices and the consumed time. [ABSTRACT FROM AUTHOR]

In many papers, random modulation of power electronic converters is described as an electromagnetic interference (EMI) mitigation technique, which contributes to the assurance of electromagnetic compatibility (EMC) of systems consisting of power electronic converters and serial transmission-based control and measuring devices. However, experimental results as well as theoretical analyses of physical phenomenon suggest that the EMI reduction is ostensible and results from the measuring procedure adopted in standards. This article presents the first known approach to compare the influence of deterministic and random modulation of transmission errors rate in serial communication systems. The comparison is based on the especially developed mathematical models, which are validated experimentally, using a simple setup with a dc/dc converter injecting a noise into the USB/RS232 converter representing a serial transmission scheme. The comparison reveals that for the studied switching frequency range from 10 kHz to 1 MHz, on average, there is no significant difference between random and deterministic modulations with respect to the probability of error rate in serial communication systems. Thus, in such systems, random modulation should not be considered as a valid EMC-assuring technique. [ABSTRACT FROM AUTHOR]

Ising machines are expected to solve combinatorial optimization problems efficiently by representing them as Ising models or equivalent quadratic unconstrained binary optimization (QUBO) models. However, upper bound exists on the computable problem size due to the hardware limitations of Ising machines. This paper propose a new hybrid annealing method based on partial QUBO extraction, called subQUBO model extraction, with multiple solution instances. For a given QUBO model, the proposed method obtains $N_I$ N I quasi-optimal solutions (quasi-ground-state solutions) in some way using a classical computer. The solutions giving these quasi-optimal solutions are called solution instances. We extract a size-limited subQUBO model as follows based on a strong theoretical background: we randomly select $N_S$ N S $(N_S

This paper presents a novel fault detection approach for industrial batch processes. The batch processes under consideration are characterized by the interaction between discrete system modes and non-stationary continuous dynamics. Therefore, a stochastic hybrid process model (SHPM) is introduced, where process variables are modeled as time-variant Gaussian distributions, which depend on hidden system modes. Transitions between the system modes are assumed to be either autonomous or to be triggered by observable events such as on/off signals. The model parameters are determined from training data using expectation-maximization techniques. A new fault detection algorithm is proposed, which assesses the likelihoods of sensor signals on the basis of the stochastic hybrid process model. Evaluation of the proposed fault detection system has been conducted for a penicillin production process, with the results showing a significant improvement over the existing baseline methods.Note to Practitioners—Automatic fault detection makes it possible to limit the effects of faults by taking countermeasures at an early stage. In this work, a data-driven fault detection method for industrial batch processes is proposed, in which the underlying process model is learned from training data. The proposed fault detection system can be used for various industrial batch processes without the need for complex and error-prone manual configuration. In contrast to many other data-driven approaches such as neural networks, only a few process cycles are required to create a robust process model. It should be noted that in data-driven fault detection methods, the training data should cover a large part of the process states that occur during error-free process cycles. The developed method is therefore particularly suitable for cyclical processes, which, however, can have alternative process paths and variability between the process cycles. [ABSTRACT FROM AUTHOR]

In order to generalize the applicability of Conditional Value at Risk, one of the most widely used measurements used in financial risk management, we develop a solution methodology for the conditional expectation (CE)-based simulation optimization problems. To optimize CE-based objective functions in a highly generalized context, we propose a gradient-free, direct search optimization method, called SNM-CE, which inherits the search framework of Stochastic Nelder-Mead (SNM) Simplex Method but further incorporates effective mechanisms designed for handling problems with CE-based objective functions. As we assume the underlying problem is complicated enough that no closed-form expression can represent the objective function, stochastic simulation is applied to estimate CE. We apply Importance Sampling (IS) as a variance reduction technique, which, combined with a newly-developed methodology, called SOCBA-mn, ensures that simulation resources are used with great efficiency. We show that SNM-CE can converge to the true global optimum with probability one (w.p.1) like SNM. An extensive numerical study and a communication system-based empirical study are both conducted to demonstrate the effectiveness, efficiency and viability of this research in both theoretical and practical settings. Note to Practitioners—This paper develops a direct search algorithm, called SNM-CE, used for solving conditional expectation-based simulation optimization problems. By tackling conditional expectation-based problems, SNM-CE fills a gap in the stochastic optimization literature which has traditionally focused on expectation- or quantile-based objective functions. SNM-CE possesses a high degree of flexibility and generalizability which users can benefit from in solving real-world applications. For example, the $\alpha $ value in the CE-based simulation optimization formulation can be freely adjusted. In other words, the quantile value above which the CE is estimated and optimized can be set according to the needs of the practitioner and/or characteristics of the problem to be solved. Also, SNM-CE is designed such that instead of setting the $\alpha $ value, the user may choose to utilize a particular numerical value of the objective function as the threshold for the CE estimation and optimization. SNM-CE is also simple in terms of implementation and, being a direct search method, does not impose many assumptions about the structure of the underlying objective function and does not necessitate the use of gradient information in the search process. As one application example, SNM-CE can be utilized to select the operational parameters which minimize the average delay in the manufacturing of semiconductors given that the delay exceeds the 90th percentile of simulated delay times. [ABSTRACT FROM AUTHOR]

In medical image analysis (MIA) and computer-assisted surgery (CAS), aligning two multiple point sets (PSs) together is an essential but also a challenging problem. For example, rigidly aligning multiple point sets into one common coordinate frame is a prerequisite for statistical shape modelling (SSM). Accurately aligning the pre-operative space with the intra-operative space in CAS is very crucial to successful interventions. In this article, we formally formulate the multiple generalized point set registration problem (MGPSR) in a probabilistic manner, where both the positional and the normal vectors are used. The six-dimensional vectors consisting of both positional and normal vectors are called as generalized points. In the formulated model, all the generalized PSs to be registered are considered to be the realizations of underlying unknown hybrid mixture models (HMMs). By assuming the independence of the positional and orientational vectors (i.e., the normal vectors), the probability density function (PDF) of an observed generalized point is computed as the product of Gaussian and Fisher distributions. Furthermore, to consider the anisotropic noise in surgical navigation, the positional error is assumed to obey a multi-variate Gaussian distribution. Finally, registering PSs is formulated as a maximum likelihood (ML) problem, and solved under the expectation maximization (EM) technique. By using more enriched information (i.e., the normal vectors), our algorithm is more robust to outliers. By treating all PSs equally, our algorithm does not bias towards any PS. To validate the proposed approach, extensive experiments have been conducted on surface points extracted from CT images of (i) a human femur bone model; (ii) a human pelvis bone model. Results demonstrate our algorithm’s high accuracy, robustness to noise and outliers. Note to Practitioners—This paper was motivated by solving the problem of registering two or more PSs. Most existing registration approaches use only the positional information associated with each point, and thus lacks robustness to noise and outliers. Three significant improvements are brought by our proposed approach. First, the normal vectors that can be extracted from the point sets are utilized in the registration. Second, the positional error distribution is assumed to be anisotropic and inhomogeneous. Third, all the PSs to be registered are treated equally that means no PS is considered as the model one. The registration problem is cast into a maximum likelihood (ML) problem and solved under the expectation maximization (EM) framework. We have demonstrated through extensive experiments that the proposed registration approach achieves significantly improved accuracy, robustness to noise and outliers. The algorithm is particularly suitable for biomedical applications involving the registration procedures, such as image-guided surgery. [ABSTRACT FROM AUTHOR]

It is found from actual data that due to generation dispatch and uncertain renewable generations and loads with complicated correlations, inferring the probabilistic distributions for uncertain inputs is challenging. Many probabilistic power flow approaches have been developed in the literature but their validations using realistic systems and data are lacking. This paper proposes a data-driven probabilistic analysis approach for system risk assessment of the miniWECC system using actual data. The sparse Gaussian process (SGP) is advocated to quantify the impacts of uncertain inputs on voltage security. SGP does not need the probability distribution function of uncertain inputs, can handle correlations and is highly computationally efficient. Results on the miniWECC system using realistic data show that SGP outperforms existing approaches and is able to quantify the voltage violation risks. [ABSTRACT FROM AUTHOR]

In this paper, we propose a data-driven approach for uncertainty moment propagation through the non-linear power system dynamics. The proposed approach relies on the linear representation of a nonlinear system using the Perron-Frobenius & Koopman operators for propagating the moments. Data from non-linear simulations is used to approximate the linear operators by matrices, thereby enabling moment propagation by matrix multiplication. Results for a large-scale system are presented to demonstrate the accuracy and speed-up of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, an approximate aggregated impedance model (AAIM) of a grid-connected wind farm with multiple wind turbine generators (WTGs) for the small-signal stability study is derived. The derivation is conducted under the condition that the WTGs inside the wind farm are of the same type and from the same manufacturer. The AAIM derived can be seen as the impedance model (IM) of a grid-connected aggregated WTG. The IM of the aggregated WTG is the weighted average of the IMs of the individual WTGs in the wind farm and hence is simple to be built. The reactance of the line connecting the aggregated WTG to the external AC grid is the maximum eigenvalue of the network reactance matrix of the grid-connected wind farm. In addition, the derivation justifies the representation of a wind farm by an aggregated WTG in the IM-based small-signal stability analysis. Finally, two example grid-connected wind farms are used to evaluate and demonstrate the effectiveness of the AAIM proposed in the small-signal stability analysis. It is also shown that the computational burden of the stability analysis of a large-scale wind farm could be effectively reduced when the AAIM is used. [ABSTRACT FROM AUTHOR]

The quantification of heating, ventilation, and air condition (HVAC) system flexibility is critical to the operations of both the grid and buildings in demand response (DR) programs. However, the flexibility quantification is challenging due to the non-linearity and non-convexity of thermal dynamics associated with HVAC components. This paper proposes a novel HVAC flexibility quantification method based on a semidefinite programming (SDP) formulation. The SDP is reformulated from the non-convex problem of HVAC power optimization, and can be solved efficiently in real-time. The physics-based HVAC model is incorporated to ensure the reliability and accuracy of solutions. The quantification results are organized into an HVAC flexibility table that can provide response strategies on adjusting HVAC setpoints in response to the grid signals received. The developed response strategies minimize occupant discomfort while satisfying grid requirements. A case study of a test building model is carried out to illustrate the flexibility quantification framework and compares the performance of two DR strategies. [ABSTRACT FROM AUTHOR]

The paper establishes convergence properties of decentralized and distributed control strategies that use a Steinmetz-based technique for mitigating voltage unbalance on distribution feeders. Reactive power from distributed energy resources (DERs), such as solar photovoltaic (PV), is used to balance voltages at critical nodes in the distribution network. Interactions between controllers are analyzed and a theoretical guarantee is derived for stability of the control algorithms. This is achieved using Banach fixed-point theory. Providing such convergence results ensures robustness of the proposed control algorithms in realistic settings where parameters are uncertain and/or slowly varying, disturbances are prevalent, and control and measurement signals are prone to delays. The convergence results are evaluated on three test distribution networks. [ABSTRACT FROM AUTHOR]

Deep learning (DL) techniques have shown promising performance for designing data-driven power system transient stability assessment (TSA) models. However, due to the deep structure of the DL, the resulting model is always a black-box and hard to explain, which hinders its practical adoption by the industry. This paper proposes an interpretable DL-based TSA model to balance the TSA accuracy and transparency. The proposed method combines the strong nonlinear modelling capability of a deep neural network and the interpretability of a decision tree (DT). Through regularizing DL-based model with the average DT path length in the training process, the proposed interpretable DL-based TSA method can visually explain the TSA decision-making process. Simulation results have shown that the proposed method can deliver highly accurate TSA results and interpretable TSA decision-making rules, which can be used for designing preventive control actions. [ABSTRACT FROM AUTHOR]

As a challenging task of high-level video understanding, weakly supervised temporal action localization has attracted more attention recently. With only video-level category labels, this task should indistinguishably identify the background and action categories frame by frame. However, it is non-trivial to achieve this in untrimmed videos, due to the unconstrained background, complex and multi-label actions. With the observation that these difficulties are mainly brought by the large variations within background and actions, we propose to address these challenges from the perspective of modeling variations. Moreover, it is desired to further reduce the variations, or learn compact features, so as to cast the problem of background identification as rejecting background and alleviate the contradiction between classification and detection. Accordingly, in this paper, we propose a two-branch relational prototypical network. The first branch, namely action-branch, adopts class-wise prototypes and mainly acts as an auxiliary to introduce priori knowledge about label dependencies and be a guide for the second branch. Meanwhile, the second branch, namely sub-branch, starts with multiple prototypes, namely sub-prototypes, to enable a powerful ability of modeling variations. As a further benefit, we elaborately design a multi-label clustering loss based on the sub-prototypes to learn compact features under the multi-label setting. The two branches are associated using the correspondences between two types of prototypes, leading to a special two-stage classifier in the s-branch, on the other hand, the two branches serve as regularization terms to each other, improving the final performance. Ablation studies find that the proposed model is capable of modeling classes with large variations and learning compact features. Extensive experimental evaluations on Thumos14, MultiThumos and ActivityNet datasets demonstrate the effectiveness of the proposed method and superior performance over state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Existing RGB-D salient object detection (SOD) models usually treat RGB and depth as independent information and design separate networks for feature extraction from each. Such schemes can easily be constrained by a limited amount of training data or over-reliance on an elaborately designed training process. Inspired by the observation that RGB and depth modalities actually present certain commonality in distinguishing salient objects, a novel joint learning and densely cooperative fusion (JL-DCF) architecture is designed to learn from both RGB and depth inputs through a shared network backbone, known as the Siamese architecture. In this paper, we propose two effective components: joint learning (JL), and densely cooperative fusion (DCF). The JL module provides robust saliency feature learning by exploiting cross-modal commonality via a Siamese network, while the DCF module is introduced for complementary feature discovery. Comprehensive experiments using five popular metrics show that the designed framework yields a robust RGB-D saliency detector with good generalization. As a result, JL-DCF significantly advances the state-of-the-art models by an average of $\sim 2.0\%$ ∼ 2. 0 % (max F-measure) across seven challenging datasets. In addition, we show that JL-DCF is readily applicable to other related multi-modal detection tasks, including RGB-T (thermal infrared) SOD and video SOD, achieving comparable or even better performance against state-of-the-art methods. We also link JL-DCF to the RGB-D semantic segmentation field, showing its capability of outperforming several semantic segmentation models on the task of RGB-D SOD. These facts further confirm that the proposed framework could offer a potential solution for various applications and provide more insight into the cross-modal complementarity task. [ABSTRACT FROM AUTHOR]

Cameras currently allow access to two image states: (i) a minimally processed linear raw-RGB image state (i.e., raw sensor data); or (ii) a highly-processed nonlinear image state (e.g., sRGB). There are many computer vision tasks that work best with a linear image state, such as image deblurring and image dehazing. Unfortunately, the vast majority of images are saved in the nonlinear image state. Because of this, a number of methods have been proposed to “unprocess” nonlinear images back to a raw-RGB state. However, existing unprocessing methods have a drawback because raw-RGB images are sensor-specific. As a result, it is necessary to know which camera produced the sRGB output and use a method or network tailored for that sensor to properly unprocess it. This paper addresses this limitation by exploiting another camera image state that is not available as an output, but it is available inside the camera pipeline. In particular, cameras apply a colorimetric conversion step to convert the raw-RGB image to a device-independent space based on the CIE XYZ color space before they apply the nonlinear photo-finishing. Leveraging this canonical image state, we propose a deep learning framework, CIE XYZ Net, that can unprocess a nonlinear image back to the canonical CIE XYZ image. This image can then be processed by any low-level computer vision operator and re-rendered back to the nonlinear image. We demonstrate the usefulness of the CIE XYZ Net on several low-level vision tasks and show significant gains that can be obtained by this processing framework. Code and dataset are publicly available at https://github.com/mahmoudnafifi/CIE_XYZ_NET. [ABSTRACT FROM AUTHOR]

Light nodes in blockchains improve the scalability of the system by storing a small portion of the blockchain ledger. In certain blockchains, light nodes are vulnerable to a data availability (DA) attack where a malicious node makes the light nodes accept an invalid block by hiding the invalid portion of the block from the nodes in the system. Recently, a technique based on LDPC codes called Coded Merkle Tree (CMT) was proposed by Yu et al. that enables light nodes to detect a DA attack by randomly requesting/sampling portions of the block from the malicious node. However, light nodes fail to detect a DA attack with high probability if a malicious node hides a small stopping set of the LDPC code. To mitigate this problem, Yu et al. used random LDPC codes that achieve large minimum stopping set size with high probability. Although effective, these codes are not necessarily optimal for this application, especially at short code lengths, which are relevant for low latency systems, IoT blockchains, etc.. In this paper, we focus on short code lengths and demonstrate that a suitable co-design of specialized LDPC codes and the light node sampling strategy can improve the probability of detection of DA attacks. We consider different adversary models based on their computational capabilities of finding stopping sets in LDPC codes. For a weak adversary model, we devise a new LDPC code construction termed as the entropy-constrained PEG (EC-PEG) algorithm which concentrates stopping sets to a small group of variable nodes. We demonstrate that the EC-PEG algorithm coupled with a greedy sampling strategy improves the probability of detection of DA attacks. For stronger adversary models, we provide a co-design of a sampling strategy called linear-programming-sampling (LP-sampling) and an LDPC code construction called linear-programming-constrained PEG (LC-PEG) algorithm. The new co-design demonstrates a higher probability of detection of DA attacks compared to approaches in earlier literature. [ABSTRACT FROM AUTHOR]

In this paper, we consider Internet of Things (IoT) based mobile edge computing (MEC) system. IoT devices are controlled by access points (APs), which do not have access to any licensed spectrum. This puts limitations on IoT devices’ offloading capability and hence effectively reduces the number of computed bits. Under such scenario, we consider spectrum sharing, where IoT networks get access to licensed spectrum by helping a primary network (owns licensed spectrum) in cooperative relaying. We aim to maximize sum of the primary network’s communication rate and IoT networks’ total number of computed bits by jointly optimizing relay selection, licensed spectrum allocation, local computation, and offloading sequence selection. Moreover, we consider important constraints, e.g., the guaranteed rate for the primary network, IoT devices’ available energy, and available licensed spectrum resource blocks (RBs). We observe that the optimization problem is combinatoric in nature, which becomes more complicated when IoT devices’ scheduling order comes into the formulation. From our analysis, we devise an optimal and computationally efficient algorithm. In the result section, we compare with relevant benchmark systems and show the efficacy of our proposed model. Moreover, we also show how IoT and primary networks benefit from spectrum sharing. [ABSTRACT FROM AUTHOR]