Search

In nonstationary environments, data distributions can change over time. This phenomenon is known as concept drift, and the related models need to adapt if they are to remain accurate. With gradient boosting (GB) ensemble models, selecting which weak learners to keep/prune to maintain model accuracy under concept drift is nontrivial research. Unlike existing models such as AdaBoost, which can directly compare weak learners' performance by their accuracy (a metric between [0, 1]), in GB, weak learners' performance is measured with different scales. To address the performance measurement scaling issue, we propose a novel criterion to evaluate weak learners in GB models, called the loss improvement ratio (LIR). Based on LIR, we develop two pruning strategies: 1) naive pruning (NP), which simply deletes all learners with increasing loss and 2) statistical pruning (SP), which removes learners if their loss increase meets a significance threshold. We also devise a scheme to dynamically switch between NP and SP to achieve the best performance. We implement the scheme as a concept drift learning algorithm, called evolving gradient boost (LIR-eGB). On average, LIR-eGB delivered the best performance against state-of-the-art methods on both stationary and nonstationary data.

Nonoccurring behavior (NOB) studies have attracted the growing attention of scholars as a crucial part of behavioral science. As an effective method to discover both NOB and occurring behaviors (OB), negative sequential pattern (NSP) mining is successfully used in analyzing medical treatment and abnormal behavior patterns. At this time, NSP mining is still an active and challenging research domain. Most of the algorithms are inefficient in practice. Briefly, the key weaknesses of NSP mining are: 1) an inefficient positive sequential pattern (PSP) mining process, 2) a strict constraint of negative containment, and 3) the lack of an effective Negative Sequential Candidate (NSC) generation method. To address these weaknesses, we propose a highly efficient algorithm with improved techniques, named sc-NSP, to mine NSP efficiently. We first propose an improved PrefixSpan algorithm in the PSP mining process, which connects to a bitmap storage structure instead of the original structure. Second, sc-NSP loosens the frequency constraint and exploits the NSC generation method of positive and negative sequential patterns mining (PNSP) (a classic NSP mining method). Furthermore, a novel pruning strategy is designed to reduce the computational complexity of sc-NSP. Finally, sc-NSP obtains the support of NSC by using the most efficient bitwise-based calculation operation. Theoretical analyses show that sc-NSP performs particularly well on data sets with a large number of elements and items in sequence. Comparison and extensive experiments along with case studies on health data show that sc-NSP is 10 times more efficient than other state-of-the-art methods, and the number of NSPs obtained is 5 times greater than other methods.

Existing programming practices for building Android apps mainly follow the one-size-fits-all strategy to include lots of functions and adapt to most types of devices. However, this strategy can result in software bloat and many serious issues, such as slow download speed, and large attack surfaces. Existing solutions cannot effectively debloat an app as they either lack flexibility or require human efforts. This work proposes a novel feature-oriented debloating approach and builds a new tool, named XDebloat, to automate this process in a flexible manner. First, XDebloat supports feature location approaches at a fine granularity. It also makes the feature location results editable and changeable. Second, XDebloat considers several Android-oriented issues (i.e., callback, UI dependencies), which are not covered in the state-of-art approaches. Third, XDebloat support two major debloating strategies: pruning-based and module-based debloating. We evaluate XDebloat with 200 open-source and 1,000 commercial apps. The results show that XDebloat can successfully remove unwanted features from apps or transform them into on-demand modules within 10 minutes. For the pruning-based debloating strategy, on average, XDebloat can remove 32.1% code from an app. For the module-based debloating strategy, XDebloat can help developers build modules automatically.

Advanced reverse analysis tools have significantly improved the ability of attackers to crack software via dynamic analysis techniques, such as symbol execution and taint analysis. These techniques are widely used in malicious fields such as vulnerability exploitation or theft of intellectual property. In this paper, we present an obfuscation strategy called execution trace obfuscation, wherein the program execution trace repeatedly switches between multiple threads. Our technique realizes equivalent code transformation by abstracting the obfuscation problems into pruning, cloning, and coloring problems in graph theory. Based on this, we further propose the cascade encryption of a function that depends on execution trace information with a key derived from the function address calculation process, followed by removing this key from the program. We have implemented a compiler-level system that inputs a source program and automatically generates an obfuscated file. Finally, random test proves the universality of obfuscation algorithm and verify the system's performance. Results shows that our system can effectively interfere advanced reverse analysis tools.

The Coronavirus Disease (COVID-19) was declared a pandemic disease by the World Health Organization (WHO), and it has not ended so far. Since the infection rate of the COVID-19 increases, the computational approach is needed to predict patients infected with COVID-19 in order to speed up the diagnosis time and minimize human error compared to conventional diagnoses. However, the number of negative data that is higher than positive data can result in a data imbalance situation that affects the classification performance, resulting in a bias in the model evaluation results. This study proposes a new oversampling technique, i.e., TRIM-SBR, to generate the minor class data for diagnosing patients infected with COVID-19. It is still challenging to develop the oversampling technique due to the data’s generalization issue. The proposed method is based on pruning by looking for specific minority areas while retaining data generalization, resulting in minority data seeds that serve as benchmarks in creating new synthesized data using bootstrap resampling techniques. Accuracy, Specificity, Sensitivity, F-measure, and AUC are used to evaluate classifier performance in data imbalance cases. The results show that the TRIM-SBR method provides the best performance compared to other oversampling techniques.

Common path pessimism removal (CPPR) is imperative for eliminating redundant pessimism during static timing analysis (STA). However, turning on CPPR can significantly increase the analysis runtime by 10-100× in large designs. Recent years have seen much research on improving the algorithmic efficiencies of CPPR, but most are architecturally constrained by either the speed-accuracy trade-off or design-specific pruning heuristics. In this paper, we introduce a novel CPPR algorithm that is provably good and practically efficient. We have evaluated our algorithm on large industrial designs and demonstrated promising performance over the current state-of-the-art. As an example, our algorithm outperforms the baseline by 36-135× faster when generating the top-10K post-CPPR critical paths on a million-gate design. At the extreme, our algorithm with one core is even 4-16× faster than the baseline with 8 cores. Our algorithm also outperforms the commercial STA engine PrimeTime up to 26.99× faster. By exploiting parallelism within the circuit graph, we can reduce the memory consumption of our algorithm by 30%, with only 3% runtime increase.

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach called EffMoP is presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and time-optimal velocity planning. Compared with existing approaches, the novelty of this work is twofold: 1) a heuristic-guided pruning strategy of motion primitives is newly designed and fully integrated into the search-based global path planner to improve the computational efficiency of graph search, and 2) a novel soft-constrained local path optimization approach is proposed, wherein the sparse-banded system structure of the underlying optimization problem is fully exploited to efficiently solve the problem. We validate the safety, smoothness, flexibility, and efficiency of EffMoP in various complex simulation scenarios and challenging real-world tasks. It is shown that the computational efficiency is improved by 66.21% in the global planning stage and the motion efficiency of the robot is improved by 22.87% compared with the recent quintic Bezier curve-based state space sampling approach.

In recent days, social media, online services, smartphones, and the Internet of Things (IoT) produces large quantities of data every second. The generated data is structured, unstructured, or semi-structured and available in various formats. Therefore, traditional approaches are not sufficient to handle such kind of data effectively. High utility pattern mining is a famous study area of data analytics that incorporates utility measures to consider user-based constraints such as number of units and benefit, in addition to frequency statistics of datasets. It is also essential to make effective decisions, and their demand is increasing in the last decade. Several techniques have been suggested for utility-based frequent pattern extraction. However, these approaches are limited to data size and operate on standalone systems. We have proposed a parallel approach named distributed memory-optimized utility mining (DMOUM) for high utility-based frequent pattern mining to address this issue. A new pruning technique is implemented that greatly reduces processing time and memory use. The proposed approach can handle huge amounts of data, i.e., big data. DMOUM approach is implemented in cluster-node architecture using Spark. The results are validated on various real-time datasets and found proposed approach has better performance than cutting edge approaches in terms of run-time, memory consumption, and scalability measures. The proposed work will provide solutions to real-time issues such as health care, education, e-commerce, and so on.

Co-location pattern mining plays an important role in spatial data mining. With the rapid growth of spatial datasets, the usefulness of co-location patterns is strongly limited by the huge amount of discovered patterns. Although several methods have been proposed to reduce the number of discovered patterns, these statistical algorithms are unable to guarantee that the extracted co-location patterns are user preferred. Therefore, it is crucial to help the decision maker discover his/her preferred co-location patterns via efficient interactive procedures. This article proposes a new interactive approach that enables the user to discover his/her preferred co-location patterns. First, we present a novel and flexible interactive framework to assist the user in discovering his/her preferred co-location patterns. Second, we propose using ontologies to measure the similarity of two co-location patterns. Furthermore, we design a pruning scheme by introducing a pattern filtering model for expressing the user's preference, to reduce the number of the final output. By applying our proposed approach over voluminous sets of co-location patterns, we show that the number of filtered co-location patterns is reduced to several dozen or less and, on average, 80% of the selected co-location patterns are user preferred.

Large deep neural network (DNN) models pose the key challenge to energy efficiency due to the significantly higher energy consumption of off-chip DRAM accesses than arithmetic or SRAM operations. It motivates the intensive research on model compression with two main approaches. Weight pruning leverages the redundancy in the number of weights and can be performed in a non-structured, which has higher flexibility and pruning rate but incurs index accesses due to irregular weights, or structured manner, which preserves the full matrix structure with a lower pruning rate. Weight quantization leverages the redundancy in the number of bits in weights. Compared to pruning, quantization is much more hardware-friendly and has become a "must-do" step for FPGA and ASIC implementations. Thus, any evaluation of the effectiveness of pruning should be on top of quantization. The key open question is, with quantization, what kind of pruning (non-structured versus structured) is most beneficial? This question is fundamental because the answer will determine the design aspects that we should really focus on to avoid the diminishing return of certain optimizations. This article provides a definitive answer to the question for the first time. First, we build ADMM-NN-S by extending and enhancing ADMM-NN, a recently proposed joint weight pruning and quantization framework, with the algorithmic supports for structured pruning, dynamic ADMM regulation, and masked mapping and retraining. Second, we develop a methodology for fair and fundamental comparison of non-structured and structured pruning in terms of both storage and computation efficiency. Our results show that ADMM-NN-S consistently outperforms the prior art: 1) it achieves 348× , 36× , and 8× overall weight pruning on LeNet-5, AlexNet, and ResNet-50, respectively, with (almost) zero accuracy loss and 2) we demonstrate the first fully binarized (for all layers) DNNs can be lossless in accuracy in many cases. These results provide a strong baseline and credibility of our study. Based on the proposed comparison framework, with the same accuracy and quantization, the results show that non-structured pruning is not competitive in terms of both storage and computation efficiency. Thus, we conclude that structured pruning has a greater potential compared to non-structured pruning. We encourage the community to focus on studying the DNN inference acceleration with structured sparsity.

Driven by real-life applications in geo-social networks, we study the problem of computing radius-bounded k-cores (RB-k-cores) that aims to find communities satisfying both social and spatial constraints. In particular, the model k-core (i.e., the subgraph where each vertex has at least k neighbors) is used to ensure the social cohesiveness, and a radius-bounded circle is used to restrict the locations of users in an RB-k-core. We explore several algorithmic paradigms to compute RB-k-cores, including a triple-vertex-based paradigm, a binary-vertex-based paradigm, and a paradigm utilizing the concept of rotating circles. The rotating-circle-based paradigm is further enhanced by several pruning techniques to achieve better efficiency. In addition, to find representative RB-k-cores, we study the diversified radius-bounded k-core search problem, which finds t RB-k-cores to cover the most number of vertices. We first propose a baseline algorithm that identifies the distinctive RB-k-cores after finding all the RB-k-cores. Beyond this, we design algorithms that can efficiently maintain the top-t candidate RB-k-cores and also achieve a guaranteed approximation ratio. Experimental studies on both real and synthetic datasets demonstrate that our proposed techniques can efficiently compute (diversified) RB-k-cores. Moreover, our techniques can be used to compute the minimum-circle-bounded k-core and significantly outperform the existing techniques.

The automatic classification of apple tree organs is of great significance for automatic pruning of apple trees, automatic picking of apple fruits, and estimation of fruit yield. However, there are some problems of dense foliage, partial occlusion and clustering of apple fruits. All of the problems above would contribute to the difficulties of organs classification and yield estimation of the apple trees. In this paper a method based on Color and Shape Multi-features Fusion and Support Vector Machine (SVM) for 3D apple tree organs classification and yield estimation was proposed. The method was designed for dwarf and densely planted apple trees at the early and late maturity stages. 196-dimensional feature vectors composed with Red Green Blue (RGB), Hue Saturation Value (HSV), Curvatures, Fast Point Feature Histogram (FPFH), and Spin Image were extracted firstly. And then the SVM based on linear kernel function was trained, after that the trained SVM was used for apple tree organs classification. Then the position weighted smoothing algorithm was used for classified apple tree organs smoothing. Then the agglomerative hierarchical clustering algorithm was used to recognize single apple fruit for yield estimation. On the same training and test set the experimental results showed that the SVM based on linear kernel function outperformed the KNN algorithm and Ensemble algorithm. The Recall, Precision and F1 score of the proposed method for yield estimation were 93.75%, 96.15% and 94.93% respectively. In summary, to solve the problems of apple tree organs classification and yield estimation in natural apple orchard, a novelty method based on multi-features fusion and SVM was proposed and achieve good performance. Moreover, the proposed method could provide technical support for automatic apple picking, automatic pruning of fruit trees, and automatic information acquisition and management in orchards.

Deep learning has been proved to be one of the most effective method in feature encoding for different intelligent applications such as video based human action recognition. However, its non-convex optimization mechanism leads large memory consumption, which hinders its deployment on the smart embedded systems with limited computation resources. To overcome this challenge, we propose a novel deep model compression technique for smart embedded systems, which realizes both the memory size reduction and inference complexity decrease within a small drop of accuracy. First, we propose an improved naive Bayes inference based channel parameter pruning to obtain a sparse model with higher accuracy. Then, to improve the inference efficiency, the improved Tucker decomposition method is proposed, where an improved genetic algorithm is used to optimize the Tucker ranks. Finally, to elevate effectiveness of our proposed method, extensive experiments are conducted. The experimental results show that our method can achieve the state-of-the-art performance compared with existing methods in terms of accuracy, parameter compression and floating point operations reduction.

This paper investigates the feasibility of applying the broad learning system (BLS) to realize a novel TakagiSugenoKang (TSK) neuro-fuzzy model, namely a broad learning-based dynamic fuzzy inference system (BL-DFIS). It not only improves the accuracy and interpretability of neuro-fuzzy models but also solves the challenging problem that models are incapable of determining the optimal architecture autonomously. BL-DFIS firstly accomplish a TSK fuzzy system under the framework of BLS, in which extreme learning machine auto-encoder (ELM-AE) is employed to obtain feature representation in a fast and analytical way, and an interpretable linguistic fuzzy rule (ILFR) is integrated into the enhancement node to ensure the high interpretability of the system. Meanwhile, the extended-enhancement unit (EEU) is designed to achieve the first-order TSK fuzzy system. On the other hand, a dynamic incremental learning algorithm with internal pruning and updating mechanism is developed for the learning of BL-DFIS, which enables the system to automatically assemble the optimal structure to obtain a compact rule base and an excellent classification performance. Experiments on benchmark datasets demonstrate that the proposed BL-DFIS can achieve a better classification performance than some state-of-the-art nonfuzzy and neuro-fuzzy methods, simultaneously using the most parsimonious model structure.

Feature selection in data with class-imbalance or missing values has received much attention from researchers due to their universality in real-world applications. However, for data with both the two characteristics above, there is still a lack of corresponding feature selection algorithm. Due to the complex coupling relationship between missing data and class-imbalance, the need for better feature selection method becomes essential. To tackle high-dimensional imbalanced data with missing values, this paper studies a new evolutionary feature selection method. Firstly, an improved F-measure based on filling risk (RF-measure) is defined to evaluate the influence of missing data on the performance of feature selection in the case of class-imbalance. Following that, taking the RF-measure as an objective function, a particle swarm based feature selection method with fuzzy clustering (PSOFS-FC) is proposed. Two new problem-specific operators or strategies, i.e., the swarm initialization strategy guided by fuzzy clustering and the local pruning operator based on feature importance, are developed to improve the performance of PSOFS-FC. Compared with state-of-the-art feature selection algorithms on several public datasets, experimental results show that PSOFS-FC can achieve excellent classification performance with relatively less running time, indicating its superiority on tackling high-dimensional imbalanced data with missing values.

As an emerging threat to Deep Neural Networks (DNNs), backdoor attacks have received increasing attentions due to the challenges posed by the lack of transparency inherent in DNNs. In this paper, we develop an efficient algorithm from the interpretability of DNNs to defend against backdoor attacks to DNN models. To extract critical neurons, we deploy sets of control gates following neurons in layers, and the function of a DNN model can be interpreted as semantic sensitivities of neurons to input samples. A backdoor identification approach, derived from the activation frequency distribution on critical neurons, is proposed to reveal anomalies of particular neurons produced by backdoor attacks. Subsequently, a feasible and fine-grained pruning strategy is introduced to eliminate backdoors hidden in DNN models, without the need of retraining. Extensive experiments demonstrate that the proposed algorithm can identify and eliminate malicious backdoors efficiently in both single-target and multi-target scenarios with the performance of a DNN model retained to a large extent.

With the rapid advances in computing and information technologies, traditional access control models have become inadequate in terms of capturing fine-grained, and expressive security requirements of newly emerging applications. An attribute-based access control (ABAC) model provides a more flexible approach for addressing the authorization needs of complex and dynamic systems. While organizations are interested in employing newer authorization models, migrating to such models pose as a significant challenge. Many large-scale businesses need to grant authorization to their user populations that are potentially distributed across disparate and heterogeneous computing environments. Each of these computing environments may have its own access control model. The manual development of a single policy framework for an entire organization is tedious, costly, and error-prone. In this paper, we present a methodology for automatically learning ABAC policy rules from access logs of a system to simplify the policy development process. The proposed approach employs an unsupervised learning-based algorithm for detecting patterns in access logs and extracting ABAC authorization rules from these patterns. In addition, we present two policy improvement algorithms, including rule pruning and policy refinement algorithms to generate a higher quality mined policy. Finally, we implement a prototype of the proposed approach to demonstrate its feasibility.

A vast amount of activation values of DNNs are zeros due to ReLU (Rectified Linear Unit), which is one of the most common activation functions used in modern neural networks. Since ReLU outputs zero for all negative inputs, the inputs to ReLU do not need to be determined exactly as long as they are negative. However, many accelerators usually do not consider such aspects of DNNs, losing a huge amount of opportunities for speedups and energy savings. To exploit such opportunities, we propose early negative detection (END), a computation pruning technique that detects the negative results at an early stage. The key to the early negative detection is the adoption of inverted twos complement representation for filter parameters. This ensures that as soon as the intermediate results become negative, the final results are guaranteed to be negative. Upon detection, the remaining computation can be skipped and the following ReLU output can be simply set to zero. We also propose a DNN accelerator architecture (ComPreEND) that takes advantage of such skipping. ComPreEND with END significantly improves both the energy efficiency and the performance according to the evaluation.

Many research areas depend on group anomaly detection. The use of group anomaly detection can maintain and provide security and privacy to the data involved. This research attempts to solve the deficiency of the existing literature in outlier detection thus a novel hybrid framework to identify group anomaly detection from sequence data is proposed in this paper. It proposes two approaches for efficiently solving this problem: i) Hybrid Data Mining-based algorithm, consists of three main phases: first, the clustering algorithm is applied to derive the micro-clusters. Second, the kNN algorithm is applied to each micro-cluster to calculate the candidates of the group's outliers. Third, a pattern mining framework gets applied to the candidates of the group's outliers as a pruning strategy, to generate the groups of outliers, and ii) a GPU-based approach is presented, which benefits from the massively GPU computing to boost the runtime of the hybrid data mining-based algorithm. Extensive experiments were conducted to show the advantages of different sequence databases of our proposed model. Results clearly show the efficiency of a GPU direction when directly compared to a sequential approach by reaching a speedup of 451. In addition, both approaches outperform the baseline methods for group detection.

Social Internet of Things (SIoT) incorporates social relationship into Internet of Things, and compositive relationship between persons, devices and persons to devices is utilized for providing better services. This paper proposes a novel type of search, namely top-k social spatial keyword search (SSKS) in SIoT networks to discover relevant users or data objects according to social, spatial and textual preferences. Existing works mainly focus on two of these preferences at the same time, and efficiently processing top-k SSKS remains challenging. To this end, we propose two algorithms to evaluate top-k SSKS in SIoT networks. The first algorithm is a forward search based algorithm, which spreads the search from the vertex of the querying user. An effective pruning strategy is established by recognizing an early termination condition according to the threefold preference. The forward search based algorithm is efficient when textual objects are dense. The second algorithm is based on index searching. We present an index namely 2HL-GIL to support spatial and textual pruning while providing fast computation of social distances in the SIoT. Then an index-based search algorithm is proposed for top-k SSKS, and it is efficient especially when textual objects are sparse. Our proposed algorithms are evaluated over two real-life social networks attached with synthetic locations and textual data. Evaluation results illustrate the effectiveness and efficiency of our proposed forward search based algorithm and index-based search algorithm.

With growing numbers of cores, the memory subsystem of manycore architectures increases in complexity. Many modern multicores are designed in a hierarchical fashion, with multiple clusters of processing elements. However, most algorithms for design-space exploration of resource allocation in multicores do not consider these complex topologies, which results in poor scaling, or worse, non-functioning algorithms. In this paper we present mpsym, a C++ library designed to alleviate this problem in an algorithm-agnostic fashion. Using methods from computational group theory, we present domain-specific algorithms to improve design-space exploration in hierarchical architecture topologies. We evaluate mpsym on multiple designspace exploration algorithms from literature. Without modifying the algorithm, our methods improve the execution time by a factor up to 8:6× on the E3S benchmark suite for complex, clustered architecture topologies. Similarly, by pruning the design space, our methods consistently improve the result of the exploration. In particular, the results from a simulated annealing heuristic on the Kalray MPPA3 Coolidge topology are over 30× better on average, while requiring less time to explore.

Real-world networks, such as phone-call networks and social networks, are often not static but temporal. Mining cohesive subgraphs from static graphs is a fundamental task in network analysis and has been widely investigated in the past decades. However, the concepts of cohesive subgraphs shift from static to temporal graphs raise many important problems. For instance, how to detect stable cohesive subgraphs on temporal networks such that the nodes in the subgraph are densely and stably connected over time. To address this problem, we resort to the conventional quasi-clique and propose a new model, called maximal ρ-stable (δ, ɣ)-quasi-clique, to capture both the cohesiveness and the stability of a subgraph. We show that the problem of enumerating all maximal ρ-stable (δ, ɣ)-quasi-cliques is NP-hard. To efficiently tackle our problem, we first devise a novel temporal graph reduction algorithm to significantly reduce the temporal graph without losing any maximal ρ-stable (δ, ɣ)-quasi-clique. Then, on the reduced temporal graph, we propose an effective branch and bound enumeration algorithm, named BB&SCM, with four carefully designed pruning techniques to accomplish the enumeration process. Finally, we conduct extensive experiments on seven real-world temporal graphs, and the results demonstrate that the temporal graph reduction algorithm can safely reduce 98% nodes of the temporal graph (with millions of nodes and edges) and BB&SCM is at least two orders of magnitude faster than the baseline algorithms. Moreover, we also evaluate the effectiveness of our model against other baseline models.

Deep neural networks designed for point clouds, also called point cloud neural networks (PCNNs), are attracting increasing attention in recent years. In this work, we propose the first model compression framework called JointPruning (JP) that is specifically designed for compressing PCNNs. Observing that the redundancies in PCNNs are largely affected by certain parameters like the number of points, we first propose a new search space specifically designed for PCNNs. By searching the optimal pruning policy in our newly proposed search space, our JP framework can prune the PCNNs at different levels and simultaneously reduce the redundancies along multiple dimensions. As the newly proposed search space consists of multiple levels and the policy value at each level is continuous in our JP framework, it is hard to directly search for the best pruning policy in such a large search space. To this end, we further propose two strategies called search space refinement and validation set extension to progressively refine the granularity of our searching process in a coarse-to-fine and easy-to-hard fashion, which can help us gradually find better pruning policies. Comprehensive experiments on three benchmark datasets demonstrate the effectiveness of our JP framework for compressing PCNNs.

We consider a class of non-convex complex modulus constrained problems (CMCPs), which has many important applications in signal processing for wireless communications, including multicast beamforming and multi-input multi-output detection. Due to the non-convex constraints, traditional optimization-based algorithms either obtain sub-optimal solutions or take exponential complexity to reach the optimum. In this paper, we propose a learning based branch-and-bound (LBB) algorithm for solving the considered CMCPs. LBB regards the search procedure of the global optimal argument-cut based branch-and-bound (AC-BB) algorithm as a sequential decision problem in a binary tree and learns the optimal pruning policy via supervised learning. We first propose to apply ensemble learning to train multiple classifiers and then combine them to achieve better performance. To tackle the imbalanced issue, we propose an undersampling-supervised learning method to sample several balanced subsets and train a classifier on each of them. We also show that the computational complexity of LBB is determined by the depth of the binary tree and give its expression in the worst case. Finally, we validate the proposed LBB with two classic problems in wireless communications, i.e., multicast beamforming and MIMO detection. Numerical results show that LBB runs significantly faster than AC-BB while achieving nearly the same optimal performance.

The greater interest of research in the field of bioinformatics and the ample amount of available data across the different domains paved the way for the generation of the dataset with high dimensionality. The number of features in the dataset with high dimensionality are very high and number of rows are less. The significance of the Frequent Colossal Closed Itemsets (FCCI) is high for diverse applications and also for the field of bioinformatics. FCCI are very prominent in the process of the decision making. Amount of information extraction from the dataset with high dimensionality is huge and this extraction is a non-trivial task. The pruning of all the inadmissible features and rows is not performed by the state-of-the-art algorithms. The proposed work articulates the pruning of all the inadmissible features and rows, an efficient pruning strategy to snip the row enumeration mining search space and closure method for checking the closeness of the rowset. An efficient row enumeration algorithm enclosing the rowset closure checking method and pruning strategy is designed to efficiently mine the complete set of FCCI. The experimental results demonstrate the effectiveness of pruning all the inadmissible features and rows.

Deep Neural Networks have achieved remarkable advancement in various intelligence tasks. However, the massive computation and storage consumption limit applications on resource-constrained devices. While channel pruning has been widely applied to compress models, it is challenging to reach very deep compressions for such a coarse-grained pruning structure without significant performance degradation. In this article, we propose an acceleration-aware fine-grained channel pruning (AFCP) framework for accelerating neural networks, which optimizes trainable gate parameters by estimating residual errors between pruned and original channels with hardware characteristics. Our fine-grained concept consists of both algorithm and structure levels. Different from existing methods that leverage a pre-defined pruning criterion, AFCP explicitly considers both zero-out and similar criteria for each channel and adaptively selects the suitable one via residual gate parameters. For structure level, AFCP adopts a fine-grained channel pruning strategy for residual neural networks and a decomposition-based structure, which further extends the pruning optimization space. Moreover, instead of using theoretical computation costs such as FLOPs, we propose the hardware predictor that bridges the gap between realistic acceleration and pruning procedure to guide the learning of pruning, which improves the efficiency of model pruning when deployed on accelerators. Extensive evaluation results demonstrate that AFCP outperforms state-of-the-art methods, and achieves a favorable balance between model performance and computation cost.

Neural Network Coding and Representation (NNR) is the first international standard for efficient compression of neural networks (NNs). The standard is designed as a toolbox of compression methods, which can be used to create coding pipelines. It can be either used as an independent coding framework (with its own bitstream format) or together with external neural network formats and frameworks. For providing the highest degree of flexibility, the network compression methods operate per parameter tensor in order to always ensure proper decoding, even if no structure information is provided. The NNR standard contains compression-efficient quantization and deep context-adaptive binary arithmetic coding (DeepCABAC) as core encoding and decoding technologies, as well as neural network parameter pre-processing methods like sparsification, pruning, low-rank decomposition, unification, local scaling and batch norm folding. NNR achieves a compression efficiency of more than 97% for transparent coding cases, i.e. without degrading classification quality, such as top-1 or top-5 accuracies. This paper provides an overview of the technical features and characteristics of NNR.

As a promising solution to boost the performance of distance-related algorithms (e.g., K-means and KNN), FPGAbased acceleration attracts lots of attention, but also comes with numerous challenges. In this work, we propose, STPAcc, an optimization framework based on structural triangle-inequality (TI) based pruning (STP) for accelerating distance-related algorithms on CPU-FPGA platforms. STPAcc provides a domainspecific language to unify distance-related algorithms effectively, a structural TI-based pruning strategy to remove unnecessary distance computations, a coarse-grained workload partitioning and mapping strategy to fully exploit the potentials of the CPUFPGA platform, and fine-grained hardware optimizations to further improve performance on the FPGA. Intensive experiments show that STPAcc designs achieve 31:42× speedup and 99:63× better energy efficiency on average over standard CPU-based implementations.

Density Peak Clustering (DPC), a popular density-based clustering approach, has received considerable attention from the research community primarily due to its simplicity and fewer-parameter requirement. However, the resultant clusters obtained using DPC are influenced by the sensitive parameter $d_c$, which depends on data distribution and requirements of different users. Besides, the original DPC algorithm requires visiting a large number of objects, making it slow. To this end, this paper investigates index-based solutions for DPC. Specifically, we propose two list-based index methods viz. (i) a simple List Index, and (ii) an advanced Cumulative Histogram Index. Efficient query algorithms are proposed for these indices which significantly avoids irrelevant comparisons at the cost of space. For memory-constrained systems, we further introduce an approximate solution to the above indices which allows substantial reduction in the space cost, provided that slight inaccuracies are admissible. Furthermore, owing to considerably lower memory requirements of existing tree-based index structures, we also present effective pruning techniques and efficient query algorithms to support DPC using the popular Quadtree Index and R-tree Index. Finally, we practically evaluate all the above indices and present the findings and results, obtained from a set of extensive experiments on six synthetic and real datasets. The experimental insights obtained can help to guide in selecting a befitting index.

A key characteristic of biological systems is the ability to update the memory by learning new knowledge and removing out-of-date knowledge so that intelligent decision can be made based on the relevant knowledge acquired in the memory. Inspired by this fundamental biological principle, this article proposes a multi-output selective ensemble regression (SER) for online identification of multi-output nonlinear time-varying industrial processes. Specifically, an adaptive local learning approach is developed to automatically identify and encode a newly emerging process state by fitting a local multi-output linear model based on the multi-output hypothesis testing. This growth strategy ensures a highly diverse and independent local model set. The online modeling is constructed as a multi-output SER predictor by optimizing the combining weights of the selected local multi-output models based on a probability metric. An effective pruning strategy is also developed to remove the unwanted out-of-date local multi-output linear models in order to achieve low online computational complexity without scarifying the prediction accuracy. A simulated two-output process and two real-world identification problems are used to demonstrate the effectiveness of the proposed multi-output SER over a range of benchmark schemes for real-time identification of multi-output nonlinear and nonstationary processes, in terms of both online identification accuracy and computational complexity.

This paper presents a multi-objective search-based approach to generate balanced maps for real-time strategy games. First, an angle-based pruning-power indicator guided evolutionary algorithm is proposed to solve the many-objective optimization problems. This algorithm divides the hyper-spherical coordinate space into a set of local partitions, where the pruning power indicator is used to access the dominance ability of the solution over its located partition, and the radius-penalized angle calculation indicator aims to guide the evolution of population along specific partition direction. In this way, both the convergence and diversity of solutions are considered, and the selection pressure toward Pareto fronts is strengthened. We show the effectiveness of the proposed strategies on a set of many-objective benchmark functions. Then, the proposed algorithm is used to generate game maps –a real-world problem derived from the procedural content generation, in which there are four objective functions to guide the map generation of MegaGlest based on the principle of fairness, playability, strategy, and interestingness of games. Experimental results on four map generation instances verify the competitiveness and effectiveness of the proposed algorithm.

Adaptive computing (AC) is a technique to dynamically select the layers to pass in a prespecified deep neural network (DNN) according to the input samples. In previous literature, AC was deemed as a standalone complexity-reduction skill. This brief studies AC through a different lens: we investigate how this strategy interacts with mainstream compression techniques in a unified complexity-reduction framework and whether its ``input sample related'' feature helps with the improvement of model robustness. Following this direction, we first propose a defensive accelerating branch (DAB) based on the AC strategy that can reduce the average computational cost and inference time of DNNs with higher accuracy compared with its counterparts. Then, the proposed DAB is jointly applied with the mainstream parameterwise compression skills, pruning and quantization, to build a unified complexity-reduction framework. Extensive experiments are conducted, and the results reveal quasi-orthogonality between the input-related and parameterwise complexity-reduction skills, which means that the proposed AC can be integrated into an off-the-shelf compressed model without hurting its accuracy. Besides, the robustness of the proposed compression framework is explored, and the experimental results demonstrate that DAB can be used as both the detector and the defensive tool when the model is under adversarial attacks. All these findings shed light on the great potential of DAB in building a unified complexity-reduction framework with both a high compression ratio and great adversarial robustness.

Classifier ensembles are characterized by the high quality of classification, thanks to their generalizing ability. Most existing ensemble algorithms use all learning samples to learn the base classifiers that may negatively impact the ensemble’s diversity. Also, the existing ensemble pruning algorithms often return suboptimal solutions that are biased by the selection criteria. In this work, we present a proposal to alleviate these drawbacks. We employ an instance selection method to query a reduced training set that reduces both the space complexity of the formed ensemble members and the time complexity to classify an instance. Additionally, we propose a guided search-based pruning schema that perfectly explores large-size ensembles and brings on a near-optimal subensemble with less computational requirements in reduced memory space and improved prediction time. We show experimentally how the proposed method could be an alternative to large-size ensembles. We demonstrate how to form less-complex, small-size, and high-accurate ensembles through our proposal. Experiments on 25 datasets show that the proposed method can produce effective ensembles better than Random Forest and baseline classifier pruning methods. Moreover, our proposition is comparable with the Extreme Gradient Boosting Algorithm in terms of accuracy.

Manual titration of positive airway pressure (PAP) is a gold standard to provide an optimal pressure for the treatment of obstructive sleep apnea-hypopnea syndrome (OSAS). Since manual titration studies were costly and time-consuming, many statistical models for predicting effective PAPs were reported. However, the prediction accuracies of the models associated with nocturnal parameters still remain low. This study proposes a fuzzy neural prediction network (FNPN) with input candidate variables, selected among easily available measurements (e.g., body mass index (BMI), waist circumstance (WC), and body composition) and OSAS related questionnaires, to rapidly predict an optimal PAP. The FNPN comprises fuzzy rules and is characterized with the ability of automatic rule growing and pruning from training data. A total of 147 participants from April 2018 to April 2019 were enrolled in Taichung Veterans General Hospital, Taiwan. After two selection processes for feature extraction, WC and BMI were the significant variables for entering the FNPN to predict optimal PAP. Experimental results showed that the average successful prediction rate of the proposed method was 71.8%. This study also found that Epworth sleepiness scales (ESS) and body composition, such as visceral fat area and percent body fat, were excluded in the final prediction model. Compared with existing models, the proposed prediction approach provided a rapid prediction of optimal PAP with higher accuracy.

In this paper, we introduce an algorithm for computing all the shortest reducts in a decision system. The proposed algorithm is based on determining the size of the shortest reducts using a small super-reduct and some new pruning methods. Once the size of the shortest reduct is determined, all other reducts of the same size are found applying the new pruning methods. The results of our experiments using several synthetic and real-world decision systems show that the proposed algorithm is, in most cases, faster than the state of the art algorithms for computing all the shortest reducts reported in the literature.

In addition to the word-level weight pruning, which excludes the 0-value weights from the neural network inference computation, it is recently demonstrated that the bit-level weight pruning, which excludes the 0-bits in the weight value representation regardless of whether the weight values are zero or not, is very effective to further accelerate the neural network computation without accuracy loss. This work overcomes the inherent limitation of the bit-level weight pruning, that is, the maximal computation speedup is bounded by the total number of non-zero bits of the weights and the bound is invariably considered “uncontrollable" (i.e., constant) for the neural network to be pruned. Precisely, this work, based on the signed-digit encoding, (1) proposes a transformation technique which converts the two’s complement representation of every weight into a set of signed-digit representations of the minimal number of essential (i.e., non-zero) bits, (2) formulates the problem of selecting signed-digit representations of weights that maximize the parallelism of bit-level multiplication on the weights into a multi-objective shortest path problem to achieve a maximal digit-index by digit-index (i.e. column-wise) compression for the weights and solves it efficiently using an approximation algorithm, and (3) proposes a supporting novel acceleration architecture ( DWP) with no additional inclusion of non-trivial hardware. In addition, we (4) propose a variant of DWP to support bit-level parallel multiplication with the capability of predicting a tight worst-case latency of the parallel processing. Through experiments on several representative models using the ImageNet dataset, it is shown that our proposed approach is able to reduce the number of essential bits by 69% on AlexNet, 74% on VGG-16, and 68% on ResNet-152, by which our accelerator is able to reduce the inference computation time by up to 3.57× over the conventional bit-level weight pruning.

The proliferation of vehicle-to-vehicle (V2V) communication techniques has resulted in collaborative vehicle localization (CVL) approaches that localize a target vehicle by leveraging the state information of nearby vehicles. However, CVL approaches typically require a large search space to locate the real position of a target vehicle and assume small measurement errors of nearby vehicle information, which limit the efficiency and robustness of the existing methods. In this paper, we propose an efficient and error-tolerant CVL approach (referred to as ET-CVL) that increases localization efficiency and accuracy even in the case of large measurement errors of nearby vehicle information. Unlike the existing CVL approaches, our approach prunes the search space for the position of the target vehicle through a pruning-based strategy that considers the relative positions of nearby vehicles. To determine the position of the target vehicle from the search space, we propose a displacement-based selection method to reduce the influence of the measurement errors of nearby vehicle information. The localization accuracy and efficiency of the proposed approach are then evaluated using simulated GPS trajectories in a large road network in New York City. The experimental results show that the proposed approach achieves higher localization efficiency and greater accuracy even with large measurement errors compared with state-of-the-art CVL approaches.

A novel behavioral modeling technique for digital predistortion of radio frequency power amplifiers (PAs) is proposed. The proposed basis-propagating selection (BAPS) model has an advantage on complexity since its basis functions are constructed as delay or a combination of existing basis functions. The BAPS model effectively optimizes the running time complexity while maintaining the generality of a full Volterra series. A greedy search procedure is applied to select the optimal basis functions. Compared with truncated Volterra models and pruned models obtained by popular pruning techniques, the superior performance of the BAPS model, in terms of accuracy and identification stability, has been fairly justified. The linearization measurements on different types of PAs illustrate that the BAPS model can robustly outperform other models from literature while achieving a low running complexity.

In this study, we propose a simple and effective quantization-aware neural network pruning method to prune and quantize deep neural networks and then run them on lightweight and cost-effective embedded industrial devices. The proposed pruning method provides a suitable pruning strategy for a given bit precision before performing the subsequent quantization. In particular, the dropout rate is set to be bounded below by the maximum number of quantization levels, which enables the regulation of the tradeoff between sparsity and performance loss. The Kullback-Leibler divergence term of the evidence lower bound is also weighted for adjusting the performance loss. From learned dropout rates, efficient bit allocation is provided to maintain randomness even through the quantization step. In supervised and reinforcement learning applications, the proposed quantization-aware pruning method was proven to be successful in obtaining sparse and lightweight neural networks while maintaining their performance even after quantization. In particular, a real inverted pendulum system was used to determine whether the proposed method works well for actual physical control systems.

The binarized neural network (BNN) inference accelerators show great promise in cost-and power-restricted domains. However, the performances of these accelerators are still severely limited by the significant redundancies in BNNs inference. In this brief, we introduce channel-aware sparse accelerator (CAA) to alleviate the performance degradations induced by the redundancies in BNNs while maintaining original accuracies. First, motivated by the observation that the convolution processes of our rebuilt rectangle kernels contain many redundant operations which can be skipped by exploiting the BNN-specific property, we convert the entire original XNOR-popcount convolutions of each neuron into channel-aware-popcount (CAP) operations for all binarized convolutional and fully-connected layers in CAA by employing rectangle kernel simplification strategy and eliminate the unnecessary operations. Meanwhile, these CAP operations can be implemented to directly gain the final output without any extra steps. Furthermore, inspired by our new observations on two specific kinds of properties of the CAP operations, we adopt group pruning approach to save the remaining redundant CAP operations. Experimental results show that our design evaluated on an embedded FPGA achieves 4.2-6.6× inference-speedup, 3.6-5.5× energy-efficiency enhancement, and 1.35× resource-efficiency improvement compared with state-of-the-art works.

Neural networks (NNs) are attractive for nonlinear impairment compensation applications in communication systems, such as optical fiber nonlinearity, nonlinearity of driving amplifiers, and nonlinearity of semiconductor optical amplifiers. Without prior knowledge of the transmission link or the hardware characteristics, optimal parameters are completely constructed from a data-driven approach by exploring training datasets, once the NN structure is given. On the other hand, due to computational power and energy consumption, especially in high-speed communication systems, the computational complexity of the optimized NN needs to be confined to the hardware, such as FPGA or ASIC without sacrificing its performance improvement. In this paper, two approaches are presented to accommodate the NN-based algorithms for high-speed communication systems. The first approach is to reduce computational complexity of the NN-based nonlinearity compensation algorithms on the basis of weight pruning (WP). WP can significantly reduce the computational complexity, especially because the nonlinear compensation task studied here results in a sparse NN. The authors have studied an enhanced approach of WP by imposing an additional restriction on the selection of non-zero weights on each hidden layer. The second approach is to implement NNs onto a silicon-photonic integrated platform, enabling power efficiency to be further improved without sacrificing the high-speed operation.

To address the difficulty of optimizing turbidity process with large inertia and long time delay, a twice-optimal control and construction pruning (TOCCP) strategy is proposed, which can be used to improve the fast stability and strong robustness of industrial process. The new approach owns the advantages of twice optimal control (TOC) and construction pruning (CP) method. The TOC constructs an infinite-dimensional state observer of the system, which can eliminate the influence of time delay on the system. The CP is adopted to improve the mid-band damping of the open-loop frequency characteristics to improve the robustness of the system. Moreover, an exhaustive algorithm is designed to find the optimal time scale, the most important parameter for adjusting the system response speed. In addition, the influences of different parameter changes on the robustness of the system are analyzed, and a practical parameter tuning formula is provided. Simulation-based comparisons of TOCCP and the other two algorithms were made. The results show that the system optimized by TOCCP has strong robustness, especially in disturbances rejection and set-point tracking. Moreover, TOCCP is practical and easily implemented to set parameters, providing guidelines for industrial application of large inertia and time-delay systems.

Although massive pruning methods are emerging for achieving structural sparsity in convolutional neural networks (CNN), most of them target structures such as ResNet. Meanwhile, previous works take more interest in pruning filters inside a residual block and keep the shortcut connection intact, leading to an imbalanced network structure. In this paper, we focus on the penalty-based method to prune already compact networks. In contrast to the broadly used L1 constraint, which shrinks the parameters uniformly, we propose a novel penalty term that is similar in shape to an upside-down Laplace distribution. The penalty allows us to impose more pressure on potential weak channels but protects others during training to avoid damaging crucial channels, especially for compact architectures. We also design a candidate selection strategy to cooperate with the penalty-based training procedure. Besides, we address the residual block pruning problem by a scaling factor elimination skill, which is often ignored in other research. Our method reduces 50% parameters of MobileNet v1/v2 with a tolerable accuracy degradation. We further conduct pruning on MobileNetv1-SSDLite to compress parameters by 60%, manifesting the ability to generalize to different visual tasks. The experiment results demonstrate that our method outperforms pruning frameworks based on channel importance without a complicated tuning for hyper-parameters like search-based methods.

Model pruning aims to reduce the parameter amount of deep neural networks while retaining the performance. Existing strategies often treat all layers equally and all layers simply share the same pruning rate. However, it is observed from our experiments that the redundancy degree differs from layer to layer. Based on this observation, this work proposes a pruning strategy depending on the layer-wise redundancy degree. Firstly, we define the redundancy degree for each layer by the norm and similarity redundancy of filters. Then a novel layer-wise strategy, Redundancy-dependent Filter Pruning (RedFiP), is proposed which prunes different proportion of filters at different layers according to the defined redundancy degree. Since the redundancy analysis and experimental results of RedFiP show that deeper layers need fewer filters, a phase-wise strategy, Phased Filter Pruning (PFP), is proposed that divides the layers into three phases and layers in each phase share the same pruning rate. The phase-wise PFP allows the layer-wise RedFiP to be easily implemented in existing structures of deep neural networks. Experimental results show that when total parameters are pruned by 40%, RedFiP outperforms the state-of-the-art strategy FPGM-Mixed by 1.83% on CIFAR-100, and even slightly outperforms the non-pruned model by 0.11% on CIFAR-10. On ImageNet-1k, RedFiP (30%) and PFP (30%) outperform FPGM-Mixed (30%) by 1.3% and 0.8% with ResNet-18.

Budget allocation problems in portfolio management are inherently multi-objective as they entail different types of assets of which performance metrics are not directly comparable. Existing asset management methods that either consolidate multiple goals to form a single objective (a priori) or populate a Pareto optimal set (a posteriori) may not be sufficient because a decision maker (DM) may not possess comprehensive knowledge of the problem domain. Moreover, current techniques often present a Pareto optimal set with too many options, making it counter-productive. In order to provide the DM with a diverse yet compact solution set, this paper proposes a three-step approach. In the first step, we employ different approximation functions to capture investment-performance relationships at the asset-type level. These simplified relationships are then used as inputs for the multi-objective optimisation model in the second step. In the final step, Pareto optimal solutions generated by a selected evolutionary algorithm are pruned by a clustering method. To measure the spread of representative solutions over the Pareto front, we present two novel indicators based on average Euclidean distance and cosine similarity between original Pareto solutions and representative solutions. Through numerical examples, we demonstrate that this approach can provide a set of representative solutions that maintain high integrity of the original Pareto front. We also put forward suggestions on choosing appropriate approximation functions, pruning methods, and indicators.

Cyber-physical systems (CPS) integrate discrete computations with continuous physical processes and can be highly nonlinear and nondeterministic. Unlike the verification of CPS, which is difficult to handle, the falsification of CPS fulfills certain requirements from testing by seeking witness behavior of these systems and is easier to conduct. However, existing falsification techniques may fail to support the general complex CPS in practice because they usually focus on certain restricted classes of systems. In this paper, we present a path-oriented, derivative-free approach to falsify safety properties in nonlinear and nondeterministic CPS. In our approach, we model the behavior of CPS by hybrid automata. Then, we enumerate candidate paths of hybrid automata, transform the feasibility of candidate paths into optimization problems and solve these optimization problems by our newly proposed classification-model-based, derivative-free optimization algorithm. We also provide two novel pruning techniques to further improve the efficiency and efficacy of our approach: a nested optimization structure with better model refinements for continuous search space pruning and a hardly feasible path prefixes guided backtracking for discrete search space pruning. We implement our approach into a tool called PDF. Our experiments showed that PDF supported the safety falsification of CPS in all of our benchmarks, and it achieved success rates no lower than 95% in only seconds on 22/28 of the benchmarks.

In recent years, RGBT tracking has become a hot topic in the field of visual tracking, and made great progress. In this paper, we propose a novel Trident Fusion Network (TFNet) to achieve effective fusion of different modalities for robust RGBT tracking. In specific, to deploy the complementarity of features of all convolutional layers, we propose a recursive strategy to densely aggregate these features that yield robust representations of target objects in two modalities. Moreover, we design a trident architecture to integrate the fused features and both modality-specific features for robust target representations. There are three main advantages. First, retaining the classification layer of each modality is beneficial to enhance feature learning of single modality, and compared with aggregate branches, single-modality branches pay more attention to the mining of modal specific information. Second, when some modality is noisy or invalid, the modality-specific branches would capture more discriminative features for RGBT tracking. Finally, the integration of aggregation branches and single-modality branches is beneficial to the complementary learning of different modalities. In addition, we also introduce a feature pruning module in each branch to prune the redundant features and avoid network overfitting. Experimental results on four RGBT tracking benchmark datasets suggest that our tracker achieves superior performance against the state-of-the-art RGBT tracking methods.

As the smallest structural unit of feature mapping, the convolution kernel in a deep convolution neural networks (DCNN) convolutional layer is responsible for the input channel features to output channel features. A specific convolution kernel belongs to a specific group from the perspective of the input channel, and it belongs to a specific filter from the perspective of the output channel. If the input and output channels are simultaneously considered in the pruning process, the performance of the pruning model can be further improved. This paper proposes progressive kernel pruning with salient mapping of input-output channels, introduces the concept of input-output channel saliency and defines single-port salient mapping channels and dual-port salient mapping channels. This study demonstrates that single-port salient mapping channels can ensure that each input channel signal has a relatively strong convolution kernel mapped to the output channel, and vice versa. The dual-port salient mapping channel is a channel with high feature mapping abilities from both the input and output directions. Additionally, the average mapping ability measure index is defined, which is used to control the kernel pruning process of the single-port salient mapping channel to switch to the kernel pruning process of the dual-port salient mapping channel. The experimental results and analysis show that the method proposed in this paper can be used to effectively prune a heavyweight model and a lightweight model and can obtain a better accuracy under the conditions of higher compression ratio and acceleration ratios. For example, when VGG-16 is pruned on CIFAR-10, the compression ratio and acceleration ration are 91.00 × and 15.24 × , respectively, and the classification accuracy of the model decreased slightly by 0.22%. When ResNet-101 is pruned on ImageNet, the compression ratio and acceleration ratio are 3.90 × and 3.38 × , respectively, and the classification accuracy of the model decreased slightly by 0.48%. The proposed method is significantly better than state-of-the-art methods.

The huge size of deep neural networks makes it difficult to deploy on the embedded platforms with limited computation resources directly. In this paper, we propose a novel trimming approach to determine the redundant parameters of the trained deep neural network in a layer-wise manner to produce a compact neural network. This is achieved by minimizing a nonconvex sparsity-inducing term of the network parameters while maintaining the response close to the original one. We present a proximal iteratively reweighted method to resolve the resulting nonconvex model, which approximates the nonconvex objective by a weighted l1 norm of the network parameters. Moreover, to alleviate the computational burden, we develop a novel termination criterion during the subproblem solution, significantly reducing the total pruning time. Global convergence analysis and a worst-case O(1/k) ergodic convergence rate for our proposed algorithm is established. Numerical experiments demonstrate the proposed approach is efficient and reliable.