Search

Understanding and predicting each individual's real-time travel destination given the origin information in urban public transportation systems is crucial for personalized traveler recommendation, targeted demand management, dynamic traffic operations and so on. Existing methods are often based on modeling the regular travel patterns through analyzing the long-term personal travel information. They are suitable for destination prediction of individual regular trips with regular travel patterns, but may not work well for occasional trips with strong randomness and uncertainty, especially for the individuals with a few historical travel data. In this paper, we focus on more challenging issue about destination prediction of occasional trips. We design a general Multi-View Deep Learning Framework (MDLF) based on the data-driven insight that a location where a user will destine to is not only related to the user's own travel preference to the location, but also influenced by crowd's travel preference and the region's characteristics of the location under certain spatiotemporal contexts. The destination of an individual's occasional trip can be predicted by combining all these complementary influencing factors. The novelty of MDLF is mainly reflected in two aspects. The first is the effective feature extraction from multiple and complementary views. The second is that a CNN (Recurrent Neural Network) based deep learning component for predicting each occasional trip's destination by calculating a moving trend score for each possible destination. We evaluate the MDLF based on two real-world smart card datasets collected by AFC (Automatic Fare Collection) Systems. The experimental results demonstrate the superiority of MDLF against other competitors.

In recent years, various deep learning architectures have been proposed to solve complex challenges (e.g. spatial dependency, temporal dependency) in traffic domain, which have achieved satisfactory performance. These architectures are composed of multiple deep learning techniques in order to tackle various challenges in traffic tasks. Traditionally, convolution neural networks (CNNs) are utilized to model spatial dependency by decomposing the traffic network as grids. However, many traffic networks are graph-structured in nature. In order to utilize such spatial information fully, it's more appropriate to formulate traffic networks as graphs mathematically. Recently, various novel deep learning techniques have been developed to process graph data, called graph neural networks (GNNs). More and more works combine GNNs with other deep learning techniques to construct an architecture dealing with various challenges in a complex traffic task, where GNNs are responsible for extracting spatial correlations in traffic network. These graph-based architectures have achieved state-of-the-art performance. To provide a comprehensive and clear picture of such emerging trend, this survey carefully examines various graph-based deep learning architectures in many traffic applications. We first give guidelines to formulate a traffic problem based on graph and construct graphs from various kinds of traffic datasets. Then we decompose these graph-based architectures to discuss their shared deep learning techniques, clarifying the utilization of each technique in traffic tasks. What's more, we summarize some common traffic challenges and the corresponding graph-based deep learning solutions to each challenge. Finally, we provide benchmark datasets, open source codes and future research directions in this rapidly growing field., 21pages, 11figures

In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), into CO nvolutional M ax O ut (COMO) — a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for ResNet or Inception for VGG-alikes, COMO works well on both linear architectures and the ones with skipped connections and residual blocks. More specifically, COMO adopts a novel split-transform-merge paradigm that extends the layers with spatial resolution reduction into multiple parallel splits. For the layer with COMO, each split passes the input feature maps through a 4D convolution operator with independent batch normalization operators for transformation, then merge into the aggregated output of the original sizes through max-pooling . Such a strategy is expected to tackle the potential classification accuracy degradation due to the spatial resolution reduction, by incorporating the multiple splits and max-pooling-based feature selection. Our experiment using a wide range of deep architectures shows that COMO can significantly improve the classification accuracy of ResNet/VGG-alike networks based on a large number of benchmark datasets. COMO further outperforms the existing solutions, e.g., Inceptions, ResNeXts, SE-ResNet, and Xception, that make networks wider, and it dominates in the comparison of accuracy versus parameter sizes.

The multi-view of an object can be used for 3D reconstruction. The method proposed in this paper generates the left and the top view of a target car through deep learning. The input of the method is only a front view of a 3D car and it isn’t necessary for the depth of the 3D car. Firstly, a rough orthographic views of the 3D car is gotten from an information constraint network which is constructed by considering the structural relation between one view and the other two views. And then the rough orthographic views is transformed into large-pixel block rough view through the nearest interpolation, at the same time, the large-pixel blocks are also migrated to improve the quality of the rough orthographic views. Finally, the generative adversarial network with perception loss is used to enhance the large-pixel block view. In addition, the three views generated by the network can be used to synthesize a 3D point cloud shell.

The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and (3) performance analysis of medical image fusion on mainly data set. Finally, the conclusion of this paper is that the current multimodal medical image fusion research results are more significant and the development trend is on the rise but with many challenges in the research field.

Remaining useful life (RUL) prediction plays an important role in guaranteeing safe operation and reducing maintenance cost in modern industry. In this paper, we present a novel deep learning method for RUL estimation based on time empirical mode decomposition (EMD) and temporal convolutional networks (TCN). The proposed framework can effectively reveal the non-stationary characteristics of bearing degradation signals and acquire time-series degradation signals which namely intrinsic mode functions through empirical mode decomposition. Furthermore, the feature information is used as the input to convolution layer and trained by TCN to predict remaining useful life. The proposed EMD–TCN model structure maintains a superior result compared to several state-of-the-art convolutional algorithms on public data sets. Experimental results show that the average score of EMD–TCN model is improved by 10–20% than traditional convolutional algorithms.

Resource management is challenging in clouds due to the dynamics and sharing characteristics. The crucial problem is how to allocate resources accurately and satisfy demands of workloads timely. The traditional solution is to use historical data to predict future resource usage. Although these resource prediction methods can predict the periodicity, they can not accurately predict mutation points due to the high dynamics and uncertainty of resource usage. To tackle this issue, in this paper we propose a resource usage prediction method - RPTCN, which is based on a deep learning method - temporal convolutional networks (TCNs) in cloud systems. We add a fully connected layer and attention mechanism to TCNs to improve the prediction accuracy. In order to explore the relationship between the usage of different resources in the temporal dimension, we use correlation analysis to screen performance indicators as multidimensional feature input for prediction. Finally, we evaluate the performance of this method on Alibaba trace v2018. Evaluations show that RPTCN improves the overall MAE and MSE by 6.50%~89.03% and 0.41%~68.82% respectively compared to baselines in dynamic and long-term prediction of resource usage. Moreover, the convergence and generalization of RPTCN are also better than the baselines.

Highlights • The performance of the deep learning algorithm was on par with that of 7 expert pathologists in discriminating melanoma from nevus using whole-slide pathological images (WSIs). • Deep learning algorithm might function as a supplemental tool to assist pathologist by automatically pre-screening and highlighting interest regions prior to review., Background Deep learning has the potential to improve diagnostic accuracy and efficiency in medical image recognition. In the current study, we developed a deep learning algorithm and assessed its performance in discriminating melanoma from nevus using whole-slide pathological images (WSIs). Methods The deep learning algorithm was trained and validated using a set of 781 WSIs (86 melanomas, 695 nevi) from PLA General Hospital. The diagnostic performance of the algorithm was tested on an independent test set of 104 WSIs (29 melanomas, 75 nevi) from Tianjin Chang Zheng Hospital. The same test set was also diagnostically classified by 7 expert dermatopathologists. Results The deep learning algorithm receiver operating characteristic (ROC) curve achieved a sensitivity 100% at the specificity of 94.7% in the classification of melanoma and nevus on the test set. The area under ROC curve was 0.99. Dermatopathologists achieved a mean sensitivity and specificity of 95.1% (95% confidence interval [CI]: 92.0%-98.2%) and 96.0% (95% CI: 94.2%-97.8%), respectively. At the operating point of sensitivity of 95.1%, the algorithm revealed a comparable specificity with 7 dermatopathologists (97.3% vs. 96.0%, P = 0.11). At the operating point of specificity of 96.0%, the algorithm also achieved a comparable sensitivity with 7 dermatopathologists (96.5% vs. 95.1%, P = 0.30). A more transparent and interpretable diagnosis could be generated by highlighting the regions of interest recognized by the algorithm in WSIs. Conclusion The performance of the deep learning algorithm was on par with that of 7 expert dermatopathologists in interpreting WSIs with melanocytic lesions. By pre-screening the suspicious melanoma regions, it might serve as a supplemental diagnostic tool to improve working efficiency of pathologists.

A technological revolution is occurring in the field of robotics with the data-driven deep learning technology. However, building datasets for each local robot is laborious. Meanwhile, data islands between local robots make data unable to be utilized collaboratively. To address this issue, the work presents Peer-Assisted Robotic Learning (PARL) in robotics, which is inspired by the peer-assisted learning in cognitive psychology and pedagogy. PARL implements data collaboration with the framework of cloud robotic systems. Both data and models are shared by robots to the cloud after semantic computing and training locally. The cloud converges the data and performs augmentation, integration, and transferring. Finally, fine tune this larger shared dataset in the cloud to local robots. Furthermore, we propose the DAT Network (Data Augmentation and Transferring Network) to implement the data processing in PARL. DAT Network can realize the augmentation of data from multi-local robots. We conduct experiments on a simplified self-driving task for robots (cars). DAT Network has a significant improvement in the augmentation in self-driving scenarios. Along with this, the self-driving experimental results also demonstrate that PARL is capable of improving learning effects with data collaboration of local robots.

The quality of Automatic Identification System (AIS) data is of fundamental importance for maritime situational awareness and navigation risk assessment. To improve operational efficiency, a deep learning method based on Bi-directional Long Short-Term Memory Recurrent Neural Networks (BLSTM-RNNs) is proposed and applied in AIS trajectory data restoration. Case studies have been conducted in two distinct reaches of the Yangtze River and the capability of the proposed method has been evaluated. Comparisons have been made between the BLSTM-RNNs-based method and the linear method and classic Artificial Neural Networks. Satisfactory results have been obtained by all methods in straight waterways while the BLSTM-RNNs-based method is superior in meandering waterways. Owing to the bi-directional prediction nature of the proposed method, ship trajectory restoration is favourable for complicated geometry and multiple missing points cases. The residual error of the proposed model is computed through Euclidean distance which decreases to an order of 10 m. It is considered that the present study could provide an alternative method for improving AIS data quality, thus ensuring its completeness and reliability.

Copy number variations (CNVs) are an important class of variations contributing to the pathogenesis of many disease phenotypes. Detecting CNVs from genomic data remains difficult, and the most currently applied methods suffer from an unacceptably high false positive rate. A common practice is to have human experts manually review original CNV calls for filtering false positives before further downstream analysis or experimental validation. Here, we propose DeepCNV, a deep learning-based tool, intended to replace human experts when validating CNV calls, focusing on the calls made by one of the most accurate CNV callers, PennCNV. The sophistication of the deep neural network algorithm is enriched with over 10 000 expert-scored samples that are split into training and testing sets. Variant confidence, especially for CNVs, is a main roadblock impeding the progress of linking CNVs with the disease. We show that DeepCNV adds to the confidence of the CNV calls with an optimal area under the receiver operating characteristic curve of 0.909, exceeding other machine learning methods. The superiority of DeepCNV was also benchmarked and confirmed using an experimental wet-lab validation dataset. We conclude that the improvement obtained by DeepCNV results in significantly fewer false positive results and failures to replicate the CNV association results.

Stock price movement prediction is commonly accepted as a very challenging task due to the volatile nature of financial markets. Previous works typically predict the stock price mainly based on its own information, neglecting the cross effect among involved stocks. However, it is well known that an individual stock price is correlated with prices of other stocks in complex ways. To take the cross effect into consideration, we propose a deep learning framework, called Multi-GCGRU, which comprises graph convolutional network (GCN) and gated recurrent unit (GRU) to predict stock movement. Specifically, we first encode multiple relationships among stocks into graphs based on financial domain knowledge and utilize GCN to extract the cross effect based on these pre-defined graphs. To further get rid of prior knowledge, we explore an adaptive relationship learned by data automatically. The cross-correlation features produced by GCN are concatenated with historical records and then fed into GRU to model the temporal dependency of stock prices. Experiments on two stock indexes in China market show that our model outperforms other baselines. Note that our model is rather feasible to incorporate more effective stock relationships containing expert knowledge, as well as learn data-driven relationship., Comment: 8pages, 4figures

With the improvement of data volume, computing power and algorithms, deep learning has achieved rapid development and showing excellent performance. Recently, many deep learning models are proposed to solve the problems in different areas. A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior, which makes it applicable to tasks such as handwriting recognition or speech recognition. However, the RNN relies heavily on the automatic learning ability to update parameters that concentrate on the data flow but seldom considers the feature extraction capability of the gate mechanism. In this paper, we propose a novel architecture to build the forget gate which is generated by multiple bases. Instead of using the traditional single-layer fully-connected network, we use a Multiple Attention (MA) based network to generate the forget gate which refines the optimization space of gate function and improve the granularity of the recurrent neural network to approximate the map in the ground truth. Due to the benefit of MA structure on the gate mechanism, the proposed MA-LSTM model achieves better feature extraction capability than other known models.

Accurate segmentation of cardiac structures can assist doctors to diagnose diseases, and to improve treatment planning, which is highly demanded in the clinical practice. However, the shortage of annotation and the variance of the data among different vendors and medical centers restrict the performance of advanced deep learning methods. In this work, we present a fully automatic method to segment cardiac structures including the left (LV) and right ventricle (RV) blood pools, as well as for the left ventricular myocardium (MYO) in MRI volumes. Specifically, we design a semi-supervised learning method to leverage unlabelled MRI sequence timeframes by label propagation. Then we exploit style transfer to reduce the variance among different centers and vendors for more robust cardiac image segmentation. We evaluate our method in the M&Ms challenge (https://www.ub.edu/mnms/), ranking 2nd place among 14 competitive teams.

Automatic brain tumor segmentation from multi-modality Magnetic Resonance Images (MRI) using deep learning methods plays an important role in assisting the diagnosis and treatment of brain tumor. However, previous methods mostly ignore the latent relationship among different modalities. In this work, we propose a novel end-to-end Modality-Pairing learning method for brain tumor segmentation. Paralleled branches are designed to exploit different modality features and a series of layer connections are utilized to capture complex relationships and abundant information among modalities. We also use a consistency loss to minimize the prediction variance between two branches. Besides, learning rate warmup strategy is adopted to solve the problem of the training instability and early over-fitting. Lastly, we use average ensemble of multiple models and some post-processing techniques to get final results. Our method is tested on the BraTS 2020 online testing dataset, obtaining promising segmentation performance, with average dice scores of 0.891, 0.842, 0.816 for the whole tumor, tumor core and enhancing tumor, respectively. We won the second place of the BraTS 2020 Challenge for the tumor segmentation task.

There is a large body of literature linking anatomic and geometric characteristics of kidney tumors to perioperative and oncologic outcomes. Semantic segmentation of these tumors and their host kidneys is a promising tool for quantitatively characterizing these lesions, but its adoption is limited due to the manual effort required to produce high-quality 3D segmentations of these structures. Recently, methods based on deep learning have shown excellent results in automatic 3D segmentation, but they require large datasets for training, and there remains little consensus on which methods perform best. The 2019 Kidney and Kidney Tumor Segmentation challenge (KiTS19) was a competition held in conjunction with the 2019 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) which sought to address these issues and stimulate progress on this automatic segmentation problem. A training set of 210 cross sectional CT images with kidney tumors was publicly released with corresponding semantic segmentation masks. 106 teams from five continents used this data to develop automated systems to predict the true segmentation masks on a test set of 90 CT images for which the corresponding ground truth segmentations were kept private. These predictions were scored and ranked according to their average So rensen-Dice coefficient between the kidney and tumor across all 90 cases. The winning team achieved a Dice of 0.974 for kidney and 0.851 for tumor, approaching the inter-annotator performance on kidney (0.983) but falling short on tumor (0.923). This challenge has now entered an "open leaderboard" phase where it serves as a challenging benchmark in 3D semantic segmentation., 24 pages, 11 figures

Coronavirus disease 2019 (COVID-19) is a highly contagious virus spreading all around the world. Deep learning has been adopted as an effective technique to aid COVID-19 detection and segmentation from computed tomography (CT) images. The major challenge lies in the inadequate public COVID-19 datasets. Recently, transfer learning has become a widely used technique that leverages the knowledge gained while solving one problem and applying it to a different but related problem. However, it remains unclear whether various non-COVID19 lung lesions could contribute to segmenting COVID-19 infection areas and how to better conduct this transfer procedure. This paper provides a way to understand the transferability of non-COVID19 lung lesions. Based on a publicly available COVID-19 CT dataset and three public non-COVID19 datasets, we evaluate four transfer learning methods using 3D U-Net as a standard encoder-decoder method. The results reveal the benefits of transferring knowledge from non-COVID19 lung lesions, and learning from multiple lung lesion datasets can extract more general features, leading to accurate and robust pre-trained models. We further show the capability of the encoder to learn feature representations of lung lesions, which improves segmentation accuracy and facilitates training convergence. In addition, our proposed Hybrid-encoder learning method incorporates transferred lung lesion features from non-COVID19 datasets effectively and achieves significant improvement. These findings promote new insights into transfer learning for COVID-19 CT image segmentation, which can also be further generalized to other medical tasks., Comment: 11 pages

Subway passenger flow forecasting, an essential component of intelligent transportation system, is critical for traffic management, public safety, urban planning. However, it is very challenging due to the high nonlinearities and complex dynamic spatio-temporal dependencies of passenger flows. In this paper, we model the subway system as a directed weighted graph and propose a novel spatio-temporal deep learning framework, Multi-STGCnet, for forecasting short-term subway passenger flow at a station level. Specifically, Multi-STGCnet is mainly composed of two components, temporal component and spatial component. (1) The temporal component employs three long short-term memory network (LSTM)-based modules to capture three temporal properties of the target station, which are the interval closeness, daily periodicity, weekly trend. (2) The spatial component designs three spatial matrixes to extract spatial correlation of a target station with all other stations classified as near neighbors, middle neighbors and distant neighbors. Respectively, it adopts three graph convolution network (GCN) and LSTM combined modules to capture the spatio-temporal influences from different neighbors. Finally, the outputs of the two components are fused with different weights to generate prediction. We evaluate Multi-STGCnet on a real world dataset from the metro system in Shenzhen, China. Experiment results demonstrate that our model outperforms multiple baselines.

Though deep learning has achieved advanced performance recently, it remains a challenging task in the field of medical imaging, as obtaining reliable labeled training data is time-consuming and expensive. In this paper, we propose a double-uncertainty weighted method for semi-supervised segmentation based on the teacher-student model. The teacher model provides guidance for the student model by penalizing their inconsistent prediction on both labeled and unlabeled data. We train the teacher model using Bayesian deep learning to obtain double-uncertainty, i.e. segmentation uncertainty and feature uncertainty. It is the first to extend segmentation uncertainty estimation to feature uncertainty, which reveals the capability to capture information among channels. A learnable uncertainty consistency loss is designed for the unsupervised learning process in an interactive manner between prediction and uncertainty. With no ground-truth for supervision, it can still incentivize more accurate teacher’s predictions and facilitate the model to reduce uncertain estimations. Furthermore, our proposed double-uncertainty serves as a weight on each inconsistency penalty to balance and harmonize supervised and unsupervised training processes. We validate the proposed feature uncertainty and loss function through qualitative and quantitative analyses. Experimental results show that our method outperforms the state-of-the-art uncertainty-based semi-supervised methods on two public medical datasets.

In the field of computer-aided diagnosis (CAD) and treatment evaluation system on hepatic disease diagnosis, the automatic segmentation of the liver from CT volume is the most basic step. It is a difficult task because the shape of liver could be changed by liver tumor, and the intensity of liver is similar to that of other adjacent tissues. In this paper, we proposed a framework based on the U-net architecture, called dense pyramid network to segment the liver from CT images automatically. The main contribution of our network is that, multiple feature maps from the previous level of hierarchy are combined as the input of each layer in the encoding part. This removes the loss of context information between different layers. The model is trained on practical enhanced CT scans, which are gained from People’s Liberation Army General Hospital (PLA). Experimental results demonstrate that our model can effectively improve the segmentation performance of liver, no matter the different shapes between livers. In the experiment, the Dice score of liver segmentation in the arterial phase, venous phase, and delay phase by dense pyramid network was about 95.97%, 96.22%, and 96.16%, respectively, which shows that our model is more suitable for multi-phase liver segmentation.

SLAM has shown great values in many fields such as self-driving cars, virtual reality and robotic localization, etc. Cloud robot based collaborative SLAM can effectively improve the efficiency of mapping tasks. However, place recognition and matching accuracy among different robots can greatly affect the map fusion performance of the entire SLAM system. Therefore, this paper presents a learning architecture for cooperative SLAM named FC-SLAM, a distributed SLAM in cloud robotic systems by taking advantage of federated learning to enhance the performance of visual-LiDAR SLAM. Additionally, we propose a federated deep learning algorithm for feature extraction and dynamic vocabulary designation which works in real-time on cloud workstation. FC-SLAM can ensure real-time collaborative SLAM by keep the original images on robot side instead of sending them to the cloud server. We test our system on open datasets and in simulated environment. The results show that it has better feature extraction performance than SIFT and ORB under illumination and viewpoint changes. Besides, map fusion is conducted to generate a global map according to place matching relation of distributed robots.

Landslide inventories could provide fundamental data for analyzing the causative factors and deformation mechanisms of landslide events. Considering that it is still hard to detect landslides automatically from remote sensing images, endeavors have been carried out to explore the potential of DCNNs on landslide detection, and obtained better performance than shallow machine learning methods. However, there is often confusion as to which structure, layer number, and sample size are better for a project. To fill this gap, this study conducted a comparative test on typical models for landside detection in the Wenchuan earthquake area, where about 200,000 secondary landslides were available. Multiple structures and layer numbers, including VGG16, VGG19, ResNet50, ResNet101, DenseNet120, DenseNet201, UNet−, UNet+, and ResUNet were investigated with different sample numbers (100, 1000, and 10,000). Results indicate that VGG models have the highest precision (about 0.9) but the lowest recall (below 0.76); ResNet models display the lowest precision (below 0.86) and a high recall (about 0.85); DenseNet models obtain moderate precision (below 0.88) and recall (about 0.8); while UNet+ also achieves moderate precision (0.8) and recall (0.84). Generally, a larger sample set can lead to better performance for VGG, ResNet, and DenseNet, and deeper layers could improve the detection results for ResNet and DenseNet. This study provides valuable clues for designing models’ type, layers, and sample set, based on tests with a large number of samples.

The Internet and computer networks are currently suffering from serious security threats. Those threats often keep changing and will evolve to new unknown variants. In order to maintain the security of network, we design and implement a dynamic network anomaly detection system using deep learning methods. We use Long Short Term Memory (LSTM) to build a deep neural network model and add an Attention Mechanism (AM) to enhance the performance of the model. The SMOTE algorithm and an improved loss function are used to handle the class-imbalance problem in the CSE-CIC-IDS2018 dataset. The experimental results show that the classification accuracy of our model reaches 96.2%, which is higher than other machine learning algorithms. In addition, the class-imbalance problem is alleviated to a certain extent, making our method have great practicality.

With the rapid development of the Internet and the growing complexity of the network topology, network anomaly has become more diverse. In this paper, we propose an algorithm named Deep Adaptive Feature Learning (DAFL) for traffic anomaly detection based on deep learning model. By setting proper feature parameters \(\theta \) on the neural network structure, DAFL can effectively generate low-dimensional new abstract features. Experimental results show the DAFL algorithm has good adaptability and robustness, which can effectively improve the detection accuracy and significantly reduce the detection time.

The Internet and computer networks are currently suffering from different security threats. This paper presents a new method called AMF-LSTM for abnormal traffic detection by using deep learning model. We use the statistical features of multi-flows rather than a single flow or the features extracted from log as the input to obtain temporal correlation between flows, and add an attention mechanism to the original LSTM to help the model learn which traffic flow has more contributions to the final results. Experiments show AMF-LSTM method has high accuracy and recall in anomaly type identification.

Network anomaly detection is the process of determining when network behavior has deviated from the normal behavior. The detection of abnormal events in large dynamic network has become increasingly important as networks grow in size and complexity. However, fast and accurate network anomaly detection is very challenging. Deep learning is a potential method for network anomaly detection due to its good feature modeling capability. This paper presents a new anomaly detection method based on deep learning models, specifically the feedforward neural network (FNN) model and convolutional neural network (CNN) model. The performance of the models is evaluated by several experiments with a popular NSL-KDD dataset. From the experimental results, we find the FNN and CNN models not only have a strong modeling ability for network anomaly detection, but also have high accuracy. Compared with several traditional machine learning methods, such as J48, Naive Bayes, NB Tree, Random Forest, Random Tree and SVM, the proposed models obtain a higher accuracy and detection rate with lower false positive rate. The deep learning models can effectively improve both the detection accuracy and the ability to identify anomaly types.

© 1979-2012 IEEE. The capabilities of (I) learning transferable knowledge across domains; and (II) fine-tuning the pre-learned base knowledge towards tasks with considerably smaller data scale are extremely important. Many of the existing transfer learning techniques are supervised approaches, among which deep learning has the demonstrated power of learning domain transferrable knowledge with large scale network trained on massive amounts of labeled data. However, in many biomedical tasks, both the data and the corresponding label can be very limited, where the unsupervised transfer learning capability is urgently needed. In this paper, we proposed a novel multi-scale convolutional sparse coding (MSCSC) method, that (I) automatically learns filter banks at different scales in a joint fashion with enforced scale-specificity of learned patterns; and (II) provides an unsupervised solution for learning transferable base knowledge and fine-tuning it towards target tasks. Extensive experimental evaluation of MSCSC demonstrates the effectiveness of the proposed MSCSC in both regular and transfer learning tasks in various biomedical domains.

Part 5: Data Processing and Big Data; International audience; Recently, Deep Learning (DL) has become super hot because it achieves breakthroughs in many areas such as image processing and face identification. The performance of DL models critically depend on hyperparameter settings. However, existing approaches that quantify the importance of these hyperparameters are time-consuming.In this paper, we propose a fast approach to quantify the importance of the DL hyperparameters, called QIM. It leverages Plackett-Burman design to collect as few as possible data but can still correctly quantify the hyperparameter importance. We conducted experiments on the popular deep learning framework – Caffe – with different datasets to evaluate QIM. The results show that QIM can rank the importance of the DL hyperparameters correctly with very low cost.