Search

With a large number of distance measures, the appropriate choice for clustering a given data set with a specified clustering algorithm becomes an important problem. In this article, an automatic distance measure recommendation method for clustering algorithms is proposed. The recommendation method consists of the following steps: (1) metadata extraction, including meta-feature collection and meta-target identification; (2) recommendation model construction using metadata; and (3) distance measure recommendation for a new data set by the recommendation model. Two different types of meta-targets and meta-learning techniques are utilized considering the possible different requirements of users. To validate the necessity and effectiveness of the distance measure recommendation method, an empirical study is conducted with 199 publicly available data sets, 9 distance measures, and 2 widely used clustering algorithms. The experimental results indicate that distance measure significantly influences the performance of the clustering algorithm for a given data set. Furthermore, performance analysis of the proposed recommendation method proves its effectiveness.

Due to the significance and value in human-computer interaction and natural language processing, task-oriented dialog systems are attracting more and more attention in both academic and industrial communities. In this paper, we survey recent advances and challenges in task-oriented dialog systems. We also discuss three critical topics for task-oriented dialog systems: (1) improving data efficiency to facilitate dialog modeling in low-resource settings, (2) modeling multi-turn dynamics for dialog policy learning to achieve better task-completion performance, and (3) integrating domain ontology knowledge into the dialog model. Besides, we review the recent progresses in dialog evaluation and some widely-used corpora. We believe that this survey, though incomplete, can shed a light on future research in task-oriented dialog systems.

The ability of story comprehension is a strong indicator of natural language understanding. Recently, Story Cloze Test has been introduced as a new task of machine reading comprehension, i.e., selecting a correct ending from two candidate endings given a four-sentence story context. Most existing methods for Story Cloze Test are essentially matching-based that operate by comparing an individual ending with a given context, therefore suffering from the evidence bias issue: both candidate endings can obtain supporting evidence from the story context, which misleads the classifier to choose an incorrect ending. To address this issue, we present a novel idea to improve story comprehension by utilizing the hints that are obtained through comparing two candidate endings. The proposed model firstly anticipates a feature vector for a possible ending solely based on the context, and then refines the feature prediction using the hints which encode the difference between two candidates. The candidate ending whose feature vector is more similar to the predicted ending vector is regarded as correct. Experimental results demonstrate that our approach can alleviate the evidence bias issue and improve story comprehension.

Background Cumulative evidence from biological experiments has confirmed that miRNAs have significant roles to diagnose and treat complex diseases. However, traditional medical experiments have limitations in time-consuming and high cost so that they fail to find the unconfirmed miRNA and disease interactions. Thus, discovering potential miRNA-disease associations will make a contribution to the decrease of the pathogenesis of diseases and benefit disease therapy. Although, existing methods using different computational algorithms have favorable performances to search for the potential miRNA-disease interactions. We still need to do some work to improve experimental results. Results We present a novel combined embedding model to predict MiRNA-disease associations (CEMDA) in this article. The combined embedding information of miRNA and disease is composed of pair embedding and node embedding. Compared with the previous heterogeneous network methods that are merely node-centric to simply compute the similarity of miRNA and disease, our method fuses pair embedding to pay more attention to capturing the features behind the relative information, which models the fine-grained pairwise relationship better than the previous case when each node only has a single embedding. First, we construct the heterogeneous network from supported miRNA-disease pairs, disease semantic similarity and miRNA functional similarity. Given by the above heterogeneous network, we find all the associated context paths of each confirmed miRNA and disease. Meta-paths are linked by nodes and then input to the gate recurrent unit (GRU) to directly learn more accurate similarity measures between miRNA and disease. Here, the multi-head attention mechanism is used to weight the hidden state of each meta-path, and the similarity information transmission mechanism in a meta-path of miRNA and disease is obtained through multiple network layers. Second, pair embedding of miRNA and disease is fed to the multi-layer perceptron (MLP), which focuses on more important segments in pairwise relationship. Finally, we combine meta-path based node embedding and pair embedding with the cost function to learn and predict miRNA-disease association. The source code and data sets that verify the results of our research are shown at https://github.com/liubailong/CEMDA. Conclusions The performance of CEMDA in the leave-one-out cross validation and fivefold cross validation are 93.16% and 92.03%, respectively. It denotes that compared with other methods, CEMDA accomplishes superior performance. Three cases with lung cancers, breast cancers, prostate cancers and pancreatic cancers show that 48,50,50 and 50 out of the top 50 miRNAs, which are confirmed in HDMM V2.0. Thus, this further identifies the feasibility and effectiveness of our method.

Unsupervised commonsense question answering is appealing since it does not rely on any labeled task data. Among existing work, a popular solution is to use pre-trained language models to score candidate choices directly conditioned on the question or context. However, such scores from language models can be easily affected by irrelevant factors, such as word frequencies, sentence structures, etc. These distracting factors may not only mislead the model to choose a wrong answer but also make it oversensitive to lexical perturbations in candidate answers. In this paper, we present a novel SEmantic-based Question Answering method (SEQA) for unsupervised commonsense question answering. Instead of directly scoring each answer choice, our method first generates a set of plausible answers with generative models (e.g., GPT-2), and then uses these plausible answers to select the correct choice by considering the semantic similarity between each plausible answer and each choice. We devise a simple, yet sound formalism for this idea and verify its effectiveness and robustness with extensive experiments. We evaluate the proposed method on four benchmark datasets, and our method achieves the best results in unsupervised settings. Moreover, when attacked by TextFooler with synonym replacement, SEQA demonstrates much less performance drops than baselines, thereby indicating stronger robustness., Comment: Accepted by ACL 2021 (long paper)

Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570 GB training data, drew a lot of attention due to the capacity of few-shot (even zero-shot) learning. However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available. In this technical report, we release the Chinese Pre-trained Language Model (CPM) with generative pre-training on large-scale Chinese training data. To the best of our knowledge, CPM, with 2.6 billion parameters and 100 GB Chinese training data, is the largest Chinese pre-trained language model, which could facilitate several downstream Chinese NLP tasks, such as conversation, essay generation, cloze test, and language understanding. Extensive experiments demonstrate that CPM achieves strong performance on many NLP tasks in the settings of few-shot (even zero-shot) learning. The code and parameters are available at https://github.com/TsinghuaAI/CPM .

Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge is required during generation. Existing approaches that integrate commonsense knowledge into generative pre-trained language models simply transfer relational knowledge by post-training on individual knowledge triples while ignoring rich connections within the knowledge graph. We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. In this paper, we propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph. We empirically show that our model outperforms existing baselines on three text generation tasks that require reasoning over commonsense knowledge. We also demonstrate the effectiveness of the dynamic multi-hop reasoning module with reasoning paths inferred by the model that provide rationale to the generation., accepted by EMNLP 2020

Background Genotyping of structural variation is an important computational problem in next generation sequence data analysis. However, in cancer genomes, the copy number variant(CNV) often coexists with other types of structural variations which significantly reduces the accuracy of the existing genotype methods. The bias on sequencing coverage and variant allelic frequency can be observed on a CNV region, which leads to the genotyping approaches that misinterpret the heterozygote as a homozygote. Furthermore, other data signals such as split mapped read, abnormal read will also be misjudged because of the CNV. Therefore, genotyping the structural variations with CNV is a complicated computational problem which should consider multiple features and their interactions. Methods Here we proposed a computational method for genotyping indels in the CNV region, which introduced a machine learning framework to comprehensively incorporate a set of data features and their interactions. We extracted fifteen kinds of classification features as input and different from the traditional genotyping problem, here the structure of variant may fall into types of normal homozygote, homozygous variant, heterozygous variant without CNV, heterozygous variant with a CNV on the mutated haplotype, and heterozygous variant with a CNV on the wild haplotype. The Multiclass Relevance Vector Machine (M-RVM) was used as a machine learning framework combined with the distribution characteristics of the features. Results We applied the proposed method to both simulated and real data, and compared it with the existing popular softwares include Gindel, Facets, GATK, and also compared with other machine learning cores: Support Vector Machine, Lanrange-SVM with OVO multiple classification, Naïve Bayes and BP Neural Network. The results demonstrated that the proposed method outperforms others on accuracy, stability and efficiency. Conclusion This work shows that the genotyping of structural variations on the CNV region cannot be solved as a traditional genotyping problem. More features should be used to efficiently complete the five-category task. According to the result, the proposed method can be a practical algorithm to correct genotype structural variations with CNV on the next generation sequence data. The source codes have been uploaded at https://github.com/TrinaZ/Mixgenotypefor academic usage only.

With the rapid development of the mobile Internet and the increasing popularity of smart terminals, various mobile social applications are emerging. Medical data has become a valuable data asset and is being continuously explored and utilized, which has greatly promoted the improvement of the medical service level. However, publishing and using user data makes the user vulnerable to reasoning attacks. Due to the special nature of the medical field, medical data not only carries the health status of patients and medical process information but also involves individual sensitive information of a large number of patients. Allowing users to fully enjoy the advantages brought by social networks while ensuring security is an important issue that needs to be solved urgently in the era of big data. In this paper, we first provide an overview of social network data privacy risks and various types of attacks. Aiming at the privacy leakage of weighted social networks, we propose a privacy protection recommendation algorithm based on differential privacy. The algorithm utilizes the change of edge weight grouping, which greatly reduces the amount of calculation and satisfies the user's rapid response. It minimizes a privacy leak of user private data under data availability while supporting personalized rankings. Compared to the most advanced methods, this method protects users from reasoning attacks and reduces the distortion of ranking results caused by data confusion to ensure the accuracy of recommendations. Experiments on real-world datasets show that our framework can achieve more effective and lasting protection for user-sensitive data.

To advance multi-domain (cross-domain) dialogue modeling as well as alleviate the shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first large-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dataset. It contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains rich annotation of dialogue states and dialogue acts at both user and system sides. About 60% of the dialogues have cross-domain user goals that favor inter-domain dependency and encourage natural transition across domains in conversation. We also provide a user simulator and several benchmark models for pipelined task-oriented dialogue systems, which will facilitate researchers to compare and evaluate their models on this corpus. The large size and rich annotation of CrossWOZ make it suitable to investigate a variety of tasks in cross-domain dialogue modeling, such as dialogue state tracking, policy learning, user simulation, etc., Accepted by TACL

Story generation, namely generating a reasonable story from a leading context, is an important but challenging task. In spite of the success in modeling fluency and local coherence, existing neural language generation models (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of long-range coherence in generated stories. We conjecture that this is because of the difficulty of associating relevant commonsense knowledge, understanding the causal relationships, and planning entities and events with proper temporal order. In this paper, we devise a knowledge-enhanced pretraining model for commonsense story generation. We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. To further capture the causal and temporal dependencies between the sentences in a reasonable story, we employ multi-task learning which combines a discriminative objective to distinguish true and fake stories during fine-tuning. Automatic and manual evaluation shows that our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence., Accept at Transactions of the Association for Computational Linguistics 2020

In recent years, with the development of mobile terminals, geographic location has attracted the attention of many researchers because of its convenience in collection and its ability to reflect user profile. To protect user privacy, researchers have adopted local differential privacy in data collection process. However, most existing methods assume that location has already been discretized, which we found, if not done carefully, may introduces huge noise, lowering collected result utility. Thus in this paper, we design a differentially private location division module that could automatically discretize locations according to access density of each region. However, as the size of discretized regions may be large, if directly applying existing local differential privacy based attribute method, the overall utility of collected results may be completely destroyed. Thus, we further improve the optimized binary local hash method, based on personalized differential privacy, to collect user visit frequency of each discretized region. This solution improve the accuracy of the collected results while satisfying the privacy of the user’s geographic location. Through experiments on synthetic and real data sets, this paper proves that the proposed method achieves higher accuracy than the best known method under the same privacy budget.

The advancements of neural dialogue generation models show promising results on modeling short-text conversations. However, training such models usually needs a large-scale high-quality dialogue corpus, which is hard to access. In this paper, we present a large-scale cleaned Chinese conversation dataset LCCC, which contains a base version (6.8 million dialogues) and a large version (12.0 million dialogues). The quality of our dataset is ensured by a rigorous data cleaning pipeline, which is built based on a set of rules and a classifier that is trained on manually annotated 110K dialogue pairs. We also release pre-training dialogue models which are trained on LCCC-base and LCCC-large respectively. The cleaned dataset and the pre-training models will facilitate the research of short-text conversation modeling. All the models and datasets are available at https://github.com/thu-coai/CDial-GPT.

The research of knowledge-driven conversational systems is largely limited due to the lack of dialog data which consist of multi-turn conversations on multiple topics and with knowledge annotations. In this paper, we propose a Chinese multi-domain knowledge-driven conversation dataset, KdConv, which grounds the topics in multi-turn conversations to knowledge graphs. Our corpus contains 4.5K conversations from three domains (film, music, and travel), and 86K utterances with an average turn number of 19.0. These conversations contain in-depth discussions on related topics and natural transition between multiple topics. To facilitate the following research on this corpus, we provide several benchmark models. Comparative results show that the models can be enhanced by introducing background knowledge, yet there is still a large space for leveraging knowledge to model multi-turn conversations for further research. Results also show that there are obvious performance differences between different domains, indicating that it is worth to further explore transfer learning and domain adaptation. The corpus and benchmark models are publicly available.

The rapid expansion of urbanization and the fast pace of life result in abundant choices with little time for people to manage routes. A proper planning route enables us to enjoy life better with fewer time and energy costs. Therefore, route planning becomes too valuable to be ignored. At the same time, with the popularity of mobile devices, location sensing, and Web 2.0 technologies, location-based social networks (e.g., Facebook Palaces and Foursquare) have attracted millions of users to share their visited locations and other information, which generates large amounts of user check-in data. These data can be used to mine users’ preferences and time information for recommending routes. In this paper, we propose FineRoute, a personalized and time-sensitive route recommendation system. We take three factors users’ preferences, proper visiting time, and transition time into consideration for the route generation. First, we infer users’ preferences by constructing a three-dimensional tensor, with three dimensions representing users, locations, and time, respectively. Second, we obtain the proper visiting time for certain locations, as well as the transition time between two locations from the check-in dataset. Moreover, we adopt Kullback–Leibler divergence in order to measure the quality of a route in terms of the proper visiting month and the proper visiting hour. Finally, we propose a route generation algorithm by extending the classic longest path algorithm. We conduct experiments on a real-world check-in dataset and the results demonstrate the effectiveness of our scheme.

Phrase/Sentence representation is one of the most important problems in natural language processing. Many neural network models such as Convolutional Neural Network (CNN), Recursive Neural Network (RNN), and Long Short-Term Memory (LSTM) have been proposed to learn representations of phrase/sentence, however, rich syntactic knowledge has not been fully explored when composing a longer text from its shorter constituent words. In most traditional models, only word embeddings are utilized to compose phrase/sentence representations, while the syntactic information of words is yet to be explored. In this article, we discover that encoding syntactic knowledge (part-of-speech tag) in neural networks can enhance sentence/phrase representation. Specifically, we propose to learn tag-specific composition functions and tag embeddings in recursive neural networks, and propose to utilize POS tags to control the gates of tree-structured LSTM networks. We evaluate these models on two benchmark datasets for sentiment classification, and demonstrate that improvements can be obtained with such syntactic knowledge encoded.

Collaborative deep learning can provide high learning accuracy even participanted users' datasets are small. In the training process, users only share their locally obtained parameters, therefore it is believed that the privacy of users' original datasets can be protected. However, we present an attack approach against users' privacy in collaborative deep learning by utilizing Generative Adversarial Network (GAN) and Membership Inference. In this attack, an attacker builds a discriminator based on users' shared parameters and then trains a GAN network locally. The GAN can refactor the training records of the collaborative deep learning system. According to the generated records, the attacker uses the extent of model overfitting on an input and gets the membership of each group of records by the simplified Membership Inference attack. We evaluate the presented attack model over datasets of complex representations of handwritten digits (MINIST) and face images (CelebA). The results show that an attacker can easily generate the original training sets and classify them to obtain the membership between users' records and their identities in the collaborative deep learning.

The flourishing of intelligent mobile devices, significantly prompts various location-Based Services (LBSs), especially continuous LBSs. However, continuous LBSs can suffer from location association attacks and trajectory attacks. The existing dummy-based schemes aren't protect users' privacy effectively since ignoring the reasonability of dummies. Therefore, in this paper we consider the Time-Sensitive Side Information(TSSI), spatiotemporal correlations and movement patterns of previous locations simultaneously to present a Dummy Filtering Algorithm(DFA). First we consider the TSSI to generate dummies. Then we propose two constraints time reachability and out-degree/indegree to cut off spatiotemporal correlations. Next, we utilize area overlapping and trend similarity to protect users' trajectory privacy. Security analysis indicates that our scheme can assure dummies' rationality and availability. Furthermore, experiments show that our scheme not only defends location association attacks and trajectory attacks effectively, but increases the ratio of available dummies from 42% up to 66.7%.

Location-based social network (LBSN) is a highly used social network at present. Detecting the community structures in LBSN plays a significant role in analyzing the network comprehensively. Modularity optimization is a traditional social network community detection method which has excellent performance. Whereas, resolution limit is an inherent problem of it. Some existing community detection methods for LBSN fail to take account of the full network information. To solve above problems, a network weighting scheme is proposed in this paper. Firstly, we combine a network embedding method that named walk2friends with this paper. In view of the shortcoming of walk2friends, we proposes a collaborative filtering method based on social relationship to improve it. Then, the learned similarity is obtained by improved walk2friends and the structural similarity is obtained by a random walk method. These two similarities are fused and then transformed into edge weight. Finally, the undirected unweighted graph is preprocessed as an undirected weighted graph. Experimental results on real networks show that when using modularity optimization method for optimization optimization the weighting scheme not only can avoid the resolution limit but also applicable to LBSN . The mined community is more cohesive than the existing work on LBSN.

With the widespread adoption of mobile devices and technical improvement of localization technologies, users can benefit from Location-Based Services (LBSs), which need users' private information. However, users' privacy are vulnerable. Even though the k-anonymity technique has been widely used, adversaries who have external spatiotemporal information still reveal users' privacy. Therefore, in this paper, we propose a novel dummy-location generation mechanism. Firstly, through computing the Super Concept-based Distance (SCD) between candidate dummies and the real location, we select 4k dummies which are semantically similar with the real one. Secondly, by checking the Euclidean distances with the real location, we choose 2k satisfied dummies. Finally, by comparing the multiplication of semantic similarities and Euclidean distance with the real location, we get k–1 dummy locations. Our scheme can generate semantically similar dummy locations, which provides better location protection. Evaluation results show that our method can significantly improve the privacy level in terms of entropy.

Edge computing is an ideal platform for pedestrian detection in smart city because of low latency and location awareness. In edge computing, data collected by IoT devices are processed on edge servers rather than being transported to cloud server. Compared with cloud computing, edge computing could avoid the possibility of pedestrians' privacy being leaked from cloud server or being stolen in the process of transmission. However, edge servers are not always safe. For instance, there are researches show that 89% of WiFi hotspots are unsecured. Hence, it is possible for attackers to know where you go at a given time of the day, which places you prefer to visit from images collected by IoT devices, such as camera, UAVs. Considering the data collected by IoT devices could include the sensitive information about users, we propose a scheme that applies differential privacy to protect the collected data. We experiment on the INRIA Person Dataset and use three deep learning networks. Results show that even though adding differential privacy makes images blurred, the deep learning network on edge servers can detect pedestrians in the images with accuracy as high as 97.3%.

Traditional customer churn is predicted by machine learning and data mining methods. The advantages of big data are not fully utilized. In this paper, we use deep learning based customer churn analysis to establish a predictive model of customer churn, to achieve a warning real time. We use stacked autoencoder network to extract features from the data and then use logistic regression (LR) to classify customers. We first pretrain stacked autoencoder network, which is a deep learning model that uses the greedy layerwise unsupervised learning algorithm to train. After pretraining each layer separately, we will stack the each layer to form stacked autoencoder network, using backpropagation (BP) algorithm to reverse tuning parameters, and then train the logistic regression layer. From the perspective of overall accuracy, we use accuracy to evaluate the model, if the company pays more attention to all the churner to be predicted, we use the recall to evaluate the model, if the company pays more attention to the predicted accuracy of the churner, we can use the precision to evaluate the model. As to the customers who predict the loss, change the shortages in the operation timely, and reduce the loss of customers.

With the development of data science and deep learning, people tend to share their data to boost the precision of the trained model. However, the data to be shared may contain a very large number of private information, which may harm user privacy. Therefore, in this paper, based on n-gram model, a differentially private trajectory data publishing framework is proposed. To mitigate the impact of noise on the utility, we novelly propose a sparse vector technique based tree pruning method to generate an adaptive n-gram model which can adapt to height-unlimited tree. We also combine n-gram model and start-end distribution to solve the intrinsic problem of existing methods based on Markov model. Finally, we use empirical experiments to verify that our method could synthesize more similar trajectories to the original dataset than the state-of-the-art method.

Aspect-level sentiment analysis aims to provide complete and detailed view of sentiment analysis from different aspects. Existing solutions usually adopt a two-staged approach: first detecting aspect category in a document, then categorizing the polarity of opinion expressions for detected aspect(s). Inevitably, such methods lead to error accumulation. Moreover, aspect detection and aspect-level sentiment classification are highly correlated with each other. The key issue here is how to perform aspect detection and aspect-level sentiment classification jointly, and effectively. In this paper, we propose the aspect-level sentiment capsules model (AS-Capsules), which is capable of performing aspect detection and sentiment classification simultaneously, in a joint manner. AS-Capsules utilizes the correlation between aspect and sentiment through shared components including capsule embedding, shared encoders, and shared attentions. AS-Capsules is also capable of communicating with different capsules through a shared Recurrent Neural Network (RNN). More importantly, AS-Capsules model does not require any linguistic knowledge as additional input. Instead, through the attention mechanism, this model is able to attend aspect related words and sentiment words corresponding to different aspect(s). Experiments show that the AS-Capsules model achieves state-of-the-art performances on a benchmark dataset for aspect-level sentiment analysis.

There is a resurgent interest in developing intelligent open-domain dialog systems due to the availability of large amounts of conversational data and the recent progress on neural approaches to conversational AI [33]. Unlike traditional task-oriented bots, an open-domain dialog system aims to establish long-term connections with users by satisfying the human need for communication, affection, and social belonging. This article reviews the recent work on neural approaches that are devoted to addressing three challenges in developing such systems: semantics , consistency , and interactiveness . Semantics requires a dialog system to not only understand the content of the dialog but also identify users’ emotional and social needs during the conversation. Consistency requires the system to demonstrate a consistent personality to win users’ trust and gain their long-term confidence. Interactiveness refers to the system’s ability to generate interpersonal responses to achieve particular social goals such as entertainment and conforming. The studies we select to present in this survey are based on our unique views and are by no means complete. Nevertheless, we hope that the discussion will inspire new research in developing more intelligent open-domain dialog systems.

The large volume of event information makes it difficult for users to find interesting events in social networks. Therefore, we would like to develop an intelligent event recommendation to reduce information overload. Specifically, by exploring the behavior of users during the selection process, we are able to find particular rules associated with various event attributes which reflect the willingness of users. However, traditional event recommendations in social networks mainly concern the basic items like time and location. It is noted that few studies have yielded specific aspects such as the influence and spread capability of events and hosts. In this paper, we propose an event recommender approach fusing multiple features that can provide users with customized contents. To be specific, we consider hybridizing features including event influence, host impact, fee, social relationship and spatiotemporal characteristics. In order to achieve better performance, we concern the match degree between user and event properties especially in terms of their content and impact. Based on the improved idea of RankNet with neural networks, we build a Learning to Rank algorithm to reveal the importance of each feature. We rectify the problem of data sparse and cold start to grasp the balance of accuracy and novelty. Extensive experiments on datasets demonstrate that our method achieves promising results in comparison with other schemes.

A verification code is an automated test method used to distinguish between humans and computers. Humans can easily identify verification codes, whereas machines cannot. With the development of convolutional neural networks, automatically recognizing a verification code is now possible for machines. However, the advantages of convolutional neural networks depend on the data used by the training classifier, particularly the size of the training set. Therefore, identifying a verification code using a convolutional neural network is difficult when training data are insufficient. This study proposes an active and deep learning strategy to obtain new training data on a special verification code set without manual intervention. A feature learning model for a scene with less training data is presented in this work, and the verification code is identified by the designed convolutional neural network. Experiments show that the method can considerably improve the recognition accuracy of a neural network when the amount of initial training data is small.

Currently, the technique choices of issuing digital fiat currencies include RSCoin, Corda and Quorum etc. RSCoin is specially designed for central banks based on Bitcoin. Corda and Quorum are permissioned distributed ledger techniques based on Bitcoin and Ethereum, respectively. There lacks a framework specially for central banks based on Ethereum. We here introduce AFCoin: a framework based on the account model and smart contract of Ethereum. AFCoin shows a possible way to issue and manage fiat currencies by central banks. It can be deployed in an evolutionary way and enjoys good efficiency, regulation and privacy properties.

Existing neural methods for data-to-text generation are still struggling to produce long and diverse texts: they are insufficient to model input data dynamically during generation, to capture inter-sentence coherence, or to generate diversified expressions. To address these issues, we propose a Planning-based Hierarchical Variational Model (PHVM). Our model first plans a sequence of groups (each group is a subset of input items to be covered by a sentence) and then realizes each sentence conditioned on the planning result and the previously generated context, thereby decomposing long text generation into dependent sentence generation sub-tasks. To capture expression diversity, we devise a hierarchical latent structure where a global planning latent variable models the diversity of reasonable planning and a sequence of local latent variables controls sentence realization. Experiments show that our model outperforms state-of-the-art baselines in long and diverse text generation., Comment: To appear in EMNLP 2019

Components symmetry commonly exists in complex systems, which usually leads to considerable redundant computations in reliability analysis. Mining and making use of symmetry information can improve computational efficiency and reduce computational cost that is related to reliability analysis of complex systems. However, few literatures provide attention to this issue in reliability area. This paper proposes a graph learning based method to measure the similarity among the components’ local topological structure in a system. Based on the learned structural role similarity, components are clustered into different groups to reduce the complexity of the system. Application of the proposed method is presented in reducing computation effort of system survival signature, and a nearly 77% decrease of computation times demonstrates the effectiveness of the proposed method.

Along with wide application of sensors, multi-dimensional time-series data are commonly available for remaining useful life (RUL) estimation. This paper proposes a joint data-driven approach that adapts two models, AdaBoost regression and Long Short-Term Memory (LSTM), to estimate the RUL based on data trajectory extension. In RUL prediction, the data trajectories in the training set contain the data up to the units’ failure while the data trajectories in the testing set do not. Although this fact has a significant negative effect on the accuracy of RUL estimation, it is considered by few literatures. The proposed approach adapts the LSTM to learn the time series dependencies of training data and then extend the trajectories of testing data, aiming at reducing the variance of the lengths of data trajectory between the training and testing sets. Then, the proposed approach adapts the AdaBoost regression to estimate the RUL using the extended time series data. The proposed approach is competitive with state-of-the-art methods by demonstrating on two degradation datasets.

In vehicular ad hoc networks (VANETs), when a vehicle receives a message, the certificate revocation list (CRL) checking process will operate before certificate and signature verification. However, large communication sources, storage space, and checking time are needed for CRLs that cause the privacy disclosure issue as well. To address these issues, in this paper, we propose an efficient anonymous batch authentication scheme (ABAH) to replace the CRL checking process by calculating the hash message authentication code (HMAC). In our scheme, we first divide the precinct into several domains, in which road-side units (RSUs) manage vehicles in a localized manner. Then, we adopt pseudonyms to achieve privacy-preserving and realize batch authentication by using an identity-based signature (IBS). Finally, we use HMAC to avoid the time-consuming CRL checking and to ensure the integrity of messages that may get loss in previous batch authentication. The security and performance analysis are carried out to demonstrate that ABAH is more efficient in terms of verification delay than the conventional authentication methods employing CRLs. Meanwhile, our solution can keep conditional privacy in VANETs.

A software process model automatically recommendation framework is proposed.Different machine learning technologies are used to construct the recommendation models.Mutual impact analyses between process models and different project factors are performed.We found process models are responsible for defect count, defect severity and software change. Among many factors that influence the success of a software project, the software process model employed is an essential one. An improper process model will be time consuming, error-prone and cost expensive, and further lower the quality of software. Therefore, how to choose an appropriate software process model is a very important problem for software development. Current works focus on the selection criteria and often lead to subjective results. In this paper, we propose a software process model recommendation method, to help project managers choose the most appropriate software process model for a new project at an early stage of development process according to historical software engineering data. The proposed method casts the process model recommendation into a classification problem. It first evaluates the different combinations of the alternative classification and attribute selection algorithms, and the best one is used to build the recommendation model with historical software engineering data; then, the constructed recommendation model is used to predict process models for a new software project with only a few data. We also analyze the mutual impacts between process models and different types of project factors, to further help managers locate the most suitable process model. We found process models are also responsible for defect count, defect severity and software change. Experiments on the data sets from 37 different development teams of different countries show that the average recommendation accuracy of our method reaches up to 82.5%, which makes it potentially useful in practice.

The performance of software defect prediction have been hindered by the imbalanced nature of software defect data. Fortunately, a variety of sampling methods have been employed to improve defect prediction performance. However, researchers and practitioners are usually burdened with selecting the optimal sampling methods for the defect data at hand. In practice, no sampling method has been found to perform best in theory and practice. Therefore it is necessary and valuable to study how to select applicable sampling methods according to the current data characteristics. This paper presents a collaborative filtering based sampling methods recommendation algorithm (CFSR) for automatically recommending applicable sampling methods for the new defect data. CFSR firstly ranks existing sampling methods with historical defect data, and then mines the data similarity between the new and historical defect data with meta-features. Finally, all the information of ranked sampling methods and data similarity are combined to build a recommendation network, with which the user-based collaborative filtering algorithm is employed to recommend appropriate sampling methods for the new defect data. A thorough experiment with five classification algorithms, two prediction performance, five recommendation performance and 12 popular sampling methods was conducted over 20 imbalanced software defect data. The experimental results firstly demonstrate the importance and necessity of present study, and then show that the proposed CFSR method is feasible and effective.

Ranking is a fundamental and widely studied problem in scenarios such as search, advertising, and recommendation. However, joint optimization for multi-scenario ranking, which aims to improve the overall performance of several ranking strategies in different scenarios, is rather untouched. Separately optimizing each individual strategy has two limitations. The first one is lack of collaboration between scenarios meaning that each strategy maximizes its own objective but ignores the goals of other strategies, leading to a sub-optimal overall performance. The second limitation is the inability of modeling the correlation between scenarios meaning that independent optimization in one scenario only uses its own user data but ignores the context in other scenarios. In this paper, we formulate multi-scenario ranking as a fully cooperative, partially observable, multi-agent sequential decision problem. We propose a novel model named Multi-Agent Recurrent Deterministic Policy Gradient (MA-RDPG) which has a communication component for passing messages, several private actors (agents) for making actions for ranking, and a centralized critic for evaluating the overall performance of the co-working actors. Each scenario is treated as an agent (actor). Agents collaborate with each other by sharing a global action-value function (the critic) and passing messages that encodes historical information across scenarios. The model is evaluated with online settings on a large E-commerce platform. Results show that the proposed model exhibits significant improvements against baselines in terms of the overall performance., WWW2018

Topic structure analysis plays a pivotal role in dialogue understanding. We propose a reinforcement learning (RL) method for topic segmentation and labeling in goal-oriented dialogues, which aims to detect topic boundaries among dialogue utterances and assign topic labels to the utterances. We address three common issues in the goal-oriented customer service dialogues: informality, local topic continuity, and global topic structure. We explore the task in a weakly supervised setting and formulate it as a sequential decision problem. The proposed method consists of a state representation network to address the informality issue, and a policy network with rewards to model local topic continuity and global topic structure. To train the two networks and offer a warm-start to the policy, we firstly use some keywords to annotate the data automatically. We then pre-train the networks on noisy data. Henceforth, the method continues to refine the data labels using the current policy to learn better state representations on the refined data for obtaining a better policy. Results demonstrate that this weakly supervised method obtains substantial improvements over state-of-the-art baselines.

Commonsense knowledge is vital to many natural language processing tasks. In this paper, we present a novel open-domain conversation generation model to demonstrate how large-scale commonsense knowledge can facilitate language understanding and generation. Given a user post, the model retrieves relevant knowledge graphs from a knowledge base and then encodes the graphs with a static graph attention mechanism, which augments the semantic information of the post and thus supports better understanding of the post. Then, during word generation, the model attentively reads the retrieved knowledge graphs and the knowledge triples within each graph to facilitate better generation through a dynamic graph attention mechanism. This is the first attempt that uses large-scale commonsense knowledge in conversation generation. Furthermore, unlike existing models that use knowledge triples (entities) separately and independently, our model treats each knowledge graph as a whole, which encodes more structured, connected semantic information in the graphs. Experiments show that the proposed model can generate more appropriate and informative responses than state-of-the-art baselines.

In this paper, we propose RNN-Capsule, a capsule model based on Recurrent Neural Network (RNN) for sentiment analysis. For a given problem, one capsule is built for each sentiment category e.g., 'positive' and 'negative'. Each capsule has an attribute, a state, and three modules: representation module, probability module, and reconstruction module. The attribute of a capsule is the assigned sentiment category. Given an instance encoded in hidden vectors by a typical RNN, the representation module builds capsule representation by the attention mechanism. Based on capsule representation, the probability module computes the capsule's state probability. A capsule's state is active if its state probability is the largest among all capsules for the given instance, and inactive otherwise. On two benchmark datasets (i.e., Movie Review and Stanford Sentiment Treebank) and one proprietary dataset (i.e., Hospital Feedback), we show that RNN-Capsule achieves state-of-the-art performance on sentiment classification. More importantly, without using any linguistic knowledge, RNN-Capsule is capable of outputting words with sentiment tendencies reflecting capsules' attributes. The words well reflect the domain specificity of the dataset.

Recommender systems (RSs) make recommendation for users by capturing their preference from historical behaviors. However, emphasizing too much on the perspective of prediction precision leads to the problem of preference overfitting, which blockades the user from touching a wide range of new but potentially interesting items. Usually, it would be beneficial for the system to attempt suitable extensions beyond users' existing preferences already expressed in the recommender system. Traditional recommender systems concentrate much more on what a user had explored than what she/he is yet to explore. Too much attention to the precision of recommendation can lead to a preference overfitting problem. Users have the need for certain amount of extension based on the existing preference. In this paper, we investigate how novel items should be recommended given the user's preference which is modeled from her/his historical behaviors. Inspired by Wundt Curve, a famous law in cognition psychology field, we propose an idea of Personal Exploration Preference (PEP) model to capture the user preference from another perspective, the strength of aspiration to explore. Then we integrate PEP model into traditional recommendation algorithms. Finally,we experimentally evaluate the improvement of PEP model to traditional algorithms on HetRec2011 dataset. Experiment results show that, on the Top-N recommendation task, PEP model can effectively capture user preference of exploration and improve recommendation quality significantly.

Mobile healthcare social networks (MHSNs) have emerged as a promising next-generation healthcare system, which will significantly improve the quality of life. However, there are many security and privacy concerns before personal health information (PHI) is shared with other parities. To ensure patients’ full control over their PHI, we propose a fine-grained and scalable data access control scheme based on attribute-based encryption (ABE). Besides, policies themselves for PHI sharing may be sensitive and may reveal information about underlying PHI or about data owners or recipients. In our scheme, we let each attribute contain an attribute name and its value and adopt the Bloom filter to efficiently check attributes before decryption. Thus, the data privacy and policy privacy can be preserved in our proposed scheme. Moreover, considering the fact that the computational cost grows with the complexity of the access policy and the limitation of the resource and energy in a smart phone, we outsource ABE decryption to the cloud while preventing the cloud from learning anything about the content and access policy. The security and performance analysis is carried out to demonstrate that our proposed scheme can achieve fine-grained access policies for PHI sharing in MHSNs.

Location-based services (LBSs) have been one of the novel uses and most popular activities in internet of things (IoT). In such location-based applications, mobile users enjoy plenty of conveniences at the cost of privacy. To protect user's location privacy, many research solutions have been proposed. In this paper, we focus on an important class of solutions, short-range communication-based spatial cloaking algorithms, which achieve k -anonymity within some collaborative groups. We first analyze the inherent drawbacks of existing P2P-based and encounter-based spatial cloaking approaches and propose a Variance-Based Attack (VBA) against them. Then we study the proposed attack on several existing spatial cloaking solutions. Finally, we propose a countermeasure R -cloak, which can mitigate VBA for current P2P cloaking algorithms. Our empirical evaluations further verify the effectiveness and efficiency of R -cloak. We design a variance-based attack to exploit weaknesses of some existing algorithms.We verify the effectiveness of our VBA on several existing algorithms.We propose a random walk-based cloaking algorithm to mitigate the proposed attack.

Picking up appropriate classification algorithms for a given data set is very important and useful in practice. One of the most challenging issues for algorithm selection is how to characterize different data sets. Recently, we extracted the structural information of a data set to characterize itself. Although these kinds of characteristics work well in identifying similar data sets and recommending appropriate classification algorithms, the extraction method can only be applied to binary data sets and its performance is not high. Thus, in this paper, an improved data set characterization method is proposed to address these problems. For the purpose of evaluating the effectiveness of the improved method on algorithm recommendation, the unsupervised learning method EM is employed to build the algorithm recommendation model. Extensive experiments with 17 different types of classification algorithms are conducted upon 84 public UCI data sets; the results demonstrate the effectiveness of the proposed method.

The class imbalance problems have been reported to severely hinder classification performance of many standard learning algorithms, and have attracted a great deal of attention from researchers of different fields. Therefore, a number of methods, such as sampling methods, cost-sensitive learning methods, and bagging and boosting based ensemble methods, have been proposed to solve these problems. However, these conventional class imbalance handling methods might suffer from the loss of potentially useful information, unexpected mistakes or increasing the likelihood of overfitting because they may alter the original data distribution. Thus we propose a novel ensemble method, which firstly converts an imbalanced data set into multiple balanced ones and then builds a number of classifiers on these multiple data with a specific classification algorithm. Finally, the classification results of these classifiers for new data are combined by a specific ensemble rule. In the empirical study, different class imbalance data handling methods including three conventional sampling methods, one cost-sensitive learning method, six Bagging and Boosting based ensemble methods, our previous method EM1vs1 and two fuzzy-rule based classification methods were compared with our method. The experimental results on 46 imbalanced data sets show that our proposed method is usually superior to the conventional imbalance data handling methods when solving the highly imbalanced problems. HighlightsWe propose a novel ensemble method to handle imbalanced binary data.The method turns imbalanced data learning into multiple balanced data learning.Our method usually performs better than the conventional methods on imbalanced data.

Statement frequency data can inform programming language research and provide a solid basis for frequency-based code analysis. This paper presents an analysis of programming language statement frequency in a large corpus of C, C++, and Java source code, comprised of more than 54 million lines of code. Across these languages, the top four work-performing statement types are Method/Function Call, Assignment, If, and Return. As compared to studies of Formula Translating System, Common Business Oriented Language and Programming Language One in the 1970s, the main change is the prevalence of method/function calls. Statement use frequency across languages is remarkably similar, and within each individual language, most statement types have a frequency distribution that occupies a small range. A more detailed examination of assignment and looping statement types shows that many assignments simply involve copying of data and that C++/Java use for statements more than C. Copyright © 2014 John Wiley & Sons, Ltd.

In 1989, Tompa and Woll first introduced a cheating problem in Shamir's (k, n) secret sharing scheme, such that malicious shareholders (cheaters) aim to cheat honest shareholders by pooling forged shares during secret reconstruction. They succeed if honest shareholders accept a forged secret. In this cheating, even a single cheater can deceive other k – 1 honest shareholders with high probability. In this paper, we consider a situation that there are k -–2 or less cheaters in secret reconstruction, and propose a (k, n) secret sharing scheme secure against such cheating. Our scheme is efficient respect to the size of shares. Furthermore, our scheme is secure regardless of the probability distribution of the secret.

Identifying bug immediately when it is introduced can help improve the validity and effectiveness of bug fixing. Predicting bugs in software code changes makes such identification possible. Buggy changes, changes that introduce bugs into source code, can be viewed as anomalies relative to clean changes for that they are rare and irregular. Thus, anomaly detection techniques can be applied to buggy change prediction. Isolation Forest, which detects anomalies based on the hypothesis that the anomalies have the shortest average path length on the constructed random forest, has exhibited its good performance on anomaly detection compared to other anomaly detection methods. In this paper, we adopt it in predicting bugs in software code changes. Empirical study with eight practical open source projects are conducted to validate the effective of Isolation Forest in bug prediction in software code changes. Results of the empirical study show that compared to traditional classification methods used in literature, Isolation Forest can achieve better clean precision, buggy recall, buggy F-measure, AUC and Gmean.

This paper deals with sentence-level sentiment classification. Though a variety of neural network models have been proposed recently, however, previous models either depend on expensive phrase-level annotation, most of which has remarkably degraded performance when trained with only sentence-level annotation; or do not fully employ linguistic resources (e.g., sentiment lexicons, negation words, intensity words). In this paper, we propose simple models trained with sentence-level annotation, but also attempt to model the linguistic role of sentiment lexicons, negation words, and intensity words. Results show that our models are able to capture the linguistic role of sentiment words, negation words, and intensity words in sentiment expression.

Concept drifts usually originate from many causes instead of only one, which result in two types of concept drifts: abrupt drifts and gradual drifts. From the point of view of speed, concept drifts pose strong challenges for data stream mining. In this paper, we propose a selective detector ensemble to detect both abrupt and gradual drifts. We first present our detector ensemble construction method, and then introduce how to use this ensemble to detect concept drifts with the proposed early-findearly-report rule.To evaluate the performance of our method, we compare it with four drift detection methods on eight publicly available data sets containing various concept drifts. The experimental results show that compared with those benchmarks, our ensemble method can effectively improve the recall and false negative rate without significantly increasing the false positive rate, and has stronger generalization ability than those single-change-indicator-based methods.

In this paper, we present an efficient privacy-preserving authentication scheme based on group signature for vehicular ad hoc networks (VANETs). Although group signature is widely used in VANETs to realize anonymous authentication, the existing schemes based on group signatures suffer from long computation delay in the certificate revocation list (CRL) checking and in the signature verification process, leading to high message loss. As a result, they cannot meet the requirement of verifying hundreds of messages per second in VANETs. In our scheme, we first divide the precinct into several domains, in which roadside units (RSUs) are responsible for distributing group private keys and managing vehicles in a localized manner. Then, we use a hash message authentication code (HMAC) to avoid time-consuming CRL checking and to ensure the integrity of messages before batch group authentication. Finally, we adopt cooperative message authentication among entities, in which each vehicle only needs to verify a small number of messages, thus greatly alleviating the authentication burden. The security and performance analysis show that our scheme is more efficient in terms of authentication speed, while keeping conditional privacy in VANETs.

Security and privacy issues in RFID technology gain tremendous popularity recently. However, existing work on RFID authentication problems always make assumptions such as (1) hash function can be fully employed in designing RFID protocols; (2) channels between readers and server are always secure. The first assumption is not suitable for EPC Class-1 Gen-2 tags, which has been challenged in many research work, while the second one cannot be directly adopted in mobile RFID applications where wireless channels between readers and server are always insecure. To solve these problems, in this paper, we propose a novel ultralightweight and privacy-preserving authentication protocol for mobile RFID systems. We only use bitwise XOR, and several special constructed pseudo-random number generators to achieve our aims in the insecure mobile RFID environment. We use GNY logic to prove the security correctness of our proposed protocol. The security and privacy analysis show that our protocol can provide several privacy properties and avoid suffering from a number of attacks, including tag anonymity, tag location privacy, reader privacy, forward secrecy, and mutual authentication, replay attack, desynchronization attack etc. We implement our protocol and compare several parameters with existing work, the evaluation results indicate us that our protocol significantly improves the system performance.