Search

Sequence learning approaches require careful tuning of parameters for their success. Pre-trained sequence models exhibit a superior performance compared to the sequence models that are randomly initialized. This work presents a sequence autoencoder based pre-trained decoder approach for sequence learning. This approach is applicable for the tasks in which the output is a sequence. In the proposed method, a SAE (Sequence Auto Encoder) is trained with an objective of reconstructing the input sequence. The weights of the pre-trained SAE are then used to initialize the decoder in the sequence model developed based on the encoder–decoder paradigm. The proposed pre-trained decoder-based approach achieves superior performance as compared to the pre-trained encoder-based approach and the pre-trained encoder- and decoder-based approach. The behavior of the suggested approach is examined using unsupervised pre-training. The proposed method is evaluated for neural machine translation and image caption generation tasks. Outcomes of the experimental studies on benchmark datasets indicate the effectiveness of the proposed approach.

Grounded in a hierarchical model of intrinsic and extrinsic motivation (HMIEM), the primary aim of this study was to test a full motivational sequence at the contextual level in a high school ensemble setting (Social-Contextual Factors → Psychological Needs → Motivation → Consequences). I specifically examined the relationships between multifaceted variables within this sequence, including teacher-created social contexts, psychosocial needs, types of motivation, and consequences. A secondary purpose of this study was to determine whether gender would impact the results of the sequence of motivational processes. Structural equation modeling analysis with a sample of 425 high school ensemble students revealed that social-contextual factors provided by the teachers were related to satisfaction of fundamental autonomy, competence, and relatedness needs, which in turn influenced intrinsic motivation, positive motivational outcomes, and persistence in musical activities. The multistep invariance analysis also revealed the model to be invariant for males and females. The results of the study supported the HMIEM and validated the application of the motivational sequence in the context of music education.

Experimental procedures for chemical synthesis are commonly reported in prose in patents or in the scientific literature. The extraction of the details necessary to reproduce and validate a synthesis in a chemical laboratory is often a tedious task requiring extensive human intervention. We present a method to convert unstructured experimental procedures written in English to structured synthetic steps (action sequences) reflecting all the operations needed to successfully conduct the corresponding chemical reactions. To achieve this, we design a set of synthesis actions with predefined properties and a deep-learning sequence to sequence model based on the transformer architecture to convert experimental procedures to action sequences. The model is pretrained on vast amounts of data generated automatically with a custom rule-based natural language processing approach and refined on manually annotated samples. Predictions on our test set result in a perfect (100%) match of the action sequence for 60.8% of sentences, a 90% match for 71.3% of sentences, and a 75% match for 82.4% of sentences., Extracting experimental operations for chemical synthesis from procedures reported in prose is a tedious task. Here the authors develop a deep-learning model based on the transformer architecture to translate experimental procedures from the field of organic chemistry into synthesis actions.

Traditionally, researchers have focused on network level intrusion detection and program level intrusion detection to improve computer security. However, neither approach is foolproof. We argue that the internal and external security of a computer system are equally important. Typically, a successful attacker manifests in the form of the attacker becoming a user on the host either with elevated or normal user privileges. At this point, user-level intrusion detection attempts to deter and curtail an attacker even after the system has been compromised. In this work, we introduce a new approach of intrusion detection based on recurrent neural networks (RNNs) to solve the long sequential problem. We build a robust user command sequence-to-sequence model by semantic modeling command. Our model implements the prediction of user command sequence and the prophesying of user behaviors. The experimental results on data sets of Purdue University, SEA and self-collected data show that an accurate, effective and efficient detection can be achieved by using the proposed approach.

Human–computer conversation is an active research topic in natural language processing. One of the representative methods to build conversation systems uses the sequence-to-sequence (Seq2seq) model through neural networks. However, with limited input information, the Seq2seq model tends to generate meaningless and trivial responses. It can be greatly enhanced if more supplementary information is provided in the generation process. In this work, we propose to utilize retrieved responses to boost the Seq2seq model for generating more informative replies. Our method, called ReBoost, incorporates retrieved results in the Seq2seq model by a hierarchical structure. The input message and retrieved results can influence the generation process jointly. Experiments on two benchmark datasets demonstrate that our model is able to generate more informative responses in both automatic and human evaluations and outperforms the state-of-the-art response generation models.

Question and Answering (QA) system is a problem in natural language processing that can be used as the system of dialogs and chatbots. It can be used as a customer service that can provide a response to the customer quickly. A QA system receives an input in the form of sentences and produces the predictive sentences that are responses to the input. Therefore, a model that can learn such conversations is needed. This research focuses on developing a chatbot based on a sequence-to-sequence model. It is trained using a data set of conversation from a university admission. Evaluation on a small dataset obtained from the Telkom University admission on Whatsapp instant messaging application shows that the model produces a quite high BLEU score of 41.04. An attention mechanism technique using the reversed sentences improves the model to gives a higher BLEU up to 44.68.

Traditional document summarization models cannot handle dialogue summarization tasks perfectly. In situations with multiple speakers and complex personal pronouns referential relationships in the conversation. The predicted summaries of these models are always full of personal pronoun confusion. In this paper, we propose a hierarchical transformer-based model for dialogue summarization. It encodes dialogues from words to utterances and distinguishes the relationships between speakers and their corresponding personal pronouns clearly. In such a from-coarse-to-fine procedure, our model can generate summaries more accurately and relieve the confusion of personal pronouns. Experiments are based on a dialogue summarization dataset SAMsum, and the results show that the proposed model achieved a comparable result against other strong baselines. Empirical experiments have shown that our method can relieve the confusion of personal pronouns in predicted summaries.

There are different text summarization process available in Natural Language Processing. Among them abstractive text summarization is one of the challenging problems in natural language processing. Abstractive text summarization contains a short and concise summary of a large text document built from the underlying message of the text. The objective of the proposed system is to create a short and understandable abstractive summary of a malayalam text document. A sequence to sequence model is used to create the summary of the document. In this work, the goal was to increase the efficiency and reduce the training loss of a sequence to sequence model thereby implementing a better abstractive text summarizer for a malayalam document.

End-to-end approaches for sequence tasks are becoming increasingly popular. Yet for complex sequence tasks, like speech translation, systems that cascade several models trained on sub-tasks have shown to be superior, suggesting that the compositionality of cascaded systems simplifies learning and enables sophisticated search capabilities. In this work, we present an end-to-end framework that exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. These hidden intermediates can be improved using beam search to enhance the overall performance and can also incorporate external models at intermediate stages of the network to re-score or adapt towards out-of-domain data. One instance of the proposed framework is a Multi-Decoder model for speech translation that extracts the searchable hidden intermediates from a speech recognition sub-task. The model demonstrates the aforementioned benefits and outperforms the previous state-of-the-art by around +6 and +3 BLEU on the two test sets of Fisher-CallHome and by around +3 and +4 BLEU on the English-German and English-French test sets of MuST-C., NAACL 2021. All code and models are released as part of the ESPnet toolkit: https://github.com/espnet/espnet

Recently, blockchain technology has been one of the most promising fields of research aiming to enhance the security and privacy of systems. It follows a distributed mechanism to make the storage system fault-tolerant. However, even after adopting all the security measures, there are some risks for cyberattacks in the blockchain. From a statistical point of view, attacks can be compared to anomalous transactions compared to normal transactions. In this paper, these anomalous transactions can be detected using machine learning algorithms, thus making the framework much more secure. Several machine learning algorithms can detect anomalous observations. Due to the typical nature of the transactions dataset (time-series), we choose to apply a sequence to the sequence model. In this paper, we present our approach, where we use federated learning embedded with an LSTM-based auto-encoder to detect anomalous transactions.

Load forecasting plays a critical part in grid operation and planning. In particular, the importance of multistep load forecasting for individual power customer is increasingly prominent. Due to the strong volatility of individual consumers’ electricity consumption behavior, traditional machine learning methods that cannot capture time dependence are difficult to obtain good prediction results. The recurrent neural network (RNN) can capture the time correlations existing in the load data, and the sequence to sequence (Seq2Seq) model combining two RNNs of the encoder and decoder is very suitable for multistep prediction. The temporal pattern attention mechanism can further capture the periodic change pattern in historical load data, which further improves time series modeling. We combined their advantages to propose a new type of multistep individual load forecasting framework, called the temporal pattern attention based sequence to sequence (TPA-Seq2Seq) model. This model can overcome the difficulty of multi-step prediction and further capture the load change pattern. The proposed framework was tested on real residential smart meter data, the results show that the proposed model has good prediction accuracy and is well suited for longer prediction sequences.

To be able to profit from natural language processing (NLP) tools for analysing historical text, an important step is spelling normalisation. We first compare and second combine two different approaches: on the one hand VARD, a rule-based system which is based on dictionary lookup and rules with non-probabilistic but trainable weights; on the other hand a language-independent approach to spelling normalisation based on statistical machine translation (SMT) techniques. The rule-based system reaches the best accuracy, up to 94% precision at 74% recall, while the SMT system improves each tested period. We obtain the best system by combining both approaches. Re-training VARD on specific time-periods and domains is beneficial, and both systems benefit from a language sequence model using collocation strength.

As the most widely used crop-specific land use data, the Cropland Data Layer (CDL) product covers the entire Contiguous United States (CONUS) at 30-meter spatial resolution with very high accuracy up to 95% for major crop types (i.e., Corn, Soybean) in major crop area. However, the quality of early-year CDL products were not as good as the recent ones. There are many erroneous pixels in the early-year CDL product due to the cloud cover of the original Landsat images, which affect many follow-on researches and applications. To address this issue, we explore the feasibility of using machine learning technology to refine and correct misclassified pixels in the historical CDLs in this study. An end-to-end deep learning-based framework for restoration of misclassified pixels in CDL image is developed and tested. By feeding the CDL time series into the artificial neural network, a crop sequence model is trained and the misclassified pixels in an original CDL map can be restored. In the experiment with the 2005 CDL data of the State of Illinois, the misclassified pixels over Agricultural Statistics Districts (ASD) #1760 were corrected with a reasonable accuracy (> 85%). The findings suggest that the proposed method provides a low-cost and reliable way to refine the historical CDL data, which can be potentially scaled up to the entire CONUS.

This work proposes the use of a pretrained sequence-to-sequence model for document ranking. Our approach is fundamentally different from a commonly adopted classification-based formulation based on encoder-only pretrained transformer architectures such as BERT. We show how a sequence-to-sequence model can be trained to generate relevance labels as “target tokens”, and how the underlying logits of these target tokens can be interpreted as relevance probabilities for ranking. Experimental results on the MS MARCO passage ranking task show that our ranking approach is superior to strong encoder-only models. On three other document retrieval test collections, we demonstrate a zero-shot transfer-based approach that outperforms previous state-of-the-art models requiring in-domain cross-validation. Furthermore, we find that our approach significantly outperforms an encoder-only architecture in a data-poor setting. We investigate this observation in more detail by varying target tokens to probe the model’s use of latent knowledge. Surprisingly, we find that the choice of target tokens impacts effectiveness, even for words that are closely related semantically. This finding sheds some light on why our sequence-to-sequence formulation for document ranking is effective. Code and models are available at pygaggle.ai.

Accurate indoor temperature forecasting can facilitate energy savings of the building without compromising the occupant comfort level, by providing more accurate control of the HVAC (heating, ventilating, and air conditioning) system. In order to make the best use of different input variables, a long short-term memory (LSTM) based sequence to sequence (seq2seq) model was proposed to make multi-step ahead forecasting. The out-of-sample forecasting capacity of the model was evaluated with regard to different forecast horizons by various evaluation metrics. A tailor-made metric was proposed to take account of the small daily-variation characteristic of indoor temperature. The model was benchmarked against Prophet and a seasonal naive model, showing that the current model is much more skillful and reliable in very short-term forecasting. A cross-series learning strategy was adopted to enable multi-zone indoor temperature forecasting with a more generalised model. Furthermore, the uncertainty in model parameters was quantified by prediction intervals created by Monte-Carlo dropout (MC-dropout) technique.

Multi-label text classification is an important yet challenging task in natural language processing. It is more complex than single-label text classification in that the labels tend to be correlated. To capture this complex correlations, sequence to sequence model has been widely applied, and achieved impressing performance for multi-label text classification. It encodes each document as contextual representations, and then decodes them to generate labels one by one. At each time step, the decoder usually adopts the attention mechanism to highlight important contextual representations to predict a related label, which has been proved to be effective. Nevertheless, the traditional attention approaches only utilize a hidden state to explore such contextual representations, which may result in prediction errors, or omit several trivial labels. To tackle this problem, in this paper, we propose “history-based attention”, which takes history information into consideration, to effectively explore informative representations for labels’ predictions in multi-label text classification. Our approach consists of two parts: history-based context attention and history-based label attention. History-based context attention considers historical weight trends to highlight important context words, which is helpful to predict trivial labels. History-based label attention explores historical labels to alleviate the error propagation problem. We conduct experiments on two popular text datasets (i.e., Arxiv Academic Paper Dataset and Reuters Corpus Volume I), it is demonstrated that the history-based attention mechanism could boost the performance to a certain extent, and the proposed method consistently outperforms highly competitive approaches.

It is expensive to collect trajectory data on a mobile phone by continuously pinpointing its location, which limits the application of trajectory data mining (e.g., trajectory prediction). In this poster, we propose a method for trajectory prediction by collecting cell-id trajectory data without explicit locations. First, it exploits the spatial correlation between cell towers based on graph embedding technique. Second, it employs the sequence-to-sequence (seq2seq) framework to train the prediction model by designing a novel spatial loss function. Experiment results based on real datasets have demonstrated the effectiveness of the proposed method.

Multi-step passenger demand forecasting is a crucial task in on-demand vehicle sharing services. However, predicting passenger demand over multiple time horizons is generally challenging due to the nonlinear and dynamic spatial-temporal dependencies. In this work, we propose to model multi-step citywide passenger demand prediction based on a graph and use a hierarchical graph convolutional structure to capture both spatial and temporal correlations simultaneously. Our model consists of three parts: 1) a long-term encoder to encode historical passenger demands; 2) a short-term encoder to derive the next-step prediction for generating multi-step prediction; 3) an attention-based output module to model the dynamic temporal and channel-wise information. Experiments on three real-world datasets show that our model consistently outperforms many baseline methods and state-of-the-art models., Comment: 7 pages

Named Entity Recognition (NER) is a basic task in Natural Language Processing (NLP). Recently, the sequence-to-sequence (seq2seq) model has been widely used in NLP task. Different from the general NLP task, 60% sentences in the NER task do not contain entities. Traditional seq2seq method cannot address this issue effectively. To solve the aforementioned problem, we propose a novel seq2seq model, named SC-NER, for NER task. We construct a classifier between the encoder and decoder. In particular, the classifier’s input is the last hidden state of the encoder. Moreover, we present the restricted beam search to improve the performance of the proposed SC-NER. To evaluate our proposed model, we construct the patent documents corpus in the communications field, and conduct experiments on it. Experimental results show that our SC-NER model achieves better performance than other baseline methods.

In this paper we describe our system for morphological analysis and lemmatization in context, using a transformer-based sequence to sequence model and a biaffine attention based BiLSTM model. First, a lemma is produced for a given word, and then both the lemma and the given word are used for morphological analysis. We also make use of character level word encodings and trainable encodings to improve accuracy. Overall, our system ranked fifth in lemmatization and sixth in morphological accuracy among twelve systems, and demonstrated considerable improvements over the baseline in morphological analysis.

In recent years, the rapid growth of motor vehicle ownership brings great pressure to the road traffic system and inevitably leads to a large number of traffic accidents. Therefore, it is a demanding task to build a well-developed system to identify the high-risk links, i.e., black spots, of a road network. However, most of the existing works focus on identifying black spots in a road network simply based on the statistic data of accidents, which leads to low accuracy. In this demonstration, we present a novel system called BSI, to predict and analyze the high-risk links in a road network by adequately utilizing the spatial-temporal features of accidents. First, BSI predicts the trend of accidents by a spatial-temporal sequence model. Then, based on predicted results, K-means method is utilized to discover the roads with the highest accident severity. Finally, BSI identifies the central location and coverage of a high-risk link by a modified DBSCAN clustering model. BSI can visualize the final identified black spots and provide the results to the user.

This research aims in detecting violent crowd flows in the context of Bangladesh. For this purpose, we have collected a dataset which includes both violent and non-violent crowd flows. Different deep learning algorithms and approaches have been applied on this dataset to detect scenarios which contain violence. Convolutional neural networks (CNN) and long short-term memory network (LSTM) based architectures have been experimented separately on this dataset and in combination as well. Moreover, a model that was already pre-trained on violent movie scenes has been used to leverage transfer learning which outperformed all other experimented approaches with an accuracy of 95.67%. Surprisingly, the sequence model alone or in combination with CNN has not performed well on this particular dataset. The proposed model is lightweight hence it can be deployed easily in any security systems consisting of CCTV cameras or unmanned aerial vehicles (UAVs).

In this work, we examine whether it is possible to achieve the state of the art performance in paraphrase generation with reduced vocabulary. Our approach consists of building a convolution to sequence model (Conv2Seq) partially guided by the reinforcement learning, and training it on the subword representation of the input. The experiment on the Quora dataset, which contains over 140,000 pairs of sentences and corresponding paraphrases, found that with less than 1,000 token types, we were able to achieve performance which exceeded that of the current state of the art.

Patients with similar conditions in the intensive care unit (ICU) may have different reactions for a given treatment. An effective personalised medicine can help save patient lives. The availability of recorded ICU data provides a huge potential to train and develop the systems. However, there is no ground truth of best treatments. This makes existing supervised learning based methods are not appropriate. In this paper, we proposed clustering based Bayesian reinforcement learning. Firstly, we transformed the multivariate time series patient record into a real-time Patient Sequence Model (PSM). After that, we computed the likelihood probability of treatments effect for all patients and cluster them based on that. Finally, we computed Bayesian reinforcement learning to derive personalised policies. We tested our proposed method using 11,791 ICU patients records from MIMIC-III database. Results show that we are able to cluster patient based on their treatment effects. In addition, our method also provides better explainability and time-critical recommendation that are very important in a real ICU setting.

Active learning is very suitable for many problems in natural language processing, where unlabeled data may be abundant, but annotation is slow and expensive. This article aims to illustrate some active learning methods for handwritten digit recognition tasks, such as the least confidence and entropy methods. We investigated the previously used sequence model query selection strategies and used some selection strategies for sample labeling in handwritten digit recognition. We also conduct a large-scale empirical comparison of using multiple corpora, which shows that our proposed method improves the technical level.

We present a machine learning approach to short tandem repeat (STR) sequence detection and extraction from massively parallel sequencing data called Fragsifier. Using this approach, STRs are detected on each read by first locating the longest repeat stretches followed by locus prediction using k-mers in a machine learning sequence model. This is followed by reference flanking sequence alignment to determine precise STR boundaries. We show that Fragsifier produces genotypes that are concordant with profiles obtained using capillary electrophoresis (CE), and also compared the results with that of STRait Razor and the ForenSeq UAS. The data pre-processing and training of the sequence classifier is readily scripted, allowing the analyst to experiment with different thresholds, datasets and loci of interest, and different machine learning models.

Previous work approaches the SQL-to-text generation task using vanilla Seq2Seq models, which may not fully capture the inherent graph-structured information in SQL query. In this paper, we first introduce a strategy to represent the SQL query as a directed graph and then employ a graph-to-sequence model to encode the global structure information into node embeddings. This model can effectively learn the correlation between the SQL query pattern and its interpretation. Experimental results on the WikiSQL dataset and Stackoverflow dataset show that our model significantly outperforms the Seq2Seq and Tree2Seq baselines, achieving the state-of-the-art performance., EMNLP18, Accepted

Getting a better understanding of user behavior is important for advancing information retrieval systems. Existing work focuses on modeling and predicting single interaction events, such as clicks. In this paper, we for the first time focus on modeling and predicting sequences of interaction events. And in particular, sequences of clicks. We formulate the problem of click sequence prediction and propose a click sequence model (CSM) that aims to predict the order in which a user will interact with search engine results. CSM is based on a neural network that follows the encoder-decoder architecture. The encoder computes contextual embeddings of the results. The decoder predicts the sequence of positions of the clicked results. It uses an attention mechanism to extract necessary information about the results at each timestep. We optimize the parameters of CSM by maximizing the likelihood of observed click sequences. We test the effectiveness of CSM on three new tasks: (i) predicting click sequences, (ii) predicting the number of clicks, and (iii) predicting whether or not a user will interact with the results in the order these results are presented on a search engine result page (SERP). Also, we show that CSM achieves state-of-the-art results on a standard click prediction task, where the goal is to predict an unordered set of results a user will click on.

In this paper, we propose a deep learning approach to tackle the automatic summarization tasks by incorporating topic information into the convolutional sequence-to-sequence (ConvS2S) model and using self-critical sequence training (SCST) for optimization. Through jointly attending to topics and word-level alignment, our approach can improve coherence, diversity, and informativeness of generated summaries via a biased probability generation mechanism. On the other hand, reinforcement training, like SCST, directly optimizes the proposed model with respect to the non-differentiable metric ROUGE, which also avoids the exposure bias during inference. We carry out the experimental evaluation with state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets. The empirical results demonstrate the superiority of our proposed method in the abstractive summarization., International Joint Conference on Artificial Intelligence and European Conference on Artificial Intelligence (IJCAI-ECAI), 2018

We propose to achieve explainable neural machine translation (NMT) by changing the output representation to explain itself. We present a novel approach to NMT which generates the target sentence by monotonically walking through the source sentence. Word reordering is modeled by operations which allow setting markers in the target sentence and move a target-side write head between those markers. In contrast to many modern neural models, our system emits explicit word alignment information which is often crucial to practical machine translation as it improves explainability. Our technique can outperform a plain text system in terms of BLEU score under the recent Transformer architecture on Japanese-English and Portuguese-English, and is within 0.5 BLEU difference on Spanish-English., Comment: BlackboxNLP workshop at EMNLP 2018

In this paper we introduce our system participating at the 2017 SemEval shared task on keyphrase extraction from scientific documents. We aimed at the creation of a keyphrase extraction approach which relies on as little external resources as possible. Without applying any hand-crafted external resources, and only utilizing a transformed version of word embeddings trained at Wikipedia, our proposed system manages to perform among the best participating systems in terms of precision.

A description of a system for identifying Verbal Multi-Word Expressions (VMWEs) in running text is presented. The system mainly exploits universal syntactic dependency features through a Conditional Random Fields (CRF) sequence model. The system competed in the Closed Track at the PARSEME VMWE Shared Task 2017, ranking 2nd place in most languages on full VMWE-based evaluation and 1st in three languages on token-based evaluation. In addition, this paper presents an option to re-rank the 10 best CRF-predicted sequences via semantic vectors, boosting its scores above other systems in the competition. We also show that all systems in the competition would struggle to beat a simple lookup base-line system and argue for a more purpose-specific evaluation scheme.

The objective of this research is to discover the rule of gameplay related to the task interdependence to analyse the behavior pattern of social gameplay. Previous literatures related to the gameplay were reviewed and game which was suitable for the gameplay of the task interdependence was selected. A party-play includes a team of five people in the experiment during the gameplay with think-aloud method and video/audio data about action protocol and verbal report were collected. The video observation and protocol analysis were conducted to analyse data. The objective coding scheme were developed from consolidated sequence model task analysis. The player`s behavior was analysed. The result was revealed that four rules and four modified rules were included into the total eight behavior pattern. A behavior graph integrated with five gameplay was written. The excellent cooperative spot and error and failure place could be identified. The social gameplay behavior graph is expected to be the key practical design guideline on whether the level design and balance design are proper.

Among other factors, student behavior during learning activities is affected by the pedagogical content they are interacting with. In this paper, we analyze this effect in the context of a problem-solving based online Physics course. We use a representation of the content in terms of its position, composition and visual layout to identify eight content types that correspond to problem solving sub-tasks. Canonical examples as well as a sequence model of these tasks are presented. Student behaviors, measured in terms of activity, help-requests, mistakes and time on task, are compared across each content type. Students request more help while working through complex computational tasks and make more mistakes on tasks that apply conceptual knowledge. We discuss how these findings can inform the design of pedagogical content and authoring tools.

Bacterial lipoproteins have many important functions owing to their essential nature and roles in pathogenesis and represent a class of possible vaccine candidates. The prediction of bacterial lipoproteins from sequence is thus an important task for computational vaccinology. A Support Vector Machines (SVM) based module for predicting bacterial lipoproteins, LIPOPREDICT, has been developed. The best performing sequence model were generated using selected dipeptide composition, which gave 97% accuracy of prediction. The results obtained were compared very well with those of previously developed methods. Availability The database is available for free at www.lipopredict.cdac.in

Text-based language identification (T-LID) of isolated words has been shown to be useful for various speech processing tasks, including pronunciation modelling and data categorisation. When the words to be categorised are proper names, the task becomes more difficult: not only do proper names often have idiosyncratic spellings, they are also often considered to be multilingual. We, therefore, investigate how an existing T-LID technique can be adapted to perform multilingual word classification. That is, given a proper name, which may be either mono- or multilingual, we aim to determine how accurately we can predict how many possible source languages the word has, and what they are. Using a Joint Sequence Model-based approach to T-LID and the SADE corpus - a newly developed proper names corpus of South African names - we experiment with different approaches to multilingual T-LID. We compare posterior-based and likelihood-based methods and obtain promising results on a challenging task.

Bacterial lipoproteins have many important functions and represent a class of possible vaccine candidates. The prediction of lipoproteins from sequence is thus an important task for computational vaccinology. Naïve-Bayesian networks were trained to identify SpaseII cleavage sites and their preceding signal sequences using a set of 199 distinct lipoprotein sequences. A comprehensive range of sequence models was used to identify the best model for lipoprotein signal sequences. The best performing sequence model was found to be 10-residues in length, including the conserved cysteine lipid attachment site and the nine residues prior to it. The sensitivity of prediction for LipPred was 0.979, while the specificity was 0.742. Here, we describe LipPred, a web server for lipoprotein prediction; available at the URL: http://www.jenner.ac.uk/LipPred/. LipPred is the most accurate method available for the detection of SpaseIIcleaved lipoprotein signal sequences and the prediction of their cleavage sites.

This paper describes the DCU submission to WMT 2014 on German-English translation task. Our system uses phrasebased translation model with several popular techniques, including Lexicalized Reordering Model, Operation Sequence Model and Language Model interpolation. Our final submission is the result of system combination on several systems which have different pre-processing and alignments.

This paper describes the Yandex School of Data Analysis Russian-English system submitted to the ACL 2014 Ninth Workshop on Statistical Machine Translation shared translation task. We start with the system that we developed last year and investigate a few methods that were successful at the previous translation task including unpruned language model, operation sequence model and the new reparameterization of IBM Model 2. Next we propose a {simple yet practical} algorithm to transform Russian sentence into a more easily translatable form before decoding. The algorithm is based on the linguistic intuition of native Russian speakers, also fluent in English.

Understanding human activities is important for user-centric and context-aware applications. Previous studies showed promising results using various machine learning algorithms. However, most existing methods can only recognize the activities that were previously seen in the training data. In this paper, we present a new zero-shot learning framework for human activity recognition that can recognize an unseen new activity even when there are no training samples of that activity in the dataset. We propose a semantic attribute sequence model that takes into account both the hierarchical and sequential nature of activity data. Evaluation on datasets in two activity domains show that the proposed zero-shot learning approach achieves 70-75% precision and recall recognizing unseen new activities, and outperforms supervised learning with limited labeled data for the new classes.

Computer intrusions are taking place everywhere, and have become a major concern for information security. Most intrusions to a computer system may result from illegitimate or irregular calls to the operating system, so analyzing the system-call sequences becomes an important and fundamental technique to detect potential intrusions. This paper proposes two models based on data mining technology, respectively called frequency patterns (FP) and tree patterns (TP) for intrusion detection. FP employs a typical method of sequential mining based on frequency analysis, and uses a short sequence model to find out quickly frequent sequential patterns in the training system-call sequences. TP makes use of the technique of tree pattern mining, and can get a quality profile from the training system-call sequences of a given system. Experimental results show that FP has good performances in training and detecting intrusions from short system-call sequences, and TP can achieve a high detection precision in han...

[1] The hidden Markov model (HMM) and related algorithms provide a powerful framework for statistical inference on partially observed stochastic processes. HMMs have been successfully implemented in many disciplines, though not as widely applied as they should be in earthquake modeling. In this article, a simple HMM earthquake occurrence model is proposed. Its performance in declustering is compared with the epidemic-type aftershock sequence model, using a data set of the central and western regions of Japan. The earthquake clusters and the single earthquakes separated using our model show some interesting geophysical differences. In particular, the log-linear Gutenberg-Richter frequency-magnitude law (G-R law) for the earthquake clusters is significantly different from that for the single earthquakes.

We describe the LIU systems for English-German and German-English translation in the WMT09 shared task. We focus on two methods to improve the word alignment: (i) by applying Giza++ in a second phase to a reordered training corpus, where reordering is based on the alignments from the first phase, and (ii) by adding lexical data obtained as high-precision alignments from a different word aligner. These methods were studied in the context of a system that uses compound processing, a morphological sequence model for German, and a part-of-speech sequence model for English. Both methods gave some improvements to translation quality as measured by Bleu and Meteor scores, though not consistently. All systems used both out-of-domain and in-domain data as the mixed corpus had better scores in the baseline configuration.

In this article, compound processing for translation into German in a factored statistical MT system is investigated. Compounds are handled by splitting them prior to training, and merging the parts after translation. I have explored eight merging strategies using different combinations of external knowledge sources, such as word lists, and internal sources that are carried through the translation process, such as symbols or parts-of-speech. I show that for merging to be successful, some internal knowledge source is needed. I also show that an extra sequence model for part-of-speech is useful in order to improve the order of compound parts in the output. The best merging results are achieved by a matching scheme for part-of-speech tags.