Search

The widespread use of social media like Facebook, Twitter, Whatsapp, etc. has changed the way News is created and published; accessing news has become easy and inexpensive. However, the scale of usage and inability to moderate the content has made social media, a breeding ground for the circulation of fake news. Fake news is deliberately created either to increase the readership or disrupt the order in the society for political and commercial benefits. It is of paramount importance to identify and filter out fake news especially in democratic societies. Most existing methods for detecting fake news involve traditional supervised machine learning which has been quite ineffective. In this paper, we are analyzing word embedding features that can tell apart fake news from true news. We use the LIAR and ISOT data set. We churn out highly correlated news data from the entire data set by using cosine similarity and other such metrices, in order to distinguish their domains based on central topics. We then employ auto-encoders to detect and differentiate between true and fake news while also exploring their separability through network analysis.

Linguists have been focused on a qualitative comparison of the semantics from different languages. Evaluation of the semantic interpretation among disparate language pairs like English and Tamil is an even more formidable task than for Slavic languages. The concept of word embedding in Natural Language Processing (NLP) has enabled a felicitous opportunity to quantify linguistic semantics. Multi-lingual tasks can be performed by projecting the word embeddings of one language onto the semantic space of the other. This research presents a suite of data-efficient deep learning approaches to deduce the transfer function from the embedding space of English to that of Tamil, deploying three popular embedding algorithms: Word2Vec, GloVe and FastText. A novel evaluation paradigm was devised for the generation of embeddings to assess their effectiveness, using the original embeddings as ground truths. Transferability across other target languages of the proposed model was assessed via pre-trained Word2Vec embeddings from Hindi and Chinese languages. We empirically prove that with a bilingual dictionary of a thousand words and a corresponding small monolingual target (Tamil) corpus, useful embeddings can be generated by transfer learning from a well-trained source (English) embedding. Furthermore, we demonstrate the usability of generated target embeddings in a few NLP use-case tasks, such as text summarization, part-of-speech (POS) tagging, and bilingual dictionary induction (BDI), bearing in mind that those are not the only possible applications.

In this paper, a Deep learning-based approach is analyzed to improve the accuracy of the Language Modelling of a system that is a part of ASR. The main intent is to build an End to End Neural Network Language Model. Generally, ASR works in three different modes. The modes are Acoustic Modeling, Lexicons, and Language Modeling. In ASR, G2P models are vital in the case of analyzing the Out Of Vocabulary words. Mostly these G2P models were fabricated using the traditional approaches such as Bi-gram, n-gram language models. The traditional approaches are deceptive in aiding the meanings, long term dependencies of the words and are also impotent in rendering the phonemes of new words. Thus Seq2Seq G2P models exceed the state of the art of the traditional approach. As we all know, the ASR (Speech to Text) system is widely used in AI applications such as voice dialing, system control, navigation, etc. The method which is applied here is the Statistical based one. A method other than the Rule-based approach is trialed out. A suitable neural network of LSTM and BI-LSTM is proposed which enhances the accuracy of the system.

Accurate automatic Identification and localization of spine vertebrae points in CT scan images is crucial in medical diagnosis. This paper presents an automatic feature extraction network, based on transfer learned CNN, in order to handle the availability of limited samples. The 3D vertebrae centroids are identified and localized by an LSTM network, which is trained on CNN features extracted from 242 CT spine sequences. The model is further trained to estimate age and gender from LSTM features. Thus, we present a framework that serves as a multi-task data driven model for identifying and localizing spine vertebrae points, age estimation and gender classification. The proposed approach is compared with benchmark results obtained by testing 60 scans. The advantage of the multi-task framework is that it does not need any additional information other than the annotations on the spine images indicating the presence of vertebrae points.

The price of a single stock is seldom independent. It has been known to brokers and fund managers, that, they heavily influence each other. Portfolios are built, on the premise of minimizing such dependencies between stocks. There have been several efforts to quantify these dependencies, predominantly using conventional statistics and correlations. Most approaches use single independent variable-based approaches to compare two stock price signals at a time. In this paper, we analyze stock sectors with multiple stocks in each, presenting a multivariate case. We apply two multivariate analysis techniques, namely multivariate variational mode decomposition (MVMD) and spatio-temporal intrinsic mode decomposition (STIMD), to analyze stock sectors based on their individual stock day-wise price series. Sector-wise data is downloaded from Google Finance. Furthermore, we quantify the dependence of a sector on each of its constituent stock, by decomposing this multivariate signal and selectively reconstructing it multiple times, excluding each one from the reconstruction process. The error of reconstruction in each case serves as a measure of how much other stocks depend on the one that was excluded.

Multiple myeloma cancer is caused by the abnormal growth of plasma cells in the bone marrow. The most commonly used method for diagnosis of multiple myeloma is Bone marrow aspiration, where the aspirate slide images are either observed visually or passed onto existing digital image processing software for the detection of myeloma cells. The current work explores the effectiveness of deep learning based object detection/segmentation algorithms such as Mask-RCNN and unet for the detection of multiple myeloma. The manual polygon annotation of the current dataset is performed using VGG image annotation software. The deep learning models were trained by monitoring the train and validation loss per epoch and the best model was selected based on the minimal loss for the validation data. From the comparison results obtained for both the models, it is observed that Mask-RCNN has competing results than unet and it addresses most of the challenges existing in multiple myeloma segmentation.

Diabetic Retinopathy(DR) is one of the diseases, which is caused by damage to blood vessels of the light sensitive tissue of the eyes. DR is graded into five different levels; normal, mild, moderate, severe and proliferative. DR diagnosis demands more time on detection from fundus images. An accurate automatic model requires sufficient data for training which is unavailable. The open-source DR datasets are highly imbalanced between the levels of DR and it is also not easy to collect more data for proliferative cases. The synthetic generation of data for such highly imbalanced classes in a dataset provides better results on classification. In this paper, an analysis of classification results for the same is carried out with augmented proliferative class (highly imbalanced class) in the EYEPACS dataset using a generative adversarial network (GAN). We generated highly diverse images for proliferative cases without any constraints. The generated proliferative images do not influence other class images and have also improved the classification results obtained by the model, over that which was trained without synthetic generation. The results obtained before and after augmentation by the proposed generative based model is compared over various model attributes.

Streaming data refers to the data that is sent to a cloud or a processing centre in real time. Even though we have limited exposure to such applications that can process data streams on a live basis and generate useful insights, we are still in infancy when it comes to complicated stream processing. Current streaming data analytics tools represents the third-/fourth-generation data processing capability in the big data hierarchy which includes the Hadoop ecosystem, Apache Storm™ and Apache Kafka™ and its likes, Apache Spark™ framework, and now, Apache Flink™ with its non-batch stateful streaming core. Each of these individually cannot handle all the aspects of a data processing pipeline, alone. It is essential to have good synergy between these technologies to cater to the management, streaming, processing and fault-tolerant requirements of various data-driven applications. Companies tailor their pipelines exclusively for their requirements, since making a general framework entails mammoth interfacing and configuration efforts that are not cost-effective for them. In this paper, we envision and implement such a generalized minimal stream processing pipeline and measure its performance, on some data sets, in the form of delays and latencies of data arrival at pivotal checkpoints in the pipeline. We virtualize this using a Docker™ container without much loss in performance.

Defects in structures will affect its natural vibrations. With the advent of pure data-driven modeling techniques such as Dynamic Mode Decomposition (DMD), the defected modes can be separated from the normal modes by using vibration data from various points on the structural element. In this work we simulate the vibrations of a cantilever beam in Abaqus® without defect and with different defects. We apply DMD to compute the spatial modes of vibration in each of these cases. Furthermore we train a Support Vector Machine (SVM) classifier with the Eigen-modes we have computed, to identify defects. We also analyze this data visually using t-SNE plots.

The neural network, one of the intelligent data mining technique that has been used by researchers in various areas for the past 10 years. Prediction and analysis of stock market data have got an important role in today’s economy. The various algorithms used for forecasting can be categorized into linear (AR, MA, ARIMA, ARMA) and non-linear models (ARCH, GARCH, Neural Network). In this paper, we are using four types of deep learning architectures i.e Multilayer Perceptron (MLP), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) for predicting the stock price of a company based on the historical prices available. Here we are using day-wise closing price of two different stock markets, National Stock Exchange (NSE) of India and New York Stock Exchange (NYSE). The network was trained with the stock price of a single company from NSE and predicted for five different companies from both NSE and NYSE. It has been observed that CNN is outperforming the other models. The network was able to predict for NYSE even though it was trained with NSE data. This was possible because both the stock markets share some common inner dynamics. The results obtained were com- pared with ARIMA model and it has been observed that the neural networks are outperforming the existing linear model (ARIMA).

In this paper we propose a deep learning framework for classification of malware. There has been an enormous increase in the volume of malware generated lately which represents a genuine security danger to organizations and people. So as to battle the expansion of malwares, new strategies are needed to quickly identify and classify malware. Malimg dataset, a publicly available benchmark data set was used for the experimentation. The architecture used in this work is a hybrid cost-sensitive network of one-dimensional Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) network which obtained an accuracy of 94.4%, an increase in performance compared to work done by [1] which got 84.9%. Hyper parameter tuning is done on deep learning architecture to set the parameters. A learning rate of 0.01 was taken for all experiments. Train-test split of 70-30% was done during experimentation. This facilitates to find how well the models perform on imbalanced data sets. Usual methods like disassembly, decompiling, de-obfuscation or execution of the binary need not be done in this proposed method. The source code and the trained models are made publicly available for further research.

Real-time prediction of domain names that are generated using the Domain Generation Algorithms (DGAs) is a challenging cyber security task. Scope to collect the vast amount of data for training favored data-driven techniques and deep learning architectures have the potential to address this challenge. This paper proposes a deep learning framework using long short-term memory (LSTM) architecture for prediction of the domain names that are generated using the DGAs. Binary classification had benign and DGA domain names and multiclass classification was performed using 20 different DGAs. For the binary classification, LSTM model gave accuracy of 98.7% and 71.3% on two different test data sets and for the multi-class classification, it gave accuracy of 68.3% and 67.0% respectively. Two diversified data sets were used to analyze the robustness of the LSTM architecture.

Despite the growth of bilingual word embeddings, there is no work done so far, for directly evaluating them for English–Tamil language pair. In this paper, we present a data resource and evaluation for the English–Tamil bilingual word vector model. In this paper, we present dataset and the evaluation paradigm for English–Tamil bilingual language pair. This dataset contains words that covers a range of concepts that occur in natural language. The dataset is scored based on the similarity rather than association or relatedness. Hence, the word pairs that are associated but not literally similar have a low rating. The measures are quantified further to ensure consistency in the dataset, mimicking the cognitive phenomena. Henceforth, the dataset can be used by non-native speakers, with minimal effort. We also present some inferences and insights into the semantics captured by word vectors and human cognition.

Economy of a country is closely related to stock market. By analysing stock market performance, we can evaluate whether a country’s economic growth is increasing or decreasing. Even though country’s economic growth can be understood by predicting stock market, it is highly unpredictable. We used dynamic mode decomposition which is a spatio-temporal, equation-free, data-driven algorithm for stock market prediction Schmid (J Fluid Mech 656:5–28, [13]) by considering stock markets as a dynamical system. How the system evolves and prediction of future state is done using DMD by decomposing a spatio-temporal system to modes having predetermined temporal behaviour. We used this property of DMD to predict stock market behaviour. In Kuttichira et al. (2017 international conference on advances in computing, communications and informatics (ICACCI), IEEE, pp 55–60, [7]) DMD was used to predict Indian stock market for minutewise data. We used daywise data as our timescale. Time series data of stock price of companies listed in National Stock Exchange were used as data. Sampled daywise stock price of companies across sector was used to predict the stock price for next few days. Predicted prices were compared with original prices and mean absolute percentage error was used to calculate the deviation for every companies. We analysed the stock price prediction for both intra- and intersector companies. We used dynamic mode decomposition to predict the stock price using historical data. We also did fine tuning of sampling windows to find out the best parameters for our data set.

At present time, malware is one of the biggest threats to Internet service security. This chapter propose a novel file agnostic deep learning architecture for malware family identification which converts malware binaries into gray scale images and then identifies their families by a hybrid in-house model, Convolutional Neural Network and Long Short Term Memory (CNN-LSTM). The significance of the hybrid model enables the network to capture the spatial and temporal features which can be used effectively to distinguish among malwares. In this novel method, usual methods like disassembly, de-compiling, de-obfuscation or execution of the malware binary need not be done. Various experiments were run to identify an optimal deep learning network parameters and network structure on benchmark and well-known data set. All experiments were run at a learning rate 0.1 for 1,000 epochs. To select a model which is generalizable, various test-train splits were done during experimentation. Additionally. this facilitates to find how well the models perform on imbalanced data sets. Experimental results shows that the hybrid model is very effective for malware family classification in all the train-test splits. It indicates that the model can work in unevenly distributed samples too. The classification accuracy obtained by deep learning architectures on all train-test splits performed better than other compared classical machine learning algorithms and existing method based on deep learning. Finally, a scalable framework based on deep learning and visualization approach is proposed which can be used in real time for malware family identification.

The SCADA based monitoring systems, having a very low sampling of one reading per 2-4 seconds is known to produce roughly 4.3 Tera Bytes (TiBs) of data annually. With synchrophasor technology, this will go up at least 100 times more as the rate of streaming is as high as 50/100 (60/120) Hz. Phasor data concentrators (PDCs) transmit byte streams encapsulating a comprehensive list of power system parameter including multiple phasor measurements, instantaneous frequency estimates, rate of change of frequency and several analog and digital quantities; this high volume and velocity of data makes it truly ‘Big Data’. This helps in making the power grid a lot more observable, enabling real-time monitoring of crucial grid events such as voltage stability, grid stress and transient oscillations. Synchrophasor technology uses the IEEE C37.118.2-2011™ Phasor Measurement Unit (PMU) / PDC communication protocol for data exchange which has no direct interface with any contemporary big data stream APIs or protocols. In this paper we propose a streaming interface in Apache Spark™, a popular big data platform, using Scala programming language, implementing a complete IEEE C37.118.2-2011™ client inside a stream receiver so that we can effortlessly receive synchrophasor data directly to Spark™ applications for real-time processing and archiving.

Tree adjoining grammars (TAGs) are mildly context sensitive formalisms used mainly in modelling natural languages. Usage and research on these psycho linguistic formalisms have been erratic in the past decade, due to its demanding construction and difficulty to parse. However, they represent promising future for formalism based NLP in multilingual scenarios. In this paper we demonstrate basic synchronous Tree adjoining grammar for English-Tamil language pair that can be used readily for machine translation. We have also developed a multithreaded chart parser that gives ambiguous deep structures and a par dependency structure known as TAG derivation. Furthermore we then focus on a model for training this TAG for each language using a large corpus of text through a map reduce frequency count model in spark and estimation of various probabilistic parameters for the grammar trees thereafter; these parameters can be used to perform statistical parsing on the trained grammar.

Comparing the semantics of a pair of sentences has been an interesting yet unstructured problem. Semantic analysis is mostly elusive due to the fact that the semantics of Natural language constructs cannot be measured, let alone be compared to one another. Methods like Latent Semantic Analysis(LSA) and Latent Dichlaret Analysis(LDA) are able to capture broader semantics between documents, but their contribution in pairwise comparison tasks which require deeper semantics may be limited. In this paper we present a local alignment based scoring scheme for sentence pairs using word embeddings and how this can be used as a feature for some popular text analysis tasks such as summarization, paraphrase comparison, topic profiling and other semantic comparison tasks. We also present a theoretical analysis on the metrics used in this approach and a separability argument using t-SNE plots. Furthermore we detail our Spark implementation model for the pairwise comparison and summarization.

Tree adjoining grammars (TAGs) are mildly context-sensitive psycholinguistic formalisms that are hard to parse. All standard TAG parsers have a worst-case complexity of O(n6), despite being one of the most linguistically relevant grammars. For comprehensive syntax analysis, especially of ambiguous natural language constructs, most TAG parsers will have to run exhaustively, bringing them close to worst-case runtimes, in order to derive all possible parse trees. In this paper, we present a new and intuitive genetic algorithm, a few fitness functions and an implementation strategy for lexicalised-TAG parsing, so that we might get multiple ambiguous derivations efficiently.

In this paper, we present the results obtained by applying deep learning techniques to classification of network protocols and applications using flow features and data signatures. We also present a similar classification of malware using their binary files. We use our own dataset for traffic identification and Microsoft Kaggle dataset for malware classification tasks. The current techniques used in network traffic analysis and malware detection is time consuming and beatable as the precise signatures are known. Deep learned features in both cases are not hand crafted and are learned form data signatures. It cannot be understood by the attacker or the malware in order to fake or hide it and hence cannot be bypassed easily.

Background/Objectives: This paper is aimed at performing real time bigdata analytics on vehicular data collected from a network of ECUs (Electronic Control Unit) in cooperated into the different automobiles. Methods/Statistical Analysis: The analytics has been performed by building a software model that is capable of handling the vehicular data in real time. Bigdata platforms like hadoop, Apache Storm, Apache Spark(real time streaming), Kafka are utilised here. Automotive sensor data from different Electronic Control Units are collected into a central data server and this data is pushed to kafka, from which the real time streaming models pulls the data and perform analysis. Findings: Automotive industry has undergone a drastic revolutionised innovation in the past decade in all of its respective segments. The industry had started utilizing the computational and mathematical aspects from top to bottom in its design strategies to achieve greater reliability on its products out on roads. Latest advancements in this field is the fully autonomous car. Today an automotive is a collection of innumerable sensors and microcontrollers which are under the command of the master ECU. A network of ECUs connected across the globe is a source tap of bigdata. Leveraging the new sources of bigdata by automotive giants boost vehicle performance, enhance loco driver experience, accelerated product designs. Statistical Projections reveal that automotive industry is likely to be the 2nd largest generator of data by mid of 2016. The contribution of this paper to the automotive industry is the real time vehicle monitoring utilizing Bigdata platforms. This can contribute to better customer-industry relations. Applications/Improvements: The model developed in this paper can contribute a lot to the automobile industry as it facilitates real time monitoring of the vehicles. This can improve customer-industry relation.

Financial forecasting is a widely applied area, making use of statistical prediction using ARMA, ARIMA, ARCH and GARCH models on stock prices. Such data have unpredictable trends and non-stationary property which makes even the best long term predictions grossly inaccurate. The problem is countered by keeping the prediction shorter. These methods are based on time series models like auto regressions and moving averages, which require computationally costly recurring parameter estimations. When the data size becomes considerable, we need Big Data tools and techniques, which do not work well with time series computations. In this paper we discuss such a finance domain problem on the Indian National Stock Exchange (NSE) data for a period of one year. Our main objective is to device a light weight prediction for the bulk of companies with fair accuracy, useful enough for algorithmic trading. We present a minimal discussion on these classical models followed by our Spark RDD based implementation of the proposed fast forecast model and some results we have obtained.