Search

In this paper, we tackle the problem of enhancing the interpretability of the results of Cluster Analysis. Our goal is to find an explanation for each cluster, such that clusters are characterized as precisely and distinctively as possible, i.e., the explanation is fulfilled by as many as possible individuals of the corresponding cluster, true positive cases, and by as few as possible individuals in the remaining clusters, false positive cases. We assume that a dissimilarity between the individuals is given, and propose distance-based explanations, namely those defined by individuals that are close to its so-called prototype. To find the set of prototypes, we address the biobjective optimization problem that maximizes the total number of true positive cases across all clusters and minimizes the total number of false positive cases, while controlling the true positive rate as well as the false positive rate in each cluster. We develop two mathematical optimization models, inspired by classic Location Analysis problems, that differ in the way individuals are allocated to prototypes. We illustrate the explanations provided by these models and their accuracy in both real-life data as well as simulated data.

Receiver Operating Characteristic curves have been widely used to represent the performance of diagnostic tests. The corresponding area under the curve, widely used to evaluate their performance quantitatively, has been criticized in several respects. Several proposals have been introduced to improve area under the curve by taking into account only specific regions of the Receiver Operating Characteristic space, that is, the plane to which Receiver Operating Characteristic curves belong. For instance, a region of interest can be delimited by setting specific thresholds for the true positive rate or the false positive rate. Different ways of setting the borders of the region of interest may result in completely different, even opposing, evaluations. In this paper, we present a method to define a region of interest in a rigorous and objective way, and compute a partial area under the curve that can be used to evaluate the performance of diagnostic tests. The method was originally conceived in the Software Engineering domain to evaluate the performance of methods that estimate the defectiveness of software modules. We compare this method with previous proposals. Our method allows the definition of regions of interest by setting acceptability thresholds on any kind of performance metric, and not just false positive rate and true positive rate: for instance, the region of interest can be determined by imposing that [Formula: see text] (also known as the Matthews Correlation Coefficient) is above a given threshold. We also show how to delimit the region of interest corresponding to acceptable costs, whenever the individual cost of false positives and false negatives is known. Finally, we demonstrate the effectiveness of the method by applying it to the Wisconsin Breast Cancer Data. We provide Python and R packages supporting the presented method.

This paper proposes a practical physical tampering detection mechanism using inexpensive commercial off-the-shelf (COTS) Wi-Fi endpoint devices with a deep neural network (DNN) on channel state information (CSI) in the Wi-Fi signals. Attributed to the DNN that identifies physical tampering events due to the multi-subcarrier characteristics in CSI, our methodology takes effect using only one COTS Wi-Fi endpoint with a single embedded antenna to detect changes in the relative orientation between the Wi-Fi infrastructure and the endpoint, in contrast to previous sophisticated, proprietary approaches. Preliminary results show that our detectors manage to achieve a 95.89% true positive rate (TPR) with no worse than a 4.12% false positive rate (FPR) in detecting physical tampering events.

Intelligent lameness detection is important for improving cow welfare. A method based on YOLOv3 deep learning algorithm and relative step size characteristic vector is proposed to classify lame and non-lame cows. Videos were decomposed into sequence frames, and leg targets of cows in each frame were detected by YOLOv3 algorithm. Relative step sizes of cow's front and rear legs were calculated based on leg coordinates, and the relative step size characteristic vector was constructed. Finally, a trained Long Short-Term Memory (LSTM) classification model was used to classify lame and non-lame cows based on the characteristic vector. A total of 210 videos were selected for verification using LSTM, support vector machine (SVM), K-Nearest Neighbour (KNN) and decision tree classifier (DTC) algorithms. Results showed that accuracy of lameness detection based on LSTM was 98.57%, which was 2.93%, 3.88%, and 9.25% higher than SVM, KNN, and DTC, respectively. True positive rate of the LSTM was 0.97, which was 0.03, 0.04 and 0.06 higher than SVM, KNN and DTC, respectively. False positive rate of LSTM was 0.03, which was 0.03, 0.06 and 0.11 lower than SVM, KNN and DTC, respectively. A bidirectional LSTM performed slightly better than LSTM but would be more demanding on hardware. Comparison of LSTM with a purely deep learning method showed the latter performed slightly better, but was less conducive to interpretation and diagnosis of lameness. The relative step size characteristic vector proposed was effective for classification of lame and non-lame cows, and could lead to intelligent detection of lameness.

We study the problem of automatically detecting if a given multi-class classifier operates outside of its specifications (out-of-specs), i.e. on input data from a different distribution than what it was trained for. This is an important problem to solve on the road towards creating reliable computer vision systems for real-world applications, because the quality of a classifier’s predictions cannot be guaranteed if it operates out-of-specs. Previously proposed methods for out-of-specs detection make decisions on the level of single inputs. This, however, is insufficient to achieve low false positive rate and high false negative rates at the same time. In this work, we describe a new procedure named KS(conf), based on statistical reasoning. Its main component is a classical Kolmogorov–Smirnov test that is applied to the set of predicted confidence values for batches of samples. Working with batches instead of single samples allows increasing the true positive rate without negatively affecting the false positive rate, thereby overcoming a crucial limitation of single sample tests. We show by extensive experiments using a variety of convolutional network architectures and datasets that KS(conf) reliably detects out-of-specs situations even under conditions where other tests fail. It furthermore has a number of properties that make it an excellent candidate for practical deployment: it is easy to implement, adds almost no overhead to the system, works with any classifier that outputs confidence scores, and requires no a priori knowledge about how the data distribution could change.

The adaptive measurement of change (AMC) refers to the use of computerized adaptive testing (CAT) at multiple occasions to efficiently assess a respondent’s improvement, decline, or sameness from occasion to occasion. Whereas previous AMC research focused on administering the most informative item to a respondent at each stage of testing, the current research proposes the use of Fisher information per time unit as an item selection procedure for AMC. The latter procedure incorporates not only the amount of information provided by a given item but also the expected amount of time required to complete it. In a simulation study, the use of Fisher information per time unit item selection resulted in a lower false positive rate in the majority of conditions studied, and a higher true positive rate in all conditions studied, compared to item selection via Fisher information without accounting for the expected time taken. Future directions of research are suggested.

A Bloom filter is a special case of an artificial neural network with two layers. Traditionally, it is seen as a simple data structure supporting membership queries on a set. The standard Bloom filter does not support the delete operation, and therefore, many applications use a counting Bloom filter to enable deletion. This paper proposes a generalization of the counting Bloom filter approach, called “autoscaling Bloom filters”, which allows adjustment of its capacity with probabilistic bounds on false positives and true positives. Thus, by relaxing the requirement on perfect true positive rate, the proposed autoscaling Bloom filter addresses the major difficulty of Bloom filters with respect to their scalability. In essence, the autoscaling Bloom filter is a binarized counting Bloom filter with an adjustable binarization threshold. We present the mathematical analysis of its performance and provide a procedure for minimizing its false positive rate., Neural Computing and Applications, 32 (8), ISSN:1433-3058, ISSN:0941-0643

Mobile ads are plagued with fraudulent clicks which is a major challenge for the advertising community. Although popular ad networks use many techniques to detect click fraud, they do not protect the client from possible collusion between publishers and ad networks. In addition, ad networks are not able to monitor the user’s activity for click fraud detection once they are redirected to the advertising site after clicking the ad. We propose a new crowdsource-based system called Click Fraud Crowdsourcing (CFC) that collaborates with both advertisers and ad networks in order to protect both parties from any possible click fraudulent acts. The system benefits from both a global view, where it gathers multiple ad requests corresponding to different ad network-publisher-advertiser combinations, and a local view, where it is able to track the users’ engagement in each advertising website. The results demonstrated that our approach offers a lower false positive rate (0.1) when detecting click fraud as opposed to proposed solutions in the literature, while maintaining a high true positive rate (0.9). Furthermore, we propose a new mobile ad charging model that benefits from our system to charge advertisers based on the duration spent in the advertiser’s website.

Biomarkers abound in many areas of clinical research, and often investigators are interested in combining them for diagnosis, prognosis, or screening. In many applications, the true positive rate for a biomarker combination at a prespecified, clinically acceptable false positive rate is the most relevant measure of predictive capacity. We propose a distribution-free method for constructing biomarker combinations by maximizing the true positive rate while constraining the false positive rate. Theoretical results demonstrate desirable properties of biomarker combinations produced by the new method. In simulations, the biomarker combination provided by our method demonstrated improved operating characteristics in a variety of scenarios when compared with alternative methods for constructing biomarker combinations., 37 pages (including appendices)

The main goal of this article is experimental research on machine learning methodology for cyberattacks diagnosing. This study based on two publicly available datasets UNSW-NB15 and NSL-KDD. Its scope included the features reduction problem and calculations of the classification efficiency. We applied Machine Learning (ML) and Deep Learning (DL) methods to classify traffic. The methods of supervised k-nearest neighbours (k-NN) and artificial neural networks (ANN) were used. Accuracy, Precision, True Positive Rate (TPR), False Positive Rate (FPR) were calculated based on a series of numerical experiments for all types of attacks and for DoS (Deny of Service) attacks only. The features number reduction for observation is achieved, and the effect of this reduction on classification accuracy was investigated. We made some conclusions about the possibility of implementing ML/DL methods in intrusion detection systems (IDS).

Classification of functional metagenomes from the microbial community plays the vital role in the metagenomics research. In this paper, an investigation was made to study the performance of beta-t random forest classifier for classification of metagenomics data. Nine key functional meta-genomic variables were selected using the beta-t test statistic from the 10 different microbial community using p-value at 5% level of significance. Then beta-t random forest classifier showed the higher accuracy (96%), true positive rate (96%) and lower false positive rate (5%), false discovery rate (5%) and misclassification error rate (5%) for classification of metagenomes. This method showed the better performance compare to Bayes, SVM, KNN, AdaBoost and LogitBoost).

An ROC curve describes the relationship between the sensitivity and specificity of a test by plotting the two against one another while varying the CV. It is helpful when the outcome of a diagnostic test is continuous or ordinal. The key to an effective diagnostic test is to accurately classify 2 distinct populations into their respective groups of diseased versus nondiseased. Choosing the optimum CV is a tradeoff between the sensitivity (true positive rate) and the false positive rate. ROC curves are an important tool in evaluating the shape of uncertainty and are a valuable method in characterizing the strengths and weaknesses of diagnostic tests. The AUC provides a single quantitative, index measure for assessing the performance of a diagnostic test. They also provide an intuitive, qualitative assessment of the interrelated dynamics influencing decisions behind diagnostic tests. Their effectiveness spans multiple fields of study and their presence in modern diagnostic analysis is likely only to grow.

The existing anti-phishing approaches use the blacklist methods or features based machine learning techniques. Blacklist methods fail to detect new phishing attacks and produce high false positive rate. Moreover, existing machine learning based methods extract features from the third party, search engine, etc. Therefore, they are complicated, slow in nature, and not fit for the real-time environment. To solve this problem, this paper presents a machine learning based novel anti-phishing approach that extracts the features from client side only. We have examined the various attributes of the phishing and legitimate websites in depth and identified nineteen outstanding features to distinguish phishing websites from legitimate ones. These nineteen features are extracted from the URL and source code of the website and do not depend on any third party, which makes the proposed approach fast, reliable, and intelligent. Compared to other methods, the proposed approach has relatively high accuracy in detection of phishing websites as it achieved 99.39% true positive rate and 99.09% of overall detection accuracy.

Objective: To study the detection of malicious web page in mobile environment by lowering the false positive rate (FPR) and False Negative Rate (FNR) in real time and also how this CMW blocks the access of malicious webpages to the user are to be studied. Methods/Statistical Analysis: Now a days, Mobile devices are progressively being used to access the webpages. Content, Layout size, Functionaity have commonly been used to perform the static analysis to check the maliciousness in desktop space. In this paper we have presented a methodology named as CMW (Classification of malicious webpages) which detects the malicious mobile webpages in mobile environment. Here we used static features of mobile webpages derived from the HTML and Javascript content, URL and leading mobile specific capabilities. We then collected over 3500 mobile benign and malicious webpages. Findings: We have extracted a feature set consists of 44 features. 11 of which are not previously identified nor used. we then used a binomial classification technique to build a model for CMW to provide 90% accuracy and 89% true positive rate. It also detects a number of malicious webpages which are not accurately detected by existing techniques such as VirusTotal and Google Safe Browsing. Application/Improvements: We have used 11 new features in our feature set. Due to these new features, the detection of malicious webpages rate will be increased and false positive and false negative rates are reduced. Keywords: Classification, Features, Machine Learning, Malicious Web Pages, Static Detection

Inspired by the human immune system, we explore the development of a new Multiple-Detector Set Artificial Immune System (mAIS) for the detection of mobile malware based on the information flows in Android apps. mAISs differ from conventional AISs in that multiple-detector sets are evolved concurrently via negative selection. Typically, the first detector set is composed of detectors that match information flows associated with malicious apps while the second detector set is composed of detectors that match the information flows associated with benign apps. The mAIS presented in this paper incorporates feature selection along with a negative selection technique known as the split detector method (SDM). This new mAIS has been compared with a variety of conventional AISs and mAISs using a dataset of information flows captured from malicious and benign Android applications. This approach achieved a 93.33% accuracy with a true positive rate of 86.67% and a false positive rate of 0.00%.

Interruption detection is the procedure of assault distinguishing proof in the PC frameworks and it clears path for the recognizable proof of entrances, breakings, and other PC-related maltreatment. However, the development of the web-based gadgets makes the discovery procedure a confused strategy, representing the requirement for the robotized framework to recognize the assaults. In view of this, the paper proposes method of intrusion detection using the Novel Brainstorm-Crow Search-based Adaptive Actor Critic Neural Network. Clustering is the way toward making a gathering of conceptual objects into classes of comparative items. The clusters are subjected to the different-advance arrangement that is advanced utilizing the proposed enhancement calculation, and in the second dimension of characterization, the interruption in the information is distinguished. The experimentation of the proposed strategy utilizing the KDD cup dataset yields a precision of 0.69, True Positive Rate of 0.68, and False Positive Rate of 0.55.