Search

Internet video providers rely on alerting workflows to identify and remedy incidents that can impact users (e.g., outages or buggy players). There is growing evidence for the need for viewership-based analytics---detecting and diagnosing incidents that manifest through changes in viewership patterns but not in other (e.g., QoE) metrics. However, both detection and diagnosis of viewership anomalies is challenging due to the contextual nature of anomalies, non-stationarity of viewership, and complex dependencies between the structure of events and how they impact different subpopulations of viewers. We present Proteas, an alerting framework for video viewership anomalies that tackles these challenges. Proteas builds on key spatiotemporal structural insights. First, across different sub-populations of viewers and days of the week, we find that the shape of the viewership curve remains invariant over multiple weeks, thus enabling anomaly detection. Second, we use the hierarchy of viewership groups to produce compact alerts. Finally, we find that common anomalies manifest with spatiotemporal signatures, which enables us to classify anomalies to produce actionable alerts. We evaluate Proteas using 3 months of real viewership data (including the onset of the COVID-19 pandemic) and show that Proteas is accurate with over 80% True Positive Rate, average precision of over 86% (i.e., few false positives) and doesn't miss any major events. In addition, we find that approximately half of Proteas's alerts refer to events not caught by other alerting workflows, thus adding value to operators' existing toolkit.

As the semiconductor process technology advances into sub-10nm regime, cell pin accessibility, which is a complex joint effect from the pin shape and nearby blockages, becomes a main cause for DRC violations. Therefore, a machine learning model for DRC hotspot prediction needs to consider both very high-resolution pin shape patterns and low-resolution layout information as input features. A new convolutional neural network technique, J-Net, is introduced for the prediction with mixed resolution features. This is a customized architecture that is flexible for handling various input and output resolution requirements. It can be applied at placement stage without using global routing information. This technique is evaluated on 12 industrial designs at 7nm technology node. The results show that it can improve true positive rate by 37%, 40% and 14% respectively, compared to three recent works, with similar false positive rates.

Organization and government networks are a target of Advanced Persistent Threats (APTs), i.e., stealthy attackers that infiltrate networks slowly and usually stay undetected for long periods of time. After an attack has been discovered, security administrators have to manually determine which hosts were compromised to clean and restore them. For that, they have to analyze a large number of hosts. In this paper, we propose an approach to efficiently reconstruct the lateral movement of attackers from a given set of indicators of compromise (IoCs) that can help security administrators to identify and prioritize potentially compromised hosts. To reconstruct attacker paths in a network, we link hosts with IoCs via two methods: k-shortest-paths and biased random walks. To evaluate the accuracy of these approaches in reconstructing attack paths, we introduce three models of attackers that differ in their network knowledge. Our results indicate that we can approximate the lateral movement of the three proposed attacker models, even when the attacker significantly deviates from them. For insider attackers that deviate up to 75% from our models, the method based on k-shortest-paths achieves a true positive rate of 88% and can significantly narrow down the set of nodes to analyse to 5% of all network hosts.

We propose deep learning in word embedding for making a diagnostic prediction model. One factor that causes uncertainties in diagnostic is the inexperience of physicians. The diagnosis errors lead to incorrect and delay in treatment, waste of time and money. To solve the problem, a differential diagnosis is a critical tool. It is powerful and does not introduce additional work to physician. Our method applied a deep learning tool together with word embedding from existing diagnosis texts in medical system. The model takes the clinical notes from a physician. The note is then used to analyze the possibilities of diseases. The output is sorted by model confidence. We validate our model with True Positive Rate (Recall), False Positive Rate (Precision) and accuracy. Our model achieves a new record of accuracy at 99.95% The highest recall rate is at 86.64% in top first prediction.

The control of vehicles can be taken over by injecting malicious controller area network (CAN) messages. To detect malicious messages, anomaly-detection systems based on intervals or payloads of CAN messages have been proposed. However, these systems cannot accurately detect malicious messages injected into mixed CAN messages, which include periodic and sporadic transmissions. Moreover, sophisticated systems leveraging machine learning are not deployable because the computers in vehicles have limited computational resource. Therefore, we propose a lightweight system to detect malicious messages injected into mixed CAN messages. The proposed system extracts features essential for detecting such messages from CAN messages; thus it is deployable in vehicles. Specifically, we use the quantized intervals and the absolute difference of payloads. We collected 44 hours of running data from 4 types of vehicles and injected 7,788 malicious messages into 33 mixed and 8 periodic CAN messages. Our result shows that our system achieved the high detection performance: a true positive rate of 97.55% and a false positive rate of 0.003%.

A method for detection of changes in one-dimensional data based on a combination of a detector consisting of competitive filters and statistical tests is elaborated. Using synthetic data, without noise and with noise of various strengths, our method is compared with the V-Box method and the CUSUM chart. The efficiency of the algorithms is evaluated in terms of the true positive rate and the false positive rate. The performance of the proposed change detector is demonstrated to be reasonable. The method can be applied to any kind of one-dimensional data, especially to time series.

In a Website Fingerprinting (WF) attack, a local, passive eavesdropper utilizes network flow information to identify which web pages a user is browsing. Previous researchers have extensively demonstrated the feasibility and effectiveness of WF, but only under the strong Single Page Assumption: the network flow extracted by the adversary always belongs to a single page. In other words, the WF classifier will never be asked to classify a network flow corresponding to more than one page, or part of a page. The Single Page Assumption is unrealistic because people often browse with multiple tabs. When this happens, the network flow induced by multiple tabs will overlap, and current WF attacks fail to classify correctly. Our work demonstrates the feasibility of WF with the relaxed Single Page Assumption: we can attack a client who visits more than one pages simultaneously. We propose a multi-tab website fingerprinting attack that can accurately classify multi-tab web pages if they are requested and sequentially loaded over a short period of time. In particular, we develop a new BalanceCascade-XGBoost scheme for an attacker to identify the start point of the second page such that the attacker can accurately classify and identify these multi-tab pages. By developing a new classifier, we only use a small chunk of packets, i.e., packets between the first page's start time to the second page's start time, to fingerprint website. Our experiments demonstrate that in the multi-tab scenario, WF attacks are still practically effective. We have an average TPR of 92.58% on SSH, and we can also averagely identify the page with a TPR of 64.94% on Tor. Specially, compared with previous WF classifiers, our attack achieves a significantly higher true positive rate using a restricted chunk of packets.

Modern malware applies a rich arsenal of evasion techniques to render dynamic analysis ineffective. In turn, dynamic analysis tools take great pains to hide themselves from malware; typically this entails trying to be as faithful as possible to the behavior of a real run. We present a novel approach to malware analysis that turns this idea on its head, using an extreme abstraction of the operating system that intentionally strays from real behavior. The key insight is that the presence of malicious behavior is sufficient evidence of malicious intent, even if the path taken is not one that could occur during a real run of the sample. By exploring multiple paths in a system that only approximates the behavior of a real system, we can discover behavior that would often be hard to elicit otherwise. We aggregate features from multiple paths and use a funnel-like configuration of machine learning classifiers to achieve high accuracy without incurring too much of a performance penalty. We describe our system, TAMALES (The Abstract Malware Analysis LEarning System), in detail and present machine learning results using a 330K sample set showing an FPR (False Positive Rate) of 0.10% with a TPR (True Positive Rate) of 99.11%, demonstrating that extreme abstraction can be extraordinarily effective in providing data that allows a classifier to accurately detect malware.

The cognitive processes that underly expert decision making in medical image interpretation are crucial to the understanding of what constitutes optimal performance. Often, if an anomaly goes undetected, the exact nature of the false negative is not fully understood. This work looks at 24 experts' performance (true positives and false negatives) during an anomaly detection task for 13 images and the corresponding gaze behavior. By using a drawing and an eye-tracking experimental paradigm, we compared expert target anomaly detection in orthopantomographs (OPTs) against their own gaze behavior. We found there was a relationship between the number of anomalies detected and the anomalies looked at. However, roughly 70% of anomalies that were not explicitly marked in the drawing paradigm were looked at. Therefore, we looked how often an anomaly was glanced at. We found that when not explicitly marked, target anomalies were more often glanced at once or twice. In contrast, when targets were marked, the number of glances was higher. Furthermore, since this behavior was not similar over all images, we attribute these differences to image complexity.

Browsing product reviews is an important information resource for taking actions such as choosing and purchasing a product. However, there are a large number of reviews on electronic commerce sites; hence, it is necessary to efficiently judge the usefulness of reviews to obtain pertinent information. Conventional judging methods make the assumption that all reviews contain useful information. However, it is apparent that some reviews do not contain useful evaluation standpoints of consumers. That is, the conventional methods have the potential to not evaluate the usefulness of the reviews accurately.In this paper, we propose a method for judging the usefulness of reviews from the evaluation standpoints of consumers. To grasp the evaluation standpoints of consumers, we utilize both comprehensiveness and specificity (also known as true positive rate and positive predictive value, respectively), which can be estimated from contributory factors influencing the usefulness of reviews.

Cane-toads (Bufo marinus) are introduced to Australia to negate the insect pests. However, cane-toad population is out of control, and has highly affected native frog species for its great reproductive capacity. Therefore, it is necessary to monitor cane-toads and control their population. Recent advances in acoustic sensors make it possible to monitor cane-toads over long periods. In this paper, we present a novel approach to analyze cane-toad calls for long-term monitoring. Specifically, recorded acoustic data is first transformed into spectrogram using short-time Fourier transform. Then, cane-toad calls are recognized with an oscillation detector. Next, recognized cane-toad calls are annotated for further analysis. Initial analysis of our method demonstrates promising results, with 71.36% true positive rate. With the oscillation detector, cane-toad calling activity is investigated over the whole year.

Python is a popular dynamic language that allows quick software development. However, Python program analysis engines are largely lacking. In this paper, we present a Python predictive analysis. It first collects the trace of an execution, and then encodes the trace and unexecuted branches to symbolic constraints. Symbolic variables are introduced to denote input values, their dynamic types, and attribute sets, to reason about their variations. Solving the constraints identifies bugs and their triggering inputs. Our evaluation shows that the technique is highly effective in analyzing real-world complex programs with a lot of dynamic features and external library calls, due to its sophisticated encoding design based on traces. It identifies 46 bugs from 11 real-world projects, with 16 new bugs. All reported bugs are true positives.

Concurrency has been a perpetual problem in Android apps, mainly due to event-based races. Several event-based race detectors have been proposed, but they produce false positives, cannot reproduce races, and cannot distinguish be- tween benign and harmful races. To address these issues, we introduce a race verification and reproduction approach named ERVA. Given a race report produced by a race detector, ERVA uses event dependency graphs, event flipping, and replay to verify the race and determine whether it is a false positive, or a true positive; for true positives, ERVA uses state comparison to distinguish benign races from harmful races. ERVA automatically produces an event schedule that can be used to deterministically reproduce the race, so developers can fix it. Experiments on 16 apps indicate that only 3% of the races reported by race detectors are harmful, and that ERVA can verify an app in 20 minutes on average.

With the development and popularization of WiFi, surfing on the Internet with mobile devices has become an indispensable part of people's daily life. However, as an infrastructure, WiFi APs are easily connected by some undesired users nearby. In this paper, we propose NiFi, a non-intrusive WiFi user identification system based on WiFi signals that enables AP to automatically identify legitimate users in indoor environment such as home, office and hotel. The core idea is that legitimate and undesired users may have different physical constraints, e.g., moving area, walking path, etc, leading to different signal sequences. NiFi analyzes and exploits the characteristics of signal sequences generated by mobile devices. NiFi proposes a practical and effective method to extract useful features and measure similarity for signal sequences, while not relying on precise user location information. We implement NiFi on Commercial Off-The-Shelf (COTS) APs, and the implementation does not require any modification to user devices. The experiment results demonstrate that NiFi is able to achieve an average identification accuracy at 90.83% with true positive rate at 98.89%.

Soundness of type systems is an important property to guarantee the absence of certain kinds of runtime errors, that is, no false negatives can occur.Unfortunately, for well-known theoretical limits, there are many programs that cannot be typed correctly, even though they will never manifest runtime errors, that is, false positives can occur.Minimizing the rate of false positives makes static type analysis more effective, especially for dynamically typed languages. In this paper we propose a new approach to type systems, aiming to distinguish true from potentially false positives, and, thus, to provide useful hints on those lines of code that definitely contain a bug that sooner or later will occur.To this aim, we define a three-valued type system for Featherweight Java which is sound in the usual sense, but can also distinguish true positives from potentially false ones.

Images/videos are often uploaded in situations like disasters. This can tax the network in terms of increased load and thereby upload latency, and this can be critical for response activities. In such scenarios, prior work has shown that there is significant redundancy in the content (e.g., similar photos taken by users) transferred. By intelligently suppressing/deferring transfers of redundant content, the load can be significantly reduced, thereby facilitating the timely delivery of unique, possibly critical information. A key challenge here however, is detecting “what content is similar,” given that the content is generated by uncoordinated user devices. Toward addressing this challenge, we propose a framework, wherein a service to which the content is to be uploaded first solicits metadata (e.g., image features) from any device uploading content. By intelligently comparing this metadata with that associated with previously uploaded content, the service effectively identifies (and thus enables the suppression of) redundant content. Our evaluations on a testbed of 20 Android smartphones and via ns3 simulations show that we can identify similar content with a 70% true positive rate and a 1% false positive rate. The resulting reduction in redundant content transfers translates to a latency reduction of 44 % for unique content.

This paper is based upon an approach for automatic detection of personal events in on-line personal photo collections and proposes a powerful exploitation of these events: We compose social events out of personal events and then automatically reveal interpersonal ties. Trying to tame the stream of big data in social networks we solely rely on image meta-data of time and space. We validate our assumptions in the wild using 1.8 million public images of more than 4100 users. The proposed approach has three main steps: (i) personal event detection using individual, unsorted photo collections, in which we make use of the spatio-temporal context embedded in digital photos to detect event boundaries within the collection; (ii) social event detection for which we use a tailored similarity measurement between personal events of different users; and (iii) an analysis of event co-participation to propagate social connections. Experiments validate that the fully automated approach is able to accurately detect 78.76% of social events and reconstruct the interpersonal ties of a user with a verified true positive rate of 45%. This rate is probably much higher: Since most interpersonal ties are undefined in the universe of social networks, our experimental ground-truth of course remains fragmentary.

This paper presents a way to identify topics and classify text messages in the SMSAll system, which is the Twitter of Pakistan (except over SMS). Among many challenges, one is to develop an unsupervised algorithm for text messages containing Urdu-English words written in roman letters. Still in 1-gram we are able to have 72%, 53% and 58% true positives for popular, medium and rare topics respectively and 48% and 40% true positives in 2 and 3-grams respectively.

Dynamic specification mining observes program executions to infer models of normal program behavior. What makes us believe that we have seen sufficiently many executions? The typestate miner generates test cases that cover previously unobserved behavior, systematically extending the execution space and enriching the specification. To our knowledge, this is the first combination of systematic test case generation and typestate mining--a combination with clear benefits: On a sample of 800 defects seeded into six Java subjects, a static typestate verifier fed with enriched models would report significantly more true positives, and significantly fewer false positives than the initial models.

The complexity of large computer systems has raised unprecedented challenges for system management. In practice, operators often collect large volume of monitoring data from system components and set up many rules to check data and trigger alerts. However, the alerts from various rules usually have different problem reporting accuracy because their thresholds are often manually set based on operators' experience and intuition. Meantime, due to system dependencies, a single problem may trigger many alerts at the same time in large systems and the critical question is which alert should be analyzed first in the following problem determination process. In this paper, we propose a novel peer review mechanism to rank the importance of alerts and the top ranked alerts are more likely to be true positives. After comparing a metric value against its threshold to generate alerts, we also compare the value with the equivalent thresholds from many other rules to determine the importance of alerts. Our approach is evaluated with a real test bed system and experimental results are also included to demonstrate its effectiveness.

Format-string bugs are a relatively common security vulnerability, and can lead to arbitrary code execution. In collaboration with others, we designed and implemented a system to eliminate format string vulnerabilities from an entire Linux distribution, using type-qualifier inference, a static analysis technique that can find taint violations. We successfully analyze 66% of C/C++ source packages in the Debian 3.1 Linux distribution. Our system finds 1,533 format string taint warnings. We estimate that 85% of these are true positives, i.e., real bugs; ignoring duplicates from libraries, about 75% are real bugs. We suggest that the technology exists to render format string vulnerabilities extinct in the near future.