Search

Proactive measurement of the delay in communication networks aims to detect congestion as early as possible and find links on which the traffic flow is obstructed. There is, however, a tradeoff between detection time and cost (e.g., bandwidth utilization). Adaptive measurement adjusts the inspection rate per each link, for effective monitoring with reduced costs. In this paper we present SDProber---a tool for proactive measurement of delays in SDN. SDProber uses probe packets that are routed by adding tailored rules to the vSwitches. It adjusts the forwarding rules to route probe packets more frequently to areas where congestion tends to occur. To increase efficiency, SDProber uses a novel approach of probing by a random walk. Adaptation is achieved by changing the probabilities that govern the random walk. Our experimental results show that SDProber provides control over the probe rates per each link and that it reduces measurement costs in comparison to baseline methods that send probe packets via shortest paths.

This paper explores the potential of enabling SDN security and monitoring services by piggybacking on SDN reactive routing. As a case study, we implement and evaluate a piggybacking based intrusion prevention system called SDN-Defense. Our study of university WiFi traffic traces reveals that up to 73% of malicious flows can be detected by inspecting just the first three packets of a flow, and 90% of malicious flows from the first four packets. Using such empirical insights, we propose to forward the first K packets of each new flow to an augmented SDN controller for security inspection, where K is a dynamically configurable parameter. We characterize the cost-benefit trade-offs of SDN-Defense using real wireless traces and discuss potential scalability issues. Finally, we discuss other applications which can be enhanced by using our proposed piggybacking approach.

Web performance impacts the popularity of a particular Web site or service as well as the load on the network, but there have been no publicly available end-to-end measurements that have focused on a large number of popular Web servers examining the components of delay or the effectiveness of the recent changes to the HTTP protocol. In this paper we report on an extensive study carried out from many client sites geographically distributed around the world to a collection of over 700 servers to which a majority of Web traffic is directed. Our results show that the HTTP/1.1 protocol, particularly with pipelining, is indeed an improvement over existing practice, but that servers serving a small number of objects or closing a persistent connection without explicit notification can reduce or eliminate any performance improvement. Similarly, use of caching and multi-server content distribution can also improve performance if done effectively.

Traffic, usage, and performance measurements are crucial to the design, operation and control of Internet protocol networks. This article describes a prototype infrastructure for the measurement, storage, and correlation of network data of different types and origins from AT&T's commercial IP network. We focus first on some novel aspects of the measurement infrastructure, then describe analyses that illustrate the power of joining different measured data sets for network planning and design.

We present a piggyback server invalidation (PSI) mechanism for maintaining stronger cache coherency in Web proxy caches while reducing overall costs. The basic idea is for servers to piggyback on a reply to a proxy client, the list of resources that have changed since the last access by the client. The proxy client invalidates cached entries on the list and can extend the lifetime of entries not on the list. This continues our prior work on piggyback cache validation (PCV) where we focused on piggybacking validation requests from the proxy cache to the server. Trace-driven simulation of PSI on two large, independent proxy log data sets, augmented with data from several server logs, shows PSI provides close to strong cache coherency while reducing the request traffic compared to existing cache coherency techniques. The best overall performance is obtained when the PSI and PCV techniques are combined. Compared to the best TTL-based policy, this hybrid policy reduces the average cost (considering response latency, request messages and bandwidth) by 7–9%, reduces the staleness ratio by 82–86%, yielding a staleness ratio of 0.001. © 1998 Published by Elsevier Science B.V. All rights reserved.

International audience; Internet exchange points (IXPs) are an important ingredient of the Internet AS-level ecosystem - a logical fabric of the Internet made up of about 30,000 ASes and their mutual business relationships whose primary purpose is to control and manage the flow of traffic. Despite the IXPs' critical role in this fabric, little is known about them in terms of their peering matrices (i.e., who peers with whom at which IXP) and corresponding traffic matrices (i.e., how much traffic do the different ASes that peer at an IXP exchange with one another). In this paper, we report on an Internet-wide traceroute study that was specifically designed to shed light on the unknown IXP-specific peering matrices and involves targeted traceroutes from publicly available and geographically dispersed vantage points. Based on our method, we were able to discover and validate the existence of about 44K IXP-specific peering links - nearly 18K more links than were previously known. In the process, we also classified all known IXPs depending on the type of information required to detect them. Moreover, in view of the currently used inferred AS-level maps of the Internet that are known to miss a significant portion of the actual AS relationships of the peer-to-peer type, our study provides a new method for augmenting these maps with IXP-related peering links in a systematic and informed manner.

Rule-based packet classification is a powerful method for identifying traffic anomalies, with network security as a key application area. While popular systems like Snort are used in many network locations, comprehensive deployment across Tier-1 service provider networks is costly due to the need for high-speed monitors at many network ingress points. Since ISPs already collect flow statistics ubiquitously, can we use it for detecting the same anomalies as the packet based rules in spite of aggregation and absence of payload information? We exploit correlations between packet and flow level information via a machine learning (ML) approach to associate packet level alarms with a feature vector derived from flow records on the same traffic. We describe a system architecture for network-wide flow- alarming and describe the steps required to establish a proof- of-concept. We evaluate prediction accuracy of candidate ML algorithms on actual packet traces. The duration of prediction effectiveness is an issue for ML approaches and more so in resource intensive network applications. Initial results show little impairment of performance over periods of one or two weeks.

There is a large and increasing amount of unwanted traffic on the Internet today, including phishing, spam, and distributed denial-of-service attacks. One way to deal with this problem is to filter unwanted traffic at the routers based on source IP addresses. Because of the limited number of available filters in the routers today, aggregation is used in practice: a single filter describes and blocks an entire range of IP addresses. This results in blocking of all (unwanted and wanted) traffic generated from hosts with IP addresses in that range. In this paper, we develop a family of algorithms that, given a blacklist containing the source IP addresses of unwanted traffic and a constraint on the number of filters, construct a set of filtering rules that optimize the tradeoff between the unwanted and legitimate traffic that is blocked. We show that our algorithms are optimal and also computationally efficient. Furthermore, we demonstrate that they are particularly beneficial when applied to realistic distributions of sources of unwanted traffic, which are known to exhibit spatial and temporal clustering.

Unexpected surges in request traffic (e.g., flash crowds) can exercise server-side resources such as access bandwidth, CPU, and disks in unanticipated ways. Administrators today do not have the requisite tools to fully understand the effect that flash crowds can have on server-side resources. As a result, most Web-servers today rely on significant over-provisioning, strict admission control, or alternatively use potentially expensive solutions like CDNs to provide high availability under load. A more fine-grained understanding of the performance of servers under emulated but controlled flash crowd like conditions can guide administrators to make more efficient provisioning and resource management decisions.We present the initial design of Mini-Flash Crowds (MFC) -- a light-weight wide-area profiling service that reveals resource bottlenecks in a Web-server infrastructure, including access bandwidth, processing resources, and back-end data management. The MFC approach is based on a set of controlled measurements in which an increasing number of distributed clients make synchronized requests to exercise specific resources of a remote server. Using a number of wide-area experiments and controlled lab tests, we show that our approach can faithfully track the impact of request loads on different server resources. Our approach is non-intrusive and thus we can use it to actively probe numerous live Web servers. We present the results from a preliminary measurement study of resource provisioning on public Web servers.

Web performance measurements and availability tests have been carried out using a variety of infrastructures over the last several years. Disruptions in the Internet can lead to Web sites being unavailable or increase user-perceived latency. The unavailability could be due to DNS, failures in segments of the physical network cutting off thousands of users, or attacks. Prompt reactions to network-wide events can be facilitated by local or remote measurement and monitoring. Better yet, a distributed set of intercommunicating measurement and monitoring entities that react to events dynamically could go a long way to handle disruptions.We have designed and built ATMEN, a triggered measurement infrastructure to communicate and coordinate across various administrative entities. ATMEN nodes can trigger new measurements, query ongoing passive measurements or historical measurements stored on remote nodes, and coordinate the responses to make local decisions. ATMEN reduces wasted measurements by judiciously reusing measurements along three axes: spatial, temporal, and application.We describe the use of ATMEN for key Web applications such as performance based ranking of popular Web sites and availability of DNS servers on which most Web transactions are dependent. The evaluation of ATMEN is done using multiple network monitoring entities called Gigascopes installed across the USA, measurement data of a popular network application involving millions of users distributed across the Internet, and scores of clients to aid in gathering measurement information upon demand. Our results show that such a system can be built in a scalable fashion.

For the last few years, large Web content providers interested in improving their scalability and availability have increasingly turned to three techniques: mirroring, content distribution, and ISP multihoming. The Domain Name System (DNS) has gained a prominent role in the way each of these techniques directs client requests to achieve the goals of scalability and availability. The DNS is thought to offer the transparent and agile control necessary to react quickly to ISP link failures or phenomenon such as flash crowds.In this paper, we investigate this assumption with the objective of quantifying the degree of responsiveness that can be expected from DNS. We use a combination of Web and DNS access measurements from several busy Web sites, as well as a large content distribution network, to characterize the behavior of end-systems and local DNS servers in terms of their adherence to DNS-based controls. Our results suggest that DNS is at best a coarse-grained mechanism, and poorly suited for applications, such as route control, which require quick response to link failures or performance degradations. We then propose several proactive techniques that, when deployed in cooperation between large content providers and important clients, have the potential to improve the responsiveness of DNS-based control.

Honeypots have been traditionally used to advertise dark address space and gather information about originators of traffic to such addresses. With simple thresholding mechanisms this technique has shown itself to be fairly effective in identifying suspicious IP addresses. Honeypots are however unsuitable to locate the precise entry point of unwanted traffic. Tracing back to the origination of such traffic is hard due to the delay and difficulty of maintaining state along the path of such traffic. We propose a novel mobile honeypot mechanism that allows unwanted traffic to be detected significantly closer to the origin. The mobility in our scheme stems from additional information that is made available to the upstream ASes as well as the changes in the set of dark address space advertised. Sharing information with a network of friendly ASes has the potential to identify and significantly lower unwanted traffic on such links.

Significant research on the Web is performed using logs collected from proxies and servers. Virtually all of these logs are from sites running the 1.0 version (or earlier) of HTTP. After four years a draft-standard specification of HTTP/1.1 has emerged. However, many proxies and servers still use HTTP/1.0, and most so-called HTTP/1.1 servers do not employ all of the key features available in the protocol. It is not feasible to evaluate the impact of HTTP/1.1 traffic using HTTP/1.0 logs. Thus, there is a need to examine ways of postulating HTTP/1.1 traffic from HTTP/1.0 traffic. This paper presents the En Passant architecture and a tool based on the architecture to convert HTTP/1.0 logs and traffic into plausible HTTP/1.1 traffic based on various parameters representing changes between the two versions of the protocol. The tool takes both the high-level information in HTTP/1.0 server logs and low-level information found in packet traces as input and generates a possible HTTP/1.1 log. We present the set of parameters and the actual feature changes between the protocol that impact the traffic by using a live packet trace as example.

We present the architecture and implementation of READY an event notification service that provides efficient, decoupled, asynchronous event notifications. READY supports consumer specifications that match over both single and compound event patterns, communication sessions that manage quality of service (QoS) for event delivery, grouping constructs for sessions and specifications, event zones and boundary routers that bound the scope of event distribution and control the mapping of events across zones.

With a key component of latency on the Web being connection set up between clients and Web servers, several ways to avoid connections have been explored. While the work in recent years on Content Distribution Networks (CDNs) have moved some content 'closer' to users at the cost of increasing DNS traffic, they have not fully exploited the available unused potential of existing protocols. We explore ways by which a variety of Web responses can be piggybacked on DNS messages. While we evaluated our idea in the Web context, the approach is generic and not restricted to Web responses. We propose an architecture for HTTP piggybacking in DNS messages and carry out a detailed performance analysis based on a trace-driven simulation study. Our architecture requires minimal extensions to existing protocols, utilizing only the allowed optional fields for these extensions. It is fully compatible and can coexist with the current Web.

Peer-to-peer applications such as Napster [4], Freenet [1], and Gnutella [2], [7] have gained much attention recently. These applications are mainly designed and used for largescale sharing of MP3 files. In such systems, end-hosts self-organize into an overlay network and share content with each other. Compared to the traditional client-server model, files are served in a distributed manner and replicated among the network on demand. Since hosts participating in peer-to-peer (P2P) networks also devote some computing resources, such systems scale with the number of hosts in terms of hardware, bandwidth, and disk space. With the wide deployment of P2P applications, the P2P traffic is becoming a growing portion of the Internet traffic. There has been very little examination of P2P traffic patterns and how they differ from traditional service models. Studying and understanding P2P traffic is thus important to provide efficient application-level content location and routing within the network. The existing applications use their own approach to do content location and routing and none of them are scalable. Napster uses a centralized server to locate content, while Gnutella clients broadcast queries to all their neighbors. [8] discusses the query locality observed in Gnutella traces and suggests caching as a short-term approach to increase Gnutella’s scalability. Recent designs such as CAN [5], Chord [9], Pastry [6], and Tapestry [10] propose distributed indexing schemes based on hashing to locate content. These systems assume a flat content delivery mesh. Each object’s location is stored at one or more nodes selected deterministically by a uniform hash function; queries for the object will be routed incrementally to the node. Although hash functions can help locate content deterministically, they lack the flexibility of keyword searching—a useful operation to find content without prior knowledge of exact object names. There is no real deployment at present and thus no measurement information is available for understanding the usability and scalability of

Content distribution networks (CDNs) are a mechanism to deliver content to end users on behalf of origin Web sites. Content distribution offloads work from origin servers by serving some or all of the contents of Web pages. We found an order of magnitude increase in the number and percentage of popular origin sites using CDNs between November 1999 and December 2000.In this paper we discuss how CDNs are commonly used on the Web and define a methodology to study how well they perform. A performance study was conducted over a period of months on a set of CDN companies employing the techniques of DNS redirection and URL rewriting to balance load among their servers. Some CDNs generally provide better results than others when we examine results from a set of clients. The performance of one CDN company clearly improved between the two testing periods in our study due to a dramatic increase in the number of distinct servers employed in its network. More generally, the results indicate that use of a DNS lookup in the critical path of a resource retrieval does not generally result in better server choices being made relative to client response time in either average or worst case situations.

The increasing deployment of overlay networks that rely on DNS tricks has led to added interest in examining DNS traffic. In this paper we report on a characterization of DNS traffic gathered over a period of several weeks at Internet Gateway Routers (IGRs) in the AT&T Common Backbone. The characterization is carried out using several novel techniques to identify clients, local DNS servers, and authoritative DNS servers. Our techniques include passive and active measurements, graph-based analysis, examination of outliers, and explicit checks against data obtained from several external sources. Our contribution is the reduction of a very large data set (over 1 terabyte of raw data) into a significantly smaller representation that is ideally suited for answering protocol-specific semantic queries quickly. After categorizing the addresses, we use the network aware clustering technique to group local DNS servers. By juxtaposing the DNS server clusters with clusters formed by Web clients obtained from a large portal Web site, we determine the distribution of identified DNS servers in busy clusters. A variety of applications are examined, ranging from identifying suspected zombies to helping Content Distribution Networks in mapping location of DNS servers.

Internet traffic has grown significantly with the popularity of the Web. Consequently user perceived latency in retrieving Web pages has increased. Caching and prefetching at the client side, aided by hints from the server, are attempts at solving this problem. We suggest techniques to group resources that are likely to be accessed together into volumes, which are used to generate hints tailored to individual applications, such as prefetching, cache replacement, and cache validation. We discuss theoretical aspects of optimal volume construction, and develop efficient heuristics. Tunable parameters allow our algorithms to predict as many accesses as possible while reducing false predictions and limiting the size of hints. We analyze a collection of large server logs, extracting access patterns to construct and evaluate volumes. We examine sampling techniques to process only portions of the server logs while constructing equally good volumes. We show that it is possible to predict requests at low cost with a high degree of precision.

The rapid growth of the World Wide Web has caused serious performance degradation on the Internet. This paper offers an end-to-end approach to improving Web performance by collectively examining the Web components --- clients, proxies, servers, and the network. Our goal is to reduce user-perceived latency and the number of TCP connections, improve cache coherency and cache replacement, and enable prefetching of resources that are likely to be accessed in the near future. In our scheme, server response messages include piggybacked information customized to the requesting proxy. Our enhancement to the existing request-response protocol does not require per-proxy state at a server, and a very small amount of transient per-server state at the proxy, and can be implemented without changes to HTTP 1.1. The server groups related resources into volumes (based on access patterns and the file system's directory structure) and applies a proxy-generated filter (indicating the type of information of interest to the proxy) to tailor the piggyback information. We present efficient data structures for constructing server volumes and applying proxy filters, and a transparent way to perform volume maintenance and piggyback generation at a router along the path between the proxy and the server. We demonstrate the effectiveness of our end-to-end approach by evaluating various volume construction and filtering techniques across a collection of large client and server logs.

Data concerning the users and usage of Online Social Networks (OSNs) has become available externally, from public resources (e.g., user profiles), participation in OSNs (e.g., establishing relationships and recording transactions such as user updates) and APIs of the OSN provider (such as the Twitter API). APIs let OSN providers monetize the release of data while helping control measurement load, e.g. by providing samples with different cost-granularity tradeoffs. To date, this approach has been more suited to releasing transactional data, with graphical data still being obtained by resource intensive methods such a graph crawling. In this paper, we propose a method for OSNs to provide samples of the user graph of tunable size, in non-intersecting increments, with sample selection that can be weighted to enhance accuracy when estimating different features of the graph., Accepted to appear in the Proceedings of the 54th Annual Allerton Conference on Communication, Control, and Computing, 2016