Search

Bias significantly undermines both the accuracy and trustworthiness of machine learning models. To date, one of the strongest biases observed in image classification models is texture bias-where models overly rely on texture information rather than shape information. Yet, existing approaches for measuring and mitigating texture bias have not been able to capture how textures impact model robustness in real-world settings. In this work, we introduce the Texture Association Value (TAV), a novel metric that quantifies how strongly models rely on the presence of specific textures when classifying objects. Leveraging TAV, we demonstrate that model accuracy and robustness are heavily influenced by texture. Our results show that texture bias explains the existence of natural adversarial examples, where over 90% of these samples contain textures that are misaligned with the learned texture of their true label, resulting in confident mispredictions., Comment: Accepted to IEEE Secure and Trustworthy Machine Learning (SaTML)

The influence of textures on machine learning models has been an ongoing investigation, specifically in texture bias/learning, interpretability, and robustness. However, due to the lack of large and diverse texture data available, the findings in these works have been limited, as more comprehensive evaluations have not been feasible. Image generative models are able to provide data creation at scale, but utilizing these models for texture synthesis has been unexplored and poses additional challenges both in creating accurate texture images and validating those images. In this work, we introduce an extensible methodology and corresponding new dataset for generating high-quality, diverse texture images capable of supporting a broad set of texture-based tasks. Our pipeline consists of: (1) developing prompts from a range of descriptors to serve as input to text-to-image models, (2) adopting and adapting Stable Diffusion pipelines to generate and filter the corresponding images, and (3) further filtering down to the highest quality images. Through this, we create the Prompted Textures Dataset (PTD), a dataset of 362,880 texture images that span 56 textures. During the process of generating images, we find that NSFW safety filters in image generation pipelines are highly sensitive to texture (and flag up to 60\% of our texture images), uncovering a potential bias in these models and presenting unique challenges when working with texture data. Through both standard metrics and a human evaluation, we find that our dataset is high quality and diverse.

The Tor anonymity network allows users such as political activists and those under repressive governments to protect their privacy when communicating over the internet. At the same time, Tor has been demonstrated to be vulnerable to several classes of deanonymizing attacks that expose user behavior and identities. Prior work has shown that these threats can be mitigated by leveraging trusted execution environments (TEEs). However, previous proposals assume that all relays in the network will be TEE-based-which as a practical matter is unrealistic. In this work, we introduce ParTEETor, a Tor-variant system, which leverages partial deployments of TEEs to thwart known attacks. We study two modes of operation: non-policy and policy. Non-policy mode uses the existing Tor relay selection algorithm to provide users incident security. Policy mode extends the relay selection algorithm to address the classes of attacks by enforcing a specific TEE circuit configuration. We evaluate ParTEETor for security, performance, and privacy. Our evaluation demonstrates that at even a small TEE penetration (e.g., 10% of relays are TEE-based), users can reach performance of Tor today while enforcing a security policy to guarantee protection from at least two classes of attacks. Overall, we find that partial deployments of TEEs can substantially improve the security of Tor, without a significant impact on performance or privacy.

In this work, we investigate \textit{texture learning}: the identification of textures learned by object classification models, and the extent to which they rely on these textures. We build texture-object associations that uncover new insights about the relationships between texture and object classes in CNNs and find three classes of results: associations that are strong and expected, strong and not expected, and expected but not present. Our analysis demonstrates that investigations in texture learning enable new methods for interpretability and have the potential to uncover unexpected biases., Comment: Accepted to ICLR 2024, Tiny Papers Track

Location data privacy has become a serious concern for users as Location Based Services (LBSs) have become an important part of their life. It is possible for malicious parties having access to geolocation data to learn sensitive information about the user such as religion or political views. Location Privacy Preserving Mechanisms (LPPMs) have been proposed by previous works to ensure the privacy of the shared data while allowing the users to use LBSs. But there is no clear view of which mechanism to use according to the scenario in which the user makes use of a LBS. The scenario is the way the user is using a LBS (frequency of reports, number of reports). In this paper, we study the sensitivity of LPPMs on the scenario on which they are used. We propose a framework to systematically evaluate LPPMs by considering an exhaustive combination of LPPMs, attacks and metrics. Using our framework we compare a selection of LPPMs including an improved mechanism that we introduce. By evaluating over a variety of scenarios, we find that the efficacy (privacy, utility, and robustness) of the studied mechanisms is dependent on the scenario: for example the privacy of Planar Laplace geo-indistinguishability is greatly reduced in a continuous scenario. We show that the scenario is essential to consider when choosing an obfuscation mechanism for a given application., Comment: M.S. Thesis (https://etda.libraries.psu.edu/catalog/25677abh5960)

Cloud file systems offer organizations a scalable and reliable file storage solution. However, cloud file systems have become prime targets for adversaries, and traditional designs are not equipped to protect organizations against the myriad of attacks that may be initiated by a malicious cloud provider, co-tenant, or end-client. Recently proposed designs leveraging cryptographic techniques and trusted execution environments (TEEs) still force organizations to make undesirable trade-offs, consequently leading to either security, functional, or performance limitations. In this paper, we introduce BFS, a cloud file system that leverages the security capabilities provided by TEEs to bootstrap new security protocols that deliver strong security guarantees, high-performance, and a transparent POSIX-like interface to clients. BFS delivers stronger security guarantees and up to a 2.5X speedup over a state-of-the-art secure file system. Moreover, compared to the industry standard NFS, BFS achieves up to 2.2X speedups across micro-benchmarks and incurs <1X overhead for most macro-benchmark workloads. BFS demonstrates a holistic cloud file system design that does not sacrifice an organizations' security yet can embrace all of the functional and performance advantages of outsourcing.

Public clouds necessitate dynamic resource allocation and sharing. However, the dynamic allocation of IP addresses can be abused by adversaries to source malicious traffic, bypass rate limiting systems, and even capture traffic intended for other cloud tenants. As a result, both the cloud provider and their customers are put at risk, and defending against these threats requires a rigorous analysis of tenant behavior, adversarial strategies, and cloud provider policies. In this paper, we develop a practical defense for IP address allocation through such an analysis. We first develop a statistical model of cloud tenant deployment behavior based on literature and measurement of deployed systems. Through this, we analyze IP allocation policies under existing and novel threat models. In response to our stronger proposed threat model, we design IP scan segmentation, an IP allocation policy that protects the address pool against adversarial scanning even when an adversary is not limited by number of cloud tenants. Through empirical evaluation on both synthetic and real-world allocation traces, we show that IP scan segmentation reduces adversaries' ability to rapidly allocate addresses, protecting both address space reputation and cloud tenant data. In this way, we show that principled analysis and implementation of cloud IP address allocation can lead to substantial security gains for tenants and their users., Comment: Replaced with version to appear in 2025 Network and Distributed Systems Security (NDSS) Symposium

Adversarial examples, inputs designed to induce worst-case behavior in machine learning models, have been extensively studied over the past decade. Yet, our understanding of this phenomenon stems from a rather fragmented pool of knowledge; at present, there are a handful of attacks, each with disparate assumptions in threat models and incomparable definitions of optimality. In this paper, we propose a systematic approach to characterize worst-case (i.e., optimal) adversaries. We first introduce an extensible decomposition of attacks in adversarial machine learning by atomizing attack components into surfaces and travelers. With our decomposition, we enumerate over components to create 576 attacks (568 of which were previously unexplored). Next, we propose the Pareto Ensemble Attack (PEA): a theoretical attack that upper-bounds attack performance. With our new attacks, we measure performance relative to the PEA on: both robust and non-robust models, seven datasets, and three extended lp-based threat models incorporating compute costs, formalizing the Space of Adversarial Strategies. From our evaluation we find that attack performance to be highly contextual: the domain, model robustness, and threat model can have a profound influence on attack efficacy. Our investigation suggests that future studies measuring the security of machine learning should: (1) be contextualized to the domain & threat models, and (2) go beyond the handful of known attacks used today., Comment: Accepted to the 32nd USENIX Security Symposium

Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledge or consent -- violating the core tenet of user privacy. In this paper, we introduce CaCTUs, a privacy-preserving smart camera system that returns control to the user; the root of trust begins with the user and is maintained through a series of cryptographic protocols designed to support popular features, such as sharing, deleting, and viewing videos live. In so doing, we demonstrate that it is feasible to implement a performant smart-camera system that leverages the convenience of a cloud-based model while retaining the ability to control access to (private) data. We then discuss how our techniques and protocols can also be extended to privacy-preserving designs of other IoT devices recording time series data., Comment: Extension of arXiv:2201.09338

Public clouds provide scalable and cost-efficient computing through resource sharing. However, moving from traditional on-premises service management to clouds introduces new challenges; failure to correctly provision, maintain, or decommission elastic services can lead to functional failure and vulnerability to attack. In this paper, we explore a broad class of attacks on clouds which we refer to as cloud squatting. In a cloud squatting attack, an adversary allocates resources in the cloud (e.g., IP addresses) and thereafter leverages latent configuration to exploit prior tenants. To measure and categorize cloud squatting we deployed a custom Internet telescope within the Amazon Web Services us-east-1 region. Using this apparatus, we deployed over 3 million servers receiving 1.5 million unique IP addresses (56% of the available pool) over 101 days beginning in March of 2021. We identified 4 classes of cloud services, 7 classes of third-party services, and DNS as sources of exploitable latent configurations. We discovered that exploitable configurations were both common and in many cases extremely dangerous; we received over 5 million cloud messages, many containing sensitive data such as financial transactions, GPS location, and PII. Within the 7 classes of third-party services, we identified dozens of exploitable software systems spanning hundreds of servers (e.g., databases, caches, mobile applications, and web services). Lastly, we identified 5446 exploitable domains spanning 231 eTLDs-including 105 in the top 10,000 and 23 in the top 1000 popular domains. Through tenant disclosures we have identified several root causes, including (a) a lack of organizational controls, (b) poor service hygiene, and (c) failure to follow best practices. We conclude with a discussion of the space of possible mitigations and describe the mitigations to be deployed by Amazon in response to this study.

Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledge or consent -- violating the core tenet of user privacy. In this paper, we present CaCTUs, a privacy-preserving smart Camera system Controlled Totally by Users. CaCTUs returns control to the user; the root of trust begins with the user and is maintained through a series of cryptographic protocols, designed to support popular features, such as sharing, deleting, and viewing videos live. We show that the system can support live streaming with a latency of 2s at a frame rate of 10fps and a resolution of 480p. In so doing, we demonstrate that it is feasible to implement a performant smart-camera system that leverages the convenience of a cloud-based model while retaining the ability to control access to (private) data., Comment: Accepted to PETS (Privacy Enhancing Technologies Symposium) 2022

Machine learning is vulnerable to adversarial examples-inputs designed to cause models to perform poorly. However, it is unclear if adversarial examples represent realistic inputs in the modeled domains. Diverse domains such as networks and phishing have domain constraints-complex relationships between features that an adversary must satisfy for an attack to be realized (in addition to any adversary-specific goals). In this paper, we explore how domain constraints limit adversarial capabilities and how adversaries can adapt their strategies to create realistic (constraint-compliant) examples. In this, we develop techniques to learn domain constraints from data, and show how the learned constraints can be integrated into the adversarial crafting process. We evaluate the efficacy of our approach in network intrusion and phishing datasets and find: (1) up to 82% of adversarial examples produced by state-of-the-art crafting algorithms violate domain constraints, (2) domain constraints are robust to adversarial examples; enforcing constraints yields an increase in model accuracy by up to 34%. We observe not only that adversaries must alter inputs to satisfy domain constraints, but that these constraints make the generation of valid adversarial examples far more challenging., Comment: Accepted to the 28th ACM Conference on Computer and Communications Security. Seoul, South Korea