Search

To share and exchange the files among Internet users, peer-to-peer (P2P) applications build another layer of overlay networks on top of the Internet infrastructure. In P2P file sharing systems, a file request takes two steps. First, a routing message is generated by the client (request initiator) and spread to the overlay network. After the process finishes, the location information of the requested file is returned to the client. In the second step, the client establishes direct connection(s) with the peer(s) who store a copy of that file to start the retrieving process. While numerous research projects have been conducted to design efficient, high-performance routing algorithms, few work concentrated on file retrieving performance. In this paper, we propose a novel and efficient algorithm - PROD to improve the file retrieving performance in DHT based overlay networks. In PROD, when a file or a portion of a file is transferred from a source peer to the client, instead of creating just one direct link between these two peers, we build an application level connection chain. Along the chain, multiple network links are established. Each intermediate peer on this chain uses a store-and-forward mechanism for the data transfer. PROD also introduces a novel topological based strategy to choose these peers and guarantees the transmission delay of each intermediate link is much lower than the direct link. We conducted extensive simulation experiments and the results shown that PROD can greatly reduce the transfer time per file in DHT base P2P systems.

With the growth of many open source operating systems comes the introduction of open source file systems that allow free modification. File system modifications can be a method of users to evade investigators who use commercial tools to analyze the suspect's media. Investigators need to be aware of what is possible and ways to circumvent the methods that users can utilize when modifying their file system to hide data.

The need for saving a great many files becomes necessary in distributed file system environment. Particularly, some file systems maintain multiple file replicas for efficient failure recovery. Under these circumstances, however, all the file replicas have to be modified identically to update a file. If there are many clients to update the same file concurrently, the final images of all replicas can be inconsistent. In this paper, we propose a consistent update method so that a file can have the identical image in all replicas after concurrent modifications by multiple clients.

Data management is one of the most important problems in grid environments. One important challenge facing grid computing is the design of a grid file system. The Global Grid Forum defines a grid file system as a human-readable resource namespace for management of heterogeneous distributed data resources, that can span across multiple autonomous administrative domains. This paper evaluates Expand, a new grid file system according to the Global Grid Forum recommendations that integrates heterogeneous data storage resources in grids using standard grid technologies: GridFTP and the OGSA ByteIO interface defined by the Open Grid Forum.

Peer-to-peer (P2P) file sharing systems are becoming increasingly popular due to their flexibility and scalability. We propose a new model to design file replication schemes for P2P file sharing systems. The model introduces expected costs for serving user requests for the files which are computed from node up/down statistics. Based on the model we introduce and develop several methods to determine the sets of nodes to store copies of the files in order to optimize certain performance metrics (e.g., maximize the system hit rate, minimize the total expected cost). We verify the effectiveness of the file replication schemes via simulation. We also outline a framework to implement the file replication schemes for P2P file sharing systems in a distributed and adaptive manner. The framework scales to a large number of nodes and files and can handle user request pattern change via file migration.

The Data Grid, like all other collaboration models, has strict rules for contributors to follow and many criteria to abide with. Namespace is one of the rules that govern the contributors. Some fields are not ready for abiding with such kinds of global rules. For instance, scientific research taxonomy (down to topics and areas of interest) is highly dynamic. Topics are confusingly interdisciplinary and sometimes have institutional influence. The intention is to design a hybrid namespace collaboration model for existing organizations to pay lower cost (initial and running) in order to join it. Data resources will be allowed to have colliding names. The contributing organization will be free to set any rules for its internal users to follow (such as taxonomy). However, it will have to take care of a very simple interface to the intended model in order to introduce new contributed resources or to query external ones. A hierarchical hybrid namespace will be maintained in order to uniquely identify data resources that have colliding names. Similar and related topics in different places in the hierarchy will be linked by semantic relations to express the degree of similarity and or relevance. Resource explorations should follow the suitable semantic relations while traversing the hierarchy of servers and data grids for more successful search.

The importance of file migration is increasing because of its potential to improve the performance of distributed office, manufacturing and hospital information systems. To encourage research in the file migration problem, the authors summarize accomplishments of researchers of the problem, provide a detailed comparison of file migration and dynamic file allocation problems, and identify important areas of research to support the development of effective file migration policies.

The built-in process registry has proven to be an extremely useful feature of the Erlang language. It makes it easy to provide named services, which can be reached without knowing the process identifier of the serving process.However, the current registry also has limitations: names can only be atoms (unstructured), processes can register under at most one name, and it offers no means of efficient search and iteration.In Ericsson's IMS Gateway products, a recurring task was to maintain mapping tables in order to locate call handling processes based on different properties. A common pattern, a form of index table, was identified, and resulted in the development of an extended process registry.It was not immediately obvious that this would be worthwhile, or even efficient enough to be useful. But as implementation progressed, designers found more and more uses for the extended process registry, which resulted in significant reduction of code volume and a more homogeneous implementation. It also provided a powerful means of debugging systems with many tens of thousand processes.This paper describes the extended process registry, critiques it, and proposes a new implementation that offers more symmetry, better performance and support for a global namespace.

This paper presents the design of two tabletop file system interfaces: OriTop, a novel associative access approach to file system interaction, where users navigate multiple file systems by selecting focus files; and the Browser, a hierarchical interface that is based upon the same mental model as conventional desktop file system access. We report a qualitative study with ten users to explore both approaches. OnTop was found to better facilitate collaboration on file access and use, while the more familiar hierarchical model of the Browser was found to be more natural on very early use and has a clear role-particularly in cases where the associative approach fails.

Most traditional cluster file systems replicate whole files to support the data availability. However, considering that the access patterns of file systems have high localities, replicating whole files is not an efficient strategy to improve the data availability. This paper presents the design and the implementation of HA-PVFS, a PVFS supporting high data availability, in which the replication is performed partially based on the locality information. Our performance results show that HA-PVFS is dynamic with little overhead over various access patterns.

One way of improving I/O performance is to carry out I/O operations in parallel, which is supported by parallel file systems. The data management techniques are very important to build a parallel file system with high performance and high reliability. A simple and efficient parallel file data management method was implemented. A fixed stripe size and redundant data placement technique was used. Experimental results showed that this method can improve the performance and the reliability of parallel file data.

The traditional file service is no longer sufficient for providing reliable and smart service to users as roaming in ubiquitous computing environments. In this paper, we propose a smart push file service model, called PushFS, which employs a push-oriented file service for ubiquitous storage. The design of PushFS introduces user-profiles to identify the environmental-context and file-semantics for retrieve, and hoards virtual directories and virtual files into lightweight working sets on-demand. We indicate that the dynamic metadata-aware file selection and organization capability increases efficiency of the file service and improves the tradeoff between proactivity and transparency. We then implement a prototype system to validate our model, which shows that PushFS is adequate to ubiquitous storage with high efficient and proactive file service.

In a P2P file-sharing system, a node finds and retrieves its desired file. If multiple nodes cache the same file to provide others, we can achieve a file-sharing system with low latency and high file availability. However, a node has to spend costs, e.g., processing load or storage capacity, on caching of a file. Consequently, a node may selfishly behave and hesitate to cache a file. In such a case, there is a possibility that unpopular files disappear from the system. In this paper, we aim to accomplish effective caching in the whole system that emerges from autonomous and selfish node behavior. We first discuss relationship between selfish node behavior and system dynamics according to evolutionary game theory. As a result, we show that a file-sharing system can be robust to file disappearance depending on a cost and demand model for caching even if nodes behave selfishly.

We present FRAC, a Framework for role-based access control in network file systems. FRAC is a reference monitor that controls the message flow between file system clients and servers. FRAC supports role hierarchies, user sessions, and static and dynamic separation of duty constraints. It also allows administrators to define dynamic policies based on access history and the environment, e.g., time of day. FRAC introduces a virtual control namespace (VCN) that provides an interface to query and update the state of the access control framework over the standard file system protocol. This namespace eliminates the need for executing specialized user agents either at the client or at the server. Therefore, FRAC does not require any modification to either the file system client or the file server. We have implemented FRAC for the widely deployed NFS protocol using FileWall, a file system proxy previously developed by us. Our experimental evaluation shows that FRAC imposes minimal overheads for the common file system operations.

In peer-to-peer (P2P) file sharing networks, the average download time is determined by how efficiently the upload bandwidth of each peer can be utilized. In this paper, we will first propose a stochastic model for P2P file sharing networks. By numerically solving the proposed model, we will be able to gain interesting insight on how the performance of a P2P file sharing network is affected by different parameters such as the piece numbers of the file, the number of neighbors of a peer, and the seed departure rate etc. We will also provide some useful guidelines on how to design an efficient P2P system based on our results.

Semantic file systems enhance standard file systems with the ability of file searching based on file semantics. The users interact with semantic file systems not only by browsing a hierarchy of directories but also by querying as information retrieval systems usually do. In this paper, we argue for a new file system paradigm, the semantic file system. We identify the issues in designing a semantic file system and propose an ontology- based solution for these issues.

Achieving high availability is a premium goal for many distributed file systems. File replication is a well-known technique that is used to achieve this goal. It generally offers reduced client latencies and increases file availability. The objective of this paper is to achieve high availability of files through a novel intelligent replication method. To achieve this goal two algorithms are proposed, namely a primary replica assignment algorithm and an intelligent replica placement algorithms. The assignment algorithm uses an intelligent criteria based scheme to decide which one from a number of replicas to be assigned as the primary copy for a set of replicas. The placement algorithm also uses an intelligent criteria based scheme to decide on replication of files. The two algorithms have different criteria and the combined effect of both facilitate for higher availability of files within the distributed file system.

According to the development of the object based storage device (OSD) and networking technologies, some OSD based clustering file systems, for example, the ActiveScale storage cluster of the Panasas Corp. and the Lustre file system of the Cluster File System Corp., are emerging in the marketplace. Typically, these systems connect multiple OSDs on the fast network. When a large number of OSDs are connected as a linear style configuration, a client machine stores a file in its local memory first and then flushes it into an OSD. However, the file stored temporarily in the client's memory may not be completely stored in the selected OSD because the size of the file can be bigger than that of the OSD's available space. In this paper, we propose a detailed algorithm and system structure for solving this problem occurring in the cluster file system using OSDs.

We consider a BitTorrent-like file sharing system, where the peers interested in downloading a large file join an overlay network. The seed node possessing the file stays in the system, whereas all other peers are non-altruistic in the sense that they leave the system as soon as they have downloaded the whole file. We consider a flash crowd scenario, where the peers join the overlay simultaneously. We show that the chunk selection algorithm is critical, propose an analytic approach to the process, and find that the encounters can be restricted to neighbours in a Chord overlay without losing much in performance.

International audience; In this article a three staged file transfer approach for the Grid is proposed. File transfer in the Grid can take place at three stages: deployment, user application execution, and retrieval (post-execution). Each stage has it's own environmental requirements, and therefore different techniques must be applied. The contributions presented in this article can be grouped in two. First, the integration of heterogeneous resource acquisition and file transfer protocols. This allows on-the-fly deployment and retrieval of files. Secondly, an asynchronous file transfer mechanism based on active objects, wait-by-necessity, and automatic continuation. The proposed file transfer model has been implemented using the ProActive middleware. ProActive provides, among others, a Grid infrastructure abstraction using deployment descriptors, and a programming model based on active objects with transparent futures. Finally, the proposed file transfer model is benchmarked and validated with a real case study: BLAST.

Peer-to-peer (P2P) file sharing systems often replicate files to multiple nodes. File replication is beneficial in the sense that it can achieve good query latency, load balance, and reliability. However, it introduces another problem of maintaining mutual consistency among replica when a file is updated. The new file has to be propagated to all of its replica. In this paper, we propose an update propagation algorithm for P2P file sharing over wireless mobile networks (MONET). Compared to the previous algorithms proposed in wired P2P file sharing systems, our algorithm has low communication overhead. It also guarantees safe delivery of updates even when the underlying network is unreliable. This means that our algorithm is well matched to the characteristics of MONET such as limited battery power of mobile nodes, lower network reliability, and frequent disconnection of mobile nodes.

With the system scale keeping grown, the key challenge is to mask the failures that arise among the system components and to improve the performance of data-intensive applications. This paper designs and implements a cluster-based distributed replication file system EDRFS to meet these critical demands. EDRFS works with a single metadata server and multiple storage nodes, deploys whole-file replication scheme at the file level, and tracks what storage node a file is replicated on. We use a linear hash algorithm to evenly distribute data and load across multiple storage nodes so as to achieve balancing workload and incremental scalability of throughput and storage capacity as the system scale grows. In addition, we employ metadata caches and file data caches in clients to enhance system performance. Furthermore, we deploy a concurrency lock scheme to avoid namespace operation bottleneck and a replicas consistency method to keep a consistent mutation order among replicas of a file. We provide the initial experimental evaluations of our prototypical system on a small-file and data-intensive workload.

Distributed file systems provide a fundamental abstraction to location-transparent, permanent storage. They allow distributed processes to co-operate on hierarchically organized data beyond the life-time of each individual process. The great power of the file system interface lies in the fact that applications do not need to be modified in order to use distributed storage. On the other hand, the general and simple file system interface makes it notoriously difficult for a distributed file system to perform well under a variety of different workloads. This has lead to today's landscape with a number of popular distributed file systems, each tailored to a specific use case. Early distributed file systems merely execute file system calls on a remote server, which limits scalability and resilience to failures. Such limitations have been greatly reduced by modern techniques such as distributed hash tables, content-addressable storage, distributed consensus algorithms, or erasure codes. In the light of upcoming scientific data volumes at the exabyte scale, two trends are emerging. First, the previously monolithic design of distributed file systems is decomposed into services that independently provide a hierarchical namespace, data access, and distributed coordination. Secondly, the segregation of storage and computing resources yields to a storage architecture in which every compute node also participates in providing persistent storage.

Servers' reintegration is a mode of file system operation that allows file servers to synchronize their data after network partitions. The reintegration design supports the main objectives of the disconnection-resilient file systems, which is to provide high available and reliable storage for files, and guarantees that file operations are executed in spite of concurrency and failures. In this paper, we show that server reintegration is efficient and practical by describing its design and implementation in paradise file system. Moreover, we report on its performance evaluation using a cluster of workstations. Our results indicate clearly that our design exhibits a significant degree of automation and conflict-free mobile file system.

We propose a new overlay network, called Generic Identifier Network (GIN), for collaborative nodes to share objects with transactions across affiliated organizations by merging the organizational local namespaces upon mutual agreement. Using local namespaces instead of a global namespace can avoid excessive dissemination of organizational information, reduce maintenance costs, and improve robustness against external security attacks. GIN can forward a query with an O(1) latency stretch with high probability and achieve high performance. In the absence of a complete distance map, its heuristic algorithms for self configuration are scalable and efficient. Routing tables are maintained using soft-state mechanisms for fault tolerance and adapting to performance updates of network distances. Thus, GIN has significant new advantages for building an efficient and scalable distributed hash table for modern collaborative applications across organizations

In this paper, we present the mathematical analysis of two important performance measures for a BitTorrent (BT) like P2P file sharing system, namely, average file downloading time and file availability. For the file downloading time, we develop a model using the "stochastic differential equation" approach, which can capture the system more accurately than some previous approach [17] and can capture various network settings and peers behavior. We study the steady-state behavior and obtain the closed-form solutions for performance measures which allow us to carry sensitivity analysis on various performance measures for various system parameters. We then extend this model to consider multiclass peers wherein some peers are behind firewalls which may impede the uploading service. We also present the mathematical model to study the file availability of a BT-like system. The model helps us gain the understanding of why the "rarest-first" chunk selection policy is used in today's BT protocol. We propose a novel chunk selection algorithm to enhance the overall system file availability. Extensive simulations are carried to validate our analysis.

Today, more and more people in knowledge communities, like research laboratories, use shared file servers to store and share their information. People in such communities often work together and their files stored in a file server have relationships with each other. Information on the relationships is usually exchanged offline and used implicitly to facilitate the management and sharing of the files. This paper proposes new functions to manage and use the relationships to make various views on the file servers. The functions provide a high-level support and are compatible with the operational framework of existing file systems.

This paper gives an overview of a novel storage management concept, called, Adaptive Object File System (AOFS). The design of the file type classification module in AOFS is emphasized. The design attempts to increase the efficiency through the dynamic tuning technique, which automatically classifies files using attributes and access pattern. The file classification, thus, allows files to be stored in the most efficient way. The key idea is to utilize the file properties, such as access pattern, owner, size, and permissions to select adaptive file system policies (e.g. disk allocation, redundancy, and caching strategies). Moreover, a metadata store is maintained to provide the best possible dynamic tuning strategy for any given operating period. The static classification initial design is done based on decision tree, while the dynamic classification adapts the Hidden Markov Model for prediction. The main goal of the AOFS design is to enhance system performance, storage efficiency and flexibility.

As the tremendous growth in peer-to-peer (P2P) applications, the issues related to file consistency become critical. In this paper, an algorithm for file Consistency Maintenance through Virtual servers (CMV) is proposed for unstructured and decentralized P2P systems. In CMV, consistency of each dynamic file is maintained by a virtual server (VS). A file update can only be accepted through the VS to ensure the one-copy serializability. The VS of a file is a logical network composed of multiple replica peers (RPs) that have replicas of the file. Mathematical analysis is performed to determine the optimal parameter selections that achieve minimum overhead messages for maintaining file consistency. The numerical results indicate that CMV is well suited for efficient file consistency maintenance in P2P systems.

The main obstacles to grid file management come from the fact that grid file resources are typically stored in heterogeneous and distributed environment and accessed through various protocols. In this paper, we propose a grid file management system called Vega [1][2] Hotfile2 for data-intensive application in widely distributed systems and grid environments. Widely distributed and heterogeneous storage resource can converge into a single uniform storage resource using virtualization techniques in Hotfile2. It is a grid middleware provides a uniform file resources view and a high level interface to access widely distributed and heterogeneous data resources. Presented here are two services that we believe are fundamental to any grid file management system: high-performance, secure transport and metadata management. In this paper, we discuss many aspects of our design and implementation in Hotfile2, present the file system interface designed to support distributed applications such as batch job. Experiments are done to illustrate its performance, extensibility, and various other features

Traditional network file systems, like NFS, do not extend to wide-area due to low bandwidth and high network latency. We present WireFS, a Wide Area File System, which enables delegation of metadata management to nodes at client sites (homes). The home of a file stores the most recent copy of the file, serializes all updates, and streams updates to the central file server. WireFS uses access history to migrate the home of a file to the client site which accesses the file most frequently. We formulate the home migration problem as an integer programming problem, and present two algorithms: a dynamic programming approach to find the optimal solution, and a non-optimal but more efficient greedy algorithm. We show through extensive simulations that even in the WAN setting, access latency over WireFS is comparable to NFS performance in the LAN setting; the migration overhead is also marginal.

Distributed computer applications usually need processes that allow them to retrieve, store, and share data in a suitable way. As a consequence, file systems become a basic concern for these kinds of applications. Typical file systems have been designed on a computer network infrastructure whose communication and data exchange support is homogeneous. The Internet encourages the construction of file systems, which deal with heterogeneous computer platforms. This work presents SoapFS, a file system that is able to manage information in a heterogeneous environment such as the Internet. This is possible because it is based on technologies like RPC-XML and SOAP. This paper shows how a set of functions available in SoapFS allows the development of robust and heterogeneous distributed applications that involve remote data retrieval and storage. SoapFS shows competitive performance compared with popular file systems and it can connect different file systems working together like a one virtual file system.

With the increasing number of applications and their variety, the need to prevent unauthorized disclosure of data and to ensure data integrity in distributed systems has led to a heavy reliance on encryption. Typically, the encryption process operates at file or directory level and applies indiscriminately one algorithm to all the data in a file. In this paper, a distributed scheme is proposed whereby secrecy is achieved through file data concealment and file location concealment, within a network of nodes. This involves first the division of a file into fragments, which are encrypted by applying different algorithms; second the random distribution of these fragments to connected nodes, third the generation of metadata for reconstructing the original file and its commitment to a physical medium and finally the deletion of both the original file and its metadata from the local node. Access to the metadata is the only way of retrieving the original file. An evaluation of the scheme is presented with some performance results.

Over the past 7 years, the San Diego Supercomputer Center has worked to produce the highest possible performance file systems available to the National Science Foundation community in the USA. Most of this was done with GPFS, IBM's parallel file system, but several distinctly different configurations were designed and implemented with numerous lessons learned in the process. All of these systems provided transfer rates in the multiple GB/s range. In this paper, we detail the configurations and their intended modes of operation and, as much as possible, show the resulting performance. We attempt to describe the advantages and disadvantages of each approach with an emphasis on the implications for future systems.

Sharing data among collaborators in widely distributed systems remains a challenge due to limitations with existing methods for defining groups across administrative domain boundaries with various file systems. Groups in traditional systems are bound to particular domains or file systems using centralized storage locations either beyond ordinary users' ability to manage, inaccessible outside a closed system, or both. We present a method for users to independently create and manage groups on any networked workstation using global user identities and to control access to shared data and storage resources based on group membership, regardless of domain boundaries or underlying file systems. Decentralized groups are decoupled from shared user databases and centralized authentication servers through the use of a virtual user namespace. We describe how owners of shared resources can define security policies through the use of caching, and demonstrate how each caching policy represents tradeoffs between performance, scalability, and consistency.

A common vision for the Grid is that it would enable heterogeneous, spatially distributed, non-trusted computers to operate as a single system for users, compilers, applications and services. However, currently available Grid solutions, while providing for such computers to cooperate, are complex and do not support the single system image principles. In this chapter, we describe a simple architecture for a portable Grid engine which produces and uses both interpreted and binary objects constructed around text-based mathematical descriptions of those objects and their classes. A first prototype engine has been implemented that achieves some functionality and a fully functional model is under development.

In this paper we describe the evolution of Global File Systems from the concept of a few years ago, to a first demonstration using hardware Fibre Channel frame encoding into IP packets, to a native GFS, to a full prototype demonstration, and finally to a production implementation. The surprisingly excellent performance of the Global File Systems over standard TCP/IP Wide Area Networks has made them a viable candidate for the support of Grid Supercomputing. The implementation designs and performance results are documented within this paper. We also motivate and describe the authentication extensions we made to the IBM GPFS file system, in collaboration with IBM. In several ways Global File Systems are superior to the original approach of wholesale file movement between grid sites and we speculate as to future modes of operation.

File synchronization problem has been solved using many different approaches. One recent approach is to use intelligent agents to synchronize files. Multisync is such a system. The original system implemented file synchronization across machines using agents. However the system did not cater for virus attack or files with different times but same contents, which can result in extraneous synchronization. The current paper presents an improvement to the system by allowing for hash based comparisons of files. Using MD5, the paper demonstrates that a marked improvement is achieved in file synchronization. It is conjectured that the proposed approach to file synchronization results in performance improvement. Experimental results and statistics have also been presented

The growth of the bandwidth available in WAN links stimulates novel usage of traditional network systems. By extending the boundary of cluster file systems to the customers' premise, it is now possible to provide home users efficient, dependable, and responsive network storage. In this paper, we identify the primary issue of network latency when implementing cluster file systems across the last miles, and propose a solution by replacing the round-based data transmission protocol of Coda file system with a rate-based one. The performance evaluation of the prototype system shows significant improvements of both throughput and response time for file-transfer operations, especially under high network latency. The result is a latency-resistant cluster file system.

In this paper, a novel workflow-aware distributed versioning file system, WAD-VFS is presented to overcome the shortcoming of traditional DFS and facilitate the high performance computing. Our preliminary simulation results are impressive and can hence serve as a supporting evidence of deploying WAD-VFS to our ongoing metacomputing project, Trellis system.

All too often, decisions about whom to trust in computer systems are driven by the needs of system management rather than data security. In particular data storage is often entrusted to people who have no role in creating or using the data-through outsourcing of data management, hiring of outside consultants to administer servers, or even collocation servers in physically insecure machine rooms to gain better network, connectivity. This paper outlines the design of SUNDR, a network file system designed to run on untrusted servers. SUNDR servers can safely be managed by people who have no permission to read or write data stored in the file system. Thus, people can base their trust decisions on who needs to use data and their administrative decisions on how best to manage the data. Moreover, with SUNDR, attackers will no longer be able to wreak havoc by compromising servers and tampering with data. They will need to compromise clients while legitimate users are logged on. Since clients do not need to accept incoming network connections, they can more easily be firewalled and protected from compromise than servers.

The paradigm of Ubiquitous computing seeks to build a computing environment that responds to user context. An ideal file system for the Ubiquitous environment is one that can successfully recognize the present context and automate file management. The intelligence in the Ubiquitous file management is achieved by applying a heuristics based clustering approach to the system. The applied heuristics are those that are used on file attributes by users to manually manage files in a traditional file system. File attributes can be used to relate files to the most appropriate work-context and also draw inter-file relationships. We discuss methods to harness the given file information from the file-system to form a context-relation wrapper over disparate files. This enables management of files as context related working sets rather than as individual files. A survey was conducted among regular computer users and the compiled results supported our context based file clustering approach. The experiments also showed promising results that confirm our model on the file heuristics, thus finding semantic relations between those files.

Secure management of file systems of large organizations can present significant challenges to system administrators, in terms of the number of users, shared access to parts of the file system for supporting large software projects, and securing and monitoring critical parts of the file system from intruders. We present interactive visualization tools for monitoring and viewing the complex access control relationships within large file systems. This tool is targeted as an aid to system administrators to manage users, software applications and shared access. We tested our tool on UNC Charlotte's Andrew File System (AFS), which contains 7043 users, 560 user groups, and about 2.1 million directories. Our system displays summary information about the file system, and two types of visualizations to explore access control relationships among classes of users. In addition, drill-down features are provided to explore the user file system structure and manage access control information of any directory within the system. All of the views are linked to permit easy navigation and features are provided that make the system scalable to larger filesystems.

With the advent of Storage Area Networks (SAN) and Network Attached Disks (NASD), a new trend in storage systems design is to move disks from behind storage servers and attach them to a SAN for direct client access. Such designs remove storage servers from the path of data transfer leading to highly scalable file systems. While there exists many file systems built using such direct attached disks, they are all optimized for traditional text-based data and are not well-suited for streaming continuous media, i.e., audio and video data. In this paper we present the architectural details of a scalable distributed continuous media file system built using SAN-attached disks. We describe the implementation details of our Linux-based prototype and show by experimention that the performance of our file system scales linearly with the number of disks and the number of clients, and the file system provides strict bandwidth guarantees for continuous media streams.

The emerging issues of personal and network mobility have created a need for the decoupling of identity from location in Internet addressing. This decoupling requires name resolution systems that can provide scalable resolution of globally unique persistent identifiers of communication endpoints, which may be users, devices, content or services. Recent developments in structured peer-to-peer overlay networks have made possible the scalable resolution of flat names, which opens up new possibilities in the area of naming and name resolution systems. In this paper we propose a scheme to provide authentication and verification in a name resolution system based on structured peer to peer networks such as distributed hash tables (DHTs). We specify how namespace security and global uniqueness may be managed with the use of public key cryptography. We also propose a framework within which independent overlay networks may compose a global namespace.

Social networks offering unprecedented content sharing are rapidly developing over the Internet. Unfortunately, it is often difficult to both locate and manage content in these networks, particularly when they are implemented on current peer-to-peer technologies. In this paper, we describe Wayfinder, a peer-to-peer file system that targets the needs of medium-sized content sharing communities. Wayfinder seeks to advance the state-of-the-art by providing three synergistic abstractions: a global namespace that is uniformly accessible across connected and disconnected operation, content-based queries that can be persistently embedded into the global namespace, and automatic availability management. Interestingly, Wayfinder achieves much of its functionality through the use of a peer-to-peer indexed data storage system called PlanetP: essentially, Wayfinder constructs the global namespace, locates specific files, and performs content searches by posing appropriate queries to PlanetP. We describe this query-based design and present preliminary performance measurements of a prototype implementation.

The explosive growth of the Web contents has led to increasing attention on scalability and availability of file system. Hence, the ways to improve the reliability and availability of system, to achieve the expected reduction in operational expenses and to reduce the operations of management of system have become essential issues. A basic technique for improving reliability of a file system is to mask the effects of failures through replication. Consistency control protocols are implemented to ensure the consistency among replicas. In this paper, we leveraged the concept of intermediate file handle to cover the heterogeneity of file system and proposed an efficient data consistency control scheme supporting dependence checking among writes and management of out-of-ordered requests for file server group. Finally, the results of experiments proved the efficiency of the proposed consistency control mechanism. Above all, easy to implement is our main design consideration.

By merging network and channel interfaces, resulting interfaces allow multiple computers to physically share storage devices. A storage area network (SAN) is a high-speed special-purpose network (or subnetwork) that interconnects different kinds of data storage devices with associated data servers on behalf of a larger network of users. In SAN, computers service local file requests directly from shared storage devices. Direct device access eliminates the server machines as bottlenecks to performance and availability. Communication is unnecessary between computers, since each machine views the storage as being locally attached. SAN provides us to very large physical storage up to 64-bit address space, but traditional file systems can't adapt to the file system for SAN because they have the limitation of scalability. In this paper, we present architectures and features of SANtopia that allows multiple machines to access and share disk and tape devices on a Fibre Channel or SCSI storage network in Linux system. It performs well as a local file system, as a traditional network file system running over IP environments, and as a high performance cluster file system running over storage area networks like Fibre Channel. SANtopia provides a key cluster enabling technology for Linux, helping to bring the availability, scalability, and load balancing benefits of clustering to Linux.

Traditional high-performance computing (HPC) systems have independent job schedulers and file systems that do not interact in substantial ways. We make the case that some integration of scheduler and file system can have three main benefits. First, the dataflow dependencies between the jobs in a workflow can be inferred by combining the scheduler's knowledge of the jobs (and possibly control-flow) and the file system's knowledge of the files accessed. Second, the dataflow information can be used to improve workflow instance concurrency when there are (potential) filename conflicts. Third, when workflows need to be re-computed, only the affected jobs need to be re-executed. We present the design and a simulation study of the Workflow-Aware File System (WaFS). Our design layers a namespace manager (NM) on top of existing file systems to provide, for example, a dataflow engine and a versioned file system. Our simulation study (with a specific set of application parameters) shows that a combined WaFS-aware file system and scheduler can significantly improve makespans for intensive workloads and be efficient in the re-computation of jobs

The popularity of distributed file systems continues to grow. Reasons they are preferred over traditional centralized file systems include fault tolerance, availability, scalability and performance. In addition, peer-to-peer (P2P) system concepts and scalable functions are being incorporated into the domain of file systems. This survey paper explores the design paradigms and important issues that relate to such systems and discusses the various research activities in the field of distributed peerto-peer file systems.