Back to Search
Start Over
PLFS
- Source :
- SC
- Publication Year :
- 2009
- Publisher :
- ACM, 2009.
-
Abstract
- Parallel applications running across thousands of processors must protect themselves from inevitable system failures. Many applications insulate themselves from failures by checkpointing. For many applications, checkpointing into a shared single file is most convenient. With such an approach, the size of writes are often small and not aligned with file system boundaries. Unfortunately for these applications, this preferred data layout results in pathologically poor performance from the underlying file system which is optimized for large, aligned writes to non-shared files. To address this fundamental mismatch, we have developed a virtual parallel log structured file system, PLFS. PLFS remaps an application’s preferred data layout into one which is optimized for the underlying file system. Through testing on PanFS, Lustre, and GPFS, we have seen that this layer of indirection and reorganization can reduce checkpoint time by an order of magnitude for several important benchmarks and real applications without any application modification.
- Subjects :
- FOS: Computer and information sciences
File system
Computer science
Computer file
Distributed computing
Device file
computer.file_format
Everything is a file
computer.software_genre
Unix file types
Virtual file system
Torrent file
File Control Block
Self-certifying File System
Journaling file system
Data file
Data_FILES
Operating system
Versioning file system
89999 Information and Computing Sciences not elsewhere classified
Fork (file system)
computer
File system fragmentation
Subjects
Details
- Database :
- OpenAIRE
- Journal :
- Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
- Accession number :
- edsair.doi.dedup.....0fe08c6c11e778ba32658d6df6e49416