Towards Simulation of Parallel File System Scheduling Algorithms with PFSsim

TitleTowards Simulation of Parallel File System Scheduling Algorithms with PFSsim
Publication TypeConference Paper
Year of Publication2010
AuthorsLiu, Y, Figueiredo, RJ, Clavijo, D, Xu, Y, Zhao, M
Conference Name5th Petascale Data Storage Workshop, Supercomputing’10
Date Published11/2010
Conference LocationNew Orleans, LA
KeywordsI/O scheduling, parallel file system, simulation
AbstractThis paper presents a new parallel storage management approach which supports the allocation of shared storage bandwidth on a per-application basis. Existing parallel storage systems are unable to differentiate I/Os from different applications and meet per-application bandwidth requirement. This limitation presents a hurdle for applications to achieve their desired performance, which will become even more challenging as high-performance computing (HPC) systems continue to scale up with respect to both the amount of available resources and the number of concurrent applications. This paper proposes a novel solution to address this challenge through the virtualization of parallel file systems (PFSes). Such PFS virtualization is achieved with user-level PFS proxies, which interpose between native PFS clients and servers and schedule the I/Os from different applications according to the resource sharing algorithm (e.g., SFQ(D)). In this way, virtual PFSes can be created on a per- application basis, each with a specific bandwidth share allocated according to its I/O requirement. This approach is applicable to different PFS-based parallel storage systems and can be transparently integrated with existing as well as future HPC systems. A prototype of this approach is implemented upon PVFS2, a widely used PFS, and evaluated with experiments using a typical parallel I/O benchmark (IOR). Results show that this approach’s overhead is very small while it can achieve very good proportional sharing under different usage scenarios.