A New Object Based File Model To Improve Parallel I/O Performance

Abstract

Advanced data intensive scienti fic applications with thousands of processing nodes are used to execute very high resolution scienti c model. The I/O requirements of data intensive applications are still straining the I/O capabilities of even the largest, most powerful file systems in use today. Poor I/O performance has been a bottleneck in these applications. The main reason for this I/O bottleneck is the non-contiguous I/O access patterns exhibited by scienti c applications. Most of the le systems stored the le as linear sequences of bytes. The problem is that application processes rarely access their data in a way that matches this le model, and a signifi cant portion of the scalability problem is the high cost of dynamically translating between the process data model and the le data model at runtime. This thesis addresses the storage problem of application data in parallel le system's legacy view. We develop a new le model and required software infrastructure. With the help of MPI views, we create intervals and store those intervals as objects. We use the data layout awareness technique to distribute the objects on I/O servers. This provides contigu- ous, single I/O operations. Result shows the signi ficant improvement in read operations.