The open archive for STFC research publications

ePubs will be undergoing scheduled maintenance on 19/06/2024 between 10:00-12:00 and the service will be unavailable during this time.

Full Record Details

Persistent URL http://purl.org/net/epubs/work/40804
Record Status Checked
Record Id 40804
Title On the Use of Parallel Input/Output in the High Performance Application Code, POLCOMS
Abstract As the problems addressed by High Performance Computational Science programs increase in size and complexity, the need for efficient input and output of datasets becomes an important factor in program design. Such datasets arise naturally as initial data and final results, but frequently programs save intermediate data during a long run to enable restarting the calculation in the event of a program or machine failure. Gathering all the data on one processor forms an inherent bottleneck, and for suitably large problems may exhaust the memory available to the master processor. We discuss the general problem of parallel I/O and the features of some of the available solutions. The checkpointing phase of a large High Performance scientific program, POLCOMS, is implemented in parallel using MPI-IO, a set of routines from the MPI-2 standard. The performance of MPI-IO is compared with that for gathering the data to one processor and writing it serially using a simple Fortran WRITE statement. We find that it is important to make sure the data representation being written out and the characteristics of access to the data are carefully selected, or serious performance degradation can be produced, as well as loss of flexibility. At its best, however, MPIIO is comparable with the naive process, and is capable of handling much larger problems.
Organisation CCLRC , CSE , CSE-HEC
Keywords CCP , parallel I/O , parallel input/output
Funding Information
Related Research Object(s):
Licence Information:
Language English (EN)
Type Details URI(s) Local file(s) Year
Report RAL Technical Reports RAL-TR-2006-029. 2006. polcoms-io.pdf 2006