NFS and MPI-IO


Up: Running an MPI Program Next: Thorough Testing Previous: Special Considerations for Running with Shared Memory

To use MPI-IO multihost on NFS file systems, NFS should be version 3, and the shared NFS directory must be mounted with the ``no attribute caching'' (noac) option set (the directory cannot be automounted). If NFS is not mounted in this manner, the following error could occur:

    MPI_Barrier: Internal MPI error: No such file or directory 
    File locking messages 
In order to reconfigure NFS to handle MPI-IO properly, the following sequence of steps are needed (root permission required):
    1. confirm you are running NFS version 3
        rpcinfo -p `hostname` | grep nfs 
     
        for example, there should be a '3' in the second column 
        fire >rpcinfo -p fire | grep nfs 
        100003    3   udp   2049  nfs 
    

    2. edit /etc/fstab for each NFS directory read/written by MPI-IO on each machine used for multihost MPI-IO. The following is an example of a correct fstab entry for /epm1:
        root >grep epm1 /etc/fstab 
        gershwin:/epm1 /rmt/gershwin/epm1 nfs bg,intr,noac 0 0 
    
    If the ``noac'' option is not present, add it and then remount this directory on each of the machines that will be used to share MPI-IO files.
        root >umount /rmt/gershwin/epm1 
        root >mount /rmt/gershwin/epm1 
    

    3. confirm that the directory is mounted noac
        root >grep gershwin /etc/mnttab 
        gershwin:/epm1 /rmt/gershwin/epm1 nfs 
        noac,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0 0 0 899911504 
    
Turning off of attribute caching may reduce performance of MPI-IO applications as well as other applications using this directory. The load on the machine where the NFS directory is hosted will increase.



Up: Running an MPI Program Next: Thorough Testing Previous: Special Considerations for Running with Shared Memory