7 The Global Map

7.1 Module m_GlobalMap - One-Dimensional Domain Decomposition Descriptor (Source File: m_GlobalMap.F90)

The GlobalMap is a datatype used to store descriptors of a one-dimensional domain decomposition for a vector on an MPI communicator. It is defined with three assumptions:

1. Each process ID owns only one segment;
2. No two segments in the decomposition overlap; and
3. The segments are laid out in identical order to the MPI rank of each process participating in the decomposition.

per process ID). It is the simpler of the two domain decomposition descriptors offerd by MCT (the other being the GlobalSegMap). It consists of the following components:

• The MCT component identification number (see the module m_MCTWorld for more information about MCT's component model registry);
• The global number of elements in the distributed vector;
• The number of elements stored locally;
• The number of elements stored on each process on the communicator over which the vector is distributed; and
• The index of the elemnent immediately before the starting element of each local segment (this choice allows for direct use of this information with MPI's scatter and gather operations). We refer to this quantity as the displacement of the segment, a term used both here and in the definition of the MCT Navigator datatype.

Both the segment displacement and length data are stored in arrays whose indices run from zero to $N-1$, where $N$ is the number of MPI processes on the communicator on which the GlobalMap is defined. This is done so this information corresponds directly to the MPI process ID's on whihc the segments reside.

This module contains the definition of the GlobalMap datatype, all-processor and an on-root creation methods (both of which can be used to create a GlobalMap on the local communicator), a creation method to create/propagate a GlobalMap native to a remote communicator, a destruction method, and a variety of query methods.

INTERFACE:

 
  module m_GlobalMap
 
   !USES
   No external modules are used in the declaration section of this module.
 
       implicit none
 
       private	! except

PUBLIC TYPES:

 
       public :: GlobalMap		! The class data structure
 
     Type GlobalMap
       integer :: comp_id                        ! Component ID number
       integer :: gsize				! the Global size
       integer :: lsize				! my local size
       integer,dimension(:),pointer :: counts	! all local sizes
       integer,dimension(:),pointer :: displs	! PE ordered locations
     End Type GlobalMap

PUBLIC MEMBER FUNCTIONS:

 
       public :: gsize
       public :: lsize
       public :: init
       public :: init_remote
       public :: clean
       public :: rank
       public :: bounds
       public :: comp_id
 
     interface gsize; module procedure gsize_; end interface
     interface lsize; module procedure lsize_; end interface
     interface init ; module procedure	&
        initd_,	&	! initialize from all PEs
        initr_		! initialize from the root
     end interface
     interface init_remote; module procedure init_remote_; end interface
     interface clean; module procedure clean_; end interface
     interface rank ; module procedure rank_ ; end interface
     interface bounds; module procedure bounds_; end interface
     interface comp_id ; module procedure comp_id_ ; end interface

SEE ALSO:

   The MCT module m_MCTWorld for more information regarding component 
   ID numbers.

REVISION HISTORY:

   21Apr98 - Jing Guo <guo@thunder> - initial prototype/prolog/code
    9Nov00 - J.W. Larson <larson@mcs.anl.gov> - added init_remote
             interface.
   26Jan01 - J.W. Larson <larson@mcs.anl.gov> - added storage for
             component ID number GlobalMap%comp_id, and associated
             method comp_id_()

7.1.1 initd_ - Collective Creation on the Local Communicator

This routine creates the GlobalMap GMap from distributed data spread across the MPI communicatior associated with the input INTEGER handle comm. The INTEGER input argument comp_id is used to define the MCT component ID for GMap. The input INTEGER argument ln is the number of elements in the local vector segment.

INTERFACE:

 
  subroutine initd_(GMap, comp_id, ln, comm)

USES:

 
       use m_mpif90
       use m_die
 
       implicit none

INPUT PARAMETERS:

 
       integer,         intent(in)  :: comp_id ! Component ID
       integer,         intent(in)  :: ln      ! the local size
       integer,         intent(in)  :: comm    ! f90 MPI communicator 
                                               ! handle

OUTPUT PARAMETERS:

 
       type(GlobalMap), intent(out) :: GMap

SEE ALSO:

   The MCT module m_MCTWorld for more information regarding component 
   ID numbers.

REVISION HISTORY:

   21Apr98 - Jing Guo <guo@thunder> - initial prototype/prolog/code

7.1.2 initr_ Create a GlobalMap from the Root Process

This routine creates the GlobalMap GMap, and propagates it to all processes on the communicator associated with the MPI INTEGER handle comm. The input INTEGER arguments comp_id (the MCT component ID number) and lns(:) need only be valid on the process whose rank is equal to root on comm. The array lns(:) should have length equal to the number of processes on comm, and contains the length of each local segment.

INTERFACE:

 
  subroutine initr_(GMap, comp_id, lns, root, comm)

USES:

 
       use m_mpif90
       use m_die
       use m_stdio
 
       implicit none

INPUT PARAMETERS:

 
       integer,               intent(in)  :: comp_id ! component ID number
       integer, dimension(:), intent(in)  :: lns     ! segment lengths
       integer,               intent(in)  :: root    ! root process ID
       integer,               intent(in)  :: comm    ! communicator ID

OUTPUT PARAMETERS:

 
       type(GlobalMap),       intent(out) :: GMap

SEE ALSO:

   The MCT module m_MCTWorld for more information regarding component 
   ID numbers.

REVISION HISTORY:

   29May98 - Jing Guo <guo@thunder> - initial prototype/prolog/code

7.1.3 init_remote_ Initialize Remote GlobalMap from the Root

This routine creates and propagates across the local communicator a GlobalMap associated with a remote component. The controlling process in this operation has MPI process ID defined by the input INTEGER argument my_root, and its MPI communinicator is defined by the input INTEGER argument my_comm. The input INTEGER argument remote_npes is the number of MPI processes on the remote component's communicator (which need be valid only on the process my_root). The input the INTEGER array remote_lns(:), and the INTEGER argument remote_comp_id need only be valid on the process whose rank on the communicator my_comm is my_root. The argument remote_lns(:) defines the vector segment length on each process of the remote component's communicator, and the argument remote_comp_id defines the remote component's ID number in the MCT component registry MCTWorld.

INTERFACE:

 
  subroutine init_remote_(GMap, remote_lns, remote_npes, my_root, &
                          my_comm, remote_comp_id)

USES:

 
       use m_mpif90
       use m_die
       use m_stdio
 
       implicit none

INPUT PARAMETERS:

 
       integer, dimension(:), intent(in)  :: remote_lns
       integer,               intent(in)  :: remote_npes
       integer,               intent(in)  :: my_root
       integer,               intent(in)  :: my_comm
       integer,               intent(in)  :: remote_comp_id

OUTPUT PARAMETERS:

 
       type(GlobalMap),       intent(out) :: GMap

SEE ALSO:

   The MCT module m_MCTWorld for more information regarding component 
   ID numbers.

REVISION HISTORY:

    8Nov00 - J.W. Larson <larson@mcs.anl.gov> - initial prototype
   26Jan01 - J.W. Larson <larson@mcs.anl.gov> - slight change--remote
             communicator is replaced by remote component ID number
             in argument remote_comp_id.

7.1.4 clean_ - Destroy a GlobalMap

This routine deallocates all allocated memory associated with the input/output GlobalMap argument GMap, and sets to zero all of its statically defined components. The success (failure) of this operation is signified by the zero (non-zero) value of the optional output INTEGER argument stat.

INTERFACE:

 
  subroutine clean_(GMap, stat)

USES:

 
       use m_die
 
       implicit none

INPUT/OUTPUT PARAMETERS:

 
       type(GlobalMap),           intent(inout) :: GMap

OUTPUT PARAMETERS:

 
       integer,         optional, intent(out)   :: stat

REVISION HISTORY:

   21Apr98 - Jing Guo <guo@thunder> - initial prototype/prolog/code
   26Jan01 - J. Larson <larson@mcs.anl.gov> incorporated comp_id.
    1Mar02 - E.T. Ong <eong@mcs.anl.gov> removed the die to prevent
             crashes and added stat argument.

7.1.5 lsize_ - Return Local Segment Length

This INTEGER function returns the length of the local vector segment as defined by the input GlobalMap argument GMap.

INTERFACE:

 
  integer function lsize_(GMap)

USES:

 
       implicit none

INPUT PARAMETERS:

 
       type(GlobalMap), intent(in) :: GMap

REVISION HISTORY:

   21Apr98 - Jing Guo <guo@thunder> - initial prototype/prolog/code

7.1.6 gsize_ - Return Global Vector Length

This INTEGER function returns the global length of a vector that is decomposed according to the input GlobalMap argument GMap.

INTERFACE:

 
  integer function gsize_(GMap)

USES:

 
       implicit none

INPUT PARAMETERS:

 
       type(GlobalMap), intent(in) :: GMap

REVISION HISTORY:

   21Apr98 - Jing Guo <guo@thunder> - initial prototype/prolog/code

7.1.7 rank_ - Process ID Location of a Given Vector Element

This routine uses the input GlobalMap argument GMap to determine the process ID (on the communicator on which GMap was defined) of the vector element with global index i_g. This process ID is returned in the output INTEGER argument rank.

INTERFACE:

 
  subroutine rank_(GMap, i_g, rank)

USES:

 
       implicit none

INPUT PARAMETERS:

 
       type(GlobalMap), intent(in)  :: GMap
       integer,         intent(in)  :: i_g

OUTPUT PARAMETERS:

 
       integer,         intent(out) :: rank

REVISION HISTORY:

    5May98 - Jing Guo <guo@thunder> - initial prototype/prolog/code

7.1.8 bounds_ - First/Last Global Indicies for a Process' Segment

This routine takes as input a process ID (defined by the input INTEGER argument pe_no), examines the input GlobalMap argument GMap, and returns the global indices for the first and last elements of the segment owned by this process in the output INTEGER arguments lbnd and ubnd, respectively.

INTERFACE:

 
  subroutine bounds_(GMap, pe_no, lbnd, ubnd)

USES:

 
       implicit none

INPUT PARAMETERS:

 
       type(GlobalMap), intent(in)  :: GMap
       integer,         intent(in)  :: pe_no

OUTPUT PARAMETERS:

 
       integer,         intent(out) :: lbnd
       integer,         intent(out) :: ubnd

REVISION HISTORY:

   30Jan01 - J. Larson <larson@mcs.anl.gov> - initial code

7.1.9 comp_id_ - Return the Component ID Number

This INTEGER query function returns the MCT component ID number stored in the input GlobalMap argument GMap.

INTERFACE:

 
  integer function comp_id_(GMap)

USES:

 
       implicit none

INPUT PARAMETERS:

 
       type(GlobalMap), intent(in) :: GMap

SEE ALSO:

   The MCT module m_MCTWorld for more information regarding component 
   ID numbers.

REVISION HISTORY:

   25Jan02 - J. Larson <larson@mcs.anl.gov> - initial version