Collective operations

May be implemented with MPI point-to-point
Implementations can optimize for small (latency), large (bandwidth), or both.
Generality of some MPI collective operations can limit performance
- Routines must assume datatypes are general and non-contiguous
- Time/memory tradeoffs (for internal temporary buffers)