CACHE: Communication Avoidance and Communication Hiding at the Extreme Scale

Efficient optimization algorithm makes large-scale auto-tuning feasible.

Efficient optimization algorithm makes large-scale auto-tuning feasible.

The goal of CACHE (Communication Avoidance and Communication Hiding at the Extreme Scale) is to have applied mathematicians and computer scientists collaborate to reduce the two major obstacles to productively using future extreme scale computer architectures:

  • The cost of communication and
  • The effort required to write efficient programs on these increasingly large and complicated machines.

Communication means moving data (either between levels of a memory hierarchy or between processors on a network) and synchronization. It is important to minimize both bandwidth costs (proportional to the number of words moved) and latency costs (proportional to the number of messages, or packets of contiguous data).

Mathematical innovation is needed to develop new algorithms that do less communication than current algorithms, which were mostly designed to minimize arithmetic operations rather than communication (sometimes minimizing the two goes together, but not always).

Computer science innovation is needed to make programming these more complicated algorithms easier on more complicated and possibly heterogeneous architectures, as well as more easily finding the fastest implementation out of the combinatorial explosion of possibilities.