Compiler Techniques for Massively Scalable Implicit Task Parallelism
|Title||Compiler Techniques for Massively Scalable Implicit Task Parallelism|
|Publication Type||Conference Paper|
|Year of Publication||2014|
|Authors||Armstrong, TG, Wozniak, JM, Wilde, M, Foster, IT|
|Conference Name||Supercomputing 2014|
|Conference Location||New Orleans, LA|
Swift/T is a high-level language for writing concise, deterministic scripts that compose serial or parallel codes implemented in lower-level programming models into large-scale parallel applications. It executes using a data-driven task parallel execution model that is capable of orchestrating millions of concurrently executing asynchronous tasks on homogeneous or heterogeneous resources. Producing code that efficiently executes at this scale requires sophisticated compiler transformations: poorly optimized code inhibits scaling with excessive synchronization and communication. We present a comprehensive set of compiler techniques for data-driven task parallelism, including novel compiler optimizations and intermediate representations. We report application benchmark studies, including unbalanced tree search and simulated annealing, and demonstrate that our techniques greatly reduce communication overhead and enable extreme scalability, distributing up to 1.47 billion dynamically load balanced tasks per second at scales of up to 524,288 cores without explicit parallelism, synchronization, or load balancing in application code.