Compiler Techniques for Massively Scalable Implicit Task Parallelism

TitleCompiler Techniques for Massively Scalable Implicit Task Parallelism
Publication TypeConference Paper
Year of Publication2014
AuthorsArmstrong, TG, Wozniak, JM, Wilde, M, Foster, IT
Conference NameSupercomputing 2014
Conference LocationNew Orleans, LA
Other NumbersANL/MCS-P5131-0414

Swift/T is a high-level language for writing concise, deterministic scripts that compose serial or parallel codes implemented in lower-level programming models into large-scale parallel applications. It executes using a data-driven task parallel execution model that is capable of orchestrating millions of concurrently executing asynchronous tasks on homogeneous or heterogeneous resources. Producing code that efficiently executes at this scale requires sophisticated compiler transformations: poorly optimized code inhibits scaling with excessive synchronization and communication. We present a comprehensive set of compiler techniques for data-driven task parallelism, including novel compiler optimizations and intermediate representations. We report application benchmark studies, including unbalanced tree search and simulated annealing, and demonstrate that our techniques greatly reduce communication overhead and enable extreme scalability, distributing up to 1.47 billion dynamically load balanced tasks per second at scales of up to 524,288 cores without explicit parallelism, synchronization, or load balancing in application code.