Argonne National Laboratory

Compiler optimization for data-driven task parallelism on distributed memory systems

TitleCompiler optimization for data-driven task parallelism on distributed memory systems
Publication TypeConference Paper
Year of Publication2013
AuthorsArmstrong, TG, Wozniak, JM, Wilde, M, Foster, IT
Conference Name19th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
Conference LocationOrlando, FL
Other NumbersANL/MCS-P5030-1013

The data-driven task parallelism execution model can support parallel programming models that are well suited for large-scale distributed-memory parallel computing, for example, simulations and analysis pipelines running on clusters and clouds. We describe a novel compiler intermediate representation and optimizations for this execution model, including adaptions of standard techniques alongside novel techniques. These techniques are applied to Swift/T, a high-level declarative language for flexible data flow composition of functions, which may be serial or use lower-level parallel programming models such as MPI and OpenMP. We show that our compiler optimizations reduce communication overhead by 70 to 93% on distributed memory systems. This makes Swift/T competitive in performance with lower-level message passing-based coordination logic for many applications, while offering developers a gentler learning curve and higher productivity.