Compiler optimization for data-driven task parallelism on distributed memory systems
|Title||Compiler optimization for data-driven task parallelism on distributed memory systems|
|Publication Type||Conference Paper|
|Year of Publication||2013|
|Authors||Armstrong, TG, Wozniak, JM, Wilde, M, Foster, IT|
|Conference Name||19th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming|
|Conference Location||Orlando, FL|
The data-driven task parallelism execution model can support parallel programming models that are well suited for large-scale distributed-memory parallel computing, for example, simulations and analysis pipelines running on clusters and clouds. We describe a novel compiler intermediate representation and optimizations for this execution model, including adaptions of standard techniques alongside novel techniques. These techniques are applied to Swift/T, a high-level declarative language for flexible data flow composition of functions, which may be serial or use lower-level parallel programming models such as MPI and OpenMP. We show that our compiler optimizations reduce communication overhead by 70 to 93% on distributed memory systems. This makes Swift/T competitive in performance with lower-level message passing-based coordination logic for many applications, while offering developers a gentler learning curve and higher productivity.