JETS: Language and System Support for Many Parallel Task Computing
|Title||JETS: Language and System Support for Many Parallel Task Computing|
|Publication Type||Conference Paper|
|Year of Publication||2011|
|Authors||Wozniak, JM, Wilde, M|
|Conference Name||Proc. 40th International Conf. on Parallel Processing Workshops|
Many-task computing is a now well-established paradigm for implementing loosely coupled applications on large-scale computing systems. However, few of the model\'s existing implementations provide efficient, low-latency support for the execution of tightly coupled applications as atomic tasks. Thus, a vast array of parallel applications can not readily be used effectively within many-task workloads. In this work, we present JETS, a middleware component that provides high performance support for many-parallel-task-computing (MPTC). JETS is based on a highly concurrent approach to parallel task dispatch and on new capabilities now available in the MPICH2 MPI implementation and the ZeptoOS Linux operating system. JETS represents an advancement over the few known examples of multi-level many-parallel-task scheduling systems by more efficiently scheduling many short-duration parallel application invocations; by overcoming the challenges of coupling the user processes of each application invocation via the messaging fabric; and by concurrently managing many application executions in various stages. We report here on the JETS architecture and its performance on both synthetic benchmarks and the NAMD molecular dynamics application.