Argonne National Laboratory

JETS: Language and System Support for Many Parallel Task Computing

TitleJETS: Language and System Support for Many Parallel Task Computing
Publication TypeConference Paper
Year of Publication2011
AuthorsWozniak, JM, Wilde, M
Conference NameProc. 40th International Conf. on Parallel Processing Workshops
Date Published09/2011
Other NumbersANL/MCS-P1885-0411

Many-task computing is a now well-established paradigm for implementing loosely coupled applications on large-scale computing systems. However, few of the model\'s existing implementations provide efficient, low-latency support for the execution of tightly coupled applications as atomic tasks. Thus, a vast array of parallel applications can not readily be used effectively within many-task workloads. In this work, we present JETS, a middleware component that provides high performance support for many-parallel-task-computing (MPTC). JETS is based on a highly concurrent approach to parallel task dispatch and on new capabilities now available in the MPICH2 MPI implementation and the ZeptoOS Linux operating system. JETS represents an advancement over the few known examples of multi-level many-parallel-task scheduling systems by more efficiently scheduling many short-duration parallel application invocations; by overcoming the challenges of coupling the user processes of each application invocation via the messaging fabric; and by concurrently managing many application executions in various stages. We report here on the JETS architecture and its performance on both synthetic benchmarks and the NAMD molecular dynamics application.