We are currently in the process of migrating all Mochi source repositories from https://xgitlab.cels.anl.gov/sds to their new home at https://github.com/mochi-hpc. We will be archiving repositories at xgitlab (marking them read only) as each is migrated. Please open issues and pull requests at the new github location moving forward!
carns
Quarterly newsletter, January 2021
Platform notes:
- If you are using Summit (or another system with a power architecture) then you should use argobots@main as a safety precaution to avoid a potential performance regression for now (https://lists.argobots.org/pipermail/discuss/2021-January/000094.html)
- If you are using a system with an OmniPath (PSM2) network fabric, then be aware that we are also chasing a performance regression there that can occur with multithreaded access (https://xgitlab.cels.anl.gov/sds/margo/-/issues/67)
- Possibly a problem with libpsm2 fairness with multithreaded access, but not confirmed
Logistics:
- All of the ANL-hosted git repositories (https://xgitlab.cels.anl.gov/sds/) will be moving within the next few months. We will communicate when that happens.
- There are no changes to policy or access (in fact, access will likely change for the better); it’s just that the xgitlab.cels instance that we are using is being decommissioned
- We are working on landing a Spack PR (https://github.com/spack/spack/pull/20273) that will introduce a “mochi-margo” package, maintained by us, to replace the out-of-date “margo” package
- Once this is done, we will likely start upstreaming more packages that depend on margo-mochi
Mochi service development news:
- Work continues on a new component called “Bedrock” that can be used to more easily bootstrap microservice compositions (https://xgitlab.cels.anl.gov/sds/bedrock).
- Bedrock is already available, and we are in the process of updating existing services to use it.
- You can think of bedrock as a general-purpose Mochi daemon that takes a JSON configuration file describing how to spin up embedded microservices
- We are actively working on performance tuning of “Benvolio”, which you can think of as a runtime I/O delegation service (i.e. that provides a more generic version of MPI-IO aggregation capabilities). https://xgitlab.cels.anl.gov/sds/benvolio
Upcoming training events:
- We plan to host a BoF session at this year’s (virtual) ECP annual meeting
in mid-April, with a mechanism for people to sign up for one on one
sessions for more detailed interaction. (https://ecpannualmeeting.com/) - Please let us know what other kinds of outreach/training you are interested in this year.
Mochi at SC 2020
Highlighted events for the Mochi project at this year’s SC conference included the following:
- Pascal Grosset, Jesus Pulido, and James Ahrens. 2020. “Personalized In Situ Steering for Analysis and Visualization,” in ISAV’20 In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV’20). Association for Computing Machinery, New York, NY, USA, 1–6. DOI:https://doi.org/10.1145/3426462.3426463
- Christopher Kelly, Sungsoo Ha, Kevin Huck, Hubertus Van Dam, Line Pouchard, Gyorgy Matyasfalvi, Li Tang, Nicholas D’Imperio, Wei Xu, Shinjae Yoo, and Kerstin Kleese Van Dam. 2020. “Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool,” in ISAV’20 In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV’20). Association for Computing Machinery, New York, NY, USA, 15–19. DOI:https://doi.org/10.1145/3426462.3426465
- P. Carns, K. Harms, B. Settlemyer, B. Atkinson and R. Ross, “Keeping It Real: Why HPC Data Services Don’t Achieve I/O Microbenchmark Performance,” in 2020 IEEE/ACM Fifth International Parallel Data Systems Workshop (PDSW), GA, USA, 2020 pp. 1-6. doi: 10.1109/PDSW51947.2020.00006
Mochi at the 2020 ECP Annual Meeting
The Mochi team at ANL presented a full day tutorial entitled “Customizing Data Services for Fun and Profit” at the 2020 Exascale Computing Project Annual Meeting. You can find materials from this session on the tutorials tab on this page or in the Mochi bootcamp git repository.
Mochi at SC 2019
Highlighted events for the Mochi project at this year’s Supercomputing included:
- Srinivasan Ramesh, Philip Carns, Robert Ross, Shane Snyder, and Allen Malony. “Profiling Composable HPC Data Services”, Work-In-Progress report at the 4th International Parallel Data Systems Workshop (PDSW 2019). LINK (paper) LINK (slides)
- Jerome Soumagne et al. “Data Services for High Performance Computing”, Birds of a Feather session. LINK
2019 Mochi Boot Camp
Thank you to the attendees of our first ever Mochi Boot Camp, held at Argonne National Laboratory, September 24-26 2019. All presentation materials and exercises are available online. The purpose of the boot camp was to introduce the Mochi project through tutorials, hands-on exercises, and in-person assistance in the design and development of data services.
Updated Mochi documentation
Please check out our new Mochi Read the Docs page! This is the official source for all Mochi documentation. We will be expanding and updating material on that page over time.
Mochi at SC18
Please stop by and check out the following events featuring Mochi technology at SC18!
- Qing Zheng et al. “Scaling Embedded In Situ Indexing with DeltaFS”, Technical program presentation.
- Matthieu Dorier et al. “Methodology for the Rapid Development of Scalable HPC Data Services”, PDSW workshop presentation.
- Jerome Soumagne et al. “Enabling Data Services for HPC”, Birds of a Feather session.
mochi-devel mailing list
Please sign up for the mochi-devel mailing list if you would like to participate in technical discussions, ask questions, or keep up to date with changes in the Mochi software packages.