The goal of the CODES project is use highly parallel simulation to explore the design of exascale storage architectures and distributed data-intensive science facilities.
Increasingly, science endeavors rely heavily on data management, analysis, and storage as part of the discovery process. To serve large communities of scientists, complex systems and instruments are deployed across multiple institutions to manage and analyze data produced from experiments, observational platforms, and computational simulation. Evaluating designs and coordinating deployment and operation of such a virtual data facility poses a significant challenge. An ability to simulate these environments would transform the approach taken to design, procurement, tuning, and upgrade of these facilities.
Our simulations build upon the Rensselaer Optimistic Simulation System (ROSS), a discrete event simulation framework that allows simulations to be run in parallel, decreasing the simulation run time of massive simulations to hours. We are using ROSS to explore topics including large-scale storage systems, I/O workloads, HPC network fabrics, distributed science systems, and data-intensive computation environments.
The CODES project is a collaboration between the Mathematics and Computer Science department at Argonne National Laboratory and Rensselaer Polytechnic Institute. We also collaborate with Lawrence Livermore National Laboratory for modeling HPC interconnect systems.
Thanks everyone for attending the 3rd summer of CODES workshop and making it a success! This year we had 28 participants from 10 different institutions. We had a number of amazing talks by the users on their work with CODES, ROSS and TraceR and on day 2, we had a tutorial about using CODES/TraceR. The slides have been posted at https://press3.mcs.anl.gov/summerofcodes2017/workshop-proceedings/
We are happy to announce the recent release of CODES version 0.6.0! This release comes after an year since the last release so there have been significant changes and additions. We are listing the major changes here (See docs/RELEASE_NOTES for full list of changes):
- C++ models can now be built and integrated with CODES. The new dragonfly model
has been implemented in C++.
- CODES can now replay collective operations by using CoRTex — a library for translating collectives to point to point operations.
- Dragonfly network model based on Cray XC topology has been added. The model can
use the network configurations of Theta and Edison systems. Custom network
configurations can be generated using C scripts.
- Fat tree network model with support for adaptive and static routing has been
added. The model can support both full and pruned fat tree configurations.
- Test suite has been extended — tests for DUMPI trace replay have been added.
- MPI rendezvous protocol can now be replayed in addition to the eager protocol.
The transition point for switching between the two protocols is configurable.
- Background network communication using uniform random workload can now be
generated. The traffic generation gets automatically shut off when the main workload
- Compatible with the most recent ROSS version that has GVT/real time sampling enabled.
The release can be downloaded from http://ftp.mcs.anl.gov/pub/CODES/releases/codes-0.6.0.tar.gz
We had a number of CODES related activities at Supercomputing this year! The research work presented used the CODES/ROSS framework to evaluate HPC network and storage systems, study application interference, enable efficient collective communication, and perform visual analysis of simulations. Details can be found at CODES@SC16.
Thanks everyone for attending this year’s Summer of CODES workshop! We had a great attendance and excellent talks. Slides will be posted in the coming days as I receive them, at the workshop website, http://press3.mcs.anl.gov/summerofcodes2016/.
I am happy to announce the release of CODES 0.5.2! This release, prepared during this year’s Summer of CODES workshop (post incoming), is primarily a bug-fix release that includes a few minor API additions and updates to the latest ROSS API. Full release notes can be found at doc/RELEASE_NOTES in the release tarball.
Downloads can be found at http://www.mcs.anl.gov/projects/codes/downloads/.
We are happy to announce the release of CODES 0.5.1! This release is a hotfix for latency calculation on the dragonfly, along with a few trivia missed in the past release. Downloads can be found at http://www.mcs.anl.gov/projects/codes/downloads/.
We are happy to announce the release of CODES 0.5.0! It’s been a full year since the last content release and there have been a massive set of changes! See doc/RELEASE_NOTES for a full list – this post covers a few of the major ones. Downloads can be found at http://www.mcs.anl.gov/projects/codes/downloads/
- codes-base and codes-net have been merged, greatly simplifying the build process. The new repository is at https://xgitlab.cels.anl.gov/codes/codes.
- we’ve added a Slim Fly network topology simulator, corresponding to the Wolfe et al. paper “Modeling a Million-node Slim Fly Network using Parallel Discrete-event Simulation” at SIGSIM-PADS’16.
- the dragonfly and torus networks have seen many improvements, including updates to the credit-based flow control mechanism, in-depth data gathering on both terminals and routers, and the ability to periodically sample the terminal/router states for gathering time-series data.
- workload processing (and the MPI simulation layer) have been greatly enhanced, improving task to LP mapping flexibility and allowing concurrently running workloads.
- a number of new APIs, including a “mapping context” construct for building more flexible implicit LP->LP mappings (now used in model-net, the local storage LP, and the resource LP) and a mechanism for RPC/callback-oriented event control flow (see the resource LP API and implementation for how this works).
Feedback is welcome!
CODES 0.4.1 has been released! This is a hotfix release that fixes an incompatibility with recent ROSS revisions. The new version can be found at http://www.mcs.anl.gov/research/projects/codes/downloads/ .
The slides for (most of) the Summer of CODES 2015 talks have been made available. Check them out over at http://press3.mcs.anl.gov/summerofcodes2015/workshop-proceedings/.
Thanks to all that attended the Summer of CODES 2015 Workshop! We had a strong first showing and a full schedule of engaging talks. We will post slides onto the workshop website (http://press3.mcs.anl.gov/summerofcodes2015/) in the coming days.