PyDarshan Documentation
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with PyDarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan
# Open darshan log
with darshan.DarshanReport('example.darshan', read_all=False) as report:
# Load some report data
report.mod_read_all_records('POSIX')
report.mod_read_all_records('MPI-IO')
# or fetch all
report.read_all_generic_records()
# ...
# Generate summaries for currently loaded data
# Note: aggregations are still experimental and have to be activated:
darshan.enable_experimental()
report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt
make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing PyDarshan usage with log files
- tests::
PyDarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating PyDarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing