darshan.backend package
The backend package provides implementions for reading the binary representation of the darshan log file. The only current implementation uses CFFI which uses the functions defined in libdarshan-util.so to read the log.
Submodules
darshan.backend.api_def_c module
The api_def_c carries a copy of CFFI compatible headers for libdarshan-util.so. These definitions must match the structure definitions for the associated darshan release.
darshan.backend.cffi_backend module
The cfii_backend package will read a darshan log using the functions defined in libdarshan-util.so and is interfaced via the python CFFI module.
- darshan.backend.cffi_backend._df_to_rec(rec_dict, mod_name, rec_index_of_interest=None)[source]
Pack the DataFrames-format PyDarshan data back into a C buffer of records that can be consumed by darshan-util C code.
Parameters
- rec_dict: dict
Dictionary containing the counter and fcounter dataframes.
- mod_name: str
Name of the darshan module.
- rec_index_of_interest: int or None
If
None, use all records in the dataframe. Otherwise, repack only the the record at the provided integer index.
Returns
buf: Raw char array containing a buffer of record(s) or a single record.
- darshan.backend.cffi_backend._log_get_heatmap_record(log)[source]
Returns a dictionary holding a heatmap darshan log record.
- Parameters:
log – Handle returned by darshan.open
- Returns:
heatmap log record
- Return type:
- darshan.backend.cffi_backend._log_get_lustre_record(log, dtype='numpy')[source]
Returns a darshan log record for Lustre.
- Parameters:
log – handle returned by darshan.open
- darshan.backend.cffi_backend._make_generic_record(rbuf, mod_name, dtype='numpy')[source]
Returns a record dictionary for an input record buffer for a given module.
- darshan.backend.cffi_backend.accumulate_records(rec_dict, mod_name, nprocs)[source]
Passes a set of records (in pandas format) to the Darshan accumulator interface, and returns the corresponding derived metrics struct and summary record.
- Parameters:
rec_dict – Dictionary containing the counter and fcounter dataframes.
mod_name – Name of the Darshan module.
nprocs – Number of processes participating in accumulation.
- Returns:
namedtuple containing derived_metrics (cdata object) and summary_record (dict).
- darshan.backend.cffi_backend.counter_names(mod_name, fcnts=False, special='')[source]
Returns a list of available counter names for the module. By default only integer counter names are listed, unless fcnts is set to true in which case only the floating point counter names are listed.
- darshan.backend.cffi_backend.fcounter_names(mod_name)[source]
Returns a list of available floating point counter names for the module.
- darshan.backend.cffi_backend.get_lib_version()[source]
Return the version information hardcoded into the shared library.
- Parameters:
None –
- Returns:
library version number
- Return type:
version (str)
- darshan.backend.cffi_backend.log_close(log)[source]
Closes the logfile and releases allocated memory.
- darshan.backend.cffi_backend.log_get_dxt_record(log, mod_name, reads=True, writes=True, dtype='dict')[source]
Returns a dictionary holding a dxt darshan log record.
- Parameters:
- Returns:
generic log record
- Return type:
Example:
The typical darshan log record provides two arrays, on for integer counters and one for floating point counters:
>>> darshan.log_get_dxt_record(log, "DXT_POSIX", "struct dxt_file_record **") {'rank': 0, 'read_count': 11, 'read_segments': array([...]), ...}
- darshan.backend.cffi_backend.log_get_exe(log)[source]
Get details about the executable (path and arguments)
- Parameters:
log – handle returned by darshan.open
- Returns:
executable path and arguments
- Return type:
string
- darshan.backend.cffi_backend.log_get_generic_record(log, mod_name, dtype='numpy')[source]
Returns a dictionary holding a generic darshan log record.
- Parameters:
log – Handle returned by darshan.open
mod_name (str) – Name of the Darshan module
- Returns:
generic log record
- Return type:
Example:
The typical darshan log record provides two arrays, on for integer counters and one for floating point counters:
>>> darshan.log_get_generic_record(log, "POSIX", "struct darshan_posix_file **") {'counters': array([...], dtype=int64), 'fcounters': array([...])}
- darshan.backend.cffi_backend.log_get_job(log)[source]
Returns a dictionary with information about the current job.
- darshan.backend.cffi_backend.log_get_modules(log)[source]
Return a dictionary containing available modules including information about the contents available for each module in the current log.
- Parameters:
log – handle returned by darshan.open
- Returns:
Modules with additional info for current log.
- Return type:
- darshan.backend.cffi_backend.log_get_mounts(log)[source]
Returns a list of available mounts recorded for the log.
- Parameters:
log – handle returned by darshan.open
- darshan.backend.cffi_backend.log_get_name_records(log)[source]
Return a dictionary resovling hash to string (typically a filepath).
- Parameters:
log – handle returned by darshan.open
hash – hash-value (a number)
- Returns:
the name records
- Return type:
- darshan.backend.cffi_backend.log_get_record(log, mod, dtype='numpy')[source]
Standard entry point fetch records via mod string.
- Parameters:
log – Handle returned by darshan.open
mod_name (str) – Name of the Darshan module
- Returns:
log record of type dtype
- darshan.backend.cffi_backend.log_lookup_name_records(log, ids=[])[source]
Resolve a single hash to it’s name record string (typically a filepath).
- Parameters:
log – handle returned by darshan.open
hash – hash-value (a number)
- Returns:
the name records
- Return type: