bginfo.h File Reference

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include "rts.h"

Go to the source code of this file.

Classes
struct	bginfo_rank_ip_pair
	Represents associated ION to rank. More...
struct	bginfo_ion_vec
	Holds IP address of an ION and associated compute nodes to this ION. More...
struct	bginfo_struct
	Holds all necessary information of bginfo. More...
Relatively generic functions
#define	bginfo_perror(msg)   _bginfo_perror((msg),__LINE__,__FILE__)
int	bginfo_ion_proc_readline (const char *fn, char *buf, int len)
int	bginfo_get_valstr_from_file (const char *fn, const char *key, char *valstr, size_t valstrsize)
void	_bginfo_perror (const char *msg, int lno, char *fn)
For a quick usage or demo
void	bginfo_print_info_full ()
	Prints BG/L information in the key=val style with the shell style comment.
void	bginfo_print_info ()
	Prints BG/L information in the key=val style with the shell style comment.
Initialization and raw-level access
int	bginfo_init ()
	Initializes bginfo library.
bginfo_struct *	bginfo_get_struct ()
Wrapper functions for rts library.
These function reads personality data on a compute node where this function is called using rts library functions. They are simply a wrapper function of BGLPersonality_* family defined in rts.h and bglpersonality.h at /bgl/BlueLight/ppcfloor/bglsys/include/. struct bginfo holds an instance of BGLPersonality which is initialized at bgl_init(). bginfo doesn't define wrapper functions for all functions in bglpersonality.h
int	bginfo_is_virtualnodemode ()
int	bginfo_num_computenodes ()
int	bginfo_num_ionodes ()
size_t	bginfo_memorysize_per_node ()
Defines
#define	BGINFO_VERSION   "0.6"
Functions
int	bginfo_driver_version (char *buf, int len)
	Gets IBM BG/L driver version as a string and copies it to buf.
int	bginfo_zepto_version (char *buf, int len)
int	bginfo_personality_get_valstr (const char *key, char *valstr, size_t valstrsize)
int	bginfo_uneven_ci_ratio ()
int	bginfo_ci_ratio ()
int	bginfo_n_ions ()
char *	bginfo_ion_ipstr (int ion_idx)
int	bginfo_find_ion (int rank)
const char *	bginfo_bglmpi_mapping ()

---

Detailed Description

bginfo is a LGPL licensed library that provides you information such as version, setting or configuration corresponding to BG/L, which are detected (mostly) at runtime.

This is very helpful when you want to keep your computational results with the current machine setting so that you can find the performance differences between each driver. Runtime status such as running mode(virtual or co-processor), number of CNs, number of IONs or version of installed software can be a factor that influences performance of your program. It's always a good idea to record the BG/L runtime information with each result so that you can compare the status between each experiment.

bginfo also provides the map of IO node and compute nodes. This mapping capablity give you an answer to your question like "what is the rank N's ION?", "which CNs are belonging to this ION?", "which CN is belonging to which ION?", etc. etc. With the CN-ION mapping capability, you can I/O benchmark against a specific set of IONs, instead of splitting by rank.

To understand this library, bginfo_print_info() in bginfo.c is a good start.

The current version of bginfo provides

• BGINFO library version
• Current date&time(UTC)
• Zepto version
• ION linux kernel version
• IBM BG/L version
• Personality version
• Processor mode(virtual or coprocessor)
• Block ID
• ION ethernet MTU
• CI(ComputeNode/IONode) ratio
• Uneveness of CI ratio
• The number of IO nodes
• IP address of each IO Node
• Th number of CNs on each I/O node
• List of CNs on each I/O node
• MPI MAPPING on virtual node mode
• TORUS size
• PSET size
• Global Interrupts
• Memory size per compute node

bginfo.c has main() function for testing. you can compile a test code like

$ mpicc -Wall -D__TEST_MAIN__ bginfo.c -o bginfo

and sumbit

$ cqsub -t 10 -n 32 -m co -O bginfo-32-co ./bginfo
$ cqsub -t 10 -n 33 -m co -O bginfo-33-co ./bginfo
$ cqsub -t 10 -n 64 -m vn -O bginfo-64-vn ./bginfo
$ cqsub -t 10 -n 1024 -m vn -O bginfo-1024-vn -e BGLMPI_MAPPING=TXYZ ./bginfo

Then check the output file (i.e. bginfo-32-co.output)

NOTE: I hope bginfo would work with a big partition but I didn't try bginfo on more than 128 nodes.

Copyright: Argonne National Laboratory, Department of Energy, and University of Chicago. 2004, 2005 and 2006

This is a GPL licensed software. See the file COPYING for complete details on your rights to copy, modify, and use this software.

[Version History]

* Fri Jan 13 2006 - Kazutomo Yoshii <[email protected]>
v0.1 - Initial Version. Basic functionalities are implemented.

* Mon Jan 16 2006 - Kazutomo Yoshii <[email protected]>
v0.2 - Added function for BG/L driver and Zepto version and get the status of mode(vn or co). 0.2 uses librts

* Wed Jan 18 2006 - Kazutomo Yoshii <[email protected]>
v0.3 - Torus,pset,gint and memorysize report funcs are added

* Thu Jan 19 2006 - Kazutomo Yoshii <[email protected]>
v0.4 - bginfo_bglmpi_mapping(), some function name cleanup

* Thu Jan 26 2006 - Kazutomo Yoshii <[email protected]>
v0.5 - print memory size by default

* Mon Jan 30 2006 - Kazutomo Yoshii <[email protected]>
v0.6 - bginfo_driver_version() fix for V1R2M1 driver

---

Define Documentation

#define BGINFO_VERSION   "0.6"

bginfo library version

---

Function Documentation

const char* bginfo_bglmpi_mapping (   )

Reports the current MPI mapping on virtual node mode. Returns: A pointer to a data string "XYZT" or "TXYZ" is returned. On error or co-processor mode, NULL is returned.

int bginfo_ci_ratio (   )

Returns: the ratio of IONs to CNs. On error, -1 is returned.

int bginfo_driver_version (  char *  buf, int  len )

Gets IBM BG/L driver version as a string and copies it to buf. Parameters: buf  driver version will be stored len  length of buf Returns: On success, the number of bytes copied to buf is returend. On error , -1 is returned.

int bginfo_find_ion (  int  rank  )

Finds ION by a given processor rank and return it as ION index which has number between 0 to (bginfo_n_ions()-1). Returns: On error, -1 is returned.

bginfo_struct* bginfo_get_struct (   )

Returns: a raw pointer to bginfo object. See also bginfo_struct. On error, return NULL

int bginfo_get_valstr_from_file (  const char *  fn, const char *  key, char *  valstr, size_t  valstrsize )

Finds a value associated with key from a specified file. Returns: On success, the number of bytes is stored to valstr is returned. On error, -1 is returned.

int bginfo_init (   )

Initializes bginfo library. Returns: 0 is returned on success. -1 is returned on error. Note that: this uses collective operation, so it needs to be called from all nodes.

char* bginfo_ion_ipstr (  int  ion_idx  )

Converts ION index to an IP address string. Returns: On error, NULL is returned. ION index has number between 0 to (bginfo_n_ions()-1). The string is returned in a statically allocated buffer in this function. So the buffer will be overwrittened.

int bginfo_ion_proc_readline (  const char *  fn, char *  buf, int  len )

Reads a line from a specifed file. The new-line charater at the end of buffer will be eliminated. Returns: On sucess, the number of bytes copied to buf is returned. It returns -1 or negative number on error.

int bginfo_is_virtualnodemode (   )

Returns: 1 when it's in virtual node mode. Otherwise, it returns 0.

size_t bginfo_memorysize_per_node (   )

Returns: the memory size on a compute node. The memory size of a node in virtual mode is the half of a node in coprocessor mode.

int bginfo_n_ions (   )

Returns: the number of IONs. On error, -1 is returned.

int bginfo_num_computenodes (   )

Returns: the number of compute nodes.

int bginfo_num_ionodes (   )

Returns: the number of I/O nodes.

int bginfo_personality_get_valstr (  const char *  key, char *  valstr, size_t  valstrsize )

Finds a value associated to key from /proc/personality.sh and copies the value to valstr. Returns: On success, the number of bytes is stored to valstr is returned. On error, -1 is returned.

void bginfo_print_info (   )

Prints BG/L information in the key=val style with the shell style comment. bginfo_print_info() is same as bginfo_print_info_full() except this prints less information.

void bginfo_print_info_full (   )

Prints BG/L information in the key=val style with the shell style comment. NOTE: this functions involes bginfo_init() in it, so you don't need to call bginfo_init() by yourself. This also means that you need to call this function from all nodes.

int bginfo_uneven_ci_ratio (   )

Checks whether all ION has the even number of CNs. Returns: 1 when the ratio is uneven between IONs. If it's even, it returns 0. In most case, it returns 0. On error, -1 is returned.

int bginfo_zepto_version (  char *  buf, int  len )

Gets Zepto version and copies it to buf. Parameters: buf  version string will be stored to buf len  the length of buf Returns: On success, the number of bytes copied to buf is returend.* On error , -1 is returned.

---