Collaboratory Interoperability Framework

DOE 2000 - Collaboratories Research

FY 1998 Highlights

1. Project Title

Collaboratory Interoperability Framework

2. Project Homepage

    http://www.mcs.anl.gov/cif

And see also:
    http://www-itg.lbl.gov/CIF/
    http://www.emsl.pnl.gov:2080/docs/collab/research/  

3. Objective

The goal of the Collaboratory Interoperability Framework (CIF) project is to develop a distributed computing software bus that supports software components for scientific collaboratories developed by diverse DoE and non DoE groups.  The goal of the project is to reduce duplication of effort, enhance interoperability and promote interlab cooperation by developing a common communication library for all projects.

4. Approach

A requirements analysis conducted prior to starting the project led to the following initial goals:

• Robust implementations of reliable multicast
• Communication service providing uniform interfaces to unicast and multicast communication
• Directory services and resource location services
• Integration of authentication and authorization services
• Investigations of CORBA and Java-based technologies

In addition, a number of application experiments are being conducted to refine further our understanding of requirements.

5. Selected FY98 accomplishments

Accomplishments during FY98 ranged from development of CIF toolkit components (CIFcomm, CIF shared state) to exploratory investigations of technology integration (e.g., Globus security, Akenti, CORBA) and application experiments (computed microtomography, camera control).  CIF technologies were demonstrated at several major conferences, including the IEEE Symposium on High Performance Distributed Computing  and SC'98.  In addition, a CIF-based collaborative data analysis system was part of the Globus project entry that won the "Best of Show" award in the HPC Challenge at SC'98.

Toolkit development achievements included:

• Design, development, and release of the CIFcomm library, providing a uniform interface to reliable and unreliable, unicast and multicast communications.  The TCP and UDP transport protocols are supported, as well as the Totem and XTP multicast protocols.   C++ and Java bindings are supported.
• Design, development, and release of the CIF shared state library, which provides C++ and Java mechanisms for manipulation of shared state as used (e.g.) for control elements in collaborative tools.
• Development of a collaborative data analysis system supporting both VR and desktop clients, with the CIF shared state library used to manage access.
• New capabilities in the widely used Nexus communication library, to support unreliable unicast and reliable and unreliable multicast.  New protocols supported by Nexus include Totem and XTP.
• Development of a directory service (the Metacomputing Directory Service) supporting the representation of numerous entities required in collaboratory applications.  Automatic discovery tools for numerous resource types.  Successful deployment of this service on an international testbed spanning over 40 organizations.
• A survey of LDAP-based schema definitions for common collaboratory objects, i.e. people, organizations, groups, security, collaborative sessions, resources, and data.
• Development of a Web-based Collaborative Sessions Directory (WSD).
• Significant improvements to the Totem protocol implementation are underway to improve robustness and ease of integration of the reliable multicast.
• Design and implementation of a new Internet-scale reliable multicast protocol is underway.

Technology integration achievements included:

• Integration of  smart card technology (used to hold PKI credentials) with CIF shared state, hence allowing an impressive demonstration of smart-card-mediated access to a collaborative data analysis session.
• Developed Java software to digitally sign Electronic Notebooks and to use public key based credentials for authentication and secure communications for collaborative tools
• Coordinating efforts with the Product Realization Environment (PRE) project to develop CORBA IDL notebook server interfaces to allow programmatic entry of data into the notebook from CORBA aware applications.

Application achievements included:

• Real-time collaborative camera control: A set of tools for the collaborative control of remote cameras was developed, based on CIF technologies.
• Computed microtomography: CIF shared state services were used extensively in a ground-breaking demonstration of quasi-real-time reconstruction of data from a beamline at the Advanced Photon Source (APS). This application combines DOE2000 ACTS and Collaboratory technology in exciting ways, supporting collaborative analysis from VR and desktop clients of data  reconstructed in quasi-real-time following collection at the APS.

On the left, an Advanced Photon Source (APS) beamline, and on the right the internal structure of a micrometeorite, as determined by our quasi-real-time reconstruction system that supports remote, collaborative analysis of  APS data.

Note that some of this work (e.g. the work on directory services and security) leverages work performed under DARPA support in the Globus project.

6. Plans for FY99

• Additional protocol integration and development of CIFcomm and CIF shard state.
• Development of CORBA interfaces, including work on use of Globus technologies to support access to scheduled resources from CORBA applications.
• Investigations of next-generation communication libraries for collaboratory applications supporting multiple flows.
• Network quality of service interfaces.
• Integration of Globus security and Akenti authorization technologies.
• Identification of standard LDAP schemas for collaboratory applications.
• Additional application experiments.
• Complete implementation of the new Internet-scale reliable multicast protocol.
• Integration of membership notification services into Nexus to better support multicast communication protocols.

7. Tool Availability

CIFcomm and CIF shared state are available online, as are instructions on their use.

8. Participants and Affiliations

Deb Agarwal, Lawrence Berkeley National Laboratory
Karlo Berket, Lawrence Berkeley National Laboratory
John Bresnahan, Argonne National Laboratory
Brett Didier, Pacific Nortwest National Laboratory
Ian Foster, Argonne National Laboratory
Joseph Insley, Argonne National Laboratory
Bruce Mah, Sandia National Laboratory
Jim Myers, Pacific Northwest National Laboratory
Brian Toonen, Argonne National Laboratory
Steve Tuecke, Argonne National Laboratory