Using the RAVI Extension to Introduce:
A HOWTO for those new to the Remote Application Virtualization Interface (RAVI)


This document is intended for a developer wishing to transform any arbitrary executable application into a web service. The resulting web service is deployable into either a Globus or Tomcat container. To accomplish this task, we'll be using the RAVI extension to the Introduce software (normally distributed as part of caGrid) which is primarily for automating this process.

Specifically, this HOWTO will walk a new user of the RAVI extension through an example of wrapping the standard UNIX ls command into a web service. Then the service will be deployed to a Globus container and a client will be generated so that a resource can be created, the program can be executed, and the resulting output can be retrieved by the client. While this example may seem trivial, any other custom (non-web service) application can be wrapped in a similar manner. This demonstration is just a simple example of the RAVI framework which allows this to be done with a few clicks of a mouse. All of the complexity of writing a web service from scratch has been taken care of for you. Further, extending the service is possible due to the base code and stubs that will be generated for your service.

This document is broken up into the following sections:

1) Requirements and Notes
2) Getting Started: Downloading and installing Introduce
3) Updating Introduce: Downloading and installing Required Extensions
4) Running Introduce and the RAVI Extension
4a) Creating a RAVI Executable based Service
4b) Creating a RAVI WS-GRAM based Service
5) Deploying the Newly Generated Web Service
6) Running the WSListing Web Service
7) Testing the WSListing Web Service via our New Client


Requirements and Notes

The following are summary notes that may help you get started very quickly, however it is advised that you read the entire document.

-------------------------------------------
System Requirements
-------------------------------------------
Java version 1.5 or newer (http://java.sun.com).
Apache Ant 1.6.5 or newer (http://ant.apache.org).
Globus Toolkit Core 4.0.3 from here:
 http://gforge.nci.nih.gov/frs/download.php/1334/ws-core-enum-4.0.3.zip
Unzip it into a directory and set GLOBUS_LOCATION env variable to that unzipped folder

-------------------------------------------
Environment Variables
-------------------------------------------
JAVA_HOME "path to the java installation"
ANT_HOME "path to the ant installation"
GLOBUS_LOCATION "path to the globus installation"

-----------------------------------------------
Execution
-----------------------------------------------

1) Unzip Introduce

2) cd introduce

3) run ant introduce

Download and install caGrid Metadata plug-in and RAVI plugin using the Menu Help -> Check For Updates

Introduce will be restarted

Getting Started: Downloading and installing Introduce

The first step is to download and install Introduce that comes packaged with the RAVI extension. A link is provided below:

http://software.cagrid.org/introduce/introduce-1.1-src.zip



Updating Introduce: Downloading and installing Required Extensions

Start Introduce by executing the following from the top level introduce directory: 
ant introduce


Following the splash screen, the application opens (shown above). Before the RAVI extension can be used, we need to check for Introduce updates which will allow it to automatically be downloaded and installed. To do this, click on the Help menu's Check for Updates menu item.


An update window will appear. Click on the Introduce Versions and Extensions item in the large window area and then click the Look for Updates button toward the upper right side.


Make sure that you check the boxes next to the CaGrid Metadata Service Extension and the Remote Application Virtualization Interface. Then click on the next button when the window looks like it does above and you should see a screen similar to this. This window will appear for a while as it downloads the necessary extensions.


If it appears that all of the updates were downloaded, click the Next button.

NOTE: In testing, it appears that sometimes only the first update is downloaded. If this is the case, click the Previous button instead of the Next button and re-select the Remote Application Virtualization Interface check box and again click Next. When it is complete, click the Next Button.


Click the Finish button and wait for Introduce to automatically restart.

Running Introduce and the RAVI Extension

When Introduce has restarted and is fully opened, click on the Create Service Skeleton button near the top left of the screen. This button opens up a dialog box where we customize the web service code that will be generated (shown below).


Enter in valid data for Step 1 through Step 4. For this example, I will be creating my project in the /tmp directory of my system and calling the project WSListing (which we'll say stands for Web Service Listing).

NOTE: Your home directory is the default installation target and that may be a better location for this example. If the window will not let you select the /tmp directory, please choose another suitable location.

Below the Step 4 field, click on the drop down box under the Service Extensions area. Select the caGrid Metadata extension and then click the Add button. After that, select the Remote Application Virtualization extension and again click the Add button.

NOTE: It is extremely important that you add the caGrid Metadata extension BEFORE adding the Remote Application Virtualization extension. This example will not work if you reverse the order of the two, or omit one of them all together.

The filled out dialog window is shown below for your reference:


Click the Create button and Introduce will start creating your new web service. After creating some amount of code, the RAVI customization dialog will appear. This dialog is shown below for reference:




Creating a RAVI Executable based Service

As you can see, we need to enter the full path to the application binary that will be run on the service side. In addition, you can add a description of the generated service as well as a version number. Below is the completed dialog for our example application. Click the OK Button.




Creating a RAVI WS-GRAM based Service

Follow the same as the above, but be sure to click the Deploy as GRAM Job check box near the bottom (as seen in picture below).

NOTE:The major difference between this example and the above is that each time the service is invoked, the executable specified to be run is submitted as a WS-GRAM job rather than on the machine hosting the service. In order to do this (after the service is deployed as seen in a later step below), you MUST edit the rave.properties file found in the $GLOBUS_LOCATION/etc/<your_service_name> and add the following text GRAM=https://my-gram-submission-host:8443/wsrf/services/ManagedJobFactoryService . Also make sure your grid/proxy credentials are valid in this case.



NOTE: For either example above, you might notice that we did not enter any command line arguments for this application (specifically because they're not required for our example). The truth is that adding arguments is beyond the scope of this document for the moment and aren't fully supported in a sane manner in the RAVI framework. For now, you can circumvent required command line argument by wrapping your executable (with the arguments) into an executable script.

Once you hit OK, the web service will be generated. This may take some time on your system. ... Eventually, you will see the following window.


For this example, no modifications will be done here. The last step that needs to be completed is to deploy your service to a container. For this example, I'll be deploying to my local Globus container.

Deploying the Newly Generated Web Service

To deploy your service to a container, click the Deploy Service button near the top of the window. A selection dialog will appear where you can select the service you'd like to deploy. By default, the newly generated one will be selected, so for this example, just click Open.

You will again see the same error dialog that you saw above (although again, it's not an error this time). You can safely ignore this window and click the Clear button near the bottom right side of the window. After this, you will reveal the Introduce Deployment window:


In the Deployment Location drop down box, make sure that the GLOBUS_LOCATION option is selected. This should be the default. Nothing else needs to be edited for this example. While a generated web service could store information in a configurable MDS4 Index Service, our example will not. Safely skip this step by just clicking the Deploy button.

This step will take some time, but when it's complete, your new service should almost be ready to go. Since we gave the absolute path for the Application, the generated service will just execute it from that location. In case you selected "Deploy as a GRAM job" option above this path is used as path to application on the remote machine the application is executable. In this release of the software you would have to manually copy the application to remote host for GRAM service to execute it.

Running the WSListing Web Service

To test if the new web service was properly deployed, we need only to start the Globus container. Upon proper deployment, you should see a listing of the service as well as the factory used for creating resources. To start the container in this example, we'll use the following command: 
$GLOBUS_LOCATION/bin/globus-start-container -nosec
A screenshot has been provided for reference below. NOTE: the existence of the WSListing and the WSListintResutltResource services in the deployment list:


Congratulations -- your service has been properly deployed!

Testing the WSListing Web Service via our New Client

To test the new WSListing web service, we need to run the newly generated web service client. You'll need to move to the top-level of the generated project code. For this example, we just change directory. 
cd /tmp/WSListing
Once in this directory, we need to first modify a file that was generated for us by Introduce. Unfortunately, it does not contain the proper data in it, so we'll have to edit it by hand. Open up the run-tools.xml file in the top-level directory in your favorite editor. The file contents should look like this: 
<project name="run-tools file" basedir="." default="runClient">
     <target name="runClient" depends="checkGlobus, defineClasspaths"
     description="Run the sample Client">
        <property name="service.url"
        value="http://localhost:8080/wsrf/services/cagrid/WSListing"/>
        <echo message="Connecting to service: ${service.url}" />
        <java
        classname="org.cagrid.introduce.wslisting.client.WSListingClient" classpathref="run.classpath" fork="yes">
            <jvmarg value="-DGLOBUS_LOCATION=${ext.globus.dir}" />
            <arg value="-url" />
            <arg value="${service.url}" />
        </java>
    </target>
</project>
We'll have to change the service.url property line to match our newly deployed service. For this example, I will be changing the line by replacing cagrid with introduce: 
  <property name="service.url" value="http://localhost:8080/wsrf/services/cagrid/WSListing"/>
to this: 
  <property name="service.url" value="http://localhost:8080/wsrf/services/introduce/WSListing"/>
Save this file and then close it. That's all for client side setup! Now you can run invoke the service with the client by running ant runClient in the same top-level directory. On the client side, this should look something like this: 
neillm@macglob /tmp/WSListing $ ant runClient
Buildfile: build.xml

setGlobus:

checkGlobus:
     [echo] Globus: /usr/local/gt-current

defineClasspaths:

defineExtendedClasspaths:

runClient:
     [echo] Connecting to service:
     http://127.0.0.1:8080/wsrf/services/introduce/WSListing
     [java] Running the Grid Service Client
     [java] Testing WSListing
     [java] Got arguments:
     [java] -url
     [java] http://127.0.0.1:8080/wsrf/services/introduce/WSListing
     [java] Attempting to write Endpoint reference to file EPRFile.epr
     [java] Endpoint reference written to file EPRFile.epr
     [java] Checking client output
     [java] Address:
     http://127.0.0.1:8080/wsrf/services/introduce/WSListingResultResource
     [java] Reference property[0]:
     [java] <ns2:WSListingResultResourceResultsKey
     xmlns:ns2="http://wslisting.introduce.cagrid.org/WSListing/Context">
     c7242f90-8cbc-11dc-8a15-8114d679ad1a</ns2:WSListingResultResourceResultsKey>
     [java] 
     [java] Working Dir: WSListing1194390333175
     [java] Status: Finished: 0
     [java] Got output file data ... decoding
     [java] Got encoded data length of 2540 bytes
     [java] Output file written properly to: .//output

BUILD SUCCESSFUL
Total time: 33 seconds
The last line of the client output tells you that the service ran the program and the output that it made was stored by the client in the output file in the current client director (still in the top-level generated code directory). You can view the output simply by cating the file like this: 
neillm@macglob /tmp/WSListing $ cat output 
CA.pl
CA.sh
WSListing1194390333175
c_rehash
cas-action
cas-enroll
cas-find-policies
...
wsrf-get-properties
wsrf-get-property
wsrf-insert-property
wsrf-query
wsrf-set-termination-time
wsrf-update-property
xmlsec1-config
On the container side, you should see output similar to the following: 
Attempting to execute.... 
Directory created: WSListing1194390333175
Command: /usr/local/gt-current/etc/introduce_WSListing/ls
Command Executed
Wrote process output data of 1903 bytes
Output data written to: WSListing1194390333175/output
Trying to get resource
ResourceKey:
{http://wslisting.introduce.cagrid.org/WSListing/Context}WSListingResultResourceResultsKey=c7242f90-8cbc-11dc-8a15-8114d679ad1a
Got output data of 1903 bytes
Total output data size is 1903 bytes
Sending encoded output data of 2540 bytes
Destroying resource - ending process and removing working directory
Destroy Process
Deleting directory WSListing1194390333175


 Testing the Service where Application is Deployed as a GRAM job 


On the client side the output looks something like this :

Macintosh-13:TeragridDemo madduri$ ant runClient 
Buildfile: build.xml

setGlobus:

checkGlobus:
     [echo] Globus: /Users/madduri/globus/cagrid-globus/install

defineClasspaths:

defineExtendedClasspaths:

runClient:
     [echo] Connecting to service: http://localhost:8080/wsrf/services/introduce/TeraGridDemo
     [java] Running the Grid Service Client
     [java] Testing TeraGridDemo
     [java] Got arguments:
     [java] -url
     [java] http://localhost:8080/wsrf/services/introduce/TeraGridDemo
     [java] Attempting to write Endpoint reference to file EPRFile.epr
     [java] Endpoint reference written to file EPRFile.epr
     [java] Checking client output
     [java] Address: http://192.168.0.102:8080/wsrf/services/introduce/TeraGridDemoResultResource
     [java] Reference property[0]:
     [java] 503ed8e0-aeb2-11dc-bfde-93cc7931dcb8
     [java] 
     [java] Working Dir: /Users/madduri/dev/tmp/TeraGridDemo1198124170087
     [java] Status: Finished:0

BUILD SUCCESSFUL
Total time: 1 minute 14 seconds






 The functionality of the generated Service  

The code to generate a GRAM job description, submitting the job to a remote GRAM service, monitoring the status of the job is automatically generated by the plug-in. The rave.properties file for this service looks like the following :


anlext2wls-96:install madduri$ cat etc/introduce_TeraGridDemo/rave.properties 
BaseDirectory=/Users/madduri/dev/tmp/
Debug=true
GRAM=https://grid-hg.ncsa.teragrid.org:8443/wsrf/services/ManagedJobFactoryService
GRAMType=PBS
ProxyPath=/tmp/x509up_u1811



So the generated service automatically created the job description to execute 
/bin/ls, submitted it to GT4 GRAM service running on Teragrid using the proxy 
/tmp/x509up_u1811. The application itself is scheduled to be executed using the
PBS scheduler running at NCSA. Since /bin/ls is already on the remote node, we 
did not have to stage the executable but in other cases the executable need to 
be staged for this release.

The output on the container hosting the application service would look something like this :


anlext2wls-96:install madduri$ bin/grid-proxy-init
Your identity: DC=org,DC=doegrids,OU=People,CN=Ravi Madduri 134710
Enter GRID pass phrase for this identity:           
 Creating proxy, please wait...
 Proxy verify OK
 Your proxy is valid until Thu Dec 20 10:15:08 CST 2007
 anlext2wls-96:install madduri$ bin/globus-start-container -nosec 
 Starting SOAP server at: http://192.168.0.102:8080/wsrf/services/ 
 With the following services:

 [1]: http://192.168.0.102:8080/wsrf/services/AdminService
 [2]: http://192.168.0.102:8080/wsrf/services/AuthzCalloutTestService
 [3]: http://192.168.0.102:8080/wsrf/services/ContainerRegistryEntryService
 [4]: http://192.168.0.102:8080/wsrf/services/ContainerRegistryService
 [5]: http://192.168.0.102:8080/wsrf/services/CounterService
 [6]: http://192.168.0.102:8080/wsrf/services/DelegationFactoryService
 [7]: http://192.168.0.102:8080/wsrf/services/DelegationService
 [8]: http://192.168.0.102:8080/wsrf/services/DelegationTestService
 [9]: http://192.168.0.102:8080/wsrf/services/ManagementService
 [10]: http://192.168.0.102:8080/wsrf/services/NotificationConsumerFactoryService
 [11]: http://192.168.0.102:8080/wsrf/services/NotificationConsumerService
 [12]: http://192.168.0.102:8080/wsrf/services/NotificationTestService
 [13]: http://192.168.0.102:8080/wsrf/services/PersistenceTestSubscriptionManager
 [14]: http://192.168.0.102:8080/wsrf/services/ResolutionService
 [15]: http://192.168.0.102:8080/wsrf/services/SampleAuthzService
 [16]: http://192.168.0.102:8080/wsrf/services/SecureCounterService
 [17]: http://192.168.0.102:8080/wsrf/services/SecurityTestService
 [18]: http://192.168.0.102:8080/wsrf/services/ShutdownService
 [19]: http://192.168.0.102:8080/wsrf/services/SubscriptionManagerService
 [20]: http://192.168.0.102:8080/wsrf/services/TestAuthzService
 [21]: http://192.168.0.102:8080/wsrf/services/TestCounterService
 [22]: http://192.168.0.102:8080/wsrf/services/TestEnumService
 [23]: http://192.168.0.102:8080/wsrf/services/TestRPCService
 [24]: http://192.168.0.102:8080/wsrf/services/TestService
 [25]: http://192.168.0.102:8080/wsrf/services/TestServiceRequest
 [26]: http://192.168.0.102:8080/wsrf/services/TestServiceWrongWSDL
 [27]: http://192.168.0.102:8080/wsrf/services/TransferService
 [28]: http://192.168.0.102:8080/wsrf/services/Version
 [29]: http://192.168.0.102:8080/wsrf/services/WidgetNotificationService
 [30]: http://192.168.0.102:8080/wsrf/services/WidgetService
 [31]: http://192.168.0.102:8080/wsrf/services/cagrid/Date
 [32]: http://192.168.0.102:8080/wsrf/services/cagrid/DateResultResource
 [33]: http://192.168.0.102:8080/wsrf/services/cagrid/HelloWorld
 [34]: http://192.168.0.102:8080/wsrf/services/cagrid/HelloWorldResultResource
 [35]: http://192.168.0.102:8080/wsrf/services/gsi/AuthenticationService
 [36]: http://192.168.0.102:8080/wsrf/services/introduce/APSDemo
 [37]: http://192.168.0.102:8080/wsrf/services/introduce/APSDemoResultResource
 [38]: http://192.168.0.102:8080/wsrf/services/introduce/GRAMSample
 [39]: http://192.168.0.102:8080/wsrf/services/introduce/GRAMSampleResultResource
 [40]: http://192.168.0.102:8080/wsrf/services/introduce/HelloWorld
 [41]: http://192.168.0.102:8080/wsrf/services/introduce/HelloWorldResultResource
 [42]: http://192.168.0.102:8080/wsrf/services/introduce/JavaSample
 [43]: http://192.168.0.102:8080/wsrf/services/introduce/JavaSampleResultResource
 [44]: http://192.168.0.102:8080/wsrf/services/introduce/SCDemo
 [45]: http://192.168.0.102:8080/wsrf/services/introduce/SCDemo1
 [46]: http://192.168.0.102:8080/wsrf/services/introduce/SCDemo1ResultResource
 [47]: http://192.168.0.102:8080/wsrf/services/introduce/SCDemoResultResource
 [48]: http://192.168.0.102:8080/wsrf/services/introduce/SimpleSample
 [49]: http://192.168.0.102:8080/wsrf/services/introduce/SimpleSampleResultResource
 [50]: http://192.168.0.102:8080/wsrf/services/introduce/TeraGridDemo
 [51]: http://192.168.0.102:8080/wsrf/services/introduce/TeraGridDemoResultResource
 [52]: http://192.168.0.102:8080/wsrf/services/introduce/WSListing
 [53]: http://192.168.0.102:8080/wsrf/services/introduce/WSListingResultResource
 [54]: http://192.168.0.102:8080/wsrf/services/introduce/WantaoDemo
 [55]: http://192.168.0.102:8080/wsrf/services/introduce/WantaoDemoResultResource
 Attempting to execute.... 
 Directory created: /Users/madduri/dev/tmp/TeraGridDemo1198124170087
 Arg: InputFile.txt
 Creating job for application /bin/date
 Factory Url: https://grid-hg.ncsa.teragrid.org:8443/wsrf/services/ManagedJobFactoryService
 Submitting job
 Job Submitted: org.globus.exec.generated.JobDescriptionType@6dac30e6
 Job:RSL: /bin/date
 Loading Proxy
 User proxy: X509_USER_PROXY=/tmp/x509up_u1811
 Generating ID:
 Submission ID: uuid:4b176b70-aeb2-11dc-bfde-93cc7931dcb8
 Not in batch mode adding listener
 2007-12-19 22:16:10,152 INFO  client.GramJob [ServiceThread-4,submit:417] 1198124170152
 2007-12-19 22:16:10,237 INFO  client.GramJob [ServiceThread-4,fetchDelegationFactoryEndpoints:647] 1198124170237
 2007-12-19 22:16:12,340 INFO  client.GramJob [ServiceThread-4,fetchDelegationFactoryEndpoints:654] 1198124172340
 2007-12-19 22:16:12,342 INFO  client.GramJob [ServiceThread-4,delegate:732] 1198124172342
 2007-12-19 22:16:13,233 INFO  client.GramJob [ServiceThread-4,delegate:741] 1198124173233
 2007-12-19 22:16:13,234 INFO  client.GramJob [ServiceThread-4,delegate:766] 1198124173234
 2007-12-19 22:16:15,406 INFO  client.GramJob [ServiceThread-4,delegate:783] 1198124175406
 2007-12-19 22:16:15,419 INFO  client.GramJob [ServiceThread-4,createJobEndpoint:958] 1198124175419
 2007-12-19 22:16:18,797 INFO  client.GramJob [ServiceThread-4,createJobEndpoint:965] 1198124178797Pendingfalse0
 ========== State Notification ==========
 Job State: Pending
 ========================================
 2007-12-19 22:16:44,172 INFO  client.GramJob [ServiceThread-4,refreshStatus:1789] Raw status query response message:
 Pendingfalse0
 2007-12-19 22:16:57,669 INFO  client.GramJob [ServiceThread-4,refreshStatus:1789] Raw status query response message:
 Pendingfalse0
 2007-12-19 22:16:58,314 INFO  client.GramJob [ServiceThread-4,refreshStatus:1789] Raw status query response message:
 Activefalse0
 ========== State Notification ==========
 Job State: Active
 ========================================
 2007-12-19 22:17:05,857 INFO  client.GramJob [ServiceThread-4,refreshStatus:1789] Raw status query response message:
 Activefalse0
 2007-12-19 22:17:18,557 INFO  client.GramJob [ServiceThread-4,refreshStatus:1789] Raw status query response message:
 Donefalse0
 ========== State Notification ==========
 Job State: Done
 ========================================
 Exit Code: 0
 (ResultResourceImpl) Error getFile /Users/madduri/dev/tmp/TeraGridDemo1198124170087/output
 java.io.FileNotFoundException: /Users/madduri/dev/tmp/TeraGridDemo1198124170087/output (No such file or directory)
 Destroying resource - ending process and removing working directory
 Destroy GRAM Job
 2007-12-19 22:17:19,061 INFO  client.GramJob [ServiceThread-5,unbind:1237] 1198124239061
 2007-12-19 22:17:19,683 INFO  client.GramJob [ServiceThread-5,unbind:1251] 1198124239683
 2007-12-19 22:17:19,691 INFO  client.GramJob [ServiceThread-5,destroy:1321] 1198124239691
 2007-12-19 22:17:20,305 INFO  client.GramJob [ServiceThread-5,destroy:1345] 1198124240305
 2007-12-19 22:17:20,321 INFO  client.GramJob [ServiceThread-5,destroyDelegatedCredential:1457] 1198124240321
 2007-12-19 22:17:21,790 INFO  client.GramJob [ServiceThread-5,destroyDelegatedCredential:1463] 1198124241790
 Deleting directory /Users/madduri/dev/tmp/TeraGridDemo1198124170087

 


 You can see how the application is polling for job status, displaying the 
 status of the execution and cleaning up after the execution.


Congratulations!  You have now properly wrapped your application as a
web service using the RAVI Framework.







Contact the authors: neillm@mcs.anl.gov,
madduri@mcs.anl.gov