A do-it-yourself framework for grid computing [come from javaworld]

A do-it-yourself framework for grid computing

Learn to build your own grid computing application using open source tools

Summary

Large-scale grid computing frameworks can be successfully used to build computational grid infrastructures, but their sophistication can also be a barrier for software designers experimenting with entry-level grid computing. This article uses a Java custom ClassLoader, a Tomcat servlet container, and the Axis SOAP (Simple Object Access Protocol) implementation to demonstrate a simple-to-use framework that supports minimal grid requirements, including machine independence, abstracted compute-task distribution and execution, and a secure, scalable infrastructure. (2,000 words; April 25, 2003)

By Anthony Karre

Page 1 of 4

internet computing is a term commonly used to describe the use of multiple networked computers to run dedicated computing applications. The most famous example is The Search for Extraterrestrial Intelligence (SETI) project, in which half a million home and corporate PC users voluntarily use their spare processing power to analyze radio telescope data for SETI.

The SETI project is typical of the Internet computing domain in that its application is both dedicated and vertically integrated. In other words, no generic use of the computing resources is or can be made, and all dataflow between the computing resource and the data server uses proprietary APIs. When an application upgrade is required, manual intervention is needed to download new executables, and typically no management layer exists for dynamically pushing new computing tasks or otherwise manipulating the available computing resources.

Transformation: Internet to grid computing

Driven by the success of the SETI project and others like it, researchers have been working to exploit the vast pool of computing resources connected to the Internet, but in a way that is secure, manageable, and extendable. In the last few years, various frameworks have been developed, with much emphasis on the management of shared resources. These frameworks typically provide the following features:

Machine independence through the use of Java and Web services

Security through Web service implementations like SOAP (Simple Object Access Protocol), with its built-in accommodation of HTTPS and user authentication

Task abstraction, that is, the ability for a management system to submit essentially arbitrary computing jobs to grid-based resources

Management of non-CPU resources, like file storage

Dynamic management of all of the above given a real-time assessment of the grid's status

Some frameworks have successfully evolved to the point where their architects have made them available in toolkit form. One example of such a toolkit is the Globus Toolkit 3.0, available through the Globus Project. This toolkit has been steadily evolving and has heavily influenced the development of the Open Grid Services Architecture (OGSA). Other published examples include a compute framework developed by Sun Microsystems researchers based on the Jxta peer-to-peer communication API.

Though powerful, these frameworks require a high degree of knowledge capital to successfully deploy and exploit. This can represent a barrier to entry of sorts for individuals or groups who wish to experiment with grid computing. For new programmers, this is like trying to use Struts for your first Web application, rather than writing your own servlets from scratch.

A minimal grid computing framework

In this article, I show you a small do-it-yourself framework that makes booting up a grid computing environment easy. This framework can be quickly developed using the common open source applications Tomcat and Axis, and, while simple, it still meets the following minimal grid needs:

Machine independence, achieved through the use of Java, Tomcat, and Axis.

Secure, scalable infrastructure, achieved through the use of SOAP-based Web services for client-server communication.

Abstracted task execution, achieved through the use of a client-side custom ClassLoader. With this ClassLoader, the client will be capable of identifying and executing arbitrary server-delivered Java thread classes.

Figure 1 illustrates a simplified view of the framework's architecture and execution sequence.

[/url]

Figure 1. Simplified grid execution sequence. Click on thumbnail to view full-size image.

Build an example application

Throughout this article, I focus on building and testing this framework, and provide examples and illustrations in the following order:

Build a server-side SOAP service using Tomcat and Axis

Create connection stubs to support client-side use of the SOAP service

Build a custom client-side ClassLoader

Build the main client application

Build a trivial compute task designed to exercise the client ClassLoader

Build the SOAP service

The SOAP service I build in this article is the closest thing to a management layer that this framework will have. The SOAP service provides a way for our grid computing application to pull the classes it needs from the SOAP server. While my example service simply delivers a single specific jar file, this service's actual production version would likely have access to multiple jar files (each containing a different computing task), and it would contain additional logic to control which JAR was delivered to whom.

The first step in providing the SOAP service is to set up the SOAP infrastructure. I chose Tomcat as the servlet container/HTTP server because it is an open source project and proves to be extremely reliable and easy to use. I chose Axis as the SOAP services provider because it too is open source, supports an easy-to-use drag-and-drop service installer, and comes with a tool that creates SOAP client-side stubs from WSDL (Web Services Description Language) files (a feature I exploit later).

After downloading and installing Tomcat 4.0.6 and Axis 1.0, I wrote the SOAP service class GridConnection. This service fetches a known jar file, loads the file into a byte array, and returns the byte array to the caller. The following code is the entire file GridConnection.java:

//// GridConnection.java

import java.util.*;

import java.io.* ;

public class GridConnection {

public byte[] getJarBytes () {

byte[] jarBytes = null ;

try {

FileInputStream fi = new FileInputStream("/Users/tkarre/MySquare/build/MySquare.jar");

jarBytes = new byte[fi.available()];

fi.read(jarBytes);

fi.close() ;

}

catch(Exception e) {}

return jarBytes ;

}

Page 2 of 4

Deploy the SOAP service

The Axis Java Web service (JWS) deployment scheme eases the deployment of our new service. With Axis, you can write a simple service (like our own GridConnection class), change the .java file suffix to .jws, then drag and drop it into Tomcat's Axis webapp directory. Axis discovers the new .jws file, compiles it, and makes it available as a SOAP endpoint. Figure 2 shows where I put this file in my test environment and where the jws class files are created. Look for the GridConnection.jws file in the output of the Unix ls command.

[url=http://www.javaworld.com/javaworld/jw-04-2003/images/jw-0425-grid2.jpg]

Figure 2. Output of Unix ls command. Click on thumbnail to view full-size image.

Create client-side connection stubs

Once I deployed the new SOAP service, I had to generate the client-side code to use the service. In this step, I used the Axis WSDL2Java command-line tool to generate the Java files that provide the stubs necessary to invoke our SOAP service. This tool completely eliminates the process of hand-coding the classes required to implement the SOAP interface's client side. The WSDL2Java tool requires a WSDL file, so I used yet another Axis feature to get one for our service.

Axis can generate WSDL viewable by a Web browser for any service locally deployed. For our deployed GridConnection class, I used the URL http://localhost:8080/axis/GridConnection.jws?wsdl.

Figure 3 shows the WSDL that Axis generates as seen from Internet Explorer.

[/url]

Figure 3. WSDL as seen from Internet Explorer. Click on thumbnail to view full-size image.

After saving the WSDL file (you can use the Internet Explorer command View Source and save it), you can continue with the stub creation. I saved the WSDL as GridConnection.wsdl, then ran the command:

java org.apache.axis.wsdl.WSDL2Java -o . -p com.perficient.grid.client GridConnection.wsdl

The -p parameter tells WSDL2Java to create the package structure for com.Perficient.grid.client, which is the package I used when I built the actual client application. Figure 4 illustrates this command's execution (don't forget to set your classpath to include the various JARs installed with Axis). Note in Figure 4 that WSDL2Java has created the package directory structure and four Java files.

[url=http://www.javaworld.com/javaworld/jw-04-2003/images/jw-0425-grid4.jpg]

Figure 4. WSDL2Java results. Click on thumbnail to view full-size image.

Here is the content of the four Java files created by the WSDL2Java tool:

/**

* GridConnection.java

* This file was auto-generated from WSDL

* by the Apache Axis WSDL2Java emitter.

package com.perficient.grid.client;

public interface GridConnection extends java.rmi.Remote {

public byte[] getJarBytes() throws java.rmi.RemoteException;

}

/**

* GridConnectionService.java

* This file was auto-generated from WSDL

* by the Apache Axis WSDL2Java emitter.

package com.perficient.grid.client;

public interface GridConnectionService extends javax.xml.rpc.Service {

public java.lang.String getGridConnectionAddress();

public com.perficient.grid.client.GridConnection getGridConnection() throws javax.xml.rpc.ServiceException;

public com.perficient.grid.client.GridConnection getGridConnection(java.net.URL portAddress) throws javax.xml.rpc.ServiceException;

}

/**

* GridConnectionServiceLocator.java

* This file was auto-generated from WSDL

* by the Apache Axis WSDL2Java emitter.

package com.perficient.grid.client;

public class GridConnectionServiceLocator extends org.apache.axis.client.Service implements com.perficient.grid.client.GridConnectionService {

// Use to get a proxy class for GridConnection.

private final java.lang.String GridConnection_address = "http://localhost:8080/axis/GridConnection.jws";

public java.lang.String getGridConnectionAddress() {

return GridConnection_address;

}

// The WSDD service name defaults to the port name.

private java.lang.String GridConnectionWSDDServiceName = "GridConnection";

public java.lang.String getGridConnectionWSDDServiceName() {

return GridConnectionWSDDServiceName;

}

public void setGridConnectionWSDDServiceName(java.lang.String name) {

GridConnectionWSDDServiceName = name;

}

public com.perficient.grid.client.GridConnection getGridConnection() throws javax.xml.rpc.ServiceException {

java.net.URL endpoint;

try {

endpoint = new java.net.URL(GridConnection_address);

}

catch (java.net.MalformedURLException e) {

return null; // unlikely as URL was validated in WSDL2Java

}

return getGridConnection(endpoint);

}

public com.perficient.grid.client.GridConnection getGridConnection(java.net.URL portAddress) throws javax.xml.rpc.ServiceException {

try {

com.perficient.grid.client.GridConnectionSoapBindingStub _stub = new com.perficient.grid.client.GridConnectionSoapBindingStub(portAddress, this);

_stub.setPortName(getGridConnectionWSDDServiceName());

return _stub;

}

catch (org.apache.axis.AxisFault e) {

return null;

}

/**

* For the given interface, get the stub implementation.

* If this service has no port for the given interface,

* then ServiceException is thrown.

public java.rmi.Remote getPort(Class serviceEndpointInterface) throws javax.xml.rpc.ServiceException {

try {

if (com.perficient.grid.client.GridConnection.class.isAssignableFrom(serviceEndpointInterface)) {

com.perficient.grid.client.GridConnectionSoapBindingStub _stub = new com.perficient.grid.client.GridConnectionSoapBindingStub(new java.net.URL(GridConnection_address), this);

_stub.setPortName(getGridConnectionWSDDServiceName());

return _stub;

}

catch (java.lang.Throwable t) {

throw new javax.xml.rpc.ServiceException(t);

}

throw new javax.xml.rpc.ServiceException("There is no stub implementation for the interface: " + (serviceEndpointInterface == null ? "null" : serviceEndpointInterface.getName()));

}

/**

* For the given interface, get the stub implementation.

* If this service has no port for the given interface,

* then ServiceException is thrown.

public java.rmi.Remote getPort(javax.xml.namespace.QName portName, Class serviceEndpointInterface) throws javax.xml.rpc.ServiceException {

java.rmi.Remote _stub = getPort(serviceEndpointInterface);

((org.apache.axis.client.Stub) _stub).setPortName(portName);

return _stub;

}

public javax.xml.namespace.QName getServiceName() {

return new javax.xml.namespace.QName("http://localhost:8080/axis/GridConnection.jws", "GridConnectionService");

}

private java.util.HashSet ports = null;

public java.util.Iterator getPorts() {

if (ports == null) {

ports = new java.util.HashSet();

ports.add(new javax.xml.namespace.QName("GridConnection"));

}

return ports.iterator();

}

/**

* GridConnectionSoapBindingStub.java

* This file was auto-generated from WSDL

* by the Apache Axis WSDL2Java emitter.

package com.perficient.grid.client;

public class GridConnectionSoapBindingStub extends org.apache.axis.client.Stub implements com.perficient.grid.client.GridConnection {

private java.util.Vector cachedSerClasses = new java.util.Vector();

private java.util.Vector cachedSerQNames = new java.util.Vector();

private java.util.Vector cachedSerFactories = new java.util.Vector();

private java.util.Vector cachedDeserFactories = new java.util.Vector();

public GridConnectionSoapBindingStub() throws org.apache.axis.AxisFault {

this(null);

}

public GridConnectionSoapBindingStub(java.net.URL endpointURL, javax.xml.rpc.Service service) throws org.apache.axis.AxisFault {

this(service);

super.cachedEndpoint = endpointURL;

}

public GridConnectionSoapBindingStub(javax.xml.rpc.Service service) throws org.apache.axis.AxisFault {

if (service == null) {

super.service = new org.apache.axis.client.Service();

} else {

super.service = service;

}

private org.apache.axis.client.Call createCall() throws java.rmi.RemoteException {

try {

org.apache.axis.client.Call _call =

(org.apache.axis.client.Call) super.service.createCall();

if (super.maintainSessionSet) {

_call.setMaintainSession(super.maintainSession);

}

if (super.cachedUsername != null) {

_call.setUsername(super.cachedUsername);

}

if (super.cachedPassword != null) {

_call.setPassword(super.cachedPassword);

}

if (super.cachedEndpoint != null) {

_call.setTargetEndpointAddress(super.cachedEndpoint);

}

if (super.cachedTimeout != null) {

_call.setTimeout(super.cachedTimeout);

}

if (super.cachedPortName != null) {

_call.setPortName(super.cachedPortName);

}

java.util.Enumeration keys = super.cachedProperties.keys();

while (keys.hasMoreElements()) {

java.lang.String key = (java.lang.String) keys.nextElement();

if(_call.isPropertySupported(key))

_call.setProperty(key, super.cachedProperties.get(key));

else

_call.setScopedProperty(key, super.cachedProperties.get(key));

}

return _call;

}

catch (java.lang.Throwable t) {

throw new org.apache.axis.AxisFault("Failure trying to get the Call object", t);

}

public byte[] getJarBytes() throws java.rmi.RemoteException {

if (super.cachedEndpoint == null) {

throw new org.apache.axis.NoEndPointException();

}

org.apache.axis.client.Call _call = createCall();

_call.setReturnType(new javax.xml.namespace.QName("http://www.w3.org/2001/XMLSchema", "base64Binary"), byte[].class);

_call.setUseSOAPAction(true);

_call.setSOAPActionURI("");

_call.setOperationStyle("rpc");

_call.setOperationName(new javax.xml.namespace.QName("http://localhost:8080/axis/GridConnection.jws", "getJarBytes"));

java.lang.Object _resp = _call.invoke(new java.lang.Object[] {});

if (_resp instanceof java.rmi.RemoteException) {

throw (java.rmi.RemoteException)_resp;

}

else {

try {

return (byte[]) _resp;

} catch (java.lang.Exception _exception) {

return (byte[]) org.apache.axis.utils.JavaUtils.convert(_resp, byte[].class);

}

These files must be included with our client application—they provide the classes necessary to connect to our SOAP endpoint and will be used by our custom ClassLoader.

Page 3 of 4

Build the custom ClassLoader

The custom ClassLoader is arguably the most important component of the grid client application—it retrieves the jar file via the SOAP service, then makes the JAR-contained classes available for execution. I started with the skeletal ClassLoader structure originally published by Ken McCrary in "Create a Custom Java 1.2-Style ClassLoader" (JavaWorld, March 2000). I then completed the ClassLoader with code specific to our application. Here is my ClassLoader GridConnectionClassLoader.java source, with inserted narrative:

// GridConnectionClassLoader.java

// Created by Anthony Karre on Wed Dec 11 2002.

// Structure of this class was kick-started by published Ken McCrary examples:

// Here is his copyright notice:

* Permission to use, copy, modify, and distribute this software

* and its documentation for NON-COMMERCIAL purposes and without

* fee is hereby granted provided that this copyright notice

* appears in all copies.

* KEN MCCRARY MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT THE

* SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED, INCLUDING

* BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY,

* FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. KEN MCCRARY

* SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT

* OF USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.

package com.perficient.grid.client ;

import java.util.Enumeration;

import java.util.NoSuchElementException;

import java.util.zip.* ;

import java.util.jar.Manifest;

import java.util.jar.JarInputStream ;

import java.util.jar.JarFile ;

import java.util.jar.JarEntry ;

import java.util.jar.Attributes;

import java.util.jar.Attributes.Name;

import java.util.Hashtable ;

import java.io.* ;

import java.net.URL ;

public class GridConnectionClassLoader extends ClassLoader {

// Provide normal constructor.

GridConnectionClassLoader() {

super();

init();

}

// Provide delegation constructor.

GridConnectionClassLoader(ClassLoader parent) {

super(parent);

init();

}

Here is the first method of interest: The init() method, using our SOAP stubs that I constructed earlier, will attempt to connect to our SOAP endpoint. If all goes well, it will fetch a byte array containing the jar file's contents with our executable task. Beyond collecting the raw bytes, the init() method also performs two critical initialization tasks—extracting the size, in bytes, of each JAR resource, and identifying the primary class that I want to execute.

For simplicity, my design requires the primary class to be a Java thread object and identified in the JAR manifest through the attribute "Task-Thread" (similar in concept to the "Main-Class" attribute often used with static main classes):

// ***********************************************************************

// Initialize the ClassLoader by using Web services (SOAP) to fetch our JAR.

// This jar file will contain the classes that compose our grid task.

// **************************************************************************

private void init() {

byte[] localjarbytes = null ;

File tempfile = null ;

FileInputStream tempfis = null ;

JarInputStream jis = null ;

try {

// Make a SOAP service to start the process of fetching the JAR.

com.perficient.grid.client.GridConnectionService service =

new com.perficient.grid.client.GridConnectionServiceLocator() ;

// Now use the service to get a stub to the SOAP service itself.

com.perficient.grid.client.GridConnection gc = service.getGridConnection() ;

// Make the actual call and fetch the bytes.

localjarbytes = gc.getJarBytes() ;

// If localjarbytes is null, then the grid connection failed. One possibility

// is that a jar file could not be found or loaded by the grid server.

// Print a message and exit.

if (localjarbytes == null) {

System.out.println("Class Loader: The grid connection returned no bytes. No Jar file is currently available.") ;

return ;

}

// At this point we can store our bytes into a temp file. This temp file

// will be a repository for the JAR, allowing the classloader to fetch

// classes as necessary.

System.out.println("Class Loader: the grid connection returned " + localjarbytes.length + " bytes") ;

// Save the size of the JAR for later use.

this.jarsize = localjarbytes.length ;

try {

// Create a temp jar file for these bytes. Having an actual file will

// allow us to retrieve various resources more easily.

tempfile = File.createTempFile("grid", ".jar") ;

// Make sure our file is deleted when the VM exits.

tempfile.deleteOnExit() ;

// Get an output stream and dump our bytes to the file.

FileOutputStream tempfileos = new FileOutputStream(tempfile) ;

tempfileos.write(localjarbytes) ;

tempfileos.close() ;

}

catch (IOException e) {

System.out.println("Class Loader: Problem with temp Jar file: " + e.getMessage()) ;

}

catch (Exception e) {

System.out.println("Class Loader: a grid connection failure occurred:\n" + e.toString() + "\n" + e.getMessage()) ;

}

try {

// Save the filename for later reference.

jarfilename = new String(tempfile.getCanonicalPath()) ;

// Try to get a manifest, if one exists. This will

// let us know what the main task thread class is.

// The main task thread is the primary work unit

// that we will be executing in our grid.

tempfis = new FileInputStream(tempfile) ;

jis = new JarInputStream(tempfis) ;

manifest = jis.getManifest() ;

Attributes attr = manifest.getMainAttributes();

if (attr != null) {

taskthread = attr.getValue("Task-Thread") ;

}

// Now scan all of the entries in the file and

// get the size of each resource/class. This is the

// only way to get the true size when the JAR is compressed.

ZipFile zipfile = new ZipFile(tempfile);

enum = zipfile.entries() ;

while (enum.hasMoreElements()) {

ZipEntry ze = (ZipEntry) enum.nextElement() ;

htSizes.put(ze.getName(), new Integer((int)ze.getSize())) ;

}

} // Try.

catch (Exception e) {

System.out.println("Class Loader: " + e.toString() + " : " + e.getMessage()) ;

}

finally {

if ( tempfis != null ) {

try {

tempfis.close();

}

catch (Exception e){}

}

if ( jis != null ) {

try {

jis.close();

}

catch (Exception e){}

}

} // Finally.

}

The findClass() method, shown below, is the method used by the ClassLoader to actually track down and load a particular class. In this implementation, I look in our jar file for the specified class. If I don't find it, I throw the standard ClassNotFoundException:

/**

* This is the method where the task of classloading

* is delegated to our custom loader.

* @param name the name of the class

* @return the resulting Class object

* @exception ClassNotFoundException if the class could not be found

protected Class findClass(String name) throws ClassNotFoundException {

FileInputStream fis = null ;

ZipInputStream zis = null ;

try {

String path = name.replace('.', '/') + ".class" ;

fis = new FileInputStream(jarfilename) ;

zis = new ZipInputStream(fis) ;

ZipEntry ze = null ;

boolean classfound = false ;

while ((ze = zis.getNextEntry()) != null) {

if (path.equals(ze.getName())) {

classfound = true ;

// Get the size of the classfile so we know how many bytes

// to read. If the size appears to be -1, then the

// jar file is compressed, and we need to consult our

// hashtable to get the uncompressed size.

int size = (int) ze.getSize() ;

if (size == -1) {

size = ((Integer) htSizes.get(path)).intValue() ;

}

// Now that we know the actual size of the class,

// fetch the data.

byte[] classBytes = new byte[size] ;

int rb = 0 ;

int chunk = 0 ;

while ((size - rb) > 0) {

chunk = zis.read(classBytes, rb, size - rb) ;

if (chunk == -1) {break ;}

rb += chunk ;

}

// Now that we have our class data, define the class.

System.out.println ("Class Loader: Found class " + name) ;

definePackage(name);

return defineClass(name, classBytes, 0, classBytes.length);

}

if (!classfound) {

// We apparently failed to locate the class within the jar file,

// so indicate the problem with an exception.

throw new ClassNotFoundException(name) ;

}

catch (Exception e) {

// We could not find the class, so indicate the problem with an exception.

throw new ClassNotFoundException(name);

}

finally {

if ( fis != null ) {

try {

fis.close();

}

catch (Exception e){}

}

if ( zis != null ) {

try {

zis.close();

}

catch (Exception e){}

}

return null ;

}

The findResource() and findResources() methods, shown below, are normally used to find and provide resources that might be in the JAR. Since all of our resources remain packed in their encapsulating JARs rather than unpacked in a filesystem someplace, these resources won't be available by URL; therefore these methods aren't extremely useful at the moment:

/**

* Identify where to load a resource from.

* Normally the resources will be extracted from a JAR

* and reside in a filesystem somewhere. Since

* we are loading our classes directly from a jar file,

* they won't really be available via URL.

* We'll always return a null to indicate

* that the resource can't be found, i.e., unavailable.

* You could also dump all of the resources into a

* local/temp filesystem and "find" them there.

* @param name the resource name

* @return URL for resource or null if not found

protected URL findResource(String name) {

return null;

}

/**

* Used for identifying resources from multiple URLS.

* We will return a null for the same reasons as above.

* @param name the resource name

* @return Enumeration of one URL

protected Enumeration findResources(final String name) throws IOException {

return null ;

}

The definePackage() method provides the final standard method for our ClassLoader. It is unchanged from the published Ken McCrary example:

/**

* Minimal package definition

private void definePackage(String className) {

// Extract the package name from the class name.

String pkgName = className;

int index = className.lastIndexOf('.');

if (index != -1) {

pkgName = className.substring(0, index);

}

// Preconditions -- need a manifest and the package

// is not previously defined.

if ( null == manifest ||

getPackage(pkgName) != null) {

return;

}

String specTitle,

specVersion,

specVendor,

implTitle,

implVersion,

implVendor;

// Look up the versioning information.

// This should really look for a named attribute.

Attributes attr = manifest.getMainAttributes();

if (attr != null) {

specTitle = attr.getValue(Name.SPECIFICATION_TITLE);

specVersion = attr.getValue(Name.SPECIFICATION_VERSION);

specVendor = attr.getValue(Name.SPECIFICATION_VENDOR);

implTitle = attr.getValue(Name.IMPLEMENTATION_TITLE);

implVersion = attr.getValue(Name.IMPLEMENTATION_VERSION);

implVendor = attr.getValue(Name.IMPLEMENTATION_VENDOR);

definePackage(pkgName,

specTitle,

specVersion,

specVendor,

implTitle,

implVersion,

implVendor,

null); // No sealing for simplicity.

}

The final three methods shown below complete our ClassLoader. The getTaskThreadName() method returns the name of the primary task thread that I want to execute (based on information found in the manifest). The isLoaded() method indicates whether the ClassLoader successfully fetched and loaded a jar file, and is used by the main class to determine whether the primary task thread is available for execution. The getResourceEntries() method returns an enumeration of the resource names in the JAR and is used for debugging:

// The getTaskThreadName is a getter for the taskthread field.

public String getTaskThreadName() {

return this.taskthread ;

}

// If a JAR has been loaded, then we will have

// a nonzero jarsize.

public boolean isLoaded() {

return (this.jarsize > 0) ;

}

// getResourceEntries returns an enumeration of the keys

// in the hashtable. The keys are the names of the JAR

// entries we picked up earlier. This is useful for

// printing the list of JAR resources.

public Enumeration getResourceEntries() {

return htSizes.keys() ;

}

private Manifest manifest = null ;

private String taskthread = null ;

private String jarfilename = null ;

private int jarsize = 0 ;

private Enumeration enum = null ;

private Hashtable htSizes = new Hashtable() ;

Page 4 of 4

Build the main client application

The main client application is responsible for instantiating the ClassLoader, then using it to load the primary task thread. In my example, I simply load and execute the thread once. A production application would likely attempt to restart a thread once it executed or even destroy the ClassLoader object and load a new one (perhaps to force the fetch of a new JAR). Here is our main application code:

// GridClientMain.java

package com.perficient.grid.client ;

import java.util.*;

import java.io.* ;

public class GridClientMain {

public static void main (String args[]) {

// Start by creating our custom classloader. The classloader, during

// initialization, will use Web services to attempt to download a JAR

// containing the grid task we will execute.

try {

GridConnectionClassLoader gcloader = new GridConnectionClassLoader () ;

// Check to see if any bytes of data were received. If the classloader is not loaded,

// then the classloader was unable to initialize properly.

if (!gcloader.isLoaded()) {

System.out.println("GridClientMain: No bytes were loaded by the classloader, so no grid task is available to be executed.") ;

}

// A JAR has been loaded by the classloader, but

// the JAR manifest may not have a registered Task-Thread attribute. This means

// that we have no way of knowing which class we are supposed to run.

// If our classloader does not have a Task-Thread name, print a message,

// then print the list of resources found in the JAR for reference.

else if (gcloader.getTaskThreadName() == null) {

System.out.println("GridClientMain: The Jar manifest did not have a Task-Thread attribute.\nPlease identify the Task-Thread and re-jar the data. Here are the known Jar resources:\n") ;

Enumeration enum = gcloader.getResourceEntries() ;

while (enum.hasMoreElements()) {

System.out.println((String) enum.nextElement()) ;

}

else {

// We have a Task-Thread, so let's try to load it. Again, note that we assume

// that the Task-Thread class is a Thread class. This is a design decision for building

// the grid.

System.out.println("GridClientMain: Attempting to load " + gcloader.getTaskThreadName()) ;

Thread task = (Thread) gcloader.loadClass(gcloader.getTaskThreadName()).newInstance();

System.out.println("GridClientMain: Attempting to start Thread...") ;

task.start();

}

catch (ClassNotFoundException e) {

System.out.println("GridClientMain: Could not find class:\n" + e.toString() + "\n" + e.getMessage()) ;

}

catch (Exception e) {

System.out.println("GridClientMain: exception:\n" + e.toString() + "\n" + e.getMessage()) ;

}

} // Main.

}

Build a trivial test compute thread

To test the grid framework, I need a test compute thread. It's unnecessary for the test thread to actually do anything, but it is necessary to construct it in a way that reasonably exercises the ClassLoader. I've designed our test thread to use multiple classes, including at least one inner class. When I execute our main application (which subsequently results in the test thread's retrieval and execution), I should see evidence that the custom ClassLoader invoked as necessary. Here is the code for our test thread:

// TestThread.java

package com.perficient.tasks ;

import java.util.*;

import com.perficient.tasks.TestThreadHelper ;

public class TestThread extends Thread {

public TestThread () {

super() ;

}

public void run() {

System.out.println("TestThread: You are now inside the TestThread thread.");

System.out.println("TestThread: Accessing another class in this jar...") ;

// Demonstrate the ability to access another class in the

// same jar file.

try {

TestThreadHelper tth = new TestThreadHelper() ;

System.out.println("TestThread: tth.getDemoString() returned " + tth.getDemoString()) ;

}

catch (Exception e) {

System.out.println("TestThread: Failed to create TestThreadHelper.") ;

System.out.println(e.toString() + " : " + e.getMessage()) ;

}

System.out.println("TestThread: Exiting TestThread thread.") ;

}

// TestThreadHelper.java

// TestThread

// Created by Anthony Karre on Sun Jan 05 2003.

package com.perficient.tasks ;

public class TestThreadHelper {

public String getDemoString () {

// demonstrate correct inner class load

TTHInnerClass myinner = new TTHInnerClass() ;

return myinner.getInnerClassString() ;

}

class TTHInnerClass {

public String getInnerClassString () {

return "innerclassworked" ;

}

Because I used the Apple Mac OS X Project Builder IDE to build this application, our test JAR contains an extra text file—manifest.txt. With this tool, you must explicitly provide a source file containing any extra manifest information not already provided by Project Builder if you want additional data merged into the normal manifest. Here is the file (containing a single line):

Task-Thread: com.perficient.tasks.TestThread

Test the grid computing framework

I tested the grid computing framework by starting the Tomcat/Axis SOAP server on my Macintosh, then running the main application from within the Apple Mac OS X Project Builder IDE. I simulated various failure conditions, including nonexistent JARs, missing Task-Thread manifest attributes, and SOAP outages caused by communication failures. In all cases, the ClassLoader behaved as designed. Figure 5 illustrates a normal execution of the grid client application. Note how the ClassLoader is invoked as needed to support class instantiation within the test thread.

[/url]

Figure 5. Java console output. Click on thumbnail to view full-size image.

Start experimenting with grid computing

While comprehensive grid computing toolkits are available and effective, they do require a certain amount of expertise to implement. The learning curve may be a barrier to developers looking to quickly test a small grid computing architecture. The do-it-yourself grid computing framework illustrated in this article is an easy vehicle for experimenting with grid computing principles. It uses common open source components familiar to most Java programmers, while preserving the main tenets of grid computing: machine independence, abstracted task execution, and secure, scalable infrastructure.

Resources

Learn about the Search for Extraterrestrial Intelligence (SETI@home) project:

[url=http://setiathome.ssl.berkeley.edu/]http://setiathome.ssl.berkeley.edu/

Global Grid Forum:

http://www.ggf.org/

Open Grid Services Architecture Working Group (OGSA WG):

http://www.ggf.org/ogsa-wg/

The Globus Project:

http://www.globus.org

Sun Microsystems Research Paper: "Framework for Peer-to-Peer Distributed Computing in a Heterogeneous, Decentralized Environment," Jerome Verbeke, Neelakanth Nadgir, Greg Ruetsch, Ilya Sharapov:

http://wwws.sun.com/software/jxta/mdejxta-paper.pdf

Project Jxta:

http://wwws.sun.com/software/jxta

Apache Tomcat homepage:

http://jakarta.apache.org/tomcat/

SOAP W3C (World Wide Web Consortium) specification:

http://www.w3.org/TR/SOAP/

The Apache Axis SOAP implementation:

http://ws.apache.org/axis/

"Create a Custom Java 1.2-Style ClassLoader," Ken McCrary (JavaWorld, March 2000):

http://www.javaworld.com/javaworld/jw-03-2000/jw-03-classload.html

The Apache Struts Web application framework: