Next Generation SOA

Last week I was on the SoaSymposium in the WorldTrade Center in Rotterdam. The theme of this years episode was apparently "Next Generation SOA". On before hand I wondered what they meant with it. Apparently I missed the blog of Anne Thomas Mane that stated that SOA is dead.
It reminded me of my own blog post on the subject. In that blog-post I opposed to the statement of soa-experts of a Dutch ICT weeklet, where SOA was declared dead in favour of EDA. Because I could not remember exactly on what article I reacted back then, I got a little anxious to get in touch with her. But the reason that Anne declared SOA dead was because of the hype that rose around the acronym by the vendors, customers and media. The acronym got so loaded with pre-assumptions that it did not stand for what it promised anymore. And so many SOA projects fail because customers think they could buy the magic box of a vendor that solve every problem. Or the magic hat of the Pixar Magician Presto. And of course vendor's would love to sell this to them. If they could... To declare SOA dead as a hype with all its false promises and focus on Service Orientation, I second completely. You can't buy or sell SOA, neither as a "Silver bullet", nor as a "Magic hat". It reminded me on this company that would sell an integration product on a blade-server. Put it in a rack and your ERP's are integrated. Yeah, right. I assume that the actual implementation is less magic.

Thomas Erl and Anne had a session to cast out the "Evil SOA" and welcome the "Good SOA". I liked the music of Mike Oldfield, but after a while it got too pressing and looped through the very first part of Tubular Bells, that it almost cast me out. But it got me the nice Soa Patterns book of Thomas Erl.

I think with Thomas and Anne that Service Orientation is very important and grow in importance. But together with EDA. Well actually, I think EDA is not an architecture on it self. As with Soa and cloud: if we're going to hype this idea we fall in the same faults again. But as stated in my former post, I think that Events and Services are (or should) two thightly coupled ideas. No loose coupling here. You use these ideas to loosely couple your functionalities. But Services are useless without events. And events (either coming through an esb or as a stream) are quite useless without services that process them.

For the rest it was a good event to network. Met a few former colleagues and business-partners. Or people with I had co-acquaintances. I saw a nice presentation of Sander Hoogendoorn and Twan van der Broek on agile SOA projects combined with ERP. I'd like to learn more on that.

VirtualBox: a virtual competitor of VMware?

I'm a frequent VMware user for years. Most of my setups are done in a Virtual Machine. It's very convenient because with a new host installation, just installing VMware and restoring the virtual machines gets you up and running.

One of the names that comes around on Virtualization on Linux is VirtualBox. I did not pay any attention to it since I was quite happy with VMware. I use VMware Server, since it's free and has about every feature VMware Workstation had in the latest version I used. That was Workstation 5.5 and the only thing Server lacked was Shared Folders.

But products evolve. And Server has become quite big. I found it quite a step to turn from 1.x to 2.0 since the footprint increased about 5 times! From 100MB to about 500MB.

Last week my colleague Erik asked me if I knew VirtualBox. I said I did, that is by name. A colleague on his project advised him to go for VirtualBox. So I thought I might give it a try.
Some of the features of VirtualBox that interested me were:

• it's about 40MB! That is very small for such a complete product. I like that. I'm fond of small but feature-rich products, like TotalCommander, Xtrans and IrfanView on Windows. Apparently there are still programmers that go for smart and compact products.
• I found articles on internet that it would be faster then VMware. I also found statements that suggest it is slower. But on a simple laptop, performance would be the decision forcing feature.
• Seamless mode: VirtualBox can have the applications run in a seperate window along with the native applications on your host. It layers the taskbar on top of the taskbar of your host. Looks neat!
• Shared Folders: this was the differating feature between Server and Workstation. I solved that lack by installing a Filezilla Server on the Windows Hosts, or using SFTP over SSH on Linux. But it is handy to have that feature.
• It's a client/standalone app: it may seem like a nonsense feature. But VMware Server comes with an Apache Tomcat application server that serves the infrastructure application. That's nice for a server based application, but on a laptop quite over done.
• 3D Acceleration: this summer I wanted to run a Windows racing game under VMware. It didn't do it. I should try it under VirtualBox with 3D acceleration on.
  
• But the important feature to have it work for me is that it should be able to run or import VMware images. It turns out that it cannot. You have to create a new VirtualBox image. But luckily you can base it on the virtual disks of a VMware image. Below I tell you how.

What I did not know was that Sun Microsystems acquired Innotek last year (2008). And since Oracle is in the run to buy Sun, Oracle gets this Virtualizing tool also. Another crown-jewel I could say. Since Oracle does have a VM tool but it is not a desktop tool. Oracle's VM has to run on the 'bare metal'.

Pre-requisites:

• Install VirtualBox: download it from http://www.virtualbox.org/wiki/Downloads. Mark that there is an Open Source and a Closed Source version. Both free, but the Closed Source is the most feature-rich. Importing and exporting is only available in the closed source version. The Open Source is apparently available in many Linux Repositories. But I went for the Closed Source.
• De-install VMware tools from the guest os before migrating. VirtualBox can install it's own guest-tools with the own drivers. For windows you can do it from the software pane in the configuration screen. Under linux (rpm based) you can remove it using rpm -e VMwareTools-5.0.0-.i386.rpm. To query the precise version you can do rpm -q VMwareTools
  

Creation of the VM
Then you can Create a new VirtualMachine in VirtualBox. You can find an how-to with screen-dumps here.
For the harddisks use the existing disks of your VMware image. If the VM is a Windows Guest, the disks are probably IDE disks. You could try to use SATA as an Additional Controller (check the checkbox) and couple them to a SATA channel in order of the disks. Denote the first one bootable. If it is a Linux guest the disks are probably SCSI- LsiLogic, so choose SCSI as additional disk.

After adding the disks and other hardware, you can start it. After starting it you can install the Guest addons. These are necessary to have the appropriate drivers. Under windows, an installer is automatically started. Under Linux a virtual cdrom is mounted. Do a 'su -' to root and start the appropriate '.run' file or do a sudo:
sudo /media/cdrom/VBoxLinuxAdditions-x86.run
In this case it is for a 32-bit Linux. For 64-bit Linux guests it should be the amd64 variant.

After the install you should restart the guest.

Then you can reconfigure your display type (resolution, color-depth) and the network adapters.

Experiences
Well my first experiences are pretty positive. It is sometimes a struggle to get the display and the networking right. It's a pity that you have to recreate the VM instead of importing the VMware image itself. It seems to me that the VMware vmx is convertable to the VirtualBox xml-config file.

I found it a little inconvenient that on clicking of the VM it starts immediately. Sometimes you just want to reconfigure it.

I have to experience the peformance still. So I don't know yet if VirtualBox on this is an advantage. But the Seamless mode is really neat.

JNDI in your Applications

Lately I had to make several services to be used from an application server, as a webservice or a service used from BPEL.

I find it convenient to first build the application or service as a java-application and test it from jDeveloper and/or using an ant-script. Although I know that it's possible to test an application from the application server (but never did it myself) it surely is more convenient to have it tested from an ordinary java-application. Also it enables you to use the service in other topologies. Actually it's my opinion that for example a Webservice should not be more then an interface or just another exposure of an existing service or application.

If your service need a database then from an application you would just create a connection using a driver, jdbc-connection and username/password. Probably you would get those values from a property-file. But within an Application Server you would use JNDI to get a DataSource and from that retrieve a Connection.

But the methods that are called from your services within the application server should somehow get a connection in a uniform manner. Because, how would you determine if the method is called from a running standalone application or from an Application Server? The preferred approach, as I see it, is to make the application in fact agnostic of it. The retrieval of a database connection should in both cases be the same. So how to achieve this?

The solution that I found is to have the methods retrieve the database connection in all cases from a JNDI Context. To enable this, you should take care of having a DataSource registered upfront, before calling the service-implementing-methods.

JNDI Context
To understand JNDI you could take a look into the JNDI Tutorial.
It all starts with retrieving a JNDI Context. From a context you could retrieve every type of Object that is registered with it. So it not necessarly be a Database connection or DataSource, although I believe this is one of the most obvious usages of JNDI.

To retrieve a JNDI Context I create a helper class:

package nl.darwin-it.crmi.jndi;
/**
* Class providing and initializing JndiContext.
*
* @author Martien van den Akker
* @author Darwin IT Professionals
*/
import java.util.Hashtable;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import nl.darwin-it.crmi.CrmiBase;

import oracle.jdbc.pool.OracleDataSource;

public class JndiContextProvider extends CrmiBase {
   public static final String DFT_CONTEXT_FACTORY = "com.sun.jndi.fscontext.RefFSContextFactory";
   public static final String DFT = "Default";
   private Context jndiContext = null;

   public JndiContextProvider() {
   }

   /**
    * Get an Initial Context based on the jndiContextFactory. If jndiContextFactory is null
    * the Initial context is fetched from the J2EE environment. Otherwise if jndiContextFactory contains the keyword
    * "Default" (in any case) the DFT_CONTEXT_FACTORY is used as an Initial Context Factory. Otherwise
    * jndiContextFactory is used.
    *
    *
    * @param jndiContextFactory
    * @return
    * @throws NamingException
    */
   public Context getInitialContext() throws NamingException {
       final String methodName = "getInitialContext";
       debug("Start "+methodName);
       Context  ctx = new InitialContext();
       debug("End "+methodName);
       return ctx;
   }

   /**
    * Bind the Oracle Datasource to the JndiName
    * @param ods
    * @param jndiName
    * @throws NamingException
    */
   public void bindOdsWithJNDI(OracleDataSource ods, String jndiName) throws NamingException {
       final String methodName = "bindOdsWithJNDI";
       debug("Start "+methodName);
       //Registering the data source with JNDI
       Context ctx = getJndiContext();
       ctx.bind(jndiName, ods);
       debug("End "+methodName);
   }

   /**
    * Unbind jndiName
    * @param jndiName
    * @throws NamingException
    */
   public void unbindJNDIName(String jndiName) throws NamingException {
       final String methodName = "unbindJNDIName";
       debug("Start "+methodName);
       //UnRegistering the data source with JNDI
       Context ctx = getJndiContext();
       ctx.unbind(jndiName);
       debug("End "+methodName);
   }

   /**
    * Set jndiContext;
    */
   public void setJndiContext(Context jndiContext) {
       this.jndiContext = jndiContext;
   }
   /**
    * Get jndiContext
    */
   public Context getJndiContext() throws NamingException {
   if (jndiContext==null){
       Context context = getInitialContext();
       setJndiContext(context);
   }
       return jndiContext;
   }
}

This package gives a Jndi Context that can be used to retrieve a DataSource kunt ophalen, using JndiContextProvider.getJndiContext().
This method retrieves an InitialContext that is cached in a private variable.
From within the application server the InitialContext is provided by the Application Server.
But when running as a standalone application a default InitialContext is provided. But this one does not have any content. Also it has no Service Provider that can deliver any objects. You could get an InitialContext by providing the InitialContext() constructor with a hash table with the java.naming.factory.initial property set to com.sun.jndi.fscontext.RefFSContextFactory.

The problem is that if you use the HashTable option, the getting of the InitialContext is not the same as within the Application Server. Fortunately if you don't provide the HashTable, the constructor is looking for the jndi.properties file, within the classpath. This file can contain several properties, but for our situation only the following is needed:

java.naming.factory.initial=com.sun.jndi.fscontext.RefFSContextFactory

This Context factory is one of the jndi-provider-factories delivered with the standard JNDI implementation. It is a simple FileSystemContextFactory,a JNDI factory that refers to the filesystem.

Now the only thing you need to do is to bind a DataSource to the Context, right before your call the possible method, for example in the main method of your applicatoin.
That DataSource you should create using the properties from the configuration file.

To create a datasource I created another helper class: JDBCConnectionProvider:

package nl.darwin-it.crmi.jdbc;
/**
 * Class providing JdbcConnections
 * 
 * @author Martien van den Akker
 * @author Darwin IT Professionals
 */
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

import java.util.Hashtable;

import javax.naming.Context;

import javax.naming.InitialContext;

import javax.naming.NamingException;

import javax.naming.NoInitialContextException;

import oracle.jdbc.pool.OracleDataSource;

public abstract class JDBCConnectionProvider {
    public static final String ORCL_NET_TNS_ADMIN = "oracle.net.tns_admin";

    /**
     * Create a database Connection
     * 
     * @param connectString
     * @param userName
     * @param Password
     * @return
     * @throws ClassNotFoundException
     * @throws SQLException
     */
    public static Connection createConnection(String driver, String connectString, String userName, 
                                              String password) throws ClassNotFoundException, SQLException {
        Connection connection = null;
        Class.forName(driver);
        // DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
        // @machineName:port:SID, userid, password
        connection = DriverManager.getConnection(connectString, userName, password);
        return connection;
    }

    /**
     * Create a database Connection after setting TNS_ADMIN property.
     * 
     * @param driver
     * @param tnsNames
     * @param connectString
     * @param userName
     * @param password
     * @return
     * @throws ClassNotFoundException
     * @throws SQLException
     */
    public static Connection createConnection(String driver, String tnsAdmin, String connectString, String userName, 
                                              String password) throws ClassNotFoundException, SQLException {
        if (tnsAdmin != null) {
            System.setProperty(ORCL_NET_TNS_ADMIN, tnsAdmin);
        }
        Connection connection = createConnection(driver, connectString, userName, password);
        return connection;
    }
    /**
     * Create an Oracle DataSource
     * @param driverType
     * @param connectString
     * @param dbUser
     * @param dbPassword
     * @return
     * @throws SQLException
     */
    public static OracleDataSource createDataSource(String driverType, String connectString, String dbUser, 
                                                 String dbPassword) throws SQLException {
        OracleDataSource ods = new OracleDataSource();
        ods.setDriverType(driverType);
        ods.setURL(connectString);
        ods.setUser(dbUser);
        ods.setPassword(dbPassword);
        return ods;
    }
    public static Connection getConnection(Context context, String jndiName) throws NamingException, SQLException {
        //Performing a lookup for a pool-enabled data source registered in JNDI tree
        Connection conn = null;
        OracleDataSource ods = (OracleDataSource)context.lookup(jndiName);
        conn = ods.getConnection();
        return conn;
    }
}

The method createDataSource will create an OracleDataSource. Today with trial&error I discovered that from a BPEL 10.1.2 server, you should get a plain java.sql.DataSource.

Then with JndiContextProvider.bindOdsWithJNDI(OracleDataSource ods, String jndiName) you can bind the DataSource with the Filesystem Context Provider.

Having done that from your implementation code you can get a database connection using JDBCConnectionProvider.getConnection(Context context, String jndiName)

I haven't discovered how yet, but somewhere (in the filesystem I presume) the DataSource is registered. So a subsequent bind will throw an exception. But since you might want to change your connection in your property file, it might be convenient to unbind the DataSource afterwards at the end of the main-method.The method JndiContextProvider.unbindJNDIName(String jndiName) takes care of that.

What took me also quite a long time to figure out is what jar-files you need to get this working when running in a standalone application. Somehow I found quite a lot examples but no list of jar-files. I even understood (wrongly as it turned out) that the base J2SE5 or 6 should deliver these packages.


The jars you need are:

• jndi.jar
• fscontext.jar
• providerutil.jar

You can find them from the BPEL PM 10.1.2 tree and probably also in a Oracle 10.1.3.x Application Server or Weblogic 10.3 (or nowadays 11g). But the JSE ones can be found via
http://java.sun.com/products/jndi/downloads/index.html
There you'll find the following packages:

• jndi-1_2_1.zip
• fscontext-1_2-beta3.zip

Conclusion
This approach gave me a database connection within the application server or from a standalone application. The name of the JNDI I fetched from a property file still. That way it is configurable from which JNDI source the connection have to be fetched (I don't like hardcodings).
In my search I found that from Oracle 11g onwards you can use the Universal Connection Pool library: http://www.oracle.com/technology/pub/articles/vasiliev-oracle-jdbc.html to have a connection pool from your application.