Darwin-IT

SoapUI Tip 3

2011-12-13T03:19:00.000-08:00

It was a little searching using the javadocs of SoapUI, but if you want to log the result of the script of a mockService or mockResponse step in a testCase, then that can be done using:

log.info("Response: "+mockResponse.getMockResult().getResponseContent())

Another nice tip I found at another blog: "SoapUI limitations and workarounds : mock response test step"
If you want to use mockServices in your testcase or testsuite, but don't want to implement mockResponse steps for them, you can create (of course) mockServices for them. They can handle multple responses and use xpath expressions to choose the right response for a particular request. But running the testcase you'll need to start the mockServices that you need in the test. this can be done by in the "Setup Script" of the testCase:


def project = testCase.getTestSuite().getProject();
def mockService = project.getMockServiceByName("Name of the MockService");
mockService.start();

After running the testcase/testsuite the mockservices should be stopped neatly. This can be done in the "TearDown Script":


def project = testCase.getTestSuite().getProject();
def mockService = project.getMockServiceByName("Name of the MockService");
def mockRunner = mockService.getMockRunner();
mockRunner.stop();

SoapUI: Properties from test Suite

2011-12-11T23:57:00.001-08:00

In my previous post I showed how to get a file from a script in SoapUI. One of the parts on building the filepath of the file to load was a property from the mockService. The same thing is possible from a test suite. There are several places at which you can define properties. Amongst them is the TestSuite.

To get the properties from a test suite, you have to "get" it. To get the test suite on which you defined the properties goes as follows:

def project = mockResponse.mockOperation.mockService.project
def testSuite = project.testSuites["TestSupport"]

The property can then be fetched with:

def filePath = testSuite.getPropertyValue( "responseFilePath")

Test cases are part of the testSuite and they can be fetched from an string-based array in the same manner as getting the testSuite:

def testCase = project.testSuites["TestSuite 1"].testCases["TestCase 1"]

Expanding SoapUI possibilities...

Load Response Message File System in SoapUI

2011-12-08T13:34:00.001-08:00

Today I figured out how to get a response from filesystem in SoapUI. I found that I could not dynamically select from multiple responses in a mockResponse-TestStep in a TestSuite. But loading from a file within a Groovy script should do the job. Advantage is also that you can have as many response files as you like. You just put a value in the filename, that you select using xpath from the request message. The folder in which you store the message can be put in a property on the mockService.I don't have the time to explain the whole lot. But the response in the mockService should be like:

<soapenv:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
   <soapenv:header>
      <darwinheader xmlns="http://www.darwin-it.nl/XMLHeader/10">
         <headerversion>${headerVersion}</headerversion>
         <messageid>${messageId}</messageid>
         <servicerequestordomain>${serviceRequestorDomain}</servicerequestordomain>
         <servicerequestorid>${serviceRequestorId}</servicerequestorid>
         <serviceproviderdomain>${serviceProviderDomain}</serviceproviderdomain>
         <serviceid>${serviceId}</serviceid>
         <serviceversion≶${serviceVersion}</serviceversion>
         <faultindication>${faultIndication}</faultindication>
         <messagetimestamp>${messageTimestamp}</messagetimestamp>
      </darwinheader>
   </soapenv:header>
   <soapenv:body>${responseBody}</soapenv:body>
</soapenv:envelope>

Here you see that the Soapenvelope with the header is given, but the diffent values are properties, as well as the responseBody.The script to get these from the request is as follows:

def method = "ChangeServiceRequest.Response 1.Script"
log.info("Start "+method)

log.info(mockRequest.requestContent)

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
// Set Namespaces
def holder = groovyUtils.getXmlHolder(mockRequest.requestContent)
holder.namespaces["soapenv"] = "http://schemas.xmlsoap.org/soap/envelope/"
holder.namespaces["ns"] = "http://www.darwin-it.nl/XMLHeader/10"
holder.namespaces["rpy"] = "http://www.darwin-it.nl/ChangeServiceRequest/2/Rpy"


log.info("Get Header Properties")
context.messageId=Math.random()
log.info("messageId: "+context.messageId)
context.headerVersion= holder.getNodeValue("//ns:DarwinHeader/ns:HeaderVersion")
log.info("headerVersion: "+context.headerVersion)
context.serviceRequestorDomain= holder.getNodeValue("//ns:DarwinHeader/ns:ServiceRequestorDomain")
log.info("ServiceRequestorDomain: "+context.serviceRequestorDomain)
context.serviceRequestorId= holder.getNodeValue("//ns:DarwinHeader/ns:ServiceRequestorId")
log.info("serviceRequestorId: "+context.serviceRequestorId)
context.serviceProviderDomain= holder.getNodeValue("//ns:DarwinHeader/ns:ServiceProviderDomain")
log.info("serviceProviderDomain: "+context.serviceProviderDomain)
context.serviceId= holder.getNodeValue("//ns:DarwinHeader/ns:ServiceId")
log.info("serviceId: "+context.serviceId)
context.serviceVersion= holder.getNodeValue("//ns:DarwinHeader/ns:ServiceVersion")
log.info("serviceVersion: "+context.serviceVersion)
context.faultIndication= holder.getNodeValue("//ns:DarwinHeader/ns:FaultIndication")
log.info("faultIndication: "+context.faultIndication)
context.messageTimestamp= holder.getNodeValue("//ns:DarwinHeader/ns:MessageTimestamp")
log.info("messageTimestamp: "+context.messageTimestamp)

def serviceRequestNr= holder.getNodeValue("//req:ChangeServiceRequest_Req/req:ServiceRequestData/req:ServiceRequestNumber")
log.info("ServiceRequestNr: "+serviceRequestNr)

def mockRunner = context.getMockRunner()
def mockService = mockRunner.mockService
def filePath = mockService.getPropertyValue( "responseFilePath")+"/FT01_ChangeServiceRequest_"+serviceRequestNr+".xml"
log.info("FileName: "+ filePath)
def File file = new File( filePath )
def fileLength = (int) file.length();
def buffer = new char[fileLength];
def inputReader = new FileReader(file);
def numChar = inputReader.read(buffer);
requestContext.responseBody = new String(buffer);
log.info("End "+method)

With "def holder = groovyUtils.getXmlHolder(mockRequest.requestContent)" you get an XML object of the request. The lines "holder.namespaces["ns"] = "http://www.darwin-it.nl/XMLHeader/10" set the namespace-abbreviations, needed to select the values from the request.
Then with "def serviceRequestNr= holder.getNodeValue("//req:ChangeServiceRequest_Req/req:ServiceRequestData/req:ServiceRequestNumber")", you can perform an xpath query, to get the serviceRequestNr from the request message in this case.

The lines

def mockRunner = context.getMockRunner()
def mockService = mockRunner.mockService
def filePath = mockService.getPropertyValue( "responseFilePath")

show how to read a property from the mockService.

Then the last lines are to determine the actual file path using this property and the queried serviceRequestNr, and to read the file.

By putting the file in the requestContext using: "requestContext.responseBody = new String(buffer);" it can be put in the response using the property "${responseBody}" like in the response-skeleton above.You might notice that it is a mixture of Java and Groovy. I'm not so familiar with Groovy yet. But I found that if it works in Java it is easy to get it work in Groovy. Actually, Groovy is just more loosely typed.I hope this clears enough to get this going.Good luck.

Drag and drop in Gmail

2011-10-19T04:57:00.000-07:00

Well, I'm probably one of the last gmail users that see's it but I just a moment ago encountered that Gmail supports drag and drop. One of the things I really missed. When I wanted to move mails to a label I had to do it using the 'move-to' poplist. But apparently sometime in the past months google implemented the possibility to grab one of more mails/conversations and drag them to a label. Great. By this the frontend mimics a rich-mail-client as Thunderbird., so that handling mail using Gmail is much more convenient. I love empty mailboxes...

Shared Folders in VirtualBox

2011-09-22T02:47:00.000-07:00

Shared Folders are a big advantage in both VirtualBox and VMware Player (>3.0.x). However I notice that in my direct neighborhood there is a unfamiliarity with it. And how to use it. That is, it turns out that a created shared folder is not mounted right a way in a started VM. So to help this out, a little how to.

To create a Shared Folder you have to go to the SharedFolders node in the VM Settings:

There you can click on the add folder button. Of course you can remove or edit existing ones.

Give here the path to the folder on your host. At the layout of the screen you could gues my host is a Windows 7. You can give in a Shared Folder name.
The other options (check boxes, I really need to do a reinstall of VB in English, however it was no option I've chosen consciously to install it in Dutch) are:

Read only
Auto Mount
Make permanent

I check the the latter 2.
Although auto mount is checked, this does not count right away for running Linux guests. Therefor you have to do a restart.

Shared Folders are automounted in linux as /media/sf_<sharedfolder name>

So as root you have to

create the folder
' change owner' it to root:vboxsf
set " group writable" with chmod
Then mount the folder

So:

makker@makker-lnx:~> cd /media
makker@makker-lnx:/media> ls -l
total 92
drwxrwx--- 1 root vboxsf  4096 Aug 25 17:07 sf_Data
drwxrwx--- 1 root vboxsf  8192 Sep 21 13:45 sf_Documents
dr-xr-xr-x 1 root root   16384 Sep 22 09:59 sf_Downloads
drwxrwx--- 1 root vboxsf 65536 Aug 23 14:26 sf_Music
makker@makker-lnx:/media> sudo mkdir sf_Projects
root's password:
makker@makker-lnx:/media> ls -l
total 96
drwxrwx--- 1 root vboxsf  4096 Aug 25 17:07 sf_Data
drwxrwx--- 1 root vboxsf  8192 Sep 21 13:45 sf_Documents
dr-xr-xr-x 1 root root   16384 Sep 22 09:59 sf_Downloads
drwxrwx--- 1 root vboxsf 65536 Aug 23 14:26 sf_Music
drwxr-xr-x 2 root root    4096 Sep 22 12:06 sf_Projects

makker@makker-lnx:/media> sudo chown root:vboxsf sf_Projects
makker@makker-lnx:/media> sudo chmod g+w  sf_Projects
makker@makker-lnx:/media> ls -l
total 96
drwxrwx--- 1 root vboxsf  4096 Aug 25 17:07 sf_Data
drwxrwx--- 1 root vboxsf  8192 Sep 21 13:45 sf_Documents
dr-xr-xr-x 1 root root   16384 Sep 22 09:59 sf_Downloads
drwxrwx--- 1 root vboxsf 65536 Aug 23 14:26 sf_Music
drwxrwxr-x 2 root vboxsf  4096 Sep 22 12:06 sf_Projects

To mount the folder use the command mount -t vboxsf <sharedfolder name> /media/sf_<sharedfolder name>
So:

makker@makker-lnx:/media> sudo mount -t vboxsf Projects /media/sf_Projects
makker@makker-lnx:/media> ls -l sf_Projects/
total 94
-rwxrwxrwx 1 root root  9776 Oct 12  2010 AIA Project in Amsterdam
....

Now, as you'll notice to use the shared folder by another user than root, you'll need to add that user to the vboxsf user group.

The easiest way to do that is to use the user-administration tool of your linux distribution.

To add it on the commandline, first list the groups you allready have:


makker@makker-lnx:/media> id
uid=1000(makker) gid=100(users) groups=100(users),6(disk),17(audio),20(cdrom),33(video),49(ftp)

Or:


makker@makker-lnx:/media> groups
users disk audio cdrom video ftp

Then use usermod (as root or via sudo) to add the group:


makker@makker-lnx:/media> sudo /usr/sbin/usermod -g users -G disk,audio,cdrom,video,ftp,vboxsf makker
makker@makker-lnx:/media> groups
users disk audio cdrom video ftp vboxsf

Mind that you have always a primary group indicated with lowercase '-g', and several secondary groups indicated with capital '-G' and a comma separated list (don't use spaces).

That's it.

Decomposing Virtual Machines

2011-09-15T06:47:00.000-07:00

It's been a while since my last post, but here's a new one.
This summer I got a VirtualMachine to do a Webcenter workshop. The VM turned out to be 30GB in size. This 30GB is basically one big Virtual Disk of logically 50GB. That is: it's dynamically allocated and can grow until 50GB.
Although it is a VirtualBox VM, the disk uses VMDK format, which is the VMware virtual disk format.
The disk contained the OS (of course), the install/setup files of the database, Weblogic Suite, Repository Creation Utillity and WebcenterSuite, and the installation itself.
This means that after installing the OS, the setup files are copied to the VM into a staging directory, from which the installation is done. The installation is put in a /u01/app folder, but physically in the same disk after the setup files. So deleting the setup files won't decrease the VM a bit.

I found it a nice project to try to split the VM getting it into a more suitable size.

The project turned out to be about the following steps:

Add new virtual disks for the Oracle/Webcenter installation and the staging-area (the setup files)
Partition, format and mount the disks
Copy/Move the staging and installation folder to their particular disks
Detach the staging disk
Transform the OS disk from VMDK to VDI
Shrink/Compact the OS disk

1. Add new virtual disks
Adding a disk to the VM is pretty simple: goto VM settings and node Storage. I'm sorry the screendumps are in dutch. Some how I got a Dutch VirtualBox installation.

Click the add disk Icon on the Sata Controller node.
Choose "create new disk":

Choose VDI as disk format. VDI is the "Virtual Disk Image" format of VirtualBox. VMDK is the "Virtual Machine Disk" format of VMware. Later I'll show how to create a VDI image out of a VMDK, but it's better to choose the right format right away. Especially because VirtualBox can't compact a VMDK.

For "transportable" VM's choose the option "Dynamically Allocated". That way the Disk Image is only the size of the needed space.

Then choose a location and preferably a sensible name. Then set the (maximum) size on an appropriate value.

Now, you could add all the disks you need right a way. In my case I need two extra disks: one for the installation (disk2; disk1 one is for the OS: the root disk) and one for staging (named the stage disk here). But it is important to know which disk get's which device name. I think you can assume that the disk that is first in the list gets "/dev/sda", the second "/dev/sdb" and so this staging disk "/dev/sdc". But I don't take the risk and add a disk and format it one by one (and thus do a restart each new disk).

2. Partition, Format and mount disks
This can in fact in two ways:

The raw way: partition, format and mount the raw disk.
Use Logical Volume Manager to create a Logical Volume in a Volume Group

2.1 the raw way:
First check with fdisk -l the disk devices:

[root@server1 ~]# fdisk -l

Disk /dev/sda: 53.6 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        6527    52323705   8e  Linux LVM

Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        2611    20971519+  8e  Linux LVM

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/dm-0: 47.7 GB, 47747956736 bytes
255 heads, 63 sectors/track, 5805 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 5804 MB, 5804916736 bytes
255 heads, 63 sectors/track, 705 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-1 doesn't contain a valid partition table

Here you see that /dev/sdc does not contain a partition table (ignore /dev/dm-0 and /dev/dm-1).
Then partition it with fdisk /dev/sdc (the device that you want to partition) and enter the following commands:

n (new partition)
p (for primary)
1 (first partition)
2 x enter, accepting the defaults for first and last cylinder
w (write partition table to disk)

[root@server1 ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305):
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Then you can create a filesystem on it (format). Based on the version of the Linux distro you can choose for ext3 or ext4 (older systems have ext2). You can do this by issueing the command mkfs.ext4 and ofcourse choose y(es) at the question to proceed anyway:

[root@server1 ~]# mkfs.ext4 /dev/sdc
mke4fs 1.41.12 (17-May-2010)
/dev/sdc is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done        
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first.  Use tune4fs -c or -i to override.

Then the disk is usable but you need to mount it. Since it in this case is a staging disk I want to mount it as such. So I create a directory /stage in the root:

[root@server1 /]# mkdir stage
[root@server1 /]# ls -l
total 186
...
drwxr-xr-x   2 root   root      4096 Sep  9 17:43 stage
...

Then create an entry in the /etc/fstab:

...
/dev/sdc                 /stage                  ext4    defaults        1 2
...

Of course it is most convenient to copy an existing row and edit it in the above way. Take care of editing the file system correctly (ext4 in this case).
You can mount the disk with mount /stage.
Then the owner is automatically root:root. To enable another user (eg. oracle) to use it you can create a directory oracle in it and change the owner to oracle:

[root@server1 ~]# cd /stage
[root@server1 stage]# mkdir oracle
[root@server1 stage]# chown oracle:oinstall oracle
[root@server1 stage]# ls -l
total 20
drwx------ 2 root   root     16384 Sep 15 17:51 lost+found
drwxr-xr-x 2 oracle oinstall  4096 Sep 15 18:05 oracle

2.2 Logical Volume Manager

Using a Logical Volume Group with a Logical Volume gives you a little more flexibility in extending your disk. Also, using the Logical Volume manager is a little simpeler. Since there is a nice gui for it in Gnome.
Start it using the administration menu:

You need to provide the root password.
Then you get in the following screen:

Here you see an uninitialized disk. Actually it is the disk that I used in my previous example. So it is (in my case at least) already partitioned and formated. That would not be the case with new disk of course.
So first you have to initialize it and then it asks to partition it.

It gives an error and suggests a reboot. That I did and I just re-initalized it. Which went ok.
After it is initialized you'll find it in the Unallocated Volumes node. You can add it to an existing volume group. That is handy if you want to extend it. Especially in an enterprise environment that needs to extend a volume with more space (for a growing database or something).

In this case we choose to create a new Volume Group.

Give it an appropriate name. You can choose an extent size. You'll probably don't want to create it too small, since that would probably cause an extend quite often.

Then you can create a Logical Volume in it:

Choose Ext4 for the filesystem. Click on "Use Remaining" to span the complete partion.
You can check "Mount" and "Mount when rebooted".
After pressing "Ok" you get the following dialog:

Of course you choose "Yes". If you check the /etc/fstab you'll find an entry like:

/dev/VolGroup02/LogVol00                /stage          ext4    defaults        1 2

3. Copy/Move the files to the new disk
Well, I assume I don't have to explain this. Fortunately in my case the installation is done in /u01/app/oracle. So I could name my mount point /u01. It's important to choose the mount point precisely as the first directory of the directory path (see step 2). To do so, you'd probably rename the directory with the installation first. The result should be that the mount point and the rest of the directory structure matches the original path. Otherwise the installation would break and the whole exercise is quite needless.

4. Detach the staging disk
You probably don't want to ship the staging disk with the VM. In this decomposition project we created it to get the staging files out of the root disk. Normally when creating a VM you create an empty staging disk and copy the setup files into it. After doing the installation you don't need the stage disk anymore. But it can be handy to refine an installation later on or redo an installation in an other VM. In those cases you'll want to backup the disk.

But anyway, at this point you can detach it from your VM. But before you do that it is extremely important to remove or comment the corresponding line (see the line above) from the /etc/fstab. If you don't you'll not be able to restart the VM again. Because Linux tries to mount the disk that can't be found when removed from the storage.

Then you can shutdown the machine, go to the storage settings of the machine and remove the disk from the sata controller.

If you removed the disk from the /etc/fstab then you can safely startup the VM again.

5. Transform the OS disk from VMDK to VDI
To be able to compact the disk, you'll neet to transform it to VDI. VirtualBox can't convert it in one go. You'll need to convert from VMDK to Raw format first and then from Raw to VDI.

5.1 Convert from VMDK to Raw
Open a command window or terminal/console session and cd to your VirtualBox installation. My host is a Windows 7 laptop and my VirtualBox Installation is in c:\Program Files\Oracle\VirtualBox>. In that directory you'll find the Vboxmanage tool. To convert the disk to raw you issue the command:

VboxManage internalcommands converttoraw  -format vmdk <path to vmdk disk> <path to new raw disk>

Spaces in folders are not so convenient, since it can break the option list in the command line. Enclose the path's to the input (vmdk) and output (raw) disk with quotes.
For example:

c:\Program Files\Oracle\VirtualBox>VboxManage internalcommands converttoraw -format vmdk "c:\Users\makker\VirtualBox VMs\PTS_WebCenter_PS4\PTS_DWN_Webcenter_PS4-disk1.vmdk" "d:\VirtualMachines\VirtualBox\Webcenter11g\PTS_Webcenter_PS4-disk1.raw"
Converting image "c:\Users\makker\VirtualBox VMs\PTS_WebCenter_PS4\PTS_DWN_Webcenter_PS4-disk1.vmdk" with size 536870912
00 bytes (51200MB) to raw...

It should be needles to say that it is not so wise to do this while running the Virtual Machine. Make sure it is down before running the conversion.

5.2 Convert from Raw to VDI
The command to convert it from raw to VDI is quite similar. Remarkably the conversion to raw is apparently an "internal command" to virtual box, but the conversion from raw to another disk format is not. The command is:

VboxManage convertfromraw <path to raw disk> <path to (new) VDI disk> --format vdi --variant Standard

For example:

c:\Program Files\Oracle\VirtualBox>VboxManage convertfromraw d:\VirtualMachines\VirtualBox\Webcenter11g\PTS_Webcenter_PS
4-disk1.raw "c:\Users\makker\VirtualBox VMs\PTS_WebCenter_PS4\PTS_Webcenter_PS4-disk1.vdi" --format vdi --variant Standard
Converting from raw image file="d:\VirtualMachines\VirtualBox\Webcenter11g\PTS_Webcenter_PS4-disk1.raw" to file="c:\Users\makker\VirtualBox VMs\PTS_WebCenter_PS4\PTS_Webcenter_PS4-disk1.vdi"...
Creating dynamic image with size 53687091200 bytes (51200MB)...

For a big disk, this can take quite an amount of time. In my case it took about 20 minutes per conversion.

Having done this, you can remove the VMDK from the storage in the Virtual Machine settings and add the VDI disk. You can do that in the same way as shown above with creating a new disk, except in stead choose "existing disk".
Make sure you put the disk on the correct sata port. For the root disk this should be port 0:
The root disk is of course the first one to be found by Linux and should get /dev/sda as device name. If it is your second it should get port 1 and thus named /dev/sdb by Linux, and so on.

6. Shrink/Compact the OS disk
Now, the disks are VDI, the root disk only contains the OS by now, since we moved the staging and installation in step 3. However: the disk is still huge: 30GB while the OS would only take about 5 to 6 GB (OS plus swap space). So it needs to shrink.
But for the compact tool, this 25GB remaining space is written and thus data. We first have to zero out the space. There is a tool for that: zerofree. Unfortunately it is not in the Oracle Enterprise Linux distribution. But OEL 5 is compatible with RedHat5. So I found my rpm here. Be sure you download the correct one (i386 or x86_64). Install it as root with

rpm -ihv zerofree-1.0.1-5.el5.i386.rpm

Well and then we have a problem. Zerofree only works on a readonly filesystem.
So we have to mount the rootdisk read only. That's not so easy since there are several processes that runs against the root disk.
Most of them are ruledout when booting the system in single user mode.
To do so login as root and issue:

[root@server1 ~]# telinit 1

This causes the system into single user mode and doing so terminating most of the services. You'll find yourself logged on as root.
However, probably there will be still some services running.
To check them out:

sh-3.2# service --status-all|grep runnnig

This will list all possibly running (or explictly stated "not running" services.
You can stop services by:

sh-3.2# service iscsid stop

Stop all running services and kill all p
Then you can try to remount the root volume rewritable:

sh-3.2# mount -n -o remount,ro -t ext3 /dev/VolGroup00/LogVol00 /

If that succeeds then you can zerofree the filesystem by:

zerofree /dev/VolGroup00/LogVol00

After this we're ready to compact the disk. You can shutdown the system by issueing the command:

sh-3.2# init 0

Go to the VirtualBox install directory again in a command window.
To compact the disk, VitualBox has the command:

VboxManage modifyhd <path to the VDI disk file> --compact

For example:

c:\Program Files\Oracle\VirtualBox>VboxManage modifyhd "c:\Users\makker\VirtualBox VMs\PTS_WebCenter_PS4\PTS_Webcenter_PS4-disk1.vdi" --compact

After a while the file is probaly a lot smaller.

Conclusion
A first blog in a long time and it became quite a story. I feel like it looks like a little VirtualBox and Oracle Enterprise Linux administration course.

The last few months I created several VM's with environments to do courses (Virtual Course Environments I call them). And several of these tasks help me with this. In fact, see it as a toolbox of VCE creation. And the 30GB VM? Exported to an OVA file it is only 11GB now...

BPM 11g Advanced Workshop

2011-06-30T06:17:00.000-07:00

Last week I was on a trip to Lissabon for a BPM 11g Advanced Workshop. In the same event there were also advanced courses on ADF and Webcenter.

It was a pleasant week since there were showers all over the place in the Netherlands. Since the pentacost weekend there nearly wasn't a dry moment over here. Portugal was cloudless and the ocean was really refreshing.

The BPM workshop was actually about building a POC on the process of handling a request on increasing the credit limit on a credit card of a fictive bank. You had to create an application with a team of 3 or 4 members involving:

ADF Web application for initiating the process using either Webservice or (better) EDN.
Advanced Business Rules for determining the allowed increase
ADF Human Task for approval screens
Implementing Conversation/Correlation constructs using BPEL to be able to have cancel or fraud-detection messages influencing the running process. BPM currently lacks functionality to handle multiple conversations/correlations.
BAM, BAM, BAM (seen as highly convincing in a POC)

Although it was not required, I for myself was so stubborn to insist on creating a Custom ADF Human Task screen to initiate the BPM Process. Since I've ran in to that problem in an earlier BPM11g try-out. I started on an investigation on what to do to rebuild an BPM10R3 project into BPM11g.

BPM11g is a real new product. The way I see it, there is little from the old BPM10 product. You could find a significant part from BPM10 in the modeller (studio). But the engine part is replaced by the one used by BPEL Process Manager as a base. On top of that all the parts needed to run BPM processes are added.
BPM10 projects tend to have loads of PBL code, snippets of program code or sometimes larger amounts of code to implement a specific automatic action. For most it is used to build up request messages to call (web)services and process the response messages. But it is not unusual to have script-tasks to calculate outcomes of business rules, or other business values. Often this is done on methods of objects in object model in the Business Category. It is the part of BPM10 I was suprised of the most. Since you would not find this in BPEL (if you would left out embedded java tasks) and it should not be needed in a tool that is targeted to Business Users. In BPM11g it is not needed, because you can use XSLT and XPath (or alike) expressions to process messages. And ADF BC to display corresponding data in your Human Task Screens.
I learned that Oracle is going to support scripting in the BPM12c release. But it will be deprecated upon introduction. Since you would not need it and should not use it. Only to be able to migrate your projects to 12c. But from there onwards you should replace your code with Data associations and Transformations. Also as a best practice: only keep the data in your project that is really needed, for example Primary/Foreign key values. All relating data needed to display in the screens you can query along using ADF BC. And that's basically what I wanted to learn about.

Another neat subject we've got presented was about the upcoming feature pack that is scheduled for the summer. It is explicitly stated as not being a Patch Set. Because it's not primarily focused on bugfixing but on adding new features. To name a few:

Business Rules made simpeler
Support for conversation/correlation on message exchanges with multiple external systems. As you can and should do now with BPEL
Migration from Oracle Workflow!

The latter I found very surprising. It has been for years stated that Oracle ends with Oracle Workflow from the end of support of the 10g product-line. A very big pity if you're asking me. I would very much support the idea that Oracle should give Oracle Workflow over to the Open Source community. But that aside. Since in 2004 Collaxa was acquired because of the BPEL PM, it was thought to be the replacement for OWF. Although until SOASuite11g there was no alternative foreseen for OWF's Business Event System. With SOASuite11g there is an alternative in ways of EDN (Event Delivery Network). But because of this migration tool apparently BPM is rightly seen as the replacement of OWF. Although it is kind of late. Because I suppose there are not yet so many OWF Customers left. Oracle should have already convinced them all over to BPEL. Particularly Oracle EBusiness Suite customers could still have complex custom workflows. For those customers the migration to BPM11g would be very interesting.

After all it was a very good and recommendable course. Having worked and talked and dined with more or less Oracle SOA/BPM Specialist all over Europe and the people from Oracle Product Mgt. I hope I have been able to give you a glimpse on the week to get tasty for another round of this event. I hope I can get into the ADF or Webcenter class... Not only because Lissabon is nice...

And now we wait ... , for

The sun breaking through in Holland
A nice vacation in France
ehm, oh yes: the SOA/BPM feature pack to be released this summer

Oracle Fusion Applications 11g Release 1 (11.1.1.5.0)

2011-06-29T00:37:00.000-07:00

Since June 7 Oracle has quietly released Fusion Applications v1.0

It can be downloaded from http://edelivery.oracle.com at this moment from it's own Fusion Application section.

If you have 64-bit Linux available you may even install it, without to many hurdles:

http://onlineappsdba.com/index.php/2011/06/15/install-oracle-fusion-applications-in-10-steps/

Official information about the available 'modules' can be found here http://www.oracle.com/us/products/applications/fusion/index.html

The official pricing is available (since June 27th) from:

http://www.oracle.com/us/corporate/pricing/fusion-applications-price-list-418746.pdf see also

http://www.cio.com/article/685156/Oracle_Fusion_Applications_Pricing_Revealed

for a discussion.

An overview can be found on http://www.oracleappshub.com/fusion/oracle-fusion-and-oracle-fusion-applications-overview/ where you can also find a short description of Fusion Applications Supply Chain, Procurement and Project Portfolio Management.

The documentation can also be downloaded from edelivery.oracle.com or may be viewed more easily at

http://www.orastudy.com/oradoc/selfstu/fusion/index.htm

or http://www.orastudy.com/oradoc/selfstu/fusion/sysint_role.htm for the documentation for the System Implementor/ntegrator job role.

The first Fusion Application book by Richard Bingham can be pre-viewed (and ordered) from McGraw-Hill:

TOC http://www.mhprofessional.com/downloads/products/0071750339/0071750339_toc.pdf

Chapter 1 http://www.mhprofessional.com/downloads/products/0071750339/0071750339-ch01.pdf

Chapter 2 http://www.oracle.com/technetwork/articles/managing-fusion-apps-418611.pdf

Ofcourse I will probably not be able to not discuss this this afternoon while delivering another Oracle Product

Portfolio Overview training. If you are interested, check out:

http://www.darwin-it.nl/training/training/oracle-overview/

Webcenter 11g VM: Add Spaces

2011-06-15T11:55:00.000-07:00

As mentioned in my earlier posts, there is a Oracle Virtual Box VM for Webcenter. But it is a VM with Webcenter Portal (11gPS3: 11.1.1.4). It turns out that Webcenter Spaces is not installed. And since I needed just that for my course-preperations I went looking for a VM that contains spaces. I was directed to the Pre-built Appliances page for the spaces VM but it turns out that the links to download the files were removed, because "in the near future" a new version will be made available.

So that leaves me with the Webcenter Portal VM. On OTN there is only one download to install Webcenter. So the software of webcenter including spaces is on the VM. The database just does not contain the repository schema's for spaces. So what to do? Well apparently the following:

Run RCU (Repository Creation Utility) for Spaces against the database
Extend the webcenter weblogic domain
Fiddle around somewhat to make things actually working (I found that necessary)

Run the RCU
To be able to run the RCU you have to download the right version (11.1.1.4) and unzip it. You can download it from here. Yannick Ogena's blog was a good starting point here, by the way. Expand the nodes under "Prerequisites & Recommended Install Process" of the 11.1.1.4.0 version. Look for "4 Repository Creation Utility (11.1.1.4.0) for Linux" and download that one. Unzip it somewhere so that you can reach it from inside the VM. A shared folder would be helpful. You could of course FTP/SCP it to the VM, but I don't like that idea since it will unnecessarly expand the vitual disks.
Then start "rcu" from the bin folder.
If you shrunk the database like I did following my previous post on this, you'll hit the error that your processes parameter is too small. It has to be at least 200.
If so increase it like:

SQL> alter system set processes=200 scope=spfile;
System altered.
SQL> shutdown immediate;
...
SQL> startup

In the "Database Connection Details" screen enter:

Hostname: localhost
Port: 1521
Service name: orcl
username: sys
password: welcome1 (all database passwords in the Webcenter Portal VM are "welcome1" so it might be handy to use that for all other schema's too)

Then you'll see a screen like:

Check the boxes like above. It will suggest however to create a new prefix "DEV1". Instead select the existing prefix "DEV". Then you'll mention that some of the schema's allready exist and won't be created. That's fine. Then finish the wizard keeping the defaults and confirm with "OK" after the checks. It will ask for creating non existent Tablespaces. Just confirm with OK.

At the end you can press close to close the RCU.

Extend the domain
Now the repository is ready, to extend the domain. To do so, start the domain configurator. You can find it in "/u01/app/oracle/product/Middleware/wlserver_10.3/common/bin". Start the script "config.sh".

It starts with the screen:

Choose Extend existing Weblogic domain and look for the webcenter domain. It is the folder "/u01/app/oracle/product/Middleware/user_projects/domains/webcenter":

When choosing next you'll be able to select the options to add to the domain. Check the Webcenter options following the next example screendumps:

Then, you'll have to add the connection properties of the different repository schemas. What you can do is select every schema that does not have orcl.localdomain as service and localhost as Host Name. Then enter in the fields in the top of the screen orcl.localdomain as service (just "orcl" is not enough, you have to add "localdomain" as a domain name). Since all the schema's have "welcome1" as a password, you can enter that in the Schema password field. Do not touch the Schema owner field. That won't be changed.

Then finish the wizard. After finishing up the wizard the domain has got some new Manged Servers added. You can start the Admin Server using the "startWeblogic.sh" command in "$DOMAIN_HOME/bin" (in our case: "/u01/app/oracle/product/Middleware/user_projects/domains/webcenter/bin").
After having started the admin server you can browse to http://localhost:7001/console. Log in as:

user name: weblogic
password: welcome1

At the left you'll see a portlet called "Domain Structure". Open up the "Environment" node and click on servers.
A table with the mananged servers is given:

Except for "WC_Collaboration" all the WC_% servers are added. Each with it's own port number.

A managed server can be started with the "startManagedWebLogic.sh" command with as an extra parameter the name of the managed server like:

startManagedWebLogic.sh WC_Spaces

You can start just the managed servers you'll need.

Tuning the domain
However, when you start the added managed servers, you'll find the error: " <Getting boot identity from user.> " in the log. It seems you can enter the weblogic user but it will fail.
I tried to add the info using the weblogic Admin Server console. But that did not work.

When you start the managed server for the first time, it will add a folder with the name of the managed server in the servers folder within the domain folder, like:

[oel50wc oracle /u01/app/oracle/product/Middleware/user_projects/domains/webcenter/servers]$ ls
AdminServer     domain_bak   WC_Collaboration  WC_Portlet  WC_Utilities
AdminServerTag  UCM_server1  WC_CustomPortal   WC_Spaces

That is: when it is not started for the first time, it will not be there.
To solve the boot-identity problem, add a folder called "security" to the managed-server folder. In that folder a file called "boot.properties" is expected. You can copy the security folder from for example the "UCM_server1" or "WC_CustomPortal" managed servers. Edit the boot.properties file with the following values:

username=weblogic
password=welcome1

At startup of the managed server both values will be encripted.

To start the managed servers you can adapt the script vmctl.sh that is provided in the VM. I did not like that, since it has lot's of double code. I like a more modular approach.

When you start all of the managed servers after some time the VM will be very busy with ... swapping! Each of the managed server is started with it's own JVM session. And it turns out that each VM gets an initial heap of 512MB and a max of 1024MB. I think that is somewhat oversized for a demo VM. You can see it in the output of the startup script. Since the first thing it does is to log the memory properties:

JAVA Memory arguments: -Xms512m -Xmx1024m -XX:CompileThreshold=8000 -XX:PermSize=128m  -XX:MaxPermSize=512m

The settings can be adapted in the "setDomainEnv.sh" script in the bin folder of the webcenter domain.
There you'll find the following values:

XMS_SUN_64BIT="256"
export XMS_SUN_64BIT
XMS_SUN_32BIT="512"
export XMS_SUN_32BIT
XMX_SUN_64BIT="512"
export XMX_SUN_64BIT
XMX_SUN_32BIT="1024"
export XMX_SUN_32BIT
XMS_JROCKIT_64BIT="256"
export XMS_JROCKIT_64BIT
XMS_JROCKIT_32BIT="256"
export XMS_JROCKIT_32BIT

The VM uses 32-bit Linux and the Sun JVM. So the values to change are "XMS_SUN_32BIT" and "XMX_SUN_32BIT" for the min and max heap size respectively. I changed them to "256" and "512". Since each managed server, including the admin server uses the same script, these values are the same for each server. If you might need to adapt for just one managed server then you probably have to copy the scripts specificly for that particular managed server. Or install a node manager...

Together with a downsized database, it has to make to run the VM better.

Webcenter 11g VM: JDeveloper project location

2011-06-09T08:14:00.000-07:00

The Webcenter VM on OTN contains a tutorial. The tutorial let you do some excersises with Jdeveloper. I put my projects under /home/oracle/Jdeveloper/mywork. But then it lets you unzip a package into $JDEV_USER_DIR/mywork. It turns out that it the environment variable points to: /u01/JDevApps. I found that it is set in the ".bashrc" script.

Webcenter 11g VM: downsizing the database

2011-06-09T05:27:00.000-07:00

This week I started with Webcenter 11g, using the Oracle VirtualBox VM that can be downloaded here.
One of the tips upfront was that the NAT network adapter should be "Internal Network" or "Host Only". Because otherwise the UCM part of the hands-on would not work.

I also changed the memory settings to 3GB at first. But that caused my Windows 7 to stutter. Aparently Windows has became very memory consumptive. On my Linux (Open Suse) it would not have be too much of a problem to raise the VM to 3GB. An 8GB laptop would be nice. So I brought the settings back to a more modest 2,5 GB.

But then I encountered that the install of Oracle DB 11g was pretty basic. And that means a memory consumption of 700GB only the lonely for the database. I remembered my earlier post to tune Oracle DB11g together with SoaSuite10g on a OEL5 VM.

It was basically about resizing the memory. What I did was to startup an XE database. There I looked at the basic memory settings. For convenience I created a plain init.ora.

For the non-dba's amongst you, you can do that by loging on as internal with:

sqlplus "/ as sysdba"

having set the ORACLE_HOME and ORACLE_SID:

ORACLE_HOME=/u01/app/oracle/product/11.1.0/db_1
ORACLE_SID=orcl

When you logged on as internal you can create an init.ora (also called a pfile) with:

create pfile from spfile;

Then you'll find an init.ora in the $ORACLE_HOME/dbs folder.

For an Oracle XE database the most interesting settings I found were:

java_pool_size=4194304
    large_pool_size=4194304
    shared_pool_size=67108864
    open_cursors=300
    sessions=20
    pga_aggregate_target=70M
    sga_target=210M

The sga_max_size was not set.
So I changed the 11g database with these settings, created a spfile from the pfile again (create spfile from pfile) started it again.

My initorcl.ora:

#orcl.__db_cache_size=222298112
#orcl.__java_pool_size=12582912
orcl.__java_pool_size=10M
orcl.__large_pool_size=4194304
....
#orcl.__pga_aggregate_target=159383552
orcl.__pga_aggregate_target=70M
#orcl.__sga_target=478150656
orcl.__sga_target=210M
orcl.__shared_io_pool_size=0
#orcl.__shared_pool_size=234881024
orcl.__shared_pool_size=100M
orcl.__streams_pool_size=0
...
#*.memory_target=635437056
*.open_cursors=300
#*.processes=150
*.sessions=20
...
*.sga_max_size=250
...

Mark that I unset the db_cache_size and memory_target. I also replaced the processes parameter with the sessions parameter being 20. These two parameters relate to eachother, one computed from the other.

I found that I had a database of 145MB!But that's some what too small, especially the shared-poolsize being about 64M, while the sga_max_size was 145M.

I changed my sga_max_size to explicitly 250M and the large_pool_size to 100M:

SQL> alter system set sga_max_size=250M scope=spfile;
System altered.
SQL> alter system set shared_pool_size=100M scope=spfile;
System altered.

Then restarting the database resulted in a database of 250M:

Total System Global Area 263639040 bytes
Fixed Size 1299284 bytes
Variable Size 209718444 bytes
Database Buffers 50331648 bytes
Redo Buffers 2289664 bytes

That looks better to me. And having it posted again refreshes it for me.

Update: I now see that in the initorcl.ora I had a sga_max_size of 250. But that should be 250M... Maybe that caused the shared_pool_size and sga_max_size too small.

Oracle Fusion Middleware PS4 (11.1.1.5)

2011-05-09T04:36:00.000-07:00

This morning I talked with a few coleagues about BPM 11g. And the current patch set.
Back to my PC and Google Reader I found out that Oracle released PS4 (11.1.1.5) last friday.

So I'm very curious what this release brings for SoaSuite 11g. I found that the announcment is made in note 1316076.1 at My Oracle Support. There is a link to a bug fix link: "Fixed Bugs List - Patch Set 4 (11.1.1.5.0) for Oracle Fusion Middleware 11g [ID 1316062.1]". But this list suggest that there was no need for a Patch Set... Or it must be that Weblogic increased a version number to 10.3.5.

The list of product included in the Patch set are:

Oracle WebLogic Server 11g R1 (10.3.5)
Oracle Fusion Middleware Repository Creation Utility 11g R1 (11.1.1.5.0)
Oracle SOA Suite 11g R1 (11.1.1.5.0)
Oracle WebCenter 11g R1 (11.1.1.5.0)
Oracle Identity Management 11g R1 (11.1.1.5.0)
Oracle Fusion Middleware Web-Tier Utilities 11g R1 (11.1.1.5.0)
Oracle JDeveloper 11g R1 (11.1.1.5.0)
Oracle Application Development Runtime 11g R1 (11.1.1.5.0)
Oracle Application Development Framework 11g R1(11.1.1.5.0)
Oracle Enterprise Content Management 11g R1 (11.1.1.5.0)
Oracle Enterprise Repository 11g R1 (11.1.1.5.0)
Oracle Business Process Management Suite11g R1 (11.1.1.5.0)
Oracle Application Integration Architecture Foundation Pack 11g R1 (11.1.1.5.0)
Oracle Service Bus 11g R1 (11.1.1.5.0)

But I'm so curious on the enhancements in SoaSuite en BPM Suite.

SoaSuite including BPM Suite can be downloaded here. But I hope Oracle comes with a Soa/BPM VirtualBox appliance for this version soon (currently it's 11.1.1.3).

SOA Gateway Service Invocation Framework: Test Service

2011-04-13T02:36:00.000-07:00

This week I'm delivering a workshop on SOA Gateway of E-Business Suite 12.1.
On implementing the Service Invocation Framework there is a good blog-article here.
I found myself running into an error using the "raise in java" button from the test screen. And I was pretty sure that my values were ok.
The error was:

Exception occured in Java Raise - Exception when Invoking Web Service -> oracle.apps.fnd.wf.bes.InvokerException: HTTP transport error: 
javax.xml.soap.SOAPException: java.security.PrivilegedActionException: 
javax.xml.soap.SOAPException: Message send failed: For input string: ""

Since I have a background in Pl/Sql and using the Oracle Workflow Business Event system in Pl/Sql, I got some of my BES-example code from the shelf and build a package to invoke the event from Pl/Sql.

Here is the package specification of my example code:

create or replace package xxx_soa_events is
  /*******************************************************************************************
    Package with functionality to support SOA Gateway Service Invocation Framework Events
  
    #Version  0.1
    #Author   Martien van den Akker, Darwin-IT Professionals
    #Usage    Methods to get system_parameters
  
    <b>Change History</b>
  
    When Who: What
    <ul>
     <li>07-apr-2011 Martien van den Akker: Initial Creation</li>
    </ul>
  ********************************************************************************************/

  /* Name create person event */
  c_evt_create_person constant varchar2(100) := 'darwin.apps.soa.create_person';
  /* Event Rule result Error */
  c_rst_error constant varchar2(10) := 'ERROR';
  /* Event Rule result Success */
  c_rst_success constant varchar2(10) := 'SUCCESS';
  /* Event Rule result Warning */
  c_rst_warning constant varchar2(10) := 'WARNING';
  /* HZ Namespace declaration*/
  c_hz_ns_decl   constant varchar2(150) := 'xmlns:hz="http://xmlns.oracle.com/apps/hz/soaprovider/plsql/hz_party_v2pub/"';
  c_hz_ns_abbrev constant varchar2(10) := 'hz';
  c_tag_soahdr   constant varchar2(20) := 'SOAHeader';
  c_tag_Resp     constant varchar2(20) := 'Responsibility';
  c_tag_Respapp  constant varchar2(20) := 'RespApplication';
  c_tag_secgrp   constant varchar2(20) := 'SecurityGroup';
  c_tag_nlslang  constant varchar2(20) := 'NLSLanguage';
  c_tag_orgid    constant varchar2(20) := 'Org_Id';
  c_val_Resp     constant varchar2(20) := 'HZ_TCA_MANAGER';
  c_val_Respapp  constant varchar2(20) := 'AR';
  c_val_secgrp   constant varchar2(20) := 'STANDARD';
  c_val_nlslang  constant varchar2(20) := 'AMERICAN';
  c_val_orgid    constant varchar2(20) := '';
  c_par_inp_hdr  constant varchar2(20) := 'WFBES_INPUT_HEADER';

  c_hz_crepers_ns_decl   constant varchar2(150) := 'xmlns:cre="http://xmlns.oracle.com/apps/hz/soaprovider/plsql/hz_party_v2pub/create_person/"';
  c_hz_crepers_ns_abbrev constant varchar2(10) := 'cre';
  c_tag_inppar           constant varchar2(20) := 'InputParameters';

  /*******************************************************************************************
    Default Rule
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Default Rule function as an example implementation for a pl/sql subscription
  
    #param    p_subscription_guid    Guid of the subscription of which this rule function is 
                                     the implementation
    #param    p_event                event object
    #return   status code
  ********************************************************************************************/
  function default_rule(p_subscription_guid in raw,
                        p_event             in out wf_event_t)
    return varchar2;
  /*******************************************************************************************
    Default Rule
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Rule function to save events as a response in Service Invocation Framework
  
    #param    p_subscription_guid    Guid of the subscription of which this rule function is 
                                     the implementation
    #param    p_event                event object
    #return   status code
  ********************************************************************************************/
  function save_response_events(p_subscription_guid in raw, p_event in out wf_event_t)

    return varchar2;
  /*******************************************************************************************
    Create a xml element
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Raise Event to Create a Person via SIF
    
    #param    p_ns_abbrev               optional namespace abbreviation
    #param    p_tag_name                name of the tag
    #param    p_value                   element value
    #param    p_attrib                  optional attribute/value pair
    #return   element
  ********************************************************************************************/
  function create_element(p_ns_abbrev in varchar2,
                          p_tag_name  in varchar2,
                          p_value     in varchar2,
                          p_attrib    in varchar2) return varchar2;
  /*******************************************************************************************
    Create a hz SOA Header
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage   Create the HZ SOA Header denoting the responsibility
  
    #return   element
  ********************************************************************************************/
  function create_hz_soaheader return varchar2;
  /*******************************************************************************************
    Create a create person data object
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage   Create the HZ SOA Header denoting the responsibility
    
    #param p_person_first_name           in varchar2
    #param p_person_middle_name          in varchar2
    #param p_person_last_name            in varchar2
    #param p_person_initials             in varchar2
    #param p_person_name_phonetic        in varchar2
    #param p_person_first_name_phonetic  in varchar2
    #param p_person_last_name_phonetic   in varchar2
    #param p_middle_name_phonetic        in varchar2
    #param p_date_of_birth               in varchar2
    #param p_place_of_birth              in varchar2
    #param p_gender                      in varchar2
    #param p_declared_ethnicity          in varchar2
    #param p_created_by_module           in varchar2
    #return   element
  ********************************************************************************************/
  function create_hz_createperson_data(p_person_first_name          in varchar2,
                                       p_person_middle_name         in varchar2,
                                       p_person_last_name           in varchar2,
                                       p_person_initials            in varchar2,
                                       p_person_name_phonetic       in varchar2,
                                       p_person_first_name_phonetic in varchar2,
                                       p_person_last_name_phonetic  in varchar2,
                                       p_middle_name_phonetic       in varchar2,
                                       p_date_of_birth              in varchar2,
                                       p_place_of_birth             in varchar2,
                                       p_gender                     in varchar2,
                                       p_declared_ethnicity         in varchar2,
                                       p_created_by_module          in varchar2)
    return clob;
  /*******************************************************************************************
    Raise Create Person Event
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Raise Event to Create a Person via SIF
  
    #param p_event_key                   event key
    #param p_person_first_name           in varchar2
    #param p_person_middle_name          in varchar2
    #param p_person_last_name            in varchar2
    #param p_person_initials             in varchar2
    #param p_person_name_phonetic        in varchar2
    #param p_person_first_name_phonetic  in varchar2
    #param p_person_last_name_phonetic   in varchar2
    #param p_middle_name_phonetic        in varchar2
    #param p_date_of_birth               in varchar2
    #param p_place_of_birth              in varchar2
    #param p_gender                      in varchar2
    #param p_declared_ethnicity          in varchar2
    #param p_created_by_module           in varchar2
  ********************************************************************************************/
  procedure raise_create_person_event(p_event_key                  in varchar2,
                                      p_person_first_name          in varchar2,
                                      p_person_middle_name         in varchar2,
                                      p_person_last_name           in varchar2,
                                      p_person_initials            in varchar2,
                                      p_person_name_phonetic       in varchar2,
                                      p_person_first_name_phonetic in varchar2,
                                      p_person_last_name_phonetic  in varchar2,
                                      p_middle_name_phonetic       in varchar2,
                                      p_date_of_birth              in varchar2,
                                      p_place_of_birth             in varchar2,
                                      p_gender                     in varchar2,
                                      p_declared_ethnicity         in varchar2,
                                      p_created_by_module          in varchar2);
end xxx_soa_events;

And below the body of the example package:

create or replace package body xxx_soa_events is
  c_package_name constant varchar2(30) := 'xxx_soa_events';
  /*******************************************************************************************
    Create an xml open tag
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    String XML Handling
  
    #param    p_tag_name                name of the tag
    #param    p_attrib                  optional attribute/value pair
    #return   tag
  ********************************************************************************************/
  function open_tag(p_tag_name in varchar2, p_attrib in varchar2)
    return varchar2 is
    l_tag varchar2(2000);
  begin
    if p_attrib is null then
      l_tag := '<' || p_tag_name || '>';
    else
      l_tag := '<' || p_tag_name || ' ' || p_attrib || ' >';
    end if;
    return l_tag;
  end;
  /*******************************************************************************************
    Create a xml close tag
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    String XML Handling
  
    #param    p_tag_name                name of the tag
    #return   tag
  ********************************************************************************************/
  function close_tag(p_tag_name in varchar2) return varchar2 is
    l_tag varchar2(2000);
  begin
    l_tag := '</' || p_tag_name || '>';
    return l_tag;
  end;
  /*******************************************************************************************
    Create a xml element
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    String XML Handling
    
    #param    p_ns_abbrev               optional namespace abbreviation
    #param    p_tag_name                name of the tag
    #param    p_value                   element value
    #param    p_attrib                  optional attribute/value pair
    #return   element
  ********************************************************************************************/
  function create_element(p_ns_abbrev in varchar2,
                          p_tag_name  in varchar2,
                          p_value     in varchar2,
                          p_attrib    in varchar2) return varchar2 is
    l_return varchar2(32767);
    l_tag    varchar2(2000);
  begin
    if p_ns_abbrev is not null then
      l_tag := p_ns_abbrev || ':' || p_tag_name;
    else
      l_tag := p_tag_name;
    end if;
    l_return := open_tag(l_tag, p_attrib) || p_value || close_tag(l_tag);
    return l_return;
  end;
  /*******************************************************************************************
    Create a xml element
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Create an XML element
    
    #param    p_ns_abbrev               optional namespace abbreviation
    #param    p_tag_name                name of the tag
    #param    p_value                   element value
    #return   element
  ********************************************************************************************/
  function create_element(p_ns_abbrev in varchar2,
                          p_tag_name  in varchar2,
                          p_value     in varchar2) return varchar2 is
    l_return varchar2(32767);
  begin
    l_return := create_element(p_ns_abbrev => p_ns_abbrev,
                               p_tag_name  => p_tag_name,
                               p_attrib    => null,
                               p_value     => p_value);
    return l_return;
  end;
  /*******************************************************************************************
    Create a hz SOA Header
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage   Create the HZ SOA Header denoting the responsibility
  
    #return   element
  ********************************************************************************************/
  function create_hz_soaheader return varchar2 is
    l_return varchar2(2000);
  begin
    l_return := create_element(p_ns_abbrev => c_hz_ns_abbrev,
                               p_tag_name  => c_tag_soahdr,
                               p_attrib    => c_hz_ns_decl,
                               p_value     => create_element(p_ns_abbrev => c_hz_ns_abbrev,
                                                             p_tag_name  => c_tag_Resp,
                                                             p_value     => c_val_Resp) ||
                                              create_element(p_ns_abbrev => c_hz_ns_abbrev,
                                                             p_tag_name  => c_tag_Respapp,
                                                             p_value     => c_val_Respapp) ||
                                              create_element(p_ns_abbrev => c_hz_ns_abbrev,
                                                             p_tag_name  => c_tag_secgrp,
                                                             p_value     => c_val_secgrp) ||
                                              create_element(p_ns_abbrev => c_hz_ns_abbrev,
                                                             p_tag_name  => c_tag_nlslang,
                                                             p_value     => c_val_nlslang) ||
                                              create_element(p_ns_abbrev => c_hz_ns_abbrev,
                                                             p_tag_name  => c_tag_orgid,
                                                             p_value     => c_val_orgid));
  
    return l_return;
  end;
  /*******************************************************************************************
    Varchar2 To Clob
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Convert a clob to a varchar2
  
    #param    p_varchar2         varchar2
    #return   clob
  ********************************************************************************************/
  function varchar_to_clob(p_varchar2 varchar2) return clob is
    l_clob clob;
  begin
    -- Open clob
    -- create the temporary CLOB's to store the xml data that is sent to and received from the xml open gateway
    dbms_lob.createtemporary(l_clob, false, dbms_lob.session);
    dbms_lob.open(l_clob, dbms_lob.lob_readwrite);
    -- Convert the input to clob
    dbms_lob.writeappend(l_clob, length(p_varchar2), p_varchar2);
    -- Close the Clobs
    dbms_lob.close(l_clob);
    -- Return
    return l_clob;
  end varchar_to_clob;
  /*******************************************************************************************
    Clob To Varchar2
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Convert a clob to a varchar2
  
    #param    p_clob         clob
    #return   varchar2
  ********************************************************************************************/
  function clob_to_varchar2(p_clob in clob) return varchar2 is
    l_length  number;
    l_message varchar2(32767);
  begin
    l_length := dbms_lob.getlength(p_clob);
    l_length := greatest(l_length, 32767);
    if l_length > 32767 then
      l_length := 32767;
    end if;
    l_message := dbms_lob.substr(p_clob, l_length, 1);
    return l_message;
  exception
    when others then
      raise;
  end clob_to_varchar2;

  /*******************************************************************************************
    Create a hz SOA Header
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage   Create the HZ SOA Header denoting the responsibility
  
    #return   element
  ********************************************************************************************/
  function create_hz_crepers_inp_elt(p_value varchar2) return varchar2 is
    l_person_elt varchar2(32767);
  begin
    l_person_elt := create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                   p_tag_name  => c_tag_inppar,
                                   p_attrib    => c_hz_crepers_ns_decl,
                                   p_value     => p_value);
  
    return l_person_elt;
  end;

  /*******************************************************************************************
    Create a create person data object
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage   Create the HZ SOA Header denoting the responsibility
    
    #param p_person_first_name           in varchar2
    #param p_person_middle_name          in varchar2
    #param p_person_last_name            in varchar2
    #param p_person_initials             in varchar2
    #param p_person_name_phonetic        in varchar2
    #param p_person_first_name_phonetic  in varchar2
    #param p_person_last_name_phonetic   in varchar2
    #param p_middle_name_phonetic        in varchar2
    #param p_date_of_birth               in varchar2
    #param p_place_of_birth              in varchar2
    #param p_gender                      in varchar2
    #param p_declared_ethnicity          in varchar2
    #param p_created_by_module           in varchar2
    #return   element
  ********************************************************************************************/
  function create_hz_createperson_data(p_person_first_name          in varchar2,
                                       p_person_middle_name         in varchar2,
                                       p_person_last_name           in varchar2,
                                       p_person_initials            in varchar2,
                                       p_person_name_phonetic       in varchar2,
                                       p_person_first_name_phonetic in varchar2,
                                       p_person_last_name_phonetic  in varchar2,
                                       p_middle_name_phonetic       in varchar2,
                                       p_date_of_birth              in varchar2,
                                       p_place_of_birth             in varchar2,
                                       p_gender                     in varchar2,
                                       p_declared_ethnicity         in varchar2,
                                       p_created_by_module          in varchar2)
    return clob is
    l_input_pars    varchar2(32767);
    l_person_data   varchar2(32767);
    l_person_rec    varchar2(32767);
    l_init_msg_list varchar2(200);
    l_clob          clob;
  
  begin
    l_init_msg_list := create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'P_INIT_MSG_LIST',
                                      p_value     => 'T');
    l_person_rec    := create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_FIRST_NAME',
                                      p_value     => p_person_first_name) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_MIDDLE_NAME',
                                      p_value     => p_person_middle_name) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_LAST_NAME',
                                      p_value     => p_person_last_name) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_INITIALS',
                                      p_value     => p_person_initials) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_NAME_PHONETIC',
                                      p_value     => p_person_name_phonetic) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_FIRST_NAME_PHONETIC',
                                      p_value     => p_person_first_name_phonetic) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PERSON_LAST_NAME_PHONETIC',
                                      p_value     => p_person_last_name_phonetic) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'MIDDLE_NAME_PHONETIC',
                                      p_value     => p_middle_name_phonetic) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'DATE_OF_BIRTH',
                                      p_value     => p_date_of_birth) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'PLACE_OF_BIRTH',
                                      p_value     => p_place_of_birth) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'GENDER',
                                      p_value     => p_gender) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'DECLARED_ETHNICITY',
                                      p_value     => p_declared_ethnicity) ||
                       create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                      p_tag_name  => 'CREATED_BY_MODULE',
                                      p_value     => p_created_by_module);
  
    l_person_data := create_element(p_ns_abbrev => c_hz_crepers_ns_abbrev,
                                    p_tag_name  => 'P_PERSON_REC',
                                    p_value     => l_person_rec);
    l_input_pars  := create_hz_crepers_inp_elt(l_init_msg_list ||
                                               l_person_data);
    l_clob        := varchar_to_clob(l_input_pars);
    return l_clob;
  end;
  /*******************************************************************************************
    Raise Create Person Event
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Raise Event to Create a Person via SIF
  
    #param p_event_key                   event key
    #param p_person_first_name           in varchar2
    #param p_person_middle_name          in varchar2
    #param p_person_last_name            in varchar2
    #param p_person_initials             in varchar2
    #param p_person_name_phonetic        in varchar2
    #param p_person_first_name_phonetic  in varchar2
    #param p_person_last_name_phonetic   in varchar2
    #param p_middle_name_phonetic        in varchar2
    #param p_date_of_birth               in varchar2
    #param p_place_of_birth              in varchar2
    #param p_gender                      in varchar2
    #param p_declared_ethnicity          in varchar2
    #param p_created_by_module           in varchar2
  ********************************************************************************************/
  procedure raise_create_person_event(p_event_key                  in varchar2,
                                      p_person_first_name          in varchar2,
                                      p_person_middle_name         in varchar2,
                                      p_person_last_name           in varchar2,
                                      p_person_initials            in varchar2,
                                      p_person_name_phonetic       in varchar2,
                                      p_person_first_name_phonetic in varchar2,
                                      p_person_last_name_phonetic  in varchar2,
                                      p_middle_name_phonetic       in varchar2,
                                      p_date_of_birth              in varchar2,
                                      p_place_of_birth             in varchar2,
                                      p_gender                     in varchar2,
                                      p_declared_ethnicity         in varchar2,
                                      p_created_by_module          in varchar2) is
    l_data             clob;
    l_parameters       wf_parameter_list_t := wf_parameter_list_t();
    l_header_parameter wf_parameter_t;
  begin
    l_header_parameter := wf_parameter_t(name  => c_par_inp_hdr,
                                         value => create_hz_soaheader);
    l_data             := create_hz_createperson_data(p_person_first_name          => p_person_first_name,
                                                      p_person_middle_name         => p_person_middle_name,
                                                      p_person_last_name           => p_person_last_name,
                                                      p_person_initials            => p_person_initials,
                                                      p_person_name_phonetic       => p_person_name_phonetic,
                                                      p_person_first_name_phonetic => p_person_first_name_phonetic,
                                                      p_person_last_name_phonetic  => p_person_last_name_phonetic,
                                                      p_middle_name_phonetic       => p_middle_name_phonetic,
                                                      p_date_of_birth              => p_date_of_birth,
                                                      p_place_of_birth             => p_place_of_birth,
                                                      p_gender                     => p_gender,
                                                      p_declared_ethnicity         => p_declared_ethnicity,
                                                      p_created_by_module          => p_created_by_module);
    l_parameters.extend;
    l_parameters(l_parameters.count) := l_header_parameter;
    wf_event.raise(p_event_name => c_evt_create_person,
                   p_event_key  => p_event_key,
                   p_event_data => l_data,
                   p_parameters => l_parameters);
  end;
  /*******************************************************************************************
    Default Rule
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Default Rule function as an example implementation for a pl/sql subscription
  
    #param    p_subscription_guid    Guid of the subscription of which this rule function is 
                                     the implementation
    #param    p_event                event object
    #return   status code
  ********************************************************************************************/
  function default_rule(p_subscription_guid in raw,
                        p_event             in out wf_event_t)
    return varchar2 is
    c_module_name constant varchar2(30) := 'default_rule';
   -- l_event_name varchar2(240);
   -- l_event_key  varchar2(240);
  begin
    /* l_event_name := p_event.getEventName;
    l_event_key  := p_event.getEventKey;
     xxx_logging.log(p_message =>  c_module_name
                              || '.'|| c_package_name
                              || ': '|| l_event_name
                              || ', '|| l_event_key
                   , p_log_level => xxx_logging.c_debug_level);
    if  p_event.parameter_list.count > 0
    then
      for l_idx in p_event.parameter_list.first..p_event.parameter_list.last
      loop
        xxx_tca_api.log( p_module_name  => c_module_name
                       , p_package_name => c_package_name
                       , p_text         => l_event_name
                       , p_parameter1   => p_event.parameter_list(l_idx).getname
                                        ||': '||p_event.parameter_list(l_idx).getvalue);
      end loop;
    end if;*/
    /*  <optional code for WARNING>
    wf_core.context( pkg_name  => c_package_name
                   , proc_name => c_module_name
                   , arg1      => p_event.geteventname( )
                   , arg2      =>  p_subscription_guid);
    wf_event.seterrorinfo( p_event => p_event
                         , p_type  => c_rst_warning);
    return c_rst_warning;*/
    return c_rst_success;
  exception
    when others then
      wf_core.context(pkg_name  => c_package_name,
                      proc_name => c_module_name,
                      arg1      => p_event.geteventname,
                      arg2      => p_subscription_guid,
                      arg3      => p_event.getEventKey);
      wf_event.seterrorinfo(p_event => p_event, p_type => c_rst_error);
      return c_rst_error;
  end default_rule;
  /*******************************************************************************************
    Default Rule
  
    #Version  0.1
    #Author   Martien van den Akker
    #usage    Rule function to save events as a response in Service Invocation Framework
  
    #param    p_subscription_guid    Guid of the subscription of which this rule function is 
                                     the implementation
    #param    p_event                event object
    #return   status code
  ********************************************************************************************/
  function save_response_events(p_subscription_guid in raw, p_event in out wf_event_t)
    return varchar2 is
    c_module_name constant varchar2(30) := 'default_rule';
    l_event_name varchar2(240);
    l_event_key  varchar2(240);
  begin
    l_event_name := p_event.getEventName;
    l_event_key  := p_event.getEventKey;
    insert into xxx_soa_response_events
      (event_key, event_name, event)
    values
      (l_event_key, l_event_name, p_event);
    return c_rst_success;
  exception
    when others then
      wf_core.context(pkg_name  => c_package_name,
                      proc_name => c_module_name,
                      arg1      => p_event.geteventname,
                      arg2      => p_subscription_guid,
                      arg3      => p_event.getEventKey);
      wf_event.seterrorinfo(p_event => p_event, p_type => c_rst_error);
      return c_rst_error;
  end save_response_events;
begin
  -- Initialization
  Null;
end xxx_soa_events;

The main method here is raise_create_person_event. It get's the parameters that are needed for the call to the Create Person API. It uses some varchar2-based functions to build up an xml request message and the Apps Soa-Header.
With this method I managed to raise an event that called the service.

The methods default_rule and save_response_events show examples of a pl/sql rulefunction that you can use to subscribe on a business event.

Script for testing TCA Api's

2011-04-12T05:48:00.000-07:00

If you try to test a service generated and deployed using the Service Provider in SOA Gateway, you might run into problems while the response message does not provide the appropriate messages on the functional errors.
You passed the security authentication and the apps-authorisation and the in the SOA Monitor the webservice shows a succesful execution. But there might be fnctional errors like a lookup violation that are not shown in the response message.

Then it might be usefull to just call the corresponding pl/sql api from a test script.

Below you'll find a script to test the Create Person of the Public Party API. It's a Pl/Sql Developer test script that you might adapt to an sql plus or sqldeveloper script.

declare
  -- FND UserName
  c_user_name          constant varchar2(30) := 'ASADMIN';
  c_responsibility_key constant varchar2(30) := 'HZ_TCA_MANAGER';
  -- Type for repsonsibility record
  type t_responsibility_rec is record(
    user_id            fnd_user.user_id%type,
    user_name          fnd_user.user_name%type,
    responsibility_key fnd_responsibility.responsibility_key%type,
    responsibility_id  fnd_responsibility.responsibility_id%type,
    appplication_id    fnd_responsibility.application_id%type);
  g_responsibility xxx_profile.t_responsibility_rec;
  -- Person Record
  p_person_rec hz_party_v2pub.person_rec_type;
  -- Error Message Fields
  l_error_text varchar2(32767);
  l_msg_count  number;
  cursor c_usr(b_user_name in fnd_user.user_name%type) is
    select usr.user_id, usr.user_name
      from fnd_user usr
     where usr.user_name = b_user_name;
  r_usr c_usr%rowtype;
  cursor c_rsp(b_responsibility_key in fnd_responsibility.responsibility_key%type) is
    select rsp.application_id,
           rsp.responsibility_id,
           rsp.responsibility_key
      from fnd_responsibility rsp
     where rsp.responsibility_key = b_responsibility_key;
  r_rsp            c_rsp%rowtype;
begin
  -- Query Responsibility Id's
  open c_usr(b_user_name => c_user_name);
  fetch c_usr
    into r_usr;
  if c_usr%found then
    g_responsibility.user_id   := r_usr.user_id;
    g_responsibility.user_name := r_usr.user_name;
    open c_rsp(b_responsibility_key => c_responsibility_key);
    fetch c_rsp
      into r_rsp;
    if c_rsp%found then
      g_responsibility.responsibility_key := r_rsp.responsibility_key;
      g_responsibility.responsibility_id  := r_rsp.responsibility_id;
      g_responsibility.appplication_id    := r_rsp.application_id;
    end if;
    close c_rsp;
  end if;
  close c_usr;
  -- Set Apps context
  fnd_global.apps_initialize(user_id      => g_responsibility.user_id,
                             resp_id      => g_responsibility.responsibility_id,
                             resp_appl_id => g_responsibility.appplication_id);
  -- Set Person Record
  p_person_rec.PERSON_FIRST_NAME          := 'Jean';
  p_person_rec.PERSON_MIDDLE_NAME         := 'Michel';
  p_person_rec.PERSON_LAST_NAME           := 'Jarre';
  p_person_rec.PERSON_INITIALS            := 'JM';
  p_person_rec.PERSON_NAME_PHONETIC       := 'sjan misjel sjar';
  p_person_rec.PERSON_FIRST_NAME_PHONETIC := 'sjan';
  p_person_rec.PERSON_LAST_NAME_PHONETIC  := 'sjar';
  p_person_rec.MIDDLE_NAME_PHONETIC       := 'misjel';
  p_person_rec.DATE_OF_BIRTH              := to_date('1948-08-26',
                                                     'yyyy-mm-dd');
  p_person_rec.PLACE_OF_BIRTH             := 'Lyon';
  p_person_rec.GENDER                     := 'MALE';
  p_person_rec.DECLARED_ETHNICITY         := 'French';
  p_person_rec.created_by_module          := 'HZ_WS';
  -- Call the procedure
  hz_party_v2pub.create_person(p_init_msg_list    => :p_init_msg_list,
                               p_person_rec       => p_person_rec,
                               p_party_usage_code => :p_party_usage_code,
                               x_party_id         => :x_party_id,
                               x_party_number     => :x_party_number,
                               x_profile_id       => :x_profile_id,
                               x_return_status    => :x_return_status,
                               x_msg_count        => :x_msg_count,
                               x_msg_data         => :x_msg_data);
  l_msg_count  := fnd_msg_pub.Count_Msg;
  l_error_text := '';
  if l_msg_count = 1 then
    l_error_text := 'API Error: ';
  end if;
  if l_msg_count >= 1 then
    for i in 1 .. l_msg_count loop
      l_error_text := nvl(l_error_text, ' ') || chr(10) || i || '. ' ||
                      fnd_msg_pub.get(p_encoded => fnd_api.g_false);
    end loop;
  
  end if;
  :error := l_error_text;
end;

SOA Gateway wsse security username token

2011-04-12T04:56:00.000-07:00

When you want to use a service generated with the EBS 12.1 SOA Gateway you'll need a wsse:security block in the soap header to authenticated against the Username token. That is when you choose to use Username-token as authentication method.
The block that you need to add to the soap header looks like:

<wsse:Security>
  <wsse:UsernameToken> 
    <wsse:Username>ASADMIN</wsse:Username>
    <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">AsAdminPassword</wsse:Password>
  </wsse:UsernameToken> 
</wsse:Security>

The namespace that the abbreviation wsse: points to is:

xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"

XML to HTML through XSL

2011-03-29T05:32:00.000-07:00

I think it was about 2005 or 2006 that we at Oracle had so-called Mudwrestle sessions. Colleagues organized workshops for each other on the latest technologies. We started at 14:00 and after some introduction we did some exercises to get familiar with the subject. Amongst subjects as Integration (InterConnect), java, linux, we had also XML. And there I got to know XSLT for the first time.

To get it "in the fingers", I thought it might be handy to have all my browser links in an xml file and have an xsl attached to have it transformed into html providing them in pop-lists and a button. With a little java script it is then possible to have the link loaded after choosing a link and pressing the button. So I created my first version of the XML and XSL. It was in the time that Internet Explorer 6 was pretty new or at least the current release. I think I was into Firefox already. IE6 and Firefox were able to load and process the xsl file when it was attached to the xml file. To do so you have to add the following tag to the top of your xml file:

<?xml-stylesheet href="links.xsl" type="text/xsl"?>

Like;

<?xml version="1.0"?>
<?xml-stylesheet href="links.xsl" type="text/xsl"?>
<link-lists
  name="Links"
  title="Internet Links"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Begin van de Lists!
  <lists>
    <list
      name="DarwinApps"
      title="Darwin Links">
      <link
        description="Darwin-IT"
        link="http://www.darwin-it.nl"
        name="Darwin-IT" />
...

The XSL that I created for it is given at the bottom of this post.
Last week I enhanced it a little to have a button to load all the links of a list at once. That is particularly handy for a list of webmail-urls to load at the start of your day.

I'm very fond of this xml/xsl combi. It is so simple, since it consist of three ascii files (xml, xsl and css). The xml file is easy to extend and after a reload in the browser the link-lists are renewed. The xsl is simple to enhance to add extra functionality. And the css makes it possible to have it a layout that complies with your taste or desktop. Skinning was never so easy. And it works in both Windows and Linux. And probably Mac OS. On Windows XP I even had it on my Active Desktop. But I can't find how to do it on Windows 7 or Linux.

Later in history, also a few years ago (2009) I created a tool in java to edit the xml file. Also this one is enhanced a few times and last week I posted a blog on this tool. See here. I made it xml-parser-vendor-independent. And it's opensource thus you might take a look at the source.

To try the xml/xsl you can open the xml-links file from here. Of course you can download it to your laptop and edit it to add your own links and remove the ones not needed. The xsl is for download here and the css-file here. Place the xml and the xsl in the same folder, the css is expected in a subfolder called "style".

Below is the source of the XSL for investigation. As you can see I set it up in a modular way. I'm used to have a template for every hierarchical level of XML. And for particular functionalities, such as the generation of the java script for each poplist. You can use the xml-tool set in the the earlier post to perform a transform of the XML with the XSL to see the resulting HTML. Have fun with it.

<?xml version="1.0"?>
<!--  (c) 2008-2011, by Martien van den Akker  
Darwin-IT Professionals
-->

<xsl:stylesheet
  version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output
    method="html" />  <!--  Variables  -->

  <xsl:variable
    name="newline">
    <xsl:text></xsl:text></xsl:variable>  <!--  main  -->

  <xsl:template
    match="/">
    <html>

      <xsl:call-template
        name="heading" />
      <body>
        <xsl:call-template
          name="body" /></body></html></xsl:template>
  <xsl:template
    name="heading">    <!--  Heading  -->

    <head>
      <title>Links</title>
      <LINK
        HREF="style/style.css"
        REL="STYLESHEET"
        TYPE="text/css" /></head>    <!--  Script to load a page  -->

    <script
      language="javascript">
      <xsl:text>function loadPage(link)
{
  newWdw=window.open(link,"_blank","");
}</xsl:text></script></xsl:template>
  <xsl:template
    name="body">    <!--  Body  -->

    <body>
      <xsl:apply-templates
        select="/link-lists" /></body></xsl:template>  <!--  /link-lists  -->

  <xsl:template
    match="link-lists"
    name="link-lists">

    <h1>
      <xsl:value-of
        select="@title" /></h1>
    <table>
      <xsl:apply-templates
        select="lists/list" /></table></xsl:template>  <!--  /link-lists/lists/list  -->

  <xsl:template
    match="lists/list"
    name="lists-list">
    <tr>
      <td>
        <xsl:value-of
          select="@title" /></td>      <!--  generate javascript for loading a link  -->

      <xsl:call-template
        name="loadLink" />      <!--  generate javascript for loading a links  -->

      <xsl:call-template
        name="loadAllLinks" />      <!--  generate a poplist and a button  -->

      <xsl:call-template
        name="writePopList" /></tr></xsl:template>  <!--  loadLink: generate a javascript that loads a link  -->

  <xsl:template
    name="loadLink">
    <xsl:text
      disable-output-escaping="yes">      <!--  script to load a link  -->

</xsl:text>
    <script
      language="javascript">
      <xsl:value-of
        select="concat($newline,'      function load', @name,'Link()')" />
      <xsl:text
        disable-output-escaping="yes">{</xsl:text>
      <xsl:value-of
        select="concat('        var l_idx = document.', @name, 'LinkSelector.select.selectedIndex;',$newline)" />
      <xsl:value-of
        select="concat('        var l_value = document.', @name, 'LinkSelector.select.options[l_idx].value;')" />
      <xsl:text
        disable-output-escaping="yes">if (l_value != "none")
        {
           loadPage(l_value);
        }
      }</xsl:text></script></xsl:template>  <!--  loadAllLinks: generate a javascript that loads all links of a poplist  -->

  <xsl:template
    name="loadAllLinks">
    <xsl:text
      disable-output-escaping="yes">      <!--  script to load a link  -->
</xsl:text> 
    <script
      language="javascript">
      <xsl:value-of
        select="concat($newline,'      function loadAll', @name,'Links()')" />
      <xsl:text
        disable-output-escaping="yes">{
        var l_idx;
        var l_value;</xsl:text>
      <xsl:value-of select="concat('        for(l_idx=0; l_idx&lt;document.', @name, 'LinkSelector.select.length; l_idx++)')"/>
   <xsl:value-of select="concat($newline,'        {',$newline)"/>
      <xsl:value-of select="concat('           l_value=document.', @name, 'LinkSelector.select.options[l_idx].value;')"/> 
      <xsl:text
        disable-output-escaping="yes">

    if (l_value != "none")
    {
            loadPage(l_value);
    }
        }
      }</xsl:text></script></xsl:template>
  <xsl:template
    name="writePopList">
    <td>
      <form
        name="{concat(@name, 'LinkSelector')}">
        <table
          border="0"
          cellspacing="0">
          <tr>
            <td
              width="250px">              <!-- Select  -->

              <select
                name="select">
                <option
                  value="none">Kies een link:</option>
                <xsl:for-each
                  select="link">
                  <option
                    value="{@link}">
                    <xsl:value-of
                      select="@name" /></option></xsl:for-each></select></td>
            <td>              <!--  Button  -->

              <input
                onclick="{concat('load',@name,'Link()')}"
                type="button"
                value="Toon" /></td>

            <td>              <!--  Button Load All -->

              <input
                onclick="{concat('loadAll',@name,'Links()')}"
                type="button"
                value="Toon alles" /></td></tr></table></form></td></xsl:template>
</xsl:stylesheet>

Vendor independent XML processing

2011-03-22T03:44:00.000-07:00

A few years ago, when I was "low in work" at the particular customer, I created a little tool set on XML processing. I called it Darwin XML Editor and XML Tester. The latter name I now find not so well choosen. So I would now call it XML Tools.

The purpose of the tool set was to figure out how to process XML in java. I already had a private xml based solution to gather my frequently used browser links into an xml file and convert it using an attached xslt stylesheet to an html page. This html page provided the links in poplists and using java script it could load the link.
It would be nice to have a hierarchical editor that showed my xml in an explorer and let me edit particular attributes in a table format.

It turned out handy for me. But the most I had a toolkit that allowed me to easily load XML as a text file and parse it, perform xpath queries and just traverse through nodes, etc. In several projects I used it to read xml-based config files. Many of those cases I probably could have done using Apache-commons.
But it is handy to have your own toolkit. Especially in cases when you have repetitive lists of entities with properties.

My project originally was written using the Oracle XMLParser. Handy, because I use JDeveloper and that comes with the parser. For some time now I had plans to look if I could make it vendor independent.

The results can be found here. It's open source, whatever that may mean. But it would be nice to leave the credit-references in place or refer to the credits. That would be nice for my ego...

I will provide here some highlights of the changes.
I used the following sites as my references:

http://www.ibm.com/developerworks/library/x-javaxpathapi.html for xpath processing. .
http://www.ling.helsinki.fi/kit/2004k/ctl257/JavaXSLT/Ch05.html for xslt processing

Parsing

It starts with parsing. The "Oracle way" I did it was:

public void parse() {

        try {
            String text = super.getText();
            if (text != null) {
                resetError();
                DOMParser parser = new DOMParser();
                InputSource inputStream = new InputSource();
                inputStream.setCharacterStream(new StringReader(text));
                parser.parse(inputStream);
                xmlDoc = parser.getDocument();

                xmlRoot = xmlDoc.getDocumentElement();
                nodeSelected = new NodeSelected((Node)xmlRoot);
                parsed = true;
                log("File: " + super.getFilePath() + " is succesfully parsed");
            } else {
                log("File: " + super.getFilePath() + " empty or not loaded!");
            }

        } catch (SAXException e) {
            setErrorCode(EC_ERROR);
            setError("Error parsing XML: " + e.toString());
            error(e);
        } catch (IOException e) {
            setErrorCode(EC_ERROR);
            setError("Error reading XML: " + e.toString());
            error(e);
        }
    }

With the following oracle imports:

import oracle.xml.parser.v2.DOMParser;
import oracle.xml.parser.v2.XMLDocument;
import oracle.xml.parser.v2.XMLNode;
import oracle.xml.parser.v2.XSLException;

The vendor independent way I used is:

/**
     * Create a new DocumentBuilder.
     * @return
     * @throws ParserConfigurationException
     */
    private DocumentBuilder newDocBuilder() throws ParserConfigurationException {
        DocumentBuilderFactory domFactory = 
            DocumentBuilderFactory.newInstance();
        domFactory.setNamespaceAware(true); // never forget this!
        DocumentBuilder docBuilder;
        docBuilder = domFactory.newDocumentBuilder();
        return docBuilder;
    }

    /**
     * Create an InputSource from the xml-text
     * @return
     */
    public InputSource getInputSource() {
        String text = super.getText();
        InputSource inputStream = null;
        if (text != null) {
            inputStream = new InputSource();
            inputStream.setCharacterStream(new StringReader(text));
        }
        return inputStream;
    }

    /**
     * Parse the XML in the file
     */
    public void parse() {
        try {
            InputSource inputSource = getInputSource();
            if (inputSource != null) {
                resetError();
                DocumentBuilder docBuilder = newDocBuilder();
                Document doc = docBuilder.parse(inputSource);
                setDoc(doc);
                parsed = true;
                log("File: " + super.getFilePath() + " is succesfully parsed");
            } else {
                log("File: " + super.getFilePath() + " empty or not loaded!");
            }
        } catch (ParserConfigurationException e) {
            setErrorCode(EC_ERROR);
            setError("Error creating parser: " + e.toString());
            error(e);
        } catch (SAXException e) {
            setErrorCode(EC_ERROR);
            setError("Error parsing XML: " + e.toString());
            error(e);
        } catch (IOException e) {
            setErrorCode(EC_ERROR);
            setError("Error reading XML: " + e.toString());
            error(e);
        }
    }

with the following imports:

import javax.xml.namespace.QName;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

Mark that this code is part of a XMLFile class that extends a TextFile class that gives me the methods to load the file into a String attribute (text).
Here I extracted the conversion from the text attribute to an InputSource object in a separate method. Also the creation of the parser (the DocumentBuilder) I put in a seperate method. This because I also need it to be able to create an empty Document.

XPath Expressions

The "Oracle way" is pretty straight forward:

/**
     * Select Nodes using Xpath
     * @param xpath
     * @return NodeList 
     */
    public NodeList selectNodes(String xpath) throws XSLException {
        XMLDocument xmlDoc = getXmlDoc();
        NodeList nl = xmlDoc.selectNodes(xpath);
        return nl;
    }

To have it "Namespace Aware" you'll need a Namespace Resolver, which is a simple class based on a HashMap, implementing an Interface:

package com.darwinit.xmlfiles.xml;
/**
 * Namespace Resolver: Helper class to do namespace aware XPath queries. 
 *
 * @author Martien van den Akker
 * @author Darwin IT Professionals
 *
 * @remark: changed to JSE 1.4 code because of BPEL PM 10.1.2
 */
import java.util.HashMap;

import oracle.xml.parser.v2.NSResolver;

public class XMLNSResolver implements NSResolver{
  private HashMap<String, String> nsMap = new HashMap<String, String>();
    
    public XMLNSResolver() {
    }

   public void addNS(String abbrev, String namespace){
       nsMap.put(abbrev, namespace);
   }
    public String resolveNamespacePrefix(String string) {
        return nsMap.get(string);
    }
}

Then using the namespace resolver the code would be something like:

/**
     * Select nodes using Xpath with Namespace included
     * @param xpath
     * @return Nodelist 
     */
    public NodeList selectNodesNS(String xpath) throws XSLException {
    XMLDocument xmlDoc = getXmlDoc();
    XMLNSResolver nsRes = getNsRes();
        NodeList nl = this.xmlDoc.selectNodes(xpath, nsRes);
        return nl;
    }

The vendor indepent way is a little more complex. But it gives you some more flexibility.

/**
     * Evaluate xpath expression 
     * 
     * @param xpathExpr the xpath expression
     * @param returnType the return type that is expected.
     * http://www.ibm.com/developerworks/library/x-javaxpathapi.html:
     * XPathConstants.NODESET => node-set maps to an org.w3c.dom.NodeList
     * XPathConstants.BOOLEAN => boolean maps to a java.lang.Boolean
     * XPathConstants.NUMBER => number maps to a java.lang.Double
     * XPathConstants.STRING => string maps to a java.lang.String
     * XPathConstants.NODE
     * 
     * @throws XPathExpressionException
     */
    public   Object evaluate(String xpathExpr, 
                    QName returnType) throws XPathExpressionException {
        XPathFactory factory = XPathFactory.newInstance();
        XPath xpath = factory.newXPath();
        XMLNSResolver nsRes = getNsRes();
        if (nsRes != null) {
            xpath.setNamespaceContext(nsRes);
        }
        XPathExpression expr = xpath.compile(xpathExpr);
        Document doc = getDoc();
        Object resultObj = expr.evaluate(doc, returnType);
        return resultObj;
    }

    /**
     * Evaluate xpath expression to a double (when a number is expected from the xpath expression)
     * 
     * @throws XPathExpressionException
     */
    public Double evaluateDouble(String xpathExpr) throws XPathExpressionException {
        Double result = null;
        Object resultObj = evaluate(xpathExpr, XPathConstants.NUMBER);
        if (resultObj instanceof Double) {
            result = (Double)resultObj;
        }
        return result;
    }

    /**
     * Select Nodes using Xpath
     * 
     * @param xpath
     * @return NodeList
     * @throws XPathExpressionException 
     */
    public NodeList selectNodes(String xpath) throws XPathExpressionException {
        NodeList nl = (NodeList)evaluate(xpath, XPathConstants.NODESET);
        return nl;
    }

The Namespace resolver changed slightly. It actually implements another interface:
But the idea is the same:

package com.darwinit.xmlfiles.xml;

import java.util.HashMap;

import java.util.Iterator;

import javax.xml.namespace.NamespaceContext;


/**
 * Namespace Resolver: Helper class to do namespace aware XPath queries. 
 *
 * @author Martien van den Akker
 * @author Darwin IT Professionals
 *
 */
public class XMLNSResolver implements NamespaceContext {
    private HashMap<String, String> nsMap = new HashMap<String, String>();
   /**
     * Constructor
     */
    public XMLNSResolver() {
    }

    /**
     * Add a Namespace
     * @param prefix
     * @param namespace
     */
    public void addNS(String prefix, String namespace) {
        nsMap.put(prefix, namespace);
    }

    /**
     * Resolve a namespace from a prefix
     * @param prefix
     * @return
     */
    public String resolveNamespacePrefix(String prefix) {
        return nsMap.get(prefix);
    }

    /**
     * Resolve a namespace from a prefix
     * @param prefix
     * @return
     */
    public String getNamespaceURI(String prefix) {
        return resolveNamespacePrefix(prefix);
    }

    /**
     * Get the prefix that is registered for a NamespaceURI 
     * However, not necessary for xpath processing
     * @param namespaceURI
     * @return
     */
    public String getPrefix(String namespaceURI) {
        throw new UnsupportedOperationException();
    }

    /**
     * Get an iterator with the prefix registered for a NamespaceURI
     * @param namespaceURI
     * @return
     */
    public Iterator getPrefixes(String namespaceURI) {
        return null;
    }
}

The differences are in the fact that you first have to 'compile' an xpath expression. And that the evaluation of the XPath on a Document expects you to provide the expected result datatype. (see the comments of the evaluate method).
Then it will result an Object that you have to cast to the particular Java Class that corresponds with the Result XML DataType.

XSLT

The last thing is transforming XSLT. The "Oracle Way" I used was:

package com.darwinit.xmlfiles.xml;

import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.HashMap;
import java.util.Hashtable;

import oracle.xml.parser.v2.XMLDocument;
import oracle.xml.parser.v2.XSLException;
import oracle.xml.parser.v2.XSLProcessor;
import oracle.xml.parser.v2.XSLStylesheet;


public class XmlTransformer {
    private void pl(String text) {
        System.out.println(text);
    }

    public XmlTransformer() {
    }

    public String transform(XMLDocument xslDoc, XMLDocument xmlDoc) {     
     return transform (xslDoc, xmlDoc, null);
    }    
    
    /**
     * Transform the xmlDoc using xslDoc into String
     * @param xslDoc
     * @param xmlDoc
     * @param parameters contains parameter that are passed to the xslt
     * @return String output
     */
    public String transform(XMLDocument xslDoc, XMLDocument xmlDoc
                           , Hashtable<String, String> parameters) {
        XSLProcessor xslProcessor = new XSLProcessor();
        XSLStylesheet xslt;
        String result = "";
        try {
            StringWriter sw = new StringWriter();
            PrintWriter pw = new PrintWriter(sw);
            xslt = xslProcessor.newXSLStylesheet(xslDoc);
            
            xslProcessor.setXSLTVersion(XSLProcessor.XSLT20);
            xslProcessor.showWarnings(true);
            xslProcessor.setErrorStream(System.err);
            
            if (parameters != null) {
            
             for (String key : parameters.keySet()) {
              String value = parameters.get(key);
              xslProcessor.setParam("", key, value);
             }
            }                      
            
            xslProcessor.processXSL(xslt, xmlDoc, pw);
            pw.flush();
            pw.close();
            sw.close();
            result = sw.toString();
        } catch (XSLException e) {

            pl(e.toString());
        } catch (IOException e) {
            pl(e.toString());
        }
        return result;
    }   
    /**
     * Clean up text from XML Leftovers
     * @param text
     * @return
     */
    public String cleanText(final String text){
        String result = text;
        result = result.replaceAll("&lt;","<");
        return result;
    }
    /**
     * Transfomr the xmlDoc using xslDoc into String
     * Cleanup XML code leftovers
     * @param xslDoc
     * @param xmlDoc
     * @return String output
     */
    public String transform2Ascii(XMLDocument xslDoc, XMLDocument xmlDoc) {
      String result = transform(xslDoc, xmlDoc);
      result = cleanText(result);
      return result;
    }

}

And the vendor indepent version of the same class:

package com.darwinit.xmlfiles.xml;

import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Hashtable;

import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

import org.w3c.dom.Document;

import com.darwinit.xmlfiles.log.LocalStaticLogger;

/**
 * XmlTransformer: Class implementing functionality to Transform XML using XSLT.
 * 
 * See also http://www.ling.helsinki.fi/kit/2004k/ctl257/JavaXSLT/Ch05.html
 * 
 * @author Martien van den Akker
 * @author Darwin IT Professionals
 */

public abstract class XmlTransformer {
 public static final String className = "XmlTransformer";
 private static LocalStaticLogger lgr;

 /**
  * Transform the xmlDoc using xslDoc into String
  * 
  * @param xslDoc
  * @param xmlDoc
  * @return
  */
 public static String transform(Document xslDoc, Document xmlDoc) {
  return transform(xslDoc, xmlDoc, null);
 }

 /**
  * Transform the xmlDoc using xslDoc into String, with parameters
  * 
  * @param xslDoc
  * @param xmlDoc
  * @param parameters
  *            contains parameter that are passed to the xslt
  * @return String output
  */
 @SuppressWarnings("static-access")
 public static String transform(Document xslDoc, Document xmlDoc,
   Hashtable<String, String> parameters) {
  final String methodName = "transform";
  lgr.logStart(className, methodName);
  String result = "";
  try {
   DOMSource xsltSource = new DOMSource(xslDoc);
   DOMSource xmlSource = new DOMSource(xmlDoc);
   StringWriter sw = new StringWriter();
   PrintWriter pw = new PrintWriter(sw);
   StreamResult streamResult = new StreamResult(pw);
   // Get the transformer factory
   TransformerFactory transFact = TransformerFactory.newInstance();
   // Get a transformer for this particular stylesheet
   Transformer trans = transFact.newTransformer(xsltSource);
   // Add parameters
   if (parameters != null) {
    for (String key : parameters.keySet()) {
     String value = parameters.get(key);
     trans.setParameter(key, value);
    }
   }
   // Do the transformation
   trans.transform(xmlSource, streamResult);
   pw.flush();
   pw.close();
   sw.close();
   // Get the result string
   result = sw.toString();
  } catch (IOException e) {
   lgr.error(className, methodName, e);
  } catch (TransformerException e) {
   lgr.error(className, methodName, e);
  }
  lgr.logEnd(className, methodName);
  return result;
 }

 /**
  * Clean up text from XML Leftovers
  * 
  * @param text
  * @return
  */
 public static String cleanText(final String text) {
  String result = text;
  result = result.replaceAll("&lt;", "<");
  return result;
 }

 /**
  * Transform the xmlDoc using xslDoc into String Cleanup XML code leftovers
  * 
  * @param xslDoc
  * @param xmlDoc
  * @return String output
  */
 public static String transform2Ascii(Document xslDoc, Document xmlDoc) {
  String result = transform(xslDoc, xmlDoc);
  result = cleanText(result);
  return result;
 }

}

The code is not much different. Most important is that you have to wrap the input, xslt and result objects into particular interface objects. The examples I found worked with File(s) as Source and Result objects. But I wanted Document objects for the XLST and Input objects. And I want to have the result into a String, to be able to process it in anyway I want.

I added also the possiblity to pass XSLT parameters. But I got it from a project I worked on and see that I haven't put that in the class in my XmlFiles project.

Conclusion

I hope this helps in understanding how to work with XML parsers. Ofcourse all is rudimentary (I have several large books on the subject in the cupboard). But I have pretty much enough with the above. The code in my XmlFiles project is basically around this code. On top of these methods I have several other helper methods to do traverse nodes or to get it in a particular way.

I thought it might be helpfull to put the Oracle Native methods side by side with the vendor-independent (JAXP) code. Doing so I would not state that one way is better than the other. The vendor-independent code have clearly the advantage that you can simply replace the parser, just by changing the class path. The code will work with the Oracle XML Parser just as good as with the Apache Xerces-J parser.
The reason I did it the Oracle way before was just because I ran into these examples first, the time I started this. Probably there still are good reasons to use the native API's.

Oracle also has a Pl/Sql Wrapper around the XML Parser. So the Oracle XML Parsing code has a Pl/Sql counter part. Might be nice to do this in a Pl/Sql way sometime.
But as the code above isn't rocket science, it isn't new also. XML Parsing in Pl/Sql can be done from Oracle 8i onwards. Also roughly ten years already.

Query ClearQuest with Java.

2011-03-17T02:49:00.000-07:00

At deployment of a new release of a project it might be required to deliver a list of solved issues in a release notes document. At least that was a requirement that my colleagues on my current project had.
They copied and pasted the issues from the Rational ClearQuest tool, which they felt was error-prone and a tedious job. If I could create a tool for it?

The first thing I did was to look into ClearQuest on how the import and export possibilities were. But although there is an export tool, it is not batch. It'll start a wizard that you have to go through. Within ClearQuest there are query possibilities. But then again: you're in the tool and we wanted to have it batch-wise, integrated with Ant.
So it would be nice to be able to query ClearQuest in Java. And fortunately I found a cqjni.jar in the ClearQuest home. This jar is a Java Native Interface set of API's on CQ.

There are, however, very few java examples. I did find an example on perl here. I could use this with some other examples to come up with my own code. So I'll lay out my steps here.

0. Some helper methods

First I'll give you some helper methods that I use in the following examples. They're in fact shortcuts to "System.out.println".

/**
  * Shortcut for output
  * 
  * @param text
  */
 public static void pl(String text) {
  System.out.println(text);
 }

 public static void pl(String text, Exception e) {
  System.out.println(text);
  e.printStackTrace();
 }

 /**
  * Output a String array.
  * 
  * @param label
  * @param array
  */
 public static void printStringArray(String label, String[] array) {
  for (int idx = 0; idx < array.length; idx++) {
   pl(label + " " + idx + ": " + array[idx]);
  }
 }

1. Create session

The first step to do is to create a CQSession:

pl("Begin");
CQSession cqSession = new CQSession();

2. List Installed Database Sets and Master Databases

To be able to query on CQ you'll have to find out what databases you can connect to. In CQ apparently databases are grouped in a DatabaseSet. And a DatabaseSet is correlated to a MasterDatabase. You can query them as follows.

CQSession session = getCqSession();
  String[] dbSets = session.GetInstalledDbSets();
  String[] masters = session.GetInstalledMasterDbs();

As I understand it, a DabaseSet is put in a database. In my setup I've two masterdatabases and a DatabaseSet in each. I assumed that the entries in the String Arrays correspond to each other: dbSets[0] correlates to masters[0], etc.

I combined the lot in a some combined methods. These are based on a helper Bean DatabaseSet.

/**
  * Get Installed ClearQuest Database Sets
  * 
  * @return
  * @throws CQException
  */
 public String[] getInstalledDbSetNames() throws CQException {
  CQSession session = getCqSession();
  String[] dbSets;
  dbSets = session.GetInstalledDbSets();
  return dbSets;
 }

 /**
  * Get Installed ClearQuest Master Databases
  * 
  * @return
  * @throws CQException
  */
 public String[] getInstalledMasterDbNames() throws CQException {
  CQSession session = getCqSession();
  String[] masters = session.GetInstalledMasterDbs();
  return masters;
 }

 /**
  * Get Installed ClearQuest Database Sets
  * 
  * @return
  */
 public Vector<DatabaseSet> getInstalledDatabaseSets() {
  final String methodName = "getInstalledDatabaseSets";
  Vector<DatabaseSet> dbSets = new Vector<DatabaseSet>();
  try {
   String[] dbSetNames = this.getInstalledDbSetNames();
   String[] masters = this.getInstalledMasterDbNames();
   for (int i = 0; i < dbSetNames.length; i++) {
    DatabaseSet dbSet = new DatabaseSet(masters[i], dbSetNames[i]);
    dbSets.add(dbSet);
   }
  } catch (CQException e) {
   pl("Exception at " + methodName, e);
  }

  return dbSets;
 }
 /**
  * List Installed ClearQuest Database Sets
  * 
  * @return
  */
 public void listInstalledDbSets() {
  final String methodName = "listInstalledDbSets";
  pl(methodName);
  Vector<DatabaseSet> dbSets;
  dbSets = getInstalledDatabaseSets();
  ListIterator<DatabaseSet> it = dbSets.listIterator();
  while (it.hasNext()) {
   DatabaseSet dbSet = it.next();
   pl(dbSet.toString());
  }
 }

3. Accessible Databases

To list the Accessible Databases I used the code that I found here. In my own code I moved some of the code to the methods above.

/**
  * List Accessible ClearQuest Databases
  * 
  * @return
  */
 public void listAccessibleDatases() {
  final String methodName = "listInstalledDatabaseSets";
  pl(methodName);
  Vector<DatabaseSet> dbSets;
  dbSets = getInstalledDatabaseSets();
  ListIterator<DatabaseSet> it = dbSets.listIterator();
  while (it.hasNext()) {
   try {
    DatabaseSet dbSet = it.next();
    pl("Handling dbSet: " + dbSet.getName());
    CQSession session = getCqSession();
    CQDatabaseDescs dbDescriptors = session.GetAccessibleDatabases(
      dbSet.getMasterDb(), "", dbSet.getName());
    int n = (int) dbDescriptors.Count();
    System.out.println(" Accessible database count: " + n);
    for (int j = 0; j < n; j++) {
     CQDatabaseDesc desc = dbDescriptors.Item(j);

     pl(" Database " + j + ": " + desc.GetDatabaseName());
    }
   } catch (CQException e) {
    pl("Exception at " + methodName, e);
   }
  }
 }

4. Connect to a database

If you know what database you want to connect to, you can logon:

pl("Logon to ClearQuest");
   cqSession.UserLogon("User", "Password", "CQ-database", "CQ-DatabaseSet");
   setCqSession(cqSession)

5. List Record Types

Record types are sort of the logical-entities in ClearQuest. You have to know the entity in which your issues are stored. To list the possible Record Types you'll have to connect to the database (see above). Then you can list the record types as follows:

/**
  * List the possible RecordTypes
  * 
  * @return
  * @throws CQException
  */
 public void listRecordTypes() {
  final String methodName = "listRecordTypes";
  pl(methodName);
  try {
   CQSession session = getCqSession();
   String[] cqRecordTypes = session.GetEntityDefNames();
   printStringArray("Record Type", cqRecordTypes);
  } catch (CQException e) {
   pl("Exception at " + methodName, e);
  }
 }

The same way you can find so-called Family Types. I did not use them further.

/**
  * List the possible Family Types
  */
 public void listFamilyTypes() {
  final String methodName = "listFamilyTypes";
  pl(methodName);
  try {
   CQSession session = getCqSession();
   String[] cqFamilyTypes = session.GetEntityDefFamilyNames();
   printStringArray("Family Type", cqFamilyTypes);
  } catch (CQException e) {
   pl("Exception at " + methodName, e);
  }

 }

6. Build a query

Now it's time to build the query. If you've build a query before in the CQ tool you'll probably be familiar with the concepts. It starts with creating a query and add columns to it:

CQQueryDef cqQueryDef = cqSession.BuildQuery("BPM");
   cqQueryDef.BuildField("id");
   cqQueryDef.BuildField("State");
   cqQueryDef.BuildField("headline");
   cqQueryDef.BuildField("Owner");
   cqQueryDef.BuildField("Fix_By");

The parameter in the BuildQuery method is the RecordType that you can query with the code in previous paragraph. In this example the record type is "BPM". The field-names are those defined in the tool on the record type. So it might be handy to build your query first in the CQ-Tool, to know which attributes/columns are to your disposal.

7. Create a filter

Of course you don't want the whole lot of issues. In a longer running project there can be thousands of them. So we'll need to add a filter. In the ClearQuest Tool you can have a hierarchy of so called filter nodes. A filter node has several expressions that are concatenated using And or Or operators. Such an expression in itself can be a other filter node or what I call a FilterField: a filter on a field.

pl("Query Issues");
   CQQueryFilterNode cqQueryFilterNode = cqQueryDef
     .BuildFilterOperator(BOOL_OP_AND);
   String[] stateValues = { "Closed", "Resolved" };
   cqQueryFilterNode.BuildFilter("State", COMP_OP_EQ, stateValues);
   String[] branchValues = { "R1.%" };
   cqQueryFilterNode.BuildFilter("Fix_by", COMP_OP_LIKE, branchValues);

Here you see that I use some constants. For the FilterOperators the possible values are:

// And operator
 public static final long BOOL_OP_AND = 1;
 // Or operator
 public static final long BOOL_OP_OR = 2;

For the Filter "Comparators" the following values are possible:

// Equality operator (=)
 public static final long COMP_OP_EQ = 1;
 // Inequality operator (<>)
 public static final long COMP_OP_NEQ = 2;
 // Less-than operator (<)
 public static final long COMP_OP_LT = 3;
 // Less-than or Equal operator (<=)
 public static final long COMP_OP_LTE = 4;
 // Greater-than operator (>)
 public static final long COMP_OP_GT = 5;
 // Greater-than or Equal operator (>=)
 public static final long COMP_OP_GTE = 6;
 // Like operator (value is a substring of the string in the given field)
 public static final long COMP_OP_LIKE = 7;
 // Not-like operator (value is not a substring of the string in the given
 // field)
 public static final long COMP_OP_NOT_LIKE = 8;
 // Between operator (value is between the specified delimiter values)
 public static final long COMP_OP_BETWEEN = 9;
 // Not-between operator (value is not between specified delimiter values)
 public static final long COMP_OP_NOT_BETWEEN = 10;
 // Is-NULL operator (field does not contain a value)
 public static final long COMP_OP_IS_NULL = 11;
 // Is-not-NULL operator (field contains a value)
 public static final long COMP_OP_IS_NOT_NULL = 12;
 // In operator (value is in the specified set)
 public static final long COMP_OP_IN = 13;
 // Not-in operator (value is not in the specified set)
 public static final long COMP_OP_NOT_IN = 14;

8. Execute the query and process the rows

Having built the query, it can be executed. First you'll have to create a CQResultSet from the session based on the Query Definition.
The query can then be executed and the result set can be processed using the MoveNext() method. The MoveNext() gives a status. As long as the status equals 1 there is a next record. With GetNumberOfColumns you can count the number of columns in the resultset. And with GetColumnLabel and GetColumnValue you can get the Column name and value at a particular index. Of course you know what columns you asked for and in what order. But traversing over the available columns is handy, because adding a column to the query does not affect this part of the code. Also it enables you to search for the value of a particular column.

CQResultSet cqResultSet = cqSession.BuildResultSet(cqQueryDef);
   long count = cqResultSet.ExecuteAndCountRecords();
   pl("count: " + count);
   long status = cqResultSet.MoveNext();
   while (status == 1) {
    pl("Status: " + status);
    for (int i = 1; i <= cqResultSet.GetNumberOfColumns(); i++) {
     pl(cqResultSet.GetColumnLabel(i) + ": "
       + cqResultSet.GetColumnValue(i));
    }
    status = cqResultSet.MoveNext();
   }
   pl("Status: " + status);
   pl("Done");

Conclusion

That's it. The example-snippets above come from a test class I put together and that can be downloaded form here. It's a test class that I created from other examples and refactored somewhat. For this blog I anonymised it and rearranged some of the methods. In my application I refactored the class into seperate classes for the dbconnection, a query class and an export class. Of course that's what you do in a project, to make it neat. But in the basics it runs down to this particular example. So I kept it in my project to first pilot a new feature.

Then I created some ant tasks and targets to do the export and run Word with a document and a macro to import the file. But that's another story...

Total Commander for Linux=>Krusader

2011-03-10T07:04:00.000-08:00

Oh, how I love Total Commander. I never found a file-manager that is so powerful. And how I missed that under Linux. I work a few years with Linux now, but on my Linux I still have TotalCommander. It can browse through archives, it can FTP and it has tabs to quickly switch locations. And I find copying/moving using to panes much more convenient than the Windows Explorer way. I kind of hate Windows Explorer. KDE-Dolfin is better, but to me still too much Windows Explorer.

Today I looked at the Total Commander website, to see that there is a new version of it. I must admit that I have a new laptop with Windows 7/64bit. And one of the minor things that I miss in TC is that it's still 32 bit. It's not big of a problem, but in the context menu all the 64 bit tools (like svn) are under a sub-menu. That's a little inconvenient.

In the FAQ I saw that a Linux version of TC is postponed indefinitely. But that there is one recommended tool: Krusader. That I missed that until now! I installed it right away and at first glance indeed it is the first tool that I find comparable to TC. Tools like midnight commander and under Windows FreeCommander or other lookalikes aren't for me.

TC works under Wine, but not too optimal. Krusader however is native Linux. So I'm going to check that out. Just another reason to tempt me install OpenSuse over my Windows 7 system...

Connection errors between JDeveloper 11g and OTN SoaBPM Appliance

2011-03-03T08:06:00.000-08:00

I' m working with the "Pre-built Virtual Machine for SOA Suite and BPM Suite 11g" for a while now.
I use it to prepare for SoaSuite and BPM Bootcamps and one thing I try to do is to use JDeveloper 11g on my Host (OpenSuse 11.2) and connect to the application server in my VirtualBox guest. In fact, that would (probably) be the architecture to use in a real developer environment.

So I changed the network adapter of the VirtualBox VM (in Device Settings) from NAT to "host-only". Then I re-activated the network-adapter in the guest (Oracle Enterprise Linux), to have it query for new ip-settings.As root I queried for the ip-address using ' ifconfig':

[root@soabpm-vm ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:1C:4C:37
          inet addr:192.168.56.101  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe1c:4c37/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:219 errors:0 dropped:0 overruns:0 frame:0
          TX packets:192 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21908 (21.3 KiB)  TX bytes:26686 (26.0 KiB)
          Interrupt:10 Base address:0xd020

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2222 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2222 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2942937 (2.8 MiB)  TX bytes:2942937 (2.8 MiB)

So the ip-address of my vm is '192.168.56.101'.

Then I changed the /etc/hosts file of the VM as:

127.0.0.1       localhost
192.168.56.101  soabpm-vm soapbm-vm.local bpm-vm.local bpm-vm
::1             localhost6.localdomain6 localhost6

And I added the same address in the host file of my OpenSuse host:

 192.168.56.101  soabpm-vm soabpm-vm.darwin-it.local

In Jdeveloper 11g I created an application server connection, named "soabpm-vm', added the credentials (weblogic/welcome1), hostname: soabpm-vm, ports 7001/7002, domain name: domain1. And then we test:

Fine. So in my BPM Bootcamp preperations I came to the point I have to map the abstract roles (swimlanes) to LDAP roles/users. Doing so you have to query the users over the application server connection. Choosing the connection it would populate the realm-poplist. In my case it would not.Testing in the JDeveloper within the VM would give me:

Mark the Realm "jazn.com" automatically populated.

So I tried to deploy a SoaProject, that builds fine, but deploying resulted in:

[03:10:41 PM] Error sending deployment request to server AdminServer [localhost:7001]  
 java.net.ConnectException: Connection refused 
[03:10:41 PM] Error sending deployment request to server AdminServer [localhost:7001]  
 java.net.ConnectException: Connection refused

After a while I noticed 'AdminServer [localhost:7001]'! That's not where I want to connect to! And it is certainly not the host I applied in the Jdev Applicatioin Server dialog.

Apparently the AdminURL of Weblogic is not correctly set during startup. This is set in the config.xml.
This can easily be found as follows:

[oracle@soabpm-vm config]$ cd
[oracle@soabpm-vm ~]$ cd bin
[oracle@soabpm-vm bin]$ . ./wls_env.sh
[oracle@soabpm-vm bin]$ cd $DOMAIN_HOME/config
[oracle@soabpm-vm config]$ ls config.xml
config.xml

Edit the config.xml and look for:

<external-dns-name>localhost</external-dns-name>

Then change it in:

<!--<external-dns-name>localhost</external-dns-name>-->
<external-dns-name>soabpm-vm</external-dns-name>

As always it is good practice to backup the config.xml. And after the change of course you'll need to restart the weblogic server.
But then the connection to the AdminServer will work, the realm can be queried and deploying will work.
Important lesson: the hostname in the setting above should not be localhost, but a meaningful host name.that can be resolved on both the SoaSuite server as well as the Client running JDeveloper.

Lightning on Thunderbird 3.x X86_64

2011-03-03T01:03:00.000-08:00

I used to use the Mozilla Calander plugin for Thunderbird: Lightning. Haven't used it for long, but today I found that it would be nice to have it back again. I found that it was on my thunderbird plugins but it was not compatible anymore with my version of Thunderbird.

Since Lightning is architecture specific you need the 64 bit version of it. That is: if you use the 64-bit Linux Thunderbird. It is available but for some unknown reason Mozilla did not publish it. Via the Ubuntu site (thanks to the Ubuntu Community) I found the location of the 1.0b2 beta version (for Thunderbird 3.1.x) here. The 1.0b1 version (for Thunderbird 3.0.x) here. Since it is version specific, with new versions you can try it with changing the version id in the url.

Download it, together with the accompanying Google Calendar provider, and install it in Thunderbird.

VirtualBox on Windows XP-2

2011-03-01T05:09:00.000-08:00

I thought it was quite simple to increase the VM-memory beyond the 1500MB limit of the GUI.
However, starting the SOABPM server in the OTN SOABPM appliance would certainly crash the VM. Very nice is the state VirtualBox will bring the VM in: the "Guru Mediation"-Mode.
Well, I would not call myself a VirtualBox-guru, so I wouldn't mediate in why the VM didn't feel as comfortable as it should. So turn-off the machine.

It turns out that the the 1500MB limit is not as arbitrary as thought. In the Issue Ticket here I read that the VMM process places the complete VM's memory in the process-memory space. In 32-bit Windows the maximum of the process-address space is 2GB.And 2GB for the kernel. Since there is addresspace needed for the VMM process, the video memory etc., the limit for the VM is set to 1500MB.
I would think that the limit is some further on the "save side", possibly something like 1750MB would do either.

Since I'm a pretty stubborn guy, I'll try 2000MB. Also to see the "Guru Mediation" mode state again... Then I'll try some saver values.

VirtualBox on Windows XP

2011-03-01T02:16:00.000-08:00

Today I installed VirtualBox 3.2.12 on a Windows XP. No not 4.0, since that was the version I happened to have with me on my external HD.

I imported the Technet SOABPM Appaliance, but I found that Virtual Box places the disks and machines in a hidden ".VirtualBox" folder within my "Documents and Settings" area. Since on this machine I have a roaming profile I don't want to have folders there with gig's on virtual disks. Logging on Windows with a roaming profile is already slow without this.

So I moved the ".VirtualBox" folder to a "C:\Data\VirtualBox" folder. Then VirtualBox won't find your machine anymore. To solve that a few things are to be done.

First change the defaults paths. In VB Go to <menu>/File/Preferences:

There you can change the paths to "C:\Data\VirtualMachines\VirtualBox\HardDisks" and "C:\Data\VirtualMachines\VirtualBox\Machines". I differentiate my machines with a VirtualBox subfolder, since I also use VMWare Player.

Then you can make a new VMware image based on the disks. To get the new machine have the same name as the old one I backed up the existing machine directory ("c:\Data\VirtualMachines\VirtualBox\Machines\vbox-oel5u4-soabpm-11gr1ps2-bp1-otn\").

Then I create the machine with the "do not think for yourself, let's do that later"-default settings.
After creating the machine with the two disks, I did a file compare with the old and new "vbox-oel5u4-soabpm-11gr1ps2-bp1-otn.xml" file in the machine folder.

I copy and paste the diffent settings using the file compare tool of TotalCommander in Edit-mode:

Copy the <description> node
<memory pagefusion="false" ramsize="2500"/>
Under windows XP (with 4GB, effective 3,5 GB memory) VirtualBox does not allow the setting higher then 1500MB. But editing it on file-level allows you to raise it beyond that limit. The appliance was made with 2048. This is pretty little for SoaSuite11g. But not to eat too much from Windows I gave it 2500.
Copy the <guestproperties> node
Important: change the SATA-controller to: <storagecontroller name="SCSI Controller" portcount="16" type="LsiLogic" usehostiocache="true"/>
I changed some other hardware settings from false to true based on the file compare. Eg. HardwareVirtEx.

Then it turns out that the changes did not reflect in VirtualBox. A refresh did not help. To force VirtualBox to reread the settings, Killed the VirtualBox service with ProcessExplorer:

That did the trick. Also I have it now running with 2500MB!

JDeveloper 11g SOA & BPM Extensions

2011-02-14T03:15:00.000-08:00

If you want to use JDeveloper 11g for Soa and/or BPM Development you need to download the SOA JDev Extension for the SOA Composite Designer and the BPM JDev extension for BPM Studio.

In all the documentation you'll find, you see an explanation on how to download them using the Help/Update utilities of JDeveloper. This might be neat and user-friendly for updating one JDeveloper instance where you have a fast internet access. But it is not so handy if you have to provide these extensions to multiple JDeveloper instances for a course for example. Then it is handy when you're able to download them and provide them on a stick.

I found these files on this JDeveloper extension page (just using Google).
So I downloaded the latest versions on my harddisk at home, where I have a good internet access. I hope this page is on-line indefinately.

Sql Developer Datamodeller 3.0 available

2011-02-01T01:31:00.000-08:00

Just read on Sue Harpers blog that SQL Developer Datamodeller 3.0 went production. It's downloadable from its OTN-page. I'm happy to know it's a free download. End 2008 I used the beta version to model a datamodel for a project I did then. I found it a very nice tool, a good replacement for the ERD/Datamodelling capabilities of Designer. There were a few "misses, and the main ones I had were:

Generation of pl/sql packages and functions was not possible
Subversion support was poor: when you save a project it recreates directories to save the xml-artifacts in. Recreating directory means removal of the .svn folders, which breaks your svn working copy.

In the new features list I saw promissing statements that these should be solved now. You should be able to generate packages and functions and there is even Subversion Integration (not only support): you can see pending changes, compare models and it shoud "recognize versioned designs". Well, isn't that great?

Since it's free and since it's a really good modelling tool, I think it is a must-use on every project that uses a datamodel. Haven't used it yet? Re-engineer your ERD by importing your datamodel from the database. It gives a quick insight in the datamodel for newcomers in your project.

See also for an early adopter of Sql Developer 3.0 the SqlDeveloper OTN-page. It includes the Datamodeller.

Change http port in Oracle XE

2011-01-24T03:40:00.000-08:00

For my current project I had to install tomcat as a j2ee server in Eclipse. It would not start since the Oracle XE database has port 8080 in use.

I found the method to change the ports in XE here. From that example I created my own little script:

set serveroutput on

select dbms_xdb.gethttpport as "HTTP-Port"
,      dbms_xdb.getftpport as "FTP-Port" 
from dual;

declare
  new_port varchar2(4):='8085';
begin

 dbms_output.put_line('Change HTTP Port to '||new_port);
 dbms_xdb.sethttpport(new_port);
end;
/

select dbms_xdb.gethttpport as "HTTP-Port"
,      dbms_xdb.getftpport as "FTP-Port" 
from dual;

In good old Dutch: "Mucho Plezieros" with it...

What to do to expose EBS services as a Webservice

2010-12-09T05:09:00.000-08:00

I got a comment (in Dutch) on my article about the EBS Adapter.
Since giving short answers on simple questions is not my strengths, I'll answer with a blogpost.

In the article you can make up that I'm not too enthusiastic about the EBS Adapter. The reality is often more nuanced than stated, you should consider the EBS adapter specific for your own situation.

Two of the main reasons to use the EBS Adapter are:

It sets the application context for you at connecting to the EBS instance
The Adapter Wizard enables you to introspect the available interfaces.

If these reasons are not applicable for you since for instance you set the Application Context yourself for whatever reason and/or you use customer pl/sql procedures, then might be too little to ratify the use.

But what would I do if I need to expose a pl/sql as a webservice from EBS?

EBS 11.5.10 indeed just works with JServe. From 12 onwards OC4J is used, at least initially in the 10.1.2 version (don't know if later relases are on 10.1.3).
So to start with, I'd use at least a managed OC4J 10.1.3.x Application server. Either single node or clustered. So not a standalone oc4j.

Then it depends mainly if you can use SoaSuite. If you can use SoaSuite I would create a BPEL Process, an Oracle ESB or (in case of SoaSuite11g) OSB service, based on the SoaSuite database adapter. Then arrange for setting the application context in the implementation block of the package where you have put the pl/sql procedure in.

If you can't use SoaSuite you could generate a webservice from JDeveloper based on the pl/sql procedure. And deploy that as a WAR or EAR file to the application server. The main disadvantage of having JDeveloper generate the webservice is that you can't influence the way the generated code calls the pl/sql procedure. So I think I would create a standalone java application that uses JNDI for getting the jdbc-connection. Then code the call of the pl/sql procedure in the java-application. Test it stand-alone. If that works then create webservice on that application. Doing so you have separated the technical code that does the job (calling the pl/sql procedure) and the actual webservice. For an example on how to use JNDI in standalone applications that also have to run on an AS see this article.

For creating the webservice it self you also have two choices:

Let JDeveloper generated the code for you. But then JDeveloper generates the wsdl and you have very little (near to nothing) influence on how it looks like. Unless you generate the webservice based on the wsdl.
Use a soap stack that supports annotations (like Sun Glassfish Metro). Using annotations you have very large influence on how the generated wsdl looks like. See for instance this article. Only in that case the wsdl will be generated at startup of the webservice application in the application server.

So my preferred way to go is either use SoaSuite/OSB or create a annotation based webservice on a standalone java-app that calls my procedure.

Reinvoke BPEL Process

2010-12-07T07:06:00.000-08:00

Back in 2008 I wrote an article on creating a BPEL Scheduler. The principle of that article I used later in a real implementation of a scheduler. In several blog-posts, amongst others of Clemens Utschig and Lucas Jellema, I read all kinds of arguments why you should or shouldn't use BPEL for scheduling.

I think however that BPEL is perfectly suitable to be used as to schedule jobs. Especially if you create a datamodel with an Apex front-end for instance (used JHeadstart myself for it) to register the schedule meta data to determine the next schedule dates.

There are a few considerations though. One of them is reinvoking the scheduler for a new scheduled-date, after performing a task. In my earlier article I used a plain invoke on the client-partnerlink. A big disadvantage of that is dat every task instance is created under the same root instance. After a few iterations the process tree under the tree finder is exorbitant big.

This is solved quite easily by replacing the invoke by a piece of embedded java:

<bpelx:exec name="ReInvokeRH" language="Java" version="1.4"><![CDATA[//Get logger
                org.apache.commons.logging.Log log = org.apache.commons.logging.LogFactory.getLog("my.log");
                // Get singletonId
                log.info("ReInvoke-Get SingletonId.");
                String singletonId = (String)getVariableData("singletonId");;
                //Construct xml message
                log.info("ReInvoke - Construct XML message.");
                String xml = "<BPELProcessProcessRequest xmlns=\"http://xmlns.oracle.com/BPELProcessProcessRequest \">\n<singletonId>"
                           + singletonId 
                           + "</singletonId>\n</BPELProcessProcessRequest >";
                log.info("ReInvoke message: " + xml);
                //Get a locator and delivery service.
                log.info("ReInvoke - Get Locator.");
                try {
                  Locator locator = getLocator();
                  log.info("ReInvoke - Initiate IDeliveryService.");
                  IDeliveryService deliveryService = (IDeliveryService)locator.lookupService(IDeliveryService.SERVICE_NAME);
                  // Construct a normalized message and send to Oracle BPEL Process Manager
                  log.info("ReInvoke - Construct a normalized message and send to Oracle BPEL Process Manager.");
                  NormalizedMessage nm = new NormalizedMessage();
                  nm.addPart("payload", xml);
                  // Initiate the BPEL process
                  log.info("ReInvoke - Initiate the BPEL process.");
                  deliveryService.post("BPELProcess", "initiate", nm);
                } catch (RemoteException e) {
                  log.error(e);
                  setVariableData("invokeError", "true");
                  setVariableData("invokeErrorMessage", "ReInvokeRH-RemoteException: "+e.toString());  
                } catch (ServerException e) {
                  log.error(e);
                  setVariableData("invokeError", "true");
                  setVariableData("invokeErrorMessage", "ReInvokeRH-ServerException: "+e.toString());
                }
                log.info("ReInvoke - Return.");]]>
              </bpelx:exec>

This will build a String message, create NormalizedMesage of it, and post it to the "Initiate" operation of the "BPELProcess" using the post method of the DeliveryService that is fetched using the locator of the bpel instance.

Doing so the parent-child relation is broken, the new instance runs in a new process-instance-tree.

Another consideration is that you might want to "listen" to external messages to do an abort or a reschedule. This can be done in several ways. For example by doing a correlated receive (using a correlation set on a field in the request message like the "singletonId" in the reinvoke example) in a parallel flow.

Problem is that at a normal reinvoke this receive has to be released. If not, the parellel flow will not end and/or the correlation set will not be freed. In the new instance you might get a runtime message that there is already a receive on the same correlation-id. In another project I solved a similar problem by calling the process instance with an abort message from within the process instance, before doing a re-invoke. This is quite costly in terms of performance since it involves a message to the bpel-service with all the overhead (even if it's in fact a WSIF call). So recently I found myself a little smarter solution.

What I basically did was to wrap the parallel flow with the receive and task-execution logic with a scope.
Instead of normally ending the branches, I throw a "finish" or "abort" exception:

<throw name="Throw_Finish" faultName="client:Finish" faultVariable="ReceiveInput_initiate_InputVariable"/>

Then this can be catched in the surrounding scope:

<catch faultName="client:Finish" faultVariable="ReceiveInput_initiate_InputVariable">

This might look strange at first sight, to end your normal (or abort) flow using a business-exception. Ending your flow is not an exception, is it?
But it will release your receive-activities and the correlation id they hold for a reinvoke on the same correlation-id.

B2B Queue to JMS

2010-11-02T01:27:00.000-07:00

Lately I was asked to help with routing messages from a object-type based AQ-queue to JMS.
The thing is that JMS works with text. So you have to transform the objecttype to text.
When the Object type is one of your own, you could extend it with a method "toXML" to give a XML message based on the attributes of the object.

In this case it was about a Oracle Integratin B2B AQ Queue, which is based on the B2B object "IP_MESSAGE_TYPE".

It turns out not too hard to translate the object type to JMS. I created a package for it which I provided for download here.

You can test it with the following code:

declare
  -- Non-scalar parameters require additional processing
  result     sys.aq$_jms_text_message;
  ip_message ip_message_type;
  payload    clob;
begin
  payload         := def_b2b.varchar_to_clob('Jet, Teun, Vuur, Schapen');
  ip_message := ip_message_type(MSG_ID           => 'Aap'
                                    ,INREPLYTO_MSG_ID => 'noot'
                                    ,FROM_PARTY       => 'Mies'
                                    ,TO_PARTY         => 'Zus'
                                    ,ACTION_NAME      => 'Lezen'
                                    ,DOCTYPE_NAME     => 'Leesplankje'
                                    ,DOCTYPE_REVISION => '1.0'
                                    ,MSG_TYPE         => 1
                                    ,PAYLOAD          => payload
                                    ,ATTACHMENT       => null);
  -- Call the function
  result := def_b2b.ip_message2jms_msg(ip_message => ip_message);
   result.get_text( :msg);
end;

As you can see a jms-accesible AQ-queue has a special system-Object-type: 'sys.aq$_jms_text_message'. There are several others, for different kinds of jms queues or topics. Also markt that the object types differ between Oracle 10g or 11g Enterprise Edition or equivalent and Oracle XE. In XE you wouldn't find a 'construct' method. You could try the solution of Peter Ebell for this.

Another thing is that the guys that asked me for help, had to do the enqueue of the message on the JMS-queue based on the enqueue on the source B2B-queue.
From Oracle they got permission to use a trigger on the Queue-table. To begin with they used a Before Row Insert trigger. Besides triggers on queue-tables are not support and certainly not the way to go, they encountered a problem with it. And that lays in the fact that the payload attribute is a CLOB. I allways found the way Oracle handled CLOBs at least "a little remarkable". On insert you create a row with an empty CLOB and then query it for update. In the queried row you upload the content to the CLOB-column. Since an AQ queue is based on a table it works essentially the same way. So on Before Row Insert the payload attribute is still empty. They solved it to use an After Delete trigger (when the message is consumed by the subscribing-process).

The way to go is actually to register a notification service on the queue using code like:

declare
  lc_reg_info      SYS.AQ$_REG_INFO;
  lc_reg_info_list SYS.AQ$_REG_INFO_LIST;
begin
  lc_reg_info := SYS.AQ$_REG_INFO('B2B.IP_IN_QUEUE:'
                                 ,DBMS_AQ.NAMESPACE_AQ
                                 ,'plsql://b2b.def_b2b.handle_inbound_notification?PR=1'
                                 ,HEXTORAW('FF'));

  lc_reg_info_list := SYS.AQ$_REG_INFO_LIST(lc_reg_info);
  dbms_aq.register(lc_reg_info_list, 1);
end;

Such a plsql notification function is a function that is required to have a particular "authograph":

PROCEDURE handle_inbound_notification(context  RAW
                                       ,reginfo  sys.aq$_reg_info
                                       ,descr    sys.aq$_descriptor
                                       ,payload  RAW
                                       ,payloadl NUMBER)

These parameters provide you with the data to fetch/dequeue the message this procedure is called for. You won't get the message itself, you have to dequeue-it explicitly. See the package for an example to implement this procedure.

You should perform the register for every queue-consumer that you want to reroute the messages for. But it is not too hard to put this in a parameterized-procedure and call it based on a query that fetches the consumer from either the dictionary or (better) the B2B repository. In fact this code is extracted from such a construct. But it was a little too much (I already put a reasonable amount of time in the package) to anonymize it and make it more generic.
If you need more help with it, I could of course provide some consultancy.

SQL Datamodeler EA 3.0

2010-10-11T06:01:00.000-07:00

Hello
The EA version of sql data modeler is available.
See http://forums.oracle.com/forums/ann.jspa?annID=1446

Features:
http://www.oracle.com/technetwork/developer-tools/datamodeler/ea1-newfeatures-176686.html

Finally it should be possible to generate packages.....

Templates in BPEL Transforms

2010-09-27T02:41:00.000-07:00

Multiple times I encountered that the Transformation tool of de BPEL designer has difficulties to cope with xpath and xslt functions in the stylesheet that it does not 'know'.
We have for example some custom xslt functions and I use some xpath 2.0 functions. And if I use them and deploy the process, they'll work. But for the mapper -tool the transform is invalid and it will not show the transformation map.

This is especially true for BPEL 10.1.2 that is still used at my current customer (there is a upgrade to 11g project on going).

Very anoying, because we have some large xsl's that use a large number of custom xslt-functions to do a cached dvm lookup.

But today I found a very nice workaround. If you hide those functions in a custom user template then the transformation map does not have difficulties with it.

I now have the following user-defined template (put at the bottom of the xsl stylesheet):

<xsl:template name="TransformLandCode">
<xsl:param name="landCode"/>
<xsl:comment>Transform Landcode gebruikmakend van cache:lookup
</xsl:comment>
<xsl:variable name="result" select="cache:lookup($landCode,&quot;LandCodeDomain_<FromSystem>To<ToSystem>&quot;)"/>
<xsl:value-of select="$result"/>
</xsl:template>

So where I had something like:

<xsl:value-of select="cache:lookup(/ns1:rootElement/ns1:subElement/ns1:LandCode,&quot;LandCodeDomain_&lt;FromSystem&gt;To&lt;ToSystem&gt;&quot;)"/>

I now call the template:

<xsl:call-template name="TransformLandCode">
<xsl:with-param name="landCode" select="/ns1:rootElement/ns1:subElement/ns1:LandCode"/>
</xsl:call-template>

Nice is by the way that the mapper also allows for adding the call statement in to the map using drag and drop. Also the parameters can be filled using dragging the lines:

And of course you can add code-snippets for it.

Headache from postfix

2010-09-22T03:00:00.000-07:00

Earlier I wrote a post on how postfix could be used in a mail integration solution. This was a result on a project that I'm doing using postfix to catch mail and transfer it via a bash-shell script, through mq to BPEL PM using the mq-adapter.

I added a logging construction in the script that give me a nice mechanism to do some logging:

...
TRUE=1
FALSE=0
...
#Logging variables
LOG_ENABLED=$TRUE
#LOG_ENABLED=$FALSE
LOG_DIR=/tmp/log
LOG_FILENAME=$LOG_DIR/routescript.log
#Check log dir and create it if it does not exist.
check_logdir(){
if [ "$LOG_ENABLED" -eq $TRUE ]; then
  if [ -d $LOG_DIR ]; then
    #log a seperation line
    log
  else
    mkdir $LOG_DIR
  fi
fi
}
#Function to display usage
usage(){
  SCRIPT_PARAMETERS=" -q queuename -s sender -r receiver";
  USAGE="Usage: `basename $0` $SCRIPT_PARAMETERS";
  echo $USAGE;
}
#Log
log(){
  if [ "$LOG_ENABLED" -eq $TRUE ]; then
    TEXT="$1 ""$2"
    echo $TEXT >>$LOG_FILENAME;
  fi
}
# First check logdir
check_logdir

#Do the rest of the script logic

It works nicely, but the solution described in the earlier post works only for mails with one receipient. This was caused by the flag 'D' in the transport-line of the master.cf:
The 'D' flag prepends a message with a property line with the receipient.

# 2010-02-10, M. van den Akker: Setup transport routescript for passing message to a bash-script
routescript   unix  -       n       n       -       -       pipe
flags=FDq. user=smtpuser argv=/bin/bash -c /home/smtpuser/routescript.sh -s $sender -r $recipient -q $nexthop

Besides removing this flag, also the argument parsing of the script should be adapted, to support for multiple receipients:

ARG_NR=0;
until [ -z "$1" ]
do
  ARG_NR=$(($ARG_NR+1));
  log "Argument $ARG_NR: " $1
  case $1 in
    "-s") PARAM=$1;;
    "-r") PARAM=$1;;
    "-q") PARAM=$1;;
    * )  case $PARAM in
            "-s") SENDER=$1;
                  log "Sender: " $SENDER;;
            "-r") RECEIVER=$1;
                  log "Receiver: " $RECEIVER;
                  if  [ -z "$RECEIVER_LIST" ]
                  then
                    RECEIVER_LIST=$RECEIVER;
                  else
                    RECEIVER_LIST="$RECEIVER_LIST,$RECEIVER";
                  fi
                  log "Receiverlist: " $RECEIVER_LIST;;
            "-q") FQN_QUEUE_NAME=$1;
                  log "Queue: " $FQN_QUEUE_NAME;;
            * ) usage;
            exit $E_WRONG_ARGS;;
         esac;;
  esac;
shift 1;
done;
#Log Parameters.
log "SENDER: " $SENDER
log "RECEIVER_LIST: " $RECEIVER_LIST
log "FQN_QUEUE_NAME: " $FQN_QUEUE_NAME

But implementing this I got the strange behaviour that posfix did call my script but the script did not get any arguments! Logging like the following gave me 0 arguments:

# Check Arguments
log "Number of arguments:  $#"
log "Arguments:  $*"

Yesterday and this morning I tried and figured and thought until my brains got nearly overheated. But since all my configs seemed alright and even the version I knew they worked, it finally stroke me that it had to with the commandline to call the script.

And I finally had it. It was due to the nasty '-c' option after '/bin/bash'!

It probably got there through an example I used. And it was removed in the test environments, but apparently not in my examples and documentation. So it also came into my earlier post. The option toggles the spawn of a new bash-session for the script. The arguments got into the 'parent' session but not into the 'child'-session that runs the script. Removing the '-c'option would do the trick.

Furthermore I moved the $receiver argument to the end since it can expand to more arguments. Although the script would not have problems with it, I found it saver.

So the resulting (correct) line in the master.cf should be:

routemq   unix  -       n       n       -       -       pipe
flags=Fq. user=smtpbridge argv=/bin/bash  /home/smtpbridge/routemq.sh -q $nexthop -s $sender -r $recipient

Index-of, replace and key-value parsing in XSLT

2010-08-31T03:15:00.000-07:00

Lately I had to change a few BPEL Processes in a way that I had to pass multiple parameters in one field. Instead of a complete base64 encoded data-object I had to pass a key to that object in a way that I could determine that the object was in fact a key-value pair. The field contains in that case in fact two key-value pairs. I thought that if I would code the key-value pair with something like $KEY="AABBCCDDEEFF" I could search for $KEY=" and then the value after the double-quote and before the next would then be the value.

The problem with XSLT is that you have functions like substring-before(), substring-after() and positional substring, where the from-position and length can be passed as numeric values. But for the latter function you need to determine the start and end position of the sub-string to extract from the input string. But apparently XSLT does not provide something like the Pl/Sql instr or Java index-of(). Also XSLT lacks a replace function in which you can replace a string within a string with a replacement string. The xslt-replace() function does a character-by-character replace.

Fortunately you can build these functions quite easilily yourself as xslt-templates using the substring-before(), substring-after(), and string-length() functions. I found the examples somewhere and adapted them a little for my purpose. Mainly to get them case-insensitive.

Index-of-ci
Here is my Case insensitive version of the Index-of template:

<!-- index-of: find position of search within string
2010-08-31, by Martien van den Akker -->
<xsl:template name="index-of-ci">
<xsl:param name="string"/>
<xsl:param name="search"/>
<xsl:param name="startPos"/>
<xsl:variable name="searchLwr" select="xp20:lower-case($search)"/>
<xsl:variable name="work">
<xsl:choose>
<xsl:when test="string-length($startPos)&gt;0">
<xsl:value-of select="substring($string,$startPos)"/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="$string"/>
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<xsl:variable name="stringLwr" select="xp20:lower-case($work)"/>
<xsl:variable name="result">
<xsl:choose>
<xsl:when test="contains($stringLwr,$searchLwr)">
<xsl:variable name="stringBefore">
<xsl:value-of select="substring-before($stringLwr,$searchLwr)"/>
</xsl:variable>
<xsl:choose>
<xsl:when test="string-length($startPos)&gt;0">
<xsl:value-of select="$startPos +string-length($stringBefore)"/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="1 + string-length($stringBefore)"/>
</xsl:otherwise>
</xsl:choose>
</xsl:when>
<xsl:otherwise>-1</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<xsl:copy-of select="$result"/>
</xsl:template>

The template expects 3 parameters:

string: the string in which is searched
search: the substring that has to be searched
startPos: position in string where the search is started

The first thing the template does is declaring a work-variable. If startPos is not given, the variable work will contain the complete input string. But if startPos is given work will contain the substring of string from startPos to the end.

Then the input and the search strings are converted to lower case into new variables: andinputLwr and searchLwr. These variables are to test case-insensitively if the search string is in the input. If that is the case then the part of the input string before the search string is determined with the substring-before() function. The string-length() of the result denotes in fact the numeric position of the search string within the input string. This is incremented by 1 or by startPos depending on startPos being filled.

The template can be called without the startPos parameter:

<xsl:call-template name="index-of-ci">
<xsl:with-param name="string" select="$input"/>
<xsl:with-param name="search" select="$keyStr"/>
</xsl:call-template>

Or with the parameter:

<xsl:call-template name="index-of-ci">
<xsl:with-param name="string" select="$input"/>
<xsl:with-param name="search" select="string('&quot;')"/>
<xsl:with-param name="startPos" select="$startIdx"/>
</xsl:call-template>

Replace
The next template replaces the fromStr in input to toStr.

<!-- replace-ci: case insensitive replace based on strings 
2010-08-31, by Martien van den Akker -->
<xsl:template name="replace-ci">
<xsl:param name="input"/>
<xsl:param name="fromStr"/>
<xsl:param name="toStr"/>
<xsl:param name="startStr"/>
<xsl:if test="string-length( $input ) &gt; 0">
<xsl:variable name="posStartStr">
<xsl:call-template name="index-of-ci">
<xsl:with-param name="string"
select="$input"/>
<xsl:with-param name="search"
select="$startStr" />
</xsl:call-template>
</xsl:variable>
<xsl:variable name="startPos">
<xsl:call-template name="index-of-ci">
<xsl:with-param name="string"
select="$input"/>
<xsl:with-param name="search"
select="$fromStr" />
<xsl:with-param name="startPos"
select="$posStartStr" />
</xsl:call-template>
</xsl:variable>
<xsl:variable name="inputLwr" select="xp20:lower-case($input)"/>
<xsl:variable name="startStrLwr" select="xp20:lower-case($startStr)"/>
<xsl:choose>
<xsl:when test="contains( $input, $startStrLwr ) and contains( $inputLwr, $fromStr )">
<xsl:variable name="stringBefore" select="substring($input,1,$startPos - 1)"/>   
<xsl:variable name="stringAfter" select="substring($input,$startPos + string-length($fromStr))"/>   
<xsl:value-of select="concat($stringBefore,$toStr)"/>         
<xsl:call-template name="replace-ci">
<xsl:with-param name="input"
select="$stringAfter"/>
<xsl:with-param name="fromStr"
select="$fromStr" />
<xsl:with-param name="toStr"
select="$toStr" />
<xsl:with-param name="startStr"
select="$startStr" />
</xsl:call-template>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="$input"/>
</xsl:otherwise>
</xsl:choose>
</xsl:if>
</xsl:template>

Here the pos of the from string is determined using the index-of-ci template described earlier.
But this is done from the position of the startStr that marks the start of the replacement search. For example, if you want to replace domain names in email-adresses, you want to start the search of the domain after the at-sign ( '@' ).
Having the start position of the 'from'-string the part of the input before and after the 'from'-string is taken using the string-before() and strina-after() functions.
The 'to'-string is concatenated to the result of the string-before(). The result of the string-after() is used to call the template recursively to search the remainder of the input-string.

Parsing keys
The following template parses a key value like $KEY="AABBCCDDEEFF"

<!-- Parse a KeyValue
2010-08-31, By Martien van den Akker -->
<xsl:template name="getKeyValue">
<xsl:param name="input"/>
<xsl:param name="key"/>
<xsl:param name="default"/>
<!-- Init variables -->
<xsl:variable name="keyStr" select="concat('$',$key,'=&quot;')"/>
<xsl:if test="string-length( $input ) &gt; 0">
<xsl:variable name="startIdxKey">
<xsl:call-template name="index-of-ci">
<xsl:with-param name="string" select="$input"/>
<xsl:with-param name="search" select="$keyStr"/>
</xsl:call-template>
</xsl:variable>
<xsl:variable name="keyLength" select="string-length($keyStr)"/>
<xsl:variable name="startIdx" select="$startIdxKey+$keyLength"/>
<xsl:variable name="endIdx">
<xsl:call-template name="index-of-ci">
<xsl:with-param name="string" select="$input"/>
<xsl:with-param name="search" select="string('&quot;')"/>
<xsl:with-param name="startPos" select="$startIdx"/>
</xsl:call-template>
</xsl:variable>
<!-- Determine value -->
<xsl:choose>
<xsl:when test="$startIdxKey&gt;=0 and $endIdx&gt;=0">
<xsl:value-of select="substring($input,$startIdx, $endIdx - $startIdx)"/>
</xsl:when>
<xsl:when test="$startIdxKey>0 and $endIdx&lt;0">
<xsl:value-of select="substring($input,$startIdx)"/>
</xsl:when>
<xsl:otherwise>
<xsl:if test="$default='Y'">
<xsl:value-of select="$input"/>
</xsl:if>
</xsl:otherwise>
</xsl:choose>
</xsl:if>
</xsl:template>

The working is quite similar to the templates above. If you understand the replace-template, you wouldn't have trouble with this one. Basically the value is search using the index-of-ci template with a concatenation of '$', the key and '="'. The string after that is the value, with a double quote as the end delimiter.
Having this template you can search for the key any where in the string, even if there are multiple key-value pairs.
The template can be called like:

<ns1:Id>
<xsl:call-template name="getKeyValue">
<xsl:with-param name="input" select="/ns2:aap/ns2:noot"/>
<xsl:with-param name="key" select="'KEY'"/>
<xsl:with-param name="default" select="'Y'"/>
</xsl:call-template>
</ns1:Id>

The parameter 'default' is optional: if it is set to 'Y' and the KEY is not found in the input string, then the value of the input is returned as result. Otherwise nothing is returned.

Conclusion
I found these templates very helpful. Using them you can do almost any string replacements in XSLT. ALso to me they function as example to cope with more advanced XSLT-challenges.

VMWare Player vs. Server, vs. Virtual Box

2010-07-01T00:18:00.000-07:00

Virtualization is great. It is very handy to have your complex installations of server software in a Virtual Machine. I do it for years and have only some client tools that do need a complex installation on my host. JDeveloper, SQLDeveloper or Pl/Sql Developer for instance. It enables you to reinstall your laptop and be up and running after having installed your virtualization product. Provided that you backup your VM's of course. Also I have a Windows based VM with some tools for administration purposes.

I started years ago with VMware Workstation 3 to 5.x. But then VMware Server came out. And back then workstation did not provide much extra above Server. And although I had a demo license for Workstation (which ended), Server was for free. Actually the only missing feature in Server I encountered was 'Shared Folders'. But that I solved using (S)Ftp. Under windows it is very convenient to have a FileZilla Server. You could solve it by defining a share in Windows. But I found that that gives some problems when you have multiple copies of a VM running on the same network. Then you have to change your host name, etc. And it is problematic if you have multiple shared open in multiple VM's while your windows account have a regularly changed password. So shared folders as an alternative to FTP is not necessary but very convenient.

Shortly after VMware Server VMware Player came out. But until today it was not possible to create new VM's in Player. And that is a 'must have' to me.

VMware 2.0 was a large step for me since I found it astonishing that the footprint from 1.0 grew by a factor 5! But since VMware dit not keep 1.0 up with the Linux Kernel changes, I did the upgrade. Now I was a pretty satisfied VMware Server user.

But there are two problems now with VMware Server 2.x.:

VMware Server 2.x does not keep up with the Linux kernel changes either. I upgraded to OpenSuse 11.2, which worked very well. But stepped back to 11.1 since VMware Server did not install with that kernel.
Since FireFox 3.6 the VMware Console does not work anymore. This is solvable by installing a seperate FF 3.5.9 for VMware usage. But this is very inconvenient.

Lately I tried Virtual Box again because of the Firefox problems and I reported to be very enthousiastic about it. And indeed it is a very good product. But unfortunately most of my VM's are VMware based. It is possible to run a VM in virtual box based on VMware files. But you have to create a new Virtual Box VM for that purpose that refer to the VMware files. And than you have to de-install the VMware Tools (which probably have installed in that VM) and install VirtualBox GuestAdditions. It's all doable and no rocket-science. But it's not a simple import-and-run.

But since 25th of may VMware Player 3.1 was released. And apparently Vmware released it as an answer to the Windows XP mode on Windows 7. To run older Windows XP compatible apps in Windows 7. But it occurs to me that they've looked closely to VirtualBox. The VMware Unity mode is quite comparable or actually the same as the Seamless Windows mode of VirtualBox. This is a very attractive feature that makes VMware Player as well as VirtualBox suitable to beat the Windows XP mode of Windows 7. I have not be able to try the XP mode of Windows 7. So I don't know about load times of the XP mode. But for VirtualBox and VMware Player you need to start the guest Windows to be able to use it. And you have to have an activated version of Windows in your VM.

And since VMware Player 3.1 I'm able to create VM's in Player. Actually, with being able to create VM's in Player and with the 'Shared Folder' feature I think you might say that Player is on the 'must-have'-feature-level of VMware Workstation 5.5. And since Player has a smaller foot print (100MB, about the same as Server 1.0.x) and does not need to have an Apache Tomcat running, no need for browser access of the console: VM's running in a seperate application window, Player is a very much better suitable for running VM's on a laptop/desktop. And a feature like VMware Unity/VirtualBox Seamless Windows mode is really nice. It is funny to have windows applications running in a window side by side with your Linux apps. Even copy and paste works. And having shared folders makes it possible to have your Guest-OS apps work with the same files as your host OS apps.

And what made me most lyric? I had my OpenSuse 11.1 connected to a beamer. In the NVidia conrols I had the CRT (as NVidia/X called it) placed right of my LCD desktop. Then I placed my PowerPoint in Unity on the Beamer area of the desktop (so I was able to see other apps on my lcd screen). And then it played the slideshow fullscreen on the beamer...
I was very happy about it, because I had PowerPoint 2007 working under Wine. But editing a file under PowerPoint2007/wine was not doable. Whey drag and drop or move an object on the canvas the canvas becomes black until you release the object. Only then it draws again. But in the VMware Unity mode, PowerPoint 2007 is usable under Linux!

To conclude, when to use what? A question I try to answer a lot lately.

If you want to run VMware VM's on your own laptop along with other apps, use VMware Player
If you want to run MS-Office 2007 or other Windows apps side by side with other applications on your Linux machine, use either VMware Player with Unity or Virtual Box with Seamless Windows mode (you may choose). But you should try if the app works properly enough under Wine, since that will give you much faster startup times and lower system-load, since you do not have to load Windows as a guest os.
If you want to run VM's on a host that has to be reachable from different location, then use VMware Server. For example, if you have a spare desktop on the attick that you want to leverage for different purposes, then use Server. With server you can start, stop and use the VM-desktop using a browser. Unfortunatly it has to be a FireFox 3.5 at most, for the current Server version (2.x).
If you look for an enterprise solution, look at Oracle VM or VMware ESX and their management products. These solutions are beyond my use. Since they run on 'bare-metal', and that is out of the question for me.

I don't have particular arguments for the choice between VirtualBox and VMware Player. There are other virtualization tools like KVM or Xen based. There are people that favor a particular solution over VMware because of performance. Especially the VMware would not be that fast in IO-operations. But I haven't seen any benchmarks that support this. I haven't done any measurements in particular but I did not experience any particular performance differences between VirtualBox and VMware. The only reason actually that I favor VMware is just that most of my VM's are based on VMware.

But I sincerely hope that Oracle works on improving both Oracle VM and VirtualBox in a way that get both products in line with eachother and eventually let both products work with the same VM-architecture. Then it would be able to use Oracle VM's on both Oracle VM and VirtualBox. Maybe that would be an extra stimulus to build VirtualBox/Oracle VM appliances.

Oracle Inserts based on DB2 selects

2010-06-16T06:19:00.000-07:00

It's been a while that I wrote an article. This week I struggled with creating insert scripts based on data from DB2 to be used in my local test database (Oracle XE) at my customer.

We use Siebel and have to integrate here and there by querying data from the Siebel DB2 database. It turns out that my local database adapter has trouble with connecting to the DB2 database. Could not find out what's wrong, so I decided that I would query the data from the Siebel tables and insert it into my local XE database. It's faster anyway (in my case) and it also allows me to manipulate the data for test-case purposes.
But querying db2 to generate insert statements isn't as obvious as I would do it in Oracle.

Here is an example script.

select
'Insert into CONTACT (PARTYROWID,KLANTID,KLANTTYPE,AANGEMAAKTOP,BANKCODE,BANKLOCATIE,KLANTSTATUS,CORRESPONSDENTIETAAL,PRIMAIRTELEFOONNUMMER,'
|| 'PRIMAIRTELEFOONTYPE,PRIMAIREMAIL,PRIMAIREMAILFORMAAT,TELEFOONPRIVE,TELEFOONOVERIG,TELEFOONMOBIEL,TELEFOONWERK,FAX,EMAIL,EMAILFORMAAT,EMAILDATUM,'
|| 'EMAILBRON,INGEZETENEVAN,NATIONALITEIT,ACHTERNAAM,VOLLEDIGEACHTERNAAM,ROEPNAAM,GESLACHTSNAAM,VOORLETTERS,VOLLEDIGEVOORNAMEN,ACADEMISCHETITEL'
||',TUSSENTITEL,ACHTERVOEGSEL,ACHTERTITEL,VOORVOEGSEL,VOORVOEGSELGESLACHTSNAAM,GEBOORTDATUM,GESLACHT,GEBOORTELAND,GEBOORTEPLAATS,EIGENHUIS,FAILLIET,SAMENLEVINGSVORM'
||',BURGERLIJKSTAAT,HUW_VOORWAARDEN,PERSONEEL,TYPEKLANT,MAATSCHAPPELIJKESTATUS,LOKALEKLANTINDELING,CENTRALEKLANTINDELING,KLANTINDELING,LOKAALINGEDEELD,BEHOEFTEPROFIEL,'
||'TAXIDENTIFICATIONNR,WOONPLAATSVERKLARING,SOFINUMMER,REDENGEENSOFINUMER,DATUMOVERLIJDEN,OVERLEDEN,AARDIDENTIFICATIEDOC,DATUMLEGITIMATIE,NRIDENTIFICATIEDOC,'
|| 'DATUMUITGIFTE,LANDUITGIFTE,PLAASTUITGIFTE,PERTELEFOONBENADEREN,PEREMAILBENADEREN,PERPOSTBENADEREN,PERSMSBENADEREN,PERFAXBENADEREN,INSOLVENCYSTATUS,IKBNUMBER)'
||' values ('
|| ''''|| coalesce(ctt.PARTYROWID,'') ||''','
|| ''''|| coalesce(ctt.KLANTID,'') ||''','
|| ''''|| coalesce(ctt.KLANTTYPE,'') ||''','
|| case when ctt.AANGEMAAKTOP is not null then 'to_date('''||varchar_format( ctt.AANGEMAAKTOP,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce(ctt.BANKCODE,'') ||''','
|| ''''|| coalesce(ctt.BANKLOCATIE,'') ||''','
|| ''''|| coalesce(ctt.KLANTSTATUS,'') ||''','
|| ''''|| coalesce(ctt.CORRESPONSDENTIETAAL,'') ||''','
|| ''''|| coalesce(ctt.PRIMAIRTELEFOONNUMMER,'') ||''','
|| ''''|| coalesce(ctt.PRIMAIRTELEFOONTYPE,'') ||''','
|| ''''|| coalesce(ctt.PRIMAIREMAIL,'') ||''','
|| ''''|| coalesce(ctt.PRIMAIREMAILFORMAAT,'') ||''','
|| ''''|| coalesce(varchar(ctt.TELEFOONPRIVE),'') ||''','
|| ''''|| coalesce(varchar(ctt.TELEFOONOVERIG),'') ||''','
|| ''''|| coalesce(varchar(ctt.TELEFOONMOBIEL),'') ||''','
|| ''''|| coalesce(varchar(ctt.TELEFOONWERK),'') ||''','
|| ''''|| coalesce(varchar(ctt.FAX),'') ||''','
|| ''''|| coalesce(varchar(ctt.EMAIL),'') ||''','
|| ''''|| coalesce(varchar(ctt.EMAILFORMAAT),'') ||''','
|| case when ctt.EMAILDATUM is not null then 'to_date('''||varchar_format( ctt.EMAILDATUM,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce(EMAILBRON,'') ||''','
|| ''''|| coalesce(ctt.INGEZETENEVAN,'') ||''','
|| ''''|| coalesce(ctt.NATIONALITEIT,'') ||''','
|| ''''|| coalesce(ctt.ACHTERNAAM,'') ||''','
|| ''''|| coalesce(ctt.VOLLEDIGEACHTERNAAM,'') ||''','
|| ''''|| coalesce(ctt.ROEPNAAM,'') ||''','
|| ''''|| coalesce(ctt.GESLACHTSNAAM,'') ||''','
|| ''''|| coalesce(ctt.VOORLETTERS,'') ||''','
|| ''''|| coalesce(ctt.VOLLEDIGEVOORNAMEN,'') ||''','
|| ''''|| coalesce(ctt.ACADEMISCHETITEL,'') ||''','
|| ''''|| coalesce(ctt.TUSSENTITEL,'') ||''','
|| ''''|| coalesce(ctt.ACHTERVOEGSEL,'') ||''','
|| ''''|| coalesce(ctt.ACHTERTITEL,'') ||''','
|| ''''|| coalesce(ctt.VOORVOEGSEL,'') ||''','
|| ''''|| coalesce(ctt.VOORVOEGSELGESLACHTSNAAM,'') ||''','
|| case when ctt.GEBOORTDATUM is not null then 'to_date('''||varchar_format( ctt.GEBOORTDATUM,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce(ctt.GESLACHT,'') ||''','
|| ''''|| coalesce(ctt.GEBOORTELAND,'') ||''','
|| ''''|| coalesce(ctt.GEBOORTEPLAATS,'') ||''','
|| ''''|| coalesce(ctt.EIGENHUIS,'') ||''','
|| ''''|| coalesce(ctt.FAILLIET,'') ||''','
|| ''''|| coalesce(ctt.SAMENLEVINGSVORM,'') ||''','
|| ''''|| coalesce( ctt.BURGERLIJKSTAAT,'') ||''','
|| ''''|| coalesce( ctt.HUW_VOORWAARDEN,'') ||''','
|| ''''|| coalesce( ctt.PERSONEEL,'') ||''','
|| ''''|| coalesce( ctt.TYPEKLANT,'') ||''','
|| ''''|| coalesce( ctt.MAATSCHAPPELIJKESTATUS,'') ||''','
|| ''''|| coalesce( ctt.LOKALEKLANTINDELING,'') ||''','
|| ''''|| coalesce( ctt.CENTRALEKLANTINDELING,'') ||''','
|| ''''|| coalesce( ctt.KLANTINDELING,'') ||''','
|| ''''|| coalesce( ctt.LOKAALINGEDEELD,'') ||''','
|| ''''|| coalesce( ctt.BEHOEFTEPROFIEL,'') ||''','
|| ''''|| coalesce( ctt.TAXIDENTIFICATIONNR,'') ||''','
|| ''''|| coalesce( ctt.WOONPLAATSVERKLARING,'') ||''','
|| ''''|| coalesce( ctt.SOFINUMMER,'') ||''','
|| ''''|| coalesce( ctt.REDENGEENSOFINUMER,'') ||''','
|| case when ctt.DATUMOVERLIJDEN is not null then 'to_date('''||varchar_format( ctt.DATUMOVERLIJDEN,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce( ctt.OVERLEDEN,'') ||''','
|| ''''|| coalesce( ctt.AARDIDENTIFICATIEDOC,'') ||''','
|| case when ctt.DATUMLEGITIMATIE is not null then 'to_date('''||varchar_format( ctt.DATUMLEGITIMATIE,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce( ctt.NRIDENTIFICATIEDOC,'') ||''','
|| case when ctt.DATUMUITGIFTE is not null then 'to_date('''||varchar_format( ctt.DATUMUITGIFTE,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end ||','
|| ''''|| coalesce( ctt.LANDUITGIFTE,'') ||''','
|| ''''|| coalesce( ctt.PLAASTUITGIFTE,'') ||''','
|| ''''|| coalesce( ctt.PERTELEFOONBENADEREN,'') ||''','
|| ''''|| coalesce( ctt.PEREMAILBENADEREN,'') ||''','
|| ''''|| coalesce( ctt.PERPOSTBENADEREN,'') ||''','
|| ''''|| coalesce( ctt.PERSMSBENADEREN,'') ||''','
|| ''''|| coalesce( ctt.PERFAXBENADEREN,'') ||''','
|| ''''|| coalesce( ctt.INSOLVENCYSTATUS,'') ||''','
|| ''''|| coalesce( ctt.IKBNUMBER,'') ||''');'
from siebel.contact ctt
where klantid='12345';

The first 'not so obvious' is the NVL-function. This is a typical Oracle function. For most purposes this can be translated to the coalesce function above. In most cases when the column is empty I want to have an "empty value". In some cases just
giving "coalesce( column-reference,'')" does not suffice. I had to cast the column explicitly to char with the varchar() function:

coalesce(varchar(ctt.TELEFOONPRIVE),'')

Here TELEFOONPRIVE is apparently a number column. The function coalesce() assumes the number datatype and can't accept an empty string as default string.

For dates it is a little more complicated. If there is a date I want to transform it to an Oracle to_date function. But then I have to be sure that the format comming from DB2 is of a standard format. I choose "YYYY-MM-DD HH24:MI:SS". If the date is empty I just want to return an empty string again. I couldn't come up with a simple construct using coalesce(). So I used the CASE WHEN-construct:

case when ctt.DATUMOVERLIJDEN is not null then 'to_date('''||varchar_format( ctt.DATUMOVERLIJDEN,'YYYY-MM-DD HH24:MI:SS')||''',''YYYY-MM-DD HH24:MI:SS'')' else 'null' end

It took me a while to find out that where in Oracle you can provide a date-format to the to_char(<date-value>, <date-format>) function, in DB2 you need the varchar_format(<date-value>, <date-format>) function for that. Luckily the accepted date-formats are the same in my case. So here I transform the date-value from DB2 to the required date-format and concatenate it with the Oracle to_date() function with the same format.

The generated insert statement(s) will look like (enter at values clause added manually for readability):

Insert into CONTACT (PARTYROWID,KLANTID,KLANTTYPE,AANGEMAAKTOP,BANKCODE,BANKLOCATIE,KLANTSTATUS,CORRESPONSDENTIETAAL,PRIMAIRTELEFOONNUMMER,PRIMAIRTELEFOONTYPE,PRIMAIREMAIL,PRIMAIREMAILFORMAAT,TELEFOONPRIVE,TELEFOONOVERIG,TELEFOONMOBIEL,TELEFOONWERK,FAX,EMAIL,EMAILFORMAAT,EMAILDATUM,EMAILBRON,INGEZETENEVAN,NATIONALITEIT,ACHTERNAAM,VOLLEDIGEACHTERNAAM,ROEPNAAM,GESLACHTSNAAM,VOORLETTERS,VOLLEDIGEVOORNAMEN,ACADEMISCHETITEL,TUSSENTITEL,ACHTERVOEGSEL,ACHTERTITEL,VOORVOEGSEL,VOORVOEGSELGESLACHTSNAAM,GEBOORTDATUM,GESLACHT,GEBOORTELAND,GEBOORTEPLAATS,EIGENHUIS,FAILLIET,SAMENLEVINGSVORM,BURGERLIJKSTAAT,HUW_VOORWAARDEN,PERSONEEL,TYPEKLANT,MAATSCHAPPELIJKESTATUS,LOKALEKLANTINDELING,CENTRALEKLANTINDELING,KLANTINDELING,LOKAALINGEDEELD,BEHOEFTEPROFIEL,TAXIDENTIFICATIONNR,WOONPLAATSVERKLARING,SOFINUMMER,REDENGEENSOFINUMER,DATUMOVERLIJDEN,OVERLEDEN,AARDIDENTIFICATIEDOC,DATUMLEGITIMATIE,NRIDENTIFICATIEDOC,DATUMUITGIFTE,LANDUITGIFTE,PLAASTUITGIFTE,PERTELEFOONBENADEREN,PEREMAILBENADEREN,PERPOSTBENADEREN,PERSMSBENADEREN,PERFAXBENADEREN,INSOLVENCYSTATUS,IKBNUMBER) 
values ('1-100BM-100','000000105727750','Person',to_date('2006-05-09 19:59:21','YYYY-MM-DD HH24:MI:SS'),'3365','336515','C','NL','','','','','','','','','','','',to_date('2008-09-15 00:00:00','YYYY-MM-DD HH24:MI:SS'),'FULFILMENT','01','',to_date('2009-07-08 00:00:00','YYYY-MM-DD HH24:MI:SS'),'NL','NL','Name','Name','','Name','I.R.S.','Iris Ronald Simon','','','','','','',to_date('1966-11-11 00:00:00','YYYY-MM-DD HH24:MI:SS'),'M','NL','Tool Town','Y','N','3','1','9','03','01','08','','1','1','Y','','','X','123456789','',null,'N','03',to_date('2005-07-29 00:00:00','YYYY-MM-DD HH24:MI:SS'),'IC4631943',to_date('2004-07-29 00:00:00','YYYY-MM-DD HH24:MI:SS'),'NL','Tool Village','N','N','Y','Y','N','','1-22A-3344');

Shared folders in VirtualBox

2010-04-21T04:47:00.000-07:00

If you have installed the VirtualBox guest-additions then you can use the SharedFolders functionality as well. This might be a usefull differentiating feature over VMware Server. VMware Workstation has this feature as well, but it will cost you about 190 dollar.

Shared Folders are folders on your host OS that you denoted to the virtualization product (VirtualBox or VMware Workstation) as being available to the guest OS. This handy because you then do not have to setup Windows or Samba shares on your host to connect to from your guest. Also it prevents you from having to setup particular network settings for it.

An alternative to shared folders, besides Windows or Samba shares is to (S)Ftp the particular files to your guest. If it is about installation files (for example the Oracle 11gR2 database) then you need to have space available in the guest-virtual disks. The virtual disks will grow accordingly. If you go for that approach then it is wise to add a temporary virtual disk to hold the installations. Afterwards you can remove the disk without the need to defrag and shrink the remaining disks.

To use the shared folders in VirtualBox you need to define a folder to share in the SharedFolders screen. This is available through the Devices main menu option:

In a Windows guest you might find the shared folders in the Windows Explorer under the network places.
In Linux guest you need to mount the share explicitly.
You first need to make a directory under the /mnt folder :


[root@oel5soa11g mnt]# mkdir Zarchief
[root@oel5soa11g mnt]# chmod a+w Zarchief/
[root@oel5soa11g mnt]# chmod a+x Zarchief/
[root@oel5soa11g mnt]# ls -l
total 8
drwxrwxrwx 2 root root 4096 Apr 21 14:11 Zarchief

Then you can mount the shared folder with the following command:


# mount -t vboxsf [SharedFolderName] /mnt/[FolderName]

For example:


[root@oel5soa11g mnt]# mount -t vboxsf Zarchief /mnt/Zarchief

FireFox 3.6 conflict with VMware Console: VirtualBox

2010-04-21T02:29:00.000-07:00

Today I wanted to work with a VMware image to play around with weblogic/soasuite 11g etc. But I ran into the nasty FireFox 3.6 conflict with the VMware Console. Since FireFox 3.6.x the console won't start, because of some time-out error. The solution would be to downgrade to FireFox 3.5. But since I have a repository installed version, I found a downgrade too tedious. Too bad that the console plugin won't install in Google Chrome. I tried Mozilla Seamonkey, but that wouldn't do the trick also.

So I desided to get VirtualBox from the stable again. I neatly installed it using the VirtualBox repository for my OpenSUSE (see the bottom of the page here).

I created a VirtualBox VM based on the VMware files of the VM I wanted to start. See this earlier blogpost for a how-to.

Naively I removed the IDE-controller. But it turns out to be needed to be able to mount the Guest-Additions-ISO file. So don't remove that.

Now I hope that there is blessing on this traject, since I have put in too much time in it already, in my opinion.

Logging in bash script

2010-04-01T02:44:00.000-07:00

To debug a bash script and to know what the environment is where a script is behaving it is convenient to log. This is what I added to my e-mail prosessing script yesterday. It is simple and of course every one else could think of it. Probably it is invented hundreds of times. Here's my solution.

#!/bin/bash
###################################################################################################
# Log
# Script to demonstrate logging
#
# author: Martien van den Akker
# (C) march 2010
# Darwin-IT Professionals
###################################################################################################
#Declarations
TRUE=1
FALSE=0
#Logging variables
LOG_ENABLED=$TRUE
#LOG_ENABLED=$FALSE
LOG_DIR=/tmp/log
LOG_FILENAME=$LOG_DIR/routemq.log


#Check log dir and create it if it does not exist.
check_logdir(){
if [ "$LOG_ENABLED" -eq $TRUE ]; then
if [ -d $LOG_DIR ]; then
#log a seperation line
log
else
mkdir $LOG_DIR
fi
fi
}

#Log
log(){
if [ "$LOG_ENABLED" -eq $TRUE ]; then
TEXT="$1 ""$2"
echo $TEXT >>$LOG_FILENAME;
fi
}
# First check logdir
check_logdir
# Log Arguments

log "Number of arguments: " $#
log "First argument: " $1
log "Second argument: " $1
log "End of script"

The script starts with a call to check_logdir(). This function checks if the LOG_DIR exists. If it exists it adds a seperation line. This is because the if has to have a command in the then section. But it is also convenient because you have a seperation line between script calls.
Then there is the log function. The log function accepts two parameters. One is the prompt, the other is a string to be concatenated to the prompt. Handy for listing parameter-values.
But you could also do a logging of only one line. For example the last line.

The logging can be enabled or disabled by commenting/uncommenting the proper line of:

LOG_ENABLED=$TRUE
#LOG_ENABLED=$FALSE

Outsourcing of server management

2010-04-01T01:43:00.001-07:00

It's like in the old Cobol days. When you were working at the Automation department of the Dutch Tax office (I got this from past co-workers that were some older than I), you worked in Apeldoorn in the east of the country, but the datacenter was in The Hague in the west. A distance of about 300 km.

You coded your Cobol on punch cards. And if you were smart then you indexed your lines. You had to do a visual code check. The punch cards were put in a box and sent by courier to the datacenter in The Hague. There the cards were 'loaded' to the mainframe and if you did your visual code checking well it compiled and executed. And then you got your output back by courier.
If the box fell of the cart, you were happy you indexed your code lines. Because then the cards could be fed to a punch-card-sorter.

That's about how I feel right now. I created a script and developed a postfix configuration. But it has to be put on the Linux development machine by a system manager. Although it's a development-server there are some good reasons to not give me root-priviliges to the development server.
And since postfix runs as root, you have to be root to change the config-files. But because I do not even have a normal user-account I cannot read the logs. The script is put in a non-root-non-postfix-useraccount. But I can't update the script myself.

So, I have to do a change and now we wait until the feedback. This already goes on for days, a few weeks/months if I include the requests for accounts and initial server setup. And that for something that could be solved in a few hours (if I exclude the requests for accounts and initial server setup), if I could get my own fingers at the keys.

But so be it. The server management here is not really 'outsourced', but it is at another geographical location. In another city. And done by other people that are also busy with other tasks. They're helpful. They really are. But the overall duration is the price the organization, the customer is paying for these policies.

Postfix for handling mail in your integration solution

2010-03-24T00:21:00.000-07:00

Sometimes there is a need for integrating with mail. You could say: that's easy, since we could use the notification service in BPEL as described here.
However, this solution requires that there is a mail-box to connect to. But what if you want to serve multiple (10s, 100s or even 1000s of) e-mail-addresses within a certain domain? Maybe multiple sub-domains within a certain main-domain? Then you would not want to create seperate mail-boxes for each address. That would give to much of an administration. Not only you have to create new mail-boxes, but also have activation agents for each of this mail-box. That would have an enormous performance pressure. So, what then?

In my previous post on handling email with SoaSuite (BPEL), I already mentioned Apache James as an email server. Nearly every Linux distribution also comes with Postfix. Postfix is a Mail Transfer Agent. It's not an e-mail server like James. It handles SMTP-messages. It will listen for SMTP-trafic and filter each message to determine if it has to be passed on to another MTA or to be handled locally. If a message has to be handled locally, it can be stored to local mailboxes or a local POP or IMAP server, for example. But it is also quite easy to let Postfix call a local script or executable. And that for mail-addresses that meet certain filter rules, like belonging to a domain. This script can then be used to have the message put/enqueued on a queue. The script can also be used to enhance the email messages with properties from the SMTP-envelope. I added that in the example too.

Now I'm not an e-mail server expert. But I had to figure out how to configure Postfix for this use-case. And although Postfix seem complicated at first sight, this turns out remarkebally easy.

Postfix installation
I choose Oracle Enterprise Linux 5 Update 4, as an alternative to Red Hat. But about any Linux-taste would do. Provided that it has a Postfix-pacYou might uninstall Sendmail (or unselect it in the package-options on install of the OS). This to prevent collision with the Postfix functionality.

Although you would do al the configuration as root (postfix runs as root), it is strongly advised to introduce an extra user for the message-handling. To own the scripts, etc.
In my setup the following folders are created:

Folder	Meaning
/etc/postfix	Configuration files and lookup tables
/usr/libexec/postfix	Postfix daemons
/var/spool/postfix	Queue files
/usr/sbin	Postfix commands

This is a quite common setup, but in other (Linux or Unix) distributions the folder locations might differ slightly.

Postfix Usecase
This schema provides an overview on how Postfix will work. Left Incoming (smtp) messages will be picked up by the Smtp-deamon. Then via "Cleanup" it will be handed over to the QueueManager, via the incoming queue. The Queuemanager will put the message on the Active queue. From there it will be picked up by either the smtp-service to forward the mail to the smtp-server on the infrastructure of your Internet Service Provider or your company. Using MA-records of the DNS-server it will determine to which smtp-server it has to be send actually.
In our use-case the pipe deamon is the interesting one. This is the one we're going to instruct to call a script.

Basic Settings
Hostname
The hostname command can be used to determine the fully qualified name. This is the one used by Postfix to determine the host. If this does not include the domain-name then the following command can be used to instruct Postfix what the FQN should be:

postconf -e myhostname=vmsmtp.darwin-it.local

Mydestination
This parameter can be left default. But if mail have to be accepted for certain domains and to be delivered using the local transport, set the following parameter in main.cf:

# 2010-02-03, M. vd. Akker: To have mail accepted for .darwin-it.local
mydestination = $myhostname, localhost.$mydomain, localhost, .darwin-it.local

For now leave it default:

mydestination = $myhostname, localhost.$mydomain, localhost

Network interfaces
To have mail from external networks, non-localhost, the inet_interfaces must be set in main.cf. In my case I run Postfix within a Virtual Machine. And I want to use my Thunderbird to send mail to Postfix. So first Postfix has to be told from which network-devices it should accept mail from. Since my VM is "hidden", for simplicity we accept mail from all network devices. But this can be narrowed down.

# RECEIVING MAIL

# The inet_interfaces parameter specifies the network interface
# addresses that this mail system receives mail on.  By default,
# the software claims all active interfaces on the machine. The
# parameter also controls delivery of mail to user@[ip.address].
#
# See also the proxy_interfaces parameter, for network addresses that
# are forwarded to us via a proxy or network address translator.
#
# Note: you need to stop/start Postfix when this parameter changes.
#
# 2010-02-10, M. van den Akker: Setup all interfaces for excepting mail.
inet_interfaces = all
#inet_interfaces = $myhostname
#inet_interfaces = $myhostname, localhost
#inet_interfaces = localhost

Specific domains to accept mail from are set with mynetworks parameters. To select sub-domains to accept mail from, change the mynetwork parameter to accept mail from your host for example.

# TRUST AND RELAY CONTROL

# The mynetworks parameter specifies the list of "trusted" SMTP
# clients that have more privileges than "strangers".
#
# In particular, "trusted" SMTP clients are allowed to relay mail
# through Postfix.  See the smtpd_recipient_restrictions parameter
# in postconf(5).
#
# You can specify the list of "trusted" network addresses by hand
# or you can let Postfix do it for you (which is the default).
#
# By default (mynetworks_style = subnet), Postfix "trusts" SMTP
# clients in the same IP subnetworks as the local machine.
# On Linux, this does works correctly only with interfaces specified
# with the "ifconfig" command.
#
# Specify "mynetworks_style = class" when Postfix should "trust" SMTP
# clients in the same IP class A/B/C networks as the local machine.
# Don't do this with a dialup site - it would cause Postfix to "trust"
# your entire provider's network.  Instead, specify an explicit
# mynetworks list by hand, as described below.
#
# Specify "mynetworks_style = host" when Postfix should "trust"
# only the local machine.
#
#mynetworks_style = class
#mynetworks_style = subnet
#mynetworks_style = host

# Alternatively, you can specify the mynetworks list by hand, in
# which case Postfix ignores the mynetworks_style setting.
#
# Specify an explicit list of network/netmask patterns, where the
# mask specifies the number of bits in the network part of a host
# address.
#
# You can also specify the absolute pathname of a pattern file instead
# of listing the patterns here. Specify type:table for table-based lookups
# (the value on the table right-hand side is not used).
#
# 2010-02-10, M. van den Akker: Setup for accepting mail from CMI and localhost.
mynetworks = 192.168.192.0/28, 127.0.0.0/8
#mynetworks = 168.100.189.0/28, 127.0.0.0/8
#mynetworks = $config_directory/mynetworks
#mynetworks = hash:/etc/postfix/network_table

Where 192.168.192.0/28 should be replaced by the address-range of the host(s) from which you want to be able to receive email.

Start and stop postfix
Postfix can be stopped by:

[root@vmsmtpserver postfix]# postfix stop

Postfix can be started by:

[root@vmsmtpserver postfix]# postfix start

If it’s not already running.

After making changes to the configuration, Postfix has to be told to reload the configuration:

[root@vmsmtpserver postfix]# postfix reload
postfix/postfix-script: refreshing the Postfix mail system

Incoming e-Mail
Postfix will have to be configured that *.darwin-it.local will be seen as a domain for virtual mailbox addresses. The virtual transport has to be configured that these messages are handled by the pipe-deamon to call an external script.

We'll instruct Postfix also to enrich the smtp-message with two Custom properties:

x-envelope-to: Recipient list from the SMTP-Header
x-envelope-from: From-email-addres from the SMTP Header

Actually we implement the enrichment in the script. But Postfix will pass these properties as parameters.

Create a transport
To have incoming mail for .darwin-it.local transported to the script, a new transport has to be configured.

This is done in the master.cf file:

# 2010-02-10, M. van den Akker: Setup transport routescript for passing message to a bash-script
routescript   unix  -       n       n       -       -       pipe
flags=FDq. user=smtpuser argv=/bin/bash -c /home/smtpuser/routescript.sh -s $sender -r $recipient -q $nexthop -

Here a new transport is created, named “routescript”. It refers to the “pipe” deamon.

The “flags=FDq” denote that the From and destination addresses from the header are added to the header.
Change the following if necessary:

The user denotes the unix-user that is used to run the script. In this case the unix-user “smtpuser” is used. Change it to the proper user (other than “root” or “postfix” ) .
argv: denotes in this case that a bash script is called. Bash is called to run the routescript.sh script, that is placed in the home folder of the smtpuser. Change the path according to the correct location of the given script.
The next parameters are parameters for the routemq.sh script:
- -s: the sender of the message, $sender refers to the from-address on the smtp-envelope
- -r: the recipient(s) of the message, $recipient refers to the to-list on the smtp-envelope
- -q: the queue on which the message has to be put. Our script was designed to queue the message on IBM mq. The property $nexthop refers to the nexthop property in the transport map. Having used this parameter to denote the particular queue enables us to change only the transport map if a queue is changed. Or reuse this transport for different queues in different transport mappings. This is a nice way to pass info about the actual/technical transport channel to be used.

Create transport map
To route the '.darwin-it.local' domain to the script a transport map has to be made in the transport file.

Add the following line to the transport map file:

# 2010-02-10, M. van den Akker: Setup transport routescript for handling darwin-it.local-messages.
.darwin-it.local routescript:queuename

Where queuename is the queue that is used for the smtp-messages. This is the 'nexthop'-parameter where the $nexthop property in the master.cf refers to.

After having changed the transport map file, it has to be compiled into an indexed binary using the postmap tool. So execute the following command:

[root@vmsmtpserver postfix]# postmap transport

Define transport map
To have the transport map used by Postfix, add the following lines to the main.cf :

# 2010-02-10, M. van den Akker: use transport map
transport_maps=hash:/etc/postfix/transport

Routescript
Make sure that the routescript.sh as given in the appendix is placed on the location as defined in the master.cf above. In the example above it is: /home/smtpuser/routescript.sh
Make it owned by the user that is used by Postfix to execute the script (smtpuser).
Then make the script executable:

[smtpuser@vmsmtpserver postfix]$ chmod +x routescript.sh

The actual script as an example is given at the end of this article.

Outgoing e-Mail
All outgoing e-mail should be forwarded to your companies or ISP's smpt-infrastructure.

To do so set the relay-host parameter to the particular smtp-server in main.cf:

# INTERNET OR INTRANET

# The relayhost parameter specifies the default host to send mail to
# when no entry is matched in the optional transport(5) table. When
# no relayhost is given, mail is routed directly to the destination.
#
# On an intranet, specify the organizational domain name. If your
# internal DNS uses no MX records, specify the name of the intranet
# gateway host instead.
#
# In the case of SMTP, specify a domain, host, host:port, [host]:port,
# [address] or [address]:port; the form [host] turns off MX lookups.
#
# If you're connected via UUCP, see also the default_transport parameter.
#
#relayhost = $mydomain
#relayhost = [gateway.my.domain]
#relayhost = [mailserver.isp.tld]
#relayhost = uucphost
#relayhost = [an.ip.add.ress]

Restart Postfix
After having made the necessary changes above it is important to restart (stop and sart) Postfix. Just doing a reload of the config will probably not suffice.

Conclusion
It took me some time to understand Postfix. I was quite overwhelmed by the options. And it took me some time to figure out how to configure it for this particular usecase. Where I had to consult a co-worker (the one that sort of made up this use-case; thanks to Hugo). But as with most other things: after all it turns out to be simple. And it might be useful for many other cases.

Appendix: the routescript.sh
Below the script is given to be called by Postfix to route the messages. This script is designed to either output the message to a file or to an IBM mq client. The mq-client has to be installed from a licensed installment of IBM. IBM also provides the MA01-support pack, in which for several OS'es an compiled executable is provided. This executable (simply called 'q') provides a commandline interface to the IBM mq-client. The mq-client can put the message to a queue, but also to standard out. This is handy for testing, where no mq-client is available.

There are 2 lines to edit:
• Q=~/bin/ma01/q : give here the proper path the q-executable from the MA01-support pack
• fi|"$Q" -O "$QUEUE_NAME" : here the output of the if-block is outputted to the q-client using the queue-name. If the queue is not available in test-environment this will give an error. For test purposes it might be usefull to comment this line use (uncomment) either the line '#fi|"$Q" -s' to have the q-client output the message to STDOUT, or '#fi>$FILENAME' to have the output directed to file.

The script will exit with the result code of the last command, which is the q-client. If that one fails, the script with exit with the result-code telling Postfix to consider the call failed. Postfix will consider the message undelivered and retry it later.

#!/bin/bash
###################################################################################################
# Route messages.
# Script to route an email message, read from pipe and output it to a channel.
#
# author: Martien van den Akker
# (C) januari 2010
# Darwin-IT Professionals
###################################################################################################
#Declartions
E_WRONG_ARGS=85;
NUMBER_OF_EXPECTED_ARGS=6;
FILENAME=/tmp/routemq-`date +%Y%m%d-%H%M%S.%N`
Q=~/bin/ma01/q
TRUE=1
HDR_ENV_FROM="x-envelope-from"
HDR_ENV_TO="x-envelope-to"

#Function to display usage
usage(){
SCRIPT_PARAMETERS="-r receiver -s sender -q queuename";
USAGE="Usage: `basename $0` $SCRIPT_PARAMETERS";
echo $USAGE;
}

# Check Arguments
if [ $# -lt $NUMBER_OF_EXPECTED_ARGS ]
then
usage
exit $E_WRONG_ARGS;
fi
until [ -z "$1" ]
do
case $1 in
"-s") SENDER=$2;;
"-r") RECEIVER=$2;;
"-q") QUEUE_NAME=$2;;
* ) usage;
exit $E_WRONG_ARGS;;
esac;
shift 2;
done;

#echo header variables and cat stdin to output.
#Edit next line to give the proper full path to the “q”-executable from the MA01-support-pack
if [ "$TRUE" ]
then
echo "$HDR_ENV_FROM: $SENDER"
echo "$HDR_ENV_TO: $RECEIVER"
cat -
fi|"$Q"  -O "$QUEUE_NAME" #  uncomment to output to $QUEUENAME (Comment previous line)
#fi|"$Q" -s # only output to stdout
#fi>$FILENAME #uncomment to output to filename

exit $? #Exit with result-code of last command, which is the "q" command.

Apache James on IBM AIX

2010-03-10T00:23:00.001-08:00

Apache James is a very nice e-mail server to be used in a development or test environment, where you need to integrate with an email system. I mentioned it in an earlier post. It supports smtp, pop, nntp (news) and imap.
It's installation is as simple as can be: just unzip the tool, set your JAVA_HOME environment variable and run the appropriate run.sh or run.bat script (given your OS being either Unix/Linux or Windows). The only thing you need besides the zip is a Java Runtime Environment that is at least of version 1.4.2.

For most systems this is all you have to do to get it running. But I found that on IBM AIX (5.x) it is a little less obvious. Getting it running is not an issue, but as soon as you want to add a user, you'll run into the error:

Exception: Security error: java.security.NoSuchAlgorithmException: SHA MessageDigest not available

And after that the telnet connection is closed.
It turns out that the security-provider packages are not registered properly. To get it right there are two things to do.

Make sure that the JAVA_HOME is pointing to the jre folder in the root folder of the java-installment on your system. So like: /usr/java5_64/jre instead of /usr/java5_64. Also make sure that there is a lib/ext folder (/usr/java5_64/jre/lib/ext) that contains a <make>jceprovider.jar, eg. sunjce_provider.jar or ibmjceprovider.jar.
Change <james-home>/bin/phoenix.sh to register the extensions:

Find the line:

JVM_EXT_DIRS="$PHOENIX_HOME/lib:$PHOENIX_HOME/tools/lib"

Change it to:

JVM_EXT_DIRS="$PHOENIX_HOME/lib:$JAVA_HOME/lib/ext:$PHOENIX_HOME/tools/lib"

Now james can be started using the run.sh/run.bat scripts and using the telnet console you should be able to succesfully add users.

It might be that on your system, the system administrators block port 25(smtp) and 110 (pop). That would prevent james to startup the smtp and pop services.
In the <james-home>/apps/james/SAR-INF/ there is a config.xml file. In that file you can find a line:

<pop3server enabled="true">

There you can choose to disable pop by changing the enabled attribute. But benaath that line there is a port element. You could change that instead to for example 8110. That would enable pop-support on the 8110 port. You should instruct your client to use that port off course.
The same counts for smtp-support. That can be found at:

<smtpserver enabled="true">

For smtp you could choose to set the port to 8025.

Oracle Sun now definitely one team

2010-01-28T01:14:00.000-08:00

Today I read the news that the European Commity approved the Sun acquisition without further demands. See webwereld.nl. Actually it is already a week ago. But today I read also the plans of a Web Open Office, or Open Web Office. Or Open Cloud Office. An alternative of Google's Docs. I'm very curious how that would look like and how it will compare to Google Docs. Open Office is a fairly mature Office Suite. So I'm looking forward to see which OpenOffice features will find their way in to the Oracle Cloud.

Further more the clouds are cleared upon MySql.

Another interesting statement I found was that Java SE should support more languages. Probably as an answer to the .Net world where you can develop in multiple languages. So lots of promisses that makes it interesting to "keep an eye in the sail" with Oracle (famous Dutch saying).

Application Server Connection in JDeveloper 10.1.3.x

2010-01-18T06:38:00.000-08:00

This is actually a tip from the dusty old box, although I imagine that there are still people around with SoaSuite/Jdeveloper 10g. I got a VMware image with SoaSuite10133. And I couldn't connect to it with JDeveloper from outside the VM. And I should have the solution somewhere but I couldn't reproduce it. The solution lies in the opmn.xml found in $ORACLE_HOME/opmn/conf.

There you'll find a notification-server element. It propbably does not have the ipaddr node below it by default. But add it like below with 0.0.0.0 as ip-addresses. Doing so enables you to connect with a jdeveloper on a remote machine.

<notification-server inter>
<ipaddr remote="0.0.0.0" request="0.0.0.0"/>
<port local="6100" remote="6200" request="6003"/>
<ssl enabled="true" wallet-file="$ORACLE_HOME/opmn/conf/ssl.wlt/default"/>
</notification-server>

I wrote about connecting to an Oracle Application Server/SoaSuite within a VM in greater detail here, but I did not include the actual opmn.xml snippet like above.

It's back: the Business Event System

2010-01-15T00:07:00.001-08:00

My first steps on integration were on Oracle AQ (from Oracle 8i onwards) and Oracle Workflow Standalone 2.6.x. One of my big dissapointments were that from Oracle 10g AS and DB, Oracle Workflow is desupported. You may use it as long as the 10g licenses last (either for database or AS). But then it is End of Life time. In 11g Oracle Workflow is not delivered anymore. And that is quite a pity, since it was quite a good Workflow tool, especially for datasbase centric applications. The nice thing is that it recides and runs completely in the database. Also it has a pretty good Eventing system. You can define events on a web-based UI. Then you can subscribe Oracle Workflow processes, Java-classes or pl/sql rule-functions on an event.

The good thing about business events is that you can make applications really independend from each other. If you have an application and you need that something happens if a certain mutation in your application is done, for example on creation of an order, the order should be interfaced to another application or logged, then you would create a function call in a database trigger, or from your ADF-BC entity-object. The side-affect is that if your called-functionality is invalid, not available or gives errors, then your calling-application would not work. Also if it is a long running process that is called, than it will all happen in the session of the end-user, who must wait until all processing is done.

And that all is wrong. The calling application should only notify an infrastructure component (like a business event system) and should not care about what is done with this notification. Doing so the end-user get's his sesson back immediately. The application won't disfunction because of corrupted subscribing applications. The application has done what it's responsible for.

The business event system then will handle all the subscriptions in the back-ground. And handle all errors, throw them to an error hospital for example.

But unfortunately the Oracle Workflow Business Event System is no more. Although it is still available in Oracle's E-Business Suite. But that is quite large to install for only BES. EBS is leaning heavily on BES, since most mutations on EBS entities have Events defined on which as an EBS-developer you can subscribe your custom-code. Even if there is no Event available for what you need to do, you shouldn't code it directly on a trigger, but let a database trigger raise your newly defined custom event. And subscribe your code on that event.

This week I delivered the OPN Bootcamp on SoaSuite11g. It was very nice and I'm looking forward to the next to come (9-11th february in Belgium). There's already written a lot on SoaSuite 11g, but little on the comeback of BES: the Event Delivery Network.
Although I like the SCA/Composite. Although I like the integration of all components. But if I must choose, I would pronounce the EDN as the most interesting and promissing addition.

In the OPN Bootcamp there is a small chapter with a little lab to smell just a little on EDN. See also the chapter in "Getting Started with SoaSuite11g". A larger explanation is found here. And here you can read about the managing of EDN.

Raising/publishing an event from a mediator is really easy. Just a few clicks. Also subscribing on a event is just a few clicks away. Also you can raise/publish events from ADF-BC or from Java. But what I miss is an explanation on how to raise an event from Pl/Sql. It is mentioned that it is possible. And in the managing-guide, you'll see how you can create database-agents. But I have to find out how to do it from Pl/Sql. I'll post it when I've found out the how-to.

I've great expectations on EDN. I hope and expect that Oracle will expand the useability further more with enabling other technologies to be able to publish and subscribe to events, the propagation of events to remote systems over WANs, etc. I think they'll do because they need a good Event System when for example BPEL and/or BPM is to replace Oracle Workflow in E-Business Suite.

But we as SOA/EDA consultants have to train Business Analists to think in events. EDN is targetted to the Business. So they have to embrace the concept of events.

Failed to compile the generated BPEL classes

2010-01-06T07:04:00.000-08:00

Today I ran in a weird problem, one moment I could compile my bpel process, a minor change later I couldn't. I got the very descriptive error (not):

Error: Failed to compile classes. Failed to compile the generated BPEL classes for %process-name%.

After a little trial and error I ran a expression in a while loop:

condition="bpws:getVariableData('EKDSuccess')=&quot;false&quot; and bpws:getVariableData('EKDTryCount')<=bpws:getVariableData('EKDMaxRetries')"

It turned out to be the quotes around the word "false". I changed them to single-quotes, like:

condition="bpws:getVariableData('EKDSuccess')='false' and bpws:getVariableData('EKDTryCount')<=bpws:getVariableData('EKDMaxRetries')"

And then it compiled. It was in 10.1.2 (old I know). So maybe in 10.1.3.x it is solved.

Passing parameters to an XSLT in BPEL

2010-01-04T23:28:00.000-08:00

Yesterday for me it became handy to be able to pass parameters to an XSLT in BPEL. I've seen the need earlier, but solved it a different way. By setting the target element with a default and then over writing that default in a later assign-copy step.

But yesterday I had to transform a document a variable number of times, for a list of elements. In my case I had a document with a list of recipients and I had to transform that to another document for each recipient. Each recipient had a number of elements with personal and address information that had to be transformed to the target. So simply defaulting and overwriting would not work, or was a lot of work.

To pass parameters as arguments to a XSLT is quite easy and neatly described in several blogs, amongst others in Sudheer Dhurjati's Blog.

In my case I had to pass an index to be able to select the particular recipient that I need to transform. And another parameter that holds a number of documents that is determined from another source:

<?xml version="1.0" encoding="UTF-8" ?>
<parameters xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://schemas.oracle.com/service/bpel/common /S:/DEV/Sources/BPEL/Processes/CRMI_COM_VersturenBerichtContactMgt/1.0/src/xsltparameters.xsd"
xmlns="http://schemas.oracle.com/service/bpel/common">
<item>
<name>AantalBijlagen</name>
<value>2</value>
</item>
<item>
<name>OntvangerIndex</name>
<value>1</value>
</item>
</parameters>

In Sudheer's blog (and in other examples) the parameters are initialized by copying an XML fragment with the particular parameters. If you have to change one of the parameters this could be done by doing an indexed xpath-expression in the <to> of the assign step:
[/prm:parameters/prm:item[prm:name='OntvangerIndex']/prm:value]
with [http://schemas.oracle.com/service/bpel/common] as [prm]
<value xmlns="http://schemas.oracle.com/service/bpel/common">1</value>

But for flexibilities sake, I would propose a slightly different approach. And for that I need an adapted version of the XSD:

<?xml version="1.0" encoding="windows-1252" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:bplcmn="http://schemas.oracle.com/service/bpel/common"
xmlns="http://schemas.oracle.com/service/bpel/common"
targetNamespace="http://schemas.oracle.com/service/bpel/common"
elementFormDefault="qualified">
<xsd:element name="parameters">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="bplcmn:item" minOccurs="1" maxOccurs="unbounded"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="item" type="bplcmn:itemType"/>
<xsd:complexType name="itemType">
<xsd:sequence>
<xsd:element name="name" type="xsd:string"/>
<xsd:element name="value" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

In this XSD I created 'item' as a seperate element based on a seperated (named) complex-type. I'm actually not so fond of nested complex-types, because it prevents you from using elements lower in the hierarchy for seperate variables.
Using this XSD, you can create a seperate item-variable, based on the item-element.
This is item-variable can be filled with a name and value, just by copying to the particular elements.
The item variable has then to be added to the parameters node using the addChildNode function:

ora:addChildNode(bpws:getVariableData('XsltParams','/bplcmn:parameters'),bpws:getVariableData('item','/bplcmn:item'))

The first argument of the addChildNode function denotes the element under which you want to add a child. In the expression above this is the 'parameters' element in the XsltParams variable. The second argument is the node you want to add as a child, in this case the 'item'-variable. Of course this expression is the <from>-expression in the copy-rule of the assignment step. The <to> will be most of the times the parent element used in the addChildNode expression:

<to variable="XsltParams" query="/bplcmn:parameters"/>

I found the description of the addChildNode function in the expression builder not so clear. So this might also be helpfull for other situations where you have to build up a node structure dynamically.

JDeveloper 11g and SoaSuite 11g: a tight combination

2009-12-22T01:31:00.000-08:00

I remember the first release of BPEL Process Manager in the Oracle Application Server architecture. It was 10.1.2. Before that there was a first global available version (I believe it was called version 2.0). The 10.1.2 release came with a developer install with a very specific version of JDeveloper with the BPEL designer. From 10.1.3 and onwards, the SoaSuite (BPEL,ESB) designers were add-ons. The idea was that with every JDeveloper you could perform SoaSuite-development. But the advice to keep up the JDeveloper version with the SoaSuite version was kept (use JDeveloper 10.1.3.5 with SoaSuite 10.1.3.5).

This week I started preparing the OPN Bootcamp on SoaSuite 11g, that I'm to give in january (from 12th to 14th in De Meern, The Netherlands, see here). I got a prepared Virtual Machine with SoaSuite 11g installed. But it is a one-cpu 2GB VM (that I could give it more memory of course), and I would like to have my sources outside the VM. So for performance and development convenience I choose to install a seperate JDeveloper.

Of course I went for the latest version (11.1.1.2, downloaded here). But I got connection failures at configuring the Application Server connection. Struggled with it for a few hours (in between struggling with another problem, though), disabling firewalls, googling around. But no clues.

This morning with fresh curage I thought:lets not be too stubborn and try the suggested version of JDeveloper (11.1.1.1). The Soa-Infra version was: v11.1.1.0.0 - 11.1.1.1.0, according to the soa-infra page. The great thing about Jdeveloper 11g is that the studio-download is about 1GB. And having installed it, you have to download the SOA Composite editor separately via the Help>Update-functionality. It's about an addtional 230MB. And I found JDeveloper 10.1.3.x with about 700MB large already (Soa-designers included)!

But turns out to solve the problem. So no backwards-compatibility. Tight version coupling.
So if you run into connection problems between Jdeveloper 11g and SoaSuite, just check the version numbers.

Oh, and if you like to join the OPN Bootcamp in january, check out the enablement pages.
The course can be given in English if required.

Multi Operations BPEL

2009-12-11T00:05:00.000-08:00

Invocation flavours
Most BPEL processes have just one entry and one exit point. Actually, if you create a new BPEL process you have the following templates:

Synchronous
Asynchronous
Empty

Empty you use when you want to start your BPEL process using an Adapter (File, Database, AQ, MQ, etc.). But often developers choose asynchronous and then figure they need to start the process using an adapter but leave the originally generated WSDL. The side effect might be that another developer might try to invoke the process/service using the original WSDL that is in fact not implemented anymore.

For the two other templates you can have the following service flavours:

Synchronous (Request/Reply)
Asynchronous Request/Reply
Fire&Forget

Fire&Forget is about the same as Asynchronous Request/Reply. Often the terms and their meanings are mixed up. One says Asynchronous but means Fire&Forget. The distinguise is in the fact that with an Asynchronous process you expect an answer and with Fire and Forget you don't. Most of the times (I did that in the past as well) one builds an asynchronous process but does not put a corresponding receive in the calling process. There is little wrong with that. However if the developer even deletes the invoke to call back to the caller, but another developer does put in a receive (it is in fact an asynchronous process, right?) then it waits for a callback that never comes. So I strongly advice in to distinguise between Asynchronous and Fire&Forget, by removing the call-back partnerlink-type and porttype from the WSDL. And so delete the corresponding call-back-invoke from the process.

Normally there are two ways for invoking these process, corresponding to the type. The "process"-operation for Synchronous and the "initiate"-operation for the Asynchronous/Fire&Forget ones.

So that's about it on the invocation flavours, right?

Multiple Operations
Well, not really. There are certain cases that you need to invoke your process in different ways. In the past I build processes that are initiated by one Operation (the initiate for Asynchronous), but I needed to send signals to the running process. For example, I build a schedule process that is initiated to wait for a certain time to do a certain task, based on information in the database. And if that is changed by an end-user, because the scheduled date/time is moved, you want to reschedule the process to have it wait for the new time.
Another reason may be that you have to impelement a certain WSDL that is agreed upon (top-down-method).

You do that by adding a new operation to the port-type in the WSDL. This new operation can have other message-types for input and output then the initiate operation.

For example, if you create a new process, like the Asynchronous MultiOperationProcess, the WSDL will look like:

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="MultiOperationProcess"
targetNamespace="http://xmlns.oracle.com/MultiOperationProcess"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:client="http://xmlns.oracle.com/MultiOperationProcess"
xmlns:plnk="http://schemas.xmlsoap.org/ws/2003/05/partner-link/">

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TYPE DEFINITION - List of services participating in this BPEL process
The default output of the BPEL designer uses strings as input and
output to the BPEL Process. But you can define or import any XML
Schema type and use them as part of the message types.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://xmlns.oracle.com/MultiOperationProcess" schemaLocation="MultiOperationProcess.xsd" />
</schema>
</types>

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MESSAGE TYPE DEFINITION - Definition of the message types used as
part of the port type defintions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<message name="MultiOperationProcessRequestMessage">
<part name="payload" element="client:MultiOperationProcessProcessRequest"/>
</message>

<message name="MultiOperationProcessResponseMessage">
<part name="payload" element="client:MultiOperationProcessProcessResponse"/>
</message>


<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PORT TYPE DEFINITION - A port type groups a set of operations into
a logical service unit.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- portType implemented by the MultiOperationProcess BPEL process -->
<portType name="MultiOperationProcess">
<operation name="initiate">
<input message="client:MultiOperationProcessRequestMessage"/>
</operation>
</portType>

<!-- portType implemented by the requester of MultiOperationProcess BPEL process
for asynchronous callback purposes
-->
<portType name="MultiOperationProcessCallback">
<operation name="onResult">
<input message="client:MultiOperationProcessResponseMessage"/>
</operation>
</portType>

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PARTNER LINK TYPE DEFINITION
the MultiOperationProcess partnerLinkType binds the provider and
requester portType into an asynchronous conversation.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<plnk:partnerLinkType name="MultiOperationProcess">
<plnk:role name="MultiOperationProcessProvider">
<plnk:portType name="client:MultiOperationProcess"/>
</plnk:role>
<plnk:role name="MultiOperationProcessRequester">
<plnk:portType name="client:MultiOperationProcessCallback"/>
</plnk:role>
</plnk:partnerLinkType>
</definitions>

Adding an operation is simple:

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="MultiOperationProcess" targetNamespace="http://xmlns.oracle.com/MultiOperationProcess"
xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:client="http://xmlns.oracle.com/MultiOperationProcess"
xmlns:plnk="http://schemas.xmlsoap.org/ws/2003/05/partner-link/">
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TYPE DEFINITION - List of services participating in this BPEL process
The default output of the BPEL designer uses strings as input and
output to the BPEL Process. But you can define or import any XML
Schema type and use them as part of the message types.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://xmlns.oracle.com/MultiOperationProcess" schemaLocation="MultiOperationProcess.xsd"/>
</schema>
</types>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MESSAGE TYPE DEFINITION - Definition of the message types used as
part of the port type defintions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<message name="MultiOperationProcessRequestAapMessage">
<part name="payload" element="client:MultiOperationProcessProcessAapRequest"/>
</message>
<message name="MultiOperationProcessRequestNootMessage">
<part name="payload" element="client:MultiOperationProcessProcessNootRequest"/>
</message>
<message name="MultiOperationProcessResponseMessage">
<part name="payload" element="client:MultiOperationProcessProcessResponse"/>
</message>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PORT TYPE DEFINITION - A port type groups a set of operations into
a logical service unit.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- portType implemented by the MultiOperationProcess BPEL process -->
<portType name="MultiOperationProcess">
<operation name="initiateAap">
<input message="client:MultiOperationProcessRequestAapMessage"/>
</operation>
<operation name="initiateNoot">
<input message="client:MultiOperationProcessRequestNootMessage"/>
</operation>
</portType>
<!-- portType implemented by the requester of MultiOperationProcess BPEL process
for asynchronous callback purposes
-->
<portType name="MultiOperationProcessCallback">
<operation name="onResult">
<input message="client:MultiOperationProcessResponseMessage"/>
</operation>
</portType>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PARTNER LINK TYPE DEFINITION
the MultiOperationProcess partnerLinkType binds the provider and
requester portType into an asynchronous conversation.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<plnk:partnerLinkType name="MultiOperationProcess">
<plnk:role name="MultiOperationProcessProvider">
<plnk:portType name="client:MultiOperationProcess"/>
</plnk:role>
<plnk:role name="MultiOperationProcessRequester">
<plnk:portType name="client:MultiOperationProcessCallback"/>
</plnk:role>
</plnk:partnerLinkType>
</definitions>

You can even change an existing operation. But then you'll have to change the initial receive:Now you can add another Receive to cater for the receive events of the other porttype. Of course you need a Correlation Set to make sure that the event is sent to the right instance of the process.

We just added an operation to the Port Type. But you can also add another PortType. That means however also another Partnerlink type:

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="MultiOperationProcess" targetNamespace="http://xmlns.oracle.com/MultiOperationProcess"
xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:client="http://xmlns.oracle.com/MultiOperationProcess"
xmlns:plnk="http://schemas.xmlsoap.org/ws/2003/05/partner-link/">
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TYPE DEFINITION - List of services participating in this BPEL process
The default output of the BPEL designer uses strings as input and
output to the BPEL Process. But you can define or import any XML
Schema type and use them as part of the message types.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<types>
<schema xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://xmlns.oracle.com/MultiOperationProcess" schemaLocation="MultiOperationProcess.xsd"/>
</schema>
</types>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MESSAGE TYPE DEFINITION - Definition of the message types used as
part of the port type defintions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<message name="MultiOperationProcessRequestAapMessage">
<part name="payload" element="client:MultiOperationProcessProcessAapRequest"/>
</message>
<message name="MultiOperationProcessRequestNootMessage">
<part name="payload" element="client:MultiOperationProcessProcessNootRequest"/>
</message>
<message name="MultiOperationProcessRequestMiesMessage">
<part name="payload" element="client:MultiOperationProcessProcessMiesRequest"/>
</message>
<message name="MultiOperationProcessResponseMessage">
<part name="payload" element="client:MultiOperationProcessProcessResponse"/>
</message>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PORT TYPE DEFINITION - A port type groups a set of operations into
a logical service unit.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- portType implemented by the MultiOperationProcess BPEL process -->
<portType name="MultiOperationProcess">
<operation name="initiateAap">
<input message="client:MultiOperationProcessRequestAapMessage"/>
</operation>
<operation name="initiateNoot">
<input message="client:MultiOperationProcessRequestNootMessage"/>
</operation>
</portType>
<portType name="MultiOperationProcessMies">
<operation name="initiateMies">
<input message="client:MultiOperationProcessRequestMiesMessage"/>
</operation>
</portType>
<!-- portType implemented by the requester of MultiOperationProcess BPEL process
for asynchronous callback purposes
-->
<portType name="MultiOperationProcessCallback">
<operation name="onResult">
<input message="client:MultiOperationProcessResponseMessage"/>
</operation>
</portType>
<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PARTNER LINK TYPE DEFINITION
the MultiOperationProcess partnerLinkType binds the provider and
requester portType into an asynchronous conversation.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<plnk:partnerLinkType name="MultiOperationProcess">
<plnk:role name="MultiOperationProcessProvider">
<plnk:portType name="client:MultiOperationProcess"/>
</plnk:role>

<plnk:role name="MultiOperationProcessRequester">
<plnk:portType name="client:MultiOperationProcessCallback"/>
</plnk:role>
</plnk:partnerLinkType>
<plnk:partnerLinkType name="MultiOperationProcessMies">
<plnk:role name="MultiOperationProcessProviderMies">
<plnk:portType name="client:MultiOperationProcessMies"/>
</plnk:role></plnk:partnerLinkType>
</definitions>

Having this you need another partnerlink in your process to get events from it:

The corresponding XSD is:

<schema attributeFormDefault="unqualified" elementFormDefault="qualified"
targetNamespace="http://xmlns.oracle.com/MultiOperationProcess" xmlns="http://www.w3.org/2001/XMLSchema">
<element name="MultiOperationProcessProcessAapRequest">
<complexType>
<sequence>
<element name="inputAap" type="string"/>
</sequence>
</complexType>
</element>
<element name="MultiOperationProcessProcessNootRequest">
<complexType>
<sequence>
<element name="inputNoot" type="string"/>
</sequence>
</complexType>
</element>
<element name="MultiOperationProcessProcessMiesRequest">
<complexType>
<sequence>
<element name="inputMies" type="string"/>
</sequence>
</complexType>
</element>
<element name="MultiOperationProcessProcessResponse">
<complexType>
<sequence>
<element name="result" type="string"/>
</sequence>
</complexType>
</element>
</schema>

Now you can implement about any WSDL with BPEL.

Multi Initiate
A BPEL process can have only one Receive with the initiate checkbox checked. So there is just one and only one entry point in the BPEL process. This should also be the very first activity in the process. Actually it isn't. That is, if you have implemented an exception handler (Catch/CatchAll) on the outermost-scope. Then the activities in the fault-handlers are the first in the BPEL process. And I found that there is a little pitfall in it. It might be that you have an exception handling bpel-process that you call from the catch-branches. No worries if it is a logging process (Fire&Forget). But if it returns a value to determine if you have to do a retry or an abort, then you have a Receive activity before the first Initiate-Receive!

But apparently you can not initiate your BPEL process from the other operations we introduced before. And that makes it only usefull in cases where you initiate your bpel process in one way and have it receive messages over the other operations. If you have to implement a certain WSDL with multiple operations that have to initiate the process: "Houston we have a problem".

Until yesterday this is what I believed as well. But I under-estimated the use of the Pick activity!
The Pick activity splits up the initiation of the process from the messages it listens to:
You can then remove the Timer Branch and add extra Branches. For Aap (Ape):

For Noot (Nut)

For Mies:
And in the branches you can for example add an assignment from the corresponding message:
If you do that for each of the branches, the example process will look like:

Don't forget to remove the original InputMessage-variable.

Unfortunately the BPEL Console does not support the extra porttype for the test screen. I haven't been able to test it with SoapUI yet. So I'm not sure if that works. Deleting the ClientMies parts of the process will work:

That leaves us with this process (belonging with the second WSDL above):

Conclusion
Now you can implement about any WSDL using BPEL, although apparently you're contracted to having one partnerlink-type and one port-type with multiple operations. For me it was a revelation to be able to initiate your process in multiple ways. It might come in handy when you have a case that has some different initiation message but a common process-part. Normally you would create seperate processes to receive the different messages and have them transform them to a common message-format and call the common process. But apparently we can do it all in one process too.

You can find my code-examples here.

BPEL Partnerlinks thoughts

2009-11-30T00:23:00.000-08:00

Today I have a support session with a colleague. One of the questions he came up with was about BPEL-partnerlinks. In his current project they have BPEL Partnerlinks to integrate with Pl/Sql procedures on E-Business Suite. They tend to introduce a partnerlink for every invoke of a Pl/Sql procedure. So how about reuse of partnerlinks?

Well at first thought the answer is simple: there is no one-to-one relationship between a partnerlink and invokes of that partnerlink. So you can introduce as many invokes on a certain partnerlink as you want.
However at second thought there are a few considerations. I'll give a few that I came up with:

Invokes preferably are done in seperate scopes. That is: I prefer to do so. That provides the possibility to catch exceptions on that scope specifically. Variables are kept local as much as possible. Escpecially when the documents they hold can be large. At least you should think about the scope of variables. So if the invoke's input and output variabables are locally declared then they cannot be reused amongst invokes.
If the input and output variables of the invokes are local and thus not shared, you need to build them up from scratch. This can introduce extra assign steps that might be avoided when declare the variables in a larger spanning scope. Extra assign activities with copy steps cost time. A little perhaps, but still they do. Shared variables are stored to dehydration store over and over again.
A shared partnerlink has thus shared MCF (managed connection factory)-properties. And possibly other adapter-related properties. Normally you probably want that. So in most cases it is advisable to share partnerlinks escpecially on adapters. Then you have to configure them only once. However in some cases you might (for instance with queues) use other parameters. Maybe you have two EBS-instances and a process calling the same procedure but on seperate instances. Then you need to split them up of course.
Be aware of transactions. The database adapter normally uses connection-pooled datasources. Database sessions are shared amongst instances. You might expect (but I won't rely on it!) that if you share a database-adapter-partnerlink (invoke it multiple times) in a synchronous process, that every invoke would use the same database-session. So for instance your application context settings, package variables, etc. are shared. Although I would check on run-time, you can use that for performance purposes. In a synchronous process you may rely on the fact that this occurs in the same database-transaction. However, if you use different partnerlinks for in fact the same Pl/Sql procedure/function call, the chance is large that you get another database-session. BPEL might need to rely on XA-transaction mechanisms. I don't think this is functionally a big problem. But might cause issues on processes with high performance requirements.
Maintenance: having multple partnerlinks for the same database procedure or even the same purpose introduces extra maintenance cost and a higher risk. I would not expect that a BPEL-project would have multiple partnerlinks for the same purpose. So in a Change Request or Test-Issue I tend to solve/change the first partnerlink and forget about possible others. And why should I be different from fellow developers? A good developer is a lazy one...
Input and Output variables are based on the message-definitions of the partnerlinks-wsdl's. So changing one partnerlink might change the message definitions. If you reuse your partnerlinks, changing them will effect all the related variables.

These are thoughts that crossed my mind in thinking it over. To be sure you need to check by testing. But I do think that in most cases you should declare a partnerlink for a certain purpose only once. In principle that's the best practice. And that will prevent a lot of possible issues. Only re-declare a partnerlink if you have good reasons for it.

Oracle AIA Links

2009-11-25T01:11:00.000-08:00

At the moment I'm delivering the OPN Bootcamp on AIA. One of the questions from the group is on documentation. So I have taken the time (while the students are busy with the labs) to scan around OTN to findout some of the interesting links on AIA.

AIA Best Practice Center: http://www.oracle.com/technology/products/applications/aia/index.html
Oracle Wiki on AIA: http://wiki.oracle.com/page/Application+Integration+Architecture
Official AIA blog: http://blogs.oracle.com/aia/
AIA OTN Forum: http://forums.oracle.com/forums/forum.jspa?forumID=577
OTN AIA Foundiation pack page: http://www.oracle.com/technology/products/applications/aia/aia_fp1_index.html
AIA Foundation Pack Tools: http://www.oracle.com/technology/products/applications/aia/aiatools.html
OTN AIA PIP page: http://www.oracle.com/technology/products/applications/aia/aiapipnew.html

Hope you'll find it usefull as well.

Back to OpenSuse 11.1

2009-11-22T06:36:00.000-08:00

Last week I encountered some troubles with my OpenSuse 11.1 installation. Apparently something in wine and else got corrupt. I had some idea's what might have caused it, but was not sure. Although I found some entries comparable to the errors I had, I could not find a sufficient solution. Since OpenSuse 11.2 turned out to be released recently, I thought: "let's give that one a shot". Everything installed smoothly.

Of course I had to install the native NVidia driver again using the one click install. OpenSuse does not ship the propietry drivers, but does provide a simple install-solution.
The multi-media support (escpecially flash and mp3) for 11.2 is installed via this post.

However after some strubbling I came to the conclusion that VMware Server 2.0.x just won't work under 11.2. The rpm does install, but the vmware-config.pl hits several nasty compile-errors. Apperently the kernel is just too new for VMWare. I found some scripts to repair the install, based on the tar.gz-deployment. But they were mainly focussed on Ubuntu, doing some preparartions using the Ubuntu package-manager. So I re-installed 11.1.
First I tried to upgrade KDE to 4.3, but that did not work well. So the third time I just installed 11.1-vanilla.
A one-click-install for multimedia support voor 11.1 I found here (basically the same blog that posted the one-click-install for 11.2.

A lass: OpenSuse 11.2 worked really fine: my laptop ran like a sun (there is a great commercial on Wind-energy on Dutch television. With an older man that hosts city-trips for foreigners. He talks really Dutch English with litterly translated Dutch sayings on wind. My favorite: "I think You think they smell an hour in the wind"). But to me VMware is a real must.
Especially this week: I'll deliver an OPN Bootcamp on AIA in De Meern, the Netherlands, from tommorow (mentioned dates are wrong: it's this week). It's based on a VMware image with an SoaSuite+AIA installment for the labs. If you're interested: there still are some seats available.

VMware Server 2.0.2 on OpenSuse 11.1

2009-11-16T05:01:00.000-08:00

Today during preparing the OPN Bootcamp on Oracle AIA Foundation Pack, I found out that VMware Server 2.0.2 is out. It has been released on october 26th.
So I downloaded the one for my system (OpenSuse 11.1 x86_64). As well as the windows variant for the bootcamp.

To upgrade an existing installed vmware server you can do (as root):

makker-laptop:/usr/bin # rpm -Uhv VMware-server-2.0.2-203138.x86_64.rpm

To query which VMware Server you have installed:

makker-laptop:/usr/bin # rpm -q VMware-server
VMware-server-2.0.2-203138

To install if you're not having VMware Server:

makker-laptop:/usr/bin # rpm -ihv VMware-server-2.0.2-203138.x86_64.rpm

Then after that you should run vmware-config.pl in /usr/bin folder as root.

In my case something went wrong in installing the vsock module. I did not see that in an earlier install. This is the message I got:

make: Entering directory `/tmp/vmware-config6/vsock-only'
make -C /lib/modules/2.6.27.37-0.1-default/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-2.6.27.37-0.1-obj/x86_64/default'                
make -C ../../../linux-2.6.27.37-0.1 O=/usr/src/linux-2.6.27.37-0.1-obj/x86_64/default/. modules
CC [M]  /tmp/vmware-config6/vsock-only/linux/af_vsock.o                                     
CC [M]  /tmp/vmware-config6/vsock-only/linux/driverLog.o                                    
CC [M]  /tmp/vmware-config6/vsock-only/linux/util.o                                         
CC [M]  /tmp/vmware-config6/vsock-only/linux/vsockAddr.o                                    
LD [M]  /tmp/vmware-config6/vsock-only/vsock.o                                              
Building modules, stage 2.                                                                  
MODPOST 1 modules                                                                           
WARNING: "VMCIDatagram_CreateHnd" [/tmp/vmware-config6/vsock-only/vsock.ko] undefined!        
WARNING: "VMCIDatagram_DestroyHnd" [/tmp/vmware-config6/vsock-only/vsock.ko] undefined!       
WARNING: "VMCI_GetContextID" [/tmp/vmware-config6/vsock-only/vsock.ko] undefined!             
WARNING: "VMCIDatagram_Send" [/tmp/vmware-config6/vsock-only/vsock.ko] undefined!             
CC      /tmp/vmware-config6/vsock-only/vsock.mod.o                                          
LD [M]  /tmp/vmware-config6/vsock-only/vsock.ko                                             
make[1]: Leaving directory `/usr/src/linux-2.6.27.37-0.1-obj/x86_64/default'                  
cp -f vsock.ko ./../vsock.o                                                                   
make: Leaving directory `/tmp/vmware-config6/vsock-only'                                      
Unable to make a vsock module that can be loaded in the running kernel:                       
insmod: error inserting '/tmp/vmware-config6/vsock.o': -1 Unknown symbol in module            
There is probably a slight difference in the kernel configuration between the                 
set of C header files you specified and your running kernel.  You may want to                 
rebuild a kernel based on that directory, or specify another directory.

You might not need it since Vsock is an optional module that is used in communication between guests and hosts. But luckily I found a solution here. Mr. Swerdna states that there is a flaw in the vmware-config.pl, for which he provides a patch. He also explains how to enable USB support in guests. So thank you very much mr. Swerdna.

PartnerlinkType not found

2009-11-10T23:16:00.000-08:00

Yesterday I was struggling in getting a dynamic partnerlink coupling to work in BPEL PM.
I would not say that I invented the principle of dynamic partnerlinks myself. But lets say that it was not my first time. And what I tried and checked, and double checked, I kept hitting the error:


Error while invoking bean "presentation manager": Cannot find partnerLinkType 2.
PartnerLinkType "{http://www.domain.nl/PROJECT/SUBPROJECT/PUB}PROJECT_PUB" is not found in WSDL at
 "http://localhost:7777/orabpel/default/MyBpelProcess/1.0/_MyBpelProcess.wsdl".
Please make sure the partnerLinkType is defined in the WSDL.

ORABPEL-08016

Cannot find partnerLinkType 2.
PartnerLinkType "{http://www.domain.nl/PROJECT/SUBPROJECT/PUB}PROJECT_PUB" is not found in WSDL at
 "http://localhost:7777/orabpel/default/MyBpelProcess/1.0/_MyBpelProcess.wsdl".
Please make sure the partnerLinkType is defined in the WSDL.

At the end I was so desperate that I thought: "Okay then, lets google the error". Afterwards I could hit myself with the question: "why didn't I do this earlier?". Because the answer was simple and can be found (amongst others) here: Just clear the WSDL cache. Apparently my wsdl updates were deployed but not replacing the occurence in the cache. Ouch.

To my defense I would say that dynamic partnerlinks aren't hard but a mistake is made easy and the dependencies are very sensitive. And also I had to copy an existing solution to adapt it to a new situation. And that is harder than build it yourself. But may be in the future I should consider Google as a closer friend...

EU Commitee against Sun-aqcuisition

2009-11-10T22:46:00.001-08:00

As I, honestly, expected the EU Commitee is against the aqcquistition of Sun by Oracle. This can be read (in Dutch) here. Main issue is the position of the MySql database. Oracle wants to keep it, but it would make Oracle not only the biggest in commercial databases but also in OpenSource.
It does impose a significant delay in the acquisition. Oracle expresses that the issues of the European Commitee are based on a misunderstanding on the database and open source marketed.

I'm curious how they see that, because it is not expressed in the quoted article.
Also the American department of Justice made explicitly clear that it approved while they're convinced that MySql does not damage the competition in the database market.

I find it curious though that the acquisition of a commercial company by another commercial company is under investigation/restriction because of a non-commercial product. I understand that the MySql holding company is in fact commercial. But you and I and every one who wants to do it can get a fork of the MySql source and start our own database. Provided that we know C++ (I conveniently assume that MySql is written in C++) , what in my case is quite rusty, and understand the sources.

So I would ask that if Oracle promises that they embrace MySql lovely caring, what would be the issue here then? And EU could just put some preconditions on the take-over. And sanctions if in a later stage Oracle turns out to not meeting those.
But then: I'm not so political. And I'm in a struggle: I'm pretty proud that one of the major forces in these matters on European Level is a Dutch woman, but at the other hand I've a history with Oracle. And I think the take over would be a good one. Or at least an interesting one. For Sun, Oracle and the ICT market.

Yesterday I found that Oracle wants to keep Sun's Glassfish Appserver as a referential implementation of J2EE. I take it with for example the SOAP toolstack Metro. Which is nice, because then Oracle has three app-servers if I'm counting right:

OC4j (originally from Orion), still base of the J2EE parts of E-Business Suite, and probably for years to come.
Weblogic (originally from BEA)
Glassfish (originally from Sun)

I heard from a colleague who attended the DBA-day of the Dutch Oracle User Grouop that Oracle would not go further with VirtualBox. That would be a pity I think. I don't think it means that it ceases to exist. But a desktop virtualization tool would nicely extent Oracle's virtualization portfolio. Oracle might be focussing on the Server market. But then, what about StarOffice/OpenOffice for example?

Well questions enough. And a nice subject to philosophize further upon. But to remember/rephrase a phrase from Mike Oldfields "Songs of a distant earth": "Only time will tell"...

Next Generation SOA

2009-10-29T00:37:00.000-07:00

Last week I was on the SoaSymposium in the WorldTrade Center in Rotterdam. The theme of this years episode was apparently "Next Generation SOA". On before hand I wondered what they meant with it. Apparently I missed the blog of Anne Thomas Mane that stated that SOA is dead.
It reminded me of my own blog post on the subject. In that blog-post I opposed to the statement of soa-experts of a Dutch ICT weeklet, where SOA was declared dead in favour of EDA. Because I could not remember exactly on what article I reacted back then, I got a little anxious to get in touch with her. But the reason that Anne declared SOA dead was because of the hype that rose around the acronym by the vendors, customers and media. The acronym got so loaded with pre-assumptions that it did not stand for what it promised anymore. And so many SOA projects fail because customers think they could buy the magic box of a vendor that solve every problem. Or the magic hat of the Pixar Magician Presto. And of course vendor's would love to sell this to them. If they could... To declare SOA dead as a hype with all its false promises and focus on Service Orientation, I second completely. You can't buy or sell SOA, neither as a "Silver bullet", nor as a "Magic hat". It reminded me on this company that would sell an integration product on a blade-server. Put it in a rack and your ERP's are integrated. Yeah, right. I assume that the actual implementation is less magic.

Thomas Erl and Anne had a session to cast out the "Evil SOA" and welcome the "Good SOA". I liked the music of Mike Oldfield, but after a while it got too pressing and looped through the very first part of Tubular Bells, that it almost cast me out. But it got me the nice Soa Patterns book of Thomas Erl.

I think with Thomas and Anne that Service Orientation is very important and grow in importance. But together with EDA. Well actually, I think EDA is not an architecture on it self. As with Soa and cloud: if we're going to hype this idea we fall in the same faults again. But as stated in my former post, I think that Events and Services are (or should) two thightly coupled ideas. No loose coupling here. You use these ideas to loosely couple your functionalities. But Services are useless without events. And events (either coming through an esb or as a stream) are quite useless without services that process them.

For the rest it was a good event to network. Met a few former colleagues and business-partners. Or people with I had co-acquaintances. I saw a nice presentation of Sander Hoogendoorn and Twan van der Broek on agile SOA projects combined with ERP. I'd like to learn more on that.

VirtualBox: a virtual competitor of VMware?

2009-10-27T11:42:00.001-07:00

I'm a frequent VMware user for years. Most of my setups are done in a Virtual Machine. It's very convenient because with a new host installation, just installing VMware and restoring the virtual machines gets you up and running.

One of the names that comes around on Virtualization on Linux is VirtualBox. I did not pay any attention to it since I was quite happy with VMware. I use VMware Server, since it's free and has about every feature VMware Workstation had in the latest version I used. That was Workstation 5.5 and the only thing Server lacked was Shared Folders.

But products evolve. And Server has become quite big. I found it quite a step to turn from 1.x to 2.0 since the footprint increased about 5 times! From 100MB to about 500MB.

Last week my colleague Erik asked me if I knew VirtualBox. I said I did, that is by name. A colleague on his project advised him to go for VirtualBox. So I thought I might give it a try.
Some of the features of VirtualBox that interested me were:

it's about 40MB! That is very small for such a complete product. I like that. I'm fond of small but feature-rich products, like TotalCommander, Xtrans and IrfanView on Windows. Apparently there are still programmers that go for smart and compact products.
I found articles on internet that it would be faster then VMware. I also found statements that suggest it is slower. But on a simple laptop, performance would be the decision forcing feature.
Seamless mode: VirtualBox can have the applications run in a seperate window along with the native applications on your host. It layers the taskbar on top of the taskbar of your host. Looks neat!
Shared Folders: this was the differating feature between Server and Workstation. I solved that lack by installing a Filezilla Server on the Windows Hosts, or using SFTP over SSH on Linux. But it is handy to have that feature.
It's a client/standalone app: it may seem like a nonsense feature. But VMware Server comes with an Apache Tomcat application server that serves the infrastructure application. That's nice for a server based application, but on a laptop quite over done.
3D Acceleration: this summer I wanted to run a Windows racing game under VMware. It didn't do it. I should try it under VirtualBox with 3D acceleration on.
But the important feature to have it work for me is that it should be able to run or import VMware images. It turns out that it cannot. You have to create a new VirtualBox image. But luckily you can base it on the virtual disks of a VMware image. Below I tell you how.

What I did not know was that Sun Microsystems acquired Innotek last year (2008). And since Oracle is in the run to buy Sun, Oracle gets this Virtualizing tool also. Another crown-jewel I could say. Since Oracle does have a VM tool but it is not a desktop tool. Oracle's VM has to run on the 'bare metal'.

Pre-requisites:

Install VirtualBox: download it from http://www.virtualbox.org/wiki/Downloads. Mark that there is an Open Source and a Closed Source version. Both free, but the Closed Source is the most feature-rich. Importing and exporting is only available in the closed source version. The Open Source is apparently available in many Linux Repositories. But I went for the Closed Source.
De-install VMware tools from the guest os before migrating. VirtualBox can install it's own guest-tools with the own drivers. For windows you can do it from the software pane in the configuration screen. Under linux (rpm based) you can remove it using rpm -e VMwareTools-5.0.0-.i386.rpm. To query the precise version you can do rpm -q VMwareTools

Creation of the VM
Then you can Create a new VirtualMachine in VirtualBox. You can find an how-to with screen-dumps here.
For the harddisks use the existing disks of your VMware image. If the VM is a Windows Guest, the disks are probably IDE disks. You could try to use SATA as an Additional Controller (check the checkbox) and couple them to a SATA channel in order of the disks. Denote the first one bootable. If it is a Linux guest the disks are probably SCSI- LsiLogic, so choose SCSI as additional disk.

After adding the disks and other hardware, you can start it. After starting it you can install the Guest addons. These are necessary to have the appropriate drivers. Under windows, an installer is automatically started. Under Linux a virtual cdrom is mounted. Do a 'su -' to root and start the appropriate '.run' file or do a sudo:

sudo /media/cdrom/VBoxLinuxAdditions-x86.run

In this case it is for a 32-bit Linux. For 64-bit Linux guests it should be the amd64 variant.

After the install you should restart the guest.

Then you can reconfigure your display type (resolution, color-depth) and the network adapters.

Experiences
Well my first experiences are pretty positive. It is sometimes a struggle to get the display and the networking right. It's a pity that you have to recreate the VM instead of importing the VMware image itself. It seems to me that the VMware vmx is convertable to the VirtualBox xml-config file.

I found it a little inconvenient that on clicking of the VM it starts immediately. Sometimes you just want to reconfigure it.

I have to experience the peformance still. So I don't know yet if VirtualBox on this is an advantage. But the Seamless mode is really neat.

JNDI in your Applications

2009-10-08T12:33:00.000-07:00

Lately I had to make several services to be used from an application server, as a webservice or a service used from BPEL.

I find it convenient to first build the application or service as a java-application and test it from jDeveloper and/or using an ant-script. Although I know that it's possible to test an application from the application server (but never did it myself) it surely is more convenient to have it tested from an ordinary java-application. Also it enables you to use the service in other topologies. Actually it's my opinion that for example a Webservice should not be more then an interface or just another exposure of an existing service or application.

If your service need a database then from an application you would just create a connection using a driver, jdbc-connection and username/password. Probably you would get those values from a property-file. But within an Application Server you would use JNDI to get a DataSource and from that retrieve a Connection.

But the methods that are called from your services within the application server should somehow get a connection in a uniform manner. Because, how would you determine if the method is called from a running standalone application or from an Application Server? The preferred approach, as I see it, is to make the application in fact agnostic of it. The retrieval of a database connection should in both cases be the same. So how to achieve this?

The solution that I found is to have the methods retrieve the database connection in all cases from a JNDI Context. To enable this, you should take care of having a DataSource registered upfront, before calling the service-implementing-methods.

JNDI Context
To understand JNDI you could take a look into the JNDI Tutorial.
It all starts with retrieving a JNDI Context. From a context you could retrieve every type of Object that is registered with it. So it not necessarly be a Database connection or DataSource, although I believe this is one of the most obvious usages of JNDI.

To retrieve a JNDI Context I create a helper class:


package nl.darwin-it.crmi.jndi;
/**
* Class providing and initializing JndiContext.
*
* @author Martien van den Akker
* @author Darwin IT Professionals
*/
import java.util.Hashtable;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

import nl.darwin-it.crmi.CrmiBase;

import oracle.jdbc.pool.OracleDataSource;

public class JndiContextProvider extends CrmiBase {
   public static final String DFT_CONTEXT_FACTORY = "com.sun.jndi.fscontext.RefFSContextFactory";
   public static final String DFT = "Default";
   private Context jndiContext = null;

   public JndiContextProvider() {
   }

   /**
    * Get an Initial Context based on the jndiContextFactory. If jndiContextFactory is null
    * the Initial context is fetched from the J2EE environment. Otherwise if jndiContextFactory contains the keyword
    * "Default" (in any case) the DFT_CONTEXT_FACTORY is used as an Initial Context Factory. Otherwise
    * jndiContextFactory is used.
    *
    *
    * @param jndiContextFactory
    * @return
    * @throws NamingException
    */
   public Context getInitialContext() throws NamingException {
       final String methodName = "getInitialContext";
       debug("Start "+methodName);
       Context  ctx = new InitialContext();
       debug("End "+methodName);
       return ctx;
   }

   /**
    * Bind the Oracle Datasource to the JndiName
    * @param ods
    * @param jndiName
    * @throws NamingException
    */
   public void bindOdsWithJNDI(OracleDataSource ods, String jndiName) throws NamingException {
       final String methodName = "bindOdsWithJNDI";
       debug("Start "+methodName);
       //Registering the data source with JNDI
       Context ctx = getJndiContext();
       ctx.bind(jndiName, ods);
       debug("End "+methodName);
   }

   /**
    * Unbind jndiName
    * @param jndiName
    * @throws NamingException
    */
   public void unbindJNDIName(String jndiName) throws NamingException {
       final String methodName = "unbindJNDIName";
       debug("Start "+methodName);
       //UnRegistering the data source with JNDI
       Context ctx = getJndiContext();
       ctx.unbind(jndiName);
       debug("End "+methodName);
   }

   /**
    * Set jndiContext;
    */
   public void setJndiContext(Context jndiContext) {
       this.jndiContext = jndiContext;
   }
   /**
    * Get jndiContext
    */
   public Context getJndiContext() throws NamingException {
   if (jndiContext==null){
       Context context = getInitialContext();
       setJndiContext(context);
   }
       return jndiContext;
   }
}

This package gives a Jndi Context that can be used to retrieve a DataSource kunt ophalen, using JndiContextProvider.getJndiContext().
This method retrieves an InitialContext that is cached in a private variable.
From within the application server the InitialContext is provided by the Application Server.
But when running as a standalone application a default InitialContext is provided. But this one does not have any content. Also it has no Service Provider that can deliver any objects. You could get an InitialContext by providing the InitialContext() constructor with a hash table with the java.naming.factory.initial property set to com.sun.jndi.fscontext.RefFSContextFactory.

The problem is that if you use the HashTable option, the getting of the InitialContext is not the same as within the Application Server. Fortunately if you don't provide the HashTable, the constructor is looking for the jndi.properties file, within the classpath. This file can contain several properties, but for our situation only the following is needed:

java.naming.factory.initial=com.sun.jndi.fscontext.RefFSContextFactory

This Context factory is one of the jndi-provider-factories delivered with the standard JNDI implementation. It is a simple FileSystemContextFactory,a JNDI factory that refers to the filesystem.

Now the only thing you need to do is to bind a DataSource to the Context, right before your call the possible method, for example in the main method of your applicatoin.
That DataSource you should create using the properties from the configuration file.

To create a datasource I created another helper class: JDBCConnectionProvider:


package nl.darwin-it.crmi.jdbc;
/**
 * Class providing JdbcConnections
 * 
 * @author Martien van den Akker
 * @author Darwin IT Professionals
 */
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

import java.util.Hashtable;

import javax.naming.Context;

import javax.naming.InitialContext;

import javax.naming.NamingException;

import javax.naming.NoInitialContextException;

import oracle.jdbc.pool.OracleDataSource;

public abstract class JDBCConnectionProvider {
    public static final String ORCL_NET_TNS_ADMIN = "oracle.net.tns_admin";

    /**
     * Create a database Connection
     * 
     * @param connectString
     * @param userName
     * @param Password
     * @return
     * @throws ClassNotFoundException
     * @throws SQLException
     */
    public static Connection createConnection(String driver, String connectString, String userName, 
                                              String password) throws ClassNotFoundException, SQLException {
        Connection connection = null;
        Class.forName(driver);
        // DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
        // @machineName:port:SID, userid, password
        connection = DriverManager.getConnection(connectString, userName, password);
        return connection;
    }

    /**
     * Create a database Connection after setting TNS_ADMIN property.
     * 
     * @param driver
     * @param tnsNames
     * @param connectString
     * @param userName
     * @param password
     * @return
     * @throws ClassNotFoundException
     * @throws SQLException
     */
    public static Connection createConnection(String driver, String tnsAdmin, String connectString, String userName, 
                                              String password) throws ClassNotFoundException, SQLException {
        if (tnsAdmin != null) {
            System.setProperty(ORCL_NET_TNS_ADMIN, tnsAdmin);
        }
        Connection connection = createConnection(driver, connectString, userName, password);
        return connection;
    }
    /**
     * Create an Oracle DataSource
     * @param driverType
     * @param connectString
     * @param dbUser
     * @param dbPassword
     * @return
     * @throws SQLException
     */
    public static OracleDataSource createDataSource(String driverType, String connectString, String dbUser, 
                                                 String dbPassword) throws SQLException {
        OracleDataSource ods = new OracleDataSource();
        ods.setDriverType(driverType);
        ods.setURL(connectString);
        ods.setUser(dbUser);
        ods.setPassword(dbPassword);
        return ods;
    }
    public static Connection getConnection(Context context, String jndiName) throws NamingException, SQLException {
        //Performing a lookup for a pool-enabled data source registered in JNDI tree
        Connection conn = null;
        OracleDataSource ods = (OracleDataSource)context.lookup(jndiName);
        conn = ods.getConnection();
        return conn;
    }
}

The method createDataSource will create an OracleDataSource. Today with trial&error I discovered that from a BPEL 10.1.2 server, you should get a plain java.sql.DataSource.

Then with JndiContextProvider.bindOdsWithJNDI(OracleDataSource ods, String jndiName) you can bind the DataSource with the Filesystem Context Provider.

Having done that from your implementation code you can get a database connection using JDBCConnectionProvider.getConnection(Context context, String jndiName)

I haven't discovered how yet, but somewhere (in the filesystem I presume) the DataSource is registered. So a subsequent bind will throw an exception. But since you might want to change your connection in your property file, it might be convenient to unbind the DataSource afterwards at the end of the main-method.The method JndiContextProvider.unbindJNDIName(String jndiName) takes care of that.

What took me also quite a long time to figure out is what jar-files you need to get this working when running in a standalone application. Somehow I found quite a lot examples but no list of jar-files. I even understood (wrongly as it turned out) that the base J2SE5 or 6 should deliver these packages.

The jars you need are:

jndi.jar
fscontext.jar
providerutil.jar

You can find them from the BPEL PM 10.1.2 tree and probably also in a Oracle 10.1.3.x Application Server or Weblogic 10.3 (or nowadays 11g). But the JSE ones can be found via
http://java.sun.com/products/jndi/downloads/index.html
There you'll find the following packages:

jndi-1_2_1.zip
fscontext-1_2-beta3.zip

Conclusion
This approach gave me a database connection within the application server or from a standalone application. The name of the JNDI I fetched from a property file still. That way it is configurable from which JNDI source the connection have to be fetched (I don't like hardcodings).
In my search I found that from Oracle 11g onwards you can use the Universal Connection Pool library: http://www.oracle.com/technology/pub/articles/vasiliev-oracle-jdbc.html to have a connection pool from your application.

AdminServer(admin)	RUNNING	OK	7001
UCM_server1	SHUTDOWN		16200
WC_Collaboration	SHUTDOWN		8890
WC_CustomPortal	SHUTDOWN		8892
WC_Portlet	SHUTDOWN		8889
WC_Spaces	SHUTDOWN		8888
WC_Utilities	SHUTDOWN		8891