Years ago I played around with VMWare Unity Mode in VMWare player or VirtualBox's Seemless mode. In those modes you start an application on your virtual machine, but the windows appear as running on your host system. Back in the days I ran OpenSuse on my laptop, and had Windows XP or 7 on a Virtual Machine for those cases I had to run PowerPoint.

I wasn't too enthousiastic about those modes. Of the two I found that VMWare implemented it the most transparent. But it was quite hard to work with multiple screens, and to start the applications using the embedded menu. Now, I didn't use this in a long time, so it might have been improved. But, lately I work with my VM's (mostly Oracle Linux)  using Vagrant most of the time. And I use MobaXterm to connect with them and shortly, I use it to start my Oracle tools from MobaXterm.

With the XWindows Server in MobaXterm, working with either SQLDeveloper or JDeveloper is very convenient. And it will allow me to cleanup even my local installments of SQLDeveloper or JDeveloper.

So, I have been looking for automating some startup and working on the most convenient setup for me. And I found it (for now) in the combination of:

• VirtualBox
• Vagrant
• MobaXTerm 
• And my All time favorite cockpit: TotalCommander

Let's go through them.

VirtualBox

It must have been around 2003 that I got introduced into VMWare Workstation by my former colleague Robert. It was an eyeopener to see that you could run multiple PCs on your laptop, separating different versions of Oracle Products that could even 'talk' to eachother!
Since all is saved in a folder of files, I could do one install and share it with colleagues.

Since Oracle acquired VirtualBox, and because of several direction-changes at VMWare (introduction and revocation of VMWare Server, VMWare player that couldn't create VMs and then it could, and then it couldn't anymore,...), after a while it made sense to me to switch to VirtualBox completely. There was a period of time that I had both on my laptop.

So, for me it is only VirtualBox now and we have come to version 6.0.4 at time of writing. Downloadable from the VirtualBox Download Page. Choose the download for your platform. Installing follows the familiar NNF-Pattern (Next-Next-Finish). And don't forget to download and install the Platform independent Extension Pack. I guess this is piece of cake for you followers.

Vagrant

This is quite new for me, since a bit over a year now. I transferred my main installations in a Vagrant project and it's still work in progress.

I already wrote a bit about my Vagrant solutions. Last few years I worked on scripting my installations of Oracle Products. Vagrant allows me to automate the creation and provisioning of my VM's. Relieving me from the need to keep multiple VMs up to date. Sharing can be both simpler but also harder. Simpler, because I could share my vagrant project and scripts. But also harder, because the provision scripts should be placed in the proper order. I should implement it in a VCS repo. And also the install-binaries should be placed with the proper name in the proper place.

I now have a Software Stage Repository on my second laptop disk, and  separate Vagrant projects. But they all have copies of provision scripts for several products. So, database, java, SOA/BPM QuickStart, Weblogic is duplicated per particular Vagrant Project. I want to split it up in a common provision folder and a Choose&Select approach in my Vagrant Projects in a way that I have a simple Vagrant provisioning in where I can refer to the provisioning of particular products.

Anyway,  starting up and eventual provisioning of a box is simple: just issue the vagrant up command in the folder with the vagrant file is all you have to do. Suspending  a box is done using vagrant suspend.

My respected con-colleague Maarten Smeets wrote quite a bit about Vagrant and lately about a few good tips.

Vagrant has reached version 2.2.4 recently and can be downloaded here. It  also follows the famous NNF-pattern. But although it allows you to choose the install directory, it is very devoted to be installed in the c:\HashiCorp\Vagrant\. I quit trying to force it elsewhere.

MobaXterm

Many of my even respected coworkers stick with their all-time favorite Putty. Putty stays ubiquitous. It is simple, but I always have found it a bit archaic. I thought I once noticed that the support was  terminated. However, just now they just released the stunning version number of 0.71. I don't mean to be sarcastic, and Putty has it's own right of existence. But give MobaXterm a try. It's loaded with nice features, including an SCP client that can follow your SSH session. And as said, also an XWindows Server. So, connect with ssh to your Linux server and run jmc, visualVM, Oracle Universal Installer, Weblogic Configurator, JDeveloper, SQLDeveloper, etc. etc. and the UI will pop-up on your desktop. Also MobaXterm includes cygwin so you're able to run a terminal session on Windows. It even allows you to do ps -ef to show your running windows apps!

I also discovered that really easy way to implement a tunnel with MobaXterm!

You can download it here, in a portable and an installer version. I choose the portable. The free edition includes games (why?) and  a limit number of sessions, macros and tunnels. But the professional edition only costs a few bucks/euros.

TotalCommander

Ever since the introduction of Windows '95, I disliked the Windows Explorer. Luckily I soon discovered Windows Commander, under pressure of Microsoft, renamed to Total Commander. And it's even  a better name, because it's about the second tool I install on a new Windows Desktop. Just after Firefox, to be able to close IE/Edge...

It's my cockpit, allows me to navigate to hot folders quickly, introspect files, navigate through archives, edit them or unpack them, multi-rename files, compare files, etc., etc. I just don't make coffee with it. One of the nice features is the button bar, where you can launch applications. And this is the thing I use for this blog.

Tie it all together

I realize that I overloaded you with sales talk about my favorite tools.

After installing all the tools and having your Vagrant project in place all can be tied together.

I have a project that provisions a VM with an Oracle Database, SQLDeveloper, SOASuite, and BPM QuickStart.

My TotalCommander Toolbar looks like:

You can right-click anywhere in the toolbar to edit it, create new buttons. A 'button' that is left empty (no command) is presented as a separation bar.

Startup & suspend the VM

Let's take a look at the Vagrant SOA Start button:

It's simple: the command is vagrant up and important here is that it should be executed in the folder where the vagrant file resides. I provide a tool tip, and I created an icon file from the Vagrant Logo using my favorite image app IrfanView.

Clicking the button will fire up the VM and potentially provision it. I copied the button to create a button to suspend the VM. The command there is vagrant suspend. But for the rest it is exactly the same.

MobaXterm local terminal

When you start MobaXterm you'll get to:
When you click on the 'Start local terminal' button, you get a shell window running in the user home folder that is presented by the tool:

You can ssh to a remote server from this terminal. Of course you can create a session to a remote server. Doing so for the first time, logging on to the particular user, allows you to save the password for that user. I've already done that, and then I can do a ssh oracle without the need to provide a password.

Start Database

On my remote server that is started already (using the TotalCommander Button), I have a script that starts the database.

When the VM is started with vagrant, by default it fires up an ssh deamon on port 2222. The command to start my database on the remote server is:

ssh oracle@localhost -p 2222  /home/oracle/bin/startDB.sh

And as you can see in the screendump: I put that in a script in the home folder.

MobaXterm provides several commandline options, that allows you to run  a script command at startup of MobaXterm. That is what I used to create a StartDatabase Button:
The command is just MobaXterm (I should put MobaXterm in a version-less folder name, with a version-less executable, or create a script for that).
As parameters I provided -newtab ./startDB.sh. This is to ensure that the script is started on a new tab in MobaXterm, in a new potential session. Little side affect is that it creates a tab on the MobaXterm every button click. So, I might end up closing a few tabs...

Start SQLDeveloper and JDeveloper (BPM QuickStart)

To start SQLDeveloper I have a sqldev.sh script with the following content:

nohup ssh oracle@localhost -p 2222  /home/oracle/bin/sqldev.sh  > sqldev.out 2>&1 &

And the button looks like:

Similarly, I have a button to start JDeveloper. And it all ends up in the following desktop:

It might need some tweeking. But for now I love it and it works like a charm.

Today I found something curious in a composite upgraded from 11g to 12c, regarding DVMs. I sometimes use DVMs in BPEL to prevent the use of complex xpath expressions with many conditions. For instance, if I need to know if a JMSType is in a certain range and if it is I need to continue, I can create a DVM that has those JVMTypes correlated to an indicator.

Now, in 12c we have a new project structure. Where in 11g, about every component is in the root of the project, in 12c those are moved to a subfolder. That is, if you would create a new project:

Folder like xsd, wsdl, xsl in 11g are renated to Schemas, WSDLs and Transformations in 12c. We decided to refactor the upgraded projects to the new structure in 12c. So our BPEL processes are moved to the BPEL subfolder. This means that when referencing a transformation (xsl) you would adapt your xslt functions as:

        <assign name="Transform2MessageProperties">
          <bpelx:annotation>
            <bpelx:pattern>transformation</bpelx:pattern>
          </bpelx:annotation>
          <copy>
            <from expression="ora:doXSLTransformForDoc('../Transformations/Transform2MessageProperties.xsl', $Receive_InkomendBericht_InkomendBericht_InputVariable.InkomendBericht)"/>
            <to variable="messageProperties"/>
          </copy>
        </assign>

Here you see that the reference to the transformation is relative to the BPEL process, and thus '../Transformations'.

Keeping things transparant and uniform, we adapted the DVM references accordingly:

              <assign name="Assign_JmsType">
                <copy>
                  <from expression="dvm:lookupValue('../DVMs/DWN_Types.dvm', 'Type', bpws:getVariableData('messageProperties','/ns1:messageProperties/ns1:type') , 'JmsType', 'onbekend')"/>
                  <to variable="jmsType"/>
                </copy>
              </assign>

However, we got exceptions like:

<bpelFault><faultType>0</faultType><subLanguageExecutionFault xmlns="http://schemas.oracle.com/bpel/extension"><part name="summary"><summary>XPath expression failed to execute.
An error occurs while processing the XPath expression; the expression is dvm:lookupValue('../DVMs/DWN_Types.dvm', 'Type', bpws:getVariableData('messageProperties','/ns1:messageProperties/ns1:type') , 'JmsType', 'onbekend')
The XPath expression failed to execute; the reason was: oracle.fabric.common.xml.xpath.XPathFunctionException: Unable to get Metadata Manager for DVM "oramds:/deployed-composites/default/DVMs/DWN_Types.dvm"
Please ensure the Metadata Manager is available.
Check the detailed root cause described in the exception message text and verify that the XPath query is correct.
</summary></part><part name="code"><code>XPathExecutionError</code></part></subLanguageExecutionFault></bpelFault>

After some investigation and trial&error, I found it very peciuliar, that the reference apparently evoluated to: oramds:/deployed-composites/default/DVMs/DWN_Types.dvm. This actualy means that it searches for the DVM in the MDS relative to the deployed composite, but outside it's own folder. After the ../default/.. folder reference it should have the composites name and version/id. 

I tried it without the '../' reference in the path, and that worked!

Conclusion:

• In SOA 12c the ora:doXSLTransformForDoc() function is apparently executed in the context of the BPEL process and thus relative references to the XSL file should be done relative to the BPEL process.
• In SOA 12c the dvm:lookupValue() function appears to be executed in the context of the composite, so the reference to the DVM file should relative to the composite (composite.xml).

Today I was struggling with helping a colleague with a deployment of a SOA Project of his.
I couldn't get it deployed. It seemed I hit the problem described here. However when trying to connect to my Derby DB I got the following error:

I was very surprised. I checked and double checked my config. And check the library:
So, I checked those folders and found that they're not existing!
Now searching around I found in these release notes that as of Update 181 (let it just be the case that I just had this version of the JDK!) Java DB isn't bundled anymore:

Following the links it turns out that you should download it here.


I choose the zip and copied and unzipped it into my jdk:

[oracle@darlin-vce jdk]$ cp /media/sf_Stage/OpenSource/JavaDB/db-derby-10.14.2.0                                                                               -bin.zip .
[oracle@darlin-vce jdk]$ unzip db-derby-10.14.2.0-bin.zip
Archive:  db-derby-10.14.2.0-bin.zip
   creating: db-derby-10.14.2.0-bin/
  inflating: db-derby-10.14.2.0-bin/KEYS
  inflating: db-derby-10.14.2.0-bin/LICENSE
  inflating: db-derby-10.14.2.0-bin/NOTICE
  inflating: db-derby-10.14.2.0-bin/RELEASE-NOTES.html
...

Then I moved/renamed the folder to 'db':

[oracle@darlin-vce jdk]$ mv db-derby-10.14.2.0-bin db
[oracle@darlin-vce jdk]$ ls db/lib/
derbyclient.jar        derbyLocale_it.jar     derbyLocale_zh_TW.jar
derby.jar              derbyLocale_ja_JP.jar  derbynet.jar
derbyLocale_cs.jar     derbyLocale_ko_KR.jar  derbyoptionaltools.jar
derbyLocale_de_DE.jar  derbyLocale_pl.jar     derbyrun.jar
derbyLocale_es.jar     derbyLocale_pt_BR.jar  derbytools.jar
derbyLocale_fr.jar     derbyLocale_ru.jar     derby.war
derbyLocale_hu.jar     derbyLocale_zh_CN.jar
[oracle@darlin-vce jdk]$

After this I'm able to connect to the JavaDB:





So, that was my discovery of the day!

At my current customer we're busy with upgrading our projects from 11g to 12c.

One of the solution my predecessors implemented, is to kick of archive processes using sensor actions.The archive processes listen to JMS Queues, that are implemented as AQ Queues. For that a Foreign Server is configured:

The Foreign Server has a reference to the datasource that points to the schema owning the queues. It has also one or more Connnection Factories:

And the queues have a mapping from a local JNDI to a remote JNDI. The remote JNDI is the name of the particular queue prefixed with Queue:

In the sensor actions we used to have a JMS Adapter configured with as a connection factory the JNDI name of the outbound connection pool, for instance eis/aqjms/DwnQueueDB. The connection factory in that outbound connection factory refers to the JNDI of the connection factory in the Foreign Server.

Now, it turned out that our archiving processes weren't kicked off. I found a few things.

Sensor property files

The sensors can be configured using in the Monitor view of the BPEL Designer. It can be accessed using the Monitor Icon top left. When an Sensor is defined you can click the attena icon. You can of course create new ones by right clicking on the activity.
Sensor actions can be edited by selecting them and click the pencil-edit-icon.


In 11g, all the artefacts land in the root folder of the composite by default. We refactored the composites by moving artefacts to respective folders, like SOA Suite 12c would do in a new project.
But we skipped the files ${bpel-process-name}_sensor.xml and ${bpel-process-name}_sensorAction.xml.  I moved those to the same folder as the BPEL process. With a refresh, the attena-icons re-appeared.

But, also the files are referenced in the composite.xml:

…
</componentType>
    <property name="configuration.sensorLocation" type="xs:string" many="false">BPEL/${bpel-process-name}_sensor.xml</property>
    <property name="configuration.sensorActionLocation" type="xs:string" many="false">BPEL/${bpel-process-name}_sensorAction.xml</property>
  </component>

These references aren't updated automatically when moving them. But it turns out that the properties are renamed as well (probably from 10g to 11g already):

• pre-11g:  bpel.config.sensorLocation => 11g/12c onwards: configuration.sensorLocation
• pre-11g:  bpel.config.sensorActionLocation=> 11g/12c onwards: configuration.sensorActionLocation

JMS Adapter doesn't register the properties

As said, we used to use the JMS Adapter. I found that after the modifications to properly reference the sensor/sensorAction files, the message was published, but not picked up. The Listening archive process had a Message Selector like: BPEL_SENSOR_NAME like 'MySensorName%'.

I have a query that allows me to select from the queue tables and introspect the queues as JMS Queues:

select  qtb.queue_table 
, qtb.queue 
, qtb.msg_id
, qtb.msg_state
, qtb.enq_timestamp
, qtb.user_data.header.replyto
, qtb.user_data.header.type type
, qtb.user_data.header.userid userid
, qtb.user_data.header.appid appid
, qtb.user_data.header.groupid groupid
, qtb.user_data.header.groupseq groupseq
, qtb.user_data.header.properties properties
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'tracking_compositeInstanceId') tracking_compositeInstanceId
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'JMS_OracleDeliveryMode') JMS_OracleDeliveryMode
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'tracking_ecid') tracking_ecid
, (select num_value from table (qtb.user_data.header.properties) prp where prp.name = 'JMS_OracleTimestamp') JMS_OracleTimestamp
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'tracking_parentComponentInstanceId') tracking_prtCptInstanceId
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'tracking_conversationId') tracking_conversationId
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'BPEL_SENSOR_NAME') bpel_sensor_name
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'BPEL_PROCESS_NAME') bpel_process_name
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'BPEL_PROCESS_REVISION') bpel_process_rev
, (select str_value from table (qtb.user_data.header.properties) prp where prp.name = 'BPEL_DOMAIN') bpel_domain
, qtb.user_data.header
, qtb.user_data.text_lob text
, qtb.expiration_reason
--, qtb.*
from ( select 'DWN_OUTBOUND_TABLE' queue_table
       , qtb.* 
       from aq$dwn_outbound_table qtb
       union all
       ...
       union all
       select 'DWN_INBOUND_TABLE' queue_table
       , qtb.*
       from AQ$DWN_INBOUND_TABLE qtb) qtb
order by enq_timestamp desc;

This query lists the contents of several queue tables (always query queue tables via their AQ$Queue_table_name view) unioned together. From that you can introspect the user data and their properties witht he dot notation. The UserData has a header object, that contains a properties collection, that holds the JMS properties. You can select those as seen above.  It turns out that Sensor Actions should set the followign properties:

• BPEL_SENSOR_NAME
• BPEL_PROCESS_NAME
• BPEL_PROCESS_REVISION
• BPEL_DOMAIN

I found that using the JMS Adapter as a publish type in the SensorAction, these properties aren't set in 12c. While they apparently were in 11g.

After some researching, with no luck, I figured that I could try using a JMS Queue configuration. I wondered what the difference would be. Well, it turned out that using a JMS Queue did work.
Reconfiguring the Sensor Action to use a JMS Queue  means:

• Set Publish Type to JMS Queue (obviously)
• The JMS Connection Factory need to hold the JNDI name of the connection factory to use. In our case the one registered at the Foreign Server. (With JMSAdapter as Publish Type, this property is called JMSConnectionName.)
• The Publish Target is now the JNDI Name of the destination. In our example (see the screen shot above) it is the local jndi of the queue to publish to. With the JMS Adapter it was the queue name.

So, in our case this worked.