Wednesday, 24 April 2019

Test Remote Asynchronous Request Response services

A few years ago, I described how you can test Asynchronous Request Response services.

The thing with Asynchronous Request Response services is, as I used to describe it, that they're in essence two complementary Request-Only (Fire and Forget) services. That is, the client submits a request to the Asynchronous Request Response service, and at a certain point waits for the response by listening to an endpoint.

To make this work, the responding Asynchronous Request Response service should be told, which endpoint it should call with the response and which correlation id should be used. The WS-Addressing standard is used for that. All nicely explained in the before mentioned article.

In most customer-cases the problem is that your Client SoapUI or ReadyAPI project should catch the response, but the service is running on a SOA Suite in the datacenter and is not allowed to get to your local machine.

MobaXterm makes it very easy to create a tunnel. You can have a remote tunnel, that enables a local listening endpoint, that forwards every request to a remote service. Very handy if you have a Vagrant project with only a NAT NetworkAdapter, where Vagrant enabled a ssh endpoint on port 2222. You can easily create a Local tunnel on port 7101, for instance, to the remote ssh session on port 2222, that enables you to get to the weblogic console on the remote VM running on http://darlin-vce:7101/console.

To create a tunnel, just open the MobaSSHTunnel - Grahpical port forwarding tool:
This will open:

You can create a new SSH tunnel or edit a current one using the cogs icon under settings. For instance, to be able to do the Local port forwarding to get to your Weblogic console on your Vagrant box, create a tunnel as follows:



On the left you can enter a local port. That is the port you can use on your localhost. On the top right you can enter an host and port for the address to post your request to (does not need to be localhost). Then bottom right you need to provide an ssh session. A bit inconvenient is that you can't select a session from the sessions pane. Provide a host, port and user to connect to your ssh server.

What happens is that MobaXterm creates an SSH session, and a local endpoint. Every thing posted to the local endpoint is posted on the remote server to the give address. In this case I can go on my browser and enter https;//localhost:7101/console and it will bring me to the Weblogic Console on my Vagrant box. Neat, isn't it?



To get the remote Async Service respond to your local machine, you can also create a we need a tunnel that works the otherway around: we need Remote Port Forwarding:

Configuring is similar to Local port forwarding, however, now on the remote server a listen endpoint is created, and everything that is posted to the localhost:7777 adress (in this example) is forwarded to the address entered on the local server. In this case it is forwarded to localhost:7777, but it could be something else.

In our ReadyAPI project I created a Groovy script as follows:
def testCase = testRunner.testCase
def env = testCase.testSuite.project.activeEnvironment.name
if (env != "o02-12c"  &&  env != "o02" ) {
  log.info "Environment: "+env+", so set callbackIp to "+InetAddress.localHost.hostAddress
  testRunner.testCase.setPropertyValue( "callbackIp", InetAddress.localHost.hostAddress)
} else {
  log.info "Environment: "+env+", so set callbackIp to localhost"
  testRunner.testCase.setPropertyValue( "callbackIp", "localhost")
}

In ReadyAPI you can define environments, with the project property activeEnvironment.name it can be queried.

If the environment points to one of our development environments, I set the callbackIp testcase property to "localhost". But for the default environment, I use InetAddress.localHost.hostAddress to get the local ip address. This will be the ip address of our CD/CI tool, that runs ReadyAPI from a script.

You can set the WS-Addressing ReplyTo address as follows, for instance:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:add="http://schemas.xmlsoap.org/ws/2003/03/addressing">
      <soapenv:Header>
       <add:ReplyTo>
         <add:Address>http://${#TestCase#callbackIp}:7777/MyMockResponseURI</add:Address>
      </add:ReplyTo>
   </soapenv:Header>
   <soapenv:Body>

Then this address is used to do the callback. Make sure the tunnel is started:

You can also have the tunnel auto started (with the blue man-running-icon) or auto-reconnected (with the purple lightning icon).

This may also be very relevant in testing services on Oracle SOA Cloud Service, or Integration Cloud.

Happy tunneling!

Thursday, 18 April 2019

Split your Vagrant provisioners

For a while now, I'm quite into Vagrant in combination with VirtualBox. A few years ago I started with trying to script FMW environments, and since my discovery, and resulting fancy, of Vagrant, I also created a project for creating and provisioning a SOA Suite box.

Until now, my project all had one shell-type provisioner looking like:
  config.vm.provision "shell", inline: <<-SHELL
    export SCRIPT_HOME=/vagrant/scripts
    . $SCRIPT_HOME/install_env.sh
    echo _______________________________________________________________________________
    echo 0. Prepare Oracle Linux
    $SCRIPT_HOME/0.PrepOEL.sh
    echo _______________________________________________________________________________
    echo 1. Create Filesystem
    $SCRIPT_HOME/1.FileSystem.sh
    echo _______________________________________________________________________________
    echo 2. Create Oracle User
    $SCRIPT_HOME/2.MakeOracleUser.sh
    #
    echo _______________________________________________________________________________
    echo 3. Java SDK 8
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installJava.sh'
   SHELL

This seems quite simple, but for my SOA Suite box, I had quite a lengthy provisioner, that somehow failed at running the RCU and therefor with the creation of the domain.  There is a synchronization thingy with the database. The database is up, but at the time it reaches the RCU creation, it isn't able to connect. When running it seperately it works like a charm.

So, don't know how to solve that, but I want to re-provision only the part of the RCU and domain creation. Last night I fiddled around with it. Following the Vagrant Up basic usage explanation, you can create multiple provisioners with different names and different types. You can then force the provisioning for certain provisioners by type or by name.

I played around with that, because I couldn't get the syntax right. Although the explanation is proper, I wanted to have it slightly different and did not got it at first. Finally, I got it working.

Let's look into it.

First I split up my shell script, and found that I can put those in a variable. I now have a init script, that adapts the Linux OS, creates a new filesystem and creates an oracle user :
  $initScript = <<-SCRIPT
    export SCRIPT_HOME=/vagrant/scripts
    echo _______________________________________________________________________________
    echo 0. Prepare Oracle Linux
    $SCRIPT_HOME/0.PrepOEL.sh
    echo _______________________________________________________________________________
    echo 1. Create Filesystem
    $SCRIPT_HOME/1.FileSystem.sh
    echo _______________________________________________________________________________
    echo 2. Create Oracle User
    $SCRIPT_HOME/2.MakeOracleUser.sh
  SCRIPT

And one for installing the FMW software:
  $installFMWScript = <<-SCRIPT
    echo _______________________________________________________________________________
    echo 3. Java SDK 8
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installJava.sh'
    echo _______________________________________________________________________________
    echo 4. Database 12c
    sudo runuser -l oracle -c '/vagrant/scripts/database/installDB.sh'
    echo _______________________________________________________________________________
    echo 5.1 SQLCL and SQLDeveloper
    sudo runuser -l oracle -c '/vagrant/scripts/database/installSqlcl.sh'
    echo _______________________________________________________________________________
    echo 5.2 SQLDeveloper
    sudo runuser -l oracle -c '/vagrant/scripts/database/installSqlDeveloper.sh'
    echo _______________________________________________________________________________
    echo 6. Fusion Middleware
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installFMW.sh'
    echo _______________________________________________________________________________
    echo 6.1 Fusion Middleware - SOA
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installSOA.sh'
    echo _______________________________________________________________________________
    echo 6.2 Fusion Middleware - SB    
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installSB.sh'
    echo _______________________________________________________________________________
    echo 6.3 Fusion Middleware - OHS    
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installOHS.sh'
    echo _______________________________________________________________________________
    echo 7. BPM Quickstart
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/installBpmQS.sh'
  SCRIPT

And one for configuring FMW, that is running the RCU and creating the domain:
  $configFMWScript = <<-SCRIPT
    echo _______________________________________________________________________________
    echo 8.1 Fusion Middleware - RCU SOA   
    sudo runuser -l oracle -c '/home/oracle/bin/startDB.sh'    
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/rcuSOA.sh'
    echo _______________________________________________________________________________
    echo 8.2 Fusion Middleware - Create Domain    
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/fmw1221_domain/1.recreateFMWDomain.sh'
    echo !!! TODO: Machine configuration update to use Plain - 5555
    echo !!! TODO: Modify domain creation and property naming to create machine in accordance to nodemanager config.
    echo _______________________________________________________________________________
    echo 8.3 Fusion Middleware - Modify Nodemanager     
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/fmw1221_domain/2.modifyNodeManager.sh'
    echo _______________________________________________________________________________
    echo 8.4 Fusion Middleware - Create Nodemanager service
    sudo runuser -l oracle -c '/vagrant/scripts/fmw/fmw1221_domain/3.createNodemanagerService.sh'
    #
  SCRIPT

Cool, so far, right?

Now, after that we need to define the 3 provisioners:

  config.vm.provision "init", type: "shell", inline: $initScript
  config.vm.provision "installFMW", type: "shell", inline: $installFMWScript
  config.vm.provision "configFMW", type: "shell", inline: $configFMWScript

These provisioners

  • init -> provisioning/config of Oracle Linux, creation of oracle user, etc. This will be about equal for every box.
  • installFMW -> installation of all FMW software.
  • configFMW -> run the RCU and create domein.
Having that in place, you can run a specific provisioner. For instance during up:
  • vagrant up --provision-with configureFMW
or when reloading the box:
  • vagrant reload --provision-with configureFMW
But the following also works:
  • vagrant provision --provision-with configureFMW
Another cool thing is that you also could set a run option on the provisioner.
  config.vm.provision "init", run: "once", type: "shell", inline: $initScript
  config.vm.provision "installFMW", type: "shell", run: "once", inline: $installFMWScript
  config.vm.provision "configFMW", type: "shell", run: "once", inline: $configFMWScript

The run option has the following possible values:
  • "once": this is actually the default, the provisioner is only executed at first up. Or if you force it to run as described above.
  • "always": the provisioner is executed at every up. This can be used for something you want to be done every time you do up. A good one would be to start the database.
  • "never": this one is interesting. This makes the provisioner optional. That means it won't be executed, unless you ask for it. A good one would be to drop the RCU and delete the domain. So that you can reprovision the repository and the domain.
Happy Vagrant Upping!