Tuesday 25 September 2018

Split your flow trace in BPEL

A quick one today. In the past we suffered from very large flowtraces in SOA Suite 11g, due to which it could happen that a flow trace wasn't parsable in EM. And therefor could not be shown.

Also, you might have other reasons to split up your flow trace. Maybe because you want to have a callee BPEL process that may run for a longer time run on, while the calling BPEL project is redeployed (although I haven't tested that yet, so I'm not sure if that would work).

I did know it should be possible to split up the flow traces by changing the ECID (Execution Context ID). But, haven't seen it and wasn't able to find it. But, today I found the how-to in Oracle Support Note 2278472.1. So, as  a note-to-myself, here it is.

In the invoke activity to a child process you should add the following property:
<bpelx:toProperty name="tracking.ecid" variable="ora:generateGUID()"/>

This will update the tracking.ecid to a GUID.You should/need to do this on the invoke only (not on the receive). It should not cause any collission or conflict, since it generates a Global Unique Identifier.

Tuesday 18 September 2018

SOA 12c MDS configuration

In a previous post I described how to have the integrated Weblogic of your SOA/BPM QuickStart refer to the same filebased MDS that you might refer to in the SOADesignTimeRepository in JDeveloper.

These days for my current customer I'm looking into upgrading 11g, as can be read from my previous posts. This customer also has a legacy with projects migrated from 10g.

In the 11g workspace there was a reference to the database MDS in the Development database. In 12c we have a designtime mds reference. I would recommend to refer that to the mds artefacts in your VCS (Version Control System: svn or git) working copy. To do so, call-up the Resources pane in JDeveloper and right click on the SOADesignTimeRepository:
Then navigate to the location in your working copy:
Mind that SOASuite expects an apps folder within this folder, so resulting references in the composite.xml, etc. are expected to start with oramds:/apps/....

Now, I migrated a few projects including a adf-config.xml. In stead of the DB MDS repo, I replaced it with a file-based reference in the adf-config.xml, refering to the SOADesignTimeRepository. If you create a new 12c SOA Application, the adf-config.xml will look like:
<?xml version="1.0" encoding="windows-1252" ?>
<adf-config xmlns="http://xmlns.oracle.com/adf/config" xmlns:adf="http://xmlns.oracle.com/adf/config/properties"
            xmlns:sec="http://xmlns.oracle.com/adf/security/config">
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="MyApplication-1234"/>
  </adf:adf-properties-child>
  <sec:adf-security-child xmlns="http://xmlns.oracle.com/adf/security/config">
    <CredentialStoreContext credentialStoreClass="oracle.adf.share.security.providers.jps.CSFCredentialStore"
                            credentialStoreLocation="../../src/META-INF/jps-config.xml"/>
  </sec:adf-security-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace path="/soa/shared" metadata-store-usage="mstore-usage_1"/>
          <namespace path="/apps" metadata-store-usage="mstore-usage_2"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property name="partition-name" value="seed"/>
              <property name="metadata-path" value="${soa.oracle.home}/integration"/>
            </metadata-store>
          </metadata-store-usage>
          <metadata-store-usage id="mstore-usage_2">
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property name="metadata-path" value="${soamds.apps.home}"/>
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
</adf-config>

In the metadata-store-usage with id mstore-usage_2 you'll find the reference ${soamds.apps.home} in the metadata-path property. This refers to the folder as choosen in your SOADesignTimeRepository.

Now, I found in the past several times that although the adf-config.xml was similar to the above, that the MDS references did not work. In those cases, as a workaround, I put the absolute path reference in the metadata-path property.

Today I found something similar because of the upgrade, and searched on MDS-01333: missing element "mds-config" This resulted in this article, that gave me the hint.

It turns out that the snippet:
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="MyApplication-1234"/>
  </adf:adf-properties-child>

get's in the way. the UID refers to the application name and some generated number. It turns out not enough to change it the name of the application with a generated number. I haven't found what the proper number should be. So I just removed that snippet and then it worked.

Monday 17 September 2018

SOA Bundelpatch for 12.2.1.3 available (since juli 2018)

Because of the version level my previous customer was on, I mostly worked with the 12.2.1.2 version of the BPM QuickStart. Recently I started at an other customer that is still on SOA Suite 11g. Since I'm looking into upgrading those the latest 12c version, I installed BPM QuickStart 12.2.1.3 again.

Doing a patch search on support.oracle.com, I found out that juli 17th, 2018, a SOA BundlePatch on 12.2.1.3 was released. It's patch 28300397.

The readme shows quite a list of bugs solved. The version of JDeveloper and the main components stay unaffected. The version changes are shown in the extensions. The vanilla, unpatched, JDeveloper shows:

And the patched JDeveloper shows:

Since it's been a year already since 12.2.1.3 was released (august 2017, if I recollect correctly), this bundle patch is welcome.

By the way, the reason that I was looking into the patches, was that I created a few .xsl files to pre-upgrade our 11g projects. And the didn't reformat properly. JDeveloper behaves strangely, apparenlty it does not recognize an .xsl file as xml. When you copy and paste it into an .xml file it does format properly. I think I have to dig into the preferences to see if this can be tweaked.


To install it, unzip the patch. I'm used to create a patches folder within the OPatch folder in the Oracle Home:
And unzip the patch in to that folder. Because the unzip functionality in Windows is limited to 256 characters in the resulting path names, it is advised to use a tool like 7Zip. Since I use TotalCommander for about everything, (file related that is), I get a neat dialog mentioning this and allowing me to keep the names.

Make sure you have closed JDeveloper.

Then open a command window and navigate to the patches folder:
Microsoft Windows [Version 10.0.17134.285]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\Windows\system32>cd \oracle\JDeveloper\12213_BPMQS\OPatch\patches\28300397

C:\oracle\JDeveloper\12213_BPMQS\OPatch\patches\28300397>set ORACLE_HOME=C:\oracle\JDeveloper\12213_BPMQS

First set the the ORACLE_HOME variable to the location where you installed JDeveloper, C:\oracle\JDeveloper\12213_BPMQS in my case.

Using opatch apply you can apply the patch:
c:\Oracle\JDeveloper\12213_SOAQS\OPatch\patches\28300397>..\..\opatch apply
Oracle Interim Patch Installer version 13.9.2.0.0
Copyright (c) 2018, Oracle Corporation.  All rights reserved.


Oracle Home       : c:\Oracle\JDeveloper\12213_SOAQS
Central Inventory : C:\Program Files\Oracle\Inventory
   from           :
OPatch version    : 13.9.2.0.0
OUI version       : 13.9.2.0.0
Log file location : c:\Oracle\JDeveloper\12213_SOAQS\cfgtoollogs\opatch\opatch2018-09-17_12-05-54PM_1.log


OPatch detects the Middleware Home as "C:\Oracle\JDeveloper\12213_SOAQS"

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   28300397

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = 'c:\Oracle\JDeveloper\12213_SOAQS')


Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '28300397' to OH 'c:\Oracle\JDeveloper\12213_SOAQS'
ApplySession: Optional component(s) [ oracle.mft, 12.2.1.3.0 ] , [ oracle.soa.workflow.wc, 12.2.1.3.0 ] , [ oracle.integ
emina, 2.0.4.0.1 ] , [ oracle.mft.apachemina, 2.0.4.0.1 ] , [ oracle.bpm.plugins, 12.2.1.3.0 ] , [ oracle.oep.examples,
 12.2.1.3.0 ]  not present in the Oracle Home or a higher version is found.

Patching component oracle.soa.all.client, 12.2.1.3.0...

Patching component oracle.integration.bam, 12.2.1.3.0...

Patching component oracle.rcu.soainfra, 12.2.1.3.0...

Patching component oracle.rcu.soainfra, 12.2.1.3.0...

Patching component oracle.soacommon.plugins, 12.2.1.3.0...

Patching component oracle.oep, 12.2.1.3.0...

Patching component oracle.integration.soainfra, 12.2.1.3.0...

Patching component oracle.integration.soainfra, 12.2.1.3.0...

Patching component oracle.soa.common.adapters, 12.2.1.3.0...

Patching component oracle.soa.procmon, 12.2.1.3.0...
Patch 28300397 successfully applied.
Log file location: c:\Oracle\JDeveloper\12213_SOAQS\cfgtoollogs\opatch\opatch2018-09-17_12-05-54PM_1.log

OPatch succeeded.

c:\Oracle\JDeveloper\12213_SOAQS\OPatch\patches\28300397>

Answer 'y' on the questions to proceed and if the oracle home is ready to be patched. And with  opatch lsinventory you can check if the patch (and possibly others) is applied:

c:\Oracle\JDeveloper\12213_SOAQS\OPatch\patches\28300397>..\..\opatch lsinventory
Oracle Interim Patch Installer version 13.9.2.0.0
Copyright (c) 2018, Oracle Corporation.  All rights reserved.


Oracle Home       : c:\Oracle\JDeveloper\12213_SOAQS
Central Inventory : C:\Program Files\Oracle\Inventory
   from           :
OPatch version    : 13.9.2.0.0
OUI version       : 13.9.2.0.0
Log file location : c:\Oracle\JDeveloper\12213_SOAQS\cfgtoollogs\opatch\opatch2018-09-17_12-11-19PM_1.log


OPatch detects the Middleware Home as "C:\Oracle\JDeveloper\12213_SOAQS"

Lsinventory Output file location : c:\Oracle\JDeveloper\12213_SOAQS\cfgtoollogs\opatch\lsinv\lsinventory2018-09-17_12-11-19PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: V2W1-MAKKER.ONT.OTA.IND.MINBZK.NL
ARU platform id: 233
ARU platform description:: Microsoft Windows Server 2003 (64-bit AMD)


Interim patches (5) :

Patch  28300397     : applied on Mon Sep 17 12:08:10 CEST 2018
Unique Patch ID:  22311639
Patch description:  "SOA Bundle Patch 12.2.1.3.0(ID:180705.1130.0048)"
   Created on 6 Jul 2018, 01:58:16 hrs PST8PDT
   Bugs fixed:
     26868517, 27030883, 25980718, 26498324, 26720287, 27656577, 27639691
     27119541, 25941324, 26739808, 27561639, 26372043, 27078536, 27024693
     27633270, 27073918, 27210380, 27260565, 27247726, 27880006, 27171517
     26573292, 26997999, 26484903, 27957338, 27832726, 27141953, 26851150
     26696469, 27494478, 27150210, 27940458, 26982712, 27708925, 26645118
     27876754, 24922173, 27486624, 26571201, 26935112, 26953820, 27767587
     26536677, 27311023, 26385451, 26796979, 27715066, 27241933, 24971871
     26472963, 27411143, 27230444, 27379937, 27640635, 26957183, 26031784
     26408150, 27449047, 27019442, 26947728, 27368311, 26895927, 27268787
     26416702, 27018879, 27879887, 27929443

Patch  26355633     : applied on Wed Sep 12 12:00:33 CEST 2018
Unique Patch ID:  21447583
Patch description:  "One-off"
   Created on 1 Aug 2017, 21:40:20 hrs UTC
   Bugs fixed:
     26355633

Patch  26287183     : applied on Wed Sep 12 11:59:56 CEST 2018
Unique Patch ID:  21447582
Patch description:  "One-off"
   Created on 1 Aug 2017, 21:41:27 hrs UTC
   Bugs fixed:
     26287183

Patch  26261906     : applied on Wed Sep 12 11:59:11 CEST 2018
Unique Patch ID:  21344506
Patch description:  "One-off"
   Created on 12 Jun 2017, 23:36:08 hrs UTC
   Bugs fixed:
     25559137, 25232931, 24811916

Patch  26051289     : applied on Wed Sep 12 11:58:45 CEST 2018
Unique Patch ID:  21455037
Patch description:  "One-off"
   Created on 31 Jul 2017, 22:11:57 hrs UTC
   Bugs fixed:
     26051289



--------------------------------------------------------------------------------

OPatch succeeded.

Friday 14 September 2018

Advanced SoapUI: Mocking a Async Request Response Service supporting WS-Addressing

Lately, I sat down with my appreciated colleague Frank, prepping for a SoapUI/ReadyAPI training due next week. After having solved some issues he had, we agreed upon designing an advanced lab.

In the past I wrote an article about how to test een Asynchronous Request Response BPEL process, with WS-Addressing. This uses SoapUI as a test client, to test an Async BPEL Process and catching the response. Frank suggested to create a SoapUI project that mocks the BPEL Process. And that's a whole other ball-game!

SoapUI does support Mock Services. But those are in fact Synchronous services: upon request they send back a response. They're very flexible in that you can determine and select the responses that are send back in several ways. You can even script the lot using an OnRequest Groovy script.

But in this case we do not want to send back a response. The thing with an Asynchronous Request Response service is that they're actually two complementary Fire & Forget services.
  1. The actual request service is a fire and forget service implemented by the service provider. It does not respond with a message, but it just starts processing the request.
  2. Then the service client implements a CallBack fire and forget service. Upon processing the request to the point that a response is build, the Service Provider will call this service with the actual response as a request.
How would you implement this with SoapUI? First, you create a Project with a TestCase as described in my referenced article. It will invoke the SOAP Service with a request and then bring up a SOAP Mock Response to catch the response.

For the Async MockService we create a MockService that catches the request. But we leave the response empty: we do not want to reply with a response immediately. In stead, we use the On Request script to call a Test Case that simulates the proces. The interesting part is to pass the info from the request: the WS-Addressing elements (ReplyTo-Address and MessageId) and message Content. But let's sort that out step-by-step.

By the way I worked this out as a Lab together with my colleague Frank, in both SoapUI and ReadyAPI simultaneously. So it works in both products. In stead of ReadyAPI's 'Virts', I stick with the SoapUI term MockServices. But the principles and code snippets work one-on-one.

Create a SoapUI project with a MockService

First create a SoapUI project. I used the wsdl and xsd that I published here on github.
Then create a MockService on the BPELProcessAsyncBinding Request binding:

  • Name it for example: BPELProcessAsyncBinding MockService.
  • Service Path: /mockBPELProcessAsyncBinding
  • Port: 8088
We don’t provide a response on purpose: it will be an async service, that will respond by doing an invoke later on.

Add the mockservice’s endpoint to the interface binding:
Remove the original endpoint from the interface, since it is a dummy endpoint ('http://localhost:7101/soa-infra/services/default/helloWorldBPELAsync/bpelprocessasync_client_ep').

Now you can test the MockService with an adhoc request.

Create a 'Client' Test case

In the SoapUI Project, create a TestSuite called TestSuite AsyncBPEL and add a TestCase, called AsyncSvcClient:
Then clone the Adhoc Test Request to the testcase and call it InvokeAsyncService:


To pick up the response we need to add a MockResponse based on the CallBack binding of the wsdl:
Base it on the CallBack Binding of the wsdl:
Take note of the Port and the Path, if you choose to use something else as 8090 and /HelloWorldCallback that I used for this article.

It is important that this step is started as soon as the request is sent. It takes time to startup the MockResponse listener. So, you need to couple it to the corresponding Soap Request step. To do so, you need to get to the properties of the AsyncReceive MockResponse step and set the start step of the MockResponse step to InvokeAsyncService:

This will ensure that when the InvokeAsyncService step is executed the AsyncReceive mock response is started, so that it can be called as soon as the ServiceProvider wants to send back its response.

Note that the xml request of the AsyncReceive step is empty, as well as the response. The response will stay unused, but the request is to capture the callback message from the service provider, as we will see later on.

Setup the Async Service Provider

The MockService inherently is a synchronous mechanism, so normally used to respond with a response message on request. Since we want to implement an asynchronous request-reply mock service, we won’t respond with a message. So the response message stays empty. How are we going to respond then? We will build a second test case, that will be executed on request from a Groovy Script on the MockService. It will build up a context from the request message and providing that to the running testcase we will provide the test case with the information to invoke the AsyncReceive step of the client test case.

Thus we create a new test case, and it will do two things:
  1. Extract the request properties from the context, they will consist of the following properties:
    1. WS Addressing ReplyTo Address
    2. WS Addressing MessageId
    3. HelloWorld Input message (payload elements)
  2. Do the Callback based on the provided information.
To implement this perform the following:
  1. Create a new TestSuite, called AsyncBPELSvcProvider and add a TestCase, called AsyncSvcProvider.
  2. Add a SOAP Request step, named CallBackAsyncSvcClient and based that on the BPELProcessAsyncCallbackBinding:
  3. As a result value provide ‘Hello’ for now.
  4. As an endpoint set http://localhost:8090/HelloWorldCallback. We will change that to a property, later , fetched from the context.
  5. Remove a possible assertion to check on the Soap Response Message (since we won’t get one).
  6. If you want to test now, you can run the AsyncSvcClient but it will wait on the AsyncReceive step. To have that execute, you should manually run the AsyncSvcProvider test case.

Now we need to have the new TestCase called from the OnRequest script of the MockService.
For that we add a few properties to the MockService, to denote the TestSuite and the containing TestCase that implements our ServiceProvider process.
Then using a basic Groovy script that we will extend later on, we make sure that that test case is ran.

  1. Add two Custom Properties:
    1. AsyncTestSuite, with value: AsyncBPELSvcProvider
    2. AsyncSvcProvTestCase, with value: AsyncSvcProvider
  2. On the OnRequest script of the Mock Service:

    Add the following script:
    def mockService = context.mockService
    def method = mockService.name+".Response 1.OnRequest Script"
    log.info("Start "+method)
    //
    def project = mockService.project
    log.info("Project "+project.name)
    def asyncTestSuiteName = mockService.getPropertyValue( "AsyncTestSuite")
    def asyncTestSuite = project.getTestSuiteByName(asyncTestSuiteName)
    log.info("TestSuite: "+asyncTestSuite.name)
    def asyncSvcProvTestCaseName = mockService.getPropertyValue( "AsyncSvcProvTestCase")
    def asyncSvcProvTestCase = asyncTestSuite.getTestCaseByName(asyncSvcProvTestCaseName)
    log.info("TestCase: "+asyncSvcProvTestCase.name)
    //Log Request
    log.info(mockRequest.requestContent)
    
    
    // Set Service Context
    def svcContext = (com.eviware.soapui.support.types.StringToObjectMap)context
    
    
    
    //Invoke Async Service Provider TestCase
    asyncSvcProvTestCase.run(svcContext, false)
    // End Method
    log.info("End "+method)
    

    What this does is the following:
    1. Define the mockService and the project objects from the context variable.
    2. Get the TestSuite and TestCase objects based on the MockService property values of the TestCase to be called.
    3. Create a serviceContext, to be used to do property transfer later on.
    4. Run the testCase using the created serviceContext.
  3. Now you can test this by invoking the AsyncSvcClient test case. You might want to remove the current content of the request of the AsyncReceive .

Transfer Request Context properties to ServiceProvider TestCase

Now we want to at least transfer the helloworld input in the request from the MockService to the service provider testcase, so that it can add it to the response message.

In the OnRequest Groovy Script we already created a context. We can simply set additional properties to that context. The values we can extract from the request, by xpath.

  1. Go to the OnRequest groovy script and extend your existing script to reflect the following:
    def mockService = context.mockService
    def method = mockService.name+".Response 1.OnRequest Script"
    log.info("Start "+method)
    //
    def project = mockService.project
    log.info("Project "+project.name)
    def asyncTestSuiteName = mockService.getPropertyValue( "AsyncTestSuite")
    def asyncTestSuite = project.getTestSuiteByName(asyncTestSuiteName)
    log.info("TestSuite: "+asyncTestSuite.name)
    def asyncSvcProvTestCaseName = mockService.getPropertyValue( "AsyncSvcProvTestCase")
    def asyncSvcProvTestCase = asyncTestSuite.getTestCaseByName(asyncSvcProvTestCaseName)
    log.info("TestCase: "+asyncSvcProvTestCase.name)
    //Log Request
    log.info(mockRequest.requestContent)
    // 
    // Added lines ==>
    def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
    // Set Namespaces and query request
    def holder = groovyUtils.getXmlHolder(mockRequest.getRequestContent())
    holder.namespaces["soapenv"] = "http://schemas.xmlsoap.org/soap/envelope/"
    holder.namespaces["bpel"] = "http://xmlns.oracle.com/ReadyAPIHellloWorldSamples/helloWorldBPELAsync/BPELProcessAsync"
    holder.namespaces["wsa"] = "http://www.w3.org/2005/08/addressing"
    def helloInput =  holder.getNodeValue("/soapenv:Envelope/soapenv:Body/bpel:process/bpel:input")
    // Set Service Context
    def svcContext = (com.eviware.soapui.support.types.StringToObjectMap)context
    svcContext.helloInput=helloInput
    // <==Added lines
    //
    log.info("helloInput: "+svcContext.helloInput)
    //Invoke Async Service Provider TestCase
    asyncSvcProvTestCase.run(svcContext, false)
    // End Method
    log.info("End "+method)
    
    This adds the following:
    1. A declaration of the groovyUtils, that is used to get an so called XmlHolder that contains the content of the Request in parsed XML Format.
    2. Declare namespace references in the holder.
    3. Query the helloInput using the xpath expression: "/soapenv:Envelope/soapenv:Body/bpel:process/bpel:input” from the request.
    4. Set this as a helloInput property on the service context.


  • Now we need to extract these properties in the AsyncSvcProvider TestCase, so that we can use it in the request of the callback. To do so add a Groovy Test Step to the AsyncSvcProvider TestCase, as a first step:

    Call it GetContextProperties, and move it as the first step in the TestCase:
  • Add the following to the script:
    def testCase=testRunner.testCase
    def testSuite=testCase.testSuite
    def methodName=testSuite.name+"."+testCase.name+".getContextProperties"
    log.info("Start MethodName: "+methodName)
    def helloInput=context.helloInput
    log.info(methodName+" Received HelloInput: "+helloInput)
    testCase.setPropertyValue("helloInput",helloInput)
    log.info("End MethodName: "+methodName)
    

    As you can see in the top right corner of the editor, you can see that besides a log variable also a context variable is provided:

    This variable will contain the properties we set in the call to the testcase from the MockService.
    As you can see we get the property from the context, and set it as a TestCase property.
  • Add the helloInput property to the AsyncSvcProvider TestCase. You don’t need to provide a value, it just needs to exist. 
  • Lastly, in the request of the CallBackAsyncSvcClient step, add ${#TestCase#helloInput} to the result:

  • Configure WS-Addressing

    In the previously mentioned blog article you can read how to create a test case that supports WS Addressing to call and test an asynchronous (BPEL) request response service. Now with the above, we have the plumbing in place to add the WS Addressing specifics to simulate and test the Asynchronous RequestResponse Service Provider.

    We need then to provide and process the following:
    • A WS Addressing Reply To Address, based on property values that matches the port and path of the AsyncReceive step.
    • A message id that is used to validate if the response back is using the correct provided messageId header value. In a real life case this message Id is used by the SOA Suite infrastructure to correlate the response to the correct process instance that requested it. This is not supported/implemented in SoapUI, since that tool is not meant for that. But we can add an assertion to check the correct responding of this property.
    To implement this, perform the following:
    1. On the AsyncSvcClient test case add the following properties:
      • callbackURI, with value: HelloWorldCallback
      • callbackPort, with value: 8090
      • callbackHost, with value: localhost
      • wsAddressingReplyToEndpoint, with value: http://${#TestCase#callbackHost}:${#TestCase#callbackPort}/${#TestCase#callbackURI}
      • wsAddressingMessageId, with no value

      You see that the wsAddressingReplyToEndpoint is dynamically build up from the previous properties. The callbackURIand the callbackPort should exactly match the values of the path and the port of the AsyncReceive step (without the initial slash):

      The property wsAddressingMessageId does not need a value: we will generate a value in another Groovy TestStep.
    2.  Add a Groovy TestStep to AsyncSvcClient test case, call it GenerateWSAMessageId,  and move it to the top, and add the following code:
      def testCase=testRunner.testCase
      def testSuite=testCase.testSuite
      def methodName=testSuite.name+"."+testCase.name+".GenerateWSAMessageId"
      log.info("Start "+methodName)
      def wsAddressingMessageId=Math.round((Math.random()*10000000000))
      testCase.setPropertyValue("wsAddressingMessageId", wsAddressingMessageId.toString())
      log.info("End "+methodName)

      This will do a randomize and multiply it with a big number to create an integer value.
    3. Now we will add the WS Addressing properties to the request. Open the InvokeAsyncService test step and click on the WS-A tab at the bottom:

      Set the following properties:
      • Check Enable WS-A Addressing
      • Set Must understand to TRUE
      • Leave WS-A Version to 200508
      • Check Add default wsa:Action
      • Set Reply to to: ${#TestCase#wsAddressingReplyToEndpoint}
      • Uncheck Generate MessageID
      • Set MessageID to: ${#TestCase#wsAddressingMessageId}
      The Reply To address and the MessageID now are based on the earlier determined properties.
    4. If you would test this, then the request that will be send will look like:
      <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:bpel="http://xmlns.oracle.com/ReadyAPIHellloWorldSamples/helloWorldBPELAsync/BPELProcessAsync">
        <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
          <wsa:Action soapenv:mustUnderstand="1">process</wsa:Action>
          <wsa:ReplyTo soapenv:mustUnderstand="1">
            <wsa:Address>http://localhost:8090/HelloWorldCallback</wsa:Address>
          </wsa:ReplyTo>
          <wsa:MessageID soapenv:mustUnderstand="1">9094853750</wsa:MessageID>
        </soapenv:Header>
        <soapenv:Body>
          <bpel:process>
            <bpel:input>Roberto</bpel:input>
          </bpel:process>
        </soapenv:Body>
      </soapenv:Envelope>

      You see that the ReplyTo Address is set (as a nested element) and the MessageId. You won’t see this in the Request XML panel, but in the http log or in the script-log since we log the request in the OnRequest script of the MockService. The WS-Addressing properties are added to the soap:header on invoke.
    5. Since we have these elements in the request, we can extract those the same way as we did with the helloInput in the OnRequest script of the MockService. Add the lines denoted with // Added lines ==> and // <==Added lines: from the following script in your script (or copy&paste complete script):
      def mockService = context.mockService
      def method = mockService.name+".Response 1.OnRequest Script"
      log.info("Start "+method)
      //
      def project = mockService.project
      log.info("Project "+project.name)
      def asyncTestSuiteName = mockService.getPropertyValue( "AsyncTestSuite")
      def asyncTestSuite = project.getTestSuiteByName(asyncTestSuiteName)
      log.info("TestSuite: "+asyncTestSuite.name)
      def asyncSvcProvTestCaseName = mockService.getPropertyValue( "AsyncSvcProvTestCase")
      def asyncSvcProvTestCase = asyncTestSuite.getTestCaseByName(asyncSvcProvTestCaseName)
      log.info("TestCase: "+asyncSvcProvTestCase.name)
      //Log Request
      log.info(mockRequest.requestContent)
      //
      def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
      // Set Namespaces and query request
      def holder = groovyUtils.getXmlHolder(mockRequest.getRequestContent())
      holder.namespaces["soapenv"] = "http://schemas.xmlsoap.org/soap/envelope/"
      holder.namespaces["bpel"] = "http://xmlns.oracle.com/ReadyAPIHellloWorldSamples/helloWorldBPELAsync/BPELProcessAsync"
      holder.namespaces["wsa"] = "http://www.w3.org/2005/08/addressing"
      def helloInput =  holder.getNodeValue("/soapenv:Envelope/soapenv:Body/bpel:process/bpel:input")
      //
      // Added lines ==>
      def wsaReplyToAddress =  holder.getNodeValue("/soapenv:Envelope/soapenv:Header/wsa:ReplyTo/wsa:Address")
      def wsaInReplyToMsgId =  holder.getNodeValue("/soapenv:Envelope/soapenv:Header/wsa:MessageID")
      // <Added lines
      //
      // Set Service Context
      def svcContext = (com.eviware.soapui.support.types.StringToObjectMap)context
      svcContext.helloInput=helloInput
      //
      // Added lines ==>
      svcContext.wsaReplyToAddress=wsaReplyToAddress
      svcContext.wsaInReplyToMsgId=wsaInReplyToMsgId
      // <Added lines
      //
      log.info("helloInput: "+svcContext.helloInput)
      //
      // Added lines ==>
      log.info("wsaReplyToAddress: "+svcContext.wsaReplyToAddress)
      log.info("wsaInReplyToMsgId: "+svcContext.wsaInReplyToMsgId)
      // <Added lines
      //
      //Invoke Async Service Provider TestCase
      asyncSvcProvTestCase.run(svcContext, false)
      // End Method
      log.info("End "+method)
      
    6. These context properties need to be extracted in the GetContextProperties of the AsyncSvcProvider test case, to set those as TestCase Properties. So, add the following properties (with no values) to the AsyncSvcProvider test case:
      • wsaReplyToAddress
      • wsaInReplyToMsgId
    7. In the GetContextProperties test step, add the lines with the added properties (or copy and paste the complete script):
      def testCase=testRunner.testCase
      def testSuite=testCase.testSuite
      def methodName=testSuite.name+"."+testCase.name+".getContextProperties"
      log.info("Start MethodName: "+methodName)
      def wsaReplyToAddress=context.wsaReplyToAddress
      def wsaInReplyToMsgId=context.wsaInReplyToMsgId
      def helloInput=context.helloInput
      log.info(methodName+" Received wsaReplyToAddress: "+wsaReplyToAddress)
      log.info(methodName+" Received wsaInReplyToMsgId: "+wsaInReplyToMsgId)
      log.info(methodName+" Received HelloInput: "+helloInput)
      testCase.setPropertyValue("wsaReplyToAddress",wsaReplyToAddress)
      testCase.setPropertyValue("wsaInReplyToMsgId",wsaInReplyToMsgId.toString())
      testCase.setPropertyValue("helloInput",helloInput)
      // End
      log.info("End MethodName: "+methodName)
      

      (Since the wsaInReplyToMsgId is an integer, it should be "toStringed"...)
    8. As a pre-final step is to adapt the CallBackAsyncSvcClient step to use the wsaReplyToAddress as an endpoint and the wsaInReplyToMsgId as a header property. Edit the endpoint in the step to ${#TestCase#wsaReplyToAddress}:

      Edit the soap header to:
         <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <wsa:MessageID>${#TestCase#wsaInReplyToMsgId}</wsa:MessageID>
         </soapenv:Header>
      

    9. The final step is to add an XPath Match assertion on the AsyncReceive to validate the response of the wsaInReplyToMsgId. Call it WSAInReplyToMessageId and provide the following xpath:
      declare namespace wsa='http://www.w3.org/2005/08/addressing';
      declare namespace bpel='http://xmlns.oracle.com/ReadyAPIHellloWorldSamples/helloWorldBPELAsync/BPELProcessAsync';
      declare namespace soapenv='http://schemas.xmlsoap.org/soap/envelope/';
      
      /soapenv:Envelope/soapenv:Header/wsa:MessageID
      

      As an Expected Result value provide: ${#TestCase#wsAddressingMessageId}.
    10. Test the completed AsyncSvcClient.

    Conclusion

    This will conclude this setup. And shows how to create a WS-Addressing supporting Asynchronous Request Response Service. I hope you got this far. In that case: I'm impressed. This is quite advanced SoapUI/ReadyAPI! stuff. But it shows the power of the tools. And if you wouldn't use this as it is, you might get some nice tips from it.

    Upgrade SOA 11g to 12c: Invalid Composite File

    You might not get it, but not every customer already moved from SOA Suite 11g to 12c. My current customer isn't for instance. Because we're in a bit of a lee period, I'm looking into upgrading their composite projects.

    Solve invalid composite files

    One thing I ran into quite immediately is that for several projects the composite.xml was invalid. It turned out that it wasn't even upgraded.

    I found this support.oracle.com article, DocID 2333742.1. It says that it's not a bug. Because the problem is in the 11g project. Now, we could discus about it, since the 11g project works, and might have been upgraded from an earlier version (10g or early 11g patchset). So, the upgrade process could have been improved. Well the solution is however quite simple.

    According to the support note, the .jpr file lacks several elements. A closer look brought me the idea that it could be narrowed down to only one element:
    <hash n="oracle.ide.model.TechnologyScopeConfiguration">
      <list n="technologyScope">
        <string v="SOA"/>
      </list>
    </hash>
    In other words, the project lacks the Techonlogy scope SOA. Apparently Oracle changed the Integration technology in SOA down the product evolution (No bug, but it would be nice that the upgrade process would take this into account). Because of this, the composite is not upgraded.

    Changing every one of the tens or hundreds of composite projects can be a tedious job. And I figured that it wouldn't be the only one problem I will run into.

    Luckily, the .jpr file is an XML file. So, with an xslt file we could pre-upgrade the .jpr file. So, based on this example, I created the following prepareJpr.xsl stylesheet:
    <?xml version="1.0"?>
    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
      <!-- Update JPR 
      @Author: M. van den Akker, Darwin-IT Professionals, 2018-09-12
      Based on: https://stackoverflow.com/questions/5876382/using-xslt-to-copy-all-nodes-in-xml-with-support-for-special-cases#5877772
      -->
      <xsl:template match="@*|node()">
            <xsl:copy>
                <xsl:apply-templates select="@*|node()"/>
            </xsl:copy>
        </xsl:template>
      <!-- Check Existence of SOA technology, because of Invalid composite file exceptions. MOS DocId 2333742.1-->
      <xsl:template match="hash[@n='oracle.ide.model.TechnologyScopeConfiguration']/list[@n='technologyScope']">  
            <xsl:copy>
                <xsl:apply-templates select="@*|node()"/>
                <!--If SOA doesn't exist then add it -->
                <xsl:choose>
            <xsl:when test="count(./string[@v='SOA'])=0">
                <xsl:comment>Add SOA technology</xsl:comment> 
                <string v="SOA"/>
                </xsl:when>
            <xsl:otherwise><xsl:comment>SOA technology already present </xsl:comment> 
                </xsl:otherwise>
          </xsl:choose>
            </xsl:copy>
        </xsl:template>
    </xsl:stylesheet>
    

    If the technology scope does not exist, it adds it with a comment to denote it was added by this utility. If it does not exist, I add a comment to denote that, to be sure that the xslt works and that the project have been handled, but considered ok.

    Then I created an prepareSOA11gApplication.xml ANT project that loops over the projects in the target application folder and runs the XSLT over every .jpr file in the application.

    It looks like something as follows (since I added some other functionality that I removed for this article):
    <?xml version="1.0" encoding="windows-1252" ?>
    <!--Ant buildfile generated by Oracle JDeveloper-->
    <!--Generated Sep 12, 2018 3:06:25 PM-->
    <project xmlns="antlib:org.apache.tools.ant" name="Prepare11gProjects" default="prepareApplication" basedir=".">
      <property file="build.properties"/>
      <taskdef resource="net/sf/antcontrib/antlib.xml">
        <classpath>
          <pathelement location="${ant-contrib.jar}"/>
        </classpath>
      </taskdef>
      <target name="prepareApplication" description="Refactor SOA 11g Project pre upgrade to 12c." depends="">
        <echo>Prepare ${SOA11gAppName} in ${SOA11gAppFolder} </echo>
        <echo>. Prepare projects</echo>
        <foreach target="PrepareProjectFile" param="projectFile">
          <fileset dir="${SOA11gAppFolder}" casesensitive="yes">
            <include name="**/*.jpr"/>
          </fileset>
        </foreach>
      </target>
      <target name="PrepareProjectFile">
        <echo message=".. Prepare ${projectFile}"></echo>
        <property name="projectFileOrg" value="${projectFile}.org"/>
        <echo message="... backup ${projectFile} to ${projectFileOrg}"></echo>
        <move file="${projectFile}" tofile="${projectFileOrg}" overwrite="false"/>
        <echo message="... transform ${projectFileOrg} to ${projectFile} using ${prepareJprXsl}"></echo>
        <xslt style="${prepareJprXsl}" in="${projectFile}.org" out="${projectFile}"/>
      </target>
    </project>
    
    I also added functionality to add the projects in a emptied .jws file, and to add a adf-config.xml file.
    That way, I get a prepared pre-upgraded workspace that contains only the projects I want to upgrade at that time.

    Adapt .jca files

    Another type of files that I encountered to be invalid after upgrade, are .jca files. Some projects showed invalid JCA adapters in the composite.
    It turns out that Oracle also changed the adapter names in the jca files. A .jca  file starts with:
    <adapter-config name="dbDatabaseAdapterService" adapter="db" wsdlLocation="../WSDLs/dbDatabaseAdapterService.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

    But in some files this line reads:
    <adapter-config name="dbDatabaseAdapterService" adapter="DB Adapter" wsdlLocation="../WSDLs/dbDatabaseAdapterService.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

    So, I need a similar XSLT file, that adapts these attributes. For other adapters changes would be:
    • AQ Adapter: aq
    • Apps Adapter: apps
    • etc.
    I would rule out that I need other adaptions too.


    Tuesday 11 September 2018

    FMW 12c Topology Suggestions

    A year or two, maybe three ago  I found this teriffic article on Fusion Middleware 12c topology suggestions.

    I find my self searching for this one from time to time. So it's time to write this little note.

    It explains which combinations of FMW products in a domain makes sense and especially what the different Server Groups mean.

    Wednesday 5 September 2018

    Zipping is easy in Java/Spring/SOASuite

    Why using Spring in SOASuite?

    This post is actually about Spring in SOASuite. Last few weeks I've come around several questions and implementations using Embedded Java in BPEL. The Java activity in Oracle BPEL is ideal for very small java snippets, to do a simple Base64 encode/decode for example. However, in the situations I encountered these weeks, the code samples were more complex. And my very first recommendation in these situations is to use the SpringContext component, a technique introduced in SOASuite 11g, that I wrote about in 2012 already. I called it 'Forget about WSIF: welcome Spring', but really: forget about Embedded Java in most cases too! So, with this article, I want to showcase the SpringContext component in SOASuite.

    A few disadvantages of the Embedded Java activity:
    • To get to, and set or manipulate, your BPEL data, you'll need to compose getVariableData() and setVariableData() functions with xpath-references. You can use temporary Assign activities with copy rules, from where you can extract the actual variable and xpath references, to copy and paste into the getVariableData() functions. But best workaround, to me is to declare a few xsd:string based variables and use Assigns before and after to exchange the data with the java snippet.
    • You can't test/debug the java snippets properly.
    • Catching and handling exceptions may not behave as you might want.
    • You can introspect variables that are referenced in your java snippet from the flow trace. But you must rely on the visual code compare between bpel and java snippet.
    The advantages of using a SpringContext component:
    • You can test your java code standalone, and just reuse the methods as is.
    • Wiring your Spring Component to your BPEL will result in a WSDL and a PartnerLink. From BPEL you just invoke it like any other service. 
    • In the flowtrace you will see the input and output variables as is (provided that you set the audit level on Development). And you can just assign the data to and fro the variables based on the wsdl message types.
    • It's all declarative.
    A disadvantage might be that it is a bit more complex. It might feel like a hazzle, so you wouldn't use it for 2 lines of Java code, probably. At the other hand, try it out, I guess you might experience it as so much easier and more reliable.

    The case at hand

    In the case at hand, as referred to in the title,  we need to read a zip file. We get files from Oracle B2B, that are saved on the filesystem with a system generated filename. That is different from the file as send, and provided using the context-disposition MIME header. The client wants to have the file moved and saved using the originating filename. We couldn't get it from B2B in an acceptable way. B2B saves the file using a filename like 'M1234987309891.2341921@myTPR1234_te2309098033.dat'. But it turns out to be a zip file that contains the file, with the name as we want it. So, we could just introspect the file to determine the name as which we want to move it. This means of course that we need to read the file, from Java. And to me it does not feel right to do that from a Java activity in BPEL. It's something more or less functional, so I want to abstract that in a service, based on a java class.

    Unzip in Java

    I found an unzip method in Java that does just about what I want here. It comes down to:
    package nl.darwinit.ziputils;
    
    import java.io.File;
    import java.io.FileInputStream;
    import java.io.FileOutputStream;
    import java.io.IOException;
    
    import java.util.zip.ZipEntry;
    import java.util.zip.ZipInputStream;
    
    import nl.darwinit.ziputils.beans.ListZipEntriesRequest;
    import nl.darwinit.ziputils.beans.ListZipEntriesResponse;
    import nl.darwinit.ziputils.beans.ZipEntryFile;
    
    public class Unzip implements IUnzip {
    
        public static final String className = "nl.darwinit.ziputils.Unzip";
    
        public Unzip() {
            super();
        }
    ...
    
    ...
        private static void unzip(String zipFilePath, String destDir) {
            File dir = new File(destDir);
            // create output directory if it doesn't exist
            if (!dir.exists())
                dir.mkdirs();
            FileInputStream fis;
            //buffer for read and write data to file
            byte[] buffer = new byte[1024];
            try {
                fis = new FileInputStream(zipFilePath);
                ZipInputStream zis = new ZipInputStream(fis);
                ZipEntry ze = zis.getNextEntry();
                while (ze != null) {
                    String fileName = ze.getName();
                    File newFile = new File(destDir + File.separator + fileName);
                    System.out.println("Unzipping to " + newFile.getAbsolutePath());
                    //create directories for sub directories in zip
                    new File(newFile.getParent()).mkdirs();
                    FileOutputStream fos = new FileOutputStream(newFile);
                    int len;
                    while ((len = zis.read(buffer)) > 0) {
                        fos.write(buffer, 0, len);
                    }
                    fos.close();
                    //close this ZipEntry
                    zis.closeEntry();
                    ze = zis.getNextEntry();
                }
                //close last ZipEntry
                zis.closeEntry();
                zis.close();
                fis.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        public static void main(String[] args) {
            Unzip unzip = new Unzip();
            int i = 0;
            for (String arg : args) {
                log("main", "arg[" + i++ + "]: " + arg);
            }
            ListZipEntriesRequest listZipEntriesRequest = new ListZipEntriesRequest();
            listZipEntriesRequest.setZipFileFolder(args[0]);
            listZipEntriesRequest.setZipFileName(args[1]);
            ListZipEntriesResponse listZipEntriesResponse = unzip.listZipEntries(listZipEntriesRequest);
            log("main", listZipEntriesResponse.toString());
        }
    }
    
    

    Besides reading all the ZipEntries in the zip file one by one, this does also save it using a FileOutputStream. But I do not want to output the files, but just get the filename. Another thing: this example shows a private static function. But for the SpringContext I need an instantiatable java class, so with a constructor, and a public method. So I created an Unzip class (with default constructor) and the following method:
        public ListZipEntriesResponse listZipEntries(ListZipEntriesRequest listZipEntriesRequest) {
            final String methodName = "ListZipEntriesResponse";
            logStart(methodName);
            ListZipEntriesResponse listZipEntriesResponse = new ListZipEntriesResponse();
            FileInputStream fileInputStream;
            //buffer for read and write data to file
            log(methodName, "List entries of " + listZipEntriesRequest.getZipFilePath());
            try {
    
                fileInputStream = new FileInputStream(listZipEntriesRequest.getZipFilePath());
                ZipInputStream zipInputStream = new ZipInputStream(fileInputStream);
                ZipEntry zipEntry = zipInputStream.getNextEntry();
                while (zipEntry != null) {
                    String fileName = zipEntry.getName();
                    log(methodName, "Entry: " + fileName);
                    ZipEntryFile zipEntryFile = new ZipEntryFile();
                    zipEntryFile.setFileName(fileName);
                    listZipEntriesResponse.addZipEntryFile(zipEntryFile);
                    //close this ZipEntry
                    zipInputStream.closeEntry();
                    zipEntry = zipInputStream.getNextEntry();
                }
                //close last ZipEntry
                zipInputStream.closeEntry();
                zipInputStream.close();
                fileInputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            logEnd(methodName);
            return listZipEntriesResponse;
        }
    

    The logStart, logEnd and log methods are just wrappers around System.out.println(). As an input I have a bean with:
    package nl.darwinit.ziputils.beans;
    
    import java.io.File;
    
    
    public class ListZipEntriesRequest implements IListZipEntriesRequest {
      
      private String zipFileFolder;
        private String zipFileName;
    
        public void setZipFileFolder(String zipFileFolder) {
            this.zipFileFolder = zipFileFolder;
        }
    
        public String getZipFileFolder() {
            return zipFileFolder;
        }
    
        public void setZipFileName(String zipFileName) {
            this.zipFileName = zipFileName;
        }
    
        public String getZipFileName() {
            return zipFileName;
        }
    
        public String getZipFilePath() {
            return getZipFileFolder() +File.separator+ getZipFileName();
        }    
    }
    

    Then as a response a bean with:
    package nl.darwinit.ziputils.beans;
    
    import java.util.ArrayList;
    import java.util.List;
    
    public class ListZipEntriesResponse implements IListZipEntriesResponse {
        private List zipEntryFiles;
    
        public void setZipEntryFiles(List zipEntryFiles) {
            this.zipEntryFiles = zipEntryFiles;
        }
    
        public List getZipEntryFiles() {
            return zipEntryFiles;
        }
    
        public void addZipEntryFile(ZipEntryFile zipEntryFile) {
            if (zipEntryFiles == null) {
                zipEntryFiles = new ArrayList();
            }
            zipEntryFiles.add(zipEntryFile);
        }
        
        public String toString(){
            StringBuffer strBuf = new StringBuffer("ListZipEntriesResponse\n");
            for (ZipEntryFile zipEntryFile : zipEntryFiles){
                strBuf.append("ZipEntryFile: "+zipEntryFile.toString()+"\n");
            }        
            return strBuf.toString();
        }
    }
    

    That uses a ZipEntryFile bean, as a List:
    package nl.darwinit.ziputils.beans;
    
    
    public class ZipEntryFile implements IZipEntryFile {
       private String fileName;
    
        public void setFileName(String fileName) {
            this.fileName = fileName;
        }
    
        public String getFileName() {
            return fileName;
        }
        
        public String toString(){
            return "fileName: "+getFileName()+"\n";
        }
    }

    Place your the source of your classes in the src of your SOA project. The classes are compiled in the the SOA/SCA-INF/classes folder (in 11g, you won't have the SOA subfolder). In 12c the sources can also be placed in the SOA/SCA-INF/src folder. But I found that this will cause a misterious nullpointer exception.

    The spring context

    My article 'Forget about WSIF: welcome Spring' neatly describes how to create a spring context in 11g. In 12c it does not differ much. You don't need to define separate Spring components per bean, though, but you actually can as shown in the article.  You do need to extract an Interface out of the main class. With only those methods you need in the interface. I actually extracted interfaces from all the beans, but you shouldn't have to.

    In this case the Service xml would look like:
    <?xml version="1.0" encoding="windows-1252" ?>
    <beans xmlns="http://www.springframework.org/schema/beans" xmlns:util="http://www.springframework.org/schema/util"
           xmlns:jee="http://www.springframework.org/schema/jee" xmlns:lang="http://www.springframework.org/schema/lang"
           xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
           xmlns:sca="http://xmlns.oracle.com/weblogic/weblogic-sca" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tool http://www.springframework.org/schema/tool/spring-tool.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd http://www.springframework.org/schema/jms http://www.springframework.org/schema/jms/spring-jms.xsd http://www.springframework.org/schema/oxm http://www.springframework.org/schema/oxm/spring-oxm.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://xmlns.oracle.com/weblogic/weblogic-sca META-INF/weblogic-sca.xsd">
      <!--Spring Bean definitions go here-->
      <sca:service name="Unzip" target="UnzipBean"
                   type="nl.minvenj.ind.indigo.ziputils.IUnzip"/>
      <bean id="UnzipBean" class="nl.minvenj.ind.indigo.ziputils.Unzip"/>
      <bean id="ListZipEntriesRequestBean" class="nl.darwinit.ziputils.beans.ListZipEntriesRequest"/>
      <bean id="ListZipEntriesResponseBean" class="nl.darwinit.ziputils.beans.ListZipEntriesResponse"/>
      <bean id="ZipEntryFileBean" class="nl.darwinit.ziputils.beans.ZipEntryFile"/>
    </beans>
    

    Wire your Spring component to the BPEL component, and then you should have a dialog telling you that the WSDL is created, probably with an accompanying wrapper wsdl for the partnerlink roles.

    And then in BPEL just invoke the code.

    Happy Spring in SOA (better that than the otherway around if you speak Dutch...).


    Monday 3 September 2018

    Docker on Oracle Linux

    It occurred to me that if you want to start using Docker there are plenty examples that use Ubuntu as a base platform. I read a book called Learning Docker that assumes Ubuntu for the examples, for instance. I know I am quite stubborn, a "know-it-better" person, but I want to be able to do the same on Oracle Linux.

    Docker on Oracle Linux turns out not too complicated. But I ran in to a caveat that I solved and want to share.

    I use Vagrant to bring up an Oracle Linux Box, based on a Vagrantfile that prepares the box.With that as a starting point, I created a script that does the complete docker installation. In the following I'll build it up for you, step by step. I'll add my project to GitHub, and provide a link to the complete script in the end.

    Init

    First some initialization and a function to read the property file:
    #!/bin/bash
    SCRIPTPATH=$(dirname $0)
    #
    # Install docker on Oracle Linux.
    # @author: Martien van den Akker, Darwin-IT Professionals.
    #
    function prop {
        grep "${1}" $SCRIPTPATH/makeDockerUser.properties|cut -d'=' -f2
    }
    
    #
    DOCKER_USER=$(prop 'docker.user')
    DOCKER_GROUP=docker

    This sets the $SCRIPTPATH property as the folder where the script resides, so I can refer to other files relative to that one.
    The function prop alows me to get a property from a property file. A smart function that I got from a colleague (thanks Rob). Based on this property file called makeDockerUser.properties:
    docker.user=oracle
    docker.password=welcome1
    

    I set the DOCKER_USER and DOCKER_GROUP properties.
    The DOCKER_GROUP property is hardcoded however, but that is the standard group that is created at installation of Docker to allow other users to use Docker.

    Install Docker Engine

    The first actual step is to install the docker engine. Now you can go for the community edition and I've seen that there are examples that pulls the docker-ce (docker Community Engine) for you. However, one of the reasons I am stubborn to stick with Oracle Linux (as you know a RedHat derivate) is that Oracle Linux is the flavor that is used with most of my customers. And if not, it is RedHat. And then I just want to rely on the standard repositories.

    To install the docker engine, I have to add the ol7_addons and the ol7_optional_latest repositories. During my OL prepare script, I already added the ol7_developer_EPEL repository. Then the docker-engine package can simply be installed by yum:

    #
    echo 1. Install Docker Engine
    echo . add ol7_addons and ol7_optional_latest repos.
    sudo yum-config-manager --enable ol7_addons
    sudo yum-config-manager --enable ol7_optional_latest
    #
    echo . install docker-engine
    sudo yum install -q -y docker-engine

    Install Curl

    For most docker related actions, it is convenient to have  curl installed as well:
    #
    echo 2. Install curl
    sudo yum install -q -y  curl 

    Add docker group to docker user

    After the docker installation, we need to add the docker group to the docker user (in my case the sort-of default oracle user):
    #
    echo 3. Add  ${DOCKER_GROUP} group to ${DOCKER_USER}
    sudo usermod -aG ${DOCKER_GROUP} ${DOCKER_USER}

    This allows the ${DOCKER_USER} (set in the initialization phase) to use the docker command.

    Check the docker install

    Now let's add a check if docker works:
    #
    echo 4. Check Docker install
    docker --version
    sudo systemctl start docker
    sudo systemctl status docker
    This lists the version of the installed docker, then starts the docker service and lists the status of the docker.

    Change the location of the docker containers

    When creating a docker container/image (I leave the difference for now), these are saved by default in the location /var/lib/docker. The thing is that this is on the root disk of your installation. And it can grow quite big. For installations of oracle software for instance, I create an extra disk that I mount on /app. It would be better to have a /data mount point as well, but for now I stick with the /app data. So, I want to have docker place my images on the secondary disk. One solution used by Tim Hall, see here, is to create a second disk, format it with BTRFS, and mount it simply to /var/lib/docker.
    I rather reconfigure docker to use another disk. This is taken from this article.

    To implement this, we first need to know which storage driver Docker uses. We get this from the command docker info, as follows:
    echo 5. Change docker default folder
    # According to oracle-base you should create a filesystem, preferably using BTRFS, for the container-home. https://oracle-base.com/articles/linux/docker-install-docker-on-oracle-linux-ol7. 
    # But let's stick with ext4.
    ## Adapted from  https://sanenthusiast.com/change-default-image-container-location-docker/
    echo 5.1. Find Storage Driver
    GREP_STRG_DRVR=$(sudo docker info |grep "Storage Driver")
    DOCKER_STORAGE_DRVR=${GREP_STRG_DRVR#*": "}
    echo "Storage Driver: ${DOCKER_STORAGE_DRVR}"

    This mentions me the overlay2 driver. Then we need to stop docker:
    echo 5.2. Stop docker
    sudo systemctl stop docker

    And then create the folder where we want to store the images:
    echo 5.3. Add reference to data folders for storage.
    DOCKER_DATA_HOME=/app/docker/data
    echo mkdir -p ${DOCKER_DATA_HOME}
    sudo mkdir ${DOCKER_DATA_HOME}

    Now I found a bit of a problem with my solution here. When I reconfigure docker to use my custom folder, it turns out that on my system the filesystem is not writable from the docker image. If you want to install software in your image, it of course wants to write the files. And this is prevented. After quite some searching, I came on this question on stackoverflow. It turns out that selinux enforces a policy that prevents writing of docker to a custom device. This can be simply circumvented by disabling the enforcing:
    #
    ##https://stackoverflow.com/questions/30091681/why-does-docker-prompt-permission-denied-when-backing-up-the-data-volume
    echo disable selinux enforcing
    sudo setenforce 0

    This disables, as said, the enforcing of selinux. I would say this should be a bit more nuanced. But I don't have that at hand. This however, solved my problem.
    Now all is left to configure docker to use the custom folder. Docker is started using a script. In Oracle Linux this is quite conveniently setup. In the folder /etc/sysconfig you find a few config scripts, amongst others a script called: docker-storage. This is a proper to add options. When you set the DOCKER_STORAGE_OPTIONS variable, it is added to the command line. So we simply need to add the line:
    DOCKER_STORAGE_OPTIONS = --graph="/app/docker/data" --storage-driver=overlay2

    , to the file /etc/sysconfig/docker-storage. This can be done with the following snippet:
    #
    DOCKER_STORAGE_CFG=/etc/sysconfig/docker-storage
    sudo sh -c "echo 'DOCKER_STORAGE_OPTIONS = --graph=\"${DOCKER_DATA_HOME}\" --storage-driver=${DOCKER_STORAGE_DRVR}' >> ${DOCKER_STORAGE_CFG}"

    And then finish up with starting docker service again:
    #
    echo 5.4 Reload deamon
    sudo systemctl daemon-reload 
    echo 5.5 Start docker again
    sudo systemctl start docker

    Conclusion

    Last week I was at the #PaaSSummerCamp in Lisbon. I did some labs with my docker installation, that resulted in the permission problem. As mentioned I resolved that and I could run the labs succesfully with docker containers from Oracle. So, I concluded that this script should suffice. You can download the complete script at my GitHub vagrant repo.