I have got questions on performance of OSB quite a few times already, during the years. A few years ago on a project I got eyes on a set of recommendations on workmanagers for OSB. Many developers know that for instance Service Call outs are blocking activities. And that you should use workmanagers to solve performance problems resulting from the use of those blocking activities.

If you do nothing on dispatch policies in OSB proxy or business services, all is done in the Default Workmanager. But since some constructions, not only service call-outs, need other threads to finish the job, you can get stuck threads because the workmanager's threadpool gets empty having all or near to all threads waiting, leaving no threads to pickup work to free the others.

More on this in the terrific blog of Anthony Reynolds on the subject: Following the thread.

By the way,  I've seen that some people by default use service call-outs for almost everything. But the default use should be the routing node with a route activity. Even in some services you need to gather information from several sources, while you can only have one route node, pick or choose a 'driving-service' to use in the Routing node. Just like creating a query on several tables where you have to choose a 'driving-table'. Then  use service call outs only to do the extra enrichment.

From that earlier project I got the following recommendations, based on the blog of Anthony Reynolds. Since I refer back to it regularly, I think it would be good to share it.

For OSB to work optimally and prevent floading WebLogic’s threadpool with hogged/stuck threads you should create 3 FairShareRequest classes in ratio of 33/33/33, to distinguish different “kinds of threadpools”.

Then create 4 workmanagers:

• FTPPollingWorkManager: file based inbound OSB proxy services. Polling a filesystem (or FTP). Uses FairShareReqClass-1, and ignores stuck threads.
• InboudWorkManager: inbound OSB proxy services, not polling file based. Also uses FairShareReqClass-1, not ignoring stuck threads.
• CallOutWorkManager: Service Call Out operations in a OSB proxy. Uses FairShareReqClass-2.
• DeliveryWorkManager: outbound business services in OSB. Uses FairShareReqClass-3.

Use these as a dispatch policy in the particular Proxy/Business Services. In OSB 11g this is:
I don't have screendumps of 12c at hand. But the idea would be the same there. I haven't learned that the thread model in 12c is architecturally different.

This week I got involved in a question on the Oracle Forums on getting the hostname of the server executing the bpel process. In itself this is not possible in BPEL. Also if you have a long running async process, the process gets dehydrated at several points (at a receive, wait, etc.). After an incoming signal, another server could process it further. You can't be sure that one server will process it to the end.

However, using Java, you can get the hostname of an executing server, quite easily. @AnatoliAtanasov suggested this question on stackOverflow. I thought that it would be fun to try this out.

Although you can opt for creating an embedded java activity, I used my earlier article on SOA and Spring Contexts to have it in a separate bean. By the way, in contrast to my suggestions in the article, you don't have to create a separate spring context for every bean you use.

My java bean looks like:

package nl.darwinit.soasuite;
import java.net.InetAddress;
import java.net.UnknownHostException;


public class ServerHostBeanImpl implements IServerHostBean {
    public ServerHostBeanImpl() {
        super();
    }
    
    public  String getHostName(String hostNameDefault){
        String hostName;
        try
        {
            InetAddress addr;
            addr = InetAddress.getLocalHost();
            hostName = addr.getHostName();
        }
        catch (UnknownHostException ex)
        {
            System.out.println("Hostname can not be resolved");
            hostName = hostNameDefault;
        }
        return hostName;
    }
    
}

The interface class I generated is:

package nl.darwinit.soasuite;

public interface IServerHostBean {
    String getHostName(String hostNameDefault);
}

Then I defined a Spring Context, getHostNameContext, with the following content

<?xml version="1.0" encoding="UTF-8" ?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:util="http://www.springframework.org/schema/util"
       xmlns:jee="http://www.springframework.org/schema/jee" xmlns:lang="http://www.springframework.org/schema/lang"
       xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:sca="http://xmlns.oracle.com/weblogic/weblogic-sca" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tool http://www.springframework.org/schema/tool/spring-tool.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd http://www.springframework.org/schema/jms http://www.springframework.org/schema/jms/spring-jms.xsd http://www.springframework.org/schema/oxm http://www.springframework.org/schema/oxm/spring-oxm.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd http://xmlns.oracle.com/weblogic/weblogic-sca META-INF/weblogic-sca.xsd">
    <!--Spring Bean definitions go here-->
    <sca:service name="GetHostService" target="ServerHostBeanImpl" type="nl.darwinit.soasuite.IServerHostBean"/>
    <bean id="ServerHostBeanImpl" class="nl.darwinit.soasuite.ServerHostBeanImpl"/>
</beans>

After wiring the context to my BPEL the composite looks like:


Then, deploying and running it, gives the following output:


Nice, isn't it?

Let's say you have a virtual machine with two network adapters, both set on 'HostOnly'.
I used to do that and set the first one of those to a fixed IP address, say 10.0.0.1. To this one I coupled the hostname, for instance darlin-vce-db, using the /etc/hosts file. That way I had a fixed, always existing network address for the database.

Together with the database, you install WebLogic, for instance to serve SOASuite, or OSB, or whatever custom application you want to serve. Now, wouldn't it be nice to be able to use WebLogic from a browser out of the virtual machine? Of course, because this is what you do nowadays: almost everywhere I come, servers are hosted on virtualized computing environments, like VMWare VSE or Oracle VM. So that's where the second adapter comes in, dynamically coupled to an address in the form of 192.168.56.101, for instance. Externally, using the etc/host file on your host OS (in my case Windows), you couple that address also to darlin-vce-db.

So you have two /etc/hosts settings for the hostname, darlin-vce-db:
In internally in the VM:

10.0.0.1        darlin-vce-db     darlin-vce-db.darwin-it.local

And externally on your host OS:

192.168.56.101  darlin-vce-db     darlin-vce-db.darwin-it.local

Nothing special, right? Well, WebLogic 11g, apparently just listens to the hostname 'darlin-vce-db', if that is entered as a listen-address. It seems not to care if the request for 'darlin-vce-db' comes from 192.168.56.101 in stead of the 10.0.0.1 to where the hostname actually is bound.

However, in this particular case WebLogic 12c seems to behave differently. If you provice 'darlin-vce-db' as listen address, that is bound to a network adapter that has 10.0.0.1 it expects that requests also come in via that network adapter. It seems to ignore requests that come in via other adapters (in my case 192.168.56.101).

You can solve this partially using Channels: in the Weblogic Console, navigate to the particular managed server, click Protocols, Channels.
Create a new channel:

Give it a name like 'Extern-Intern' or something else properly denoting the purpose of it and choose 'http' as a protocol:

Then Provide the internal address, for instance 'darlin-vce-db', and the external listen address:

Leave the ports to the default listen-port, in this case. Then finish the wizard.
Although this helps in connecting to the WebLogic console, EM, or with the same method on the SOA Server, to the SOA Composer (soaserver:port/soa/composer), BPM Workspace (soaserver:port/bpm/composer) etc., this will not work for JDeveloper.

When trying to deploy a soa composite from JDeveloper, you define/choose a ApplicationServer with connection to the AdminServer. But in case of deploying a composite, the AdminServer figures out which running SOA Servers there are, and let JDeveloper provide the composite to those servers. But then the soaserver(s) refuse(s) the connections from JDeveloper. Testing the ApplicationServer connection will show success for the Http connection to the AdminServer, but fails for all the other components.

The solution is then to denote a particular Network Adapter/ip address and make sure that internally and externally the particular hostname is coupled to that same particular ip-address.

Unfortunately I noticed this nice initiative only yesterday: OTN Appreciation Day. I did not had a change to cook something up, but I do like to add some mustard after the meal, as we say in Dutch.

In the titles of the 'OTN Appreciation Day'-blogs I miss BPEL. In 2004,when Oracle acquired Collaxa, I worked at Oracle Consulting in the Netherlands. I worked with Oracle Workflow and Interconnect. Oracle wasn't yet into SOA really. But with BPEL PM they acquired the tool that stood at the base of SOA Suite, together with Webservices Manager and what we now know as the Mediator. And it changed my professional life, really. Its extremely powerful, especially with the added  JCA Adapters, xslt-mapper, and since 11g the SCA architecture wich enables you to assemble composite applications with Adapters BusinessRules, Human Workflow, Mediator, Spring Components and of course ... BPEL 2.0.

Sorry, Tim, for being late.

At my current customer I have SQLDeveloper open the whole day, and regularly I come back to it to query my throttle-table to see if my requests have been picked up. But regularly my database connection have been broken because of being idle. Probably because of a nasty firewall between my remote development desktop and the database.

Googling on it I found an article of That Jeff Smith on busy connections. That blog is really one to follow. Feed it to your Feedly if your a recent SQL Developer user.

But in my search I found the 'keepAlive-4' extension for SQL Developer 4. Download the KeepAlive zip, go to Help->Check for Updates in SQLDeveloper and install it via the 'Install From Local File' option. Then look for the Keep Alive icon in the toolbar:
Click on it and enter a check frequency, of at least 60 seconds. I try 180 seconds.

You can disable it by clicking it again. Another click and it asks for a new interval again.

To day I ran into a trap that trapped me before...

I tried to edit a file adapter service in SOA/BPM QuickStart 12.2.1. And got the error 'Service Directory is not valid' in a pop-up dialog, that prevents me from editing.

The problem is described in this forum thread.

The problem is a space in the path to the JDeveloper project. Now spaces are a drag in filenames and paths (I hate it that Microsoft Windows introducced "Program Files" as the default folder for installing applications).

The solution is in Windows quite simple actually: create a substitute drive like:

echo define drive S: referring to current folder
subst s: /d
subst s: "d:\Data\svn\Applicatie Integration"

Put this in a .bat file and run it before starting JDeveloper and load the application workspaces from the S: drive.

I had this already in place because of my first run into this trap. Why I tripped again? It turned out I accidentally had opened the application from the d: drive again...

To day I had some problems with deploying my composite to the server of my current customer. Apparently the Server had some problems with datasources. But since there are many I did not feel much for checking them one by one with the console. Using Google I got the following examples

• Testing DataSource Status using WLST: but this one only uses the current JDBCServiceRuntine, thus that of the AdminServer
• WebLogic DataSource Monitoring Using WLST: this one lists all the properties of all the datasources of all the running servers

Thus I had to combine those two, sauce it with my own way of wlst-scripting (see my other blog entries). Also I want a tabular form, that got me into troubles with printing the result of the testPool() method. But I came up with the following script, testDS.py:

#############################################################################
# Test Datasources on a domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-10-04
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
  print 'Call script as: '
  print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
  print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
  print 'Property file should contain the following properties: '
  print "adminUrl=darlin-vce:7001"
  print "adminUser=weblogic"
  print "adminPwd=welcome1"
#
#
def connectToadminServer(adminUrl, adminServerName):
  print(lineSeperator)
  print('Try to connect to the AdminServer')
  try:
    connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
  except NameError, e:
    print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
    print('Try to connect to the AdminServer adminUser and adminPwd properties')
    connect(adminUser, adminPwd, adminUrl)
#
#
def main():
  try:
    pad='                                                                               '
    print(lineSeperator)
    print('Check datasources for domain')
    print(lineSeperator)
    print ('Connect to the AdminServer: '+adminServerName)
    connectToadminServer(adminUrl, adminServerName)
    print(lineSeperator)
    allServers=domainRuntimeService.getServerRuntimes();
    if (len(allServers) > 0):
      for tempServer in allServers:
        jdbcServiceRT = tempServer.getJDBCServiceRuntime();
        dataSources = jdbcServiceRT.getJDBCDataSourceRuntimeMBeans();
        print('\nServer '+tempServer.getName())
        if (len(dataSources) > 0):
          print('Datasource                                                  '[:30]+'\tState\tTest')
          for dataSource in dataSources:
            testPool = dataSource.testPool()
            dataSourceName = dataSource.getName()+pad
            dataSourceNamePad=dataSourceName[:30]
            if (testPool == None):
              print dataSourceNamePad+'\t'+dataSource.getState()+'\tOK'
            else:
              print dataSourceNamePad+'\t'+dataSource.getState()+'\tFailure: '
              print testPool
    print(lineSeperator)
    print('Done...')
    print(lineSeperator)
  except NameError, e:
    print('Apparently properties not set.')
    print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
    usage()
  except:
    apply(traceback.print_exception, sys.exc_info())
    exit(exitcode=1)
#
main();

Of course it's easy to extent the table with properties from the monitor script in WebLogic DataSource Monitoring Using WLST.

Which runs pretty neat. Run it with a shell script like the following testDS.sh script:

#!/bin/bash
#############################################################################
# Test DataSources  using wlst.
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-06-27
#
#############################################################################
#
. fmw12c_env.sh
echo
echo Test Datasources
wlst.sh ./testDS.py -loadProperties fmw.properties

For the fmw12c_env.sh and fmw.properties files look here.