Our most appreciated Oracle ACE Director Tim Hall organizes this yearly initiative, with this years name Oracle Ground Breakers Appreciation Day, and appointed this day to blog about our favorite Oracle Technology, Service or sub-community.

Last week I presented the 'Oracle Kubernetes Managed Weblogic Revival', the introduction of the Weblogic Kubernetes Operator opens the future for Weblogic.

This week I deliver our Weblogic 12c Tuning and Troubleshooting training for ATOS The Netherlands in Groningen. So, hmmm. what to blog, on this years Ground Breakers Appreciation day? There are several other technologies that I use and follow, but mostly around Fusion Middleware: SOA Suite, BPM Suite and Oracle Service Bus. But also Oracle Integration Cloud, that in fact heavily depend on this technologies. And honestly, bottom line here is Oracle Weblogic.

I frequently hear voices that state that Customers should move away from Weblogic. Honestly, I don't relate to that. It has served customers very well over the last decade under the Oracle brand and before. And I still think it was a smart move of Oracle to acquire it and make it a strategic part of the Oracle platform.

Last few years I've been active on the community.oracle.com forums, where I've grown to level 13, almost level 14, by answering questions and participating in discussions around Fusion Middleware technologies. My first thank therefor is to this community, for having me participating.

My second thank goes to the whole Weblogic and related Fusion Middleware toolstack. During the Tuning and Troubleshooting training I again realize how smart and rich the Weblogic Suite is. Although, I stated before that Oracle could do something about the footprint. It seems to me that there are quite some duplicate libraries or different versions of the same library. And maybe some old parts could be cut out: maybe only support SAML2.0 and improve that, for instance.


One great, but quite rarely used feature of Weblogic is the Weblogic Diagnostic Framework. And especially the Policies and Actions part. It is quite difficult to configure, the console's UI does not help here and there, and to think of usages of it. However, every time I present it, I find myself thinking: I should use this more often in my daily developments.

So I started to create a wlst script to create a Diagnostic Module, create a few collectors,  a JMS Notification Action and 2 policies on it.  It is actually the solution to Lab 6 of our training. To me it is a start to be able to expand this. You could  create a version per technology: OSB, SOA Suite, or custom application like MedRec. And you can create a more generic version that based on different property files configure different collectors, policies and actions specific for that target environment.

WLDF Diagnostic Module

The script first creates a diagnostic module like this:

def createDiagnosticModule(diagModuleName, targetServerName):
  module=getMBean('/WLDFSystemResources/'+diagModuleName)
  if module==None:
    print 'Create new Diagnostic Module'+diagModuleName
    edit()
    startEdit()
    cd('/')
    module = cmo.createWLDFSystemResource(diagModuleName)
    targetServer=getMServer(targetServerName)
    module.addTarget(targetServer)
    # Activate changes
    save()
    activate(block='true')
    print 'Diagnostic Module created successfully.'
  else:
    print 'Diagnostic Module'+diagModuleName+' already exists!'
  return module

It checks if the Diagnostic Module already exists as a WLDFSystemResource. If not, it will create it as module = cmo.createWLDFSystemResource(diagModuleName) and target it to a targetServer.

Collectors

Then for creating a collector I created the following function:

def createCollector(diagModuleName, metricType, namespace, harvestedInstances,attributesCsv):
  harvesterName='/WLDFSystemResources/'+diagModuleName+'/WLDFResource/'+diagModuleName+'/Harvester/'+diagModuleName 
  harvestedTypesPath=harvesterName+'/HarvestedTypes/';
  print 'Check Collector '+harvestedTypesPath+metricType
  collector=getMBean(harvestedTypesPath+metricType)
  if collector==None:
    print 'Create new Collector for '+metricType+' in '+diagModuleName
    edit()
    startEdit()
    cd(harvestedTypesPath)
    collector=cmo.createHarvestedType(metricType)
    cd(harvestedTypesPath+metricType)
    attributeArray=jarray.array([String(x.strip()) for x in attributesCsv.split(',')], String)
    collector.setHarvestedAttributes(attributeArray)
    collector.setHarvestedInstances(harvestedInstances)
    collector.setNamespace(namespace)
    # Activate changes
    save()
    activate(block='true')
    print 'Collector created successfully.'
  else:
    print 'Collector '+metricType+' in '+diagModuleName+' already exists!'
  return collector

Again, it first checks for the existing of the collector as a so called HarvestedType, within a WLDFResource in the Diagnostic Module. If not it creates it. Here you need to provide the metricType as a HavervestedType. And then attributes that you want to collect. The function expects it as a comma separated values string, that it converts to an array via a List.
Then you can provide Metric Type Instances or None if you want to collect it over all instances.

You can call this as:

createCollector(diagModuleName, 'weblogic.management.runtime.JDBCDataSourceRuntimeMBean','ServerRuntime', None, 'ActiveConnectionsCurrentCount,CurrCapacity,LeakedConnectionCount')

or if you want to add instances, it's also done by creating an array:

    harvestedInstancesList=[]
    harvestedInstancesList.append('com.bea:ApplicationRuntime=medrec,Name=TTServer_/medrec,ServerRuntime=TTServer,Type=WebAppComponentRuntime')
    harvestedInstances=jarray.array([String(x.strip()) for x in harvestedInstancesList], String)    
    createCollector(diagModuleName, 'weblogic.management.runtime.WebAppComponentRuntimeMBean','ServerRuntime', harvestedInstances,'OpenSessionsCurrentCount')

This is a bit more complicated, since the strings describing the instances that you want to add are comma seperated values them selfs.

Actions

Creating an action is again pretty simple, for a JMS Notification that is:

def createJmsNotificationAction(diagModuleName, actionName, destination, connectionFactory):
  policiesActionsPath='/WLDFSystemResources/'+diagModuleName+'/WLDFResource/'+diagModuleName+'/WatchNotification/'+diagModuleName
  jmsNotificationPath=policiesActionsPath+'/JMSNotifications/'
  print 'Check notification action '+jmsNotificationPath+actionName
  jmsNtfAction=getMBean(jmsNotificationPath+actionName)
  if jmsNtfAction==None:
    print 'Create new JMS NotificationAction '+actionName+' in '+diagModuleName
    edit()
    startEdit()
    cd(policiesActionsPath)
    jmsNtfAction=cmo.createJMSNotification(actionName)
    jmsNtfAction.setEnabled(true)
    jmsNtfAction.setTimeout(0)
    jmsNtfAction.setDestinationJNDIName(destination)
    jmsNtfAction.setConnectionFactoryJNDIName(connectionFactory)
    # Activate changes
    save()
    activate(block='true')
    print 'JMS NotificationAction created successfully.'
  else:
    print 'JMS NotificationAction '+actionName+' in '+diagModuleName+' already exists!'
  return jmsNtfAction

There are different types of actions, so they're created differently. You can add one using the console and record that. It's what I did and then transformed the recorded script to functions as shown here.

Policies

Policies can be created with the following function. You need to provide a rule type and a rule expression, plus a array of actions you want to add:

def createPolicy(diagModuleName, policyName, ruleType, ruleExpression, actions):  
  policiesActionsPath='/WLDFSystemResources/'+diagModuleName+'/WLDFResource/'+diagModuleName+'/WatchNotification/'+diagModuleName
  policiesPath=policiesActionsPath+'/Watches/'
  print 'Check Policy '+policiesPath +policyName
  policy=getMBean(policiesPath +policyName)
  if policy==None:
    print 'Create new Policy '+policyName+' in '+diagModuleName
    edit()
    startEdit()
    cd(policiesActionsPath)
    policy=cmo.createWatch(policyName)
    policy.setEnabled(true)
    policy.setExpressionLanguage('EL')
    policy.setRuleType(ruleType)
    policy.setRuleExpression(ruleExpression)
    policy.setAlarmType('AutomaticReset')
    policy.setAlarmResetPeriod(300000)
    cd(policiesPath +policyName)
    set('Notifications', actions)
    schedule=getMBean(policiesPath +policyName+'/Schedule/'+policyName)
    schedule.setMinute('*')
    schedule.setSecond('*')
    schedule.setSecond('*/15')
    # Activate changes
    save()
    activate(block='true')
    print 'Policy created successfully.'
  else:
    print 'Policy '+policyName+' in '+diagModuleName+' already exists!'
  return policy

An example of calling this is:

actionsList=[]
    actionsList.append('com.bea:Name=JMSAction,Type=weblogic.diagnostics.descriptor.WLDFJMSNotificationBean,Parent=[TTDomain]/WLDFSystemResources[TTDiagnostics],Path=WLDFResource[TTDiagnostics]/WatchNotification[TTDiagnostics]/JMSNotifications[JMSAction]')
    actions=jarray.array([ObjectName(action.strip()) for action in actionsList], ObjectName)    
    createPolicy(diagModuleName,'HiStuckThreads', 'Harvester', 'wls:ServerHighStuckThreads(\"30 seconds\",\"10 minutes\",5)', actions)

As you can see the actions to add are actually expressions to the MBeans of the actions configured earlier. It apparently depend on the type and the diagnostic module that contains it. So I could create a function that assembles this expression. If you want a custom rule expression you can create it as follows:

actionsList=[]
ruleExpression='wls:ServerGenericMetricRule(\"com.bea:Name=MedRecGlobalDataSourceXA,ServerRuntime=TTServer,Type=JDBCDataSourceRuntime\",\"WaitingForConnectionHighCount\",\">\",0,\"30 seconds\",\"10 minutes\")'
    createPolicy(diagModuleName,'OverloadedDS', 'Harvester', ruleExpression, actions)

Again this is an expression that could be assembled using a function.

Conclusion

The complete script can be reviewed and downloaded from my GitHub Repo.

I hit two flies with one beat: Thank you Ground Breakers, fellow ACEs and other Oracle enthousiasts, and I guess my first article about the Weblogic Diagnostic Framework (but not my first one to include WLST scripts...). Happy OGB Appreciation Day y'all!