A year or two, maybe three ago I found this teriffic article on Fusion Middleware 12c topology suggestions.
I find my self searching for this one from time to time. So it's time to write this little note.
It explains which combinations of FMW products in a domain makes sense and especially what the different Server Groups mean.
Showing posts with label Oracle Service Bus. Show all posts
Showing posts with label Oracle Service Bus. Show all posts
Tuesday, 11 September 2018
Wednesday, 20 December 2017
OSB 12c Customization in WLST, some new insights: use the right jar for the job!
Problem setting and investigation
Years ago I created a Release & Deploy framework for Fusion Middleware, also supporting Oracle Service Bus. Recently revamped it to use 12c. It uses WLST to import the OSB service to the Service Bus, including the execution the customization file.There are lots of examples to do this, but I want to zoom in on the execution of the customization file.
The WLST function that does this, that I use is as follows:
#=======================================================================================
# Function to execute the customization file.
#=======================================================================================
def executeCustomization(ALSBConfigurationMBean, createdRefList, customizationFile):
if customizationFile!=None:
print 'Loading customization File', customizationFile
inputStream = FileInputStream(customizationFile)
if inputStream != None:
customizationList = Customization.fromXML(inputStream)
if customizationList != None:
filteredCustomizationList = ArrayList()
setRef = HashSet(createdRefList)
print 'Filter to remove None customizations'
print "-----"
# Apply a filter to all the customizations to narrow the target to the created resources
print 'Number of customizations in list: ', customizationList.size()
for customization in customizationList:
print "Add customization to list: "
if customization != None:
print 'Customization: ', customization, " - ", customization.getDescription()
newCustomization = customization.clone(setRef)
filteredCustomizationList.add(newCustomization)
else:
print "Customization is None!"
print "-----"
print 'Number of resulting customizations in list: ', filteredCustomizationList.size()
ALSBConfigurationMBean.customize(filteredCustomizationList)
else:
print 'CustomizationList is null!'
else:
print 'Input Stream for customization file is null!'
else:
print 'No customization File provided, skip customization.'
The parameter ALSBConfigurationMBean can be fetched with:
...
sessionName = createSessionName()
print 'Created session', sessionName
SessionMBean = getSessionManagementMBean(sessionName)
print 'SessionMBean started session'
ALSBConfigurationMBean = findService(String("ALSBConfiguration.").concat(sessionName), "com.bea.wli.sb.management.configuration.ALSBConfigurationMBean")
...
The other parameter is the createdRefList, that is build up from the default ImportPlan during import of the config jar:
...
print 'ÒSB project', project, 'will get updated'
osbJarInfo = ALSBConfigurationMBean.getImportJarInfo()
osbImportPlan = osbJarInfo.getDefaultImportPlan()
osbImportPlan.setPassphrase(passphrase)
operationMap=HashMap()
operationMap = osbImportPlan.getOperations()
print
print 'Default importPlan'
printOpMap(operationMap)
set = operationMap.entrySet()
osbImportPlan.setPreserveExistingEnvValues(true)
#boolean
abort = false
#list of created artifact refences
createdRefList = ArrayList()
for entry in set:
ref = entry.getKey()
op = entry.getValue()
#set different logic based on the resource type
type = ref.getTypeId
if type == Refs.SERVICE_ACCOUNT_TYPE or type == Refs.SERVICE_PROVIDER_TYPE:
if op.getOperation() == ALSBImportOperation.Operation.Create:
print 'Unable to import a service account or a service provider on a target system', ref
abort = true
else:
#keep the list of created resources
print 'ref: ',ref
createdRefList.add(ref)
if abort == true :
print 'This jar must be imported manually to resolve the service account and service provider dependencies'
SessionMBean.discardSession(sessionName)
raise
print
print 'Modified importPlan'
printOpMap(operationMap)
importResult = ALSBConfigurationMBean.importUploaded(osbImportPlan)
printDiagMap(importResult.getImportDiagnostics())
if importResult.getFailed().isEmpty() == false:
print 'One or more resources could not be imported properly'
raise
...
The meaning is to build up a set of references of created artefact, to narrow down the customizations to only execute them on the artefacts that are actually imported.
Now, back to the executeCustomization function. It first creates an InputStream on the customization file:
inputStream = FileInputStream(customizationFile)
on which it builds a list of customizations using the .fromXML method of the Customization object:
customizationList = Customization.fromXML(inputStream)
These customizations are interpreted from the Customization file. If you open that you can find several customization elements:
<cus:customization xsi:type="cus:EnvValueActionsCustomizationType">
<cus:description/>
...
<cus:customization xsi:type="cus:FindAndReplaceCustomizationType">
<cus:description/>
...
<cus:customization xsi:type="cus:ReferenceCustomizationType">
<cus:description/>
These all are mapped to subclasses of the Customization. And now the reason that I write this blogpost is that I ran into a problem with my import tooling. In the EnvValueActionsCustomizationType the endpoint replacements for the target environments is done. And the weren't executed. In fact these customizations were in the customizationList, but as a None/Null object. Thus, executing this complete list using ALSBConfigurationMBean.customize(filteredCustomizationList) would run in an exception, refering to a null object in the customization list. That's why they're filtered out. But why weren't these interpreted by the .fromXml() method?
Strangely enough in the javaAPI docs of 12.2.1 the EnvValueActionsCustomization does not exist, but the EnvValueCustomization does. But searching My Oracle Support shows in Note 1679528.2: 'A new customization type EnvValueActionsCustomizationType is available in 12c which is used when creating a configuration plan file.' and here in the Java API doc (click on com.bea.wli.config.customization) it is stated that EnvValueCustomization is deprecated and EnvValueActionsCustomization should be used in stead.
Apparently the docs is not updated completely....
And it seems that I used a wrong jar file: The customization file was created using the Console, and executing the customization file using the console did execute the endpoint replacements. So I figured that I must be using a wrong version of the jar file.
So I searched on my BPM quickstart installation (12.2.1.2) for the class EnvValueCustomization:
Jar files containing EnvValueCustomization
- C:\Oracle\JDeveloper\12210_BPMQS\osb\lib\modules\oracle.servicebus.configfwk.jar/com\bea\wli\config\customization\EnvValueCustomization.class
- C:\Oracle\JDeveloper\12210_BPMQS\oep\spark\lib\spark-osa.jar/com\bea\wli\config\customization\EnvValueCustomization.class
- C:\Oracle\JDeveloper\12210_BPMQS\oep\common\modules\com.bea.common.configfwk_1.3.0.0.jar/com\bea\wli\config\customization\EnvValueCustomization.class
Jar files containing EnvValueActionsCustomization:
- C:\Oracle\JDeveloper\12210_BPMQS\osb\lib\modules\oracle.servicebus.configfwk.jar/com\bea\wli\config\customization\EnvValueActionsCustomization.class
Solution
It turns out that in my ANT script I used:<path id="library.osb">
<fileset dir="${fmw.home}/oep/common/modules">
<include name="com.bea.common.configfwk_1.3.0.0.jar"/>
</fileset>
<fileset dir="${weblogic.home}/server/lib">
<include name="weblogic.jar"/>
<include name="wls-api.jar"/>
</fileset>
<fileset dir="${osb.home}/lib">
<include name="alsb.jar"/>
</fileset>
</path>
Where I should use:
<path id="library.osb">
<fileset dir="${fmw.home}/osb/lib/modules">
<include name="oracle.servicebus.configfwk.jar"/>
</fileset>
<fileset dir="${weblogic.home}/server/lib">
<include name="weblogic.jar"/>
<include name="wls-api.jar"/>
</fileset>
<fileset dir="${osb.home}/lib">
<include name="alsb.jar"/>
</fileset>
</path>
Conclusion
It took me quite some time to debug this. But learned how the customization works. I found quite some examples that use com.bea.common.configfwk_1.X.0.0.jar. And apparently during my revamping, I updated this class path (actually I had 1.7, and found only 1.3 in my environment). But, somehow Oracle found it sensible to replace it with oracle.servicebus.configfwk.jar while keeping the old jar files.So use the right Jar for the job!
Friday, 1 December 2017
OSB: Disable Chunked Streaming Mode recommendation
Intro
These weeks I got involved in a document generation performance issue. This ran for several months, maybe years even. But it stayed quite unclear what the actual issue was.Often we got complaints that document generation from the front-end application (based on Siebel) was taking very long. End users often hit the button several times, but with no luck. Asking further, it did not mean that there appeared a document in the content management system (Oracle UCM/WCC). So, we concluded that it wasn't so much a performance issue, but an exception along the process of document generation. Since we upgraded BI Publisher to 12c, it was figured that it might got something to do with that. But we did not find any problems with BI Publisher, itself. Also, there was an issue with Siebel it's self, but that's also out of the scope of this article.
The investigation
First, on OSB the retry interval of the particular Business Service was decreased from 60 seconds to 10. And the performance increased. Since the retry interval was shorter, OSB does a retry on shorter notice. But of course this did not solve the problem.As Service developers we often are quite laconical about retries. We make up some settings. Quite default is an interval of 30 seconds and a retry count of 3. But, we should actually think about this and figure out what the possible failures could be and what a sensible retry setting would be. For instance: is it likely that the remote system is out of order? What are the SLA's for hoisting it back up again? If the system startup is 10 minutes, then a retry count of 3 and interval of 30 seconds is not making sense. The retries are done long before the system's up again. But of course, in our case sensible settings for system outage would cause delays being too long. We apparently needed to cater for network issues.
Last week our sysadmins encountered network failures, so they changed the LoadBalancer of BIP Publisher, to get chunks/packets of one requests routed to the same BI Publisher node. I found SocketReadTimeOuts in the logfiles. And from the Siebel database a query was done and plotted out in Excel showing lots of request in the 1-15 seconds range, but also some plots in ranges around 40 seconds and 80 seconds. We wondered why these were.
The Connection and Read TimeOut settings on the Business Service were set to 30s. So I figured the 40 and 80 seconds range could have something to do with a retry interval of 10s added to a time out of 30 seconds.
I soon found out that in OSB on the Business Service, the Chunked Streaming Mode was enabled. This is a setting we struggled with a lot. Several issues we encountered were blamed on this one. As a Helpdesk employee would ask you if you have restarted your system, on OSB questions I would ask you about this setting first... Actually, I did for this case, long before I got actively involved.
Chunked Streaming Mode explained
Let's start with a diagram:Now, the Chunked transfer encoding is an HTTP 1.1 specification. It is an improvement that allows clients to process the data in chunks right after the chunk is read. But in most (of our) cases a chunk on itself is meaning-less, since a SOAP Request/XML Document need to be parsed as a whole.
The Load Balancer also process the chunks as separate entities. So,by default, it will route the first one to the first endpoint, and the other one to the next. And thus each SP Managed Server gets an incomplete message and there for a so-called Bad Request. This happens with big requests, where for instance a report is requested together with the complete content. Then chances are that the request is split up in chunks.
But although the SysAdmins adapted the SP Load Balancer, and although I was involved in the BIPublisher 12c setup, even I forgot about the BIP12c OHS! And even when the LoadBalancer tries to keep the chunks together, then again the OHS will mess with them. Actually, if the LoadBalancer did not keep them together, the OHS instances could reroute them again to the correct end-node.
The Solution
So for all those Service Bus developers amongst you, I'd like you to memorize two concepts: "Chunked Streaming Mode" and "disable", and the latter in combination with the first, of course.In short: remember to set Chunked Streaming Mode to disable in every SOAP/http based Business Service. Especially with services that send potentially large requests, for instance document check-in services on Content/Document Management Systems.
The proof of the pudding
After some discussion and not being able to test it on the Acceptance Test environment, due to rebuilds, we decided to change this in production (I would/should not recommend that, at least not right away).And this was the result:
This picture shows that the first half of the day, plenty requests were retried at least once, and several even twice. Notice the request durations around the 40 seconds (30 seconds read timeout + 10 seconds retry interval) and 80 seconds. But since 12:45, when we disabled the Chunked Streaming Mode we don't see any timeout exceptions any more. I hope the end users are happy now.
Or how a simple setting can throw a spanner in the works. And how difficult it is to get such a simple change into production. Personally I think it's a pity that the Chunked Streaming Mode is enabled by default, since in most cases it causes problems, while in rare cases it might provide some performance improvements. I think you should rationalize the enablement of it, in stead of actively needing to disable it.
Tuesday, 4 July 2017
Weblogic log level mapping
For quite some time now I wondered about the differences in log-levels. For instance, if you configure the log levels in the classes in SB12c or SOA Suite, You see levels as INCIDENT , ERROR, and TRACE, even with several sub leverls (1, 16, 32). But on the Server Log configuration in Weblogic, you see levels a s ERROR, NOTICE and DEBUG, or TRACE. And then in Java, JDK logging we even have other log levels.
Today I experienced the urge to look it up and fout this nice document. It describes the configuration of logging in Fusion Middleware. It contains this nice table:
Today I experienced the urge to look it up and fout this nice document. It describes the configuration of logging in Fusion Middleware. It contains this nice table:
ODL
|
WebLogic Server
|
Java
|
| OFF | OFF | 2147483647 - OFF |
| INCIDENT_ERROR:1 | (EMERGENCY) | 1100 |
| INCIDENT_ERROR:4 | EMERGENCY | 1090 |
| INCIDENT_ERROR:14 | ALERT | 1060 |
| INCIDENT_ERROR:24 | CRITICAL | 1030 |
| ERROR:1 | (ERROR) | 1000 - SEVERE |
| ERROR:7 | ERROR | 980 |
| WARNING:1 | WARNING | 900 - WARNING |
| WARNING:7 | NOTICE | 880 |
| NOTIFICATION:1 | INFO | 800 – INFO |
| NOTIFICATION:16 | (DEBUG) | 700 - CONFIG |
| TRACE:1 | (DEBUG) | 500 – FINE |
| TRACE:1 | DEBUG | 495 |
| TRACE:16 | (TRACE) | 400 - FINER |
| TRACE:32 | (TRACE) | 300 - FINEST |
| TRACE:32 | TRACE | 295 |
This helps quite neatly. Especially, because to have ODL logging get through to the Server log-files, you need to set the appropriate level on the Weblogic server.
Labels:
Java
,
Oracle Service Bus
,
SOA Suite
,
Weblogic
Thursday, 1 December 2016
Note to myself: when handling large payloads
Today I stumbled on a question in the communities about handling large payloads in BPEL/XSLT. Although I know that SOASuite from 11g onwards can do paging of XML to disk, I never had the need. However, you could need it from time to time. And it's good to know how to do it.
It's noted on My Oracle Support with Doc ID 1327970.1. Which refers to the 11g documentation on Managing Large Documents and Large Numbers of Instances.
Learning all the time....
It's noted on My Oracle Support with Doc ID 1327970.1. Which refers to the 11g documentation on Managing Large Documents and Large Numbers of Instances.
Learning all the time....
Monday, 31 October 2016
OSB Thread handling recommendations
I have got questions on performance of OSB quite a few times already, during the years. A few years ago on a project I got eyes on a set of recommendations on workmanagers for OSB. Many developers know that for instance Service Call outs are blocking activities. And that you should use workmanagers to solve performance problems resulting from the use of those blocking activities.
If you do nothing on dispatch policies in OSB proxy or business services, all is done in the Default Workmanager. But since some constructions, not only service call-outs, need other threads to finish the job, you can get stuck threads because the workmanager's threadpool gets empty having all or near to all threads waiting, leaving no threads to pickup work to free the others.
More on this in the terrific blog of Anthony Reynolds on the subject: Following the thread.
By the way, I've seen that some people by default use service call-outs for almost everything. But the default use should be the routing node with a route activity. Even in some services you need to gather information from several sources, while you can only have one route node, pick or choose a 'driving-service' to use in the Routing node. Just like creating a query on several tables where you have to choose a 'driving-table'. Then use service call outs only to do the extra enrichment.
From that earlier project I got the following recommendations, based on the blog of Anthony Reynolds. Since I refer back to it regularly, I think it would be good to share it.
For OSB to work optimally and prevent floading WebLogic’s threadpool with hogged/stuck threads you should create 3 FairShareRequest classes in ratio of 33/33/33, to distinguish different “kinds of threadpools”.
Then create 4 workmanagers:
I don't have screendumps of 12c at hand. But the idea would be the same there. I haven't learned that the thread model in 12c is architecturally different.
If you do nothing on dispatch policies in OSB proxy or business services, all is done in the Default Workmanager. But since some constructions, not only service call-outs, need other threads to finish the job, you can get stuck threads because the workmanager's threadpool gets empty having all or near to all threads waiting, leaving no threads to pickup work to free the others.
More on this in the terrific blog of Anthony Reynolds on the subject: Following the thread.
By the way, I've seen that some people by default use service call-outs for almost everything. But the default use should be the routing node with a route activity. Even in some services you need to gather information from several sources, while you can only have one route node, pick or choose a 'driving-service' to use in the Routing node. Just like creating a query on several tables where you have to choose a 'driving-table'. Then use service call outs only to do the extra enrichment.
From that earlier project I got the following recommendations, based on the blog of Anthony Reynolds. Since I refer back to it regularly, I think it would be good to share it.
For OSB to work optimally and prevent floading WebLogic’s threadpool with hogged/stuck threads you should create 3 FairShareRequest classes in ratio of 33/33/33, to distinguish different “kinds of threadpools”.
Then create 4 workmanagers:
- FTPPollingWorkManager: file based inbound OSB proxy services. Polling a filesystem (or FTP). Uses FairShareReqClass-1, and ignores stuck threads.
- InboudWorkManager: inbound OSB proxy services, not polling file based. Also uses FairShareReqClass-1, not ignoring stuck threads.
- CallOutWorkManager: Service Call Out operations in a OSB proxy. Uses FairShareReqClass-2.
- DeliveryWorkManager: outbound business services in OSB. Uses FairShareReqClass-3.
I don't have screendumps of 12c at hand. But the idea would be the same there. I haven't learned that the thread model in 12c is architecturally different.
Wednesday, 13 July 2016
Create WebLogic users for ServiceBus 12c - Part 2
Last week I wrote a blog about how to create WebLogic users for ServiceBus 12c. However, I did not now how to assign a particular Application Specific Role to the weblogic user, for particular ServiceBus privileges. I did find out what particular Roles there were (see the blog). But how to assign them I found out just today.
So here it is:
Add this to the createUsers.py script in my previous article.
Then add the following call to add a role to a group:
This needs the following property in the property file:
In the comments I provided the possible values that are described in the docs (see again my previous article).
A description of the grantAppRole can be found here in the 11g docs.
The possible parameters of the function are:
With this you can enhance the createUsers script to create actual ServiceBus users. For SOASuite or other components you can get the Application Specific Roles by querying the Application Stripe in EM.
So here it is:
#
#
def grantOSBAppRoleToWlsGroup(osbAppRole, wlsGroup):
#
# Grant OSB AppRole
# http://docs.oracle.com/cd/E23943_01/web.1111/e13813/custom_infra_security.htm#WLSTC1398
# grantAppRole(appStripe, appRoleName,principalClass, principalName)
# appStripe: Specifies an application stripe.
# appRoleName: Specifies a role name.
# principalClass: Specifies the fully qualified name of a class.
# principalName: Specifies the principal name.
#grantAppRole("Service_Bus_Console","Monitor","oracle.security.jps.service.policystore.ApplicationRole","SBMonitor")
#grantAppRole("Service_Bus_Console","Tester","weblogic.security.principal.WLSUserImpl","weblogic")
try:
print('Grant OSB Role: '+osbAppRole+' to WebLogic Group: '+wlsGroup)
grantAppRole("Service_Bus_Console",osbAppRole,"weblogic.security.principal.WLSGroupImpl",wlsGroup)
print('Grant Succeeded')
except:
print('Failed to grant role '+ osbAppRole+' to '+wlsGroup+'.')
print('Check if role not already granted.')
Add this to the createUsers.py script in my previous article.
Then add the following call to add a role to a group:
#
# Grant AppRole
grantOSBAppRoleToWlsGroup(grpOsbDevOSBAppRole, grpOsbDevName)
This needs the following property in the property file:
# Possible App Roles: MiddlewareAdministrator, Developer, Composer, Deployer, Tester, Monitor, MiddlewareOperator, ApplicationOperator, APICurator grpOsbDevOSBAppRole=Developer
In the comments I provided the possible values that are described in the docs (see again my previous article).
A description of the grantAppRole can be found here in the 11g docs.
The possible parameters of the function are:
Argument
|
Definition
|
| appStripe | Specifies an application stripe. For SB12c it is: 'Service_Bus_Console'. You can get it from the pop-list in the EM, WebLogic Domain Menu->Security->Application Roles->Application Stripes Pull Down. (See here). |
| appRoleName | Specifies a role name. For SB12c this is one of: MiddlewareAdministrator, Developer, Composer, Deployer, Tester, Monitor, MiddlewareOperator, ApplicationOperator, APICurator |
| principalClass | Specifies the fully qualified name of a class. Unclear from the docs what to use. But I found (actually on a BI-EE blog):
|
| principalName | Specifies the principal name. |
With this you can enhance the createUsers script to create actual ServiceBus users. For SOASuite or other components you can get the Application Specific Roles by querying the Application Stripe in EM.
Labels:
FMWInstallation
,
Oracle Service Bus
,
SOA Suite
,
Weblogic
,
WLST
Monday, 11 July 2016
Reset your Datasources
In most SOASuite and Oracle ServiceBus projects the Database Adapter is used. And often it is used to invoke PL/Sql functions and procedures. Actually, it's my favorite interaction method with the database, since the re-entrancy of the database adapter wizard is best there. To do a re-entrant update of a select or DML operation when for instance a column is added, is simply put often quite problematic.
But the thing is with Pl/Sql that when you update a pl/sql package the package state is altered and when calling it from another session you'll get a 'ORA-04068: existing state of packages has been discarded' error. And this is will occur for every connection in the pool of your datasource.
The solution is to reset the datasource. This can be done easily in the WebLogic Administration Console (http://adminserver:port/console). But nowadays you can do it as easily in Enterprise Manager Fusion Middleware Control. This can be quite convenient for developers since often you have EM already open because of your SOASuite tests.
To do this open the Weblogic Domain menu and select the option 'JDBC DataSources':
Select the DataSource you want to reset, for the example I choose the 'EDNDataSource', but probably you'd not do this for one, but for a custom DataSource:
Click the Control tab, select the DataSource in the table and click Reset:
Reset will drive WebLogic to recreate all the connections in the DataSource. This prevents popping up the message multiple times.
The functionality in Configuring, Monitoring and Controlling the Datasource is similar as in the WebLogic Admin console. Only the layout is a little different.
But the thing is with Pl/Sql that when you update a pl/sql package the package state is altered and when calling it from another session you'll get a 'ORA-04068: existing state of packages has been discarded' error. And this is will occur for every connection in the pool of your datasource.
The solution is to reset the datasource. This can be done easily in the WebLogic Administration Console (http://adminserver:port/console). But nowadays you can do it as easily in Enterprise Manager Fusion Middleware Control. This can be quite convenient for developers since often you have EM already open because of your SOASuite tests.
To do this open the Weblogic Domain menu and select the option 'JDBC DataSources':
Select the DataSource you want to reset, for the example I choose the 'EDNDataSource', but probably you'd not do this for one, but for a custom DataSource:
Click the Control tab, select the DataSource in the table and click Reset:
Reset will drive WebLogic to recreate all the connections in the DataSource. This prevents popping up the message multiple times.
The functionality in Configuring, Monitoring and Controlling the Datasource is similar as in the WebLogic Admin console. Only the layout is a little different.
Wednesday, 6 July 2016
OSB 12c Logging part 2
Two weeks ago, I wrote about how to set the log level voor SB Pipelines (12c) to be able to see the logging of the Log activity in the WebLogic Server logs.
Today I encountered that for a developer at my customer the complete oracle.osb.logging.pipeline logger was missing in the log-configuration. So setting the level from EM (Fusion Middleware Control) following the article above is a little hard.
I could not find why in that case the logger was missing. But I did find a simple solution.
In the paragraph '7.1.4 ODL Log Configuration' of the Administering Oracle Service Bus documentation, I found that you can change the logging via the EM, wlst and the logging.xml file. This file can be found in ${osb.domain.home}/config/fmwconfig/servers/${osbserver.name}, eg. 'c:\Data\JDeveloper\SOA\system12.2.1.0.42.151011.0031\DefaultDomain\config\fmwconfig\servers\DefaultServer\'.
Go to the end of the file:
Copy the last logger and rename it to create an entry for the oracle.osb.logging.pipeline logger:
Set the level and remove the useParentHandlers attribute.
Restart your server and then you should find the option in the EM OSB Log Configuration. If you have multiple OSB servers you'd probably need to update this change for every osb server, since the logging.xml resides in a server specific sub-folder. I haven't tried it to add it for one server and change it to see if it is automatically added to the other server. Would be a nice experiment.
Today I encountered that for a developer at my customer the complete oracle.osb.logging.pipeline logger was missing in the log-configuration. So setting the level from EM (Fusion Middleware Control) following the article above is a little hard.
I could not find why in that case the logger was missing. But I did find a simple solution.
In the paragraph '7.1.4 ODL Log Configuration' of the Administering Oracle Service Bus documentation, I found that you can change the logging via the EM, wlst and the logging.xml file. This file can be found in ${osb.domain.home}/config/fmwconfig/servers/${osbserver.name}, eg. 'c:\Data\JDeveloper\SOA\system12.2.1.0.42.151011.0031\DefaultDomain\config\fmwconfig\servers\DefaultServer\'.
Go to the end of the file:
... <logger name='com.sun.xml.ws' level='WARNING' useParentHandlers='true'/> </loggers> </logging_configuration>
Copy the last logger and rename it to create an entry for the oracle.osb.logging.pipeline logger:
... <logger name='com.sun.xml.ws' level='WARNING' useParentHandlers='true'/> <logger name='oracle.osb.logging.pipeline' level='TRACE:16' /> </loggers> </logging_configuration>
Set the level and remove the useParentHandlers attribute.
Restart your server and then you should find the option in the EM OSB Log Configuration. If you have multiple OSB servers you'd probably need to update this change for every osb server, since the logging.xml resides in a server specific sub-folder. I haven't tried it to add it for one server and change it to see if it is automatically added to the other server. Would be a nice experiment.
Friday, 24 June 2016
A couple of notes of the automatic generation of a SOA Suite/OSB domain
Earlier you could have enjoyed my article on the automatic generation of a SOA/OSB domain. Earlier this week I encountered some issues with a domain created at a customer this way.
I got the change to dive into that this week and luckily not only I learned a lot again, but I found the problem as well. I adapted my scripts. I won't repost them completely, I've created a github account, and try to place them there in the near future.
But I'll cover the changes, especially those that caused the problems.
When you create a domain using the config.sh/.cmd wizard, and choose to configure the nodemanager, you'll be asked for the nodemanager user and password. You'll get a per domain nodemanager for free and the domain is enrolled for the nodemanager for you. You might need to adapt the nodemanager.properties in <domain_home>/nodemanager and set the property SecureListener to false. The default is true, while in the Machine definition in the domain the nodemanager/machine is configured to SSL. If you disable the SecureListener, you need to set the property in the wls-console under Machines to Plain instead of SSL. And adapt the listener-port.
The docs on configuring the nodemanager for 12.2.1 can be found here. One of the biggest changes between 11g and 12cR1 is that in 12cR2 you get a per domain nodemanager, with a nodemanager home in the domain home, instead of a nodemanager home in the FMW_HOME, the home of the binaries. Also in the bin folder of the domain you'll find scripts to start and stop the nodemanager.
So for a domain configured in the config-wizard, you need little to do to get the nodemanager working.
But using the scripts in my previous article, you'll find that although a nodemanager password file is created, something is missed for a proper startup of your servers using the nodemanager. You'll find that although the nodemanager starts, connecting to it using nmConnect(), fails with: a message like:
You can go to the Domain->security settings in the weblogic console, and change the nodemanager password, but apparently this is not enough. You'll need to explicitly enroll the domain against the nodemanager. To do so:
It can be called with a shell script (enrollDomain.sh) like:
I also adapted the function 'createUnixMachine' in the createSoaBpmDomain.py script:
If this is a absolute path, 'nothing is on the hand'. Your domain should start correclty. But if you choose to use a relative path, like 'logs', then starting the AdminServer might succeed (in my case) but starting the managed servers fail with a 'No such file or directory' message:
Now it turns out that this is caused by the JavaArgs that are generated and set in the script. I found that if you set JavaArgs you need to set redirects for weblogic.Stdout and weblogic.Stderr, like:
Where logFolder should be an absolute path. This has to do with the context in which the nodemanager is starting the server. From that context the relative reference apparently does not evaluate to the proper location.
You can however, leave the Java args empty. So I changed my scripts to not use the getServerJavaArgs function, anymore, but get them from the property file. I replaced the xxxJavaArgsBase with xxxJavaArgs variables. And left them empty.
The configurator doesn't set the JavaArgs, it leaves them over to the setDomain.sh/.cmd and setStartupEnv.sh/.cmd. If you do so, you can use a relative path, and the servers will start properly.
I got the change to dive into that this week and luckily not only I learned a lot again, but I found the problem as well. I adapted my scripts. I won't repost them completely, I've created a github account, and try to place them there in the near future.
But I'll cover the changes, especially those that caused the problems.
Enrollment
When you create a domain using the config.sh/.cmd wizard, and choose to configure the nodemanager, you'll be asked for the nodemanager user and password. You'll get a per domain nodemanager for free and the domain is enrolled for the nodemanager for you. You might need to adapt the nodemanager.properties in <domain_home>/nodemanager and set the property SecureListener to false. The default is true, while in the Machine definition in the domain the nodemanager/machine is configured to SSL. If you disable the SecureListener, you need to set the property in the wls-console under Machines to Plain instead of SSL. And adapt the listener-port.
The docs on configuring the nodemanager for 12.2.1 can be found here. One of the biggest changes between 11g and 12cR1 is that in 12cR2 you get a per domain nodemanager, with a nodemanager home in the domain home, instead of a nodemanager home in the FMW_HOME, the home of the binaries. Also in the bin folder of the domain you'll find scripts to start and stop the nodemanager.
So for a domain configured in the config-wizard, you need little to do to get the nodemanager working.
But using the scripts in my previous article, you'll find that although a nodemanager password file is created, something is missed for a proper startup of your servers using the nodemanager. You'll find that although the nodemanager starts, connecting to it using nmConnect(), fails with: a message like:
WLSTException: Error occured while performing nmConnect : Cannot connect to Node Manager. : Access to domain 'osb_domain' for user 'weblogic' denied.
You can go to the Domain->security settings in the weblogic console, and change the nodemanager password, but apparently this is not enough. You'll need to explicitly enroll the domain against the nodemanager. To do so:
- Start the AdminServer, using the startWebLogic.sh/.cmd script in the domain home.
- Start wlst.sh/.cmd
- Connect to the AdminServer using: connect(adminUser, adminPwd, adminURL)
- Perform an NodeManager Enroll using: nmEnroll(soaDomainHome, nodeManagerHome)
- Stop the AdminServer
- (Re)Start the nodemanager
- In wlst: Connect to the nodemanager using nmConnect(....)
- Perform nmStart('AdminServer')
#############################################################################
# Create a SOA/BPM/OSB domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-06-23
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = 'enrollDomain.py'
#
#Home Folders
soaDomainHome = domainsHome+'/'+soaDomainName
nodeManagerHome = soaDomainHome+'/'+'nodemanager'
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl=localhost:7101"
print "adminUser=weblogic"
print "adminPwd=welcome1"
#
#
def main():
try:
#
# Section 1: Base Domain + Admin Server
print (lineSeperator)
print ('Enroll '+soaDomainName+' for NodeManager')
print('\nConnect to AdminServer ')
print (lineSeperator)
adminURL=adminListenAddress+':'+adminListenPort
connect(adminUser, adminPwd, adminURL)
#
print('\nPerform nmEnroll')
print (lineSeperator)
#
nmEnroll(soaDomainHome, nodeManagerHome)
#
print ('\nFinished')
#
print('\nExiting...')
exit()
except NameError, e:
print 'Apparently properties not set.'
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#call main()
main()
exit()
It can be called with a shell script (enrollDomain.sh) like:
#!/bin/bash . fmw12c_env.sh echo echo Enroll domain wlst.sh enrollDomain.py -loadProperties darlin-vce-db-osb.properties
I also adapted the function 'createUnixMachine' in the createSoaBpmDomain.py script:
#
# Create a Unix Machine
def createUnixMachine(serverMachine,serverAddress, serverPort, nmType):
print('\nCreate machine '+serverMachine+' with type UnixMachine')
print (lineSeperator)
cd('/')
create(serverMachine,'UnixMachine')
cd('UnixMachine/'+serverMachine)
create(serverMachine,'NodeManager')
cd('NodeManager/'+serverMachine)
set('ListenAddress',serverAddress)
set('ListenPort',int(serverPort))
set('NMType',nmType)
This allows for the non-default setting of the nodemanager to another listen port and nodemanager type, based on the following properties in the property file:
# # Server Settings nmType=Plain server1Machine=darlin-vce-db server1Address=darlin-vce-db server1Port=5555 server2Enabled=false server2Machine=darlin-vce-db server2Address=darlin-vce-db2 server2Port=5555An example of the property file can be found in the earlier article: automatic generation of a SOA/OSB domain.
Logging
In the example property file in my scripts you can set a location for the logging:# Logs logsHome=/u01/app/work/logs
If this is a absolute path, 'nothing is on the hand'. Your domain should start correclty. But if you choose to use a relative path, like 'logs', then starting the AdminServer might succeed (in my case) but starting the managed servers fail with a 'No such file or directory' message:
wls:/nm/osb_domain> nmStart('Adminserver')
Starting server Adminserver ...
Traceback (innermost last):
File "<console", line 1, in ?
File "<iostream", line 188, in nmStart
File "<iostream>", line 553, in raiseWLSTException
WLSTException: Error occurred while performing nmStart : Error Starting server Adminserver : Received error message from Node Manager Server: [Server start command for WebLogic server 'Adminserver' failed due to: [No such file or directory]. Please check Node Manager log and/or server 'Adminserver' log for detailed information.]. Please check Node Manager log for details.
Use dumpStack() to view the full stacktrace :
wls:/nm/osb_domain
Now it turns out that this is caused by the JavaArgs that are generated and set in the script. I found that if you set JavaArgs you need to set redirects for weblogic.Stdout and weblogic.Stderr, like:
'-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1532m -Dweblogic.Stdout='+logsHome+'AdminServer.out -Dweblogic.Stderr='+logsHome+'AdminServer_err.out'
Where logFolder should be an absolute path. This has to do with the context in which the nodemanager is starting the server. From that context the relative reference apparently does not evaluate to the proper location.
You can however, leave the Java args empty. So I changed my scripts to not use the getServerJavaArgs function, anymore, but get them from the property file. I replaced the xxxJavaArgsBase with xxxJavaArgs variables. And left them empty.
The configurator doesn't set the JavaArgs, it leaves them over to the setDomain.sh/.cmd and setStartupEnv.sh/.cmd. If you do so, you can use a relative path, and the servers will start properly.
Labels:
BPM Suite
,
FMWInstallation
,
Oracle Service Bus
,
SOA Suite
,
Weblogic
,
WLST
ServiceBus 12c: Logging
As a developer you probably 'log-a-lot' in OSB. (Funny term, perfectly to mock people that have the tendency to excessively add log activities/statements to their code. And hey, if you're being mocked like this: I'm happy to join you, let's make it a 'Geuzennaam').
So as a log-a-lot, I was questioned by a OSB developer this week on a OSB11g->SB12c upgrade that I support, that his logs weren't visible in the server-logs in de 12cR2 SOAQuickStart Integrated Weblogic.
We reviewed the logging settings of the server, all set on Debug.
But it turns out that in the SB Logging configuration in EM all is set to Warning (Inherited).
To check it out, go to http://localhost:7101/em (on a default configured SOA QuickStart Integrated Weblogic domain).
Click on the Target Navigator and Navigate to soa-infra->Service Bus.
Then in the Service Bus menu, navigate to Logs -> Log Configuration:
Then you need the Log-Levels tab:
Here you see that you can set another level on different SB subsystems. It can be interesting to check out some of those. For instance, I think it was oracle.osb.debug.instancetracking (but I'm not sure, I checked several of them) enable the logging of message contents and variable changes. You see that all the log-level settings are set to Warning, inherited from the level above.
But the one that is of interest for the pipeline-logging is oracle.osb.logging.pipeline:
Set that one on Trace. I used TRACE:16 (FINER). But it turns out that the levels here are different from those in the pipeline log activity and that of those of the WebLogic Server logs. I haven't got a mapping at hand, but this one let's Debug messages through.
Hit the apply button top right:
This should enable the logging of the Pipeline.
So as a log-a-lot, I was questioned by a OSB developer this week on a OSB11g->SB12c upgrade that I support, that his logs weren't visible in the server-logs in de 12cR2 SOAQuickStart Integrated Weblogic.
We reviewed the logging settings of the server, all set on Debug.
But it turns out that in the SB Logging configuration in EM all is set to Warning (Inherited).
To check it out, go to http://localhost:7101/em (on a default configured SOA QuickStart Integrated Weblogic domain).
Click on the Target Navigator and Navigate to soa-infra->Service Bus.
Then in the Service Bus menu, navigate to Logs -> Log Configuration:
Then you need the Log-Levels tab:
Here you see that you can set another level on different SB subsystems. It can be interesting to check out some of those. For instance, I think it was oracle.osb.debug.instancetracking (but I'm not sure, I checked several of them) enable the logging of message contents and variable changes. You see that all the log-level settings are set to Warning, inherited from the level above.
But the one that is of interest for the pipeline-logging is oracle.osb.logging.pipeline:
Set that one on Trace. I used TRACE:16 (FINER). But it turns out that the levels here are different from those in the pipeline log activity and that of those of the WebLogic Server logs. I haven't got a mapping at hand, but this one let's Debug messages through.
Hit the apply button top right:
This should enable the logging of the Pipeline.
Wednesday, 15 June 2016
Servicebus Overview diagrams in 12cR2 not opened for upgraded services
In ServiceBus 12c you get a 'composite'-alike service overview for your project. It shows you how the proxy services (like Exposed Services in SOASuite) via pipelines are 'wired'to business services (like Referenced Services). This is nice!
If you upgrade a project from 11g or 12cR1 (12.1.3) to 12cR2 (12.2.1) this fails. Initially you might see a (correct) diagram, but after restarting JDeveloper this is empty. You'll get a Class cast exception:
Also after opening a earlier upgraded project, OSB diagrams fail to open and instead generate this java.lang.ClassCastException.
I search on support.oracle.com and found this document: 'Unable to open OSB Diagrams upgraded from 12.1.3 to 12.2.1 (Doc ID 2124208.1)'
It refers to the patch:
If you upgrade a project from 11g or 12cR1 (12.1.3) to 12cR2 (12.2.1) this fails. Initially you might see a (correct) diagram, but after restarting JDeveloper this is empty. You'll get a Class cast exception:
java.lang.ClassCastException: oracle.tip.tools.ide.fabric.addin.CompositeNode cannot be cast to oracle.sb.tooling.ide.sca.internal.sca.SbCompositeNode at oracle.sb.tooling.ide.sca.internal.sync.CompositeEditorListener.editorOpened(CompositeEditorListener.java:72)
Also after opening a earlier upgraded project, OSB diagrams fail to open and instead generate this java.lang.ClassCastException.
I search on support.oracle.com and found this document: 'Unable to open OSB Diagrams upgraded from 12.1.3 to 12.2.1 (Doc ID 2124208.1)'
It refers to the patch:
- 22226040: java.lang.NullPointer for XQuery File ver 1.0 in JDEV 12.2.1 OSB Proj
Automatic Patching of SOA/BPM QuickStarts
Earlier I wrote how to automatically install the SOA/BPM QuickStarts. Actually, I'm quite busy with doing automatic/scripted installs for SOA/BPM Suite and OSB, as you might have read.
At my current customer we encountered that in the last months there are many one-off-patches released on support.oracle.com. We selected a pretty large bunch of patches and apply them one by one is a tedious job. But the thing is with these automatic installs that you want to have a uniform installation for each developer so each developer should have the same patches installed, in the same location. And you probably want to be able to quickly do a re-install to a uniform setup.
So I figured out how to do a silent install of the patches and to do this in a loop.
Out of the selected patches I found 5 catechories:
I have one main script called 'installQSPatches.bat' that loops over the files in each sub-folder:
And it needs to be started in an elevated (as Administrator) command window.
For each patch it calls the applyPatch.bat script:
This one figures out what the patchnumber (%PATCH_NR%) is based on the patch-file-name, and if that patch already exists in the %FMW_HOME%/Opatch/patches folder. If not it will unzip the patch file to that folder, resulting in a sub-folder named with the patch number. Then it will apply the patch using Opatch in silent mode.
For the unzip, it uses a simple ANT build file, since the Windows Command screen does not support a commandline-unzip (as far as I could find). The ANT script is as follows:
When I finished this I thought I should convert it to a complete ANT script. But maybe later.
The selected patches for 12.2.1:
The selected JDeveloper patches (sub-folder 001) were:
The selected Service Bus patches (sub-folder 002) were:
For BPM (sub-folder 005) the following merged patches were selected:
Download the applicable patches for your situation and put them in the appropriate folder. If you use the SOA QuickStart in stead of BPM QuickStart, you should skip the BPM Related patches of course.
UPDATE september 26th, 2016: Since this is a Windows script: you need to run this on the same drive as the QuickStart installer. It does a change dir to the patch and then an 'Opatch apply'. But if it's not run on the same drive, the change dir succeeds, but it's on another drive so still not current.
At my current customer we encountered that in the last months there are many one-off-patches released on support.oracle.com. We selected a pretty large bunch of patches and apply them one by one is a tedious job. But the thing is with these automatic installs that you want to have a uniform installation for each developer so each developer should have the same patches installed, in the same location. And you probably want to be able to quickly do a re-install to a uniform setup.
So I figured out how to do a silent install of the patches and to do this in a loop.
Out of the selected patches I found 5 catechories:
- 001: JDeveloper patches, where only a few we found possibly applicable for the SOA/BPM QuickStarts
- 002: ServiceBus related patches
- 003: SOA Suite merged patches
- 004: SOA Suite related other patches
- 005: BPM Suite merged patches
- 006: BPM Suite related patches
I have one main script called 'installQSPatches.bat' that loops over the files in each sub-folder:
@echo off rem check SOA12.2 QS setlocal set FMW_HOME=C:\oracle\JDeveloper\12210_BPMQS set ORACLE_HOME=%FMW_HOME% set SOA_PATCH_SOURCE=SOA set SOA_PATCH_HOME=%FMW_HOME%\Opatch\patches set CUR_DIR=%~dp0 echo Current Dir: %CUR_DIR% if exist "%FMW_HOME%" goto :SOAQS_HOME_EXISTS echo %FMW_HOME% not installed yet! Install first! goto :DONE :SOAQS_HOME_EXISTS echo %FMW_HOME% exists, install Patches echo ____________________________________________________ call %FMW_HOME%\wlserver\server\bin\setWLSEnv.cmd echo ____________________________________________________ :JDEV_PATCHES echo - echo JDeveloper Patches echo ____________________________________________________ for %%f in (001\*.zip) do ( echo %%f call applyPatch %%f ) :SB_PATCHES echo - echo ServiceBus Patches echo ____________________________________________________ for %%f in (002\*.zip) do ( echo %%f call applyPatch %%f ) :SOA_MERGE_PATCHES echo - echo SOA Suite Merged Patches echo ____________________________________________________ for %%f in (003\*.zip) do ( echo %%f call applyPatch %%f ) :SOA_PATCHES echo - echo SOA Suite Patches echo ____________________________________________________ for %%f in (004\*.zip) do ( echo %%f call applyPatch %%f ) :BPM_MERGE_PATCHES echo - echo BPM Suite Merged Patches echo ____________________________________________________ for %%f in (005\*.zip) do ( echo %%f call applyPatch %%f ) :BPM_PATCHES echo - echo BPM Suite Patches echo ____________________________________________________ for %%f in (006\*.zip) do ( echo %%f call applyPatch %%f ) :DONE echo Done installing patches endlocalYou need to shutdown JDeveloper and IntegratedWeblogic before starting this script.
And it needs to be started in an elevated (as Administrator) command window.
For each patch it calls the applyPatch.bat script:
set PATCH=%1 set PATCH_NR=%PATCH:~5,8% echo ____________________________________________________ echo Check Patch %PATCH% for patch nr %PATCH_NR% if exist "%FMW_HOME%\Opatch\patches\%PATCH_NR%" goto :PATCH_EXISTS echo "%FMW_HOME%\Opatch\patches\%PATCH_NR%" does not exist. set SOA_PATCH_HOME=%FMW_HOME%\Opatch\patches rem set SOA_PATCH_HOME=c:\temp\patches echo .. Unzip %SOA_PATCH_SOURCE%\%PATCH% to %SOA_PATCH_HOME% call ant -f ant-zip.xml unzip -Dzip-file=%PATCH% -Dunzip-destination=%SOA_PATCH_HOME% cd %ORACLE_HOME%\Opatch\patches\%PATCH_NR% echo .. Apply %ORACLE_HOME%\Opatch\patches\%PATCH_NR% call %ORACLE_HOME%\Opatch\opatch apply -silent cd %CUR_DIR% goto :DONE :PATCH_EXISTS echo Patch %PATCH_NR% already exits! :DONE echo Done for patch %PATCH_NR% echo ____________________________________________________
This one figures out what the patchnumber (%PATCH_NR%) is based on the patch-file-name, and if that patch already exists in the %FMW_HOME%/Opatch/patches folder. If not it will unzip the patch file to that folder, resulting in a sub-folder named with the patch number. Then it will apply the patch using Opatch in silent mode.
For the unzip, it uses a simple ANT build file, since the Windows Command screen does not support a commandline-unzip (as far as I could find). The ANT script is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<project name="zip" default="zip" basedir=".">
<target name="zip">
<zip destfile="${zip-file}.zip" basedir="${folder-to-zip}" excludes="dont*.*" />
</target>
<target name="unzip">
<unzip src="${zip-file}" dest="${unzip-destination}" />
</target>
</project>
When I finished this I thought I should convert it to a complete ANT script. But maybe later.
The selected patches for 12.2.1:
The selected JDeveloper patches (sub-folder 001) were:
- 22283405 JDEV 12.2.1 - NULLPOINTEREXCEPTION ENCOUNTERED (Patch)
- 23266774 VALIDATION ERRORS WHEN CLICKING ON MANDATORY FIELDS IN JDEV12.2.1 (Patch)
- 22463346 NOT STRESS SOA: MULTIPLE ERROR MESSAGES OF DEFINITIONMANAGER.LOCKINGLOGGER (Patch)
- 21890657 CREATE REST THROWS ERROR 500 FOR REFERENCED ENTITY (Patch)
The selected Service Bus patches (sub-folder 002) were:
- 23223332 Need to provide a holistic solution in main line for Bug 22887808
- 21824551 NPE while trying to read OWSM keystore
- 21168191 UnsupportedOperationException from ServiceAccountRuntimeCache
- 23184618 Http Transport throws NullPointerException
- 21827583 Deploying existing .sbar with Maven
- 22738111 OSB Java Call out method's not visible from JDeveloper
- 20119834 need to trim headers size if size exceeds 998 characters.
- 22358699 OSB12C fn-bea:inlinedXML does not work properly
- 22374613 In 12.2.1 the OSB Projet pom files still says 12.1.3
- 20196110 JDev OSB Extension has missed export Split-Join and Proxy Flow as png
- 22187224 OSB 12.2.1 - MessageID changes between request and response messages
- 22392646 Maven could not be used in Jdev in 12.2.1.0.0
- 22602059 12C: OSB pipeline based on XML - $body structure incomplete - missing node
- 22276364 12C: OSB pipeline based on XML - $body structure unavailable
- 21659900 OSB removes WSA headers on outbound request
- 23543517 MERGE REQUEST ON TOP OF 12.2.1.0.0 FOR BUGS 23527297 22875806 22995356 23062804
- 23138916 MERGE REQUEST ON TOP OF 12.2.1.0.0 FOR BUGS 21549249 21572567
- 23106839 MERGE REQUEST ON TOP OF 12.2.1.0.0 FOR BUGS 21826430 22912570
- 23134140 Diagnostic Tracking Bug for Bug 23108573 v2
- 23056585 XQuery transformation is not showing all the types of XSD in the design view
- 21904101 Error when running ValidateComposite to project with bpel calling HWF
- 23205706 MALFORMEDURLEXCEPTION EXCEPTION ON JDEV 12.2.1.0.0 - BAM 12C IDE CONNECTION
- 23193066 BPEL polls from UMS Adapter results in too large CONVERSATION_ID error
- 21698320 SELECTING ELEMENT IN LARGE DOCUMENT TAKES FREEZES 2+MINUTES
- 23108573 TrackingContextProperty causes ClassCastException in SOA 12c Spring Composite
- 21925552 SOA Maven Plugin requires deploying sar to server when running mvn install
- 23186275 JDeveloper 12.2.1 JMS Adapter not displaying Elements from the imported xsd file
- 22337707 Naming conflict with EDN event subscribers from single Mediator component
- 23052343 16.2.3 : WS JOB REQUEST ENDS UP IN ERROR STATE WHEN INVOKED VIA OTD URL
- 22026475 JDEV : Adapter : Nullpointerexception while creating the BPEL Process
- 22815366 Unable to use WSDL containing an unsupported Notification Operation portType
- 22978098 Indicators configured on Response payload coming from a DB ADapter do not work
- 21835972 SFTP FileListing not work as expected. Need similiar bug fix as in bug 21176154
- 22648699 SOA Suite12.2.1-JCA files not getting updated with Configuration Plan values
- 22300448 JDeveloper doesn't add a libraries file group to existing SOA deployment profile
- 16548396 Can not emulate fault from the SOA Test UI
For BPM (sub-folder 005) the following merged patches were selected:
- 22571194 NPE in o.bpm.project.sca.loader.impl.ElementContainer:82
- 23283093 NPE in o.bpm.project.sca.loader.impl.ElementContainer:198
- 22191778 In the Business Architecture Modeling links section, buttons are missing
- 21325503 12c compilation err-crm: oracle.bpm.services : extncontentpublicmodeleventaction
- 22736087 BPM Workspace not refreshing the task list after Initiator task page is closed
- 22018713 MeasurementPublisher has been optimized
- Software 22217468 Refresh is not working properly in Impact Analysis Report - 500 error
- 22018703 MIAuditLog has been optimized
- 22690780 workspace Title & logo unchanged on loginpage after patch 22272135
- 19536412 Case Service handles namespaces incorrectly in SOAP XML messages
- 22348182 EDG RC3a :Task comments cannot be added
- 22087208 When approving a task it immediately opens the next task in your tasklist
- 22753983 12c e-manager authorization issues - List view
- 22111729 BPM instance is not released after completing an HT ith the voting patern
- 22581888 Configure global option to enable task details to appear as pop-up window in 12c
- 23077279 Composer crash when creating Org Unit or Application system for human task
- 23125361 REST hrefs for attachments containing spaces in their names do not work.
- 23205514 Flexfields not properly displayed in parallel task with SDO
- 23301503 weblogic user or Administrator group shouldn't be necessary to login to composer
- 23491602 Studio generated project_properties.wsdl missing the correlation mapping
- 21792613 NOT STRESS BPM REST:get WebFormService - no valid constructor error
Download the applicable patches for your situation and put them in the appropriate folder. If you use the SOA QuickStart in stead of BPM QuickStart, you should skip the BPM Related patches of course.
UPDATE september 26th, 2016: Since this is a Windows script: you need to run this on the same drive as the QuickStart installer. It does a change dir to the patch and then an 'Opatch apply'. But if it's not run on the same drive, the change dir succeeds, but it's on another drive so still not current.
Labels:
BPM Suite
,
FMWInstallation
,
JDeveloper
,
Oracle Service Bus
,
SOA Suite
Thursday, 9 June 2016
Scripted Domain Creation for SOA/BPM, OSB 12.2.1
Recently I blogged about the automatic install of SOASuite and ServiceBus 12.2.1. It catered for the installation of the binaries and the creation of a repository.
What it does not handles is the creation of a domain. The last few weeks I worked on a script to do that. It's based on a wlst script for 12.1.3 by Edwin Biemond. This one was quite educational for me. But as denoted: it was made for 12.1.3. And I wanted more control on what and how certain artefacts are created. So gave it my own swing, also to adapt it for some wishes of my current customer.
Although there are certainly some improvements thinkable, and probably will be done in future uses, for now it is ready to share with the world.
One of the changes I did was to divide the main function in sections to make clear what the structure of the script is. I also moved some of the duplicate code or functional parts into separate functions.
Let me describe the sections first.
It then creates boot.properties files for nodemanager and AdminServer and set the password of the NodeManager. Finally setting the Applications home folder.
It supports the following components:
I realize that 'server' in this context might be a little confusing. In serverYAddress and serverYMachine, I actually mean a server-host, not a managed or admin server.
For each component to configure (soa, osb, etc.) a cluster, denoted with for instance osbClr or soaClr, is created.
When you extend the domain with SOA Suite or OSB then automatically a managed server called 'soa_server1' or 'osb_server1' created with the appropriate deployments targeted to it. In the script of Edwin these are removed and new ones are created. I found problems with that and found that it's quite unnecessary, since we can rename the current ones with the given name in the property file, denoted with soaSvr1 or osbSvr1, etc., as is done with the AdminServer. So I leave the already created ones, but rename them to the desired value.
These first servers are added to the appropriate cluster, what causes to re-target the deployments to that cluster, magically.
Then if enabled, as with osbSvr2Enabled or soaSvr2Enabled, etc., the particular 'second' servers are created and added to the particular cluster.
In this script the determination is done based on so-called ServerGroups. This provides a means to differentiate in memory settings for the particular servers, which was lacking in 11g.
So in this sections all the Managed and Admin Servers are added to a particular ServerGroup.
Save it with a name like darlin-vce-db.properties, but adapted for each particular environment.
And although it creates a 'per domain' nodemanager configuration, you would need to adapt it for your particular needs to get the domain started. I only tested this by starting the Admin server using the startWeblogic.sh script.
Having such a script is such a valuable asset: it allows you to (re-)create your domains repeatably in a standard way, ensuring that different environments (dev, test, acc, prod) are created similarly.
One, last thing though, the script somehow registers the creation of the domain and thus the use of the datasources in the repository. So you can't just throw away the domain and recreate it to the current Repository. You'll need to rereate the Repository as well.
What it does not handles is the creation of a domain. The last few weeks I worked on a script to do that. It's based on a wlst script for 12.1.3 by Edwin Biemond. This one was quite educational for me. But as denoted: it was made for 12.1.3. And I wanted more control on what and how certain artefacts are created. So gave it my own swing, also to adapt it for some wishes of my current customer.
Although there are certainly some improvements thinkable, and probably will be done in future uses, for now it is ready to share with the world.
One of the changes I did was to divide the main function in sections to make clear what the structure of the script is. I also moved some of the duplicate code or functional parts into separate functions.
Let me describe the sections first.
1. Create Base domain
The script starts with creating a base domain. It reads the default wls template 'wls.jar'. Sets the log properties of the domain. It then adapts the AdminServer to- change name as set in the property file: you can have your own naming convention.
- change listen addres + port
- Set default SSL settings
- Set log file properties.
It then creates boot.properties files for nodemanager and AdminServer and set the password of the NodeManager. Finally setting the Applications home folder.
2. Extending domain with templates
The second section extends the domain with templates. Another improvement I did is that you can select which components you want to add by toggling the appropriate 'xxxEnabled' switches in the property file, where xxx stands for the component (for instance 'soa', 'bpm', 'osb', 'bam', 'ess', etc.)It supports the following components:
- ServiceBus
- SOA and BPM Suite and B2B
- BAM
- Enterprise Scheduler Service
3. DataSources
Section 3 takes care of setting the datasources to the created repository based on the repository user '{Prefix}_STB', via the 'LocalScvTblDataSource' datasource. In the property file you need to set- soaRepositoryDbUrl: jdbc connect string to the repository database
- soaRepositoryDbUserPrefix=prefix used in the Repository creation
- soaRepositoryStbPwd=Password for the {Prefix}_STB user.
4. Create UnixMachines, Clusters and Managed Servers
This section creates Machine definitions of type 'Unix', based on the properties:- server1Address=darlin-vce-db.darwin-it.local
- server1Machine=darlin-vce-db
- server2Enabled=true
- server2Address=darlin-vce-db2.darwin-it.local
- server2Machine=darlin-vce-db2
I realize that 'server' in this context might be a little confusing. In serverYAddress and serverYMachine, I actually mean a server-host, not a managed or admin server.
For each component to configure (soa, osb, etc.) a cluster, denoted with for instance osbClr or soaClr, is created.
When you extend the domain with SOA Suite or OSB then automatically a managed server called 'soa_server1' or 'osb_server1' created with the appropriate deployments targeted to it. In the script of Edwin these are removed and new ones are created. I found problems with that and found that it's quite unnecessary, since we can rename the current ones with the given name in the property file, denoted with soaSvr1 or osbSvr1, etc., as is done with the AdminServer. So I leave the already created ones, but rename them to the desired value.
These first servers are added to the appropriate cluster, what causes to re-target the deployments to that cluster, magically.
Then if enabled, as with osbSvr2Enabled or soaSvr2Enabled, etc., the particular 'second' servers are created and added to the particular cluster.
5. Add Servers to ServerGroups
New in 12c is the concept of ServerGroups. In 11g you had only one definition of USER_MEM_ARGS in the setDomainEnv.sh/cmd. So these counted for each server (admin or managed) that are started using the start(Managed)Weblogic.sh/cmd scripts. But in 12c the determination of the USER_MEM_ARGS are done in a separate script: setStartupEnv.sh/cmd.In this script the determination is done based on so-called ServerGroups. This provides a means to differentiate in memory settings for the particular servers, which was lacking in 11g.
So in this sections all the Managed and Admin Servers are added to a particular ServerGroup.
6. Create boot properties files
Lastly, for each created managed server a boot.properties file with the username password is created. Smart: I used to do this every single time by hand...The example property file
Here's an example of the property file:############################################################################# # Properties voor Creeëren SOADomain # # @author Martien van den Akker, Darwin-IT Professionals # @version 1.0, 2016-04-15 # ############################################################################# # fmwHome=/u01/app/oracle/FMW12210 # soaDomainName=osb_domain domainsHome=/u01/app/work/domains applicationsHome=/u01/app/work/applications productionMode=true # # Server Settings server1Address=darlin-vce-db.darwin-it.local server1Machine=darlin-vce-db server2Enabled=true server2Address=darlin-vce-db2.darwin-it.local server2Machine=darlin-vce-db2 # # Properties for AdminServer adminServerName=AdminServer adminListenAddress=darlin-vce-db adminListenPort=7001 adminJavaArgsBase=-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1532m # Properties for OSB osbEnabled=true osbJavaArgsBase=-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1024m osbClr=OsbCluster osbSvr1=OsbServer1 osbSvr1Port=8011 osbSvr2Enabled=true osbSvr2=OsbServer2 osbSvr2Port=8012 # Properties for SOA soaEnabled=true bpmEnabled=true b2bEnabled=true soaJavaArgsBase=-XX:PermSize=256m -XX:MaxPermSize=752m -Xms1024m -Xmx1532m soaClr=SoaCluster soaSvr1=SoaServer1 soaSvr1Port=8001 soaSvr2Enabled=true soaSvr2=SoaServer2 soaSvr2Port=8002 # Properties for ESS essEnabled=true essJavaArgsBase=-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1024m essClr=essCluster essSvr1=EssServer1 essSvr1Port=8021 essSvr2Enabled=true essSvr2=EssServer2 essSvr2Port=8022 # Properties for BAM bamEnabled=true bamJavaArgsBase=-XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1532m bamClr=BamCluster bamSvr1=BamServer1 bamSvr1Port=9001 bamSvr2Enabled=true bamSvr2=BamServer2 bamSvr2Port=9002 # AdminUser adminUser=weblogic adminPwd=welcome1 # SoaRepository Settings soaRepositoryDbUrl=jdbc:oracle:thin:@darlin-vce-db.darwin-it.local:1521/pdborcl soaRepositoryDbUserPrefix=DEV soaRepositoryStbPwd=DEV_STB # Logs logsHome=/u01/app/work/logs fileCount=10 fileMinSize=5000 fileTimeSpan=24 rotationType=byTime # # Settings webtierEnabled=false jsseEnabled=false
Save it with a name like darlin-vce-db.properties, but adapted for each particular environment.
The script
(And of course I don't mean the band that my daughter likes...)#############################################################################
# Create a SOA/BPM/OSB domain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-04-09
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = 'createSoaBpmDomain.py'
#
#Home Folders
wlsHome = fmwHome+'/wlserver'
soaDomainHome = domainsHome+'/'+soaDomainName
soaApplicationsHome = applicationsHome+'/'+soaDomainName
#
# Templates for 12.1.3
#wlsjar =fmwHome+'/wlserver/common/templates/wls/wls.jar'
#oracleCommonTplHome=fmwHome+'/oracle_common/common/templates'
#wlservicetpl=oracleCommonTplHome+'/oracle.wls-webservice-template_12.1.3.jar'
#osbtpl=fmwHome+'/osb/common/templates/wls/oracle.osb_template_12.1.3.jar'
#applCoreTpl=oracleCommonTplHome+'/wls/oracle.applcore.model.stub.1.0.0_template.jar'
#soatpl=fmwHome+'/soa/common/templates/wls/oracle.soa_template_12.1.3.jar'
#bamtpl=fmwHome+'/soa/common/templates/wls/oracle.bam.server_template_12.1.3.jar'
#bpmtpl=fmwHome+'/soa/common/templates/wls/oracle.bpm_template_12.1.3.jar'
#essBasicTpl=oracleCommonTplHome+'/wls/oracle.ess.basic_template_12.1.3.jar'
#essEmTpl=fmwHome+'/em/common/templates/wls/oracle.em_ess_template_12.1.3.jar'
#ohsTpl=fmwHome+'/ohs/common/templates/wls/ohs_managed_template_12.1.3.jar'
#b2bTpl=fmwHome+'/soa/common/templates/wls/oracle.soa.b2b_template_12.1.3.jar'
#
# Templates for 12.2.1
wlsjar =fmwHome+'/wlserver/common/templates/wls/wls.jar'
oracleCommonTplHome=fmwHome+'/oracle_common/common/templates'
wlservicetpl=oracleCommonTplHome+'/wls/oracle.wls-webservice-template.jar'
osbtpl=fmwHome+'/osb/common/templates/wls/oracle.osb_template.jar'
applCoreTpl=oracleCommonTplHome+'/wls/oracle.applcore.model.stub_template.jar'
soatpl=fmwHome+'/soa/common/templates/wls/oracle.soa_template.jar'
bamtpl=fmwHome+'/soa/common/templates/wls/oracle.bam.server_template.jar'
bpmtpl=fmwHome+'/soa/common/templates/wls/oracle.bpm_template.jar'
essBasicTpl=oracleCommonTplHome+'/wls/oracle.ess.basic_template.jar'
essEmTpl=fmwHome+'/em/common/templates/wls/oracle.em_ess_template.jar'
ohsTpl=fmwHome+'/ohs/common/templates/wls/ohs_managed_template.jar' # need to be validated!
b2bTpl=fmwHome+'/soa/common/templates/wls/oracle.soa.b2b_template.jar' # need to be validated!
#
# ServerGroup definitions
adminSvrGrpDesc='WSM-CACHE-SVR WSMPM-MAN-SVR JRF-MAN-SVR'
adminSvrGrp=["WSM-CACHE-SVR" , "WSMPM-MAN-SVR" , "JRF-MAN-SVR"]
essSvrGrpDesc="ESS-MGD-SVRS"
essSvrGrp=["ESS-MGD-SVRS"]
soaSvrGrpDesc="SOA-MGD-SVRS"
soaSvrGrp=["SOA-MGD-SVRS"]
bamSvrGrpDesc="BAM12-MGD-SVRS"
bamSvrGrp=["BAM12-MGD-SVRS"]
osbSvrGrpDesc="OSB-MGD-SVRS-COMBINED"
osbSvrGrp=["OSB-MGD-SVRS-COMBINED"]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
print 'Call script as: '
print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
print 'Property file should contain the following properties: '
print "adminUrl='localhost:7101'"
print "adminUser='weblogic'"
print "adminPwd='welcome1'"
#
# Create a boot properties file.
def createBootPropertiesFile(directoryPath,fileName, username, password):
print ('Create Boot Properties File for folder: '+directoryPath)
print (lineSeperator)
serverDir = File(directoryPath)
bool = serverDir.mkdirs()
fileNew=open(directoryPath + '/'+fileName, 'w')
fileNew.write('username=%s\n' % username)
fileNew.write('password=%s\n' % password)
fileNew.flush()
fileNew.close()
#
# Create Startup Properties File
def createAdminStartupPropertiesFile(directoryPath, args):
print 'Create AdminServer Boot Properties File for folder: '+directoryPath
print (lineSeperator)
adminserverDir = File(directoryPath)
bool = adminserverDir.mkdirs()
fileNew=open(directoryPath + '/startup.properties', 'w')
args=args.replace(':','\\:')
args=args.replace('=','\\=')
fileNew.write('Arguments=%s\n' % args)
fileNew.flush()
fileNew.close()
#
# Set Log properties
def setLogProperties(logMBeanPath, logFile, fileCount, fileMinSize, rotationType, fileTimeSpan):
print '\nSet Log Properties for: '+logMBeanPath
print (lineSeperator)
cd(logMBeanPath)
print ('Server log path: '+pwd())
print '. set FileName to '+logFile
set('FileName' ,logFile)
print '. set FileCount to '+str(fileCount)
set('FileCount' ,int(fileCount))
print '. set FileMinSize to '+str(fileMinSize)
set('FileMinSize' ,int(fileMinSize))
print '. set RotationType to '+rotationType
set('RotationType',rotationType)
print '. set FileTimeSpan to '+str(fileTimeSpan)
set('FileTimeSpan',int(fileTimeSpan))
#
#
def createServerLog(serverName, logFile, fileCount, fileMinSize, rotationType, fileTimeSpan):
print ('\nCreate Log for '+serverName)
print (lineSeperator)
cd('/Server/'+serverName)
create(serverName,'Log')
setLogProperties('/Server/'+serverName+'/Log/'+serverName, logFile, fileCount, fileMinSize, rotationType, fileTimeSpan)
#
# Change DataSource to XA
def changeDatasourceToXA(datasource):
print 'Change datasource '+datasource
print (lineSeperator)
cd('/')
cd('/JDBCSystemResource/'+datasource+'/JdbcResource/'+datasource+'/JDBCDriverParams/NO_NAME_0')
set('DriverName','oracle.jdbc.xa.client.OracleXADataSource')
print '. Set UseXADataSourceInterface='+'True'
set('UseXADataSourceInterface','True')
cd('/JDBCSystemResource/'+datasource+'/JdbcResource/'+datasource+'/JDBCDataSourceParams/NO_NAME_0')
print '. Set GlobalTransactionsProtocol='+'TwoPhaseCommit'
set('GlobalTransactionsProtocol','TwoPhaseCommit')
cd('/')
#
#
def createCluster(cluster):
print ('\nCreate '+cluster)
print (lineSeperator)
cd('/')
create(cluster, 'Cluster')
#
# Create a Unix Machine
def createUnixMachine(serverMachine,serverAddress):
print('\nCreate machine '+serverMachine+' with type UnixMachine')
print (lineSeperator)
cd('/')
create(serverMachine,'UnixMachine')
cd('UnixMachine/'+serverMachine)
create(serverMachine,'NodeManager')
cd('NodeManager/'+serverMachine)
set('ListenAddress',serverAddress)
#
# Add server to Unix Machine
def addServerToMachine(serverName, serverMachine):
print('\nAdd server '+serverName+' to '+serverMachine)
print (lineSeperator)
cd('/Servers/'+serverName)
set('Machine',serverMachine)
#
# Determine the Server Java Args
def getServerJavaArgs(serverName,javaArgsBase,logsHome):
javaArgs = javaArgsBase+' -Dweblogic.Stdout='+logsHome+'/'+serverName+'.out -Dweblogic.Stderr='+logsHome+'/'+serverName+'_err.out'
return javaArgs
#
# Change Managed Server
def changeManagedServer(server,listenAddress,listenPort,javaArgs):
print '\nChange ManagedServer '+server
print (lineSeperator)
cd('/Servers/'+server)
print '. Set listen address and port to: '+listenAddress+':'+str(listenPort)
set('ListenAddress',listenAddress)
set('ListenPort' ,int(listenPort))
# ServerStart
print ('. Create ServerStart')
create(server,'ServerStart')
#cd('ServerStart/'+server)
#print ('. Set Arguments to: '+javaArgs)
#set('Arguments' , javaArgs)
# SSL
cd('/Server/'+server)
print ('. Create server SSL')
create(server,'SSL')
cd('SSL/'+server)
print ('. Set SSL Enabled to: '+'False')
set('Enabled' , 'False')
print ('. Set SSL HostNameVerificationIgnored to: '+'True')
set('HostNameVerificationIgnored', 'True')
#
if jsseEnabled == 'true':
print ('. Set JSSEEnabled to: '+ 'True')
set('JSSEEnabled','True')
else:
print ('. Set JSSEEnabled to: '+ 'False')
set('JSSEEnabled','False')
#
# Create a Managed Server
def createManagedServer(server,listenAddress,listenPort,cluster,machine,
javaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan):
print('\nCreate '+server)
print (lineSeperator)
cd('/')
create(server, 'Server')
cd('/Servers/'+server)
javaArgs=getServerJavaArgs(server,javaArgsBase,logsHome)
changeManagedServer(server,listenAddress,listenPort,javaArgs)
createServerLog(server, logsHome+'/'+server+'.log', fileCount, fileMinSize, rotationType, fileTimeSpan)
print('Add '+server+' to cluster '+cluster)
cd('/')
assign('Server',server,'Cluster',cluster)
addServerToMachine(server, machine)
#
# Adapt a Managed Server
def adaptManagedServer(server,newSrvName,listenAddress,listenPort,cluster,machine,
javaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan):
print('\nAdapt '+server)
print (lineSeperator)
cd('/')
cd('/Servers/'+server)
# name of adminserver
print '. Rename '+server+' to '+ newSrvName
set('Name',newSrvName )
cd('/Servers/'+newSrvName)
javaArgs=getServerJavaArgs(newSrvName,javaArgsBase,logsHome)
changeManagedServer(newSrvName,listenAddress,listenPort,javaArgs)
createServerLog(newSrvName, logsHome+'/'+newSrvName+'.log', fileCount, fileMinSize, rotationType, fileTimeSpan)
print('Add '+newSrvName+' to cluster '+cluster)
cd('/')
assign('Server',newSrvName,'Cluster',cluster)
addServerToMachine(newSrvName, machine)
#
# Change Admin Server
def changeAdminServer(adminServerName,listenAddress,listenPort,javaArguments):
print '\nChange AdminServer'
print (lineSeperator)
cd('/Servers/AdminServer')
# name of adminserver
print '. Set Name to '+ adminServerName
set('Name',adminServerName )
cd('/Servers/'+adminServerName)
# address and port
print '. Set ListenAddress to '+ server1Address
set('ListenAddress',server1Address)
print '. Set ListenPort to '+ str(listenPort)
set('ListenPort' ,int(listenPort))
#
# ServerStart
print 'Create ServerStart'
create(adminServerName,'ServerStart')
#cd('ServerStart/'+adminServerName)
#print '. Set Arguments to: '+javaArguments
#set('Arguments' , javaArguments)
# SSL
cd('/Server/'+adminServerName)
print 'Create SSL'
create(adminServerName,'SSL')
cd('SSL/'+adminServerName)
set('Enabled' , 'False')
set('HostNameVerificationIgnored', 'True')
#
if jsseEnabled == 'true':
print ('. Set JSSEEnabled to: '+ 'True')
set('JSSEEnabled','True')
else:
print ('. Set JSSEEnabled to: '+ 'False')
set('JSSEEnabled','False')
#
#
def main():
try:
#
# Section 1: Base Domain + Admin Server
print (lineSeperator)
print ('1. Create Base domain '+soaDomainName)
print('\nCreate base wls domain with template '+wlsjar)
print (lineSeperator)
readTemplate(wlsjar)
#
cd('/')
# Domain Log
print('Set base_domain log')
create('base_domain','Log')
setLogProperties('/Log/base_domain', logsHome+soaDomainName+'.log', fileCount, fileMinSize, rotationType, fileTimeSpan)
#
# Admin Server
adminJavaArgs = getServerJavaArgs(adminServerName,adminJavaArgsBase,logsHome)
changeAdminServer(adminServerName,adminListenAddress,adminListenPort,adminJavaArgs)
createServerLog(adminServerName, logsHome+adminServerName+'.log', fileCount, fileMinSize, rotationType, fileTimeSpan)
#
print('\nSet password in '+'/Security/base_domain/User/weblogic')
cd('/')
cd('Security/base_domain/User/weblogic')
# weblogic user name + password
print('. Set Name to: ' +adminUser)
set('Name',adminUser)
cmo.setPassword(adminPwd)
#
if productionMode == 'true':
print('. Set ServerStartMode to: ' +'prod')
setOption('ServerStartMode', 'prod')
else:
print('. Set ServerStartMode to: ' +'dev')
setOption('ServerStartMode', 'dev')
#
print('write Domain...')
# write path + domain name
writeDomain(soaDomainHome)
closeTemplate()
#
createAdminStartupPropertiesFile(soaDomainHome+'/servers/'+adminServerName+'/data/nodemanager',adminJavaArgs)
createBootPropertiesFile(soaDomainHome+'/servers/'+adminServerName+'/security','boot.properties',adminUser,adminPwd)
createBootPropertiesFile(soaDomainHome+'/config/nodemanager','nm_password.properties',adminUser,adminPwd)
#
es = encrypt(adminPwd,soaDomainHome)
#
readDomain(soaDomainHome)
#
print('set Domain password for '+soaDomainName)
cd('/SecurityConfiguration/'+soaDomainName)
set('CredentialEncrypted',es)
#
print('Set nodemanager password')
set('NodeManagerUsername' ,adminUser )
set('NodeManagerPasswordEncrypted',es )
#
cd('/')
setOption( "AppDir", soaApplicationsHome )
#
print('Finished base domain.')
#
# Section 2: Templates
print('\n2. Extend Base domain with templates.')
print (lineSeperator)
print ('Adding Webservice template '+wlservicetpl)
addTemplate(wlservicetpl)
# SOA Suite
if soaEnabled == 'true':
print ('Adding SOA Template '+soatpl)
addTemplate(soatpl)
else:
print('SOA is disabled')
# BPM
if bpmEnabled == 'true':
print ('Adding BPM Template '+bpmtpl)
addTemplate(bpmtpl)
else:
print('BPM is disabled')
# OSB
if osbEnabled == 'true':
print ('Adding OSB template '+osbtpl)
addTemplate(osbtpl)
else:
print('OSB is disabled')
#
print ('Adding ApplCore Template '+applCoreTpl)
addTemplate(applCoreTpl)
#
if bamEnabled == 'true':
print ('Adding BAM Template '+bamtpl)
addTemplate(bamtpl)
else:
print ('BAM is disabled')
#
if webtierEnabled == 'true' == true:
print ('Adding OHS Template '+ohsTpl)
addTemplate(ohsTpl)
else:
print('OHS is disabled')
#
if b2bEnabled == 'true':
print 'Adding B2B Template '+b2bTpl
addTemplate(b2bTpl)
else:
print('B2B is disabled')
#
if essEnabled == 'true':
print ('Adding ESS Template'+essBasicTpl)
addTemplate(essBasicTpl)
print ('Adding ESS Em Template'+essEmTpl)
addTemplate(essEmTpl)
else:
print('ESS is disabled')
#
dumpStack()
print ('Finished templates')
#
# Section 3: Change Datasources
print ('\n3. Change datasources')
print 'Change datasource LocalScvTblDataSource'
cd('/JDBCSystemResource/LocalSvcTblDataSource/JdbcResource/LocalSvcTblDataSource/JDBCDriverParams/NO_NAME_0')
set('URL',soaRepositoryDbUrl)
set('PasswordEncrypted',soaRepositoryStbPwd)
cd('Properties/NO_NAME_0/Property/user')
set('Value',soaRepositoryDbUserPrefix+'_STB')
#
print ('Call getDatabaseDefaults which reads the service table')
getDatabaseDefaults()
#
if soaEnabled == 'true':
changeDatasourceToXA('EDNDataSource')
if osbEnabled == 'true':
changeDatasourceToXA('wlsbjmsrpDataSource')
changeDatasourceToXA('OraSDPMDataSource')
changeDatasourceToXA('SOADataSource')
#
if bamEnabled == 'true':
changeDatasourceToXA('BamDataSource')
#
print 'Finshed DataSources'
#
# Section 4: Create UnixMachines, Clusters and Managed Servers
print ('\n4. Create UnixMachines, Clusters and Managed Servers')
print (lineSeperator)
cd('/')
#
createUnixMachine(server1Machine,server1Address)
if server2Enabled == 'true':
createUnixMachine(server2Machine,server2Address)
#
addServerToMachine(adminServerName,server1Machine)
#
cd('/')
# SOA Suite
if soaEnabled == 'true':
createCluster(soaClr)
adaptManagedServer('soa_server1',soaSvr1,server1Address, soaSvr1Port,soaClr,server1Machine,
soaJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
if soaSvr2Enabled == 'true':
createManagedServer(soaSvr2,server2Address,soaSvr2Port,soaClr,server2Machine,
soaJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
else:
print('Do not create SOA Server2')
#
# OSB
if osbEnabled == 'true':
createCluster(osbClr)
adaptManagedServer('osb_server1',osbSvr1,server1Address,osbSvr1Port,osbClr,server1Machine,
osbJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
if osbSvr2Enabled == 'true':
createManagedServer(osbSvr2,server2Address,osbSvr2Port,osbClr,server2Machine,
osbJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
else:
print('Do not create OSB Server2')
#
# BAM
if bamEnabled == 'true':
createCluster(bamClr)
adaptManagedServer('bam_server1',bamSvr1,server1Address,bamSvr1Port,bamClr,server1Machine,
bamJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
if bamSvr2Enabled == 'true':
createManagedServer(bamSvr2,server2Address,bamSvr2Port,bamClr,server2Machine,
bamJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
else:
print('Do not create BAM Server2')
#
# ESS
if essEnabled == 'true':
createCluster(essClr)
adaptManagedServer('ess_server1',essSvr1,server1Address,essSvr1Port,essClr,server1Machine,
essJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
if essSvr2Enabled == 'true':
createManagedServer(essSvr2,server2Address,essSvr2Port,essClr,server2Machine,
essJavaArgsBase,fileCount,fileMinSize,rotationType,fileTimeSpan)
else:
print('Do not create ESS Server2')
#
print ('Finshed creating Machines, Clusters and ManagedServers')
#
# Section 5: Add Servers to ServerGroups.
print ('\n5. Add Servers to ServerGroups')
print (lineSeperator)
cd('/')
print 'Add server groups '+adminSvrGrpDesc+ ' to '+adminServerName
setServerGroups(adminServerName, adminSvrGrp)
# SOA
if soaEnabled == 'true':
print 'Add server group '+soaSvrGrpDesc+' to '+soaSvr1+' and possibly '+soaSvr2
setServerGroups(soaSvr1, soaSvrGrp)
if soaSvr2Enabled == 'true':
setServerGroups(soaSvr2, soaSvrGrp)
#
# OSB
if osbEnabled == 'true':
print 'Add server group '+osbSvrGrpDesc+' to '+osbSvr1+' and possibly '+osbSvr2
setServerGroups(osbSvr1, osbSvrGrp)
if osbSvr2Enabled == 'true':
setServerGroups(osbSvr2, osbSvrGrp)
#
if bamEnabled == 'true':
print 'Add server group '+bamSvrGrpDesc+' to '+bamSvr1+' and possibly '+bamSvr2
setServerGroups(bamSvr1, bamSvrGrp)
if bamSvr2Enabled == 'true':
setServerGroups(bamSvr2, bamSvrGrp)
#
if essEnabled == 'true':
print 'Add server group '+essSvrGrpDesc+' to '+essSvr1+' and possibly '+essSvr2
setServerGroups(essSvr1, essSvrGrp)
if essSvr2Enabled == 'true':
setServerGroups(essSvr2, essSvrGrp)
#
print ('Finshed ServerGroups.')
#
updateDomain()
closeDomain();
#
# Section 6: Create boot properties files.
print ('\n6. Create boot properties files')
print (lineSeperator)
# SOA
if soaEnabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+soaSvr1+'/security','boot.properties',adminUser,adminPwd)
if soaSvr2Enabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+soaSvr2+'/security','boot.properties',adminUser,adminPwd)
#
# OSB
if osbEnabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+osbSvr1+'/security','boot.properties',adminUser,adminPwd)
if osbSvr2Enabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+osbSvr2+'/security','boot.properties',adminUser,adminPwd)
#
if bamEnabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+bamSvr1+'/security','boot.properties',adminUser,adminPwd)
if bamSvr2Enabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+bamSvr1+'/security','boot.properties',adminUser,adminPwd)
#
if essEnabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+essSvr1+'/security','boot.properties',adminUser,adminPwd)
if essSvr2Enabled == 'true':
createBootPropertiesFile(soaDomainHome+'/servers/'+essSvr2+'/security','boot.properties',adminUser,adminPwd)
#
print ('\nFinished')
#
print('\nExiting...')
exit()
except NameError, e:
print 'Apparently properties not set.'
print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
usage()
except:
apply(traceback.print_exception, sys.exc_info())
stopEdit('y')
exit(exitcode=1)
#call main()
main()
exit()
Conclusion
As said, although I think this script is already quite adaptable using the property file, of course there are many improvements thinkable for your particular situation. It creates a 'skeleton' SOA or Service Bus domain, but you might need to adapt for network topologies, security settings.And although it creates a 'per domain' nodemanager configuration, you would need to adapt it for your particular needs to get the domain started. I only tested this by starting the Admin server using the startWeblogic.sh script.
Having such a script is such a valuable asset: it allows you to (re-)create your domains repeatably in a standard way, ensuring that different environments (dev, test, acc, prod) are created similarly.
One, last thing though, the script somehow registers the creation of the domain and thus the use of the datasources in the repository. So you can't just throw away the domain and recreate it to the current Repository. You'll need to rereate the Repository as well.
Labels:
FMWInstallation
,
Oracle Service Bus
,
SOA Suite
,
Weblogic
,
WLST
Wednesday, 25 May 2016
Automatic install of SOA Suite and Service Bus 12cR2.
Lately I worked on a set of scripts to automatically install Weblogic Infrastructure, SOA/BPM Suite, Service Bus, etc. Since I implemented a reworked set yesterday at another customer it might be nice to describe them here.
The scripts help in installing the software and creating the Repository. I started to create a script for creating the domain, but haven't it working yet. A good starting poing would be this blog of Edwin Biemond for the 12cR1 (12.1.3) version. If I managed have it working for 12c related to my other scripts I will get back to it. Probably a nice reference would also be this description of Lucas Jellema (also 12.1.3).
To create the scripts I followed the Enterprise Deployment guide for SOASuite 12c, Install tasks documentation. To administer your different environments (dev, test, acc, prod) of the Fusion Middleware the Enterprise Deployment Workbook might come in handy. And then there is the Installing and Configuring Oracle SOA Suite and Oracle Business Process Management.
The scripts are based on my earlier work on the automatic install of the quickstarts under Linux.
By the way: for these scripts I use shell (bash) under Linux. But since the response files use references that you'd probably want to have based on properties (I would) I should rework those using something like awk/sed (which I don't know) or ANT (which I do know, but need an ANT installation. But maybe in a next phase.
For this installation we need the following downloads, from edelivery:
The scripts and software is placed in a folderstructure containing the following sub-folders:
The scripts and response (.rsp) files I'll explain below. In each product subfolder there is the downloaded zip file (containing the installation-jar file) and the accompanying response file. In the scripts folders there are the product installation scripts and the master script install.sh. So create a folder structure as above and place the downloaded products and the provided scripts in the appropriate folder.
So here we go.
Adapt the location of the FMW_HOME and possibly the (desired or current) location of your JAVA_HOME. The other 'homes' are relative to the FMW_HOME: these are the locations within the FMW_HOME where the products are installed (In 11g these were Oracle_SOA1 or Oracle_OSB1.
Update the JAVA_INSTALL_RPM according to the downloaded rpm as placed in the Java subfolder. Again adapt the JAVA_HOME in the fmw12c_env.sh accordingly.What this script does is check if the folder as in JAVA_HOME exists. If not then apparently the denoted version is not installed and so it does.
Sudo grants to oracle-user
To be able to run the script above (since it uses rpm via sudo) we need to adapt the sudo-ers file.
Log on as root via the command:
Edit de sudoers file:
Uncomment the lines for the Cmnd_Alias-es SOFTWARE en SERVICES (remove the hash ’#’ at the beginning of the line):
And add the follwing two lines at the end of the file:
Save the file (use an exclamation mark in the ‘:wq!’ command, since sudoers is readonly.After this you can run the installJava.sh.
The install script is as follows:
Save it as installFMW.sh under scripts. As in the installJava.sh this script checks if the FMW_HOME already exists. If not it checks on the availability of the installer-jar. If not then it checks the zip file that should contain the installer-jar. If so then it will unzip the zipfile. If the zip file does not exist then it stops with a message. You can unzip the zip-file prior in starting the scripts, because that is the primary requirement. You can leave the jar file for subsequent installation on other servers. It would be handy if you put this on a shared staging repository folder.
If in the end the jar-file exists it starts the installer with java from the JAVA_HOME and performs a silent install using a a response file. This is a file that is recorded at the end of a manual installation session and contains the choices made in the Oracle Universal Installer wizard. It is placed together with the zip file in the product folder.
It looks like as follows:
Save it as fmw_12.2.1.0.0_infrastructure.rsp under OracleFMW12cInfrastructure.
If you choose to use another FMW_HOME as suggested, you'll need to change the ORACLE_HOME variable in the file accordingly. This is one of the elements that I want to have replaced automatically using a property, based on the FMW_HOME env-variable.
Save it as installSOA.sh under scripts.
This installs the software for both SOA and BPM. The choice to include BPM or not are made at creation of the domain. Or adapt the INSTALL_TYPE element in the response file below. this one use BPM, but if you adapt it to SOA (I haven't got the actual value at hand, but assume it would be SOA) I assume the BPM software is omitted.
As in the FMW infrastructure installation we need a response file:
Save it as fmw_12.2.1.0.0_soa.rsp under SOASuiteAndBPM.
This one is a little smaller then the FMW-infra one. And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Save it as installSB.sh under scripts.
This installs the software for both SOA and BPM. The choice to include BPM or not are made at creation of the domain.
As in the FMW infrastructure installation we need a response file:
Save it as fmw_12.2.1.0.0_osb.rsp under ServiceBus.
This one is a little smaller then the FMW-infra one. And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Save it as installMFT.sh under scripts.
Again we need a response file:
Save it as fmw_12.2.1.0.0_mft.rsp under ManagedFileTransfer.
And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Save it as install.sh under scripts.
The commandline interface of the RCU is described here. In that document the commandline interface and options are described. In turns out (but not described) that the RCU also supports a response file.
The rcu install script is as follows:
Save it as rcuSOA.sh under scripts.
This script uses both a respone file and a password file.
The response file is as follows:
Regarding the elements you want to fill using properties, this one is the largest. Important are mostly:
I think properties like connectString, databaseType, dbUser, dbRole speak more or less for them selves. The property 'schemaPrefix' need to be adapted according to the target environment. This can be something like DEV, TST, ACC or PRD. Or SOAO, SOAT, SOAA, SOAP (the last one is funny...)
Then the component list. For SOA and MFT there are several required components. These can be found here in the 12.1.3 docs. For 12.2.1 the list of component id's can be founde here. Unfortunately there you can't find the requirements in detail as in 12.1.3.
Then there is a password file. If you set useSamePasswordForAllSchemaUsers to true, you need only two: the sys password and the generic schema password. If as in this example the value is false you need to specify them for each schema. The password file I use looks like:
The first password in the list is the system password. Then in the order of the components the passwords are listed. A few remarks:
But first for me it would be a challence to create a domain script in wlst. So hopefully I get to write about that soon.
The scripts help in installing the software and creating the Repository. I started to create a script for creating the domain, but haven't it working yet. A good starting poing would be this blog of Edwin Biemond for the 12cR1 (12.1.3) version. If I managed have it working for 12c related to my other scripts I will get back to it. Probably a nice reference would also be this description of Lucas Jellema (also 12.1.3).
To create the scripts I followed the Enterprise Deployment guide for SOASuite 12c, Install tasks documentation. To administer your different environments (dev, test, acc, prod) of the Fusion Middleware the Enterprise Deployment Workbook might come in handy. And then there is the Installing and Configuring Oracle SOA Suite and Oracle Business Process Management.
The scripts are based on my earlier work on the automatic install of the quickstarts under Linux.
By the way: for these scripts I use shell (bash) under Linux. But since the response files use references that you'd probably want to have based on properties (I would) I should rework those using something like awk/sed (which I don't know) or ANT (which I do know, but need an ANT installation. But maybe in a next phase.
For this installation we need the following downloads, from edelivery:
Product
|
Jar File
|
Zip file
|
Note
|
| Fusion Middleware Infrastructure | fmw_12.2.1.0.0_infrastructure.jar | V78156-01.zip | OracleFMW12cInfrastructure |
| SOA & BPM Suite | fmw_12.2.1.0.0_soa.jar | V78169-01.zip | SOASuiteAndBPM |
| Service Bus | fmw_12.2.1.0.0_osb.jar | V78173-01.zip | ServiceBus |
| Managed File Transfer | fmw_12.2.1.0.0_mft.jar | V78174-01.zip | ManagedFileTransfer |
The scripts and software is placed in a folderstructure containing the following sub-folders:
Folder Name
|
Containing
|
| Java | Java jdk U74+ rpm: jdk-8u74-linux-x64.rpm |
| ManagedFileTransfer |
|
| OracleFMW12cInfrastructure |
|
| rcu |
|
| scripts |
|
| ServiceBus |
|
| SOASuiteAndBPM |
|
The scripts and response (.rsp) files I'll explain below. In each product subfolder there is the downloaded zip file (containing the installation-jar file) and the accompanying response file. In the scripts folders there are the product installation scripts and the master script install.sh. So create a folder structure as above and place the downloaded products and the provided scripts in the appropriate folder.
So here we go.
Setting the environment
First I need a fmw12c_env.sh script to set some basic environment variables and especially the location of the FMW_HOME, where the software is going to be installed:#!/bin/bash echo set Fusion MiddleWare 12cR2 environment export JAVA_HOME=/usr/java/jdk1.8.0_74 export FMW_HOME=/u01/app/oracle/FMW12210 export SOA_HOME=$FMW_HOME/soa export OSB_HOME=$FMW_HOME/osb export MFT_HOME=$FMW_HOME/mft
Adapt the location of the FMW_HOME and possibly the (desired or current) location of your JAVA_HOME. The other 'homes' are relative to the FMW_HOME: these are the locations within the FMW_HOME where the products are installed (In 11g these were Oracle_SOA1 or Oracle_OSB1.
Install Java
For the 12cR2 version of the we need an Java 8 Installment. Of course preferably the latest version but at least above Update 65. I used update 74, but you can change it to a later update. The script for the installation is as follows:#!/bin/bash . $PWD/fmw12c_env.sh export JAVA_INSTALL_HOME=$PWD/../Java export JAVA_INSTALL_RPM=jdk-8u74-linux-x64.rpm # echo JAVA_HOME=$JAVA_HOME if [ ! -d "$JAVA_HOME" ]; then # Install jdk echo Install jdk 1.8 sudo rpm -ihv $JAVA_INSTALL_HOME/$JAVA_INSTALL_RPM else echo jdk 1.8 already installed fiSave it as installJava.sh under scripts.
Update the JAVA_INSTALL_RPM according to the downloaded rpm as placed in the Java subfolder. Again adapt the JAVA_HOME in the fmw12c_env.sh accordingly.What this script does is check if the folder as in JAVA_HOME exists. If not then apparently the denoted version is not installed and so it does.
Sudo grants to oracle-user
To be able to run the script above (since it uses rpm via sudo) we need to adapt the sudo-ers file.
Log on as root via the command:
[oracle@darlin-vce- db ~]$ su - Password: Last login: Fri Feb 26 06:44:05 EST 2016 on pts/0
Edit de sudoers file:
[root@darlin-vce- db ~]# vi /etc/sudoers
Uncomment the lines for the Cmnd_Alias-es SOFTWARE en SERVICES (remove the hash ’#’ at the beginning of the line):
## Installation and management of software Cmnd_Alias SOFTWARE = /bin/rpm, /usr/bin/up2date, /usr/bin/yum ## Services Cmnd_Alias SERVICES = /sbin/service, /sbin/chkconfig
And add the follwing two lines at the end of the file:
## Extra rights for oracle to do for instance rpm without password. oracle ALL= NOPASSWD: SERVICES, SOFTWARE
Save the file (use an exclamation mark in the ‘:wq!’ command, since sudoers is readonly.After this you can run the installJava.sh.
Install Infrastructure
First we need to install the Fusion Middleware InfraStructure.This is a Weblogic Server delivery that includes a RCU for the infrastructure schema's in the database. You can't use the 'vanilla' delivery of weblogic server, you'll need this one.The install script is as follows:
#!/bin/bash
. $PWD/fmw12c_env.sh
#
export FMW_INSTALL_HOME=$PWD/../OracleFMW12cInfrastructure
export FMW_INSTALL_JAR=fmw_12.2.1.0.0_infrastructure.jar
export FMW_INSTALL_RSP=fmw_12.2.1.0.0_infrastructure.rsp
export FMW_INSTALL_ZIP=V78156-01.zip
#
# Fusion Middlware Infrastucture
if [ ! -d "$FMW_HOME" ]; then
#Unzip FMW
if [ ! -f "$FMW_INSTALL_HOME/$FMW_INSTALL_JAR" ]; then
if [ -f "$FMW_INSTALL_HOME/$FMW_INSTALL_ZIP" ]; then
echo Unzip $FMW_INSTALL_HOME/$FMW_INSTALL_ZIP to $FMW_INSTALL_HOME/$FMW_INSTALL_JAR
unzip $FMW_INSTALL_HOME/$FMW_INSTALL_ZIP -d $FMW_INSTALL_HOME
else
echo $FMW_INSTALL_HOME/$FMW_INSTALL_ZIP does not exist
fi
else
echo $FMW_INSTALL_JAR already unzipped.
fi
if [ -f "$FMW_INSTALL_HOME/$FMW_INSTALL_JAR" ]; then
echo Install Fusion Middleware Infrastucture 12cR2
$JAVA_HOME/bin/java -jar $FMW_INSTALL_HOME/$FMW_INSTALL_JAR -silent -responseFile $FMW_INSTALL_HOME/$FMW_INSTALL_RSP
else
echo $FMW_INSTALL_JAR not available!
fi
else
echo $FMW_HOME available: Fusion Middleware 12c Infrastucture already installed.
fi
Save it as installFMW.sh under scripts. As in the installJava.sh this script checks if the FMW_HOME already exists. If not it checks on the availability of the installer-jar. If not then it checks the zip file that should contain the installer-jar. If so then it will unzip the zipfile. If the zip file does not exist then it stops with a message. You can unzip the zip-file prior in starting the scripts, because that is the primary requirement. You can leave the jar file for subsequent installation on other servers. It would be handy if you put this on a shared staging repository folder.
If in the end the jar-file exists it starts the installer with java from the JAVA_HOME and performs a silent install using a a response file. This is a file that is recorded at the end of a manual installation session and contains the choices made in the Oracle Universal Installer wizard. It is placed together with the zip file in the product folder.
It looks like as follows:
[ENGINE] #DO NOT CHANGE THIS. Response File Version=1.0.0.0.0 [GENERIC] #Set this to true if you wish to skip software updates DECLINE_AUTO_UPDATES=true # MOS_USERNAME= # MOS_PASSWORD=<SECURE VALUE> #If the Software updates are already downloaded and available on your local system, then specify the path to the directory where these patches are available and set SPECIFY_DOWNLOAD_LOCATION to true AUTO_UPDATES_LOCATION= # SOFTWARE_UPDATES_PROXY_SERVER= # SOFTWARE_UPDATES_PROXY_PORT= # SOFTWARE_UPDATES_PROXY_USER= # SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE> #The oracle home location. This can be an existing Oracle Home or a new Oracle Home ORACLE_HOME=/u01/app/oracle/FMW12210 #Set this variable value to the Installation Type selected. e.g. Fusion Middleware Infrastructure, Fusion Middleware Infrastructure With Examples. INSTALL_TYPE=Fusion Middleware Infrastructure #Provide the My Oracle Support Username. If you wish to ignore Oracle Configuration Manager configuration provide empty string for user name. MYORACLESUPPORT_USERNAME= #Provide the My Oracle Support Password MYORACLESUPPORT_PASSWORD=<SECURE VALUE> #Set this to true if you wish to decline the security updates. Setting this to true and providing empty string for My Oracle Support username will ignore the Oracle Configuration Manager configuration DECLINE_SECURITY_UPDATES=true #Set this to true if My Oracle Support Password is specified SECURITY_UPDATES_VIA_MYORACLESUPPORT=false #Provide the Proxy Host PROXY_HOST= #Provide the Proxy Port PROXY_PORT= #Provide the Proxy Username PROXY_USER= #Provide the Proxy Password PROXY_PWD=<SECURE VALUE> #Type String (URL format) Indicates the OCM Repeater URL which should be of the format [scheme[Http/Https]]://[repeater host]:[repeater port] COLLECTOR_SUPPORTHUB_URL=
Save it as fmw_12.2.1.0.0_infrastructure.rsp under OracleFMW12cInfrastructure.
If you choose to use another FMW_HOME as suggested, you'll need to change the ORACLE_HOME variable in the file accordingly. This is one of the elements that I want to have replaced automatically using a property, based on the FMW_HOME env-variable.
Install SOA and BPM Suite
The script for installation of the SOA and BPM Software is more or less the same as the FMW Infrastructure:#!/bin/bash
. $PWD/fmw12c_env.sh
#
export SOA_INSTALL_HOME=$PWD/../SOASuiteAndBPM
export SOA_INSTALL_JAR=fmw_12.2.1.0.0_soa.jar
export SOA_INSTALL_RSP=fmw_12.2.1.0.0_soa.rsp
export SOA_INSTALL_ZIP=V78169-01.zip
#
# SOA and BPM Suite 12c
if [[ -d "$FMW_HOME" && ! -d "$SOA_HOME" ]]; then
#
#Unzip SOA&BPM
if [ ! -f "$SOA_INSTALL_HOME/$SOA_INSTALL_JAR" ]; then
if [ -f "$SOA_INSTALL_HOME/$SOA_INSTALL_ZIP" ]; then
echo Unzip $SOA_INSTALL_HOME/$SOA_INSTALL_ZIP to $SOA_INSTALL_HOME/$SOA_INSTALL_JAR
unzip $SOA_INSTALL_HOME/$SOA_INSTALL_ZIP -d $SOA_INSTALL_HOME
else
echo $SOA_INSTALL_HOME/$SOA_INSTALL_ZIP does not exist!
fi
else
echo $SOA_INSTALL_JAR already unzipped
fi
if [ -f "$SOA_INSTALL_HOME/$SOA_INSTALL_JAR" ]; then
echo Install SOA and BPM Suite 12cR2
$JAVA_HOME/bin/java -jar $SOA_INSTALL_HOME/$SOA_INSTALL_JAR -silent -responseFile $SOA_INSTALL_HOME/$SOA_INSTALL_RSP
else
echo $SOA_INSTALL_JAR not available!.
fi
else
if [ ! -d "$FMW_HOME" ]; then
echo $FMW_HOME not available: First install Fusion Middlware Infrastucture
fi
if [ -d "$SOA_HOME" ]; then
echo $SOA_HOME available: SOA Already installed
fi
fi
Save it as installSOA.sh under scripts.
This installs the software for both SOA and BPM. The choice to include BPM or not are made at creation of the domain. Or adapt the INSTALL_TYPE element in the response file below. this one use BPM, but if you adapt it to SOA (I haven't got the actual value at hand, but assume it would be SOA) I assume the BPM software is omitted.
As in the FMW infrastructure installation we need a response file:
[ENGINE] #DO NOT CHANGE THIS. Response File Version=1.0.0.0.0 [GENERIC] #Set this to true if you wish to skip software updates DECLINE_AUTO_UPDATES=true # MOS_USERNAME= # MOS_PASSWORD=<SECURE VALUE> #If the Software updates are already downloaded and available on your local system, then specify the path to the directory where these patches are available and set SPECIFY_DOWNLOAD_LOCATION to true AUTO_UPDATES_LOCATION= # SOFTWARE_UPDATES_PROXY_SERVER= # SOFTWARE_UPDATES_PROXY_PORT= # SOFTWARE_UPDATES_PROXY_USER= # SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE> #The oracle home location. This can be an existing Oracle Home or a new Oracle Home ORACLE_HOME=/u01/app/oracle/FMW12210 #Set this variable value to the Installation Type selected. e.g. SOA Suite, BPM. INSTALL_TYPE=BPM
Save it as fmw_12.2.1.0.0_soa.rsp under SOASuiteAndBPM.
This one is a little smaller then the FMW-infra one. And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Install Service Bus
The script for installation of the Service Bus Software is more or less the same as the SOA and BPM:#!/bin/bash
. $PWD/fmw12c_env.sh
#
export OSB_INSTALL_HOME=$PWD/../ServiceBus
export OSB_INSTALL_JAR=fmw_12.2.1.0.0_osb.jar
export OSB_INSTALL_RSP=fmw_12.2.1.0.0_osb.rsp
export OSB_INSTALL_ZIP=V78173-01.zip
#
# ServiceBus 12c
if [[ -d "$FMW_HOME" && ! -d "$OSB_HOME/bin" ]]; then
#
#Unzip ServiceBus
if [ ! -f "$OSB_INSTALL_HOME/$OSB_INSTALL_JAR" ]; then
if [ -f "$OSB_INSTALL_HOME/$OSB_INSTALL_ZIP" ]; then
echo Unzip $OSB_INSTALL_HOME/$OSB_INSTALL_ZIP to $OSB_INSTALL_HOME/$OSB_INSTALL_JAR
unzip $OSB_INSTALL_HOME/$OSB_INSTALL_ZIP -d $OSB_INSTALL_HOME
else
echo $OSB_INSTALL_HOME/$OSB_INSTALL_ZIP does not exist!
fi
else
echo $OSB_INSTALL_JAR already unzipped
fi
if [ -f "$OSB_INSTALL_HOME/$OSB_INSTALL_JAR" ]; then
echo Install ServiceBus 12cR2
$JAVA_HOME/bin/java -jar $OSB_INSTALL_HOME/$OSB_INSTALL_JAR -silent -responseFile $OSB_INSTALL_HOME/$OSB_INSTALL_RSP
else
echo $OSB_INSTALL_JAR not available!
fi
else
if [ ! -d "$FMW_HOME" ]; then
echo $FMW_HOME not available: First install Fusion Middlware Infrastucture
fi
if [ -d "$OSB_HOME" ]; then
echo $OSB_HOME available: ServiceBus Already installed
fi
fi
Save it as installSB.sh under scripts.
This installs the software for both SOA and BPM. The choice to include BPM or not are made at creation of the domain.
As in the FMW infrastructure installation we need a response file:
[ENGINE] #DO NOT CHANGE THIS. Response File Version=1.0.0.0.0 [GENERIC] #Set this to true if you wish to skip software updates DECLINE_AUTO_UPDATES=true # MOS_USERNAME= # MOS_PASSWORD=<SECURE VALUE> #If the Software updates are already downloaded and available on your local system, then specify the path to the directory where these patches are available and set SPECIFY_DOWNLOAD_LOCATION to true AUTO_UPDATES_LOCATION= # SOFTWARE_UPDATES_PROXY_SERVER= # SOFTWARE_UPDATES_PROXY_PORT= # SOFTWARE_UPDATES_PROXY_USER= # SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE> #The oracle home location. This can be an existing Oracle Home or a new Oracle Home ORACLE_HOME=/u01/app/oracle/FMW12210 #Set this variable value to the Installation Type selected. e.g. Service Bus. INSTALL_TYPE=Service Bus
Save it as fmw_12.2.1.0.0_osb.rsp under ServiceBus.
This one is a little smaller then the FMW-infra one. And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Install Managed File Transfer
The script for installation of the Managed File Transfer Software is again more or less the same as the SOA and BPM:#!/bin/bash
. $PWD/fmw12c_env.sh
#
export MFT_INSTALL_HOME=$PWD/../ManagedFileTransfer
export MFT_INSTALL_JAR=fmw_12.2.1.0.0_mft.jar
export MFT_INSTALL_RSP=fmw_12.2.1.0.0_mft.rsp
export MFT_INSTALL_ZIP=V78174-01.zip
#
# MFT 12c
if [[ -d "$FMW_HOME" && ! -d "$MFT_HOME/bin" ]]; then
#
#Unzip MFT
if [ ! -f "$MFT_INSTALL_HOME/$MFT_INSTALL_JAR" ]; then
if [ -f "$MFT_INSTALL_HOME/$MFT_INSTALL_ZIP" ]; then
echo Unzip $MFT_INSTALL_HOME/$MFT_INSTALL_ZIP to $MFT_INSTALL_HOME/$MFT_INSTALL_JAR
unzip $MFT_INSTALL_HOME/$MFT_INSTALL_ZIP -d $MFT_INSTALL_HOME
else
echo $MFT_INSTALL_HOME/$MFT_INSTALL_ZIP does not exist!
fi
else
echo $MFT_INSTALL_JAR already unzipped
fi
if [ -f "$MFT_INSTALL_HOME/$MFT_INSTALL_JAR" ]; then
echo Install MFT 12cR2
$JAVA_HOME/bin/java -jar $MFT_INSTALL_HOME/$MFT_INSTALL_JAR -silent -responseFile $MFT_INSTALL_HOME/$MFT_INSTALL_RSP
else
echo $MFT_INSTALL_JAR not available!
fi
else
if [ ! -d "$FMW_HOME" ]; then
echo $FMW_HOME not available: First install Fusion Middlware Infrastucture
fi
if [ -d "$MFT_HOME" ]; then
echo $MFT_HOME available: MFT Already installed
fi
fi
Save it as installMFT.sh under scripts.
Again we need a response file:
[ENGINE] #DO NOT CHANGE THIS. Response File Version=1.0.0.0.0 [GENERIC] #Set this to true if you wish to skip software updates DECLINE_AUTO_UPDATES=true # MOS_USERNAME= # MOS_PASSWORD=<SECURE VALUE> #If the Software updates are already downloaded and available on your local system, then specify the path to the directory where these patches are available and set SPECIFY_DOWNLOAD_LOCATION to true AUTO_UPDATES_LOCATION= # SOFTWARE_UPDATES_PROXY_SERVER= # SOFTWARE_UPDATES_PROXY_PORT= # SOFTWARE_UPDATES_PROXY_USER= # SOFTWARE_UPDATES_PROXY_PASSWORD=<SECURE VALUE> #The oracle home location. This can be an existing Oracle Home or a new Oracle Home ORACLE_HOME=/u01/app/oracle/FMW12210
Save it as fmw_12.2.1.0.0_mft.rsp under ManagedFileTransfer.
And again here the ORACLE_HOME should be adapted in the case you choose to use another FMW_HOME location.
Install the lot
You could run the scripts above one-by-one. Or have them called using a master script:#!/bin/bash echo _______________________________________________________________________________ echo Java SDK 8 ./installJava.sh echo echo _______________________________________________________________________________ echo Fusion Middleware Infrastructure ./installFMW.sh echo echo _______________________________________________________________________________ echo SOA & BPM Suite ./installSOA.sh echo echo _______________________________________________________________________________ echo ServiceBus ./installSB.sh echo echo _______________________________________________________________________________ echo Managed File Transfer ./installMFT.sh
Save it as install.sh under scripts.
Repository Creation
When the software is installed, it's time to create the repository. This requires:- a database, for instance an 11g XE, 11gR2 latest or 12c
- Sys password
The commandline interface of the RCU is described here. In that document the commandline interface and options are described. In turns out (but not described) that the RCU also supports a response file.
The rcu install script is as follows:
#!/bin/bash . $PWD/FMW12c_env.sh echo Run rcu for SOA Infrastucture export RCU_INSTALL_HOME=$PWD/../rcu export RCU_SOA_RSP=rcuSOA.rsp export RCU_SOA_PWD=rcuSOAPasswords.txt #export RCU_SOA_PWD=rcuSOAPasswords-same.txt $FMW_HOME/oracle_common/bin/rcu -silent -responseFile $RCU_INSTALL_HOME/$RCU_SOA_RSP -f < $RCU_INSTALL_HOME/$RCU_SOA_PWD
Save it as rcuSOA.sh under scripts.
This script uses both a respone file and a password file.
The response file is as follows:
#RCU Operation - createRepository, generateScript, dataLoad, dropRepository, consolidate, generateConsolidateScript, consolidateSyn, dropConsolidatedSchema, reconsolidate operation=createRepository #Enter the database connection details in the supported format. Database Connect String. This can be specified in the following format - For Oracle Database: host:port:SID OR host:port/service , For SQLServer, IBM DB2, MySQL and JavaDB Database: Server name/host:port:databaseName. For RAC database, specify VIP name or one of the Node name as Host name.For SCAN enabled RAC database, specify SCAN host as Host name. connectString=darlin-vce-db:1521:PDBORCL #Database Type - [ORACLE|SQLSERVER|IBMDB2|EBR|MYSQL] - default is ORACLE databaseType=ORACLE #Database User dbUser=sys #Database Role - sysdba or Normal dbRole=SYSDBA #This is applicable only for database type - EBR #edition= #Prefix to be used for the schema. This is optional for non-prefixable components. schemaPrefix=DEV #List of components separated by comma. Remove the components which are not needed. componentList=UCSUMS,MDS,WLS,STB,OPSS,IAU,IAU_APPEND,IAU_VIEWER,SOAINFRA,ESS,MFT #Specify whether dependent components of the given componentList have to be selected. true | false - default is false #selectDependentsForComponents=false #If below property is set to true, then all the schemas specified will be set to the same password. useSamePasswordForAllSchemaUsers=false #This allows user to skip cleanup on failure. yes | no. Default is no. #skipCleanupOnFailure=no #Yes | No - default is Yes. This is applicable only for database type - SQLSERVER. #unicodeSupport=no #Location of ComponentInfo xml file - optional. #compInfoXMLLocation= #Location of Storage xml file - optional #storageXMLLocation= #Tablespace name for the component. Tablespace should already exist if this option is used. #tablespace= #Temp tablespace name for the component. Temp Tablespace should already exist if this option is used. #tempTablespace= #Absolute path of Wallet directory. If wallet is not provided, passwords will be prompted. #walletDir= #true | false - default is false. RCU will create encrypted tablespace if TDE is enabled in the database. #encryptTablespace=false #true | false - default is false. RCU will create datafiles using Oracle-Managed Files (OMF) naming format if value set to true. #honorOMF=false #Variable required for component SOAINFRA. Database Profile (SMALL/MED/LARGE) SOA_PROFILE_TYPE=SMALL #Variable required for component SOAINFRA. Healthcare Integration(YES/NO) HEALTHCARE_INTEGRATION=NO
Regarding the elements you want to fill using properties, this one is the largest. Important are mostly:
- connectString=darlin-vce-db:1521:PDBORCL
- databaseType=ORACLE
- dbUser=sys
- dbRole=SYSDBA
- schemaPrefix=DEV
- componentList=UCSUMS,MDS,WLS,STB,OPSS,IAU,IAU_APPEND,IAU_VIEWER,SOAINFRA,ESS,MFT
- useSamePasswordForAllSchemaUsers=false
I think properties like connectString, databaseType, dbUser, dbRole speak more or less for them selves. The property 'schemaPrefix' need to be adapted according to the target environment. This can be something like DEV, TST, ACC or PRD. Or SOAO, SOAT, SOAA, SOAP (the last one is funny...)
Then the component list. For SOA and MFT there are several required components. These can be found here in the 12.1.3 docs. For 12.2.1 the list of component id's can be founde here. Unfortunately there you can't find the requirements in detail as in 12.1.3.
Then there is a password file. If you set useSamePasswordForAllSchemaUsers to true, you need only two: the sys password and the generic schema password. If as in this example the value is false you need to specify them for each schema. The password file I use looks like:
welcome1 DEV_UMS DEV_MDS DEV_WLS DEV_WLS_RUNTIME DEV_STB DEV_OPSS DEV_IAU DEV_IAU_APPEND DEV_IAU_VIEWER DEV_SOAINFRA DEV_ESS DEV_MFT
The first password in the list is the system password. Then in the order of the components the passwords are listed. A few remarks:
- The component UCSUMS (User Messaging Services) result in a schema DEV_UMS (provided that he schemaPrefix = DEV).
- I use here passwords that equal the schema names. You probably would not do that in acceptance and/or production, but maybe you do in Dev and test. However, in the example it is handy to know at which place which password need to go.
- The component WLS needs two passwords, since it results in two schema's: DEV_WLS and DEV_WLS_RUNTIME. It is not documented (I could not find it) but it took me considerable time, since afte DEV_WLS the passwords did not match and it complained about a missing password. Looking in a manual created repository I found that it also created the DEV_WLS_RUNTIME.
- For Managed File Transfer (MFT) also Enterprise Schedule Service (ESS) is needed. As well as the prerequisites for SOAINFRA.
- SOAINFRA is needed for both SOA&BPM and Service Bus. So even if you only install Service Bus, you need to install SOAINFRA.
Conclusion
As said I these scripts help in installing the software and installing the Repository. They use shell scripts but it should not be too hard to translate them to ANT or other tooling like Ansible or Puppet if you're into one of those. To me it would be a nice finger-practice to translate it to ANT to be able to dynamically adapt the response files. I'd probably do that in the near future. And it would be a nice learning path to implement this in Ansible or Puppet.But first for me it would be a challence to create a domain script in wlst. So hopefully I get to write about that soon.
Labels:
FMWInstallation
,
Linux
,
Oracle Service Bus
,
SOA Suite
Subscribe to:
Posts
(
Atom
)









