Tuesday 30 September 2008

Tuning Soasuite 10133 and 11gDB under VMWare

In earlier posts I wrote about how to install the Oracle DB11g and SoaSuite 10133 in an Oracle Enterprise Linux based VM. Earlier this year I also wrote how to rename your SoaSuite installation when having renamed your host.

I did in fact a more or less default installation of both 11gDB and a SoaSuite 10133. But since the VM was to run on a 2GB laptop for courses I gave the VM only 1.6GB of memory. The database was sized so that it claimed about 640MB and the default J2EE+Webserver+soasuite installtion of the midtier resulted in two OC4J instances that both had a minimum heapsize of 512MB and a max of 1024. So at starting both database and soasuite I ran out of memory what results in a guest-os that gets very befriended with the harddrive (swapping all over the place).

So I took some time in getting the system tuned.

The database
First step was getting the database downsized. Earlier I shrinked the sga_max_size to about 470MB. So that was allready an improvement of about 170MB.

But that was not enough for me. So what I did was to startup an XE database. There I looked at the basic memory settings. For convenience I created a plain init.ora.
For the non-dba's amongst you, you can do that by loging on as internal with:
sqlplus "/ as sysdba"
having set the ORACLE_HOME and ORACLE_SID:
ORACLE_HOME=/u01/app/oracle/product/11.1.0/db_1
ORACLE_SID=orcl
When you logged on as internal you can create an init.ora (also called a pfile) with:
create pfile from spfile;
Then you'll find an init.ora in the $ORACLE_HOME/dbs folder.

For an Oracle XE database the most interesting settings I found were:
  • java_pool_size=4194304
  • large_pool_size=4194304
  • shared_pool_size=67108864
  • open_cursors=300
  • sessions=20
  • pga_aggregate_target=70M
  • sga_target=210M
The sga_max_size was not set.
So I changed the 11g database with these settings, created a spfile from the pfile again (create spfile from pfile) started it again.

My initorcl.ora:
#orcl.__db_cache_size=222298112
#orcl.__java_pool_size=12582912
orcl.__java_pool_size=10M
orcl.__large_pool_size=4194304
....
#orcl.__pga_aggregate_target=159383552
orcl.__pga_aggregate_target=70M
#orcl.__sga_target=478150656
orcl.__sga_target=210M
orcl.__shared_io_pool_size=0
#orcl.__shared_pool_size=234881024
orcl.__shared_pool_size=100M
orcl.__streams_pool_size=0
...
#*.memory_target=635437056
*.open_cursors=300
#*.processes=150
*.sessions=20
...
*.sga_max_size=250
...

Mark that I unset the db_cache_size and memory_target. I also replaced the processes parameter with the sessions parameter being 20. These two parameters relate to eachother, one computed from the other.

I found that I had a database of 145MB! I could start the middletier, but then I could nog logon myself because of the shared-poolsize being to small. This turned out to be about 64M, while the sga_max_size (that I did not set) was 145M.

I changed my sga_max_size to explicitly 250M and the large_pool_size to 100M:
SQL> alter system set sga_max_size=250M scope=spfile;
System altered.
SQL> alter system set shared_pool_size=100M scope=spfile;
System altered.
Then restarting the database resulted in a database of 250M:
Total System Global Area 263639040 bytes
Fixed Size 1299284 bytes
Variable Size 209718444 bytes
Database Buffers 50331648 bytes
Redo Buffers 2289664 bytes

That looks better to me.

Total System Global Area 263639040 bytes
Fixed Size 1299284 bytes
Variable Size 209718444 bytes
Database Buffers 50331648 bytes
Redo Buffers 2289664 bytes

That looks better to me.

The MidTier
The changes in the middle tier are a little less complicated. In fact you have to change two settings in opmn. So go to the $ORACLE_HOME/opmn/conf directory of the middle tier.
There you'll find a file called opmn.xml.

In that file look for:
process-type id="home" module-id="OC4J" status="enabled"
Below that you'll find a node with start-parameters, having a data sub-node with "java-options". In the value-attribute of that node change -ms512M -mx1024M into -ms128M -mx128M. These are the minimum and maximum heapsizes. The home oc4j only needs 128M. It's recommended to give the OC4J at startup the max heapsize right away. Then it need not to grow.

Look again for:
process-type id="oc4j_soa" module-id="OC4J" status="enabled"
Find the same start-parameters, and do the same change but then give it heapsizes of 384M: -ms384M -mx384M.

Conclusion
This gave me aVM with a soasuite and 11g database that runs quite fine in a 1.6GB VM.
These settings are just "wet-thumb"-values. I must strictly say that these are not valid values for a production environment and even
might not be valid for a regular development environment with a significant number of developers.

But in my case it all fits, having even 40MB of memory left. According to "top" my VM is not swapping!

Wednesday 24 September 2008

Connecting to Oracle DB 11g in OEL50 under VM takes a long time

I previously described how to install 11g Database. I found that it took a long time when connecting to the database from outside the VM, using sqldeveloper, sqlplus, Pl/Sql Developer.

I use a host-only and a bridged adapter in the VM. It took me a while but I found that it has something to do with a DNS-lookup that the database does during the connection process. In my /etc/resolv.conf a reference to the host for the name server is registered. But on my host I don't have a DNS server. It should get it from the physical/bridged network or not at all.

I resolved it by uncommenting the lines in the /etc/resolv.conf (place a semi-colon before each line). Also you change your networksettings in the network-devices. in Oracle Enterprise Linux. On one of the tabs you'll find an entry to the prefered dns-server. And a domain. You should clear those lines.

Pl/Sql Developer under Wine 2

In my previous post I mentioned that running Pl/Sql developer under wine goes fine, but it does not show the icons on the buttons. Indeed it was the case with me. But somehow they appeared magically.

However, the other "minusses" still stand.

Friday 19 September 2008

Pl/Sql Developer under Wine

This week I installed Pl/Sql Developer under wine. It was pretty easy. To have it working you need to install an Oracle Instant client. Probably you could install a complete Oracle Client, but the Instant Client will do and it just gives you enough to run Pl/Sql developer.

I unzipped the 10gR2 instant client (Windows 32-bit) into /home/makker/.wine/drive_c/oracle/product/instantclient_10_2.
I also put the sql-plus addendum there, but that did not work.
Then you place a valid tnsnames.ora in the subdirectory /home/makker/.wine/drive_c/oracle/product/instantclient_10_2/Network/Admin.

Install Pl/Sql developer (I just ran the installer under wine). When starting Pl/Sql developer you first have to go to the preferences and then the connection part. (menu=> tools => preferences). There you have to point Pl/Sql developer to your instant client in a windows way: C:\oracle\product\instantclient_10_2\oci.dll. After doing that, restart Pl/Sql Developer. Then it will load the oci.dll.

Then, provided that you have a database running and a valid tnsnames.ora you can connect to your database. In my case the connection to my 11g database in the VM is very, very slow. I haven't figured out yet what causes it. But I got the same behaviour using SqlDeveloper.

I'm very pleased having Pl/Sql developer running under Linux. There are however a few points to figure out and/or improve:
  • Buttons in the button bar are not shown.
  • Some features just don't work, like the macro-recorder.
  • Sometimes when switching applications, the Pl/Sql developer pane is not repainted correctly or at all. I have to play with switch "shade" (right-click in the taskbar) on and of, to get Pl/Sql Developer shown again.
But for me Pl/sql developer is most productive tool for the job. So I accept these "instabilities" under wine.

Tuesday 16 September 2008

Integrating Hyperion DRM 9.3.2 with SoaSuite 10133

My current customer is implementing Hyperion DRM. In DRM Organizational hierarchies are stored. These hierarchies have to be exported to several output formats for several client-systems.

We advised to use Oracle SoaSuite for the integration in stead of building exports for every single target-system. But how do you get the exports out of DRM into SoaSuite? The original idea was to have a schedular call DRM to run the export to a (XML-) file and have SoaSuite polling to that file.

I've looked into the integration possibilities of DRM. DRM has been said to have WebServices but we could, upfront, not find out if the webservices are just Soap-Services (without WSDL's) or "real" Webservices described with WSDL's. It turns out that DRM has WSDL-described webservices. The url to the WSDL should be something like:
http://--drm-server--/mdm_ntier/--service--.asmx?WSDL
Where --drm-server-- is the host where the DRM server with the webbrowser is running, and --service-- is the particular service. So something like:
http://winxp.darwin-it.local/mdm_ntier/SessionMgr.asmx?WSDL

It turns out that the DRM webservices have multipart message-types. The ESB of Oracle SoaSuite 10133 does not like them. BPEL PM seems not having a problem with them. But the wsdl's use imported schemas from:
<s:import namespace="http://schemas.xmlsoap.org/soap/encoding/" >
<s:import namespace="http://schemas.xmlsoap.org/wsdl/" >
These need an internet connection to be validated and be used in JDeveloper. What you could do is put them on a local webserver and modify the wsdl's accordingly.

But having solved that it also turns out that BPEL gets a response message that does not seem to conform the WSDL. The message I got was:
"trailing block elements must have an id attribute"
So I created a webservice-proxy on a WSDL and that works fine. Using the HTTP-Analyser of JDeveloper I intercepted the response and although the webservice proxy did accept the response, and apparently is confident with the wsdl, it seems to mee that the response does not match the wsdl. And BPEL PM agrees with me. Or better, I agree with BPEL PM.

So that did not get us any further. I've learned from a contact at Oracle Development that the Webservices from DRM indeed are not supported by BPEL PM. In DRM 11 there are changes made to the webservices in a way that some simpler ones should be accepted by BPEL PM. But apperently others still aren't.

One could wonder how this could be? Aren't webservices just invented to have technology agnostic and flexible integration? I read somewhere that the way the wsdl's of DRM are created are quite regular in the .Net world. DRM is build in Delphi (I was really surprised to see such a high-end Bussiness Application being build with Delphi, since Turbo Pascal was my favorite programming environment on college/university).

DRM also delivers java-API's. To use them you need an sdk from DRM, that can be downloaded here.

We created a java-class using the examples in the api-documentation that is delivered with DRM. Look for mdm_ntier_api_932.pdf. Unfortunately I could not find this information on OTN for you.
This Java class connects with the master-data-management server. Then it looks-up an export, starts a job and then gets the output into a string. This string is then returned. Our businessguys defined some standard exports that deliver the data into an XML message.

On this java class we created a webservice, using the webservice generation wizard from JDeveloper. Actually, since the api's are a layer on the DRM webservices, in fact they are webservice proxies, we created a webservice on several DRM webservices.
This webservice is then callable from BPEL PM.
When deploying the webservice to the AS, you should deploy the jar's from the mdm_ntier_apis-sdk also, with your deployment-descriptor. I tried to upload them as separate shared libraries in OC4J, but that didn't work.

The exported XML message is parsed in BPEL PM using the parse-xml ora:parseEscapedXML function. To be able to transfer it we had to add a namespace in the root element, using
concat(substring-before(bpws:getVariableData('Receive_Export_onResult_InputVariable','payload','/ns1:RunExportProcessResponse/ns1:result'),'<MDMMetadata'),'<MDMMetadata xmlns="http://xmlns.customer.com/drm" ',substring-after(bpws:getVariableData('Receive_Export_onResult_InputVariable','payload','/ns1:RunExportProcessResponse/ns1:result'),'<MDMMetadata'))


In the Workflow Development Kit that can be downloaded using the link above, the same approach is used. So apparently Oracle also found that BPEL PM does not support the DRM Webservices and state that this is the way to go.

I did not put in any code in this blog-entry. But most of the java-code I got from the examples. Except for transferring the export-output into a string. But that is also quite straight-forward. And generating the webservice is just playing the wizard with the defaults.

Monday 15 September 2008

OpenSuse Fonts

As stated in my previous post I re-installed OpenSuse. In an earlier install a few months back I had the problem that my fonts were to big after installing the NVidia propietry driver.
Back then I had no clue what caused this problem. It appears that installing the propietry driver initiated the problem.
On a blog I thought I found a solution. But somehow it didn't resolve it for me.
What I also tried is to add windows fonts to my fonts system. Now I wanted those anyway since I have to exchange documents with colleagues that have windows. And for our company we have selected a company-style-font.

But what it I found that in my personal fonts settings the fonts DPI were not set correctly. Since I have a pretty high resolution laptop-screen (1920x1200) this is probably interpreted incorrectly. By default it is set to disabled, but you can set it 120 or 96. Setting it to 96 I got reasonably sized window-fonts.

Multicast DNS reprise

This weekend I had to reinstall my laptop. Maybe I could have repaired it, but I had some problems with a partition, had a partion with windows that I hardly used but was taking space and OpenSuse 11 was out for a while. So I desided to reinstall my laptop but now with OpenSuse 11.

Of course doing such a rigorous action will drive you in those problems you solved before and one of those are my Multicast DNS problems I wrote about earlier. But simply turning off the Avahi deamos did not help. I had to do another thing.

Luckily I found pretty quick another tip from the forums.opensuse.org. I added the following line to my /etc/host.conf file:
mdns off

This helped. Being able to write this blog-entry is the proof.

I had to disable the avahi-services also. Apparently adding the line in the /etc/host.conf is an additional action.

Tuesday 2 September 2008

Java Swing XML Editor and Tester

Today I fine-tuned my XML Editor and Testing tools that I published earlier. The main improvements are that I integrated the XML Editor in my XMLTesting tool. For this I had to enhance and restructure my XML Editor to make it a "plugable" JPanel. Another thing is that I had to make sure that changes in one panel is propagated to the central File-Objects. This latter need some improvements because the capturing of the window-change-events are aparently not so evident in Java Swing. And also I keep copies of the File Objects. I need to improve that in the near future.
You can download my xml-editor here. And the xmltesting-tool here. Just unpack the zips into a directory and adapt the scripts (I included a windows bat file and a linux shell script) so that the your java virtual machine is referenced properly. Maybe I should change it so that it relies on having java in your path.

Monday 1 September 2008

Wrap-Up of the SOA Partner Community Forum August 2008

Last week I went, as many others, the Oracle Soa Partner Community Forum at Cap Gemini in Utrecht, the Netherlands.

It was an interesting week. Although I must say that for me the most interesting was that I did not hear any significant different than I learned past two weeks. I wrote an article with some follow ups recently about the BEA acquisition of Oracle. The original one is being resyndicated here, what is something that I'm pretty proud of (sorry). So I'm quite pleased that what I stated before is still standing.

But I want to add some conclusions to the story.

ESB
There is some discussion about the position of the Oracle ESB. Is it going to be BEA's AL Service Bus? But what about the Oracle SoaSuite ESB? Well, I learned now explicitly that when you search for a standalone ESB, than the Oracle ESB, fka AquaLogic Service Bus (BEA's Service Bus) is promoted. The Oracle ESB is going to be around as a mediator in the SoaSuite 11g. I would certainly take good notice of the pricelists and discuss with your Oracle Sales Rep. Because, I would be curious about the pricing of the ESB's. Probably you would have to pay for the new Oracle ESB, while, if I understood correctly, when you would use the current SoaSuite10g ESB standalone (without BPEL, OWSM, etc.) you get it free. Provided that you have an Enterprise Edition License of Oracle AS 10g. So I would wonder if the SoaSuite Mediator (the former SoaSuite ESB) in 11g will have a separate line on the pricelist.
Since I'm a technical consultant, and the pricing-policies of Oracle (and probably other vendors) is something like "higher-math" don't take my word for it.

AIA and SoaSuite10g/11g
Oracle's Application Integration Architecture was also promoted during the Forum. I think it's a very interesting subject. Also when you can't use a pre-built Process Integration Pack (PIP) the AIA Foundation Pack could provide you a lot of plumbing, like error-handling, that prevents you from having to break your brains on. Unfortunately I don't have experience in implementing a custom integration using AIA Foundation pack , yet. But I would recommend it only if you plan to build a large integration from scratch. If you've done a lot allready with the SoaSuite or you plan to do just some small ESB integrations than it would probably be too expensive or at least take too much time to learn and to put in place. Than you'll be faster with a doing an own implementation. But it mainly depends on your requirements.

One thing I couldn't get asked, although I raised my hands several times, is how AIA is going to embrace SoaSuite 11g. AIA now heavily depends on the capabilities of SoaSuite 10g. I'm curious on how AIA is going to use SCA for example. But there are also several PIP's ready. Migrating them to SoaSuite 11g will gain a lot of knowledge that other SoaSuite 10g customers will be interested in!

Another thing I read between the lines is that SoaSuite 11g is not expected at OpenWorld but somewhere in Fiscal Year 09. That is somewhere between now and may 31st 2009. Together with the fact that AIA is still relying on 10g, I dare to say that SoaSuite 10g will be around for quite a while. Don't be affraid to invest in SoaSuite10g. I think that it is a good, reliable product.

Weblogic Application Server
On Thursday I attended a workshop on Oracle WebLogic 10.3. Unfortunately the labs were mainly about creating some J2EE projects using Weblogic Workshop (which is a packaging of Weblogic addons on Eclipse). Although it was nice, and although I lost time strugling with a VMware image, I was mainly interested in how Weblogic was different from Oracle AS. It was nice to see that, from my perspective, Eclipse and JDeveloper are more or less the same in user-experience and functionally. Eclipse looks nice and I was pleased to have somewhat the same user-experience I have with JDeveloper. Of course this is a matter of taste, some pure-java-adepts might disagree. The thing is that JDeveloper will be Oracles main J2EE development platform. And that's fine with me.

Regarding Weblogic, I haven't been able to see in what way it's better than Oracle's OC4J. Both support the J2EE and in theory they both should support properly designed J2ee-applications. In fact most of the Oracle Soa products are interchangeable, running on both OC4J and Weblogic. So the differences will be in the way reliability, manageability and grid-functionality are implemented. So I was browsing a little through the html-interface. The labs relied on the starting and stopping of the Weblogic J2EE server from Eclipse. More or less like JDeveloper that is shipped with a standalone OC4J container. I was not too enthousiastic about the web-interface. I couldn't find the particular JSP's and Webservices of my deployed applications (I found the EAR's). In fact I ran into a locking problem between Workshop and the Weblogic web interface.
I'm very curious about the Enterprise Manager integration, the starting and stopping and maintenance of the Weblogic Server. But I think I'll wait until Oracle comes with a Universal Installer that installs SoaSuite11g with a WebLogic Server. Beause I think that's the moment it is interesting for customers that are new to Weblogic too. About the same thing more or less counted for BPEL Process Manager in the past.

Oracle Coherence
The last thing I want to remark from the Soa Partner Forum was the presentation of Dave Chappel about Oracle's Data Grid, implemented by Oracle Coherence. I heard the "expanded" presentation on the OGH meeting in De Meern on monday eavening first. To be honest, I took some time to "land" with me. I understood the principles, but I struggled with the consequeses for the implementation of Services (BPEL PM, ESB, Webservices). But it is a very interesting technology. Not at least because it is not only a Data grid but a Compute Grid as well. You can store data (like parse xml-messages) in the grid but also let the Grid perform methods on it. See also the blog of Lucas Jellema on it.

It occurred to me though that it is interesting for pretty specific implementations however. There is quite some custom-building involved. At the moment I think it is primarily interesting when having to do orchestrations with a lot of data enrichment and data-mutation based on business rules using custom services. It would be more interesting in the future when the SoaSuite can be plugged on to Coherence in a way that BPEL PM, ESB and Oracle Rules Engine is using the same data-objects in Coherence. Coherence could be used as a high-available and highly-performant (since in-memory) dehydration store.

All togehter it is a product that you'll going to here a lot more of I ex[ect. Something to watch out for, because it will grow in importance. Especially with an increased integration with the other Oracle products.