Friday, 11 April 2014

Service Provider initiated SSO on WLS11g using SAML2.0

Introduction

At a recent customer I got the assignment to implement a SAML 2.0 configuration.

The customer is in this setup a Service Provider. They provide a student-administration application for the Dutch Higher Education Sector, like Colleges and Universities. The application conventionally is implemented on premise. But they like to move to a SaaS model. One institute is going to use the application from 'the cloud'. In the Dutch education sector, an organization called SurfConext serves as an authentication broker.

A good schematic explanation of the setup is in the Weblogic 11g docs:



When a user connects to the application, Weblogic finds that the user is not authenticated: it lacks a SAML2.0 token (2). So when configured correctly the browser is rerouted to SurfConext (3). On an authentication request SurfConext displays a so-called ‘Where Are You From’ (WAYF) page, on which a user can choose the institute to which he or she is connected. SurfConext then provides a means to enter the username and password (4). On submit SurfConext validates the credentials against the actual IdP, which is provided by the user’s institute (5). On a valid authentication, SurfConext provides a SAML2.0 token identifying the user with possible assertions (6). The page is refreshed and redirected to the landing page of the application (7).

For Weblogic SurfConext is in fact the Identity Profider, although in fact, based on the choice on the WAYF page, it reroutes the authentication request to the IdP of the particular institute.

Unfortunately I did not find a how-to of that particular setup in the docs. Although I found this.  But I did find the following blog: https://blogs.oracle.com/blogbypuneeth/entry/steps_to_configure_saml_2, that helped me much. Basically the setup is only the service provider part of that description.

So let me walk you through it. This is a larger blog, in fact I copy&paste larger parts from the configuration document I wrote for the customer

Configure Service provider

Pre-Requisites

To be able to test the setup against a test-IdP of SurfConext the configured Weblogic need to be reachable from internet. Appropriate firewall and proxy-server configuration need to be done upfront to enable both SurfConext to connect to the Weblogic Server as well as a remote user.

All configuration regarding url’s need to be done using the outside url’s configured above.

A PC with a direct internet connection that is enabled to connect through these same URL’s is needed to test the configuration. When connecting a pc to the intranet of the customer enables the pc to connect to internet, but the internal network configuration prevented connecting to the weblogic server using the remote url’s.

During the configuration a so called SAML Metadata file is created. This file is requested by SurfConext to get acquainted with the Service Provider. This configuration can change through reconfigurations. So SurfConext requests this through a HTTPS url. This url need to be configured, and also remotely connectable. An option is the htdocs folder of a webserver that is connectable through https. In other SAML2 setups you might need to upload the metadata-file to the identity provider's server.

You also need the SAML metadata of SurfConext. It can be downloaded from: https://wiki.surfnet.nl/display/surfconextdev/Connection+metadata.

Update Application


The application need to be updated and redeployed to use the weblogic authenticators instead of the native logon-form. To do so the web.xml need to be updated. In the web.xml (in the WEB-INF of the application war file) look for the following part:
  <login-config>
    <auth-method>FORM</auth-method>
    <realm-name>jazn.com</realm-name>
    <form-login-config>
      <form-login-page>/faces/security/pages/Login.jspx>/form-login-page>
      <form-error-page>/loginErrorServlet>/form-error-page>
    </form-login-config>
  </login-config>
And replace it with:
  <login-config>
    <auth-method>BASIC>/auth-method>
    <realm-name>myrealm>
  </login-config>
Repackage and redeploy the application to weblogic.

Add a new SAML2IdentityAserter

Here we start with the first steps to configure Weblogic: create a SAML2IdentityAsserter on the Service Provider domain.
  1. Login to ServiceProvider domain - Weblogic console
  2. Navigate to “Security Realms”:
  3.  Click on ”myrealm” 
  4. Go to the tab  ”Providers–>Authentication” :
  5. Add a new “SAML2IdentityAsserter”
  6. Name it for example: “SurfConextIdentityAsserter”:
  7. Click Ok, Save and activate changes if you're in a production domain (I'm not going to repeat that every time again in the rest of this blog). 
  8. Bounce the domain (All WLServers including AdminServer)

Configure managed server to use SAML2 Service Provider 

In this part the managed server(s) serving the application need to be configured for the so called 'federated services'. It need to know how to behave as a SAML2.0 Service Provider.
 So perform the following steps:
  1.  Navigate to the managed server, and select the “Federation Services–>SAML 2.0 Service Provider” sub tab:

  2. Edit the following settings:
  3. FieldValue
    EnabledCheck
    Preferred BindingPOST
    Default URLhttp://hostname:portname/application-URI.
    This URL should be accessible from outside the organization, that is from SurfConext.
  4. Click Save.
  5. Navigate to the managed server, and select the “Federation Services–>SAML 2.0 General” sub tab:
  6. Edit the following settings:
  7. FieldValue
    Replicated Cache EnabledUncheck or Check if needed
    Contact Person Given NameEg. Jean-Michel
    Contact Person SurnameEg. Jarre
    Contact Person TypeChoose one from the list, like 'technical'.
    Contact Person CompanyEg. Darwin-IT Professionals
    Contact Person Telephone NumberEg. 555-12345
    Contact Person Email Addressinfo@hatseflats.com
    Organization NameEg. Hatseflats B.V.
    Organization URLwww.hatseflats.com
    Published Site URLhttp://www.hatseflats.com:7777/saml2
    This URL should be accessible from outside the organization, that is from SurfConext. The Identity Provider needs to be able to connect to it.
    Entity IDEg. http://www.hatseflats.com
    SurfConext expect an URI with at least a colon (‘:’), usually the URL of the SP.
    Recipient Check EnabledUncheck.
    When checked Weblogic will check the responding Url to the URL in the original request. This could result in a ‘403 Forbidden’ message.
    Single Sign-on Signing Key Aliasdemoidentity
    If signing is used the alias of the proper private certificate in the keystore that is configured in WLS is to be provided.
    Single Sign-on Signing Key Pass PhraseDemoIdentityPassPhrase
    Confirm Single Sign-on Signing Key Pass PhraseDemoIdentityPassPhrase
  8. Save the changes and export the IDP metadata into a XML file: 
    1. Restart the server
    2. Click on 'Publish Meta Data'
    3. Restart the server
    4. Click on 'Publish Meta Data'
    5. Provide a valid path, like /home/oracle/Documents/... and click 'OK'.
    6. Copy this to a location on a http-server that is remotely connectable through HTTPS and provide the url to SurfConext.

Configure Identity Provider metadata on SAML Service Provider in Managed Server

Add new “Web Single Sign-On Identity Provider Partner” named for instance "SAML_SSO_SurfConext".
  1. In Admin Console navigate to the myrealm Security Realm and select the “Providers–>Authentication
  2. Select the SurfConextIdentityAsserter SAML2_IdentityAsserter and navigate to the “Management” tab:
  3. Add a new “Web Single Sign-On Identity Provider Partner
    1. Name it: SAML_SSO_SurfConext
    2. Select “SurfConext-metadata.xml”
    3. Click 'OK'.
  4. Edit the created SSO Identity Provider Partner “SAML_SSO_SurfConext” and Provide the following settings:
  5. FieldValue
    NameSAML_SSO_SurfConext
    EnabledCheck
    DescriptionSAML Single Sign On partner SurfConext
    Redirect URIs/YourApplication-URI
    These are URI’s relative to the root of the server.

Add SAMLAuthenticationProvider

In this section an Authentication provider is added.
  1. Navigate to the ‘Providers->Authentication’ sub tab of the ‘myrealm’ Security Realm:
  2. Add a new Authentication Provider. Name it: ‘SurfConextAuthenticator’ and select as type: 'SAMLAuthenticator'.
    Click on the new Authenticator and set the Control Flag to ‘SUFFICIENT’:
  3. Return to the authentication providers and click on 'Reorder'.
    Use the selection boxes and the arrow buttons to reorder the providers as follows:
    The SurfConext authenticator and Identity Asserter should be first in the sequence.

Set all other authentication providers to sufficient

The control flag of the Default Authenticator is by default set to ‘REQUIRED’. That means that for an authentication request this one needs to be executed. However, for the application we want the SAMLAuthentication be Sufficient, thus that the other authenticators need not to be executed. So set these other ones (if others including the DefaultAuthenticator exist) to ‘SUFFICIENT’ as well.

Enable debug on SAML

To enable debug messages on SAML, navigate to the 'Debug' tab of the Managed Server:
Expand the nodes ‘weblogic -> security’. Check the node ‘Saml2’ and click 'Enable'. This will add SAML2 related logging during authentication processes to the server.log. To disable the logging, check the node or higher level nodes and click 'Disable'.

Deploy the Identity Name Mapper

SurfConnext generates a userid for each connected user. SurfConext provides two options for this: a persistent userid throughout all sessions or a userid per session. Either way, the userid is generated as a GUID that is not registered within the customers application and also on itself not to relate to known users in the application. In the SAML token however, also the username is provided. To map this to the actual userid that Weblogic provides to the application, an IdentityMapper class is needed. The class implements a certain interface of weblogic, and uses a custom principal class that implements a weblogic interface as well. The implementation is pretty straightforward. I found an example that uses an extra bean for a Custom Principal. The IdentityMapper class is as follows:
package nl.darwin-it.saml-example;

import com.bea.security.saml2.providers.SAML2AttributeInfo;
import com.bea.security.saml2.providers.SAML2AttributeStatementInfo;
import com.bea.security.saml2.providers.SAML2IdentityAsserterAttributeMapper;
import com.bea.security.saml2.providers.SAML2IdentityAsserterNameMapper;
import com.bea.security.saml2.providers.SAML2NameMapperInfo;

import java.security.Principal;

import java.util.ArrayList;
import java.util.Collection;
import java.util.logging.Logger;

import weblogic.logging.LoggingHelper;

import weblogic.security.service.ContextHandler;


public class SurfConextSaml2IdentityMapper implements SAML2IdentityAsserterNameMapper,
                                                      SAML2IdentityAsserterAttributeMapper {
  public static final String ATTR_PRINCIPALS = "com.bea.contextelement.saml.AttributePrincipals";
  public static final String ATTR_USERNAME = "urn:mace:dir:attribute-def:uid";

  private Logger lgr = LoggingHelper.getServerLogger();
  private final String className = "SurfConextSaml2IdentityMapper";


  @Override
  public String mapNameInfo(SAML2NameMapperInfo saml2NameMapperInfo,
                            ContextHandler contextHandler) {
    final String methodName = className + ".mapNameInfo";
    debugStart(methodName);
    String user = null;

    debug(methodName,
          "saml2NameMapperInfo: " + saml2NameMapperInfo.toString());
    debug(methodName, "contextHandler: " + contextHandler.toString());
    debug(methodName,
          "contextHandler number of elements: " + contextHandler.size());

    // getNames gets a list of ContextElement names that can be requested.
    String[] names = contextHandler.getNames();

    // For each possible element
    for (String element : names) {
      debug(methodName, "ContextHandler element: " + element);
      // If one of those possible elements has the AttributePrinciples
      if (element.equals(ATTR_PRINCIPALS)) {
        // Put the AttributesPrincipals into an ArrayList of CustomPrincipals
        ArrayList<CustomPrincipal> customPrincipals =
          (ArrayList<CustomPrincipal>)contextHandler.getValue(ATTR_PRINCIPALS);
        int i = 0;
        String attr;
        if (customPrincipals != null) {
          // For each AttributePrincipal in the ArrayList
          for (CustomPrincipal customPrincipal : customPrincipals) {
            // Get the Attribute Name and the Attribute Value
            attr = customPrincipal.toString();
            debug(methodName, "Attribute " + i + " Name: " + attr);
            debug(methodName,
                  "Attribute " + i + " Value: " + customPrincipal.getCollectionAsString());
            // If the Attribute is "loginAccount"
            if (attr.equals(ATTR_USERNAME)) {
              user = customPrincipal.getCollectionAsString();
              // Remove the "@DNS.DOMAIN.COM" (case insensitive) and set the username to that string
              if (!user.equals("null")) {
                user = user.replaceAll("(?i)\\@CLIENT\\.COMPANY\\.COM", "");
                debug(methodName, "Username (from loginAccount): " + user);
                break;
              }
            }
            i++;
          }
        }

        // For some reason the ArrayList of CustomPrincipals was blank - just set the username to the Subject
        if (user == null || "".equals(user)) {
          user = saml2NameMapperInfo.getName(); // Subject = BRID

          debug(methodName, "Username (from Subject): " + user);
        }

        return user;
      }
    }

    // Just in case AttributePrincipals does not exist
    user = saml2NameMapperInfo.getName(); // Subject = BRID
    debug(methodName, "Username (from Subject): " + user);

    debugEnd(methodName);

    // Set the username to the Subject
    return user;

    // debug(methodName,"com.bea.contextelement.saml.AttributePrincipals: " + arg1.getValue(ATTR_PRINCIPALS));
    // debug(methodName,"com.bea.contextelement.saml.AttributePrincipals CLASS: " + arg1.getValue(ATTR_PRINCIPALS).getClass().getName());


    // debug(methodName,"ArrayList toString: " + arr2.toString());
    // debug(methodName,"Initial size of arr2: " + arr2.size());


  }
  /*  public Collection<Object> mapAttributeInfo0(Collection<SAML2AttributeStatementInfo> attrStmtInfos, ContextHandler contextHandler) {
      final String methodName = className+".mapAttributeInfo0";
        if (attrStmtInfos == null || attrStmtInfos.size() == 0) {
            debug(methodName,"CustomIAAttributeMapperImpl: attrStmtInfos has no elements");
            return null;
        }

        Collection<Object> customAttrs = new ArrayList<Object>();

        for (SAML2AttributeStatementInfo stmtInfo : attrStmtInfos) {
            Collection<SAML2AttributeInfo> attrs = stmtInfo.getAttributeInfo();
            if (attrs == null || attrs.size() == 0) {
                debug(methodName,"CustomIAAttributeMapperImpl: no attribute in statement: " + stmtInfo.toString());
            } else {
            for (SAML2AttributeInfo attr : attrs) {
                if (attr.getAttributeName().equals("AttributeWithSingleValue")){
                        CustomPrincipal customAttr1 = new CustomPrincipal(attr.getAttributeName(), attr.getAttributeNameFormat(),attr.getAttributeValues());
                        customAttrs.add(customAttr1);
                }else{
                    String customAttr = new StringBuffer().append(attr.getAttributeName()).append(",").append(attr.getAttributeValues()).toString();
                    customAttrs.add(customAttr);
                }
            }
            }
        }
        return customAttrs;
    }       */

  public Collection<Principal> mapAttributeInfo(Collection<SAML2AttributeStatementInfo> attrStmtInfos,
                                                ContextHandler contextHandler) {
    final String methodName = className + ".mapAttributeInfo";
    Collection<Principal> principals = null;
    if (attrStmtInfos == null || attrStmtInfos.size() == 0) {
      debug(methodName, "AttrStmtInfos has no elements");
    } else {
      principals = new ArrayList<Principal>();
      for (SAML2AttributeStatementInfo stmtInfo : attrStmtInfos) {
        Collection<SAML2AttributeInfo> attrs = stmtInfo.getAttributeInfo();
        if (attrs == null || attrs.size() == 0) {
          debug(methodName,
                "No attribute in statement: " + stmtInfo.toString());
        } else {
          for (SAML2AttributeInfo attr : attrs) {
            CustomPrincipal principal =
              new CustomPrincipal(attr.getAttributeName(),
                                  attr.getAttributeValues());
            /*  new CustomPrincipal(attr.getAttributeName(),
                   attr.getAttributeNameFormat(),
                attr.getAttributeValues()); */
            debug(methodName, "Add principal: " + principal.toString());
            principals.add(principal);
          }
        }
      }
    }
    return principals;
  }

  private void debug(String methodName, String msg) {
    lgr.fine(methodName + ": " + msg);
  }

  private void debugStart(String methodName) {
    debug(methodName, "Start");
  }

  private void debugEnd(String methodName) {
    debug(methodName, "End");
  }

}
The commented method ‘public Collection<Object> mapAttributeInfo0’ is left in the source as an example method. The CustomPrincipal bean:
package nl.darwin-it.saml-example;

import java.util.Collection;
import java.util.Iterator;

import weblogic.security.principal.WLSAbstractPrincipal;
import weblogic.security.spi.WLSUser;


public class CustomPrincipal extends WLSAbstractPrincipal implements WLSUser{
   
  private String commonName;  
      private Collection collection;  
    public CustomPrincipal(String name,  Collection collection) {  
        super();  
        // Feed the WLSAbstractPrincipal.name. Mandatory  
        this.setName(name);  
        this.setCommonName(name);  
        this.setCollection(collection);  
    }  
      
    public CustomPrincipal() {  
        super();  
    }  
      
    public CustomPrincipal(String commonName) {  
        super();  
        this.setName(commonName);  
        this.setCommonName(commonName);  
    }  
       
    public void setCommonName(String commonName) {  
        // Feed the WLSAbstractPrincipal.name. Mandatory  
        super.setName(commonName);  
        this.commonName = commonName;  
        System.out.println("Attribute: " + this.getName());  
        // System.out.println("Custom Principle commonName is " + this.commonName);  
    }  
            
    public Collection getCollection() {  
        return collection;  
    }  
      
    public String getCollectionAsString() {  
        String collasstr;  
        if(collection != null && collection.size()>0){  
            for (Iterator iterator = collection.iterator(); iterator.hasNext();) {  
                collasstr = (String) iterator.next();  
                return collasstr;  
            }  
        }  
        return "null";  
    }  
      
    public void setCollection(Collection collection) {  
        this.collection = collection;  
        // System.out.println("set collection in CustomPrinciple!");  
        if(collection != null && collection.size()>0){  
            for (Iterator iterator = collection.iterator(); iterator.hasNext();) {  
                final String value = (String) iterator.next();  
                System.out.println("Attribute Value: " + value);  
            }  
        }  
    }  
      
    @Override  
    public int hashCode() {  
        final int prime = 31;  
        int result = super.hashCode();  
        result = prime * result + ((collection == null) ? 0 : collection.hashCode());  
        result = prime * result + ((commonName == null) ? 0 : commonName.hashCode());  
        return result;  
    }  
      
    @Override  
    public boolean equals(Object obj) {  
        if (this == obj)  
            return true;  
        if (!super.equals(obj))  
            return false;  
        if (getClass() != obj.getClass())  
            return false;  
        CustomPrincipal other = (CustomPrincipal) obj;  
        if (collection == null) {  
            if (other.collection != null)  
                return false;  
        } else if (!collection.equals(other.collection))  
                return false;  
        if (commonName == null) {  
            if (other.commonName != null)  
                return false;  
        } else if (!commonName.equals(other.commonName))  
                return false;  
        return true;  
    }  
       
} 
Package the classes as a java archive (jar) and place it in a folder on the weblogic server. For instance $DOMAIN_HOME/lib. Although the $DOMAIN_HOME/lib is in the classpath for many usages, for this usage the jar file is not picked-up by the class-loaders. Probably due to the class-loader hierarchy. To have the jar-file (SurfConextSamlIdentityMapper.jar) in the system class path, add the complete path to the jar file to the classpath on the Startup-tab on both the AdminServer as well as the Managed server. In this the AdminServer is needed, since class is configured through the Realm, and during the configuration the existence of the class is checked. Apparently it is required to also add the weblogic.jar before the SurfConextSamlIdentityMapper.jar to the startup-classpath. Then restart the AdminServer as well as the managed servers.

Configure the Identity Name Mapper

Now the Identity Name mapper class can be configured:
  1. In Admin Console navigate to the myrealm Security Realm and select the “Providers–>Authentication
  2. Select the SurfConextIdentityAsserter SAML2_IdentityAsserter and navigate to the “Management” tab:
  3. Edit the created SSO Identity Provider Partner “SAML_SSO_SurfConext”.

    Provide the following settings:
    FieldValue
    Identity Provider Name Mapper Class Namenl.darwin-it.saml-example.SurfConextSaml2IdentityMapper

Test the application

At this point the application can be tested. Browse using the external connected PC to the application using the remote URL. For instance: https://www.hatseflats.com:7777/YourApplication-URI. If all is well, the browser is redirected to SurfConext’s WhereAreYouFrom page. Choose the following provider:

Connect as ‘student1’ with password ‘student1’ (or one of the other test creditials like student2, student3, see https://wiki.surfnet.nl/display/surfconextdev/Test+and+Guest+Identity+Providers). After a succesfull logon, the browser should be redirected to the application. The choosen credential should of course be known as a userid in the application.

Conclusion

This is one of the bigger stories on this blog. I actually edited the configuration document as a blog entry. I hope you'll find it usefull. With this blog you have a complete how-to for the ServiceProvider part for an ServiceProvider Initiated SSO setup.

SAML2 seemed complicated to me at first. And under the covers it still might be. But it turns out that Weblogic11g has a great implementation for it, that is neatly configurable. It's a little pity that you need a mapper class for the identity-mapping. It would be nice if you could configure the attribute-value to be returned as a userid. But the mapper class is not that complicated.

Thursday, 10 April 2014

JDeveloper XSL Mapper tip

Of course you know already that in Jdeveloper you can create xsl maps, just by drawing lines between source and target elements. In many cases you need functions or complex expressions in between. Those are "drag-and-dropable" as well. I found that you can even drop a function on a line and the function will be added to the expression. So with a little thought of the sequence of "drag-and-drops" of functions you can assemble pretty complex expressions together just by using the mouse. Although I'm not affraid to hack back in the source code of the xsl for quickness, I found that this allowed me to spare a few switches between the Design and the Source tab. That is convenient, since hacking the source and switching back to the Design tab will cause the Designer to initialize again, driving you to expand again all the nodes you were working on. Understandable, but inconvenient with large XSD's.

What I did not know until recently is how to set a variable to an element. So what I did before was to hack in the source a piece of code like:
<ns0:countryCode>
  <xsl:value-of select="$landCodeNL" />
</ns0:countryCode>


It turns out that you can do that by "drag-and-drop" as well. In the component-palette you need to select the "Advanced" functions:
 At the bottom you find  a xpath-expression element. Drag-and-drop that in the design area and connect it to the target element.





When you edit it you can just type in your expression, for instance just a variable. When you start with a dollar sign, it even gives you a drop-down-list with available variables. Just pick the right one and our done.

I admit, no high-standard tip, but convenient enough though, for me at least.

Wednesday, 9 April 2014

SQLServer: date conversions

In my current project I need to query an MS SqlServer database.
Unfortunately the dates are stored as a BigInt instead of a proper date datatype.
So I had to find out how to do compare the dates with the systemdate, and how to get the system date. To log this for possible later use, as an exception, a blog about SqlServer.

To get the system date, you can do:
(SELECT dt=GETDATE()) a 
It's maybe my Oracle background, but I would write this like:
(SELECT GETDATE() dt) a 
An alternative is:
select CURRENT_TIMESTAMP
I found this at this blog. Contrary to the writer of that blog I would prefer this version, since I found that it works on Oracle too. There are several ways to convert this to a bigint, but the most compact I found is:
  ( SELECT  YEAR(DT)*10000+MONTH(dt)*100+DAY(dt) sysdateInt
FROM
  -- Test Data
  (SELECT  GETDATE() dt) a ) utl
The way I wrote this, makes it usefull as a subquery or a joined query:
SELECT
  Ent.* ,
  CASE
    WHEN Ent.endDate  IS NOT NULL
    AND Ent.endDate-1 < sysdateInt
    THEN Ent.endDate-1
    ELSE sysdateInt
  END refEndDateEntity ,
  utl.sysdateInt
FROM
  SomeEntity Ent,
  ( SELECT  YEAR(DT)*10000+MONTH(dt)*100+DAY(dt) sysdateInt
FROM
  -- Test Data
  (SELECT  GETDATE() dt) a ) utl;
To convert a bigint to a date, you can do the following:
CONVERT(DATETIME, CONVERT(CHAR(8), ent.endDate))
However, I found that although this works in a select clause, in the where-clause this would run into a "Data Truncation" error. Maybe it is due to the use of SqlDeveloper and thus a JDBC connection to SqlServer, but I'm not so enthousiastic about the error-responses of SqlServer... I assume the error has to do with the fact that it has to do with the fact that SqlServer has to interpret a column-value of a row when it did not already selected it, that is when evaluating wheter to add the row (or not) to the result set. So to make it work I added the construction as a determination value in the select clause of a 1:1 view on the table, and use that view in stead of the table. Then the selected value can be used in the where clause.

Thursday, 3 April 2014

SoapUI: Property Based Assertions

In many cases you need to or are allowed to send a messageId and a correlationId to a service. In the response these Id's are mirrored back. In SoapUI it is pretty easy to generate those ID's using Groovy. The code for this is as follows:
def guidVal = "${java.util.UUID.randomUUID()}";
def testCase = testRunner.testCase;
testCase.setPropertyValue("MessageId", guidVal);
def msgDate = new Date() ;
def msgDateStr =  msgDate.format("yyyy-MM-dd'T'HH:mm:ss");
testCase.setPropertyValue("MessageTimeStamp",msgDateStr);
This as you can conclude from the code this is from a Groovy test step in a test Case. In the succeeding soap request you can have something like the following to have the property values embedded in the message:
        <ber:berichtHeader>
            <ber:messageData>
               <ber:messageId>${#TestCase#MessageId}
               <ber:messageTimestamp>${#TestCase#MessageTimeStamp}
            </ber:messageData>
            <ber:correlatieId>${#TestCase#MessageId}
        </ber:berichtHeader>
Here you see that I use the same id for both messageId as well as the correlationId. A correlationId might have a longer lifespan as a messageId. In my (simple) case we have just one-to-one conversations. In the response of the message you might find something like the following:
         <abct:berichtHeader>
            <abct:messageData>
               <abct:messageId>23c20898-9164-449d-87ef-3d9ed96ba946
               <abct:messageTimestamp>2014-04-03T13:40:25.147+02:00
               <abct:refToMessageId>23bf6b7a-61f9-4c91-b932-b288c5e358be
            </abct:messageData>
            <abct:correlatieId>23bf6b7a-61f9-4c91-b932-b288c5e358be
         </abct:berichtHeader>
Nice. Apparently SoapUI generated global unique message id's and apparently my service mirrored them back. But how do I test that automatically? The thing is in this that I don't know the expected value at designtime, since the messageId's are generated at runtime. But the nice thing in SoapUI is that almost everywhere you can define properties at several level's (General, Project, TestSuite, TestCase, etc.) and reference them almost every where. For instance, you can use properties to buildup a soap-endpoint, using properties at project level. So you could define an endpoint as follows:
http://${#Project#ServiceEndpoint}/${#Project#ServiceURI}
This can be used in assertions as well. So define an Xpath-Match assertion on your response and instead of a fixed expected value, give in
${#TestCase#messageId}
Like:
As you can see you reference the TestCase property by prefixing it with #TestCase. This refers to the current, running TestCase. For the sharp readers: my property reference in the assert is lower-init-case, where in the groovy script and the message it has an initial capital. I found though that the property is apparently not case-sensitive.

Friday, 28 March 2014

JDeveloper BPMN Bug: Activity Name conflict - Follow Up

A few weeks ago I reported about my experiences with JDeveloper 11g PS6 in Malta. I had to borrow a laptop to do the Adaptive Case Management workshop. It was a HP laptop with Ubuntu 12. Somehow this combination with VirtualBox 4.3.6 lead into a bug in JDeveloper. Creating a new project would introduce a dash ('-') in the ID of each created activity.

In the mean time I have a new laptop, a very nice Asus N56. This morning I was in the opportunity to startup the VM that I exported and backed up from the borrowed laptop. And what a surprise: creating a new project just works! No naming conflicts. Also, of course, creating a new activity is ok.

A strange case indeed.

Thursday, 27 March 2014

Hierarchical XML from SQL

Years ago I wrote an article (in Dutch) on the XML functions in Oracle SQL. It can be found here.
It describes how to create an xml document as an XMLType with an Oracle SQL Query.

The query that is described is based on a pretty simple table, with no relationships. I'm creating a new course based on a datamodel we created years ago, that contains data. I wanted to abstract some of that data as xml, but then: how about the foreign key relations?

It turns out pretty straightforward, that you can probably figure out yourself. But hey, I'm not a bad guy, so how about sharing it to you?

The query selects employees with their addresses and goes as follows:

select xmlelement("emp:employees"
  , xmlattributes( 'http://xmlnls.darwin-it.nl/doe/xsd/v1/employees' as "xmlns"
                 , 'http://xmlnls.darwin-it.nl/doe/xsd/v1/employees' as "xmlns:emp"
                 , 'http://xmlnls.darwin-it.nl/doe/xsd/v1/employees' as "xmlns:ads")
  , xmlagg
  ( xmlelement
    ( "emp:employee"
    , xmlforest
      ( emp.title as "emp:title"
      , emp.firstname as "emp:firstName"
      , emp.last_name as "emp:lastName"
      , emp.gender as "emp:gender"
      , emp.birth_date as "emp:birthDate"
      )
    , ( select xmlelement( "emp:addresses"
        , xmlagg(
             xmlelement("ads:address"
             , xmlattributes( ate.code as "ads:type"
                            , ate.description as "ads:description")
             , xmlforest
               ( ads.adress_line1 as "ads:addressLine1"
               , ads.adress_line2 as "ads:addressLine2"
               , ads.adress_line3 as "ads:addressLine3"
               , ads.postal_code as "ads:postalCode"
               , ads.city as "ads:city"
               , ads.country as "ads:country"
               )
             )
          )
        )
        from doe_party_addresses pae
        join doe_addresses ads on ads.id = pae.ads_id
        join doe_address_types ate on ate.id= pae.ate_id
        where pae.emp_id = emp.id
        ) 
      )
    )
  )  xml
from doe_employees emp;
The doe_addresses table is joined with the doe_employees table via a couple-table named doe_party_addresses. This is because an employee can have multiple addresses, but with different types. An employee can have a Business address and a Home address. Like a customer can have a shipping, billing and visiting addresses. Also an address can be used by multiple parties. You see here that the addresses are selected as a subselect. The output of the sub-select is an XMLType that can be embedded in the xmlelement (and other xml-) functions. What you also see is that I added namespace declarations as xmlattributes on the top-level element. The element and attribute names are prefixed with the corresponding 'emp:' or 'ads:' namespace-prefixes. Oh, and the output of the query is something like:
<?xml version="1.0" encoding="UTF-8" ?>
<emp:employees xmlns="http://xmlnls.darwin-it.nl/doe/xsd/v1/employees"
               xmlns:emp="http://xmlnls.darwin-it.nl/doe/xsd/v1/employees"
               xmlns:ads="http://xmlnls.darwin-it.nl/doe/xsd/v1/employees">
  <emp:employee>
    <emp:title>Mr.</emp:title>
    <emp:firstName>Ed</emp:firstName>
    <emp:lastName>Bushes</emp:lastName>
    <emp:gender>M</emp:gender>
    <emp:birthDate>2068-01-20</emp:birthDate>
    <emp:addresses>
      <ads:address ads:type="HOME" ads:description="Home address">
        <ads:addressLine1>Maasstraat 19</ads:addressLine1>
        <ads:postalCode>3812HS</ads:postalCode>
        <ads:city>Amersfoort</ads:city>
        <ads:country>NETHERLANDS</ads:country>
      </ads:address>
    </emp:addresses>
  </emp:employee>
  <emp:employee>
    <emp:title>Mr.</emp:title>
    <emp:firstName>M.</emp:firstName>
    <emp:lastName>Outback</emp:lastName>
    <emp:gender>M</emp:gender>
    <emp:birthDate>2061-10-14</emp:birthDate>
    <emp:addresses>
      <ads:address ads:type="HOME" ads:description="Home address">
        <ads:addressLine1>Rocky Road 2</ads:addressLine1>
        <ads:postalCode>20001</ads:postalCode>
        <ads:city>StoneHench</ads:city>
        <ads:country>UNITED KINGDOM</ads:country>
      </ads:address>
    </emp:addresses>
  </emp:employee>
...
  <emp:employee>
    <emp:title>Ms.</emp:title>
    <emp:firstName>Debby</emp:firstName>
    <emp:lastName>Waters</emp:lastName>
    <emp:gender>F</emp:gender>
    <emp:birthDate>2082-01-20</emp:birthDate>
    <emp:addresses>
      <ads:address ads:type="WORK" ads:description="Work address">
        <ads:addressLine1>Darwinplein 11</ads:addressLine1>
        <ads:postalCode>4321PS</ads:postalCode>
        <ads:city>Amsterdam</ads:city>
        <ads:country>NETHERLANDS</ads:country>
      </ads:address>
      <ads:address ads:type="HOME" ads:description="Home address">
        <ads:addressLine1>Newtonweg 41</ads:addressLine1>
        <ads:postalCode>6543AB</ads:postalCode>
        <ads:city>Maasland</ads:city>
        <ads:country>NETHERLANDS</ads:country>
      </ads:address>
    </emp:addresses>
  </emp:employee>
  <emp:employee>
    <emp:title>Mr.</emp:title>
    <emp:firstName>Wally</emp:firstName>
    <emp:lastName>Waters</emp:lastName>
    <emp:gender>M</emp:gender>
    <emp:birthDate>2063-06-13</emp:birthDate>
    <emp:addresses></emp:addresses>
  </emp:employee>
</emp:employees>

Tuesday, 25 March 2014

OSB: Remove artefacts with wlst

In my current project I needed to restructure my projects. Where I had all projects in one configuration, I wanted multiple configurations to do partial deployments. So I created configurations per highlevel business service, and merged canonical and application specific services into one osb project. But at deployment time, for OSB moved services are actually new services that conflict with the existing services at the current location, because now there are two versions of the same service with the same endpoint-uri exist. That is, it gets OSB in a conflicting state, that it can't activate. So I created a series of scripts to remove artefacts. My starting point was this Oracle forum entry.
  
import wlstModule

from java.util import Collections
from com.bea.wli.sb.util import Refs
from com.bea.wli.config import Ref
from com.bea.wli.sb.management.configuration import SessionManagementMBean
from com.bea.wli.sb.management.configuration import ALSBConfigurationMBean
#from com.bea.wli.config.component import AlreadyExistsException

import sys


#=======================================================================================
# Entry function to delete a  project from the alsbConfiguration 
#=======================================================================================
def deleteProject(alsbConfigurationMBean, projectName):
     try:
          print "Trying to remove " + projectName
          projectRef = Ref(Ref.PROJECT_REF, Ref.DOMAIN, projectName)                  
          if alsbConfigurationMBean.exists(projectRef):
               print "#### removing OSB project: " + projectName
               alsbConfigurationMBean.delete(Collections.singleton(projectRef))
               print "#### removed project: " + projectName
          else:
               failed = "OSB project <" + projectName + "> does not exist"
               print failed
          print
     except:
          print "Error whilst removing project:", sys.exc_info()[0]
          raise
  
#=======================================================================================
# Entry function to undeploy a project from an OSB Configuration
#=======================================================================================
def undeployProjectFromOSBDomain(projectName):
     try:
          domainRuntime()      
          sessionName  = "UndeployProjectStateSession_" + str(System.currentTimeMillis())         
          sessionManagementMBean = findService(SessionManagementMBean.NAME,SessionManagementMBean.TYPE)
          print "SessionMBean started session"
          sessionManagementMBean.createSession(sessionName)
          print 'Created session <', sessionName, '>'
          alsbConfigurationMBean = findService(ALSBConfigurationMBean.NAME + "." + sessionName, ALSBConfigurationMBean.TYPE)
          deleteProject(alsbConfigurationMBean, projectName)   
          sessionManagementMBean.activateSession(sessionName, "Complete project removal with customization using wlst")
     except:
          print "Error whilst removing project:", sys.exc_info()[0]
          discardSession(sessionManagementMBean, sessionName)
          raise

#=======================================================================================
# Entry function to undeploy multiple projects from an OSB Configuration
#=======================================================================================
def undeployProjectsFromOSBDomain(projectNames):
     try:
          domainRuntime()      
          sessionName  = "UndeployProjectStateSession_" + str(System.currentTimeMillis())         
          sessionManagementMBean = findService(SessionManagementMBean.NAME,SessionManagementMBean.TYPE)
          print "SessionMBean started session"
          sessionManagementMBean.createSession(sessionName)
          print 'Created session <', sessionName, '>'
          alsbConfigurationMBean = findService(ALSBConfigurationMBean.NAME + "." + sessionName, ALSBConfigurationMBean.TYPE)
          for projectName in projectNames:
            print '\nDelete project '+projectName
            deleteProject(alsbConfigurationMBean, projectName)   
          sessionManagementMBean.activateSession(sessionName, "Complete project removal with customization using wlst")
     except:
          print "Error whilst removing project:", sys.exc_info()[0]
          discardSession(sessionManagementMBean, sessionName)
          raise

def discardSession(sessionManagementMBean, sessionName):
     if sessionManagementMBean != None:
          if sessionManagementMBean.sessionExists(sessionName):
               sessionManagementMBean.discardSession(sessionName)
               print "Session discarded" 
I did a little restructure so that I can remove multiple projects in one session. This is needed since some projects are inter-related. So must be removed together to get a consistent state. To remove multiple projects I have the following main function:
def main():
  projectNames=["Project1", "Project2", "Project3", "Project4", "Project5", "Project6", "Project7", "Etc." ]
  undeployProjectsFromOSBDomain(projectNames)
# Call the main function
main()
I call this script from an ANT based patch-framework. To succesfully call this wlst script you need the following jar files in your classpath:
  • ${osb.home}/modules/com.bea.common.configfwk_1.7.0.0.jar 
  • ${osb.home}/lib/sb-kernel-api.jar 
  • ${osb.home}/sb-kernel-impl.jar 
  • ${osb.home}/osb-coherence-client.jar (I don't understand why, but this is needed for the Ref-class that is also available in the com.bea.common.configfwk_1.7.0.0.jar) 
But you should also be ok with:
  • ${osb.home}/modules/com.bea.common.configfwk_1.7.0.0.jar 
  • ${osb.home}/lib/alsb.jar (this jar contains references to the other alsb-jar files in the lib folder in it's manifest file)
Here '${osb.home}' relates to the 'Oracle_OSB1' folder in your Fusion Middleware Home.
Based on the scripts above I also created a function to remove a single Proxy service:
#=======================================================================================
# Entry function to find a proxy service
#=======================================================================================
def findProxyService(folder, serviceName, sessionName):
  print "Find proxy service: " + folder + "/" + serviceName
  pxyConf = "ProxyServiceConfiguration." + sessionName
  mbean = findService(pxyConf, 'com.bea.wli.sb.management.configuration.ProxyServiceConfigurationMBean')
  folderRef = Refs.makeParentRef(folder + '/')
  serviceRef = Refs.makeProxyRef(folderRef, serviceName)
  return serviceRef, mbean
#=======================================================================================
# Entry function to remove a proxyservice from OSB configuration 
#=======================================================================================
def undeployProxyFromOSBDomain(relativePath, proxyServiceName):
     try:
          domainRuntime()
          
          sessionName  = "UndeployProxySession_" + str(System.currentTimeMillis())
          print "Trying to remove " + proxyServiceName
          
          sessionManagementMBean = findService(SessionManagementMBean.NAME,SessionManagementMBean.TYPE)
          print "SessionMBean started session"
          sessionManagementMBean.createSession(sessionName)
          print 'Created session <', sessionName, '>'
          serviceRef, sessionBean = findProxyService(relativePath, proxyServiceName, sessionName)
          alsbConfigurationMBean = findService(ALSBConfigurationMBean.NAME + "." + sessionName, ALSBConfigurationMBean.TYPE)
          if alsbConfigurationMBean.exists(serviceRef):
            print "#### removing OSB proxy service: " + proxyServiceName
            alsbConfigurationMBean.delete(Collections.singleton(serviceRef))
            sessionManagementMBean.activateSession(sessionName, "Complete service removal with customization using wlst")
          else:
            failed = "OSB project <" + proxyServiceName + "> does not exist"
            print failed
            discardSession(sessionManagementMBean, sessionName)
            #raise Failure(failed)
          print
     except:
          print "Error whilst removing project:", sys.exc_info()[0]
          discardSession(sessionManagementMBean, sessionName)
          raise
It should not be too hard to create one for business services out of this one. Removing folders is slightly easier:
#=======================================================================================
# Entry function to remove a folder from OSB configuration 
#=======================================================================================
def removeFolderFromOSBDomain(folder):
     try:
          domainRuntime()
         
          sessionName  = "RemoveFolderSession_" + str(System.currentTimeMillis())
          print "Trying to remove " + folder
          
          sessionManagementMBean = findService(SessionManagementMBean.NAME,SessionManagementMBean.TYPE)
          print "SessionMBean started session"
          sessionManagementMBean.createSession(sessionName)
          print 'Created session <', sessionName, '>'
          folderRef = Refs.makeParentRef(folder)
          alsbConfigurationMBean = findService(ALSBConfigurationMBean.NAME + "." + sessionName, ALSBConfigurationMBean.TYPE)
          if alsbConfigurationMBean.exists(folderRef):
            print "#### removing OSB folder: " + folder
            alsbConfigurationMBean.delete(Collections.singleton(folderRef))
            sessionManagementMBean.activateSession(sessionName, "Complete service removal with customization using wlst")
          else:
            failed = "OSB folder <" + folder + "> does not exist"
            print failed
            discardSession(sessionManagementMBean, sessionName)
            #raise Failure(failed)
          print
     except:
          print "Error whilst removing project:", sys.exc_info()[0]
          discardSession(sessionManagementMBean, sessionName)
          #raise
Don't forget the import statements from the first script for these latter ones.