Friday, 26 August 2016

Mount NTFS disk in Linux 7

Today I wanted to pass an old disk in a usb-case to my son. It was from an old Windows Laptop and even though I'm administrator, I wasn't able to read the documents in an other user's folder.

So I thought, let's do it from an Oracle Linux 7 VM, as root. But it turns out that Oracle linux did not support NTFS by default.

But with the trick in this link I managed to do it.

To sum up, especially for my self:

Add the EPEL-7 repository from Fedora:
# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

# rpm -ivh epel-release-latest-7.noarch.rpm

This worked for me. Although I had to step over my issue that I add a Fedora repo to Oracle Linux.... An other option could have been (probably if under Redhat Linux or CentOS:
# yum install epel-release
...
# yum clean all
...
# yum update
...

Install the NTFS-3g package:
# yum install ntfs-3g -y

And enable NTFS support for FileManagers:
# yum install ntfsprogs -y

Then I got to the FileManager and I could browse the disk. It was mounted at '/run/media/oracle/803638DB3638D3BE'.

Wednesday, 24 August 2016

Generate Admin Channels to improve Weblogic Admin Console performance (and of FMW-Control)

At one of my customers we have quite an impressive domain configuration. It's a FMW domain with SOA, OSB, BAM, WSM, MFT in clusters of 4 nodes. The thing is that when having started all the servers, the console becomes slooooooowwwww. Not to speak of FMW Control (em).

One suggestion is to set the 'Invocation Timeout Seconds' under MyDomain->Configuration->General->Advanced to a value like 2. And 'Management Operation Timeout' under Preferences->Shared Preferences to a value like 5:

This surely makes the console responsive again. But it actually means that the console gives up right away when querying for the (health) state of the servers. So in stead of a health of 'OK', you get a 'server unreachable' message.

When having a lot of servers in the domain, they all share the same Admin Channel, and this seams to get over flooded. AdminServer does not get the responses in time. Sometimes a new request leads to a proper response, but in fact it takes a lot of time.

To reduce the load on the default channel, we created an admin channel per managed server. Since it's a lot of servers, and we need to do it on several environments, so I created a wlst-script for it.
The script createAdminChannels.py:
#############################################################################
# Create AdminChannels for WebLogic Servers
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.1, 2016-08-24
#
#############################################################################
# Modify these values as necessary
import sys, traceback
scriptName = sys.argv[0]
#
#
lineSeperator='__________________________________________________________________________________'
#
#
def usage():
  print 'Call script as: '
  print 'Windows: wlst.cmd '+scriptName+' -loadProperties localhost.properties'
  print 'Linux: wlst.sh '+scriptName+' -loadProperties environment.properties'
  print 'Property file should contain the following properties: '
  print "adminUrl=localhost:7001"
  print "adminUser=weblogic"
  print "adminPwd=welcome1"
#
#
def connectToadminServer(adminUrl, adminServerName):
  try:
    print(lineSeperator)
    print('Try to connect to the AdminServer')
    try:
      connect(userConfigFile=usrCfgFile, userKeyFile=usrKeyFile, url=adminUrl)
    except NameError, e:
      print('Apparently user config properties usrCfgFile and usrKeyFile not set.')
      print('Try to connect to the AdminServer adminUser and adminPwd properties')
      connect(adminUser, adminPwd, adminUrl)
  except WLSTException:
    message='Apparently AdminServer not Started!'
    print (message)
    raise Exception(message)
#
#
def createAdminChannel(serverName, adminListenPort):
  print(lineSeperator)
  channelName = serverName+'-AdminChannel'
  try:
    cd('/Servers/'+serverName+'/NetworkAccessPoints/'+channelName)
    print('Channel '+channelName +' for '+serverName+' already exists.')
  except WLSTException: 
    try:
      print('Create Admin Channel for server: '+serverName+', with port: '+adminListenPort)
      cd('/Servers/'+serverName)
      cmo.createNetworkAccessPoint(channelName)
      cd('/Servers/'+serverName+'/NetworkAccessPoints/'+channelName)
      cmo.setProtocol('admin')
      cmo.setListenPort(int(adminListenPort))
      cmo.setEnabled(true)
      cmo.setHttpEnabledForThisProtocol(true)
      cmo.setTunnelingEnabled(false)
      cmo.setOutboundEnabled(false)
      cmo.setTwoWaySSLEnabled(false)
      cmo.setClientCertificateEnforced(false)
      print ('Succesfully created channel: '+channelName)
    except WLSTException:
      apply(traceback.print_exception, sys.exc_info())
      message='Failed to create channel '+channelName+'!'
      print (message)
      raise Exception(message)
#
#
def main():
  try:
    print (lineSeperator)
    print ('Start Osb Cluster')
    print (lineSeperator)
    print('\nConnect to AdminServer ')
    connectToadminServer(adminUrl, adminServerName)
    print(lineSeperator)
    print('Start Edit Session')
    edit()
    startEdit()
    #
    #Create Admin Channels
    # Administrators
    print('\nCreate Admin Channels')
    serverNameList=serverNames.split(',')
    serverAdminPortList=serverAdminPorts.split(',')
    #
    idx=0
    for serverName in serverNameList:
      adminPort=serverAdminPortList[idx]
      createAdminChannel(serverName, adminPort)
      idx=idx+1
    # Activate changes
    print(lineSeperator)
    print('Activate Changes')
    save()
    activate(block='true')
    #
    print('\nExiting...')
    exit()
  except NameError, e:
    print('Apparently properties not set.')
    print "Please check the property: ", sys.exc_info()[0], sys.exc_info()[1]
    usage()
  except:
    apply(traceback.print_exception, sys.exc_info())
    stopEdit('y')
    exit(exitcode=1)
#call main()
main()
exit()


The shell script to call it, createAdminChannels.sh:
#!/bin/bash
#############################################################################
# Create AdminChannels
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 2.1, 2016-08-24
#
#############################################################################
#  
. fmw12c_env.sh
export PROPERTY_FILE=$1
echo
echo Create Admin Channels
wlst.sh ./createAdminChannels.py -loadProperties $PROPERTY_FILE

And the example property file, darlin-vce-db.properties:
#############################################################################
# Properties voor Creeëren SOADomain
#
# @author Martien van den Akker, Darwin-IT Professionals
# @version 1.0, 2016-04-15
#
#############################################################################
#
# Properties for AdminServer
adminServerName=Adminserver
adminUrl=darlin-vce-db:7001
# AdminUser
adminUser=weblogic
adminPwd=welcome1
#
serverNames=AdminServer,OsbServer1,SoaServer1
serverAdminPorts=7101,8111,8101
#

Call the script as $> createAdminChannels.sh darlin-vce-db.properties

In the property file you'll need to name every server in the property serverNames. And for each server the particular Admin Listen Port in serverAdminPorts, in the exact same order. Start with the AdminServer.


At the end of the script the changes are activated and then the AdminServer listens over https on the changed port.

Important: the servers need to be down, except for the the AdminServer.

Unfortunately the infrastructure database was apparently down. So I haven't been able to start SOA, BAM, etc. to see if it is performant now. But I have high hopes...

Update wlst.sh
When you update the AdminServer as above, it will by default use the DemoTrust keystore. And probably the listenaddress might not necessarily match the url that is used in the connect-URL.

So by default you might run into errors when trying to connect to the AdminServer using wlst.
First you need to adapt the Admin URL to something like 'tls://darlin-vce-db:7101'. Explicitly prefix with 'tls://' and adapt the port to the new admin-listen-port.

Then the wlst script need to be adapted. The following parameters need to be added to the wlst command:
  • -Dweblogic.security.SSL.ignoreHostnameVerification=true
  • -Dweblogic.security.TrustKeyStore=DemoTrust
To do so, find out where the wlst.sh/cmd script is located. Under linux you can perform:
[oracle@darlin-vce-db bin]$ which wlst.sh
/u01/app/oracle/FMW12210/oracle_common/common/bin/wlst.sh

Of course after setting the weblogic environment (see one of my earlier blogs describing the fmw12c_env.sh script).

Edit the wlst.sh file and go to the bottom of the file, and add the properties to the JVM_ARGS variable:
...
JVM_ARGS="${WLST_PROPERTIES} ${JVM_D64} ${UTILS_MEM_ARGS} ${CONFIG_JVM_ARGS} -Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.security.TrustKeyStore=DemoTrust"
if [ -d "${JAVA_HOME}" ]; then
 eval '"${JAVA_HOME}/bin/java"' ${JVM_ARGS} weblogic.WLST '"$@"'
else
 exit 1
fi
But setting the CONFIG_JVM_ARGS in a script like fmw12c_env.sh might be a better idea:
...
export CONFIG_JVM_ARGS='-Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.security.TrustKeyStore=DemoTrust'
...



Unable to login with a SQL Authenticator

For a project, we are migrating Forms to ADF.
There is also a number of reports which are not to be migrated yet.
Therefore, we need to keep the users in the database.
As we do not want to maintain two user stores, we thought it to be a good idea to create an authenticator in WebLogic to authenticate to the database.
There are loads of blog posts / support notes on how to configure a SQL Authenticator, so I won't repeat this procedure.
Take a look at Oracle Support Document 1342157.1 or a post from Edwin Biemond


However, we noticed a flaw in these (old) references.


In WebLogic 12c, when we create a SQL Authenticaotr, there is a filed named Identity Domain.
From the Oracle documentation, we learn:
All Authentication providers included in WebLogic Server support identity domains. If the identity domain attribute is set on an Authentication provider, that Authentication provider can authenticate only users who are defined in that identity domain.
...
An identity domain is a logical namespace for users and groups, typically representing a discrete set of users and groups in the physical datastore. Identity domains are used to identify the users associated with particular partitions.


As we do not use partitions in our domain, there is no use for an Identity domain.
But we did not know that when we setup the authenticator (and who reads the entire manual right??)


So following the previously mentioned resources, we created the SQL authenticator and entered the domain name in the Identity Domain field.
This resulted in a not working authenticator.
Symptoms:
  • When a user tried to login using database credentials, the autentication faild Always
  • No error message in the log
  • No activity in the datase (no query was executed to check the credentials)
To further analyse the issue, we added som Java options to the Admin server, using the setDomainEnv script:
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.kernel.debug=true -Dweblogic.log.StdoutSeverity=Debug -Dweblogic.log.LogSeverity=Debug -Dweblogic.StdoutDebugEnabled=true -Dweblogic.log.LoggerSeverity=Debug -Dweblogic.debug.DebugSecurityAtn=true" 
 export JAVA_OPTIONS 

This gave us more insight in the issue. The log file now revealled:
####<23-aug-2016 15:13:02 uur CEST> <Debug> <SecurityAtn> <BSCC6112> <DefaultServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <d489addb-c960-41f1-babb-f912aec329bc-00000036> <1471957982959> <[severity-value: 128] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <weblogic.security.providers.authentication.shared.DBMSAtnLoginModuleImpl.login exception: 
java.security.PrivilegedActionException: javax.security.auth.login.LoginException: javax.security.auth.callback.UnsupportedCallbackException: Unrecognized Callback: class weblogic.security.auth.callback.IdentityDomainUserCallback 
weblogic.security.auth.callback.IdentityDomainUserCallback@31789514
 at java.security.AccessController.doPrivileged(Native Method)
 at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:114)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:497)
 at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
...


Unfortunately, we still had no clue.
As you have read from the beginning of the post, you might think that setting the Identity Domain field to DOMAIN might solve the issue (the log states partition-name: DOMAIN) but no, that's no solution either.

The trick is to leave the Identity Domain filed blank. After that the authenticator worked like a charm.