Use SoapUI to test your SFTP based services

SoapUI is my favorite tool to do unit tests. I'd try to keep my self to test based development and build up tests along with the development service. For SOAP or REST based services this goes quite intuitively using SoapUI. For database driven it is a little harder, but SoapUI has a nice JDBC activity, that supports DML as well as callable statements as stored procedures.

But for files and especially SFTP its a little more complicated. For a while I'm working on a filebased integration with SAP as source system.

I configured and defined the SOASuite FTP adapter to use my SSH user (oracle) on my VirtualBox VM. Until now I tested it using the SFTP/SCP client from MobaXTerm (enthousiastically recommended: download here). But not so handy for unit tests.

I wanted to automate this using SoapUI. With some searching I found that JCraft Java Secure Channel library was the best and easiest option. I did take a look at Apache Commons Net. But couldn't get it to work so easily. Download the jsch-0.1.54.jar (or newer) file and copy it to the ./bin/ext folder in your SoapUI home:

And restart SoapUI.

Create a new empty SoapUI project, create a TestSuite called something like 'Utilities' and a TestCase called 'TC-FTP'. Add the following properties to the TestCase:

Property	Value
ftpHost	darlin-vce-db
ftpPort	22
ftpUsername	oracle
ftpPassword	welcome1
localFilePath	d:/Projects/2016MySapProject/ExampleFiles/SAP-MESSAGE.XML
remoteFileDir	/home/oracle/SapHR/in

In the TestCase add a Groovy Script called FTP add the script below. I took the example from snip2code.com (also found elsewhere) and refactered it to:

//Send Files to SSH Location
//
//Download jsch-0.1.54.jar from http://www.jcraft.com/jsch/ and copy it to  SoapUI-Home/bin/ext location
//Example from: https://www.snip2code.com/Snippet/413499/SoapUI-Groovy-Script-compatible-SFTP-fil

//import org.apache.commons.net.ftp.FTPSClient
import com.jcraft.jsch.*
//
// Properties
// 
def testCase = testRunner.testCase;


def String ftpHost = testCase.getPropertyValue("ftpHost")
def int ftpPort = testCase.getPropertyValue("ftpPort").toInteger()
def String ftpUsername = testCase.getPropertyValue("ftpUsername")
def String ftpPassword = testCase.getPropertyValue("ftpPassword")
def String localFilePath = testCase.getPropertyValue("localFilePath")
def String remoteFileDir = testCase.getPropertyValue("remoteFileDir")
//
//
Session session = null
Channel channel = null
try {
  log.info("Starting sftp upload process")
  JSch ssh = new JSch()
      
  session = ssh.getSession(ftpUsername, ftpHost, ftpPort)
  session.setConfig("StrictHostKeyChecking", "no"); //auto accept secure host
  session.setPassword(ftpPassword)
  session.connect()
  log.info("Connected to session")
      
  channel = session.openChannel("sftp")
  channel.connect()
  log.info("Connected to channel")
      
  ChannelSftp sftp = (ChannelSftp) channel;
  sftp.put(localFilePath, remoteFileDir);
  log.info("File Uploaded " + localFilePath + " TO " + remoteFileDir)
    
} catch (JSchException e) {
  e.printStackTrace()
  log.info("JSchException " + e.message)
} catch (SftpException e) {
  e.printStackTrace()
  log.info("SftpException " + e.message)
} finally {
  if (channel != null) {
    channel.disconnect()
    log.info("Disconnected from channel")
  }
  if (session != null) {
    session.disconnect()
    log.info("Disconnected from session")
  }
  log.info("sftp upload process complete")
}

Changes I did was to define the input values based on the properties from the testcase. And move the session and channel variable declartions out of the try, to get it accessible from the finally branch. And to replace e.printStackTrace from the logging by e.message, to have a propery message (e.printStackTrace returns null) in the logging.

The reason that I suggest to have it in a separate test cases is to enable it to be called from actual testcases with parameters. To do so add to your test case a call-test case activity:

Set the following properties:

Choose 'Run primary TestCase (wait for running to finish, Thread-Safe)' option as Run Mode.

And provide the property values.

This script copies a file from a file location and uploads it. But I want to be able to insert some runtime specific options to refer to in asserts and later JDBC calls. To check on specific running instances. So I want to be able to adapt the content running in my test case. Actually I want to upload a string fetched from a property, maybe with expanded properties.

So I copied the testcase and groovy activity and adapted the script to:

//Send Files to SSH Location
//
//Download jsch-0.1.54.jar from http://www.jcraft.com/jsch/ and copy it to  SoapUI-Home/bin/ext location
//Example from: https://www.snip2code.com/Snippet/413499/SoapUI-Groovy-Script-compatible-SFTP-fil

//import org.apache.commons.net.ftp.FTPSClient
import com.jcraft.jsch.*
import java.nio.charset.StandardCharsets
//
// Properties
// 
def testCase = testRunner.testCase;
//
def String ftpHost = testCase.getPropertyValue("ftpHost")
def int ftpPort = testCase.getPropertyValue("ftpPort").toInteger()
def String ftpUsername = testCase.getPropertyValue("ftpUsername")
def String ftpPassword = testCase.getPropertyValue("ftpPassword")
def String fileContent = testCase.getPropertyValue("fileContent")
def String remoteFile = testCase.getPropertyValue("remoteFile")
//
Channel channel = null
Session session = null
try {
  log.info("Starting sftp upload process")  
  JSch ssh = new JSch()
      
  session = ssh.getSession(ftpUsername, ftpHost, ftpPort)
  session.setConfig("StrictHostKeyChecking", "no"); //auto accept secure host
  session.setPassword(ftpPassword)
  session.connect()
  log.info("Connected to session")
      
  channel = session.openChannel("sftp")
  channel.connect()
  log.info("Connected to channel")
      
  ChannelSftp sftp = (ChannelSftp) channel; 

  byte[] fileContentBytes =   fileContent.getBytes(StandardCharsets.UTF_8)
  InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);
  log.info("Start uploaded to " + remoteFile)
  sftp.put(fileInputStream, remoteFile);
  log.info("File Content uploaded to " + remoteFile)
    
} catch (JSchException e) {
  e.printStackTrace()
  log.info("JSchException " + e.message)
} catch (SftpException e) {
  e.printStackTrace()
  log.info("SftpException " + e.message)
} catch (Exception e) {
  e.printStackTrace()
  log.info("Exception " + e.message)
} finally {
  if (channel != null) {
    channel.disconnect()
    log.info("Disconnected from channel")
  }
  if (session != null) {
    session.disconnect()
    log.info("Disconnected from session")
  }
  log.info("sftp upload process complete")
}

here the lines and related properties:

def String localFilePath = testCase.getPropertyValue("localFilePath")
def String remoteFileDir = testCase.getPropertyValue("remoteFileDir")

are changed to:

def String fileContent = testCase.getPropertyValue("fileContent")
def String remoteFile = testCase.getPropertyValue("remoteFile")

Then the lines:

  byte[] fileContentBytes =   fileContent.getBytes(StandardCharsets.UTF_8)
  InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);

convert the fileContent property value to an InputString. And that is given as an input to the statement sftp.put(fileInputStream, remoteFile);. Notice that since we upload file content, we need to provide a remoteFile path, including file name, insead of a remote directory. And that we need an extra import java.nio.charset.StandardCharsets.

It would be nice if the guys from SmartBear add both put and get as a seperate activity.