Polling SFTP from a Custom Script Receiver

When no built-in source receiver fits, for example, an SFTP folder containing a mix of metadata text files and binary attachments that must be downloaded together, or a layout where the listing call and the read call need different paths, a Custom Script Receiver can take over the pickup. The script lists the remote folder, downloads each matching file, saves a local copy, removes or archives the source copy, and creates one inbound message per row of a metadata text file.

The canonical building blocks are qie.listSFTPEndpoint(...) to enumerate the folder, qie.readSFTPFile(...) to download a single file, qie.writeFile(...) to persist the bytes locally, and qie.addInboundMessage(...) to enqueue messages into the channel. Credentials live in a system variable of type Credentials; everything else that varies by environment lives in channel cache.

qie.listSFTPEndpoint returns leaf file names, not full paths

The array returned by qie.listSFTPEndpoint(...) contains just the file name of each match, for example report_20260609.txt, not /outbound/metadata/report_20260609.txt. qie.readSFTPFile(...) expects the full remote path. The script must join the source folder and the returned name itself before calling read; passing the bare leaf name returns a null result and the next getNode call fails with Cannot call method "getNode" of null.

Configuration

Set these on the channel's Cache tab before running the script. Putting each value in channel cache keeps the script identical across test and production:

Key	Example
sftpHost	vendor-sftp.example.com
sftpPort	22
sftpSourcePath	/outbound/metadata
sftpDownloadAllFilename	*
sftpSourceFilename	report_*.txt
sftpSourceAfterDownloadAction	delete or leave
sftpCredentialsVariableName	VendorSftpCredentials
sftpTimeoutMs	30000
localInboundPath	/qie/interfaces/vendor/inbound

Create the system variable named in sftpCredentialsVariableName as a Credentials variable and store the SFTP username and password there. The script retrieves them with qie.getVarUsername(...) and qie.getVarPassword(...) so plain-text credentials never appear in the script.

The source-node script

The script runs in two passes. Pass 1 downloads every matching file to the local inbound folder. Pass 2 walks the metadata text files that were just saved and enqueues one inbound message per data row. The two-pass shape matters when a row in the metadata references a binary file by name. By the time the row becomes a message, the referenced binary is guaranteed to already exist locally.

var sftpHost                      = channelCache.getValue('sftpHost');
var sftpPort                      = parseInt(channelCache.getValue('sftpPort', '22'), 10);
var sftpSourcePath                = channelCache.getValue('sftpSourcePath');
var sftpDownloadAllFilename       = channelCache.getValue('sftpDownloadAllFilename', '*');
var sftpSourceFilename            = channelCache.getValue('sftpSourceFilename');
var sftpSourceAfterDownloadAction = channelCache.getValue('sftpSourceAfterDownloadAction', 'leave');
var sftpCredentialsVariableName   = channelCache.getValue('sftpCredentialsVariableName');
var sftpTimeoutMs                 = parseInt(channelCache.getValue('sftpTimeoutMs', '30000'), 10);
var localInboundPath              = channelCache.getValue('localInboundPath');

var sftpUserName     = qie.getVarUsername(sftpCredentialsVariableName);
var sftpPassword     = qie.getVarPassword(sftpCredentialsVariableName);
var deleteRemoteFile = StringUtils.equals(sftpSourceAfterDownloadAction, 'delete');

// Pass 1: list every matching file and download it locally.
var remoteFiles = qie.listSFTPEndpoint(
   sftpHost, sftpPort,
   sftpSourcePath + '/' + sftpDownloadAllFilename,
   true, true, '',
   sftpUserName, sftpPassword
);

var metadataFiles = [];

for (var i = 0; remoteFiles != null && i < remoteFiles.length; i++) {
   // listSFTPEndpoint returns leaf names only — prepend the source folder before reading.
   var leafName     = remoteFiles[i];
   var remotePath   = sftpSourcePath + '/' + leafName;
   var localPath    = localInboundPath + '/' + leafName;

   var downloaded = qie.readSFTPFile(
      sftpHost, sftpPort, remotePath,
      deleteRemoteFile, '',
      true, true, '',
      sftpUserName, sftpPassword,
      sftpTimeoutMs
   );

   var encodedBytes = downloaded.getNode('/ftpFile/encodedBytes');
   var fileContents = qie.base64Decode(encodedBytes);

   qie.writeFile(localPath, fileContents, true);
   qie.info('Downloaded ' + remotePath);

   if (StringUtils.endsWith(leafName, StringUtils.substringAfter(sftpSourceFilename, '*'))) {
      metadataFiles[metadataFiles.length] = { name: leafName, text: fileContents };
   }
}

// Pass 2: with every file on disk, create one inbound message per data row.
for (var m = 0; m < metadataFiles.length; m++) {
   var lines      = StringUtils.splitByWholeSeparator(metadataFiles[m].text, '\n');
   var headerLine = null;
   var rowCount   = 0;

   for (var lineIndex = 0; lines != null && lineIndex < lines.length; lineIndex++) {
      // StringUtils.trim drops a trailing \r so CRLF files behave the same as LF files.
      var currentLine = StringUtils.trim(lines[lineIndex]);

      if (StringUtils.isBlank(currentLine)) {
         continue;
      }

      if (headerLine == null) {
         headerLine = currentLine;
      } else {
         qie.addInboundMessage(headerLine + '\n' + currentLine, metadataFiles[m].name);
         rowCount = rowCount + 1;
      }
   }

   qie.info('Created ' + rowCount + ' inbound message(s) from ' + metadataFiles[m].name);
}

Each created message carries the header row followed by one data row, so downstream mapping can read columns by name (via qie.parseCSVString or a parsed CSV source) without needing to look back at the metadata file. The second argument to qie.addInboundMessage is preserved as the source file name on each message, so HL7-style file-name functions and message-history searches still work.

Disposition of the source file

qie.readSFTPFile decides what happens to the remote copy after the download succeeds based on its deleteFile and archivePath arguments:

• Delete after download: pass deleteFile = true and leave archivePath blank. The script above does this when sftpSourceAfterDownloadAction = delete.
• Move to a remote archive folder: pass an archivePath value. The remote file is moved there once the bytes are downloaded.
• Leave in place: pass deleteFile = false and an empty archivePath. The remote file remains and is picked up again on the next scan, which is appropriate only when the local script keeps state to avoid reprocessing.

The local copy is always overwritten if a file with the same name already exists. qie.writeFile(path, content, true) creates parent directories as needed and replaces any prior file at path.

Common pitfalls

A few traps that are not obvious from the function signatures alone:

• Rhino script gotchas. The script above uses StringUtils.trim to strip the trailing \r after splitting on \n, and supplies defaults to channelCache.getValue for the values that become integers. Both patterns sidestep the NaN-to-Integer errors documented in Common Scripting Pitfalls.
• Wildcard matching for sftpSourceFilename. The script uses StringUtils.substringAfter(sftpSourceFilename, '*') plus endsWith to keep the matcher simple. It handles *.txt, prefix_*.txt, and prefix_* correctly. If a pattern needs ? or multiple wildcards (*report*.txt), check the full file name against the pattern explicitly rather than trying to hand-roll a general matcher.

See the Channel Cache and System Variables pages for how to define the values referenced by this script.