If you don't need to transform the message, but only need to move the files from one folder to another or an SFTP endpoint, the best way to do it is using Streams. This process would be as follows:
1) Create a custom receiver, this reciever reads the list of filenames and filters out any you don't want. Instead of reading the whole file, you add a message that is the full file path only. You will only send in a single file at a time. The script should skip processing if there is a message inbound, processing, or outbound to prevent duplicate processing of files.
2) In the mapping node, you create a new java.io.File(source.getNode('/')) object that will be passed into the send SFTP command. By passing in the File object, the FTP command will stream the FTP instead of pushing the bytes directly meaning the whole file is never loaded into memory at any time.
3) Create a second mapping function that will stream the file bytes to a archive location if desired.
4) After moving the file where you want it to go, you will then delete the source file from another mapping function. This will prevent it from being processed a second time.
5) Since the file move happened in a mapping node, you will send the message to a discard destination node.
This exmple channel shows you how to accomplish this process. NOTE: By using a WS Connection, we are able to store the username and password to the SFTP endpoint without exposing it in plain text.
