Recursive Subdirectory Scanning for File, Network Share, and FTP Sources
Overview
QIE adds an Include All Subdirectories option to File, Network Share, and FTP source nodes. When this option is enabled, QIE scans the configured source path recursively instead of only looking in the top-level folder.
This is useful when partners, devices, or upstream jobs drop files into date-based, customer-based, or location-based subfolders under a common root.
How to Configure It
- Open the source node.
- Configure the source path as usual.
- Enable Include All Subdirectories.
- Save the channel and test the source path.
The same option is available on:
- File sources
- Network Share sources
- FTP sources
What Changes When It Is Enabled
- QIE scans the source path and nested folders beneath the configured source path.
- The source path test actions include matching files found in subfolders.
- QIE preserves the source-relative folder structure as source metadata for the message.
- If the archive directory lives under the same source root, QIE skips that archive tree during recursive scans, so archived files are not picked up again.
Source-Relative Example:
- Source 'Path' set to: /incoming/files/source-root/
- Include All Subdirectories enabled
- File found in: /incoming/files/source-root/sub1/sub2/
- source-relative path: sub1/sub2
- So source.getFileDirectory() returns: sub1/sub2
Archiving Behavior
When recursive scanning is enabled via Include All Subdirectories, the incoming file may have a relative path such as orders/2026/03/input.hl7.
If the archive path does not use source metadata, QIE flattens that relative path into a single archive filename:
orders/2026/03/input.hl7 -> orders_2026_03_input.hl7
If you want to preserve the original folder structure in the archive, use the new source metadata tags in the archive path, especially {s-metadata:FILE_DIRECTORY}.
Example
If the source path is:
/inbound/*.hl7
and files arrive in:
/inbound/west/2026/04/order1.hl7
/inbound/east/2026/04/order2.hl7
then enabling Include All Subdirectories allows QIE to find both files from the same source node.
Notes
- The Include All Subdirectories option is only for directory-style sources (File, Network Share, and FTP). If you only want QIE to watch the top folder, leave it disabled.
- Relative folder information becomes especially valuable when combined with source metadata node tags or the Code Wizard functions like source.getFileDirectory().
Archive Exclusion When Recursive Scanning Is Enabled Example
When recursive scanning is enabled via Include All Subdirectories, if the archive root directory lives under the source root, QIE skips that archive tree during recursive scans, so archived files are not picked up again. This behavior excludes the files in the archive root and its subfolders.
With this configuration and assumptions:
- Channel Cache region set to: west
- Source 'Path' set to: /incoming/files/source-root/
- Include All Subdirectories is: enabled
- Archive Path ("Move file to") set to: /incoming/files/source-root/archive-root/{cc:region}/{s-metadata:FILE_DIRECTORY}/{s-metadata:FILE_NAME}
- This file found in: /incoming/files/source-root/sub1/sub2/example.hl7
then the source relative path will be:
`sub1/sub2`
The {s-metadata:FILE_DIRECTORY} and {s-metadata:FILE_NAME} node tags will resolve to:
sub1/sub2
and
example.hl7
The Archive Exclusion folder (not scanned by the recursive scan including subfolders) is:
archive-root
The file will be archived to:
/incoming/files/source-root/archive-root/west/sub1/sub2/example.hl7
Notice that when source-metadata tags are used the relative folder and filename of the archived file is not flattened and the folder structure returned by {s-metadata:FILE_DIRECTORY} is preserved under the archive-root. Also notice that node tags like cc:region} can be used to alter where the file is archived.
If the archive-root folder is not under the source-root folder, then it is not scanned and no exclusion is applied.
Also see these related questions:
How to use source.getFileDirectory()?
What source metadata node tags are available in QIE?
