readLockMinAge for incoming FTP delay

Question

We have an issue that a larger file is getting picked up by QVERA FTP from remote server while the file is being written.  Hoping to add a 5 minute aging delay for the file before we attempt to pick it up.

 

Found the apache code readLockMinAge, and added it as custom apache command parameters to the inbound FTP setup.

With just the code no time parameters it is successful, but we error if we add in the time.

I tried readLockMinAge=300s  and readLockMinAge=30000ms and readLockMinAge=30000

All three are producing an error in the FTP connection

org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: sftp://69.8.217.154:22//Erie_GL?connectTimeout=30000&fileExist=Fail&passiveMode=true&password=mXq3ZrA%245%40-%3C%5BU7P&readLockMinAge=30000ms&separator=UNIX&soTimeout=10000&stepwise=false&throwExceptionOnConnectFailed=true&username=qveradftp due to: Failed to resolve endpoint: sftp://69.8.217.154:22//Erie_GL?connectTimeout=30000&fileExist=Fail&passiveMode=true&password=mXq3ZrA%245%40-%3C%5BU7P&readLockMinAge=30000ms&separator=UNIX&soTimeout=10000&stepwise=false&throwExceptionOnConnectFailed=true&username=qveradftp due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{readLockMinAge=30000ms}]

If I remove everything after the = then I can test successfully and see the file  that is pending.

 

Answer 1

At this time, QIE uses version 2.12.0 of Apache Camel.  The readLockMinAge parameter has been added in 2.15 of Apache Camel (http://camel.apache.org/file2.html ).  It sounds like a valuable feature.  I'll create an enhancement request and discuss it with the team.

In the meantime, you could try using the readLock option.  

This will attempt a rename the file to see if you have an exlusive lock:

readLock=rename

If your volume is low on the interface and can afford to wait between each FTP request, you can wait an interval and see if the timestamp has changed:

readLock=changed&readLockCheckInterval=30000&readLockTimeout=60000

Camel FTP gives the following guidance on this option:

The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval

Hope that helps.