Is there any constraint on the amount of files an FTP Source will pull when set to execute on a schedule?

0 votes
asked Sep 19, 2023 by nathan-b-7857 (330 points)
I am testing the behavior of schedule executions using an FTP Source and I see that it pulled all 240 files I inserted into a folder. Is there a limit on the amount of files it can pull? Is it based on the amount of memory available?

2 Answers

0 votes
Qvera Interface Engine does not have any imposed limits on the number of files/messages you can download and process. You will need to have enough available memory and CPU to handle the processing of those messages.

We are always happy to help look at your process with you when it is running and help optimize and assist with looking at the memory needs you will need.
answered Sep 19, 2023 by brandon-w-8204 (33,270 points)
0 votes

Hi Brandon, we may potentially receive several 2GB files at a given scheduled read. To accommodate other running feeds at the time, is there a way to limit the amount of files read? If so, what could the basis of that be on?

answered Sep 20, 2023 by nathan-b-7857 (330 points)