How do I know when a channel is done processing a CSV file?

Question

I have a channel responsible for processing CSVs from a folder on an SFTP host.  The CSVs will have a header row, and I want each row transformed and sent to the DB/destination individually.  All is working as it should by setting the record grouping to 1 row in each group.  My question is how can I determine when the last row in the file has been processed?  I'd like to make a web service call at that time.  I've tried to keep track of the current row by incrementing a variable, but when trying to compare to source.getRowCount, the total row count seems to be 1 (due to the grouping).  If I change the grouping back to blank/all, source.getRowCount returns a total number of rows, but the pipeline doesn't seem to iterate over all rows and just the first row is produced to the destination.

Answer 1

YOu do have a couple of options:

1. Change the grouping to blank or all and then add a mapping and use a for loop and that would use the qie.spawnNewMessage for each row. Once they are all spawned then call the web service. We may need to check that there are no messages processing before the call as well.

2. Change the grouping to blank similar to number one but instead of a spawn new message move the logic from all your nodes into that script and when it is completed then call the webservice.

3. We can change the source node to be a custom script where we pick up the file. We then do a for loop and on the last record when we do the qie.addInboundMessage we put "lastRow" or something like that in the file name field. In the script, we can check for that and if that is in the file name call the web service.

We are happy to help with the implementation of any of these options as well. Just email support@qvera.com for assistance.