Sidebar

Within the context of a single message, can initiate concurrent tasks?

0 votes
323 views
asked May 8, 2018 by mike-r-7535 (13,830 points)
I want to multithread the process of a single message. For example, I need to make a series of long running WebService calls or Database queries from different databases.  I want to increase my throughput by running these tasks in parallel.  What is the best approach to accomplish this?

1 Answer

0 votes

Before taking this approach, you should always evaluate the trade offs.  Splitting a process into sub tasks will complicate the workflow, but it has the potential of increasing throughput if your processors are busy waiting for a series of long running tasks which can be done in any order.

Here is a link to a configuration that should demonstrate how you could initiate subtasks and aggregate the results.

It actually involves two channels, a temporary H2 database, and some published functions to create, drop, delete, insert, update, and retrieve the records of that temporary database.

 

Task Initiator Channel

- At startup, drops and creates the temporary table to coordinate the tasks

- Loops through patientIds

  o Inserts tasks into the database

  o Sends messages to Task Executor channel for it to process the requests

- Loops through patientIds

  o checks database for tasks with responses, aggregates a message, and deletes the tasks once completed

 o repeats the loop if not all tasks are complete (I have a pause of 1 second, and I make 5 passes through the channel.  This can be changed.)

 

Task Executor Channel

- Receives a task, and returns an HTTP 202 immediately (This allows the Task Initiator to move onto the next patientId)

- Executes the task (this would be your webservice calls), and when completes the task by updating the database with the response or the newPatient object

 

Again, my opinion is that this approach should only be used if your channel is struggling to keep up with the demand, you have the available resources to increase processing, and there are multiple long running tasks that can be broken up and run independently.  Otherwise, you will not get the benefit of the added complexity.

answered May 8, 2018 by mike-r-7535 (13,830 points)
...