 |
In the order processing example introduced in the previous patterns , each order item that is not currently in stock could be supplied by one of multiple external suppliers. However, the suppliers may or may not have the respective item in stock themselves, they may charge a different price and may be able to supply the part by a different date. To fill the order in the best way possible, we should request quotes from all suppliers and decide which one provides us with the best term for the requested item.
How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?
Use a Scatter-Gather that broadcasts a message to multiple recipients and re-aggregates the responses back into a single message
The Scatter-Gather routes a request message to the a number of recipients. It then uses an Aggregator to collect the responses and distill them into a single response message.
There are two variants of the Scatter-Gather that use different mechanisms to send the request messages to the intended recipients:
- Distribution via a Recipient List allows the Scatter-Gather to control the list of recipients but requires the Scatter-Gather to be aware of each recipient's message channel.
- Auction-style Scatter-Gathers use a Publish-Subscribe Channel to broadcast the request to any interested participant. This option allows the Scatter-Gather to use a single channel but at the same time relinquishes control.
... Read the entire pattern in the book Enterprise Integration Patterns
Example: Serverless Loan Broker on AWSNEW
The serverless Loan Broker example includes a Scatter-Gather that routes requests for a loan quote to interested banks and selects the best offer from the incoming responses. The example implements both a a Recipient List using AWS Step Functions' Map state (see Part 2) and a Publish-Subscribe Channel using the Amazon Simple Queuing Service (SQS) (see Part 3).
Example: Mulesoft ESBNEW
Mulesoft's enterprise service bus includes a Scatter-Gather construct, described as Scatter-Gather Router and can be configured using the scatter-gather element.
Although each route is independently executed, the Scatter-Gather Router configures a pre-defined number of paths, so it doesn't broadcast to an unknown number of recipients. Knowing all recipients makes the completeness condition of "Wait for all" (see Aggregator) a natural choice. Route errors propagate to the Scatter-Gather, which in turn flags an error, passing on any results that were successfully passed by the individual routes. The aggregation strategy is concatenation, turning single variable values into a list of all route results.
Example: Azure Durable Functions (Fan out/fan in)NEW
Durable functions, which are used to orchestrate serverless Azure Functions, make it easy to implement Scatter-Gather, following the Fan out/fan in construct. which is supported by the WhenAll method, which waits for all functions to execute:
Fan out/fan in with Azure Durable Functions
def orchestrator_function(context: df.DurableOrchestrationContext):
work_batch = yield context.call_activity("GetVendors", None)
parallel_tasks = [ context.call_activity("Vendor", b) for b in work_batch ]
outputs = yield context.task_all(parallel_tasks)
total = sum(outputs)
The number of recipients can be calculated at run-time, and the completeness condition can support a variety of strategies, including "Wait for all" (Task.WhenAll), "First Best" (Task.WhenAny), or "Fixed Timeout" (Task.WaitAll).
Related patterns:
Aggregator, Introduction to Composed Messaging Examples, Asynchronous Implementation with MSMQ, Asynchronous Implementation with TIBCO ActiveEnterprise, Composed Message Processor, Publish-Subscribe Channel, Recipient List, Return Address, Splitter
Integration Pattern Language
