Queue based load leveling

Intent

Use a queue that acts as a buffer between a task and a service that it invokes in order to smooth intermittent heavy loads that may otherwise cause the service to fail or the task to time out. This pattern can help to minimize the impact of peaks in demand on availability and responsiveness for both the task and the service.

Class diagram

Applicability

• This pattern is ideally suited to any type of application that uses services that may be subject to overloading.
• This pattern might not be suitable if the application expects a response from the service with minimal latency.

Tutorials

• Queue-Based Load Leveling Pattern

Real world example

• A Microsoft Azure web role stores data by using a separate storage service. If a large number of instances of the web role run concurrently, it is possible that the storage service could be overwhelmed and be unable to respond to requests quickly enough to prevent these requests from timing out or failing.

Credits

• Queue-Based Load Leveling pattern