Check out MockMotor

September 1, 2016

Primer: Limiting the Concurrency

Putting a cap on the maximum number of parallel request in GenericParallel.

p8iAlVe

Parallel calls are an extremely powerful tool and, like any powerful tool, can be dangerous.

Hitting a backend service with 5 parallel requests will reduce the response time.

Hitting the same service with 50 parallel requests may bring that service down, causing a prolonged downtime.

Overload Protection

To prevent accidental overloading, GenericParallel uses the concept of maximum concurrency.

A call to GenericParallel is not allowed to initiate more than the given number of requests in parallel. If there are more requests to execute, they are split into batches and those batches are executed sequentially.

For instance, a call to GenericParallel may contain 12 requests, while the maximum concurrency is 5 (by default). The requests then are split into 3 batches, which are executed one after the other in sequence. The requests within the batch are still executed in parallel:

Batch 1: Requests 1, 2, 3, 4, 5
Batch 2: Requests 6, 7, 8, 9, 10
Batch 3: Requests 11, 12

The developer may control the maximum concurrency for each call to GenericParallel. This is done by providing the GPSMaxParallel attribute. For example, let’s increase the concurrency to 10 for a service we know scales up very well:

<typ:GPS GPSMaxParallel="10" xmlns:typ="http://genericparallel/types">
  <typ:Requests>
   <typ:Request GPSTarget="Profile/RetrieveProfile">
     <RetrieveProfile>
       <AccountNo>12345678</AccountNo>
     </RetrieveProfile> 
   </typ:Request> 

   <typ:Request GPSTarget="Profile/RetrieveProfile">
     <RetrieveProfile>
       <AccountNo>12343654</AccountNo>
     </RetrieveProfile> 
   </typ:Request> 

   ... and dozens of additional account numbers here ...

  </typ:Requests> 
</typ:GPS>

If you would like further details about limiting the concurrency in GenericParallel, please read the documentation.

If you would like to implement the batching in your own split-join services, feel free to download GenericParallel to learn from the provided code.

Vladimir Dyuzhev, author of GenericParallel

About Me

My name is Vladimir Dyuzhev, and I'm the author of GenericParallel, an OSB proxy service for making parallel calls effortlessly and MockMotor, a powerful mock server.

I'm building SOA enterprise systems for clients large and small for almost 20 years. Most of that time I've been working with BEA (later Oracle) Weblogic platform, including OSB and other SOA systems.

Feel free to contact me if you have a SOA project to design and implement. See my profile on LinkedIn.

I live in Toronto, Ontario, Canada.  canada   Email me at info@genericparallel.com

Search

Categories

Tags