OSB Throttling Value Is Per Domain

Throttling value, unlike the max threads constraint, is divided equally between all managed servers in the domain.

Throttling allows you to limit the number of concurrent requests to a backend server.

What is confusing is that the value is not per managed server, but for the whole domain.

This makes sense, if you think of this use-case:

You have a backend service that can only serve 20 connections at a time. Any more than that and it becomes slow and eventually crashes.

When you set the throttling value for the business service to 20, you want to guarantee that cap, no matter how the managed servers are reconfigured.

So, if we set the throttling value to 20, each managed server can open:

• In a domain with 2 managed servers: ²⁰⁄₂ = 10 connections

• In a domain with 4 managed servers: ²⁰⁄₄ = 5 connections

• In a domain with 8 managed servers: ²⁰⁄₈ = 2.5 => 3 connections; when the value is fractional, OSB rounds it up to the nearest integer; so the real throttling in this case is 8 * 3 = 24, and not the specified 20

Ramifications For Split-Join And GenericParallel Usage

This tricky behaviour of throttling may cause unexpected failures when used in conjunction with Split-Join and GenericParallel (which is based on Split-Join).

You may have carefully estimated your capacity needs, and assigned a throttling value of say 40 to a business service in a 4-node domain. That’s 10 per managed server.

Then you aligned Split-Join or GenericParallel service to this value. You custom-code Split-Join (or configure GenericParallel) to break the incoming messages into batches of 10 concurrent requests.

All is good until the domain is scaled horizontally (or the throttling value is reduced to harden the backend service against the load), so there are fewer connections available to each managed server.

At this point calls from Split-Join/GenericParallel start to fail for those requests in the batch that do not get the connection allocated for them.

The batch size in Split-Join/GenericParallel and the throttling value must be coordinated.