Throttling, Unbalanced & Over-Allocated

A service’s domain-wide throttling value may be different from what you have configured for it.

I have mentioned already that the throttling value assigned to a service on a specific managed server is calculated as the throttling value from the configuration divided by the number of managed servers, and rounded up if not whole.

There are curious extensions from this statement.

Servers May Have Unequal Throttling (e.g. 6, 6, 4)

Huh? How?

Imagine you have a domain with 2 managed servers and a service with throttling configured for 12 connections.

Managed servers MS1 and MS2 get 6 connections each. This is because when MS1 and MS2 were starting, they read in the domain configuration that there are 2 managed servers in total, so 12 / 2 = 6.

Now clone MS1 (creating MS3) and start it too. The service on the newly created server gets a throttling value of 4. This is logical too, for we now have 3 managed servers, and 12 / 3 = 4.

But test the traffic through MS1 and MS2, and you’ll see that they still have throttling set to 6!

This is because they still remember the domain configuration as it was at their startup time, and it had 2 MS then.

Try to reconfigure the service. Say, set the throttling to 24. MS1 and MS2 will get 12 connections each, and MS3 gets 8.

The only reliable way to re-distribute the throttling equally is to restart MS1 and MS2.

Throttling Can Be Larger Than You Think

It is worth noting that in the scenario above, the total number of connections the domain now supports is 6 + 6 + 4 = 16, which is 33% higher than the value you’d think it does support if you were looking at the configuration alone. However, changing the number of managed servers is rare, so let’s consider this case an academic one.

There is a much more probable case that may cause the over-allocation of resources: the rounding up.

Rounding up is intended to guarantee the allocation of at least one connection even if the throttling number is less than the number of managed servers.

If the configured value cannot be divided by the number of managed servers without a remainder, the managed server value gets rounded up to the next integer.

The immediate result is you end up with a few more (than you configured) connections.

For instance, a throttling value of 11 in a 4-node cluster will lead to 11 / 4 = 2.75 => 3 connections per managed server, which in total gives 4 x 3 = 12 maximum open connections, a 9% overuse.

Note we’d get the same 12 connections if we’ve configured the service throttling at only 9: 9 / 4 = 2.25 => 3, with 33% overuse.

In a 4-node cluster you may get 3 more connections, and in 8 node cluster - 7 more connections. The backend server may or may not be able to handle this extra permitted load.

Again, this state is not immediately visible from looking at the throttling configuration alone.