You should realize that not all services are equally useful. Some, like submitting orders, are directly generating revenue to your company. Some others, like getting orders history, while important, can be sacrificed to let orders get submitted. When the system is under a higher then usual load, how can we dynamically shutdown non-essential services to release more resources to the essential ones? Our Playground: the Orders Service For our experiments, I’m going to use a mock service that has two operations:
A service’s domain-wide throttling value may be different from what you have configured for it. I have mentioned already that the throttling value assigned to a service on a specific managed server is calculated as the throttling value from the configuration divided by the number of managed servers, and rounded up if not whole. There are curious extensions from this statement. Servers May Have Unequal Throttling (e.g. 6, 6, 4) Huh?
Throttling value, unlike the max threads constraint, is divided equally between all managed servers in the domain. Throttling allows you to limit the number of concurrent requests to a backend server. What is confusing is that the value is not per managed server, but for the whole domain. This makes sense, if you think of this use-case: You have a backend service that can only serve 20 connections at a time.
Use throttling in business services to protect the OSB from stuck threads. Throttling allows us to limit the number of outgoing requests currently in progress. At first, I thought it would only be useful to protect shaky downstream services, which crash if hit with too many concurrent requests at once. And throttling is useful for that. But not only for that. It is also useful to protect the OSB itself from too many stuck threads.