Throttling web service / application

There are different throttling techniques one can use to throttle a network applications, Following are some well established and simple techniques used in different network applications.

• Token Bucket
• Leaky Bucket
• Rate Limiting

Out of which Token Bucket suits very well for Web Services with the following reasoning,

• Failing fast on the incoming requests because there is no tokens available. That helps the caller to strategize immediately what to do in case of running out of limits.
• Failing fast also helps in not building a log task queue at the callers side.
• New milk is better than old milk (milk and wine policy) because keeping an old request with no relevance to the caller is just waste of resources if relevance is what you care about.

In my coming post I shall implement a simple throttler that uses the Token Bucket algorithm in Java.