At work we use (HAProxy)->(Varnish->HAProxy)->(IIS). Each parenthesis is a computer/cluster, basically we have a 4 node HAP cluster talking to a 4 node Varnish-HAP cluster, in which each computer has varnish talking to a local HAP (it can also talk to the cluster’s other HAP but their weight is negligible, only used if local machine’s HAP is dead). We mostly use HAProxy because it has much better load balancing and queueing options than Varnish.

Total time for a request is under 9ms on a primed cache usually and we serve 30,000 -> 150,000 hits per second. Our peak CPU usage on the machines has been 5% at 150k hits/s.

One limitation that this scheme does have is the inability to split the cached data between multiple instances (with consistent hashing), largely because the current nginx memcached module does not have the consistent hashing logic. This could be addressed by extending the nginx module though!

That would be discouraging as the advantage of memcached is that you can farm out the requests.

If both nginx and the client library (in this case ruby’s) used the same caching algorithm would caching against a farm work? If so, what would the nginx.conf look like? What is the algorithm that nginx uses to cache to memcached?

Good article. Thanks.