Comments on: Zero-Downtime Restarts with HAProxy

By: Don Ferguson

Don Ferguson — Fri, 26 Feb 2010 01:05:51 +0000

Ilya, great stuff. There’s one part I don’t understand. webA and bkpA refer to the same process; when the iptable rule is removed, old sessions continue to be served by bkpA. Presumably new requests are served by webB. Now if we shut down webA (which is bkpA), won’t those sessions be intererupted? If we leave bkpA running to continue serving old sessions, won’t we have trouble restarting webA, since it’ll want to listen on port 1000, which bkpA is still listening to?

By: Mattias Berge

Mattias Berge — Tue, 09 Feb 2010 12:48:13 +0000

Wouldn’t a simple REJECT for TCP state NEW have the same effect? ESTABLISHED connections finishing, new connections will be rejected and haproxy will detect it as down immediately?

:-)

By: findchris

findchris — Fri, 05 Feb 2010 22:05:40 +0000

Someone stole your post, FYI:
http://titopanicker.net/blog/?p=59

By: nowayjose

nowayjose — Thu, 07 Jan 2010 18:03:34 +0000

You are using HA proxy and assuming it solves a problem that does not exist. Please don’t.

For years people have been deploying applications with ZERO downtime and WITHOUT HA PROXY.

By: oliver

oliver — Thu, 02 Jul 2009 13:59:03 +0000

Hi Ilya,

I quickly added a stateless page so no session are created and now it works perfectly! Sorry for the post but as the server were marked as down even without the option httpchk I thougt it is sufficient.

Oliver

By: Ilya Grigorik

Ilya Grigorik — Thu, 02 Jul 2009 13:20:42 +0000

Oliver: Ah, you do need the option httpchk in there! That’s how HAProxy determines that the server is down (after two unsuccessful hits to your server). Otherwise it’ll continue dispatching requests to that host, and will redispatch (if you have it enabled) once it determines that server is down.

By: oliver

oliver — Thu, 02 Jul 2009 12:59:38 +0000

Hi Ilya,
I rechecked your recommended chapter and the only difference is that I do not declare “option httpchk HEAD / HTTP/1.0″ as this creates a new jsession every 2 seconds on my tomcat servers.

if I start the maintenance mode for the server then it is marked as down in the haproxy interface while the backup instance is still up. The chapter “4.2 Soft-stop using backup server” states that clients with a valid cookie will still reach the backup server. So if I already got a cookie assigned and my server is in maintenance mode what will take care that I do nor reach the backup again? Does the cookie expire?

This is my test:
1. Loggin in my application
2. check which server I got assigned
3. put the server server in maintenance mode
4. logout of my application - > invalidate the jsession
5. if log in again im still get the same server which is in maintenance.

Thank you for your help,
Oliver

By: Ilya Grigorik

Ilya Grigorik — Wed, 01 Jul 2009 03:28:32 +0000

Oliver, it sounds like something may have gone wrong in the way you disabled the server (guessing: routing). Take a close look at section “4.2 Soft-stop using backup servers” in the HAProxy doc, it describes the scenario you’re looking for in great detail.

By: oliver

oliver — Tue, 30 Jun 2009 15:10:19 +0000

Hi,
I followed your example with sticky sessions but clients
who’s session are expired and the related server is in maintenance mode are directed to the server again
and new sessions are created. So I can not restart the server.

I think this is because of the cookie usage in the backup server definition?

By: till

till — Fri, 08 May 2009 10:48:45 +0000

We got a coyotepoint loadbalancer and what we do is, we quiesce the server which was assigned to a cluster, watch the status until connections are cleared and then restart the server, or take it down for maintenance, etc..

Everything else seems super complicated.