HA Nginx in Docker with keepaliveD
Do you need to create HA proxy and thinking of Nginx ? Or maybe thinking even further … about Nginx and docker ? So something really simple and what you can defenitely take to next level.
In my scenario I need something which you could call … hmmm a “service gateway” ?! Which i a nutshell is solution which exposes HA loadbalancer ( and in future also DNS with connection to Consul ).
Raw and basic design could look as follow :
So what we have here are 2 hosts that will expose a VIP address. So nothing really edge cutting right 😀 And as recently I work a lot with Ubuntu following steps are geared towards that OS.
Installing:
installation is really plain simple. You can use APT to get package installed by running:
|
1 |
apt-get update && apt-get install keepalived |
And thats it for installation part 🙂 nothing like quick install 🙂
Configuring:
Configuration is something on which you can spend some more time tuning it to your needs. It does have a lot of options and I recommend just do a bit of reading. I will highlight here only bare metal basics to get you running. But complex scenarios are well possible.
Also you will notice that I do not use multicasting but switched to unicast
First thing which you want to configure is binding settings ( otherwise we want to be able to get solution running )
|
1 2 |
echo "net.ipv4.ip_nonlocal_bind=1" >> /etc/sysctl.conf sysctl -p |
Next we create configuration file for our service
|
1 |
vi /etc/init/keepalived.conf |
and once thats done you can paste contents of the snipet below
|
1 2 3 4 5 6 7 8 |
description "lb and ha service" start on runlevel [2345] stop on runlevel [!2345] respawn exec /usr/local/sbin/keepalived --dont-fork |
Once done we create config file /etc/keepalived/keepalived.conf ( on our first node )
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
vrrp_instance VI_1 { interface eth0 state MASTER virtual_router_id 91 priority 100 virtual_ipaddress { # YOUR VIP ADDRESS # } unicast_src_ip #YOUR-1st-NODE-ADDRESS unicast_peer { #YOUR-2nd-NODE-ADDRESS } } |
And on the other machine you place the same config but switch addresses in unicast source and peer
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
vrrp_instance VI_1 { interface eth0 state MASTER virtual_router_id 91 priority 100 virtual_ipaddress { # YOUR VIP ADDRESS # } unicast_src_ip #YOUR-2nd-NODE-ADDRESS unicast_peer { #YOUR-1st-NODE-ADDRESS } } |
More details on configuration you can find here >>> http://www.keepalived.org/LVS-NAT-Keepalived-HOWTO.html ( I have found this link to be full of useful information )
Bringing service to live:
Now you might say that I have configured both as masters. But in this case first one to be online will become master.
Now on both nodes you can execute
|
1 2 3 4 |
# Start service service keepalived start # show ip addresses ip addr show eth0 |
And on one you should see your VIP address being online. voilla! HA service running and operational
Testing Nginx:
Now time has come to test nginx. for purposes of this demo I have setup both machines to host docker container of nginx
Great! So both are listening on correct VIP address. One is displaying message “Hello from Nginx-1” and second “Hello from Nginx-2”. Lets test that from client machine …
Initial request from our client machine :
And let me know disable network interface on host-1 and once thats done we make request again
The error you see here is kinda my fault (but wanted to highlight this ) since my keepaliveD service was stopped on the host. once I started the service it responded from the other host.
Summary:
So now whats ur options ? Well quite a lot – as you can i.e. setup glusterFS and replicate your nginx config files / or use consul – explore consul template and use that for nginx dynamic files …
If you have any interesting use case scenarios leave them in comments!
