Keepalived是一个基于VRRP协议来实现的服务高可用方案, 具体的作用就不说了, 官网在这里
项目运作过程中, nginx常用来作为反向代理来配置用户访问的各项具体服务, 但如果它本身挂了的时候, 该怎么去通过一个高可用方案来实现多nginx之间的服务切换, 就是Keepalived需要做的事情了.
实现原理
假设有两台服务器192.168.20.2与192.168.20.3, 这两台服务器上各自部署了Keepalived服务, 并且这两个的Keepalived服务在配置文件中使用同一个虚拟IP(即VIP)
初始状态下,192.168.20.2被配置为Master服务, 192.168.20.3被配置为Backup服务, 当两个服务启动后, 默认情况下, VIP会绑定在Master所在主机的网卡上, 当用户访问VIP时, 实际访问的是Master所在的主机192.168.20.2, Master服务会通过VRRP协议与Backup服务联系, 以某种定时沟通的方式告诉Backup自己的"存活"状态, 如果在某个时间点, Backup服务没有收到Master的存活状态, 它则会认为Master服务所在的主机挂掉了, 它会把指定的VIP地址绑定到自己的网卡上, 如此一来, 用户通过VIP的访问流, 就切换到Backup所在主机了
案例
安装keepalived服务
以下命令在CentOS7环境下执行, 其他环境去看官网或者百度
1
2
3
4
|
yum install -y keepalived
# 设置开机启动
systemctl enable keepalived
|
Master节点(192.168.21.121)
Master节点的基础配置(/etc/keepalived/keepalived.conf)如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
# 标记当前主机为主节点
state MASTER
# 把虚拟IP添加到 ens33 网卡设备上
interface ens33
# 虚拟路由的id, 同一个Keepalived集群中该值必须相同, 否则会被识别为两个Keepalived集群
virtual_router_id 255
# 抢占模式下, 分数越高的自动被设置为Master节点
priority 100
# 通知其他Backup节点自己存活状态的时间间隔, 单位是秒
advert_int 1
# 连接keepalived服务集群的认证信息
authentication {
auth_type PASS
auth_pass 1111
}
# 虚拟IP的地址, 可以设置多个, 但是这里只设置一个, 就是这个IP会被绑定到"被激活"节点的网卡上
virtual_ipaddress {
192.168.21.119
}
}
|
启动keepalived服务
1
|
systemctl start keepalived
|
如果设置正确, keepalived的日志应该是类似这样子的(注意看日志中):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
9月 13 17:48:18 node-121 systemd[1]: Starting LVS and VRRP High Availability Monitor...
9月 13 17:48:18 node-121 Keepalived[8137]: Starting Healthcheck child process, pid=8138
9月 13 17:48:18 node-121 Keepalived[8137]: Starting VRRP child process, pid=8139
9月 13 17:48:18 node-121 systemd[1]: Started LVS and VRRP High Availability Monitor.
9月 13 17:48:18 node-121 Keepalived_healthcheckers[8138]: Initializing ipvs
9月 13 17:48:18 node-121 Keepalived_healthcheckers[8138]: Opening file '/etc/keepalived/keepalived.conf'.
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: Registering Kernel netlink reflector
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: Registering Kernel netlink command channel
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: Registering gratuitous ARP shared channel
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: Opening file '/etc/keepalived/keepalived.conf'.
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) removing protocol VIPs.
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: Using LinkWatch kernel netlink reflector...
9月 13 17:48:18 node-121 Keepalived_vrrp[8139]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
# 注意看日志中有一行:`Entering MASTER STATE`, 标志着当前Keepalived服务进入了Master状态
9月 13 17:48:19 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) Transition to MASTER STATE
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) Entering MASTER STATE
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) setting protocol VIPs.
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.21.119
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:20 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 17:48:25 node-121 Keepalived_vrrp[8139]: Sending gratuitous ARP on ens33 for 192.168.21.119
|
注意看日志中有一行:Entering MASTER STATE, 标志着当前Keepalived服务进入了Master状态
此时查看网卡信息, 输出的结果应该是这样的
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
[root@node-121 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:ed:ac:c8 brd ff:ff:ff:ff:ff:ff
inet 192.168.21.121/23 brd 192.168.21.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
# 下面这一行, 就可以看出来虚拟IP已经成功的绑定到当前主机上的ens33网卡了
inet 192.168.21.119/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::86fd:580d:b0ba:f474/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4699:150c:3a51:5226/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::b600:f95d:39e6:ee4e/64 scope link noprefixroute
valid_lft forever preferred_lft foreve
|
Backup节点(192.168.21.122)
Master节点的基础配置(/etc/keepalived/keepalived.conf)如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
# 标记当前主机为备节点
state BACKUP
interface ens33
# 注意这个值, 只有与Master保持相同, 才被当做备用节点加入Master中的keepalived集群
virtual_router_id 255
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.21.119
}
}
|
启动keepalived服务
1
|
systemctl start keepalived
|
如果设置正确, keepalived的日志应该是类似这样子的:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
9月 13 17:51:08 node-122 systemd[1]: Starting LVS and VRRP High Availability Monitor...
9月 13 17:51:08 node-122 Keepalived[1935]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
9月 13 17:51:08 node-122 Keepalived[1935]: Opening file '/etc/keepalived/keepalived.conf'.
9月 13 17:51:08 node-122 Keepalived[1936]: Starting Healthcheck child process, pid=1937
9月 13 17:51:08 node-122 Keepalived[1936]: Starting VRRP child process, pid=1938
9月 13 17:51:08 node-122 Keepalived_healthcheckers[1937]: Initializing ipvs
9月 13 17:51:08 node-122 Keepalived_healthcheckers[1937]: Opening file '/etc/keepalived/keepalived.conf'.
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: Registering Kernel netlink reflector
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: Registering Kernel netlink command channel
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: Registering gratuitous ARP shared channel
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: Opening file '/etc/keepalived/keepalived.conf'.
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) removing protocol VIPs.
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: Using LinkWatch kernel netlink reflector...
# 注意看日志中有一行:`Entering BACKUP STATE`, 标志着当前Keepalived服务进入了Master状态
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) Entering BACKUP STATE
9月 13 17:51:08 node-122 Keepalived_vrrp[1938]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
9月 13 17:51:08 node-122 systemd[1]: Started LVS and VRRP High Availability Monitor.
|
注意看日志中有一行:Entering BACKUP STATE, 标志着当前Keepalived服务进入了 BACKUP 状态
再看一下当前主机的网卡信息, 正常情况下, 网卡信息中是不会出现虚拟IP的, 如果出现了虚拟IP, 一定是配置文件中的virtual_router_id字段值与Master中的不一致,导致被识别为另一个keepalived集群
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@node-122 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:e1:f8:00 brd ff:ff:ff:ff:ff:ff
inet 192.168.21.122/23 brd 192.168.21.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::86fd:580d:b0ba:f474/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4699:150c:3a51:5226/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::b600:f95d:39e6:ee4e/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft fore
|
测试
注意看Master与Backup的配置信息中,使用的虚拟IP与真实主机IP是在同一个网段内的, 现在通过同一网段内的其他主机访问虚拟IP的结果如下:
1
2
3
4
5
6
7
8
9
10
|
# ping 192.168.21.119
PING 192.168.21.119 (192.168.21.119): 56 data bytes
64 bytes from 192.168.21.119: icmp_seq=0 ttl=64 time=10.553 ms
64 bytes from 192.168.21.119: icmp_seq=1 ttl=64 time=9.187 ms
64 bytes from 192.168.21.119: icmp_seq=2 ttl=64 time=8.319 ms
64 bytes from 192.168.21.119: icmp_seq=3 ttl=64 time=8.633 ms
^C
--- 192.168.21.119 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 8.319/9.173/10.553/0.855 ms
|
可以看到虚拟IP是可以正常访问的
根据前面说的Keepalived原理, 当Master上的Keepalived服务由于某种原因没有通知Backup时, Backup会把虚拟IP绑定到自己的网卡上, 可以通过停用Master的Keepalived服务来模拟实现, 并通过ping命令来验证虚拟IP是否切换成功
1
|
systemctl stop keepalived
|
在Master上停用keepalived服务后, 会发现ping的结果有如下变化:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# ping 192.168.21.119
64 bytes from 192.168.21.119: icmp_seq=22 ttl=64 time=2.403 ms
64 bytes from 192.168.21.119: icmp_seq=23 ttl=64 time=2.276 ms
64 bytes from 192.168.21.119: icmp_seq=24 ttl=64 time=2.373 ms
64 bytes from 192.168.21.119: icmp_seq=25 ttl=64 time=2.093 ms
# 注意这里,由于停止了Master上的keepalived服务, 虚拟IP出现了一定时间内的断层
Request timeout for icmp_seq 26
Request timeout for icmp_seq 27
64 bytes from 192.168.21.119: icmp_seq=28 ttl=64 time=2.742 ms
64 bytes from 192.168.21.119: icmp_seq=29 ttl=64 time=2.338 ms
64 bytes from 192.168.21.119: icmp_seq=30 ttl=64 time=2.188 ms
64 bytes from 192.168.21.119: icmp_seq=31 ttl=64 time=2.205 ms
64 bytes from 192.168.21.119: icmp_seq=32 ttl=64 time=2.242 ms
|
且Backup所在的Keepalived服务日志有如下提示:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
# 注意, 这里的的日志显示, Backup上的Keepalived服务已经切换到Master状态了
9月 13 18:22:02 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) Transition to MASTER STATE
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) Entering MASTER STATE
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) setting protocol VIPs.
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.21.119
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:03 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.119
9月 13 18:22:08 node-122 Keepalived_vrrp[1938]: Sending gratuitous ARP on ens33 for 192.168.21.11
|
再来看一下BackUp所在主机的网卡信息
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
[root@node-122 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:e1:f8:00 brd ff:ff:ff:ff:ff:ff
inet 192.168.21.122/23 brd 192.168.21.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
# 注意看, 虚拟IP已经被绑定到Backup所在的网卡ens33上了
inet 192.168.21.119/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::86fd:580d:b0ba:f474/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4699:150c:3a51:5226/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::b600:f95d:39e6:ee4e/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
|
此时再回头来看看Master上的网卡信息:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@node-121 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:ed:ac:c8 brd ff:ff:ff:ff:ff:ff
inet 192.168.21.121/23 brd 192.168.21.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::86fd:580d:b0ba:f474/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::4699:150c:3a51:5226/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::b600:f95d:39e6:ee4e/64 scope link noprefixroute
valid_lft forever preferred_lft forever
|
可以看到虚拟IP的信息已经被取消掉了(如果这里依然有虚拟IP的信息, 那肯定是某些地方出问题了)
另外
本文中,仅仅通过ping命令来确定keepalived服务的切换状态, 还可以通过在两个服务器上分别部署一套nginx服务, 来模拟访问web页面, 来验证keepalived服务的可用性, 这里不再说了