在大型网站建设中,我们通常会用一些负载均衡技术(LVS、Nginx、Haproxy),将请求分发到后端的服务集群中。
这时,负载均衡均衡节点就成为单点故障的节点,为了保证系统的高可用,可以引入keepalived,将多个负载均衡节点联合起来作为一个整体对外服务,从而防止单点故障。
keepalived是以VRRP协议为基础实现的,VRRP全称Virtual Router Redundancy Protocol,是一种路由容错协议,也可以叫做备份路由协议。一组VRRP路由器协同工作,共同构成一台虚拟路由器。该虚拟路由器对外表现为一个具有MAC地址和唯一固定IP地址的逻辑路由器。实际工作时,这一组路由器中的一个作为master节点,其余作为backup节点。工作期间,master会定时发送VRRP协议包,当master节点出现故障,backup节点收不到VRRP包,其中一个backup节点(priority大的)会自动成为master节点,继续提供服务,此过程完全自动,不需要人工干预。
keepalived利用这个原理,将不同的服务器节点(负载均衡节点、web服务器)组件成一个服务器组,对外提供统一的(VIP)地址,从而保证系统高可用。
$ mkdir -p /opt/k8s/keepalive-haproxy
$ cd /opt/k8s/keepalive-haproxy
$ wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz
$ tar -xzvf keepalived-2.0.20.tar.gz
$ cd /opt/k8s/keepalive-haproxy/keepalived-2.0.20
# 执行configure,prefix指定安装目录
$ mkdir -p /opt/k8s/keepalived/
$ ./configure --prefix=/opt/k8s/keepalived/
$ make & make install
# 查看安装目录
$ ls /opt/k8s/keepalived/
bin etc sbin share
下面来以keepalived结合tomcat来实现一个web服务器的双机热备。网络topology如下:
配置主节点的keepalived
$ mkdir /opt/k8s/keepalived/conf
$ cd /opt/k8s/keepalived/conf
$ cat > keepalived.conf<<EOF
global_defs {
router_id web1
}
vrrp_instance VI_1 {
state MASTER #设置为主服务器
interface wlo1 #监测网络接口
virtual_router_id 55 #主、备必须一样
priority 200 #(主、备机取不同的优先级,主机值较大,备份机值较小,值越大优先级越高)
advert_int 1 #VRRP Multicast广播周期秒数
authentication {
auth_type PASS #VRRP认证方式,主备必须一致
auth_pass 1111 #(密码)
}
virtual_ipaddress {
192.168.0.100/24 #VRRP HA虚拟地址
}
}
EOF
配置备用节点的keepalived
$ mkdir /opt/k8s/keepalived/conf
$ cd /opt/k8s/keepalived/conf
$ cat > keepalived.conf<<EOF
global_defs {
router_id web2
}
vrrp_instance VI_1 {
state BACKUP #设置为主服务器
interface wlp3s0 #监测网络接口
virtual_router_id 55 #主、备必须一样
priority 100 #(主、备机取不同的优先级,主机值较大,备份机值较小,值越大优先级越高)
advert_int 1 #VRRP Multicast广播周期秒数
authentication {
auth_type PASS #VRRP认证方式,主备必须一致
auth_pass 1111 #(密码)
}
virtual_ipaddress {
192.168.0.100/24 #VRRP HA虚拟地址
}
}
EOF
在主备节点上启动keepalived服务
$ /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf
查看日志
主节点(ubuntu系统,在syslog中)
$ tail -f /var/log/syslog | grep Keepalived
Mar 13 17:21:42 slave Keepalived[11843]: Starting Keepalived v2.0.20 (01/22,2020)
Mar 13 17:21:42 slave Keepalived[11843]: Running on Linux 5.3.0-28-generic #30~18.04.1-Ubuntu SMP Fri Jan 17 06:14:09 UTC 2020 (built for Linux 4.15.18)
Mar 13 17:21:42 slave Keepalived[11843]: Command line: '/opt/k8s/keepalived/sbin/keepalived' '-f' '/opt/k8s/keepalived/conf/keepalived.conf'
Mar 13 17:21:42 slave Keepalived[11843]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:21:42 slave Keepalived[11843]: Remove a zombie pid file /run/keepalived.pid
Mar 13 17:21:42 slave Keepalived[11844]: Starting VRRP child process, pid=11845
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering Kernel netlink reflector
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering Kernel netlink command channel
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering gratuitous ARP shared channel
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: (VI_1) Entering BACKUP STATE (init)
Mar 13 17:21:45 slave Keepalived_vrrp[11845]: (VI_1) Entering MASTER STATE
从节点
$ tail -f /var/log/syslog | grep Keepalived
Mar 13 17:24:30 master Keepalived[3118]: Starting Keepalived v2.0.20 (01/22,2020)
Mar 13 17:24:30 master Keepalived[3118]: Running on Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 (built for Linux 4.15.18)
Mar 13 17:24:30 master Keepalived[3118]: Command line: '/opt/k8s/keepalived/sbin/keepalived' '-f' '/opt/k8s/keepalived/conf/keepalived.conf'
Mar 13 17:24:30 master Keepalived[3118]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:24:30 master Keepalived[3118]: Remove a zombie pid file /run/keepalived.pid
Mar 13 17:24:30 master Keepalived[3120]: Starting VRRP child process, pid=3122
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering Kernel netlink reflector
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering Kernel netlink command channel
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering gratuitous ARP shared channel
Mar 13 17:24:30 master Keepalived_vrrp[3122]: (VI_1) Entering BACKUP STATE (init)
查看主节点网路设备
$ ip addr show wlo1
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether c4:8e:8f:d0:33:89 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.114/24 brd 192.168.0.255 scope global dynamic noprefixroute wlo1
valid_lft 4948sec preferred_lft 4948sec
inet 192.168.0.100/24 scope global secondary wlo1
valid_lft forever preferred_lft forever
inet6 fe80::db1e:449b:5431:7392/64 scope link noprefixroute
valid_lft forever preferred_lft forever
部署web服务
基于spring boot构建了一个应用,利用embeded-tomcat,里面提供了一个restful接口,对应暴露的端口设置成8086
@RequestMapping("/health")
public String health(HttpServletRequest request) {
return "OK";
}
在主备节点上都启动spring boot应用
通过虚拟IP访问
$ curl http://192.168.0.100:8086/health
OK
停掉主节点的keepalived进程
$ ps -elf | grep -e keepa -e PID
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
1 S root 11844 1 0 80 0 - 4188 ep_pol 17:21 ? 00:00:00 /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf
5 S root 11845 11844 0 80 0 - 4188 ep_pol 17:21 ? 00:00:00 /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf
0 S root 19389 16663 0 80 0 - 5383 pipe_w 18:06 pts/0 00:00:00 grep --color=auto -e keepa -e PID
$ kill -9 11844
查看主从节点网路设备
主节点
$ ip addr show wlo1
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether c4:8e:8f:d0:33:89 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.114/24 brd 192.168.0.255 scope global dynamic noprefixroute wlo1
valid_lft 5707sec preferred_lft 5707sec
inet6 fe80::db1e:449b:5431:7392/64 scope link noprefixroute
valid_lft forever preferred_lft forever
从节点
$ ip addr show wlp3s0
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether d0:c5:d3:57:73:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.107/24 brd 192.168.0.255 scope global dynamic noprefixroute wlp3s0
valid_lft 4597sec preferred_lft 4597sec
inet 192.168.0.100/24 scope global secondary wlp3s0
valid_lft forever preferred_lft forever
inet6 fe80::1fda:e90a:207a:67e4/64 scope link noprefixroute
valid_lft forever preferred_lft forever
通过虚拟IP访问
$ curl http://192.168.0.100:8086/health
OK
重新启动web1上的keepalived进程,再次查看主节点上的wlo1网卡,会发现虚拟IP又自动漂移回来
上一小节我们通过模拟keepalived进程down掉,验证了VIP自动迁移。可是当主节点上的keepalived功能正常,而只是web服务出现故障,按照之前的简单配置,VIP是不会自动迁移的,这时服务就变的不可访问了。解决方法是在keepalived的配置文件中追加vrrp_script
示例如下:
编写检测脚本
$ mkdir /opt/k8s/keepalived/script
$ cd /opt/k8s/keepalived/script
$ cat > checkproxy.sh<<EOF
#!/bin/bash
count = `ps aux | grep -v grep | grep haproxy | wc -l`
if [ $count > 0 ]; then
exit 0
else
exit 1
fi
在配置文件中追加检测
...
vrrp_script checkhaproxy
{
script "/home/checkproxy.sh"
interval 3
weight -150
}
vrrp_instance test
{
...
track_script
{
checkhaproxy
}
...
}
原文:https://www.cnblogs.com/gaofeng-henu/p/12488821.html