KeepAlived使用教程
[TOC]
KeepAlived使用教程
核心配置文件说明
1 | global_defs { |
keepalived问题
keepalived组播防火墙问题
防火墙开启keepalived组播
组播ip默认是:
vrrp_mcast_group4
配置内容,默认224.0.0.18centos7
1
2
3
4
5
6firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
firewall-cmd --reload
--in-interface ens33为网卡名称
--destination 224.0.0.18为组播ip
KeeyAlived安装
双机主备
主机配置情况
虚拟ip:192.168.231.130
网卡:ens33
主机 角色 192.168.231.131 MASTER 192.168.231.132 BACKUP
配置环境变量
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18配置文件目录
export keepalived_dir=/home/keepalived
keepalived脚本目录
export keepalived_script=${keepalived_dir}/script
虚拟ip
export keepalived_vrrp_ip=192.168.231.130
认证密码
export keepalived_pass=keepalived
虚拟路由id
export keepalived_virtual_id=130
vrrp名称
export keepalived_vrrp_name=VI_1
keepalived的容器名
export keepalived_docker_name=keepalived
创建keepalived目录
mkdir -p ${keepalived_dir}
创建keepalived脚本目录
mkdir -p ${keepalived_script}创建配置文件
192.168.231.131
1
2
3
4
5
6
7
8
9
10
11主机网卡名称
export keepalived_interface=ens33
节点状态(节点为MASTER,备份节点为BACKUP)
export keepalived_state=MASTER
主机ip
export keepalived_ip=192.168.231.131
路由id
export keepalived_router_id=131
优先级配置(0-254的值)
export keepalived_priority=100
执行创建配置文件命令192.168.231.132
1
2
3
4
5
6
7
8
9
10
11主机网卡名称
export keepalived_interface=ens33
节点状态(节点为MASTER,备份节点为BACKUP)
export keepalived_state=BACKUP
主机ip
export keepalived_ip=192.168.231.132
路由id
export keepalived_router_id=132
优先级配置(0-254的值)
export keepalived_priority=80
执行创建配置文件命令创建配置文件命令
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30创建配置文件
${keepalived_dir}/keepalived.conf<<EOF
global_defs {
# 路由id:当前安装keepalived的节点主机标识符,保证全局唯一
router_id ${keepalived_router_id}
## 修改执行脚本的用户为root
script_user root
enable_script_security
## 默认是224.0.0.18
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance ${keepalived_vrrp_name} {
state ${keepalived_state} ## 主节点为MASTER,备份节点为BACKUP
interface ${keepalived_interface} ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id ${keepalived_virtual_id} ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip ${keepalived_ip} ## 本机ip地址
priority ${keepalived_priority} ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass ${keepalived_pass} ## 认证密码
}
virtual_ipaddress {
${keepalived_vrrp_ip} ## 虚拟ip,可以指定多个
}
}
EOF启动keepalived
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21${keepalived_dir}/run.sh<<EOF
docker run -d --net=host \\
--privileged \\
--name ${keepalived_docker_name} \\
--user root \\
--restart=always \\
-v /etc/localtime:/etc/localtime:ro \\
-v /lib/modules:/lib/modules \\
-v /var/run/docker.sock:/var/run/docker.sock \\
-v $(which docker):/usr/bin/docker \\
-v /etc/docker:/etc/docker \\
-v ${keepalived_dir}/keepalived.conf:/usr/local/etc/keepalived/keepalived.conf \\
-v ${keepalived_script}:/etc/keepalived/script \\
osixia/keepalived --copy-service
EOF
chmod +x ${keepalived_dir}/run.sh
sh ${keepalived_dir}/run.sh
容器操作宿主机docker命令
docker exec ${keepalived_docker_name} bash -c 'mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2'
双主热备
双机热备的这种主从模式的缺点是无论部署了多少个KeeyAlived主机,也就只能有一台主机在工作.然而双主热备的模式则会让所用的KeeyAlived主机工作
安装方式是在双机热备的基础上进行修改
主机配置情况
虚拟ip:192.168.231.130 , 192.168.231.129
网卡:ens33
主机 192.168.231.131 192.168.231.132
配置通用环境变量
1
2
3
4
5
6设置新的vrrp名称
export keepalived_vrrp_name=VI_2
虚拟ip
export keepalived_vrrp_ip=192.168.231.129
虚拟路由id
export keepalived_virtual_id=129修改配置文件
192.168.231.131
1
2
3
4
5
6
7
8
9主机网卡名称
export keepalived_interface=ens33
节点状态(节点为MASTER,备份节点为BACKUP)
export keepalived_state=BACKUP
主机ip
export keepalived_ip=192.168.231.131
优先级配置(0-254的值)
export keepalived_priority=80
执行添加配置文件命令192.168.231.132
1
2
3
4
5
6
7
8
9主机网卡名称
export keepalived_interface=ens33
节点状态(节点为MASTER,备份节点为BACKUP)
export keepalived_state=MASTER
主机ip
export keepalived_ip=192.168.231.132
优先级配置(0-254的值)
export keepalived_priority=100
执行添加配置文件命令添加配置文件内容命令
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20添加配置文件内容
>${keepalived_dir}/keepalived.conf<<EOF
vrrp_instance ${keepalived_vrrp_name} {
state ${keepalived_state} ## 主节点为MASTER,备份节点为BACKUP
interface ${keepalived_interface} ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id ${keepalived_virtual_id} ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip ${keepalived_ip} ## 本机ip地址
priority ${keepalived_priority} ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass ${keepalived_pass} ## 认证密码
}
virtual_ipaddress {
${keepalived_vrrp_ip} ## 虚拟ip,可以指定多个
}
}
EOF查看完整的配置文件信息
192.168.231.131
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51global_defs {
# 路由id:当前安装keepalived的节点主机标识符,保证全局唯一
router_id 131
## 修改执行脚本的用户为root
script_user root
enable_script_security
## 默认是224.0.0.18
vrrp_mcast_group4 224.0.0.18
}
vrrp_script check_nginx { ## 定义脚本文件信息
script "sh /etc/keepalived/script/check_nginx.sh" ## 脚本文件位置
interval 2 ##每隔两秒运行上一行脚本
weight -10 ## 如果脚本运行失败,则降级权重-10
}
vrrp_instance VI_1 {
state MASTER ## 主节点为MASTER,备份节点为BACKUP
interface ens33 ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id 130 ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip 192.168.231.131 ## 本机ip地址
priority 100 ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass keepalived ## 认证密码
}
virtual_ipaddress {
192.168.231.130 ## 虚拟ip,可以指定多个
}
track_script { ## 运行脚本
check_nginx ## 指定要运行的脚本名称
}
}
vrrp_instance VI_2 {
state BACKUP ## 主节点为MASTER,备份节点为BACKUP
interface ens33 ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id 129 ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip 192.168.231.131 ## 本机ip地址
priority 80 ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass keepalived ## 认证密码
}
virtual_ipaddress {
192.168.231.129 ## 虚拟ip,可以指定多个
}
}192.168.231.132
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52global_defs {
# 路由id:当前安装keepalived的节点主机标识符,保证全局唯一
router_id 132
## 修改执行脚本的用户为root
script_user root
enable_script_security
## 默认是224.0.0.18
vrrp_mcast_group4 224.0.0.18
}
vrrp_script check_nginx { ## 定义脚本文件信息
script "sh /etc/keepalived/script/check_nginx.sh" ## 脚本文件位置
interval 2 ##每隔两秒运行上一行脚本
weight -10 ## 如果脚本运行失败,则降级权重-10
}
vrrp_instance VI_1 {
state BACKUP ## 主节点为MASTER,备份节点为BACKUP
interface ens33 ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id 130 ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip 192.168.231.132 ## 本机ip地址
priority 80 ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass keepalived ## 认证密码
}
virtual_ipaddress {
192.168.231.130 ## 虚拟ip,可以指定多个
}
track_script { ## 运行脚本
check_nginx ## 指定要运行的脚本名称
}
}
vrrp_instance VI_2 {
state MASTER ## 主节点为MASTER,备份节点为BACKUP
interface ens33 ## 绑定虚拟IP的网络接口(网卡),与本机IP地址所在的网络接口相同(服务器主机网卡)
virtual_router_id 129 ## 虚拟路由ID号(主备节点一定要相同)
mcast_src_ip 192.168.231.132 ## 本机ip地址
priority 100 ##优先级配置(0-254的值)
advert_int 1 ## 组播信息发送间隔,俩个节点必须配置一致,默认1s
authentication { ## 认证匹配
auth_type PASS ## 认证方式:PASS
auth_pass keepalived ## 认证密码
}
virtual_ipaddress {
192.168.231.129 ## 虚拟ip,可以指定多个
}
}重启keepalived
1
docker restart ${keepalived_docker_name}
KeeyAlived执行定时脚本
定时检测容器存活
keepalived.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21global_defs {
...
## 修改执行脚本的用户为root
script_user root
enable_script_security
}
vrrp_script check_nginx { ## 定义脚本文件信息
script "sh /etc/keepalived/script/check_nginx.sh" ## 脚本文件位置
interval 2 ##每隔两秒运行上一行脚本
weight -10 ## 如果脚本运行失败,则降级权重-10
}
vrrp_instance VI_1 { ## vrrp名称
...
track_script { ## 运行脚本
check_nginx ## 指定要运行的脚本名称
}
}check_nginx.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28脚本文件名
export keepalived_script_name=check_nginx.sh
要检测的容器名
export keepalived_script_docker=nginx
创建配置文件
${keepalived_script}/${keepalived_script_name}<<EOF
!/bin/bash
A=\`docker ps -f name=${keepalived_script_docker} | grep ${keepalived_script_docker} | wc -l\`
B=\`docker ps -af name=${keepalived_script_docker} | grep ${keepalived_script_docker} | wc -l\`
判断对应的docker容器是否存活
if [ \$A -eq 0 ];then
if [ \$B -eq 1 ];then
docker start ${keepalived_script_docker}
sleep 3
fi
if [ \`docker ps -f name=${keepalived_script_docker} | grep ${keepalived_script_docker} | wc -l\` -eq 0 ];then
docker stop ${keepalived_docker_name}
fi
fi
EOF
设置脚本文件权限
chmod +x ${keepalived_script}/${keepalived_script_name}
安装LVS
LVS文章:https://wsgzao.github.io/post/lvs-keepalived/
主机配置情况
虚拟ip:192.168.231.128
网卡:ens33
主机 角色 192.168.231.133 MASTER 192.168.231.134 BACKUP 安装ipvsadm
1
yum install ipvsadm
配置文件
先按照双机主备或者双主热备的方式安装keepalived
192.168.231.133,192.168.231.134
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52访问lvs的端口号
export lvs_port=80
健康检查的时间间隔,秒
export lvs_loop=10
LVS调度算法:rr|wrr|lc|wlc|lblc|sh|dh
export lvs_argo=wlc
添加lvs集群基础内容
>${keepalived_dir}/keepalived.conf<<EOF
virtual_server ${keepalived_vrrp_ip} ${lvs_port} {
delay_loop ${lvs_loop} # 健康检查的时间间隔,秒
lb_algo ${lvs_argo} # LVS调度算法:rr|wrr|lc|wlc|lblc|sh|dh
lb_kind DR # LVS模式:NAT|DR|TUN
persistence_timeout 300 # 持久化超时时间,单位是秒。默认是6分钟。
persistence_granularity 255.255.255.255 # 子网掩码,默认255.255.255.255
protocol TCP # 网络协议:TCP|UDP|SCTP
}
EOF
添加负载均衡的服务器ip地址和端口
virtual_server 192.168.231.128 80 {
...
负载均衡的服务器ip地址和端口
real_server 192.168.231.131 80 { # real_server ip prot
weight 1 # 给服务器指定权重。默认是1.
TCP_CHECK {
connect_port 80 # 检查的端口
connect_timeout 3 # 检查超时时间(秒)
retry 2 # 重试次数
delay_before_retry 3 # 间隔时间(秒)
}
# notify_up <STRING> # 当服务器健康检查成功时,执行的脚本。
# notify_down <STRING> # 当服务器健康检查失败时,执行的脚本。
# uthreshold <INT> # 到这台服务器的最大连接数。
# lthreshold <INT> # 到这台服务器的最小连接数。
}
real_server 192.168.231.132 80 { # real_server ip prot
weight 1 # 给服务器指定权重。默认是1.
TCP_CHECK {
connect_port 80 # 检查的端口
connect_timeout 3 # 检查超时时间(秒)
retry 2 # 重试次数
delay_before_retry 3 # 间隔时间(秒)
}
# notify_up <STRING> # 当服务器健康检查成功时,执行的脚本。
# notify_down <STRING> # 当服务器健康检查失败时,执行的脚本。
# uthreshold <INT> # 到这台服务器的最大连接数。
# lthreshold <INT> # 到这台服务器的最小连接数。
}
}real_server的服务器配置lo虚拟ip
192.168.231.131 , 192.168.231.132
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41lo网卡名称
export lo_name=ifcfg-lo
新的lo网卡文件名
export lo_new_filename=${lo_name}:0
新的lo网卡名称
export lo_new_name=lo:0
lvs虚拟ip
export keepalived_vrrp_ip=192.168.231.128
安装工具
yum install net-tools
添加 TYPE=Loopback
echo "TYPE=Loopback" >> /etc/sysconfig/network-scripts/${lo_name}
添加新的lo网卡 ifcfg-lo:0
cat > /etc/sysconfig/network-scripts/${lo_new_filename} << EOF
DEVICE=${lo_new_name}
IPADDR=${keepalived_vrrp_ip}
NETMASK=255.255.255.255
ONBOOT=yes
EOF
加载新的网卡
ifup ${lo_new_name}
查看ip是否配置成功
ip addr
添加虚拟ip到host,重启系统会丢失
route add -host ${keepalived_vrrp_ip} dev ${lo_new_name}
查看是否生效
route -n
添加到开机执行命令
echo "route add -host ${keepalived_vrrp_ip} dev ${lo_new_name}" >> /etc/rc.local
查看添加是否成功
cat /etc/rc.localreal_server的服务器配置ARP
192.168.231.131 , 192.168.231.132
在
/etc/sysctl.conf
添加一下内容1
2
3
4
5
6
7
8
9
10
11打开文件
vi /etc/sysctl.conf
configration for lvs
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.ip_forward = 1执行
sysctl -p
让配置生效
LVS命令
命令文档:https://blog.51cto.com/10978134/2122118
命令 | 作用 |
---|---|
ipvsadm -Ln |
查看集群列表 |
ipvsadm -Ln --stats |
查看集群状态 |
ipvsadm -Ln --persistent-conn |
查看持久化连接 |
ipvsadm -Lnc |
查看连接请求过期时间以及请求源ip和目标ip |
ipvsadm -Ln --timeout |
查看过期时间 |
ipvsadm -h |
帮助文档 |
man ipvsadm |
详细帮助文档 |
ipvsadm -C |
清空集群配置 |