快速入门-Keepalived+Nginx-HA高可用-不啰嗦-直接上-边实践边学

off999 2025-01-21 20:38 59 浏览 0 评论

HI，朋友们，不啰嗦系列又更新了，今天的主要内容是Nginx，Nginx作为大家熟知的中间件，在很多业务场景中，或者说业务规模不大的场景中，一般我们都不去考虑所谓的HA和负载均衡，因为其自身的稳定性还是非常优异的，还没有见过单机版的Nginx崩溃，当然作为ITer的我们还是需要掌握其HA方案，今天就介绍下Keepalived + Nginx的HA高可用方案。其本质就是Keepalive虚拟出一个IP，可以实现IP漂移来保证一主一备的Nginx正常工作。

今天参与实验的主机有

192.168.31.165 应用Keepalived、Nginx
192.168.31.175 应用Keepalived、Nginx
192.168.31.155 web程序Python+Flask
192.168.31.185 web程序Python+Flask
192.168.31.195 模拟客户端访问
使用到的域名 x.little_nb.com y.little_nb.com
虚拟ip地址 192.168.31.200

今天实验的架构图如下：

今天的应用都是运行在容器中，首先我们先来安装docker

使用阿里云的源

yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum -y install docker-ce

systemctl enable docker

systemctl start docker

docker安装好以后我们来安装nginx，这里我为了快速，没有使用compose，直接命令创建的docker。

docker pull nginx 没有指定版本默认latest，等待镜像拉取完成

docker run -d \

--name nginx \

-v /data/nginx/html:/usr/share/nginx/html \

-v /data/conf/nginx.conf:/etc/nginx/nginx.conf \

-p 9000:80 \

-p 9001:9001 \

nginx:latest

直接复制创建，html文件随便创建一个，本地需要创建/data/nginx/html文件和文件夹 /data/nginx/nginx.conf文件和文件夹，/data/nginx/log文件夹，有日志可以配置到此路径，不过没有暂时没有映射。

在html文件夹下创建 index.html

内容如下

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

<style>

body {

width: 35em;

margin: 0 auto;

font-family: Tahoma, Verdana, Arial, sans-serif;

}

</style>

</head>

<body>

<h1>Welcome to nginx!</h1>

</body>

</html>

在conf文件夹下面创建nginx.conf

user root;

worker_processes 1;

events {

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$http_x_forwarded_for"';

sendfile on;

keepalive_timeout 65;

upstream callback {

#ip_hash;

server 192.168.31.185:8331 ;

server 192.168.31.155:8331 ;

}

server {

listen 80;

server_name x.little_nb.com;

location / {

proxy_pass http://callback;

root /usr/share/nginx/html;

index index.html index.htm;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root /home/nginx-demo/html;

}

server {

listen 9001;

server_name y.little_nb.com;

location / {

root /usr/share/nginx/html;

index index.html index.htm;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root /home/nginx-demo/html;

}

格式大家自己调一下

9000端口是业务端口，9001是测试端口

以上nginx安装31.165、31.175同样的步骤。

接下来就是keepalived安装，docker pull keepalived,拉去keepalived的最新镜像

然后要创建一些文件和文件夹

/data/keepalived/keepalived.conf

/data/keepalived/nginx_check.sh

/data/keepalived/sendMail.py

31.175上keepalived.conf文件内容这是MASTER角色

! Configuration File for keepalived

vrrp_script chk_nginx {

script "/usr/local/etc/keepalived/nginx_check.sh"

interval 1

weight -2

}

vrrp_instance VI_1 {

state MASTER

interface enp12s0

virtual_router_id 120

mcast_src_ip 192.168.31.175

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.31.200/24

}

notify_master "python3 /usr/local/etc/keepalived/sendMail.py master"

notify_backup "python3 /usr/local/etc/keepalived/sendMail.py backup"

notify_fault "python3 /usr/local/etc/keepalived/sendMail.py fault"

}

31.165上的keepalived.conf文件内容这是BACKUP角色

! Configuration File for keepalived

vrrp_script chk_nginx {

script "/usr/local/etc/keepalived/nginx_check.sh" #nginx检测脚本

interval 1 #平均检测时间

weight -2 #失败就降权重，优先级降低，虚拟Vip转移

}

vrrp_instance VI_1 {

state BAKCUP #定义你的角色

interface enp5s0 #匹配你自己网卡名称

virtual_router_id 120 #router id一直组队需要

mcast_src_ip 192.168.31.165 #本机iP

priority 99 #优先级定义

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.31.200/24 #想定的虚拟IP

}

notify_master "python3 /usr/local/etc/keepalived/sendMail.py master"

notify_backup "python3 /usr/local/etc/keepalived/sendMail.py backup"

notify_fault "python3 /usr/local/etc/keepalived/sendMail.py fault"

}

31.165的sendMail.py 脚本

#!/usr/bin/env python

# -*- coding: UTF-8 -*-

import os

import sys

import socket

import smtplib

from email.mime.text import MIMEText

def SendMail(argv):

if len(argv) < 2:

print ('Error: parameter invalid!')

return

state = argv[1]

ip = '192.168.31.165'

subject = "keepalived: %s change to state: %s" % (ip, state)

conent = subject

message = MIMEText(conent,"plan","utf-8")

sender = "abc@163.com"

sender_pwd = "替换成你的授权码"

recver = "456@qq.com"

message['Subject'] = subject

message['To'] = recver

message['From'] = sender

smtp = smtplib.SMTP_SSL("smtp.163.com")

smtp.connect("smtp.163.com",465)

smtp.login(sender,sender_pwd)

smtp.sendmail(sender,[recver],message.as_string())

smtp.close()

if __name__ == "__main__":

SendMail(sys.argv)

31.175的sendMail.py脚本

#!/usr/bin/env python

# -*- coding: UTF-8 -*-

import os

import sys

import socket

import smtplib

from email.mime.text import MIMEText

def SendMail(argv):

if len(argv) < 2:

print ('Error: parameter invalid!')

return

state = argv[1]

ip = '192.168.31.175'

subject = "keepalived: %s change to state: %s" % (ip, state)

conent = subject

message = MIMEText(conent,"plan","utf-8")

sender = "abc@163.com"

sender_pwd = "替换成你的授权码"

recver = "456@qq.com"

message['Subject'] = subject

message['To'] = recver

message['From'] = sender

smtp = smtplib.SMTP_SSL("smtp.163.com")

smtp.connect("smtp.163.com",465)

smtp.login(sender,sender_pwd)

smtp.sendmail(sender,[recver],message.as_string())

smtp.close()

if __name__ == "__main__":

SendMail(sys.argv)

31.165的nginx_check.sh

#!/bin/bash

if [ "$(curl -sL -w '%{http_code}' http://192.168.31.165:9001 -o /dev/null)" = "200" ]; then

echo "Success"

exit 0

else

echo "Fail"

exit 1

31.175的nginx_check.sh

if [ "$(curl -sL -w '%{http_code}' http://192.168.31.175:9001 -o /dev/null)" = "200" ]; then

echo "Success"

exit 0

else

echo "Fail"

exit 1

docker run --cap-add=NET_ADMIN --cap-add=NET_BROADCAST --cap-add=NET_RAW --net=host -d --name keepalived \

-v /etc/localtime:/etc/localtime \

-v /data/keepalived/keepalived.conf:/usr/local/etc/keepalived/keepalived.conf \

-v /data/keepalived/nginx_check.sh:/usr/local/etc/keepalived/nginx_check.sh \

-v /data/keepalived/sendMail.py:/usr/local/etc/keepalived/sendMail.py \

osixia/keepalived:latest

docker 跑起来

这个高可用的原理就是原本挂载nginx的192.168.31.165和31.175依靠keepalived虚拟出来的31.200来达到一边故障另外一台主机可以继续工作，不管是keepalive还是其所在宿主机或者nginx服务失败，都会使31.200漂移到另外一台机器，以保证nginx指向的后端31.155和31.185能够继续给前端提供服务

keepalived开启后 ip地址会出现在已有网卡下面

首先我从195访问后端本地host做了解析，我用的域名，这个和nginx.conf配置是对应的

curl x.little_nb.com:9000

测试四表五链 (返回)

keepalived目前虚拟ip是挂载在31.175，因为195访问域名等于nginx做解析31.200，后端显示来访IP是31.175，证明目前31.200挂载在31.175上面

将175上的nginx关闭，模拟nginx故障

这个时候会发生两件事，第一,175上的nginx检测脚本会生效，keepalived优先级会降级，

175上的keepalived降级

虚拟ip地址31.200会迁移到31.165上去

31.165上接收到了提示，说175降级到了98，我们是99.重新选举，然后31.200挂载在31.165上

第二，31.165上的邮件通知会发出来，提示keepalived角色状态发生改变

这个时候在从195的客户端访问域名解析到后端，就应该是31.165上的nginx在作用了