Redis Replication+Keepalived

发表于 更新于 分类于

概述

以下是测试环境:

系统环境 IP地址 节点类型
CentOS6.5 192.168.1.92 Master
CentOS6.5 192.168.1.93 Slave(用于备份服务器)
CentOS6.5 192.168.1.94 VIP
CentOS6.5 192.168.1.17 Client

Master和Slave都执行以下操作，参考文档进行安装：
https://redis.io/topics/quickstart

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ yum install ntp -y
$ ntpdate -u cn.pool.ntp.org
$ yum install gcc -y
$ wget http://download.redis.io/redis-stable.tar.gz
$ tar xvzf redis-stable.tar.gz
$ cd redis-stable
$ make
$ yum install tcl -y
$ make test
$ cp src/redis-server /usr/local/bin/
$ cp src/redis-cli /usr/local/bin/

$ mkdir /etc/redis
$ mkdir /var/redis
$ cp utils/redis_init_script /etc/init.d/redis_6379
$ cp redis.conf /etc/redis/6379.conf
$ mkdir /var/redis/6379

编辑配置文件，确保执行了以下的更改，测试环境可以不设置密码：
[root@localhost redis-stable]# vim /etc/redis/6379.conf

  1. Set daemonize to yes (by default it is set to no).
  2. Set the pidfile to /var/run/redis_6379.pid (modify the port if needed).
  3. Change the port accordingly. In our example it is not needed as the default port is already 6379.
  4. Set your preferred loglevel.
  5. Set the logfile to /var/log/redis_6379.log
  6. Set the dir to /var/redis/6379 (very important step!)
  7. Set the bind to 0.0.0.0
  8. Set the requirepass (very important step!)

启动redis服务试试:

1
2
3
4
[root@localhost ~]# service redis_6379 start
Starting Redis server...
[root@localhost ~]# redis-cli ping
PONG

启动以后检查日志文件，发现会有很多警告信息，执行以下操作解决：

1
2
3
4
5
6
$ echo 511 > /proc/sys/net/core/somaxconn
$ echo "echo 511 > /proc/sys/net/core/somaxconn" >> /etc/rc.local
$ sysctl vm.overcommit_memory=1
$ echo "vm.overcommit_memory=1" >> /etc/sysctl.conf
$ echo never > /sys/kernel/mm/transparent_hugepage/enabled
$ echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local

LVS安装

通过lvs源码安装：
1
2
3
4
5
6
7
8
[root@localhost ~]# yum install ipvsadm -y
[root@localhost ~]# ipvsadm -v
ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1)
------------------------------------------------------or------------------------------
[root@localhost]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
[root@localhost ipvsadm-1.26]# tar zxf ipvsadm-1.26.tar.gz
[root@localhost ipvsadm-1.26]# make
[root@localhost ipvsadm-1.26]# make install

检查linux内核是否集成lvs模块,开启路由转发功能,以及ip_vs模块：

1
2
3
4
5
6
7
8
9
10
11
12
[root@localhost ~]# modprobe -l | grep ipvs
[root@localhost ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
[root@localhost ~]# echo "echo 1 > /proc/sys/net/ipv4/ip_forward" >> /etc/rc.local
[root@localhost ~]# modprobe ip_vs
[root@localhost ~]# modprobe ip_vs_wrr
[root@localhost ~]# echo "modprobe ip_vs" >> /etc/rc.local 
[root@localhost ~]# echo "modprobe ip_vs_wrr" >> /etc/rc.local
[root@localhost ~]# lsmod | grep ip_vs
ip_vs_wrr               2179  1 
ip_vs                 126705  3 ip_vs_wrr
libcrc32c               1246  1 ip_vs
ipv6                  336368  47 ip_vs,bridge,ip6t_REJECT,nf_conntrack_ipv6,nf_defrag_ipv6

keepalived安装

以下是Master和Slave都要进行的操作：
从官网下载源码包 http://www.keepalived.org/download.html ,安装相应的依赖库：

1
2
3
4
5
6
7
[root@localhost]# yum install -y gcc gcc-c++ makepcre pcre-devel kernel-devel openssl-devel libnl-devel popt-devel libnfnetlink-devel openssl-devel popt-static
[root@localhost]# yum update glib* -y
[root@localhost opt]# wget http://www.keepalived.org/software/keepalived-1.2.24.tar.gz
[root@localhost opt]# tar zxf keepalived-1.2.24.tar.gz
[root@localhost opt]# cd keepalived-1.2.24
[root@localhost]# ./configure --prefix=/usr/local/keepalived
[root@localhost]# make && make install

创建keepalived服务：

1
2
3
4
5
6
[root@localhost ~]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/keepalived
[root@localhost ~]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived
[root@localhost ~]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived
[root@localhost ~]# ln -s /usr/local/keepalived/etc/keepalived /etc/keepalived
[root@localhost ~]# chmod +x /usr/local/keepalived/etc/rc.d/init.d/keepalived
[root@localhost ~]# keepalived -v

master配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
[root@localhost ~]# vim /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     jun.chen@cyberkey.cn
   }
   notification_email_from redis_master@cyberkey.cn
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id REDIS-HA
}

vrrp_script chk_redis {
	#script "pidof redis-server"
    script "/etc/keepalived/scripts/redis_check.sh"
    interval 2
    timeout 2
    weight -100
    fall 2
    rise 1
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 58
    priority 200
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.94/24 dev eth0 scope global label eth0:0
    }
    track_script {
        chk_redis
    }
    #notify_master "/usr/local/bin/redis-cli SLAVEOF NO ONE"
    #notify_backup "/usr/local/bin/redis-cli SLAVEOF 192.168.1.93 6379"
    notify_master "/etc/keepalived/scripts/redis_master.sh"
    notify_backup "/etc/keepalived/scripts/redis_backup.sh"
}

slave配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
[root@localhost ~]# vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     jun.chen@cyberkey.cn
   }
   notification_email_from redis_slave@cyberkey.cn
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id REDIS-HA
}

vrrp_script chk_redis {
	#script "pidof redis-server"
    script "/etc/keepalived/scripts/redis_check.sh"
    interval 2
    timeout 2
    weight -100
    fall 2
    rise 1
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 58
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.94/24 dev eth0 scope global label eth0:0
    }
    track_script {
        chk_redis
    }
    #notify_master "/usr/local/bin/redis-cli SLAVEOF NO ONE"
    #notify_backup "/usr/local/bin/redis-cli SLAVEOF 192.168.1.92 6379"
    notify_master "/etc/keepalived/scripts/redis_master.sh"
    notify_backup "/etc/keepalived/scripts/redis_backup.sh"

}

master脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@localhost ~]# mkdir /etc/keepalived/scripts
[root@localhost ~]# vim /etc/keepalived/scripts/redis_check.sh
#!/bin/bash
/usr/local/bin/redis-cli ping
if [ $? -eq 0 ];then
   exit 0
else
   exit 1
fi 
[root@localhost ~]# vim /etc/keepalived/scripts/redis_master.sh
#!/bin/bash
/usr/local/bin/redis-cli SLAVEOF NO ONE
[root@localhost ~]# vim /etc/keepalived/scripts/redis_backup.sh
#!/bin/bash
/usr/local/bin/redis-cli SLAVEOF 192.168.1.93 6379
[root@localhost ~]# chmod +x /etc/keepalived/scripts/*

slave脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@localhost ~]# mkdir /etc/keepalived/scripts
[root@localhost ~]# vim /etc/keepalived/scripts/redis_check.sh
#!/bin/bash
/usr/local/bin/redis-cli ping
if [ $? -eq 0 ];then
   exit 0
else
   exit 1
fi
[root@localhost ~]# vim /etc/keepalived/scripts/redis_master.sh
#!/bin/bash
/usr/local/bin/redis-cli SLAVEOF NO ONE
[root@localhost ~]# vim /etc/keepalived/scripts/redis_backup.sh
#!/bin/bash
/usr/local/bin/redis-cli SLAVEOF 192.168.1.92 6379
[root@localhost ~]# chmod +x /etc/keepalived/scripts/*

防火墙设置

打开redis端口和vrrp端口：

1
2
3
4
[root@localhost ~]# vim /etc/sysconfig/iptables
-A INPUT -p vrrp -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 6379 -j ACCEPT
[root@localhost ~]# service iptables restart

测试效果

启动master节点的redis和keepalived,启动slave节点的redis和keepalived.查看主从关系:

1
2
[root@localhost ~]# redis-cli -h 192.168.1.94 INFO | grep slave0
slave0:ip=192.168.1.93,port=6379,state=online,offset=785,lag=0

停掉master节点的redis或者keepalived，在重启redis或者keepalived,查看主从关系:

1
2
[root@localhost ~]# redis-cli -h 192.168.1.94 INFO | grep slave0
slave0:ip=192.168.1.92,port=6379,state=online,offset=2339,lag=0

还可以试试添加一个key值后同步的效果。