Rhel6-pacemaker+drbd配置文档

系统环境: rhel6 x86_64 iptables and selinux disabled

主机: 192.168.122.119 server19.example.com

192.168.122.25 server25.example.com(注:时间需同步)

192.168.122.1 desktop36.example.com

所需的包:drbd-8.4.3.tar.gz

yum仓库配置:

[rhel-source]

name=Red Hat Enterprise Linux $releasever - $basearch - Source

baseurl=ftp://192.168.122.1/pub/yum

enabled=1

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[HighAvailability]

name=Instructor Server Repository

baseurl=ftp://192.168.122.1/pub/yum/HighAvailability

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

enabled=1

[LoadBalancer]

name=Instructor Server Repository

baseurl=ftp://192.168.122.1/pub/yum/LoadBalancer

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

enabled=1

[ResilientStorage]

name=Instructor Server Repository

baseurl=ftp://192.168.122.1/pub/yum/ResilientStorage

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

enabled=1

[ScalableFileSystem]

name=Instructor Server Repository

baseurl=ftp://192.168.122.1/pub/yum/ScalableFileSystem

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

enabled=1

#配置pacemaker

以下步骤在server19和server25上实施：

[root@server19 ~]# yum install corosync pacemaker -y

以下步骤在server19或server25上实施：

[root@server19 ~]# cd /etc/corosync/

[root@server19 corosync]# corosync-keygen (生成该key需要不断的敲打键盘)

[root@server19 corosync]# cp corosync.conf.example corosync.conf

[root@server19 corosync]# vim corosync.conf

# Please read the corosync.conf.5 manual page

compatibility: whitetank

totem {

version: 2

secauth: off

threads: 0

interface {

ringnumber: 0

bindnetaddr: 192.168.122.0

mcastaddr: 226.94.1.1

mcastport: 5405

ttl: 1

}

}

logging {

fileline: off

to_stderr: yes

to_logfile: yes

to_syslog: yes

logfile: /var/log/cluster/corosync.log

debug: off

timestamp: on

logger_subsys {

subsys: AMF

debug: off

}

}

amf {

mode: disabled

}

service {

ver: 0

name: pacemaker

use_mgmtd: yes

}

[root@server19 corosync]# scp corosync.conf authkey root@192.168.122.25:/etc/corosync/

以下步骤在server19和server25上实施：

[root@server19 corosync]# /etc/init.d/corosync start

此时查看日志tail -f /var/log/cluster/corosync.log会有如下错误:

Jul 27 02:31:31 [1461] server19.example.com pengine: notice: process_pe_message: Configuration ERRORs found during PE processing. Please run "crm_verify -L" to identify issues.

解决方法如下:

[root@server19 corosync]# crm（注：redhat6.4后crm这个命令没有集成在pacemaker包中,需要另外安装crmsh）

crm(live)# configure

crm(live)configure# property stonith-enabled=false

crm(live)configure# commit

crm(live)configure# quit

[root@server19 corosync]# crm_verify -L(检测配置是否有错误)

此时执行crm_mon进入监控页面，若两台主机均处于Online状态说明配置成功.

以下配置只需在任意一台机子上实施,所有配置会自动同步到另一台机子上.

#添加虚拟IP

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# primitive vip ocf:heartbeat:IPaddr2 params ip=192.168.122.178 cidr_netmask=32 op monitor interval=30s

crm(live)configure# commit

crm(live)configure# quit

#忽略法定人数的检查

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# property no-quorum-policy=ignore

crm(live)configure# commit

crm(live)configure# quit

# 添加apache服务

1.以下步骤在server19和server25上实施：

[root@server19 corosync]# vim /etc/httpd/conf/httpd.conf

<Location /server-status>

SetHandler server-status

Order deny,allow

Deny from all

Allow from 127.0.0.1

</Location>

[root@server19 corosync]# echo `hostname` > /var/www/html/index.html

2.以下步骤在server19或server25上实施：

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# primitive apache ocf:heartbeat:apache params configfile=/etc/httpd/conf/httpd.conf op monitor interval=1min

crm(live)configure# commit

qcrm(live)configure# quit

此时执行crm_mon有可能会发现vip和apache运行在不同的机子上:

解决方法(将vip和apache绑定):

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# colocation apache-with-vip inf: apache vip

crm(live)configure# commit

crm(live)configure# quit

此时访问192.168.122.178可访问到server19上的页面

#配置主备

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# location master-node apache 10: server19.example.com

crm(live)configure# commit

crm(live)configure# quit

#配置fence

以下步骤在desktop36上实施：

[root@desktop36 ~]# yum list fence*

[root@desktop36 ~]# yum install fence-virtd.x86_64 fence-virtd-libvirt.x86_64 fence-virtd-multicast.x86_64 fence-virt-0.2.3-9.el6.x86_64 -y

[root@desktop36 ~]# fence_virtd -c

Module search path [/usr/lib64/fence-virt]:

Available backends:

libvirt 0.1

Available listeners:

multicast 1.1

Listener modules are responsible for accepting requests

from fencing clients.

Listener module [multicast]:

The multicast listener module is designed for use environments

where the guests and hosts may communicate over a network using

multicast.

The multicast address is the address that a client will use to

send fencing requests to fence_virtd.

Multicast IP Address [225.0.0.12]:

Using ipv4 as family.

Multicast IP Port [1229]:

Setting a preferred interface causes fence_virtd to listen only

on that interface. Normally, it listens on the default network

interface. In environments where the virtual machines are

using the host machine as a gateway, this *must* be set

(typically to virbr0).

Set to ‘none‘ for no interface.

Interface [none]: virbr0

The key file is the shared key information which is used to

authenticate fencing requests. The contents of this file must

be distributed to each physical host and virtual machine within

a cluster.

Key File [/etc/cluster/fence_xvm.key]:

Backend modules are responsible for routing requests to

the appropriate hypervisor or management layer.

Backend module [checkpoint]: libvirt

The libvirt backend module is designed for single desktops or

servers. Do not use in environments where virtual machines

may be migrated between hosts.

Libvirt URI [qemu:///system]:

Configuration complete.

=== Begin Configuration ===

backends {

libvirt {

uri = "qemu:///system";

}

}

listeners {

multicast {

interface = "virbr0";

port = "1229";

family = "ipv4";

address = "225.0.0.12";

key_file = "/etc/cluster/fence_xvm.key";

}

}

fence_virtd {

module_path = "/usr/lib64/fence-virt";

backend = "libvirt";

listener = "multicast";

}

=== End Configuration ===

Replace /etc/fence_virt.conf with the above [y/N]? y

注:以上设置除“Interface”处填写虚拟机通信接口和Backend module填写libvirt外,其他选项均可回车保持默认。

[root@desktop36 ~]# mkdir /etc/cluster

[root@desktop36 ~]# dd if=/dev/urandom of=/etc/cluster/fence_xvm.key bs=128 count=1

以下步骤在server19和server25上实施：

[root@server19 corosync]# mkdir /etc/cluster

[root@server19 corosync]# yum install fence-virt-0.2.3-9.el6.x86_64 -y

以下步骤在desktop36上实施：

[root@desktop36 ~]# scp /etc/cluster/fence_xvm.key root@192.168.122.119:/etc/cluster/

[root@desktop36 ~]# scp /etc/cluster/fence_xvm.key root@192.168.122.25:/etc/cluster/

[root@desktop36 ~]# /etc/init.d/fence_virtd start

[root@desktop36 ~]# netstat -anuple | grep fence

udp 0 0 0.0.0.0:1229 0.0.0.0:* 0 823705 6320/fence_virtd

注:可查看到1229端口说明fence_virtd启动成功.

以下步骤在server19或server25上实施：

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# cib new stonith

crm(stonith)configure# quit

[root@server19 corosync]# crm

crm(live)# configure

crm(live)configure# primitive vmfence stonith:fence_xvm params pcmk_host_map="server19.example.com:vm1 server25.example.com:vm2" op monitor interval=30s

crm(live)configure# property stonith-enabled=true

crm(live)configure# commit

crm(live)configure# quit

测试：将server19断网或执行echo c > /proc/sysrq-trigger模拟内核崩溃，看服务是否被接管，并且server19断电重启.

#配置drbd

分别给server19和server25加一块相同大小的虚拟硬盘

以下步骤在server19或server25上实施：

[root@server19 kernel]# yum install kernel-devel make -y

[root@server19 kernel]# tar zxf drbd-8.4.3.tar.gz

[root@server19 kernel]# cd drbd-8.4.3

[root@server19 drbd-8.4.3]# ./configure --enable-spec –with-km

此时会出现如下问题:

(1)configure: error: no acceptable C compiler found in $PATH

(2)configure: error: Cannot build utils without flex, either install flex or pass the --without-utils option.

(3)configure: WARNING: No rpmbuild found, building RPM packages is disabled.

(4)configure: WARNING: Cannot build man pages without xsltproc. You may safely ignore this warning when building from a tarball.

(5)configure: WARNING: Cannot update buildtag without git. You may safely ignore this warning when building from a tarball.

解决方法如下:

(1)[root@server19 drbd-8.4.3]# yum install gcc -y

(2)[root@server19 drbd-8.4.3]# yum install flex -y

(3)[root@server19 drbd-8.4.3]# yum install rpm-build -y

(4)[root@server19 drbd-8.4.3]# yum install libxslt -y

(5)[root@server19 drbd-8.4.3]# yum install git -y

[root@server19 kernel]# mkdir -p ~/rpmbuild/SOURCES

[root@server19 kernel]# cp drbd-8.4.3.tar.gz ~/rpmbuild/SOURCES/

[root@server19 drbd-8.4.3]# rpmbuild -bb drbd.spec

[root@server19 drbd-8.4.3]# rpmbuild -bb drbd-km.spec

[root@server19 drbd-8.4.3]# cd ~/rpmbuild/RPMS/x86_64/

[root@server19 x86_64]# rpm -ivh *

[root@server19 x86_64]# scp ~/rpmbuild/RPMS/x86_64/* root@192.168.122.25:/root/kernel/

以下步骤在server25上实施：

[root@server25 kernel]# rpm -ivh *

以下步骤在server19和server25上实施：

[root@server19 ~]# fdisk -cu /dev/vda

划分分区（一般只划一个分区）,类型为Linux LVM的

[root@server19 ~]# pvcreate /dev/vda1

[root@server19 ~]# vgcreate koenvg /dev/vda1

[root@server19 ~]# lvcreate -L 1G -n koenlv koenvg

以下步骤在server19或server25上实施：

[root@server19 drbd.d]# cd /etc/drbd.d/

[root@server19 drbd.d]# vim drbd.res

resource koen {

meta-disk internal;

device /dev/drbd1;

syncer {

verify-alg sha1;

}

net {

allow-two-primaries;

}

on server19.example.com {

disk

/dev/mapper/koenvg-koenlv;

address 192.168.122.119:7789;

}

on server25.example.com {

disk

/dev/mapper/koenvg-koenlv;

address 192.168.122.25:7789;

}

}

[root@server19 drbd.d]# scp drbd.res root@192.168.122.25:/etc/drbd.d/

以下步骤在server19和server25上实施：

[root@server19 drbd.d]# drbdadm create-md koen

[root@server19 drbd.d]# /etc/init.d/drbd start

以下步骤在server19上实施：

[root@server19 drbd.d]# drbdsetup /dev/drbd1 primary –force

(此条命令将server19设置成primary节点，并同步数据)

此时可以执行watch cat /proc/drbd 查看同步状态，当同步完成后继续往下配置，创建文件系统.

[root@server19 drbd.d]# mkfs.ext4 /dev/drbd1

[root@server19 drbd.d]# mount /dev/drbd1 /var/www/html/

注意:两台主机上的/dev/drbd1 不能同时挂载,只有状态为 primary 时,才能被挂载使 用,而此时另一方的状态为 secondary。

测试：在server19上将/dev/drbd1挂在到/var/www/html/下 ,进到/var/www/html/中随意编辑一些文件，然后卸载/dev/drbd1(umount /var/www/html/)，执行drbdadm secondary koen和drbdadm primary koen将server25设置为主节点，在server25上挂载/dev/drbd1,最后查看/var/www/html/下的内容是否同步,

附:拉伸设备

以下步骤在server19和server25上实施：

[root@server19 ~]# lvextend -L +1000M /dev/mapper/koenvg-koenlv

以下步骤在server19或server25上实施：

[root@server19 ~]# drbdadm resize koen

以下步骤在primary节点上实施：

[root@server25 ~]# mount /dev/drbd1 /var/www/html/

[root@server25 ~]# resize2fs /dev/drbd1

#将pacemaker和drbd整合

以下步骤在server19或server25上实施：

[root@server19 ~]# crm

crm(live)# configure

crm(live)configure# primitive webdata ocf:linbit:drbd params drbd_resource=koen op monitor interval=60s

crm(live)configure# ms webdataclone webdata meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true

crm(live)configure# primitive webfs ocf:heartbeat:Filesystem params device="/dev/drbd1" directory="/var/www/html" fstype=ext4

crm(live)configure# group webgroup vip apache webfs

crm(live)configure# colocation apache-on-webdata inf: webgroup webdataclone:Master

crm(live)configure# order apache-after-webdata inf: webdataclone:promote webgroup:start

crm(live)configure# commit

crm(live)configure# quit

附：使用iscsi存储

以下步骤在desktop36上实施：

[root@desktop36 ~]# yum install scsi-target-utils.x86_64 -y

[root@desktop36 ~]# vim /etc/tgt/targets.conf

<target iqn.2013-07.com.example:server.target1>

backing-store /dev/vg_desktop36/iscsi-test

initiator-address 192.168.122.112

initiator-address 192.168.122.234

</target>

[root@desktop36 ~]# /etc/init.d/tgtd start

以下步骤在server19和server25上实施：

[root@server19 ~]# iscsiadm -m discovery -t st -p 192.168.122.1

[root@server19 ~]# iscsiadm -m node -l

使用fdisk -cu对iscsi设备进行分区并且格式化.

注:此操作只需要在一个节点上进行即可,另一个节点会自动同步.

以下步骤在server19或server25上实施：

[root@server19 ~]# crm

crm(live)# configure

crm(live)configure# primitive iscsi ocf:heartbeat:Filesystem params device=/dev/sda1 directory=/var/www/html fstype=ext4 op monitor interval=30s

crm(live)configure# colocation apache-with-iscsi inf: apache iscsi

crm(live)configure# commit

crm(live)configure# quit

Rhel6-pacemaker+drbd配置文档

原文：http://www.cnblogs.com/xautlmx/p/4381164.html