编程知识 cdmana.com

Front end load and high availability configuration (hproxy + keepalived)

Front end load and high availability configuration (Hproxy+Keepalived)

The topology is as follows :

wKiom1MNgV2wnG36AAFE7GdVpLI209.jpg

In order to facilitate future maintenance and operation , Using a relatively simple technology , But understand. , The technology is simple, just relative to the user , Developers simplify user configuration , But it's still powerful . For load balancing , We use haproxy technology , High availability adoption keepalived technology .

notes :Haproxy Can provide high availability 、 Load balancing and based on TCP and HTTP Agent for application , Support virtual host , And it's free .

       Haproxy No multiprocessing or multithreading is used , It's about implementing an event driven 、 Single process model , So you don't have to be limited by memory and so on , But usually , Its scalability will be poor .

       Keepalived Is a similar to layer3,4,5 Software for the exchange mechanism , In other words, it is usually said that 3,4,5 layer

In exchange for .

        Layer3,4&5 Working in IP/TCP The protocol stack IP layer ,TCP layer

        Layer3:keepalived Use layer3 The way you work ,keepalived A... Is sent to the servers in the server cluster on a regular basis ICMP Data packets , therefore ,layer3 The way is to use the server IP Whether the address is valid is taken as the standard of whether the server works normally

        Layer4:Layer4 Mainly with TCP The state of the port determines whether the server is working properly .

        Layer5:Layer5 It's working in a specific application layer , It will check whether the server program is running properly according to the user's settings , If it doesn't match the user's settings , This server will be removed from the server cluster .

        Keepalived It is mainly used for RealServer Health check and LoadBalance The host and Slave Between the host failover The implementation of the

Haproxy To configure (Master)

notes : This experiment is just a test environment , about WEB The server is just a simple implementation .

Connect the server with the environment shown in the figure ,IP etc. , We are going to haproxy—keepalived In order to configure .

Haproxy Service installation

notes : Please download the required software by yourself , It's best to use the latest version .

notes : Turn off before configuring iptables And selinux

1. Change host name

We use 10.1.13.231 The server is Keepalived master server , Name it Master.KPLD;

10.1.13.232 It's a secondary server , Name it :slave.KPLD

[root@localhost ~]# cat /etc/hosts

127.0.0.1  localhost Master.KPLD localhost4 localhost4.localdomain4

::1        localhost localhost.localdomain localhost6 localhost6.localdomain6

[root@localhost ~]# cat/etc/sysconfig/network

NETWORKING=yes

HOSTNAME=Master.KPLD

2. Unpack the installation haproxy software package

# tar zxvf haproxy-1.3.20.tar.gz


[root@Master ~]# tar -zxvfhaproxy-1.3.20.tar.gz

[root@Master ~]# cd haproxy-1.3.20

[root@Master haproxy-1.3.20]# uname -r

2.6.32-279.el6.x86_64      // Here to check the kernel version number , install haproxy When , To specify the correct version number

          [root@Master haproxy-1.3.20]# vim Makefile        

             64 PREFIX = /usr/local It is amended as follows : 64PREFIX = /usr/local/haproxy

Because the decompressed file is compiled , Someone uses it directly during installation —prefix= Indicate installation path , I did it a few times , Discovery is not going to work , So you want to put the installation files in a directory , It can only be modified Makefile file

      [root@Master haproxy-1.3.20]# make TARGET=linux26   // Corresponding to the above kernel version

      [root@Master haproxy-1.3.20]# make install

      [root@Master haproxy-1.3.20]# mkdir /usr/local/haproxy/etc   // Create a directory of configuration files

      [root@Master haproxy-1.3.20]# cp -p examples/haproxy.cfg/usr/local/haproxy/etc/    //haproxy There are configuration file templates under the source code , We just need copy In the past , You can modify it.

3.Haproxy Service configuration

[root@Masteretc]# vim haproxy.cfg

global

      log 127.0.0.1   local0

       log 127.0.0.1   local1 notice

       #log loghost    local0 info

       maxconn 4096

       chroot /usr/share/haproxy

       uid 99

       gid 99

       daemon                 // To configure haproxy Enter the background operation mode

       nbproc 2                 // establish 2 Processes enter deamon mode

       pidfile /usr/local/haproxy/haproxy.pid

       #debug

       #quiet


defaults

       log    global

       mode   http       // The default mode mode

       option httplog

       option dontlognull   // Automatically end completed Links

       retries 3

       option redispatch       // When serverid The corresponding server is down , Force a redirect to another healthy server

       maxconn 2000

       contimeout      5000

       clitimeout      50000

       srvtimeout      50000

       stats uri /haproxy_stats         // Visit the statistics page url

       stats realm user \ passwd        // Statistics page password box prompt text

       stats auth haproxy:haproxy       // Statistics page user name and password

       stats hide-version               // hide haproxy Version information

listen  www.rhel.com 0.0.0.0:80

       cookie SERVERID rewrite

       balanceroundrobin

#       balance source                 // It's best to use this , The same client can be guaranteed , Fixed access to a server

       server 192.168.1.10 192.168.1.10:80 cookie 192.168.1.10 check inter 2000 rise 2fall

       server  192.168.1.20 192.168.1.20:80 cookie192.168.1.20 check inter 2000 rise 2 fall 5

//  Server definition ,checkinter 2000 It refers to the detection of heart frequency ,rise finger 3 The second positive solution is that the server is available ,fall 5 Failure 5 I think the server is not available , You can also set weights weigth  Numbers

After configuration , You need to create a file , It corresponds to the configuration

[root@Master etc]# mkdir /usr/share/haproxy

4. Log configuration

     [root@Masteretc]# vim /etc/rsyslog.conf      // Log profile

Add these two lines

local0.*/var/log/haproxy.log                // increase local0 Log file , The same below

local1.* /var/log/haproxy.log

             [root@Master etc]# vim/etc/sysconfig/rsyslog

Modify this trip :SYSLOGD_OPTIONS="-r -m 0"

              [root@Master etc]# servicersyslog restart       // Restart the logging process

After restarting, you can see /var/log/ There it is haproxy.log This file

5. start-up haproxy The server

[root@Master ~]#/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg

Take a look at the process

[root@Mastersbin]# netstat -tulpn | grep haproxy

tcp        0     0 0.0.0.0:10001  0.0.0.0:*   LISTEN     3451/./haproxy      

udp 0 00.0.0.0:49372 0.0.0.0:*                              3451/./haproxy      

[root@Mastersbin]#

Start successfully

6. Enable routing and forwarding

[root@Mastersbin]# echo "1">/proc/sys/net/ipv4/ip_forward

7.Haproxy(Slave)

Slave Installation and Master The installation is the same , We just need to modify it Slave The host name of , I'm not going to repeat it here

Configure server Web1 And Web2

notes :httpd The server is only used for testing at this stage , To the back end Nginx After the server is set up , this httpd The server is shutting down !

1. modify Web1 And Web2 Host name

[root@localhost~]# hostname Web1.KPLD      // If necessary, it can be configured as a permanent host name

root@Web1 ~]#yum install httpd –y             // install apache

root@Web1 ~]# echo "Web1.KPLD">/var/www/html/index.html

[root@Web1 ~]#service httpd start

       Web2.rhel.com The configuration is basically the same , You need to change the host name , establish index.html Change the content of the web page

[root@Web2 ~]# echo "Web2.KPLD">/var/www/html/index.html

One . test Haproxy Is the service installed successfully

Enter in the address bar Master/Slave Of ip Address ( there Master And Slave It's right keepalived What high availability says , Yes haproxy Load balancing doesn't matter )

wKiom1MNgb3SA2VHAABgZq4p_wU711.jpg

Click refresh

wKioL1MNgZfTDKegAABivgO3a-4957.jpg

Because we use polling , So it's polling between the two servers .

We can also make the graphical interface view Haproxy The server usage status configured under the service

wKiom1MNgb2Bq3A9AADlqOkyMQ4271.jpg

In higher versions , Added more advanced features , such as , You can take the server off the shelf directly .

Keepalived Service configuration (Master)

We have achieved in haproxy Load balancing of , however , We can't publish two addresses at once , So use keepalived High availability features to achieve virtual ip, To achieve the goal of high availability .

1.keepalived Installation

[root@Master ~]# yum install kernel-devel openssl-devel popt-devel    // Install dependent files

[root@Master ~]# tar -zxvf keepalived-1.2.2.tar.gz

[root@Master ~]# cd keepalived-1.2.2

[root@Master keepalived-1.2.2]# ./configure--with-kernel-dir=/usr/src/kernels/2.6.32-279.el6.x86_64/

If the following prompt appears , That is success

Keepalivedconfiguration

------------------------

Keepalivedversion       : 1.2.2

Compiler                 : gcc

Compilerflags           : -g -O2

Extra Lib                : -lpopt -lssl -lcrypto

Use IPVSFramework       : Yes

IPVS sync daemonsupport : Yes

IPVS uselibnl           : No

Use VRRPFramework       : Yes

Use Debugflags          : No

[root@Masterkeepalived-1.2.2]#

[root@Masterkeepalived-1.2.2]# make && make install

[root@Masterkeepalived-1.2.2]# cp keepalived/etc/init.d/keepalived.rh.init/etc/init.d/keepalived                            //copy The startup script

[root@Masterkeepalived-1.2.2]# chmod +x /etc/init.d/keepalived    // Executable rights

[root@Masterkeepalived-1.2.2]# chkconfig --level 35 keepalived on  // stay 35 Level boot up

[root@Masterkeepalived-1.2.2]# cp keepalived/etc/init.d/keepalived.sysconfig/etc/sysconfig/keepalived                        //copy Start initialization file

[root@Masterkeepalived-1.2.2]# cp /usr/local/sbin/keepalived /usr/sbin/    //copy Startup file

2.keepalived Service configuration

[root@Master keepalived-1.2.2]# vim/usr/local/etc/keepalived/keepalived.conf

! ConfigurationFile for keepalived


global_defs {

  notification_email {

    acassen@firewall.loc

    failover@firewall.loc

    sysadmin@firewall.loc

  }

  notification_email_from Alexandre.Cassen@firewall.loc

  smtp_server 192.168.200.1

  smtp_connect_timeout 30

  router_id LVS_DEVEL

}


vrrp_instanceVI_1 {

   state MASTER

   interface eth2

   virtual_router_id 60

   priority 100

   advert_int 1

   authentication {

       auth_type PASS

       auth_pass 1111

   }

   virtual_ipaddress {

      10.1.13.230

   }

}

virtual_server 10.1.13.230 80 {

  delay_loop 6

  lb_algo rr

  lb_kind DR

  nat_mask 255.255.255.0

  persistence_timeout 50

  protocol TCP


   real_server 10.1.13.231 80 {

      weight 1

      TCP_CHECK {

          connect_timeout 3

          nb_get_retry 3

          delay_before_retry 3

      }

   }

  real_server 10.1.13.232 80 {

      weight 1

      TCP_CHECK {

          connect_timeout 3

          nb_get_retry 3

          delay_before_retry 3

      }

   }

}

Because of the startup script /etc/init.d/keepalived The configuration of , The default configuration file is config:/etc/keepalived/keepalived.conf, therefore , We're going to create a link

[root@Masterkeepalived]# ln -s /usr/local/etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf

start-up keepalived that will do

[root@Masterkeepalived-1.2.2]# service keepalived start

Keepalived To configure (Slave)

Slave Configuration and Master The configuration is basically the same , Just modify a few aspects , The full code is no longer posted here , Write only the differences


1. Will be a place state MASTER  It is amended as follows state BACKUP

2. Will be a place priority 100   It is amended as follows priority 50

Back end NGINX Server configuration

The company does not have this server , The company's real servers are IIS, I don't like windows Server side , So it's not used in the test IIS

1. Install dependent software

[root@localhost~]# yum install gcc openssl-devel pcre-devel zlib-devel

2. establish Nginx user

# useradd nginx-s /sbin/nologin

3. Unzip the software and install

# tar -zxvfnginx-1.4.2.tar.gz

# cd nginx-1.4.2

# ./configure--user=nginx --group=nginx --prefix=/usr/local/nginx--with-http_stub_status_module --with-http_ssl_module

# make&& make install

notes :--user,--group: Specify the start of each nginx Accounts and groups for

--with-http_stub_status_module: Install allow state module

--with-http_ssl_module: install ssl modular

4. start-up nginx

#/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

notes : stay web1 And web2 Make a home page for each ,web1 The content of Web1.KPLD,web2 The content of Web2.KPLD

5. Configure static routing

Because our servers are internal networks , Can't communicate with external networks , however Keepalived We use DR Model , therefore , When a request arrives , It is returned directly to the client by the internal server , therefore , We must ensure that the internal host can be connected to the external network

stay /etc/rc.local Add the following route in the

  route add defaultgw 192.168.200.253

6. Except for the configuration in web Configuration on the server IP outside , Also need to do for intranet SNAT, It can be on two Keepalived Do on SNAT

iptables -t nat -A POSTROUTING -s 192.168.200.0/24 -j SNAT--to 10.1.13.231-10.1.13.232



test Keepalived

Because the setting is Master by keepalived Primary server , So it can be Master Check to see if there is a virtual address on

[root@Master keepalived]# ip addr

You should be able to find the following

2: eth2:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 00:0c:29:3a:4d:dd brdff:ff:ff:ff:ff:ff

   inet 10.1.13.231/24 brd 10.1.13.255 scopeglobal eth2

   inet 10.1.13.230/32 scope global eth2               // It can be seen that , fictitious IP appear .

   inet6 fe80::20c:29ff:fe3a:4ddd/64 scopelink

      valid_lft forever preferred_lft forever

You can also install ipvsadm To view the configuration

     [root@Masterkeepalived]# yum install ipvsadm

Use ipvsadm –L Check it out.

Now? Use virtual ip Address to view the web page

wKioL1MNggbzC8DMAABo9SFNrqo976.jpg

Now let's test high availability , hold Master Of keepalived stop, See if it's going to happen vip The transfer of

stay Master Up close keepalived service , see Slave On ip addr Does it appear? vip

[root@Master keepalived]# servicekeepalived stop

stay Slave Check out

[root@Slave ~]# ip addr

2: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 08:00:27:a5:0e:93 brdff:ff:ff:ff:ff:ff

   inet 10.1.13.232/24 brd 10.1.13.255 scopeglobal eth1

   inet 10.1.13.230/24 scope global secondaryeth1          // fictitious IP appear

   inet6 fe80::a00:27ff:fea5:e93/64 scope link

     valid_lft forever preferred_lftforever

Then through the browser to view the web page , To view the .


It's normal , High availability is also successful .

Configuration troubleshooting

Keepalived Configuration problems and solutions

A wrong :

configure: error:

 !!!OpenSSL is not properly installed on your system. !!!

 !!!Can not include OpenSSL headers files.            !!!

install openssl-devel

  yum install openssl-devel


Error 2 :

configure: error: Popt libraries isrequired

install popt Development kit

 yum install popt-devel

Error of three :

[root@Master keepalived-1.2.2]# servicekeepalived start

Starting Keepalived for LVS: /bin/bash:keepalived: command not found

                                                         [FAILED]

resolvent :

[root@Slave ~]# cp/usr/local/sbin/keepalived /usr/sbin/


Error four :

    Ipvs Protocol not available

  Dec31 10:51:02 Slave Keepalived_healthcheckers: Registering Kernel netlink commandchannel

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Configuration is using : 14529 Bytes

Dec 31 10:51:02 Slave Keepalived:Healthcheck child process(19805) died: Respawning

Dec 31 10:51:02 Slave Keepalived: StartingHealthcheck child process, pid=19807

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: IPVS: Can't initialize ipvs: Protocol not available

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Netlink reflector reports IP 10.1.13.232 added

resolvent :

Manual loading ip_vs modular

modprobe ip_vs

modprobe ip_vs_wrr

And add boot up

 #cat /etc/rc.local

   /sbin/modprobeip_vs

/sbin/modprobe ip_vs_wrr



Error five :

 Master The server and Slave Virtual servers IP Address , When the primary server is disconnected , Cannot switch from server , The log shows

Dec 18 22:37:24 localhost Keepalived_vrrp:bogus VRRP packet received on eth1 !!!

Dec 18 22:37:24 localhost Keepalived_vrrp:VRRP_Instance(VI_1) ignoring received advertisment...

Dec 18 22:37:25 localhost Keepalived_vrrp:ip address associated with VRID not present in received packet : 10.1.13.230

Dec 18 22:37:25 localhost Keepalived_vrrp:one or more VIP associated with VRID mismatch actual MASTER advert

The main reason for this is virtual_router_id Incorrect configuration , By default 51, In some cases, however, changes are needed , For example, change it to :virtual_router_id 60 , The switch will succeed


版权声明
本文为[osc_ sxdofc9c]所创,转载请带上原文链接,感谢

Scroll to Top