编程知识 cdmana.com

Learn the high availability of redis from master-slave replication

Preface

Now when you encounter high concurrency scenarios , Caching should be the first step in performance optimization , It can relieve the pressure of database and improve the access efficiency at the same time , and Redis It should be the first choice for most application scenarios . But as soon as possible Redis No matter how good the performance is , In today's high concurrency scenarios , One server is responsible for reading and writing , Machine performance and memory bottlenecks are inevitable , There must be some small partners who will think of clusters , Right , That's right , Just before the cluster , The optimization strategy of master-slave replication mode can solve many problems , If the master-slave mode can't resist high concurrency , It's not too late to cluster again ; Let's talk about Redis Master-slave replication of .

For a better demonstration , Set up a cloud server ,Linux Environmental Science ; Convenient at the same time , It can also be more in line with the actual application scenarios ;

Text

Master slave copy : Master slave refers to having more than one Redis The server , One of the main servers , Others are slave servers , Master slave relationship can be realized by command or configuration ; Replication refers to synchronizing data from the primary server to the slave server , Data can only be synchronized one way from the master server to the slave server ; Its functions are as follows :

Read / write separation : Master server copy write , Each slave server is responsible for reading ; According to the principle of 28 ,80% The operation of reading is reading , Only 20% Write , So to a certain extent, it also solves the single machine bottleneck problem ;• Data persistence is more secure : The master and slave servers perform persistence operations , Any service outage will not affect data recovery , Single point of failure problem is avoided ;• other : Master slave replication is to implement sentinel mode and Redis The premise of clustering , This follow-up will say .

All right. , Old rules , After understanding its function , Next, I'll do the actual operation and then summarize .

Implement master-slave replication

Each station Redis When the server starts , The default is the primary server (Master), By command info replication see , As shown in the figure below :

Start practice, build a master and two slave environment , as follows :

1. Configuration file modification

Because it's simulated on the same machine , So will redis Copy the configuration file in three copies , The main modifications are as follows :

• Profile name : Respectively redis.conf、redis6388.conf、redis6399.conf;•port: port , The three configuration files are modified as follows 6379、6388、6399, This is to avoid port conflicts on the same machine ;•pidfile: Modify this file name , Avoid file conflicts , Change the file name , Revise them to redis.pid、redis6388.pid、redis6399.pid;•dbfilename: Persistent filename , Avoid persistent file conflicts , Revise them to dump.rdb、dump6388.rdb、dump6399.rdb;

2. start-up redis The server

Then specify the configuration file to start redis-server, stay redis in bin Execute under directory ./redis-server redis.conf./redis-server redis6388.conf./redis-server redis6399.conf Command is enough , By command ps -ef | grep redis View startup redis effect , as follows :

3.1 Use commands to implement master-slave relationships

By default , The three servers started are all the primary servers by default , It can be realized from a simple relation of two , Here we use 6379 This server is mainly ,6388 and 6399 Two servers are slave , Configure the master-slave relationship only for the slave node server configuration , as follows :

1. Connect them separately 6388 and 6399 Two servers , perform slaveof masterip masterport that will do :

    2. From the node display no problem , Look at the master node status information , Even on 6379 Look at this one , as follows :

3. The master-slave relationship has been maintained , Let's look at data replication , Usually the master node is responsible for writing , Synchronize data to slave nodes , The slave node is responsible for reading ; Now there's no data on all three servers , Next, write point data to the master node , See if you can synchronize to the slave node :

notes : If the master-slave relationship is realized by command , When you restart from the server , The master-slave relationship is lost , You have to execute the command manually , Therefore, it is recommended to configure in the form of configuration file ;

3.2 Master slave configuration is realized through configuration file

It's still very simple , The master server doesn't move , Only modify the slave server configuration file , Then restart , According to the learning experience of the previous chapter , Open profile , Find it directly REPLICATION The module makes the following configuration changes :

replicaof <masterip> <masterport> : Specify master ip And port , Such as :replicaof 127.0.0.1 6379;•masterauth <master-password>: Specify the master node password , If the master node is configured with a password , Just go through this configuration ;

After revising these two items , Restart the server , As simple as the command line , Only the master-slave relationship established through the configuration file will not be lost because the server is shut down ; The demo of the configuration file is not a screenshot , It's just that it's not the same way , Everything else is consistent with the command line ;

A small summary

• Deserve never be worthy of the Lord , That is, only for the slave server configuration , The primary server does not need to be configured ;• A master node has multiple slaves , The slave node can only have one master node ; The slave node can also be the master node of other slave nodes , as follows :

      The screenshot of the demonstration in this way is not provided , Let's have a good practice , Here's an explanation , although 6388 yes 6399 The Lord of section   spot , But the role is still 6379 The slave node , It's just that there are other slave nodes under it ;   • If the master-slave is implemented by command , After the primary server is disconnected , Reconnect the , The master-slave relationship is still in ; After being disconnected from the server, the reconnection relationship disappears , You need to manually execute the command to reassign the master node ;   • If you want to disconnect and reconnection, the master-slave relationship is still , It is recommended to implement the configuration file ;

Is it easy to build master-slave replication , Whether it's a command or a configuration file , It's easy to implement ; But is there any doubt about my partner : How data is synchronized between master and slave nodes ? What are the other parameters for master-slave replication ?

Master slave replication principle

Through the above practice , Data written to the primary server , It synchronizes to the slave servers without awareness , What happened in the middle ? Let's take a brief look at ;

stay Redis2.8 The previous synchronization methods were all synchronized in full mode , And then in order to improve efficiency , There are two ways of data replication , One is full replication , One is partial replication :

• Copy in full : That is, the data in the primary server , All synchronized to the slave server ; It is generally used to synchronize the initialization data from the server ;• Partial reproduction : That is, the asynchronous incremental data , Synchronize to slave server , There's no need to synchronize it all over again ; It is generally used in the case that data cannot be synchronized due to network interruption , When it's back to normal , Partial synchronization of data during interruption ;

For the convenience of viewing log analysis , Use two redis The server builds the master-slave , Remove the relationship that was built before , Restart by modifying the configuration file ;

Here we use 6388 Master node (Master),6399 As a slave node (Slave), When the connection 6399 Carry out orders slaveof 127.0.0.1 6388 When configuring the master-slave relationship , The master and slave nodes print logs as follows :

The picture above shows when the master-slave relationship was first established , the Copy in full , The general process is as follows :

Brief description :

1. The slave node establishes a connection with the master node , Then send a synchronization request psync;2. The master node sends messages to the slave nodes ,replid and offset, These two parameters are followed by the key data to determine whether to partially copy ;3. Master node fork All the data will be generated by the subprocess RDB file ;4. Write commands received during the master node are stored in the copy buffer ;5. When the master node RDB After the file is completed, it is sent to the slave node ;6. Received file from node , Clear the old data first ;7. After clearing data from the node , Load the received data into memory ;8. The master node sends the data in the replication buffer to the slave node ;9. The command is received from the node and executed , Finally synchronize to the latest data ;

The master-slave relationship is to communicate through the network , There may be network interruption or network jitter , As a result, the short-term data cannot be synchronized to the slave node in time , Ideally , When the connection is restored , You want to synchronize only the data during the interruption , So as to improve the synchronization efficiency , Partial reproduction The process is as follows :

Brief description :

1. When the network is interrupted between master and slave , The slave will attempt to reconnect the master ;2. in the meantime , The command is copied to the primary node in the buffer ;3. When the network recovers , Connect the slave node to the master node , Will request to publish a synchronization request psync, And bring the data sent by the master node before replid and offset;4. The master node receives a request from the slave node , Will verify the received replid With the master node replid match , Mismatches are replicated in full ; It will also verify offset Whether the data offset is within the legal range , If there is too much data during the interruption , Cause copy buffer data to exceed , Master slave node offset Data offset inconsistent , Full replication will also be done ;5. When it comes from a node replid and offset When the verification passes , Then make a partial copy , And keep track of the latest offset;

We will not simulate the partial replication process here , Leave it to your partner . The demonstration process is as follows : Two machines that have built a master-slave relationship , Manually simulate network disconnection , Write data to the primary server during disconnection , After a while, the network will be restored , Check the log printing of master and slave servers ;

Configuration parameters related to master-slave replication

The above demonstration only configures some parameters in the configuration file , There are other parameters that can be configured to easily implement the function , Here, combined with the content of master-slave replication, we can review the significance of relevant configuration parameters , as follows :

slaveof: Set this machine as a slave , Specify the IP And port ;•masterauth: If the host needs a password , Through this setting ;•slave-serve-stale-data: If the slave is disconnected from the host or the data is synchronizing , Get data whether to continue , If set to yes, The data can also be read normally , Set to no, Get error data and return ;•slave-read-only: The slave is configured as read-only mode , The default is yes, Read only mode is also strongly recommended ;•repl-diskless-sync: Whether to use diskless mode to transfer data from master to slave , using Socket The way , By default , If the machine disk performance is not good , And the network environment is good , You can try this pattern ;•repl-diskless-sync-delay: When transferring data in diskless mode , Wait for a specified time before the server starts to deliver data , Waiting for the transfer queue from the server , Improve data transmission efficiency , The default is 5 second ;•repl-ping-slave-period: Set the slave to send to the host ping Message interval , It means heartbeat detection ;•repl-timeout: Set timeout , The time when the master-slave transmits data , If it exceeds the specified time , Re connect from opportunity . Generally, when the master-slave replication data is relatively large , You can make it bigger , The default is 60s ;•repl-disable-tcp-nodelay: Whether master-slave replication data uses TCP_NODELAY, The default is no, On behalf of not using , Indicates that the host synchronizes data immediately , Failure to ensure data consistency ; If set to yes, Merge smaller TCP The packets are sent together , High delay , But it can improve bandwidth performance , But it is recommended not to use ;•repl-backlog-size: Used to set the size of the copy buffer , This buffer is used to synchronize incremental data after the slave is disconnected and reconnected , You don't have to copy it all for a certain period of time , Improve synchronization efficiency ; The default is 1mb, It can be modified as needed ;•repl-backlog-ttl: Set the interval time when the slave is disconnected from the host , Beyond this time , Set up backlog The buffer will be released , The default is 3600s;•slave-priority: For sentinel mode selection , That is, when the host is down , Choose the slave with higher priority to replace the host that is down , Fast recovery ; The default value is 100;•min-slaves-to-write 3: Less than available slave servers 3 Time , The primary server can only read , Can't write , In general, and min-slaves-max-lag Matching , Default not to use , Configure as needed ;•min-slaves-max-lag 10: The delay from the server exceeds 10 seconds , The main server can only read , Can't write , In general, and min-slaves-to-write Matching , Default not to use , Configure as needed ;

What are the problems with master-slave replication

Master slave replication alleviates the bottleneck of single node performance and storage , What's the problem ?

• There will be a delay in the master-slave replication architecture , As soon as possible , Also have , Especially when the amount of data and concurrency is large ; There is no good way to deal with the delay in the current master-slave architecture ,MySql、SqlServer So it is with ;• Be overdue Key The data in the early version can't invalidate the data from the server in time , Can upgrade to redis3.2 And then solve it , Because the expiration judgment is added ;• After the master node goes down , You can only manually reconfigure the master-slave relationship , From the server can execute slaveof no one Command rewriting back to the master server role , Then reconfigure the master-slave relationship ;• If there are too many slave nodes , When you start to initialize data full synchronization or multiple slave nodes are disconnected and reconnected , It will cause the master node to IO A sharp increase ;

summary

Master slave replication drills seem simple , But we still need to learn from practice , With relevant configuration parameters , This makes it more suitable for application scenarios ; The configuration is not fixed , It's about whether the application scenario is appropriate . With the failure of the master node, there is no automatic election , Next time, on this basis, talk about sentinel mode , Let automatic elections not be a problem ;

A handsome guy made ugly by the program , Focus on "Code Variety industry ", Study with me ~



版权声明
本文为[osc_ jxamvmuz]所创,转载请带上原文链接,感谢
https://cdmana.com/2020/12/20201225122829151W.html

Scroll to Top