编程知识 cdmana.com

How to deal with the big data impact of double 11! Does the server crash first or does the programmer crash first?

Double eleven is coming ！ In a big traffic situation like double eleven , Rush purchase 、 The order will be very large , If the load capacity of business application system is limited , Unexpected requests , It puts a lot of pressure on the system , So as to bring down the business application system .

that , In the face of heavy traffic , How to control the flow ？

Traffic control strategy of service interface ： shunt 、 Downgrade 、 Current limiting, etc. . This paper discusses the lower limit flow strategy , Although it reduces the access frequency and concurrency of the service interface , But in exchange for high availability of service interface and business application system .

* Current limiting algorithm

The common current limiting algorithms are leaky bucket algorithm 、 Token bucket algorithm .

1、 Leaky bucket algorithm

Leaky bucket （Leaky Bucket） The algorithm is very simple , water ( request ) Enter the leaky bucket first , The leaky bucket comes out of the water at a certain speed （ The interface has a response rate ）, When the water inflow speed is too high, it will overflow directly （ Access frequency exceeds interface response rate ）, And then reject the request , It can be seen that the leaky bucket algorithm can forcibly limit the data transmission rate .

The schematic diagram is as follows ：

So there are two variables here , One is the size of the barrel , Support how much water can be stored when the flow suddenly increases （burst）, The other is the size of the hole in the bucket （rate）.

Because the leakage rate of the leaky bucket is a fixed parameter , therefore , Even if there is no resource conflict in the network （ No congestion ）, The leaky bucket algorithm can't burst the stream （burst） To port rate . therefore , Leaky bucket algorithm is inefficient for the traffic with burst characteristics .

2、 Token bucket algorithm

Token bucket algorithm （Token Bucket） and Leaky Bucket Algorithm with the same effect but opposite direction , Easier to understand .

Over time , The system will press constant 1/QPS The time interval （ If QPS=100, The interval is 10ms） Add... To the barrel Token（ Imagination is the opposite of leakage , There's a tap that keeps adding water ）, If the bucket is full, it will not be added . When a new request comes , Take one of them Token, without Token Can take it to block or refuse service .

Another advantage of token bucket is that it can change speed conveniently . Once you need to speed up , Increase the rate of tokens placed in the bucket as needed . It's usually timed （ such as 100 millisecond ） Add a certain number of tokens to the bucket , Some variant algorithms calculate the number of tokens that should be added in real time .

* ReteLimiter Complete current limiting 、 The rush purchase scenario is realized

RateLimiter yes Guava The implementation class based on token bucket algorithm is provided ,API Easy to use , And adjust the generation according to the actual situation of the system Token Rate .

maven introduce :

Code :

result :

rateLimiter.acquire() Return the waiting time of this execution , We can see clearly that , No need to wait for the first time , It will take two seconds to execute ,

rateLimiter.acquire() This method blocks the thread , Continue to execute downward until the token can be retrieved in the token bucket .

*  Rush to achieve 1

We simulate flash buying , All in all 35 Commodity , Simulation our server can only withstand 20 concurrent , exceed 20 Servers will hang up , We use it RateLimiter To limit the flow ,

We didn't pass parameters , Of course, the actual scene is a lot more complicated .

Use Jmeter Open test 100 Threads to test ：

Statistics of the success of the purchase of goods data , Is precisely 35 We've been able to buy .

We see that the request to enter at the beginning does not need to wait , Direct execution , The waiting time for those coming in later is getting longer and longer , Because I can't get the token .

But this method has one obvious drawback , It is not suitable for use in practice , Because the user asked to come in , If you can't get the token, you need to wait for execution , The experience was terrible . So here's another way .

*  Rush to achieve 2

because RateLimiter How many tokens are generated in unit time , for example 0.1 Second generation 1 individual , It's up to luck , You just happened to be in the middle of something 1 In an hour .

Then you can grab , Here 0.1 The other requests in seconds are blind , Can only hope for the next 0.1 second , And in terms of user experience , He can't be stuck there waiting .

So you need to judge quickly , The user during a certain period of time , Do you have a chance to get a token , This is where we need to use it TryAcquire（long timeout、TimeUnit、Unit） Method , Specify a timeout period , Once it is determined that timeout The token can't be obtained in time , Just go back to false.

Be careful , It's not really waiting here timeout Time , It's judged to be even after timeout Time , You can't get a token . There's no need to wait for this .

result ：

Statistics in all our log files ：

 Panic buying ：35;   Panic buying failed ：23;   Can't get the token , Not involved in rush purchase ：42;   The total is us 100 Threads .

* summary

There are many ways to achieve this , In a rush to buy 2 In the case , Divide according to fixed unit time , Each unit time generates a token , Available for purchase .

We can think of the time when we usually participate in the rush to buy seconds , Sometimes you have a fast Internet connection , But I can't get it , Now I see .

The real rush is not that simple , The difficulty is much more complicated . So it's not just in code that streaming can be implemented .

The instantaneous traffic peak will break down the load of the server , When hundreds of thousands of people rush to buy several goods , The connection port can't come in , Not to mention token allocation in the interface .

If you want to learn programming , Become a good programmer

↓ ↓ ↓

—— 【 Worthy of attention 】 Programmer Technology Exchange Club ！ ——

involves ：C/C++、windows Programming 、 Network programming 、QT Interface development 、Linux Programming 、 Game programming 、 Network security and so on ......

Scroll to Top