编程知识 cdmana.com

Dubbogo 3.0: hand in hand with grpc to the cloud native Era

author |  Li Zhixin Yu Yu

since 2011 year Dubbo After open source , Adopted by a large number of small and medium-sized companies , Has always been the most popular in China RPC frame .2014 year Due to the adjustment of Ali's internal organizational structure ,Dubbo Maintenance was suspended for a while , Then, as Spring Cloud The advent of , The integration of the two systems has boosted the popularity of microservices .

But the world is changing fast , Since docker As a representative of the container technology and with k8s After taking the stage for the representative container arrangement technology , The age of cloud primordial has arrived . In the age of cloud Nativity , Immutable infrastructure brings immutable middleware infrastructure to the original middleware :gRPC Unified the underlying communication layer ,protobuf Unified serialization protocol , With envoy + istio As a representative of the service mesh Gradually unifies the service control plane and the data plane .

dubbogo Its natural mission is :Bridging the gap between Java and Go. keep Go Application and Java At the same time of application Interconnection , With the help of Go Language ( In fact, the first cloud native language ) The advantages of the cloud embrace the original era .dubbogo Community 2020 Years of hard work to build three arrows : Alignment that has been released dubbo 2.7 Of dubbogo v1.5 edition 、 To be released in the near future sidecar In form dubbo-go-proxy project as well as In progress dubbogo 3.0.

In a word dubbogo 3.0 That is : New communication protocol 、 New serialization protocol 、 New application registration model and new service governance capabilities ! This paper mainly discusses dubbogo 3.0 The new communication protocol and application level service registration discovery model of .

1. dubbogo 3.0 vs gRPC

Enemy and know yourself , Only then can we make progress .dubbogo 3.0 The improvement of communication layer mainly draws lessons from gRPC.

gRPC agreement , In a nutshell http2 On the basis of the agreement , Specific protocols have been added header:“grpc-” At the beginning header Field , Use specific unpacking tools (protobuf) Serializing data , So as to achieve RPC call .  picture .png

as everyone knows ,gRPC Almost no service governance capability , And Alibaba cloud has dubbo The framework has both RPC And service governance capabilities , The overall strength is not inferior to gRPC. but “ We all use gRPC” Against this background ,dubbogo 3.0 The new communication protocol must be Perfect compatibility gRPC, Fully compatible with services deployed by developers , And on this basis continue to have dubbo Protocol and service governance capabilities , And then a series of new strategies : such as mesh Support 、 Application level service registration, etc .

2. dubbogo 3.0 vs dubbogo 1.5

What we have now dubbo 2.7 The agreement has been implemented as much as possible gRPC Support for . Developers can use protoc-gen-dubbo Tools will pb IDL Protocol conversion to framework supported stub, And then with the help of the bottom gRPC conn Of RPC The process , Transfer existing service governance capabilities from top to bottom to gRPC, So it's realized gRPC Service support .

dubbo-go v1.5.x Also support gRPC Of Stream call . and unary RPC similar , Supported by a framework stub, In the underlying gRPC stream Call based on , Will flow RPC The ability and framework to incorporate . But because of dubbo v2.7.x / dubbo-go v1.5.x It doesn't support streaming calls by itself , So there's no right gRPC stream Call the upper layer of service governance support .

The problem for developers is : We are using dubbo-go2.7 Conduct grpc When the protocol is transmitted , More or less, it's not that reassuring .

And the upcoming dubbo-go 3.0 The agreement will address the problem at its root .

3. Three levels of protocol compatibility

The author thinks , The support of a service framework for third party agreements can be divided into three levels : Application level 、 Protocol level 、 Transport layer .

A framework if it's in an agreement sdk Encapsulate the interface on top of , It can be considered as an application level support , Such a framework needs to follow the lower level sdk The interface of , Poor scalability .

Framework at the protocol level , From the configuration layer to the service governance layer, the framework provides , The protocol layer below this layer to the network transport layer uses a fixed communication protocol , Such a framework can solve the problem of service governance , But the framework itself cannot fully fit into third party agreements , If not, there will be a weakening of support for third-party agreements , Like the one above dubbo-go 1.5 Yes stream rpc Defects in support .

If you want to further support more third party agreements , You need to start at the transport layer , Really understand the specific fields of the third party agreement 、 The underlying protocols that depend on ( such as HTTP2) Frame model and data stream of , And then develop the data interaction module which is completely consistent with the third party protocol , As the bottom layer of this framework . The advantage of this is that it gives the protocol maximum scalability , On the basis of compatibility with existing protocols , Optional addition of fields needed by developers , So as to realize the function that the existing protocol can't realize , Like dubbogo 3.0 Will support the counter pressure strategy .

4. be based on HTTP2 Communication process of

gRPC Once based on HTTP2 Of unary rpc The main process of call transmission is as follows :

  • client send out Magic Information PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n
  • server Receive and check for correctness
  • client and server Send each other setting frame , Send... When received ACK confirm
  • client send out Header frame , contain gRPC Agreement field , With End Headers As Header End mark
  • client And then send Data frame , contain RPC Called request Information , With End Stream As Data End mark
  • server Call the function to get the result
  • server send out Header frame , contain gRPC Agreement field , With End Headers As Header End mark
  • server And then send Data frame , contain RPC Call the returned response Information
  • server And then send it again Header frame , contain RPC State and message Information , With End Stream As this time RPC Call end flag

It includes gRPC Call information of HTTP2 Header The frame is shown below :

in addition , stay gRPC Of stream in call , Can be found in server Send many times in the process of end return Data, Send after the call is finished Header End RPC The process , And report status information .

dubbogo 3.0 The communication layer of HTTP2 The same communication process is used on the communication protocol , To guarantee and gRPC Communication skills at the bottom of .

5. dubbogo 3.0 Expected communication architecture

Except for the communication protocol HTTP2 Outside ,dubbogo 3.0 Will adopt be based on google protobuf Of triple agreement 【 It's called dubbo3 agreement 】 As dubbogo 3.0 The serialization protocol of , by dubbo In the future, more programming languages will be supported to lay the foundation of communication protocol level . Currently designed dubbogo 3.0 The transmission model is as follows :

  • To ensure simultaneous support for unary RPC and stream RPC, stay server End sum client Add data stream structure in the end , Complete data transfer in the form of asynchronous calls .
  • Continue to support the original TCP Communication capability
  • stay HTTP2 On top of the communication protocol of dubbo3 agreement ,decode Process compatible gRPC Used protobuf, Guarantee and gRPC The service gets through .

6. Application level service registration discovery

6.1 Introduction to application level service registration discovery

dubbogo 3.0 Using the next generation service registration discovery system , The old version of “ Interface level registration discovery ”, Use “ Discovery application level ”.

In short , Interface level registration discovery , In the registry with RPC The service is key, Take the instance list as value To organize data , And what we've introduced is “ Application granularity service discovery ”, It takes the name of the app (Application) As key, A set of instances deployed with this application (Instance) List as value. This makes two differences :

  1. The data mapping relationship has changed , from RPC Service -> Instance Turn into Application -> Instance
  2. There's less data , The registry is gone RPC Service And related configuration information

It can be said that , The amount of data stored and pushed by the model based on application granularity is the same as that of the application 、 The number of instances is proportional to , Only when the number of our applications increases or the number of instances of our applications increases , Address push pressure will rise .

For the model based on interface granularity , The amount of data is positively related to the number of interfaces , Given the fact that an application usually publishes multiple interfaces , Its order of magnitude is generally tens of times larger than the application granularity . Another key point is , The definition of interface is more about the internal behavior of business side , The opacity of cluster size evaluation caused by interface granularity , Examples 、 Application growth is usually in the planning of operation and maintenance side , Good controllability .

The industrial and Commercial Bank of China used to calculate these two production models : The application level service registration model can change the amount of data on the registry to the original 1.68%, The new model can make zookeeper Easy to do 10 Ten thousand levels of service and 10 Ten thousand nodes .

6.2 The introduction of metadata center synchronization mechanism

The result of less data in the data center , yes RPC Service related data disappears in the registry , Only application - instance These two levels of data . To ensure that this part is missing RPC Service data can still be Consumer Right perception , We are Consumer and Provider A separate communication channel has been established , At present, there are two specific options for metadata synchronization , Namely :

  • The built-in MetadataService.
  • Independent metadata Center , Coordinate data through metadata clusters refined in .

6.3 Compatible with older versions dubbo-go

In order to make the whole development process for the old dubbo-go Users are more transparent , At the same time, avoid specifying provider The impact on scalability ), We designed a set of RPC Mapping relationship between service and application name , To try in consumer Done automatically RPC Service to provider Application name conversion .

This design allows dubbogo 3.0 At the same time, keep to dubbo v2.6.x、dubbo v2.7.x and dubbo v3.0.x Three big versions are interconnected .

7. Unified routing support

The concept of routing can be understood as from all the existing IP In the address list , According to specific routing rules , Pick out what you need ip Address subset . The routing process needs to be filtered according to the configured routing rules , Finally, take the intersection of all routing rules to get the result . Multiple routes are like pipelining , Form a routing chain , Filter the final destination address set from all address tables , Then the access address is selected by load balancing strategy .

7.1 Routing chain

The logic of routing chain can be simply understood as target = rn(...r3(r2(r1(src)))). For each of these router Internal logic , It can be abstracted as the input address addrs-in And router According to the full address addrs-all To achieve segmentation good n individual Disjoint The address pool addrs-pool-1 ... addrs-pool-n Take the intersection as the output address according to the rules defined by the implementation . And so on , Complete the calculation of the whole routing chain .

7.2 failover**

** In the routing rule configuration file, you can configure failover Field . When the address search fails, you can failover, Choose other subset , And carry it out in order , Until we find the address , Otherwise, the last address can not find the exception . ** 7.3 Bottom route ** In the routing rule configuration of , You can configure one without any conditions match, The end result is at least one subset Selected to , In order to achieve the function of address null protection .


As 2020 year dubbogo Community building and will be in 2021 One of the three arrows that came out at the beginning of the year ,dubbogo 3.0 It will bring a different and new development experience , Please look forward to !

If you have any questions , Welcome to join the communication group 【 Nailing group no. 31363295】:

Cloud native - Middleware teams recruit for dubbo3 (java & go)、dapr、arthas Interested developers . It can be nailed in northlatitude.

Author's brief introduction

** Li Zhixin  **(GitHubID LaurenceLiZhixin), Alibaba cloud cloud cloud native middleware Team Development Engineer ,dubbogo Community developers , Students majoring in software engineering in Sun Yat sen University , Good at using Go Language , Focus on cloud native and micro services and other technical directions . Yu Yu (github @AlexStocks),dubbo-go Project and community leaders , A programmer with more than ten years of working experience in server infrastructure , Have been involved in improving Redis/Muduo/Pika/Dubbo/Sentinel-go And other well-known projects , At present, I am engaged in container arrangement and service mesh Work .

版权声明
本文为[Dubbo go open source community]所创,转载请带上原文链接,感谢
https://cdmana.com/2020/12/20201224214750770j.html

Scroll to Top