编程知识 cdmana.com

Review of offline Salon of coding Devops II: best practices of SDK testing

lecturer : Pan Zhigang

Head of acoustic network quality and efficiency Department , exceed 14 Year server 、 Mobile terminals 、 Audio and video codec, automotive electronics and other cross industry experience , Responsible for establishing test infrastructure and automated testing solutions , Lead the construction of continuous integration test ecosystem . Now head of the quality and efficiency Department of acoustic network , Responsible for continuous optimization of quality and efficiency , Focus on technology innovation, enabling Team Software Assurance , Through the efficient combination of software and hardware , Explore the best solution for product delivery .


SDK Testing is different from APP test , Not only from the perspective of the end user , You still need to stand on APP From the perspective of developers . In the face of different industry needs , How to ensure the quality is solid , It's a track to explore the unknown . This issue of push will bring you the voice of Pan Zhigang, the director of sound network R & D efficiency 《SDK Testing best practices —— Build an integrated application platform for quality assurance 》, Share the evolution of the integrated application platform and how to integrate the basic capabilities , Ensure efficient execution of testing and delivery , Improve quality and efficiency .

1.0 GUI Driven Test

SDK( Software development kit ) It is the main external product delivery of sound network , Is used for specific software packages 、 software framework 、 Hardware platform and operating system to create application software development application set , With the traditional sense of APP、 The products perceived by peripheral applications or end customers are different , Invisible to end users .

In the early days, in order to protect SDK The quality of the test , Testers need to be based on SDK Delivered API Set up GUI demo. For example, in a real-time Internet communication interface , Users need to join the corresponding channel for corresponding audio and video communication , In this kind of interface, we will design corresponding Button、 The drop-down list , Or small icons , Each corresponding element reflects the corresponding interface implementation capability . As shown in the figure below , At the bottom 4 individual Button It's the microphone 、 camera 、 Hang up button , Corresponding API Interface enableLocalAudio、enableLocalVide、startScreenCapture and leaveChannel, The signal bar icon in the upper right corner is get onNetworkQuality Interface . Through this simple Demo, The tester designs the corresponding test case, Ensure that each interface can be called normally , Based on this to ensure the quality standards delivered in the initial iteration .

However, as more and more platforms are delivered , Delivery needs to be based on the desktop 、 Mobile 、web End , The desktop includes Windows,macOS and Linux, Mobile terminals include Android and ios, More and more platforms are designed accordingly demo It is bound to require testers to invest more resources , meanwhile API It's growing . So automation is the inevitable trend .

2.0 GUI Demo Test Automation

2.0 Stage is GUI Demo Test Automation, Developers layered the platform .

Pictured above , above iOS、OSX、Android Etc. is the platform for external delivery , The following is the third-party open source tools used by the corresponding platform , Such as Appium and Selenium, The middle layer should be sub packed accordingly , Its purpose is to improve the test efficiency , Use a set case Covering all delivered platforms . Realization 70% The degree of automation of the team has been able to save half the time , Greatly improve the test efficiency .

3.0 API Demo Test

3.0 The stage goes to API Demo Test. The test subject of sound network is SDK,SDK The point of concern is API Function realization 、 Platform adaptation 、 For developers 、 Performance power consumption packet size , Integration build packaging ; and APP Focus on business functions 、 User interaction 、 End user 、 Interface operation and program installation . in the light of SDK、APP Two completely different test points , The sound network has redesigned a set for SDK Automatic testing framework of ——Wayang Testframework.

Wayang The principle comes from a puppet show in Indonesia , At the front is a puppet , Backstage performers control the front puppet to do the corresponding action through the line and dexterous hand . There are three different objects in such a system , The object on the left is test client, In the middle is test server, On the right is the corresponding test demo.Test client It's like the performer behind the puppet show , You need to know what your test requirements are , Design corresponding test case;test demo It's like a puppet at the front end , According to the persistent request from the test end, the corresponding behavior call will be made . All active and passive calls are based on code output . In the whole system, all interface calls and corresponding callbacks are based on the output of the code terminal , No need to care about the implementation of the interface . And automation 2.0 And handcraft 1.0 comparison , At present, we can finish more than one hundred per day API test , Automated test coverage can exceed 80%.

4.0 AIO

At the completion of Wayang After practice , Voice network is still thinking about whether there can be further optimization practice . As product delivery cycles get tighter , And more demand intervention , From a testing point of view , It's not enough to think about the testing link , The whole product delivery needs to be taken into consideration —— Including pre build 、 pack 、 test 、 deliver . therefore , Here's the introduction of AIO Idea .

AIO It's a combination of box court and sandbox . that , What are box court and sandbox ? The box court can provide the infrastructure , It is equivalent to providing corresponding traps in the game 、 Enemy or treasure chest , How to win or defeat is up to you . It means opening the sandbox to the smallest unit of users , A typical example is my world 、 Lego , The smallest unit is a cube. The test unit of acoustic network is based on API, So in the whole delivery , Box court is the guarantee of infrastructure (e.g: The Internet 、 Power Supply , Test environment ) Stable operation , As for sandbox, it splits the build 、 test 、 Launch three sections online .

In the integration of sound network AIO Within the framework , Contains a series of corresponding module.

AIO The architecture includes device clusters . Because different platform delivery must cover a variety of situations , The compatibility of different devices needs to be considered . The dispatch center ensures that all equipment is delivered on time in the expected settings , So there needs to be a service gateway . The data center will analyze SDK The product is clear log Output ; The last piece is build release ,ACCS The platform includes compiling 、 Release 、 Crash report 、 Data analysis 、 Automatic test and other functional modules . The following basic capabilities represent the lower level elements , Such as link simulation 、 Physical connection control 、 Human computer interaction, etc .

Back to what I said Wayang Characteristics of , There needs to be one client Corresponding to one demo.Client The performer knows what needs to be done , And then let demo To do something about it . Based on this situation , Sound network has been further improved . adopt API driven test, Voice network set up a separate soloWayang app, Inside test iterator The generator can keep testing API Keep calling . Based on test farm Concurrent test , Run on all devices soloWayang, All the corresponding API Will be tested to ensure that problems are found and addressed .

In the testing section , There will be a lot of data generated , Include SDK data、demo data、test data and server data. How to do reasonable and effective pre mining of these data ?

In the traditional mode , The value of data is to analyze data when problems arise . If you reverse your thinking , If we can collect and pre analyze the data in advance , You can take the initiative to identify and address risks before they are exposed . Sound network data analysis platform through Beats、Logstash Cleaning data from different platforms , Eliminate invalid information .Kibana Through the corresponding filtering , Be able to list out the corresponding problems . for instance , Four hundred devices one night , It is found that a corresponding device appears log abnormal , adopt Kibana You can give early warning , Find out in time whether there is really only one device in this problem , Or there are commonalities in several machines . In the past, whether there was a corresponding problem in mining the data of several devices manually , It's hard to imagine whether it's related to a system 、 It's about a chip , Or is it related to a particular network scene . Through the reasonable filtering of data analysis platform , It can help us find problems effectively through the collection of various evidences , Solve the problem as soon as possible .

Q: For mobile phones APP To do the test. , If you need hundreds of mobile phones connected at the same time , Do a performance test environment . But the support capacity of a computer is limited , It is possible to reach the limit by connecting more than a dozen mobile phones at the same time , How to do scale out and do performance environment ?

A: If it's for Android phones , We have an early node connected at the same time 30 Android devices proved to be feasible , It is recommended to confirm the configuration of nodes and peripherals . More machine connections can be deployed by clustering test farm. in addition , Can be in the corresponding test app Design independent performance test components in the application , It is conducive to the horizontal expansion of performance testing .

Click to get the video and PPT Information

Original statement , This article is authorized by the author + Community publication , Unauthorized , Shall not be reproduced .

If there is any infringement , Please contact the yunjia_community@tencent.com Delete .


Scroll to Top