Abstract ： For businesses AI The pain points and difficulties in development and application , To bring you from practice to help enterprises build a mature and efficient AI Develop process solutions .
In today's digital transformation wave sweeping the world ,AI Technology has become a recognized upgrade focus in the industry , Is bringing innovative value to the business in more and more areas . With the industry AI The landing accelerated gradually , Enterprises are developing AI In the process of application , There is often a lack of preparation 、 There is a shortage of talents 、 High development costs and other issues , Hindering AI The implementation practice of .
stay 2020 year 12 month 20 Japan QCon Global software development conference 2020 Shanghai station , Hua Wei Yun AI Chen Liang, chief technical preacher Join hands with four Huawei AI Doctor in the field , It's called “ The whole stack, the whole scene AI Application development practice ” Sharing of special topics . This topic is for enterprises AI The pain points and difficulties in development and application , Starting from practice, help enterprises to build mature and efficient AI Development process .
This paper summarizes the contents shared by Dr. Wang Jun and Dr. Bai Xiaolong .
MindSpore： The end-to-side cloud unifies training and reasoning AI Computing framework
First , Huawei advanced AI Algorithm Engineer Dr. Wang Jun deeply analyzes Huawei MindSpore AI Computing framework .
stay AI field ,AI Computing frameworks play a very central role . Over the past eight years ,AI The framework has gone through three stages of development ： The first stage mainly lays the theoretical foundation , In the second stage, distributed training is introduced 、 Calculation chart 、 Dynamic graph and other concepts ,TesnorFlow and PyTorch The two frameworks also occupy the leading position in the industry at this stage .
Now AI The development of the framework has gradually entered the third stage , But the industry is still exploring the specific evolution direction of this stage . It comes down to , Industry vs AI The requirements of the framework are mainly divided into user mode 、 There are three levels of operation state and deployment state .
- User mode . This level mainly determines how users use it , It can be divided into dynamic graph and static graph . Static graphs are efficient , But not friendly to developers ; Dynamic graphs are easy to program , But the underlying performance will be much slower . Users hope to combine the advantages of the two modes as much as possible , This is a problem for the framework .
- Running state . At this level , Users want the framework to run as efficiently as possible . Now , One of the major difficulties facing the industry is the specific way to implement distributed training with the segmentation model . The goal of distributed training is to make full use of large-scale computing resources to improve model performance , But how to segment the model reasonably , Ensuring the efficiency of distributed training is a time-consuming and laborious task , It brings obstacles to the scale expansion of the model .
- Deployment state . In the process of model deployment , On the one hand, users hope that the algorithm can be well implemented and produce results , On the other hand, we don't want the privacy and security problems of user data . Therefore, the framework needs to ensure data security while paving the way for implementation .
For the above needs , Huawei has developed MindSpore AI Computing framework , Through many innovations, it has solved a series of problems that the traditional framework can't solve .
Facing the contradiction between dynamic graph and static graph in user state ,MindSpore Introduced the name MindIR The middle representation of the layer .
IR（Intermediate Representation） It is a translation intermediary between source code and object code , well IR advantageous to AI Model compilation, optimization and execution , yes AI Framework is the basis of efficient training and reasoning .MindIR The technical route chosen is Functional Graph IR, from ParameterNode、ValueNode and CNode Make up a directed acyclic graph . stay MindSpore Automatic differentiation ,MindIR Calculation ParameterNode and CNode The gradient contribution of , And return to the final ParameterNode Gradient of , Instead of computing ValueNode Gradient of .
MindIR You can make dynamic or static graphs into IR Express , So that developers can combine dynamic diagram and static diagram in the development process , Switch to use at any time as needed .
In the running state , To help developers free themselves from the heavy task of manually slicing models ,MindSpore Automatic parallel scheme is provided .
This scheme does not need to change the user code structure , After the user enables the scheme, the framework will automatically search for parallel policies , And then the segmentation operator , Finally, the whole graph is segmented .
The framework generates segmentation strategies based on automatic planning , In the process, we will try to cut off some obviously infeasible strategies , Finally, the whole graph segmentation is generated . The goal of the overall strategy is to maximize the computational communication ratio , Try to spend more time on forward calculation and reverse calculation , Instead of the communication between cards . The segmentation strategy also supports users to manually adjust , Both manual and automatic modes can be used in combination . This solution also supports optimizer parallelism , Finally, it can significantly reduce the extra memory cost of distributed training .
For the purpose of further improving the performance of the algorithm ,MindSpore In addition, the second-order optimization and the fusion strategy of graph calculation are introduced .
Training deep learning models requires a lot of computation , Training convergence time is longer . Second order optimization method can effectively accelerate the convergence of the model , Reduce the number of iterations , But at the same time, a lot of complex computation will be introduced , The depth of its application in training is limited . So for computing frameworks , How to solve the second order matrix efficiently is a technical difficulty .MindSpore The way to deal with it is to inverse the matrix , Consider only the elements on the diagonal and the influence in each layer . At the same time, the frame doesn't do full second order , The first-order and second-order optimization will be carried out alternately , Determining the second order computation based on dynamic results . Based on these optimizations ,MindSpore It saves end-to-end training time 20%.
Operator fusion is AI Framework common optimization methods . Modern computer hardware architecture adopts multi-level memory system , Data operation requires step-by-step input and output . Operator fusion is to merge the computing logic of adjacent operators , Save the number of input and output step by step , So as to improve time efficiency .MindSpore It also introduces the integration of graphics and computing . In the end 15% End to end performance improvement .
In the hardware layer ,MindSpore Native support for high performance ascension AI chip , And according to the technical characteristics of shengteng, the software and hardware are optimized . Under the joint action of a variety of optimization measures ,MindSpore+ The solution of shengteng chip is based on V100 The plan of the project has improved 70%-80% Performance of , And in 5 In January, the industry was acquired AI The number one result on the list .
In deployment state , In order to AI The model is deployed to the end side to achieve end cloud collaboration ,MindSpore Provides lightweight MindSpore Lite edition .Lite Version and cloud full version use a unified MindIR, But on the end side, the hardware of the end side is optimized . These optimizations include some operator substitution （ Replace some heavy computation with some light computation ）, I'll also do some operators to move forward and backward , Avoid redundant computing . For a single operator , Will try to lift it on the end side CPU Operating efficiency on .
These measures combine to achieve good performance . In huawei P40 On the cell phone ,MindSpore Lite Most of the performance metrics of are better than TF Lite.
Now ,MindSpore Open source community ecology is developing rapidly , It is playing a more and more important role in the domestic developer community .MindSpore Open source this year , The code is hosted on the code cloud platform . at present MindSpore The published 1.0 Commercial version , Coming out at the end of the month 1.1 edition . The new version will support 40 Multiple generic models , go online 270 Multiple applications .MindSpore There are also many interest groups in the community to exchange various topics , New members are also welcome to continue to join .
ModelArts： Speed up the whole process AI Application development and deployment
With AI Computing framework , The next step is to have a set of end-to-end AI Development platform to achieve AI Fast landing of applications . Hua Wei Yun ModelArts Architect Dr. Bai Xiaolong shared Huawei cloud ModelArts "One-stop" work style AI Development platform and key technologies , The application case .
AI The core process of development mainly includes preparation （ The project design ）、 Data preparation 、 Algorithm selection and development 、 model training 、 Model evaluation and tuning 、 Application generation / Evaluation and release 、 Application maintenance sub process . Each sub process involves a lot of complex work , Cost of existence 、 The threshold 、 The challenges of efficiency and credibility .
Due to the present AI The number of developers is far less than the number of software developers , How to let more people start in a low threshold way AI Development 、 to ground AI Application has become a core issue . In all subprocesses , Hua Wei Yun ModelArts By providing a large number of preset algorithms and frameworks , as well as WorkFlow Arrangement ability , Minimize AI The threshold of application development , And it speeds up AI The whole process of application development and deployment . Hua Wei Yun ModelArts The whole platform is divided into two parts: the basic platform and the upper preset suite , Yes, yes ModelArts Pro Make end-to-end industry applications AI Easier to develop . among , The basic platform covers AI Development 、 The capabilities required for the deployment and trading process , The preset suite mainly provides immersion with certain generalization ability for specific application scenarios AI Engineering and WorkFlow Templates , Including vision related for intelligent perception WorkFlow、 Intelligent cognition oriented NLP dependent WorkFlow、 For decision intelligence WorkFlow etc. . Take the object detection task in the field of computer vision as an example , Users don't need to AI Domain knowledge , Just a few simple steps （ Upload data 、 Annotation data 、 Training 、 test ） You can complete a AI application development , In a way that's low code or even zero code AI Application development threshold .
When developers need to go deep into AI When applying the sub processes of development ,ModelArts Provides a number of key technologies to help reduce costs and increase efficiency , And lower the threshold .
1） Data preparation subprocess
- Intelligent data enhancement ： New trainable data can be automatically generated based on cross domain migration , Thus, the workload of data collection and data annotation is reduced .
- Intelligent data filtering , Automatic filtering is about 40% The dirty data . Take parking space recognition as an example , The platform can automatically screen out obstructions 、 Overexposure 、 Too dark and so on , Save subsequent development costs .
- Intelligent annotation ：1） Support is based on semi supervised / Hybrid intelligent tagging for active learning , It can mix unlabeled and labeled data for training , Reduce the number of iterations required for active learning , get 5 Double labeling efficiency and improve labeling accuracy . The platform will also automatically generate optimization suggestions , It is difficult to judge the sample , It is suggested that the data collection should be more optimized and the data enhancement should be more difficult .2） Support interactive intelligent annotation , Take target detection as an example , Just confirm without drawing a bounding box , Take image segmentation as an example , There is no need to manually label the outline by polygon , Only a few points need to be marked to realize automatic contour recognition . This intelligent interactive ability will greatly reduce the amount of bidding .
- Intelligent data analysis , According to the brightness of the image 、 saturation 、 The resolution of the 、 Complexity 、 Color and other information to give analysis suggestions . This will be a comparative analysis of the subsequent training set and verification set 、 Super parameter selection provides quantitative basis .
2） Algorithm development and selection of subprocesses
- Algorithm to choose ： To solve a problem AI Application problems , Most Algorithm Engineers or data scientists don't develop a new algorithm from scratch , Instead, choose the existing algorithm , Do a quick experiment , And then continue to improve iterations . In order to achieve real inclusion AI Development , bring AI The choice of algorithm is more convenient ,ModelArts Offers dozens of high performance 、 High precision preset algorithm library , It can be used directly without any manual modification . Most algorithms are more accurate than open source versions 0.5%~6%, High performance 30%~100%. The bottom layer of some preset algorithms is based on self-developed EI-Backbone Pre training model , It can improve the accuracy of multiple data sets . adopt AI Asset sharing trading center ——AI Gallery, Users can purchase or subscribe to these excellent preset algorithm assets , You can also upload and contribute new preset algorithms .
- Algorithm development ： If the existing preset algorithm can not meet the requirements , Developers can use ModelArts Provided Serveless turn Notebook development environment , Compared to the existing development environment in the industry , The cost of development and debugging can be reduced several times , And in Notebook in ,ModelArts Platform preset a large number of algorithm development examples , Developers can make secondary development based on these samples .
3） Model training sub process
- Training accelerates ：ModelArts Built in training acceleration framework MoXing, The bottom layer is connected with common computing engines （ Such as MindSpore、TensorFlow、PyTorch etc. ） Through a series of optimization measures （ data 、 Calculation 、 Training strategy 、 Optimizer 、 Distributed communication, etc ） To improve the training performance of the model , Maximum support 4096 Card distributed training , Training performance is No.1 in the industry . The platform also implements the same set of algorithm code, which is compatible with single card and multi card , Also support in different AI Switching between devices , Users can change a configuration item from training mode to verification mode or prediction mode .
- Flexibility training ： When multiple users share training resources , It can allow users' training tasks to achieve dynamic scaling of resources under the premise of unchanged model accuracy . In this way, when the whole cluster is idle , You can expand the existing training assignments , The training speed is doubled . The user is in ModelArts When you train, you have high performance 、 Standard and economy are three models to choose from . The economic model will reuse the underlying layer as much as possible , The cost can be greatly reduced .
- Federal training ： In order to ensure data security during training ,ModelArts It also supports the federal training program , A self-developed federated aggregation algorithm is proposed FedAMP, And through adaptive methods , Make federations between users with similar data stronger .
4） Model evaluation and tuning subprocess
- Multiple search ： Developers just need to modify 3 One line of code improves accuracy or performance ,ModelArts It can also be aimed at the homework with longer training time , Implementation of fast parameter adjustment strategy , Greatly reduce the search time of hyper parameters . It also provides visualization ability to sort multiple search results according to various indicators, which is convenient for developers to select quickly .
- Model evaluation and intelligent diagnosis ：ModelArts It can automatically give tuning suggestions for the accuracy or performance of the model . Besides , For the robustness of the model , The platform has built-in robustness tools , It can automatically identify the security problems existing in the algorithm and give suggestions for improvement and optimization .
5） Application generation 、 Evaluation and release sub process
- Model arrangement and reasoning framework ： The cost of model reasoning is high 、AI The development process is difficult to close loop ,ModelArts Platform introduced Serveless End to side cloud hybrid deployment and self-developed high-performance reasoning framework . The reasoning framework at the bottom of the platform can automatically generate reasoning diagrams according to the operators described by users ; When the model is deployed in the cloud , The platform supports automatic elastic scaling and dynamic loading , In edge deployment, one click distribution is supported .
6） Application maintenance sub process
- Data collection and difficult case diagnosis ：AI After the model goes online ,ModelArts The platform can automatically collect model running data , Identify and filter out data drift , Automatically generate diagnostic recommendations .
On the whole, Huawei cloud ModelArts The key technologies are summarized as follows ：
Hua Wei Yun ModelArts Since release , In the park 、 Industry 、 City 、 traffic 、 family 、 water 、 Environmental protection and other fields are widely used .ModelArts The platform can well meet the needs of the whole scene AI Application requirements , Including intelligent perception 、 There are mature practices in the fields of intelligent cognition and intelligent decision making . Typical examples include automatic bread recognition and checkout machines in bakeries 、OCR Word semantic recognition 、 game AI（ Smart vs ）、 Drug production and so on .ModelArts The end cloud collaboration scheme has also been implemented in scenarios such as parking and self payment .
HiLens： End cloud collaboration multimodality AI Application development and practice
The development of artificial intelligence is experiencing from perception to cognition , From single mode to multimode , The evolution from cloud side to end cloud collaboration . Dr. Xia Fei shared Huawei's AI algorithm HiLens Platform development end cloud collaboration multimodality AI Application and landing practice .
Huawei HiLens End cloud collaboration multimodality AI Develop application platform , Provide a simple and easy-to-use development framework 、 Out of the box development environment 、 rich AI Skill market and cloud management platform , Connect a variety of end-side computing devices . Get through the online and offline development and deployment process , Developers only need to complete multimodality once in the cloud AI application development , It can be deployed to the cloud side , You can also deploy to the end side and the edge side , It can solve the cloud collaboration of enterprise development end well AI Application requirements .HiLens The platform can also easily access multimodal data , And support low code development process , Developers only need a few simple steps to complete the end cloud collaboration AI Application development and deployment . at present ,HiLens Already in the smart store 、 Smart Family 、 Wisdom Education 、 Smart transportation and other end cloud collaborative scenarios play a role .
ModelArts Pro In the industry, multimodality AI Application practice of development
With the industry AI The landing accelerated gradually , In the industry, multimodality AI There are also many challenges in development , For example, the industry has high business complexity ,AI Application development needs rich industry experience ; Custom code is a lot of work , Time consuming , Can't get the business online quickly ; Universal AI Have limitations , Industry specific business processes are not supported . Hua Wei Yun EI Dr. Zhu Shenggao, director of agent technology, shared how to use Huawei cloud ModelArts Pro Enterprise class AI Application Development Suite acceleration AI Scene landing .
Hua Wei Yun ModelArts Pro It's for the enterprise class AI Professional development platform created by application , The platform is in the form of preset workflow and function suite , Precipitated the industry AI Application of practical scenarios Know-How, Make the enterprise realize zero base 、 Zero code fast AI Development . in addition , Enterprises can also flexibly arrange new industry workflow according to their needs , be based on AI market , Users can also share different industry scenarios with each other AI workflow .ModelArts Pro With “ Teach them to fish ” The way to help enterprises build AI Ability , Empower... In different industries AI Application Developer , Give Way AI Become accessible .
Logistics Oriented 、 oil 、 retail 、 Finance 、 Medical care 、 Traffic and other industry scenes , Huawei cloud is based on leading algorithms and industry knowledge , The first launch of ModelArts Pro Character recognition 、 Vision 、 natural language processing 、 Knowledge map and other development kits and end cloud collaboration multimodality AI Application Development Suite HiLens, And identify it in the self-service form 、 Food type identification 、 Well logging reservoir identification and other business scenarios , Improve the industry in an all-round way AI Development efficiency and industry landing effect , Bring a whole new industry AI Landing mode .
Solve the industry landing pain point , advance AI extensive use ： Huawei cloud joins hands with developers to welcome Pratt & Whitney AI Time
Four Huawei AI In depth sharing of field doctors , For developers, it analyzes the full stack and all scenarios of Huawei AI Many advantages and capabilities of the development platform , And solve the problem of enterprises in AI There are many concerns and problems in the process of application practice . In huawei AI Computing framework 、 Supported by development platform and Development Suite , Enterprises can quickly develop AI application , Embedded business processes easily , And we can see the actual effect and return in the short term .
Based on computer vision 、 Voice semantics and decision optimization AI In the field of long-term basic research and various industries AI Project practice , Huawei cloud AI Continuous innovation in the field of Technology , Constantly introduce innovative ability to meet market demand , And keep pushing AI Develop ecological construction , Pratt & Whitney and developers meet together AI In a new era .
本文为[Huawei cloud developer community]所创，转载请带上原文链接，感谢