编程知识 cdmana.com

To get started with user-defined tbe operator, you might as well start with single operator development

Abstract : Take single operator development as an example , Let you know the whole process of operator development and testing .

Why customize operators

Deep learning algorithm is composed of computing units , We call these computational units operators (Operator, abbreviation Op). An operator is a mapping from function space to function space O:X→X; In a broad sense , An operation on any function can be regarded as an operator . For us , The operator we developed is a computational function involved in the network model . stay Caffe in , The operator corresponds to the computational logic in the layer , for example : Convolution layer (Convolution Layer) Convolution algorithm in , It's an operator ; Fully connected layer (Fully-connected Layer, FC layer) The weight summation process in , It's also an operator .

Ascend Model transformation navigation

In most cases , Because of the rise AI The software stack supports most operators , Developers don't need to develop custom operators , Just provide a deep learning model file , Through the offline model generator (OMG) The offline model file can be obtained by transformation , So as to further use the process choreographer (Matrix) Build specific applications . In that case , Why do you need custom operators ? This is because in the process of model transformation, the operator does not support , For example, shengteng AI The software stack does not support operators in the model 、 Developers want to modify the calculation logic in existing operators 、 Or developers want to develop their own operators to improve computing performance , At this time, it is necessary to develop a user-defined operator .

TBE Operator development process

Rise AI The software stack provides TBE Operator development framework , Developers can use... Based on this framework Python Language development custom operators . First , Let's find out what it is TBE.TBE The full name is Tensor Boost Engine, Tensor acceleration engine , It is an operator development tool developed by Huawei , Can be used in development NPU(Neural-network Processing Unit: Neural network processor ) Upper TBE operator , The tool is a well-known open source project in the industry TVM(Tensor Virtual Machine) On the basis of , A set of Python API To implement development activities . In this development practice ,NPU It's about rising AI processor .

adopt TBE There are two ways to develop operators : Domain specific language development (DSL Development ) and TVM Primitive development (TIK Development ).DSL Development is relatively simple , For entry-level developers . Its characteristics are TBE Tools provide automatic optimization mechanisms , Give a better scheduling process , Developers only need to understand neural networks and TBE DSL Related knowledge , You can specify the target generation code , It is further compiled into a dedicated kernel .TIK It's hard to develop , For TVM Programming and Da Vinci structure are very familiar with the use of developers . The interface in this way is low level , The hardware scheduling of data flow and operators should be controlled by developers themselves . As an introductory course , What we're using this time is DSL The development way .

TBE Operator development process

Next , Let's take a simple single operator development as an example , Take a look at the development process .

  • The goal is :

use TBE-DSL How to develop a Sqrt operator

  • Determine the operator function :

Sqrt The function of the operator is to Tensor The root of each atomic value in , The mathematical expression is y=

  • Determine the computing interface to use :

Based on the current TBE Description of the computation that the framework can support API, It can be expressed by the following formula Sqrt The calculation of the operator

The implementation of operator code can be divided into the following steps :

1) Operator input parameter

shape:Tensor Properties of , Express Tensor The shape of the , use list or tuple Type said , for example (3, 2, 3)、(4, 10);

dtype:Tensor Data type of , Using string type to express , for example “float32”、“float16”、“int8” etc. .

2) Input Tensor Place holder

data = tvm.placeholder(shape, name="data", dtype=input_dtype)

tvm.placeholder() yes TVM Framework of the API, It is used to occupy the data received during the execution of the operator , Popular understanding and C In language %d、%s equally , Back to a Tensor object , Used in the above example data Express ; The parameter for shape,name,dtype, Is for Tensor Object properties .

3) Define the calculation process

4) Define the scheduling process

5) Operator construction

6) Test verification

Wait a minute , It's not over yet . Only in the simulation environment can the correctness of the operator function be verified , The development of custom operators is complete .

ST Testing process

We need to use ST test ( namely System Test The system test ) In the simulation environment, test the correctness of operator logic and whether it can be generated correctly .o and .json file . Want to know how to test it ? Instead of looking at a long boring text description , Better come Sandbox lab Experience it for yourself , It must be more intuitive .

Read here , Are you more confident in your custom operator development ability ? Why don't you come to China to study for cloud college 、 Experimentalize 、 Let's verify it with a textual research book ? here , This is the micro certification : Based on ascension AI Processor operator development

→ Click directly to Huawei cloud College ,get More new skills

 

Click to follow , The first time to learn about Huawei's new cloud technology ~

版权声明
本文为[Huawei cloud developer community]所创,转载请带上原文链接,感谢
https://cdmana.com/2020/12/20201225110425811O.html

Scroll to Top