编程知识 cdmana.com

Autumn wind, modelarts "Ai market algorithm fast SCNN guide" autumn fat paste up

This article is shared from Huawei cloud community 《 I'll bring you some fresh food in autumn | ModelArts AI Market algorithm Fast-SCNN Instructions for use 》, author :Tianyi_Li

Abstract : I'd like to send my friends a fresh one ModelArts AI Market algorithm Fast-Scnn( Hereinafter referred to as the algorithm ) Use the script , Make sure it's easy to get started .

It's double eleven , The autumn wind blows , You can't patronize chopping hands, right , It's time to stick autumn fat , Try something fresh , I sent the fresh ones out of the oven ModelArts AI Market algorithm Fast-Scnn( Hereinafter referred to as the algorithm ), Attached are the use secrets , Make sure it's easy to get started .

This algorithm uses Cityscapes High quality dimensioning data set train Set and val Set , Use train Set training , stay val The test on the set achieves mIOU=68.668 The accuracy of .

1. Prepare the dataset

The data set format supported by this algorithm is Cityscapes Data sets .

Cityscapes The dataset contains data from 50 A variety of stereo video sequences recorded in street scenes in different cities , And the larger 20000 Weak annotation framesets and 5000 High quality pixel level comments for frames . therefore , The dataset is a magnitude larger than the existing similar data set . For more information about annotated classes and examples of annotations, you can find the following Data set official website Found on the .

Cityscapes The data set is designed for :
(1) Evaluate the performance of visual algorithm in the main tasks of semantic city scene understanding : Pixel level , Instance level and panoramic semantic markup ;
(2) Support is designed to take advantage of a large number of ( weak ) Research on annotation data , For example, it is used to train deep neural network .

1.1 Download datasets

can Click on this link. download gtFine_trainvaltest.zip and leftImg8bit_trainvaltest.zip Two documents , As shown in the figure below :

gtFine_trainvaltest.zip The directory structure after decompression is as follows ( This algorithm does not need to decompress itself , Compression packages can be used directly ):

└─gtFine
├─test
├─train
└─val

leftImg8bit_trainvaltest.zip The directory structure after decompression is as follows ( This algorithm does not need to decompress itself , Compression packages can be used directly ):

└─leftImg8bit
├─test
├─train
└─val
If you need to train with your own dataset , You need to sort the data catalog into the same form as above . For a more detailed description of the dataset, see Cityscapes Data set official website Description above .

1.2 install OBS Browser+ And upload the dataset

ModelArts Use object storage services (OBS) To store data , Achieve security 、 High reliability and low cost storage requirements .OBS Browser+ Is a storage service for accessing and managing objects (Object Storage Service,OBS) Graphic tools for , Support perfect barrel management and object management operations .OBS Browser+ The graphical interface of can make it very convenient for users to be on the local OBS Conduct management , for example : Creating buckets 、 Upload and download files 、 Browsing files, etc .

The specific operation steps are as follows :
(1) Download this OBS Browser+, Decompress after download , double-click exe Installation , Run after installation is complete ;

(2) The login interface is shown in the following figure , You are required to fill in the account name 、Access Key ID(AK) and Secret Accsee Key(SK), Reference resources This document , obtain AK and SK, Many products on Huawei cloud need access keys , Please keep the key file properly , Then fill in your Huawei cloud account name and the newly obtained one with reference to the figure below AK and SK, Click login ;

(3) Refer to the below , Click on “ Creating buckets ”, Enter the bucket name , Be careful : The region should choose North China - Beijing IV 、 Standard storage 、 private 、 Close more AZ, Bucket name needs to be customized ,OBS Bucket names should be globally unique , If prompt bucket name already exists , You need to change it to another name , For example, this article sets the bucket name fast-scnn. The bucket name you set must be different from this , See the instructions below fast-scnn, Please take the initiative to replace the bucket name with your own , The following will not be prompted ;

(4) Click on the bucket name , Into the barrel , Click on “ New folder ”, Enter the folder name , Such as “train_input”, Click to enter the folder , New again “datasets”, Click on “ Upload ”->“ Add files ”-> Select the local download of good dataset compression package ( The reason why we choose the compression package format is if the dataset is larger , Uploading compressed packages is much faster than uploading folders ) –> determine , As shown in the figure below ;

(5) Click on OBS Browser+ Left side “ task management ”, You can view the data upload progress . As shown in the figure below , Click Settings , In the basic settings , Set the maximum concurrency number to the maximum value 50, It can speed up data upload

2. Subscription algorithm

Click on the... On the top right of this page 【 subscribe 】 Button . Then click on... Below the page 【 next step 】 Button , Click again 【 confirm the payment 】 Button , Finally, click 【 determine 】 Button to my subscription page , You can see the algorithm you just subscribed to . Click on 【 Application console 】 Hyperlinks , Choose North China - Four areas of Beijing , Enter the algorithm management page .

As shown in the figure below , Click on “ Sync ” Button , Synchronization algorithm , You can click the refresh button , Refresh Status . When the state becomes ready , Indicates the synchronization is successful , Please use the latest version of , It should be 6.0.0 edition , Basic version to subscription .

3. Create training assignments

Click on the image above “ Create training assignments ”, Fill in the training parameters according to the table below :

Click next , Submit , The state of the training assignment will experience “ initialization ”、“ Deploying ”、“ Running ” and “ The successful running ” Four states . After the training operation is successful , Specified in the table above “ Model output ” Path will automatically generate model Catalog , There are model files in this directory 、ModelArts Platform reasoning script (config.json、customize_service.py) And other necessary files for running the model .

4. Model import

After the model and related necessary documents are prepared , You can import the generated model into ModelArts Model management . The specific operation is as follows :

(1) stay ModelArts On the left navigation bar of the console, click “ Model management ” -> “ Model ”, Click on the page on the right “ Import ”. Fill in the name in the import model page , Choose the source of the metamodel , You can choose from training directly ( This method is recommended , Easy and convenient , Seamless connection with training ), You can also get it from OBS Choose from . If from OBS Choose from , You need to choose to model The next level of a directory ; for example , The directory that can be selected this time is obs://fast-scnn/algorithms/train_output, As shown in the figure below :

Be careful : After choosing the metamodel path ,“AI engine ” It will automatically fill . If it fails to fill automatically , Check that the metamodel path is model The next level of a directory , perhaps model Whether the directory contains the model configuration file config.json.

(2) Click on “ Create... Now ”, It takes a little time to wait for model import and build , When the model version status is “ normal ” after , That means the model import is successful .

5. Create online services

stay ModelArts On , Models can be deployed as online services , Then upload the picture to predict , Observe the prediction results directly on the web , This algorithm supports CPU and GPU Deploy .

The specific steps of deployment to online service are as follows :

(1) stay ModelArts Select... In the left navigation bar “ Deploy online -> Online services ”, Then click... On the page “ Deploy ”;

(2) Fill in the parameters on the deployment page , Among them in “ Model list ” Select the model and version to import , Calculation node specification selection CPU that will do ;

(3) Click on “ next step ”, After the parameters are confirmed , Click on “ Submit ”.

After submission , You can view the deployment progress in the online services list , When the status changes to “ Running ” after , Click on the service name , Go to the details page , Click on “ forecast ”, Upload pictures for testing . The test results are shown in the following figure , On the right is the predicted result , Different numbers indicate different categories .

6. Create a batch service

stay ModelArts On , You can also deploy the model as a batch service , from OBS Load test set images for prediction , Then output the forecast results to OBS, This algorithm supports CPU and GPU Deploy .

The specific steps of deployment for batch service are as follows :

(1) stay ModelArts Select... In the left navigation bar “ Deploy online -> Mass service ”, Then click... On the page “ Deploy ”;

(2) Fill in the parameters on the deployment page , Among them in “ Model list ” Select the model and version to import , Fill in the input data directory and output data directory , Calculation node specification selection “CPU 2 nucleus 8GB”, The number of calculated nodes is set to 1;

The input data directory location here is the location where the image to be predicted is stored , Note that only the pictures to be predicted can be stored in this location , And the output data directory location is empty folder , Self defining .

(3) Click on “ next step ”, After the parameters are confirmed , Click on “ Submit ”.

After submission , You can view the deployment progress in the batch services list , When the status changes to “ Running ” after , It means that we are predicting , When the status changes to “ Run complete ”, It means that this batch of pictures has been predicted to end , The forecast result is a batch of txt file , Save in the above figure OBS Output data directory location , You can go to this directory to see the results .

7. Model to evaluate

Refer to section 2 The steps in the section , establish “ Training assignment ”, Set training parameters according to the following table :

Click next , Submit , The state of the training assignment will experience “ initialization ”、“ Deploying ”、“ Running ” and “ The successful running ” Four states . After the training operation is successful , Specified in the table above “ Model output ” Path will automatically generate _result Catalog , It contains the reasoning .png Pictures and reasoning _results.txt file . among _results.txt The file contains the validation results , As shown in the figure .

Okay , That's the end of it , How do you feel this time ? We look forward to sharing and using experience and feeling .

Click to follow , The first time to learn about Huawei's new cloud technology ~

版权声明
本文为[Huawei cloud developer community]所创,转载请带上原文链接,感谢

Scroll to Top