This article is shared from Huawei cloud community 《 I'll bring you some fresh food in autumn | ModelArts AI Market algorithm Fast-SCNN Instructions for use 》, author ：Tianyi_Li
Abstract ： I'd like to send my friends a fresh one ModelArts AI Market algorithm Fast-Scnn（ Hereinafter referred to as the algorithm ） Use the script , Make sure it's easy to get started .
It's double eleven , The autumn wind blows , You can't patronize chopping hands, right , It's time to stick autumn fat , Try something fresh , I sent the fresh ones out of the oven ModelArts AI Market algorithm Fast-Scnn（ Hereinafter referred to as the algorithm ）, Attached are the use secrets , Make sure it's easy to get started .
This algorithm uses Cityscapes High quality dimensioning data set train Set and val Set , Use train Set training , stay val The test on the set achieves mIOU=68.668 The accuracy of .
1. Prepare the dataset
The data set format supported by this algorithm is Cityscapes Data sets .
Cityscapes The dataset contains data from 50 A variety of stereo video sequences recorded in street scenes in different cities , And the larger 20000 Weak annotation framesets and 5000 High quality pixel level comments for frames . therefore , The dataset is a magnitude larger than the existing similar data set . For more information about annotated classes and examples of annotations, you can find the following Data set official website Found on the .
Cityscapes The data set is designed for :
(1) Evaluate the performance of visual algorithm in the main tasks of semantic city scene understanding ： Pixel level , Instance level and panoramic semantic markup ;
(2) Support is designed to take advantage of a large number of （ weak ） Research on annotation data , For example, it is used to train deep neural network .
1.1 Download datasets
can Click on this link. download gtFine_trainvaltest.zip and leftImg8bit_trainvaltest.zip Two documents , As shown in the figure below ：
gtFine_trainvaltest.zip The directory structure after decompression is as follows （ This algorithm does not need to decompress itself , Compression packages can be used directly ）：
leftImg8bit_trainvaltest.zip The directory structure after decompression is as follows （ This algorithm does not need to decompress itself , Compression packages can be used directly ）：
If you need to train with your own dataset , You need to sort the data catalog into the same form as above . For a more detailed description of the dataset, see Cityscapes Data set official website Description above .
1.2 install OBS Browser+ And upload the dataset
ModelArts Use object storage services （OBS） To store data , Achieve security 、 High reliability and low cost storage requirements .OBS Browser+ Is a storage service for accessing and managing objects （Object Storage Service,OBS） Graphic tools for , Support perfect barrel management and object management operations .OBS Browser+ The graphical interface of can make it very convenient for users to be on the local OBS Conduct management , for example ： Creating buckets 、 Upload and download files 、 Browsing files, etc .
The specific operation steps are as follows ：
（1） Download this OBS Browser+, Decompress after download , double-click exe Installation , Run after installation is complete ;
（2） The login interface is shown in the following figure , You are required to fill in the account name 、Access Key ID（AK） and Secret Accsee Key（SK）, Reference resources This document , obtain AK and SK, Many products on Huawei cloud need access keys , Please keep the key file properly , Then fill in your Huawei cloud account name and the newly obtained one with reference to the figure below AK and SK, Click login ;
（3） Refer to the below , Click on “ Creating buckets ”, Enter the bucket name , Be careful ： The region should choose North China - Beijing IV 、 Standard storage 、 private 、 Close more AZ, Bucket name needs to be customized ,OBS Bucket names should be globally unique , If prompt bucket name already exists , You need to change it to another name , For example, this article sets the bucket name fast-scnn. The bucket name you set must be different from this , See the instructions below fast-scnn, Please take the initiative to replace the bucket name with your own , The following will not be prompted ;
（4） Click on the bucket name , Into the barrel , Click on “ New folder ”, Enter the folder name , Such as “train_input”, Click to enter the folder , New again “datasets”, Click on “ Upload ”->“ Add files ”-> Select the local download of good dataset compression package （ The reason why we choose the compression package format is if the dataset is larger , Uploading compressed packages is much faster than uploading folders ） –> determine , As shown in the figure below ;
（5） Click on OBS Browser+ Left side “ task management ”, You can view the data upload progress . As shown in the figure below , Click Settings , In the basic settings , Set the maximum concurrency number to the maximum value 50, It can speed up data upload
2. Subscription algorithm
Click on the... On the top right of this page 【 subscribe 】 Button . Then click on... Below the page 【 next step 】 Button , Click again 【 confirm the payment 】 Button , Finally, click 【 determine 】 Button to my subscription page , You can see the algorithm you just subscribed to . Click on 【 Application console 】 Hyperlinks , Choose North China - Four areas of Beijing , Enter the algorithm management page .
As shown in the figure below , Click on “ Sync ” Button , Synchronization algorithm , You can click the refresh button , Refresh Status . When the state becomes ready , Indicates the synchronization is successful , Please use the latest version of , It should be 6.0.0 edition , Basic version to subscription .
3. Create training assignments
Click on the image above “ Create training assignments ”, Fill in the training parameters according to the table below ：
Click next , Submit , The state of the training assignment will experience “ initialization ”、“ Deploying ”、“ Running ” and “ The successful running ” Four states . After the training operation is successful , Specified in the table above “ Model output ” Path will automatically generate model Catalog , There are model files in this directory 、ModelArts Platform reasoning script （config.json、customize_service.py） And other necessary files for running the model .
4. Model import
After the model and related necessary documents are prepared , You can import the generated model into ModelArts Model management . The specific operation is as follows ：
（1） stay ModelArts On the left navigation bar of the console, click “ Model management ” -> “ Model ”, Click on the page on the right “ Import ”. Fill in the name in the import model page , Choose the source of the metamodel , You can choose from training directly ( This method is recommended , Easy and convenient , Seamless connection with training ), You can also get it from OBS Choose from . If from OBS Choose from , You need to choose to model The next level of a directory ; for example , The directory that can be selected this time is obs://fast-scnn/algorithms/train_output, As shown in the figure below ：
Be careful ： After choosing the metamodel path ,“AI engine ” It will automatically fill . If it fails to fill automatically , Check that the metamodel path is model The next level of a directory , perhaps model Whether the directory contains the model configuration file config.json.
（2） Click on “ Create... Now ”, It takes a little time to wait for model import and build , When the model version status is “ normal ” after , That means the model import is successful .
5. Create online services
stay ModelArts On , Models can be deployed as online services , Then upload the picture to predict , Observe the prediction results directly on the web , This algorithm supports CPU and GPU Deploy .
The specific steps of deployment to online service are as follows ：
（1） stay ModelArts Select... In the left navigation bar “ Deploy online -> Online services ”, Then click... On the page “ Deploy ”;
（2） Fill in the parameters on the deployment page , Among them in “ Model list ” Select the model and version to import , Calculation node specification selection CPU that will do ;
（3） Click on “ next step ”, After the parameters are confirmed , Click on “ Submit ”.
After submission , You can view the deployment progress in the online services list , When the status changes to “ Running ” after , Click on the service name , Go to the details page , Click on “ forecast ”, Upload pictures for testing . The test results are shown in the following figure , On the right is the predicted result , Different numbers indicate different categories .
6. Create a batch service
stay ModelArts On , You can also deploy the model as a batch service , from OBS Load test set images for prediction , Then output the forecast results to OBS, This algorithm supports CPU and GPU Deploy .
The specific steps of deployment for batch service are as follows ：
（1） stay ModelArts Select... In the left navigation bar “ Deploy online -> Mass service ”, Then click... On the page “ Deploy ”;
（2） Fill in the parameters on the deployment page , Among them in “ Model list ” Select the model and version to import , Fill in the input data directory and output data directory , Calculation node specification selection “CPU 2 nucleus 8GB”, The number of calculated nodes is set to 1;
The input data directory location here is the location where the image to be predicted is stored , Note that only the pictures to be predicted can be stored in this location , And the output data directory location is empty folder , Self defining .
（3） Click on “ next step ”, After the parameters are confirmed , Click on “ Submit ”.
After submission , You can view the deployment progress in the batch services list , When the status changes to “ Running ” after , It means that we are predicting , When the status changes to “ Run complete ”, It means that this batch of pictures has been predicted to end , The forecast result is a batch of txt file , Save in the above figure OBS Output data directory location , You can go to this directory to see the results .
7. Model to evaluate
Refer to section 2 The steps in the section , establish “ Training assignment ”, Set training parameters according to the following table ：
Click next , Submit , The state of the training assignment will experience “ initialization ”、“ Deploying ”、“ Running ” and “ The successful running ” Four states . After the training operation is successful , Specified in the table above “ Model output ” Path will automatically generate _result Catalog , It contains the reasoning .png Pictures and reasoning _results.txt file . among _results.txt The file contains the validation results , As shown in the figure .
Okay , That's the end of it , How do you feel this time ？ We look forward to sharing and using experience and feeling .