Building a Custom Model for Image Classification with ML Kit - Huawei Developers

1. Intro
If you're looking to develop a customized and cost-effective deep learning model, you'd be remiss not to try the recently released custom model service in HUAWEI ML Kit. This service gives you the tools to manage the size of your model, and provides simple APIs for you to perform inference. The following is a demonstration of how you can run your model on a device at a minimal cost.
The service allows you to pre-train image classification models, and the steps below show you the process for training and using a custom model.
2. Implementation
a. Install HMS Toolkit from Android Studio Marketplace. After the installation, restart Android Studio.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
b. Transfer learning by using AI Create.
Basic configuration
* Note: First install a Python IDE.
AI Create uses MindSpore as the training framework and MindSpore Lite as the inference framework. Follows the steps below to complete the basic configuration.
i) In Coding Assistant, go to AI > AI Create.
ii) Select Image or Text, and click on Confirm.
iii) Restart the IDE. Select Image or Text, and click on Confirm. The MindSpore tool will be automatically installed.
HMS Toolkit allows you to generate an API or demo project in one click, enabling you to quickly verify and call the image classification model in your app.
Before using the transfer learning function for image classification, prepare image resources for training as needed.
*Note: The images should be clear and placed in different folders by category.
Model training
Train the image classification model pre-trained in ML Kit to learn hundreds of images in specific fields (such as vehicles and animals) in a matter of minutes. A new model will then be generated, which will be capable of automatically classifying images. Follow the steps below for model training.
i) In Coding Assistant, go to AI > AI Create > Image.
ii) Set the following parameters and click on Confirm.
Operation type: Select New Model.
Model Deployment Location: Select Deployment Cloud.
iii) Drag or add the image folders to the Please select train image folder area.
iv) Set Output model file path and train parameters.
v) Retain the default values of the train parameters as shown below:
Iteration count: 100
Learning rate: 0.01
vi) Click on Create Model to start training and to generate an image classification model.
After the model is generated, view the model learning results (training precision and verification precision), corresponding learning parameters, and training data.
Model verification
After the model training is complete, you can verify the model by adding the image folders in the Please select test image folder area under Add test image. The tool will automatically use the trained model to perform the test and display the test results.
Click on Generate Demo to have HMS Toolkit generate a demo project, which automatically integrates the trained model. You can directly run and build the demo project to generate an APK file, and run the file on a simulator or real device to verify image classification performance.
c. Use the model.
Upload the model
The image classification service classifies elements in images into logical categories, such as people, objects, environments, activities, or artwork, to define image themes and application scenarios. The service supports both on-device and on-cloud recognition modes, and offers the pre-trained model capability.
To upload your model to the cloud, perform the following steps:
i) Sign in to AppGallery Connect and click on My projects.
ii) Go to ML Kit > Custom ML, to have the model upload page display. On this page, you can also upgrade existing models.
Load the remote model
Before loading a remote model, check whether the remote model has been downloaded. Load the local model if the remote model has not been downloaded.
Code:
localModel = new MLCustomLocalModel.Factory("localModelName")
.setAssetPathFile("assetpathname")
.create();
remoteModel =new MLCustomRemoteModel.Factory("yourremotemodelname").create();
MLLocalModelManager.getInstance()
// Check whether the remote model exists.
.isModelExist(remoteModel)
.addOnSuccessListener(new OnSuccessListener<Boolean>() {
@Override
public void onSuccess(Boolean isDownloaded) {
MLModelExecutorSettings settings;
// If the remote model exists, load it first. Otherwise, load the existing local model.
if (isDownloaded) {
settings = new MLModelExecutorSettings.Factory(remoteModel).create();
} else {
settings = new MLModelExecutorSettings.Factory(localModel).create();
}
final MLModelExecutor modelExecutor = MLModelExecutor.getInstance(settings);
executorImpl(modelExecutor, bitmap);
}
})
.addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Exception handling.
}
});
Perform inference using the model inference engine
Set the input and output formats, input image data to the inference engine, and use the loaded modelExecutor(MLModelExecutor) to perform the inference.
Code:
private void executorImpl(final MLModelExecutor modelExecutor, Bitmap bitmap){
// Prepare input data.
final Bitmap inputBitmap = Bitmap.createScaledBitmap(srcBitmap, 224, 224, true);
final float[][][][] input = new float[1][224][224][3];
for (int i = 0; i < 224; i++) {
for (int j = 0; j < 224; j++) {
int pixel = inputBitmap.getPixel(i, j);
input[batchNum][j][i][0] = (Color.red(pixel) - 127) / 128.0f;
input[batchNum][j][i][1] = (Color.green(pixel) - 127) / 128.0f;
input[batchNum][j][i][2] = (Color.blue(pixel) - 127) / 128.0f;
}
}
MLModelInputs inputs = null;
try {
inputs = new MLModelInputs.Factory().add(input).create();
// If the model requires multiple inputs, you need to call add() for multiple times so that image data can be input to the inference engine at a time.
} catch (MLException e) {
// Handle the input data formatting exception.
}
// Perform inference. You can use addOnSuccessListener to listen for successful inference which is processed in the onSuccess callback. In addition, you can use addOnFailureListener to listen for inference failure which is processed in the onFailure callback.
modelExecutor.exec(inputs, inOutSettings).addOnSuccessListener(new OnSuccessListener<MLModelOutputs>() {
@Override
public void onSuccess(MLModelOutputs mlModelOutputs) {
float[][] output = mlModelOutputs.getOutput(0);
// The inference result is in the output array and can be further processed.
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Inference exception.
}
});
}
3. Summary
By utilizing Huawei's deep learning framework, you'll be able to create and use a deep learning model for your app by following just a few steps!
Furthermore, the custom model service for ML Kit is compatible with all mainstream model inference platforms and frameworks on the market, including MindSpore, TensorFlow Lite, Caffe, and Onnx. Different models can be converted into .ms format, and run perfectly within the on-device inference framework.
Custom models can be deployed to the device in a smaller size after being quantized and compressed. To further reduce the APK size, you can host your models on the cloud. With this service, even a novice in the field of deep learning is able to quickly develop an AI-driven app which serves a specific purpose.
Learn More
For more information, please visit HUAWEI Developers.
For detailed instructions, please visit Development Guide.
You can join the HMS Core developer discussion on Reddit.
You can download the demo and sample code from GitHub.
To solve integration problems, please go to Stack Overflow.

---

It seems there are many new fuctions on ML Kit. Wonderful!

---

Hi, have you tried searching for public APIs?