Easily Capture Body Motion with HUAWEI ML Kit’s Skeleton Detection - Huawei Developers

Are you the kind of person who tends to go stiff and awkward when there’s a camera on you, and ends up looking unnatural in photos? If so, posture snapshots can help. All you need to do is select a posture template, and the camera will automatically take snapshots when it detects your body is in that position. This means photographs are only taken when you’re at your most natural. In this post, I'm going to show you how to integrate HUAWEI ML Kit's skeleton detection function into your apps. This function locates 14 skeleton points, and easily captures images of specific postures.
Skeleton Detection Function Development
1. Preparations
Before you get started, you need to make the necessary preparations. Also, ensure that the Maven repository address for the HMS Core SDK has been configured in your project, and the skeleton detection SDK has been integrated.
1.1 Configure the Maven Repository Address to the Project-Level build.gradle File
Code:
buildscript {
repositories {
google()
jcenter()
maven { url 'https://developer.huawei.com/repo/' }
}
dependencies {
classpath "com.android.tools.build:gradle:3.3.2"
}
}
Code:
allprojects {
repositories {
google()
jcenter()
maven { url 'https://developer.huawei.com/repo/' }
}
}
1.2 Add SDK Dependencies to the App-Level build.gradle File
Code:
dependencies {
implementation 'com.huawei.hms:ml-computer-vision-skeleton-model:2.0.1.300'
implementation 'com.huawei.hms:ml-computer-vision-skeleton:2.0.1.300'
implementation 'com.huawei.hms:ml-computer-vision-base:2.0.1.300'
}
2 Code Development
2.1 Static Image Detection
2.1.1 Create a Skeleton Analyzer
Code:
MLSkeletonAnalyzer analyzer = MLSkeletonAnalyzerFactory.getInstance().getSkeletonAnalyzer();
2.1.2 Create an MLFrame Using a Bitmap
The image resolution should be not less than 320 x 320 pixels and not greater than 1920 x 1920 pixels.
Code:
// Create an MLFrame using the bitmap.
MLFrame frame = MLFrame.fromBitmap(bitmap);
2.1.3 Call the asyncAnalyseFrame Method to Perform Skeleton Detection
Code:
Task<List<MLSkeleton>> task = analyzer.asyncAnalyseFrame(frame); task.addOnSuccessListener(new OnSuccessListener<List<MLSkeleton>>() {
public void onSuccess(List<MLSkeleton> skeletons) {
// Process the detection result.
}
}).addOnFailureListener(new OnFailureListener() {
public void onFailure(Exception e) {
// Detection failure.
}
});
2.1.4 Stop the Analyzer and Release Resources When the Detection Ends
Code:
try {
if (analyzer != null) {
analyzer.stop();
}
} catch (IOException e) {
// Exception handling.
}
2.2 Dynamic Video Detection
2.2.1 Create a Skeleton Analyzer
Code:
MLSkeletonAnalyzer analyzer = MLSkeletonAnalyzerFactory.getInstance().getSkeletonAnalyzer();
2.2.2 Create the SkeletonAnalyzerTransactor Class to Process the Detection Result
This class implements the MLAnalyzer.MLTransactor<T> API. You can use the transactResult method to obtain the detection result and implement specific services.
Code:
public class SkeletonAnalyzerTransactor implements MLAnalyzer.MLTransactor<MLSkeleton> {
@Override
public void transactResult(MLAnalyzer.Result<MLSkeleton> results) {
SparseArray<MLSkeleton> items = results.getAnalyseList();
// You can process the detection result as required. For example, calculate the similarity in this method to perform an operation, such as taking a photo when a specific posture has been detected.
// Only the detection result is processed. Other detection APIs provided by ML Kit cannot be called.
// Convert the result encapsulated using SparseArray to an ArrayList for similarity calculation.
List<MLSkeleton> resultsList = new ArrayList<>();
for (int i = 0; i < items.size(); i++) {
resultsList.add(items.valueAt(i));
}
// Calculate the similarity between the detection result and template.
// templateList is a list of skeleton templates. Templates can be generated by detecting static images. The skeleton detection service supports single-person and multi-person template matching.
float result = analyzer.caluteSimilarity(resultsList, templateList);
}
@Override
public void destroy() {
// Callback method used to release resources when the detection ends.
}
}
2.2.3 Set the Detection Result Processor to Bind the Analyzer
Code:
analyzer.setTransactor(new SkeletonAnalyzerTransactor());
2.2.4 Create the LensEngine Class
This class is provided by the HMS Core ML SDK. It captures dynamic video streams from the camera, and sends them to the analyzer. The camera display size should be not less than 320 x 320 pixels and not greater than 1920 x 1920 pixels.
Code:
// Create LensEngine.
LensEngine lensEngine = new LensEngine.Creator(getApplicationContext(), analyzer)
.setLensType(LensEngine.BACK_LENS)
.applyDisplayDimension(1280, 720)
.applyFps(20.0f)
.enableAutomaticFocus(true)
.create();
2.2.5 Open the Camera
You can obtain and detect the video streams, then stop the analyzer and release resources when the detection ends.
Code:
if (analyzer != null) {
try {
analyzer.stop();
} catch (IOException e) {
// Exception handling.
}
}
if (lensEngine != null) {
lensEngine.release();
}
Let's take a look at the dynamic video.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
You can do all sorts of things with HUAWEI ML Kit's skeleton detection function:
Create virtual images to simulate live action in motion sensing games.
Provide posture guidance to enhance workouts and rehabilitation training.
Detect unusual behavior in video surveillance footage.

---

Well explained, will it support non -huawei devices if suppose i installed HMS Core APK.

---

Good Article Thanks :victory:

---

Nice explanation. Can you please provide a scenario where I can use this service. Thank you