Expert: Find yoga pose using Huawei ML kit skeleton detection - Part 2 - Huawei Developers

Introduction
In this article, I will cover live yoga pose detection. In my last article, I’ve written yoga pose detection using the Huawei ML kit. If you have not read my previous article refer to link Beginner: Find yoga pose using Huawei ML kit skeleton detection - Part 1.
In this article, I will cover live yoga detection.
Definitely, you will have a question about how does this application help?
Let’s take an example, most people attend yoga classes due to COVID-19 nobody is able to attend the yoga classes. So using the Huawei ML kit Skeleton detection record your yoga session video and send it to your yoga master he will check your body joints which is shown in the video. And he will explain what are the mistakes you have done in that recorded yoga session.
Integration of Skeleton Detection
1. Configure the application on the AGC.
2. Client application development process.
Configure application on the AGC
Follow the steps.
Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.
Step 2: Create an app by referring to Creating a Project and Creating an App in the Project
Step 3: Set the data storage location based on the current location.
Step 4: Enabling ML. Open AppGallery connect, choose Manage API > ML Kit.
Step 5: Generating a Signing Certificate Fingerprint.
Step 6: Configuring the Signing Certificate Fingerprint.
Step 7: Download your agconnect-services.json file, paste it into the app root directory.
Client application development process
Follow the steps.
Step 1: Create an Android application in the Android studio(Any IDE which is your favorite).
Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.
1
2apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'
Root level gradle dependencies.
1
2maven { url 'https://developer.huawei.com/repo/' }
classpath 'com.huawei.agconnect:agcp:1.4.1.300'
Step 3: Add the dependencies in build.gradle
1
2
3implementation 'com.huawei.hms:ml-computer-vision-skeleton:2.0.4.300'
implementation 'com.huawei.hms:ml-computer-vision-skeleton-model:2.0.4.300'
implementation 'com.huawei.hms:ml-computer-vision-yoga-model:2.0.4.300'
To achieve the Skeleton detection example, follow the steps.
1. AGC Configuration
2. Build Android application
Step 1: AGC Configuration
1. Sign in to AppGallery Connect and select My apps.
2. Select the app in which you want to integrate the Huawei ML kit.
3. Navigate to Project Setting> Manage API > ML Kit
Step 2: Build Android application
In this example, I’m detecting yoga poses live using the camera.
While building the application, follow the steps.
Step 1: Create a Skeleton analyzer.
1
2private static MLSkeletonAnalyzer analyzer = null;
analyzer = MLSkeletonAnalyzerFactory.getInstance().skeletonAnalyzer
Step 2: Create SkeletonAnalyzerTransactor class to process the result.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65import android.app.Activity
import android.util.Log
import app.dtse.hmsskeletondetection.demo.utils.SkeletonUtils
import app.dtse.hmsskeletondetection.demo.views.graphic.SkeletonGraphic
import app.dtse.hmsskeletondetection.demo.views.overlay.GraphicOverlay
import com.huawei.hms.mlsdk.common.LensEngine
import com.huawei.hms.mlsdk.common.MLAnalyzer
import com.huawei.hms.mlsdk.common.MLAnalyzer.MLTransactor
import com.huawei.hms.mlsdk.skeleton.MLSkeleton
import com.huawei.hms.mlsdk.skeleton.MLSkeletonAnalyzer
import java.util.*
class SkeletonTransactor(
private val analyzer: MLSkeletonAnalyzer,
private val graphicOverlay: GraphicOverlay,
private val lensEngine: LensEngine,
private val activity: Activity?
) : MLTransactor<MLSkeleton?> {
private val templateList: List<MLSkeleton>
private var zeroCount = 0
override fun transactResult(results: MLAnalyzer.Result<MLSkeleton?>) {
Log.e(TAG, "detect success")
graphicOverlay.clear()
val items = results.analyseList
val resultsList: MutableList<MLSkeleton?> = ArrayList()
for (i in 0 until items.size()) {
resultsList.add(items.valueAt(i))
}
if (resultsList.size <= 0) {
return
}
val similarity = 0.8f
val skeletonGraphic = SkeletonGraphic(graphicOverlay, resultsList)
graphicOverlay.addGraphic(skeletonGraphic)
graphicOverlay.postInvalidate()
val result = analyzer.caluteSimilarity(resultsList, templateList)
if (result >= similarity) {
zeroCount = if (zeroCount > 0) {
return
} else {
0
}
zeroCount++
} else {
zeroCount = 0
return
}
lensEngine.photograph(null, { bytes ->
SkeletonUtils.takePictureListener.picture(bytes)
activity?.finish()
})
}
override fun destroy() {
Log.e(TAG, "detect fail")
}
companion object {
private const val TAG = "SkeletonTransactor"
}
init {
templateList = SkeletonUtils.getTemplateData()
}
}
Step 3: Set Detection Result Processor to Bind the Analyzer.
1analyzer!!.setTransactor(SkeletonTransactor(analyzer!!, overlay!!, lensEngine!!, activity))
Step 4: Create LensEngine.
1
2
3
4
5
6lensEngine = LensEngine.Creator(context, analyzer)
.setLensType(LensEngine.BACK_LENS)
.applyDisplayDimension(1280, 720)
.applyFps(20.0f)
.enableAutomaticFocus(true)
.create()
Step 5: Open the camera.
Step 6: Release resources.
1
2
3
4
5
6
7
8
9
10
11if (lensEngine != null) {
lensEngine!!.close()
}if (lensEngine != null) {
lensEngine!!.release()
}if (analyzer != null) {
try {
analyzer!!.stop()
} catch (e: IOException) {
Log.e(TAG, "e=" + e.message)
}
}
Result
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Tips and Tricks
Check dependencies downloaded properly.
Latest HMS Core APK is required.
If you are taking an image from a camera or gallery make sure the app has camera and storage permission.
Conclusion
In this article, we have learned the integration of the Huawei ML kit, and what is skeleton detection, how it works, what is the use of it, how to get the Joints point from the skeleton detection, types of detections like TYPE_NORMALand TYPE_YOGA.
Reference
Skeleton Detection
Happy coding

---

Basavaraj.navi said:
Introduction
In this article, I will cover live yoga pose detection. In my last article, I’ve written yoga pose detection using the Huawei ML kit. If you have not read my previous article refer to link Beginner: Find yoga pose using Huawei ML kit skeleton detection - Part 1.
In this article, I will cover live yoga detection.
Definitely, you will have a question about how does this application help?
Let’s take an example, most people attend yoga classes due to COVID-19 nobody is able to attend the yoga classes. So using the Huawei ML kit Skeleton detection record your yoga session video and send it to your yoga master he will check your body joints which is shown in the video. And he will explain what are the mistakes you have done in that recorded yoga session.
Integration of Skeleton Detection
1. Configure the application on the AGC.
2. Client application development process.
Configure application on the AGC
Follow the steps.
Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.
Step 2: Create an app by referring to Creating a Project and Creating an App in the Project
Step 3: Set the data storage location based on the current location.
Step 4: Enabling ML. Open AppGallery connect, choose Manage API > ML Kit.
Step 5: Generating a Signing Certificate Fingerprint.
Step 6: Configuring the Signing Certificate Fingerprint.
Step 7: Download your agconnect-services.json file, paste it into the app root directory.
Client application development process
Follow the steps.
Step 1: Create an Android application in the Android studio(Any IDE which is your favorite).
Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.
1
2apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'
Root level gradle dependencies.
1
2maven { url 'https://developer.huawei.com/repo/' }
classpath 'com.huawei.agconnect:agcp:1.4.1.300'
Step 3: Add the dependencies in build.gradle
1
2
3implementation 'com.huawei.hms:ml-computer-vision-skeleton:2.0.4.300'
implementation 'com.huawei.hms:ml-computer-vision-skeleton-model:2.0.4.300'
implementation 'com.huawei.hms:ml-computer-vision-yoga-model:2.0.4.300'
To achieve the Skeleton detection example, follow the steps.
1. AGC Configuration
2. Build Android application
Step 1: AGC Configuration
1. Sign in to AppGallery Connect and select My apps.
2. Select the app in which you want to integrate the Huawei ML kit.
3. Navigate to Project Setting> Manage API > ML Kit
Step 2: Build Android application
In this example, I’m detecting yoga poses live using the camera.
While building the application, follow the steps.
Step 1: Create a Skeleton analyzer.
1
2private static MLSkeletonAnalyzer analyzer = null;
analyzer = MLSkeletonAnalyzerFactory.getInstance().skeletonAnalyzer
Step 2: Create SkeletonAnalyzerTransactor class to process the result.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65import android.app.Activity
import android.util.Log
import app.dtse.hmsskeletondetection.demo.utils.SkeletonUtils
import app.dtse.hmsskeletondetection.demo.views.graphic.SkeletonGraphic
import app.dtse.hmsskeletondetection.demo.views.overlay.GraphicOverlay
import com.huawei.hms.mlsdk.common.LensEngine
import com.huawei.hms.mlsdk.common.MLAnalyzer
import com.huawei.hms.mlsdk.common.MLAnalyzer.MLTransactor
import com.huawei.hms.mlsdk.skeleton.MLSkeleton
import com.huawei.hms.mlsdk.skeleton.MLSkeletonAnalyzer
import java.util.*
class SkeletonTransactor(
private val analyzer: MLSkeletonAnalyzer,
private val graphicOverlay: GraphicOverlay,
private val lensEngine: LensEngine,
private val activity: Activity?
) : MLTransactor<MLSkeleton?> {
private val templateList: List<MLSkeleton>
private var zeroCount = 0
override fun transactResult(results: MLAnalyzer.Result<MLSkeleton?>) {
Log.e(TAG, "detect success")
graphicOverlay.clear()
val items = results.analyseList
val resultsList: MutableList<MLSkeleton?> = ArrayList()
for (i in 0 until items.size()) {
resultsList.add(items.valueAt(i))
}
if (resultsList.size <= 0) {
return
}
val similarity = 0.8f
val skeletonGraphic = SkeletonGraphic(graphicOverlay, resultsList)
graphicOverlay.addGraphic(skeletonGraphic)
graphicOverlay.postInvalidate()
val result = analyzer.caluteSimilarity(resultsList, templateList)
if (result >= similarity) {
zeroCount = if (zeroCount > 0) {
return
} else {
0
}
zeroCount++
} else {
zeroCount = 0
return
}
lensEngine.photograph(null, { bytes ->
SkeletonUtils.takePictureListener.picture(bytes)
activity?.finish()
})
}
override fun destroy() {
Log.e(TAG, "detect fail")
}
companion object {
private const val TAG = "SkeletonTransactor"
}
init {
templateList = SkeletonUtils.getTemplateData()
}
}
Step 3: Set Detection Result Processor to Bind the Analyzer.
1analyzer!!.setTransactor(SkeletonTransactor(analyzer!!, overlay!!, lensEngine!!, activity))
Step 4: Create LensEngine.
1
2
3
4
5
6lensEngine = LensEngine.Creator(context, analyzer)
.setLensType(LensEngine.BACK_LENS)
.applyDisplayDimension(1280, 720)
.applyFps(20.0f)
.enableAutomaticFocus(true)
.create()
Step 5: Open the camera.
Step 6: Release resources.
1
2
3
4
5
6
7
8
9
10
11if (lensEngine != null) {
lensEngine!!.close()
}if (lensEngine != null) {
lensEngine!!.release()
}if (analyzer != null) {
try {
analyzer!!.stop()
} catch (e: IOException) {
Log.e(TAG, "e=" + e.message)
}
}
Result
Tips and Tricks
Check dependencies downloaded properly.
Latest HMS Core APK is required.
If you are taking an image from a camera or gallery make sure the app has camera and storage permission.
Conclusion
In this article, we have learned the integration of the Huawei ML kit, and what is skeleton detection, how it works, what is the use of it, how to get the Joints point from the skeleton detection, types of detections like TYPE_NORMALand TYPE_YOGA.
Reference
Skeleton Detection
Happy coding
Click to expand...
Click to collapse