Related
This article will help you understand how easily you can add any Huawei Mobile Services into your android application.
===== Article Takeaway =====
After reading this article you will be able to add any SDK into your android project without any hustle or blockage. It will increase the efficiency as well as reduces error chances.
Automatic HMS Kit search.
Automatic Permission Addition
Code snippets for API usage.
Add API by dragging the code snippets
#android #hms #huawei #androiddev #huaweifacts
bit.ly/HMS-Toolkit
Tshrsri said:
This article will help you understand how easily you can add any Huawei Mobile Services into your android application.
===== Article Takeaway =====
After reading this article you will be able to add any SDK into your android project without any hustle or blockage. It will increase the efficiency as well as reduces error chances.
Automatic HMS Kit search.
Automatic Permission Addition
Code snippets for API usage.
Add API by dragging the code snippets
#android #hms #huawei #androiddev #huaweifacts
bit.ly/HMS-Toolkit
Click to expand...
Click to collapse
Hi, Tshrsri. Where can I see the complete article?
HarmonyOS is a future-proof distributed operating system open to you as part of the initiatives for the all-scenario strategy, adaptable to a mobile office, fitness and health, social communication, and media entertainment, to name a few. Unlike a legacy operating system that runs on a standalone device, HarmonyOS is built on a distributed architecture designed based on a set of system capabilities. It is able to run on a wide range of device forms.
More related to HarmonyOS ,please refer to:
https://www.reddit.com/r/HuaweiDevelopers/comments/lx9jqd
1. Get to know HarmonyOS
1）What is HarmonyOS：
Click to expand...
Click to collapse
LINK https://device.harmonyos.com/en/docs/start/learn/oem_des_define-0000001055232642
What Is HarmonyOS? What ability? What does technical architecture look like?
HarmonyOS is a future-proof distributed operating system open to you as part of the initiatives for the all-scenario strategy, adaptable to a mobile office, fitness and health, social communication, and media entertainment, to name a few. Unlike a legacy operating system that runs on a standalone device, HarmonyOS is built on a distributed architecture designed based on a set of system capabilities. It is able to run on a wide range of device forms.
For application developers, HarmonyOS adopts distributed technologies to make application development possible on different device forms. With HarmonyOS, you have the choice to focus on upper-layer service logic and develop applications in a much easier and more efficient way.
For device developers, HarmonyOS uses a component-based software design to tailor itself to particular device forms based on their respective resource capabilities and service characteristics.
2） Security guide:
Click to expand...
Click to collapse
What are the security mechanisms and recommended practices of HarmonyOS in terms of hardware, system, data, device interconnection, and application security?
Link：https://device.harmonyos.com/en/docs/security/sec-guides/oem_security_guide-0000001050032745
3）Obtain the source code.
Click to expand...
Click to collapse
There are several methods for obtaining the source code. Select one based on the site requirements.
LINK：https://device.harmonyos.com/en/doc...EN_TOPIC_0000001050769927__section61172538310
4）Obtain the compilation toolchain, device development tool (HUAWEI DevEco Device Tool), application development tool (HUAWEI DevEco Studio), and tool usage guide.
Click to expand...
Click to collapse
LINK：https://device.harmonyos.com/en/docs/start/get-tools/oem_tool-0000001055705774
5）API reference：
Click to expand...
Click to collapse
https://device.harmonyos.com/en/docs/develop/apiref/abilitykit-0000001054598111
2. Develop WLAN connection products
a) Have general knowledge of the Hi3861 development board: WLAN module, which provides connection capabilities for various IoT devices.
https://device.harmonyos.com/en/docs/start/introduce/oem_wifi_start_des-0000001050168548
b) Set up the Hi3861 environment, including preparing the software and hardware, and installing the compilation and development environment.
https://device.harmonyos.com/en/docs/start/introduce/oem_quickstart_3861_build-0000001054781998
c) Develop the first example program of the Hi3861: Compile and burn the HarmonyOS to complete the first Hello World program.
https://device.harmonyos.com/en/docs/start/introduce/oem_wifi_start_helloword-0000001051930719
d) Development example of LED peripheral control: Call the NDK interface of HarmonyOS to control the GPIO and implement LED blinking.
https://device.harmonyos.com/en/docs/develop/demo/oem_wifi_sdk_des-0000001050059068
e) One-Hop scenario development guide: Develop WLAN connection products that provide the FA experience.
https://device.harmonyos.com/en/docs/develop/demo/oem_device_wifi_touch_des-0000001054809169
f) Third-party SDK integration: Integrate vendor SDKs into HarmonyOS.
https://device.harmonyos.com/en/docs/develop/demo/oem_device_wifi_sdk-0000001054412155
3. Develop camera products with screens.
[Device Software Development]
a) Understand the Hi3516 development board: It has the screen camera module and develops HarmonyOS applications based on the development board.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_3516-0000001052670587
b) Set up the Hi3516 environment, including preparing the software and hardware, and installing the compilation and development environments.
https://device.harmonyos.com/en/doc..._quickstart_3516_environment-0000001054501981
c) Develop the first application of the Hi3516: Compile and burn the HarmonyOS to complete the first application Hello World.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_first_example-0000001051610926
d) Example of developing the first Hi3516 driver: Develop a new driver using HarmonyOS and complete the first driver Hello World.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_driveexample-0000001054448621
e) Screen and camera control development example: Use HarmonyOS to control the screen and camera.
https://device.harmonyos.com/en/docs/develop/demo/oem_device_iotcamera_control_des-0000001055101239
[Application Software Development]
a) Set up the development environment: Install HUAWEI DevEco Studio.
https://developer.harmonyos.com/en/docs/documentation/doc-guides/tools_overview-0000001053582387
b) JavaScript application development interface: describes the framework, components, and interfaces of JavaScript application development.
https://device.harmonyos.com/en/docs/develop/apiref/js-framework-file-0000000000611396
c) Visual application development example: Use HarmonyOS to develop vision applications.
https://device.harmonyos.com/en/docs/develop/demo/oem_camera_fa_des-0000001050065839
4. Develop screenless camera products.
a) Have general knowledge of the Hi3518 development board: The camera module without a screen provides camera capabilities for various IoT devices.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_hi3518-0000001050170473
b) Set up the Hi3518 environment, including preparing the software and hardware, and installing the compilation and development environments.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_environment-0000001052450997
c) Develop the first example program of the Hi3518: Compile and burn the HarmonyOS to complete the first program Hello World.
https://device.harmonyos.com/en/docs/start/introduce/oem_camera_start_example-0000001051610926
d) Camera control development example: Use HarmonyOS to control the camera.
https://device.harmonyos.com/en/docs/develop/demo/oem_device_iotcamera_control_des-0000001054203800
5. Chip Adaptation Reference
a) Kernel development guide: describes the basic functions, file system, standard library, and commissioning functions of the HarmonyOS light kernel and provides development guidance.
https://device.harmonyos.com/en/docs/develop/kernel/oem_kernal_user_process-0000001050032733
b) Driver usage guide: provides guidance for using the HDF driver framework, driver platform, and peripheral functions.
https://device.harmonyos.com/en/docs/develop/drive/oem_drive_hdfdev-0000001051715456
6. Contribution component
a) Component development specifications: basic concepts of components and how to define components according to the specifications.
https://device.harmonyos.com/en/docs/develop/bundles/oem_bundle_standard_des-0000001050129846
b) Component development guide: Develop HarmonyOS components and distributions.
https://device.harmonyos.com/en/docs/develop/bundles/oem_bundle_guide_des-0000001050770981
7. Code cloud warehouse
Code cloud docs repository: OpenHarmony is an open-source version of HarmonyOS. It provides developer documents, such as quick start, development guide, and API reference. Welcome to the document open-source project and improve developer documents together.
All right, that’s all we’ve got so far. We will update the new resource editors here. Hope it is helpful for you. Welcome to post and leave a message.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
IntroductionHello everybody! In this article, I will be introducing the brand new version of HMS Unity Plugin version 2.0! Yes, it is out and it will make your life much easier, even when compared to the previous version of the plugin 1.2.0!
The HMS Unity Plugin is a tool that helps developers to quickly integrate Huawei Mobile Services (HMS) to their games in Unity without worrying about the boilerplate codes that has to be written in the background. All the necessary backend code is dealt with for you, and all you have to do is to focus on your own game and HMS features.
In this article, I will be using my own game as a scenario to quickly explain the steps required for you to integrate many HMS kits with a few clicks and/or lines of code. To be specific, I am using the version 2.0.1 for this article.
Before diving into details, you can check out here, the plugin’s GitHub page, if you think you are experienced enough with the plugin and that you do not require any specific scenario to start using the plugin. Readme already has general details about how to start using the kit.
Please note that I developed this game and used the plugin for this article series in Unity editor version 2019.4.18f1. For 2020 version of the Unity, you should be able to do the same steps with the same version of the plugin. For 2018 or other versions, please check out here, the releases page, for the corresponding plugin release (if any) and for the latest updates.
I further suggest that you download the latest version of the plugin for the corresponding Unity version, although I have used version 2.0.1 here. Details on how to use/import the plugin will follow, but I wanted to let you know beforehand.
About My GameLet me quickly introduce my game to you, so that during the phases of integration, it makes sense to you why I am implementing the plugin features in this way.
It is a hypercasual game called Raining Apocalypse, where you simply escape from the rain! You are the cool fire character that run horizontally. The more rains you run away from the more points you score. So, as you can see, very simple and affordable to implement.
I developed this game using a Udemy course and the source code is not completely written by me. However, we will not focus on the game development parts anyway, we will integrate the HMS kits to our game to increase our chances of survival in the game industry.
It consists of two scenes: main menu scene and game scene. It has several scripts attached, some of which we will use in the integration.
I will integrate “Account Kit, Ads Kit, Push Kit, Game Service, IAP, Analytics Kit and Crash Kit” to my game. For this part 1 of the article, I will talk about Account Kit, Ads Kit, Analytics Kit, Crash Kit and Push Kit. Game Service and IAP will be talked about in Part 2 of the series.
It looks too many and seems to cost the developers a lot of time, but not with Plugin 2.0 and you will see how, if you read on.
Development ProcessTo start using the plugin you must go here, the readme of the official GitHub page of the plugin and complete the phases, starting from the phase called “1 — Register your app at Huawei Developer” to “4 — Connect your game with any HMS Kit”, so that the AppGallery Connect side of your application is done. HMS requires AppGallery Connect configurations to correctly work with the in-game features you want to implement.
I will, from now on, assume that you have completed the first 3 phases and now ready to implement the 4th phase and onwards. You can continue using the following phases for the kits as well; but here, I will merge them with a real life scenario, i.e. my game, so that you can better understand how those features would work in your own game. By this way, you can decide easily where to put those methods in your own game.
Now, assuming that you are done with the AGC side, let’s get to the coding. I will talk about AGC side a bit in some of the kits I explain as well.
Coding PhaseLet’s start with Account Kit and Ads Kit. The main menu scene of my game has a logo and a play button to start the game. What I want to add is a banner ad to the bottom and also to implement sign in functionality. By this way, I will show 320*50 (or any size you choose) ads to the users at the beginning of my game. Also, since I do not want to proceed to the game without the user signed in, I will implement the sign in functionality so that when I try to use the other kits in-game, I do not have to deal with sign in process again.
To start using any kit, you must import the plugin (downloaded from the releases page) and click on the kits that you want to use. Mine currently looks like this:
Clicking them will add a manager to your game screen. They are coming with DontDestroyOnLoad() function by default, so you do not have to worry about carrying it over to the other scenes. However, it is strongly suggested that you do this in the outermost scene, so this carry-over process is smooth and bug-free. I do it in my main menu scene because it comes first, and the game scene opens the second.
I have my own manager script called EnesGameManager.cs to control the behaviour of kits and game-specific functionalities.
Before every use of managers, you must call them with .Instance because the plugin uses the singleton pattern to implement the managers. That is also helpful in changing scenes all you like because the plugin will delete the unnecessary copies of the manager and will make sure that you only work with the one and the only instance. This will save you from coding overhead and wrong instance usage.
Account KitAfter ticking the box, HMSAccountManager is added to the scene. All you have to do is to add the below line to anywhere you choose as per your own game logic. You can place it inside a new method to add a button click to the play button, for example.
HMSAccountManager.Instance.SignIn();
Important Note: If you plan to use GameService, it has automatic sign-in functionality. Instead of using Account Kit, you can just enable the GameService, tick the box (“Initialize on Start”) under Game Service tab and your app will do auto sign-in every time the user opens the app. If this is not what you want, use of Account Kit is required and GameService function should be manually initialized. All of the details about this can be found in the part 2 of the article, which is here.
GameService is dependent on AccountKit anyway, so even if you do not use Account Kit to login (which is perfectly fine), GameService login system will use it for you in the background.
Ads Kit — Part 1Ads Kit has three types of advertisements as of the publish date of this article. If you are reading this later than the publish date, some new types of ads might have been added to the plugin, so I would suggest you to check them out as well. Ads Kit, right now, supports “Banner Ads, Interstitial Ads and Rewarded Ads”. I will be using banner ads and interstitial ads in my game. Since the interstitial ads will be implemented in the game scenes, in this part 1, I will only show how to implement banner ads.
To enable the ads you want to use, just tick the corresponding boxes. That is very easy. Also, if you want to use test ads like me, check that box too. It will replace any ID you enter above with the test ID, so you can test the layout of ads in your game for example, before getting a real ID. If you have the ID already you can uncheck the test ID box, enter the ID and click save. Replacing does not mean you will lose the already-entered ID, but you will see the test ads from now on, until you uncheck the box.
The default size is 320*50 and the default position is POSITION_BOTTOM. If you require different sizes and positions, you can only change them inside of manager script in this version of the plugin. You can check Huawei -> Scripts -> Ads folder to reach the manager script and configure as per your needs.
If you build now, you should get your ads like below. You can work on alignment yourself to match the elements.
There is one more thing left. This banner ad will show in every screen you will be opening from now on. For my case, it will be present on my game screen if I press play button; which is not what I want. Thus, in the game scene, I will call below one line code to hide the ads because it is the behaviour I want. You can call this code piece in any Start() function of the active object scripts.
HMSAdsKitManager.Instance.HideBannerAd();
As you can see, no initialization etc. required. HMSAdsKitManager will remain on your other scenes and can be directly called by using its instance.
If you do not want the banner ads in your main screen and want it in other scenes, you can always use the above hide code to hide it in the beginning and call the below show method to show it anywhere you like.
HMSAdsKitManager.Instance.ShowBannerAd();
That’s it for banner ads. Very short, very simple.
Ads Kit — Part 2I also want to use interstitial ads in my app. The scenario I want to use it is when the user dies, before showing a lose screen to retry, I want to show interstitial ads, unless the user has purchased the remove_ads product (which I will talk about in IAP section).
To implement this, I need to put the code into the proper place in my own game code. There is no one way to do it. If you have a similar scenario like me, you can also follow my code and find your corresponding code and implement your own interstitial ads logic.
In my player script, I have the TakeDamage function where I control the damage inflicted on my player. It also controls the death, when the health drops below zero. So here, when the player dies, I call interstitial ad show code to display the user interstitial ads. The if check can be ignored now and will be talked about in IAP section.
public void TakeDamage(int damageAmount)
{
source.Play();
health -= damageAmount;
if (health <= 0)
{
healthDisplay.text = "0";
HMSAnalyticsManager.Instance.SendEventWithBundle("$GameCompleted", "Score", score.ToString());
Destroy(gameObject);
if (!GameObject.Find("EnesGameManager").GetComponent<EnesGameManager>().isAdsRemoved)
HMSAdsKitManager.Instance.ShowInterstitialAd();
losePanel.SetActive(true);
HMSAchievementsManager.Instance.GetAchievementsList();
}
else
{
updateHealthDisplay();
}
}
Basically, just calling the ShowInterstitialAd() function is enough to show the ads. Once the player dies, it will be shown immediately and after the user closes it, the lose panel will be shown.
Like most kits, there are callbacks that you may want to implement. For example, interstitial ads has “OnInterstitialAdClosed” callback that can be implemented. If you want to control what will happen right after the user closes the interstitial ad, you can implement this by using the code below in start function.
void Start()
{
//...
HMSAdsKitManager.Instance.OnInterstitialAdClosed = OnInterstitialAdClosed;
}
The second “OnInterstitialAdClosed” is the name of the function that you will be defining. So it can be changed to whatever you like. You can generate a new function manually by using the name you defined in the right hand side of the equal sign. However, since some callbacks come with parameters from the plugin, it is recommended that you show the potential fixes and generate one from there automatically. By this way, you will also which parameters, if any, will be returned from the callback. You can check other methods in all kits by typing “HMS…Manager.Instance.On…” and see which callbacks are supported.
Analytics KitIn the code above, you can see an AnalyticsManager instance is used. That is the whole implementation of analytics kit I have in my game. After I enabled it from the Kit Settings menu in the Unity Editor, I can call HMSAnalyticsManager however I like, as in the other managers. Here, my scenario for Analytics Kit is sending the score of the player to the server to see how much score people are achieving. You can use, just as I did, SendEventWithBundle function to send whatever items need to be sent as your game logic requires. The function requires an EventID, key and a value to be sent that can be of type string or int. That’s it and now you can check if the parameters are coming to AGC console by checking the tab “Real-time Overview”
Some of the events are predefined. As long as you enable Analytics Kit, they will be sent to the console anyway, even if you do not code anything in your game. If you cannot see your own custom event here, I would suggest adding that even manually from the Events tab close to bottom in AGC console. You can also look for support from the Huawei Developer website and/or forum.
Crash KitCrash Kit, when clicked, is automatically implemented thanks to the plugin. You should see crash reports in AGC Console, if any crash appears. You can also deliberately cause your app to crush, but how to do so is outside the scope of this article.
Push KitPush Kit, like Crash Kit, is also automatically enabled when you tick the box besides it. It is complete and ready to use. Simple as that, you should go to AGC Console, create a notification and you should receive notification depending on the time you set.
Tips & Tricks
Do not forget to get the agconnect-services.json file from the AGC and paste it to the StreamingAssets folder. The folder must be placed inside Assets folder of the Unity files.
In case any method or code piece that I shared will not work and you face any compiler complaints, make sure that you have imported the right libraries with “using” keyword. For any usage of the plugin, first line below is required and the other two lines are needed for most functions related to the kits. Make sure you imported them to use them smoothly.
using HmsPlugin;
using HuaweiMobileServices.Game;
using HuaweiMobileServices.Utils;
It is very normal that your game is different and the scenario you wanted this kits may differ from mine. That’s why, I always talked about how my need/scenario for the kit and then implemented it. If you are having trouble how to convert this to your own game logic, try to understand what I did and where, so you can implement the similar functionality where you want it to be in your own game.
Please make sure that your package name ends with either . Huawei or with .HUAWEI if you want to use IAP in your game. If you used another package name, please change it for AppGallery. Otherwise, IAP will not work. IAP will be talked about in part 3 but I wanted to warn here, so if you do not have an existing game yet, you can start off right.
ConclusionIn this article, we integrated many kits to our game. This was not just a “to integrate, do this” kind of article, but rather I tried to show you a real game scenario, so while you are applying this article to your game, you will hopefully have a better understanding. With Plugin 2.0, the speed of integration has increased dramatically, reduced down from days to perhaps hours depending on the complexity of your app.
I hope that this article has been helpful for you. You can always ask questions below, if you have anything unanswered in your mind.
Remaining two kits will be talked about here, in the part 2.
See you there!
References
HMS Unity Plugin 2.0 Branch (Github Page)
Documentation of every single kit in Huawei Docs (Links are present in the GitHub readme)
Checkout in forum
As an industry-leading analytics company for mobile games, GameAnalytics empowers developers, helping them to build more engaging user experiences through a strong understanding of their game's core metrics. GameAnalytics provides critical insights to help developers tap into the potential of their mobile games through collecting, analyzing, and presenting game performance data. This includes everything from basic metrics (such as active users, retention, and playtime,) to more advanced analytics regarding ad revenue, virtual currency, and level progression.
The partnership brings mutual benefits to both parties; Huawei can continue growing its ecosystem with more platform partners and further support the success of its game developers. Simultaneously, GameAnalytics can leverage Huawei's technological capabilities, enhancing its Android SDK to support OAID (Huawei Ads Kit) across all mobile devices. This enables HMS ecosystem game developers to integrate with the platform, to unlock deeper analysis and continue to grow their games.
Used by nearly 100,000 developers and over 63,000 studios worldwide, GameAnalytics supports around 100,000 active games, providing necessary data to help mobile developers achieve their growth goals. GameAnalytics can be easily integrated in less than 15 minutes, and its core analytics tool has and will continue to remain free.
How to get started
If you’re a Huawei game developer, or want to become one and use GameAnalytics, here’s what you need to do:
1 Download and integrate our Android SDK. Head to our documentation to learn how.
2 Integrate Huawei Ads SDK by following these steps.
3 And you’re done! The Android SDK can now automatically detect if the OAID is available to be used.
Please note, if both GAID and OAID are available on a player’s device, then GAID will be used as a primary device ID, though both will be collected and available in features like the Player Warehouse, Event Export, or Raw Export. If you have any questions, just get in touch with our friendly support team.
Find more here:https://gameanalytics.com/product-updates/gameanalytics-partners-with-huawei
For more information, please visit Huawei Partner Site and GameAnalytics website SDK integration guide
Martin Bieber said:
As an industry-leading analytics company for mobile games, GameAnalytics empowers developers, helping them to build more engaging user experiences through a strong understanding of their game's core metrics. GameAnalytics provides critical insights to help developers tap into the potential of their mobile games through collecting, analyzing, and presenting game performance data. This includes everything from basic metrics (such as active users, retention, and playtime,) to more advanced analytics regarding ad revenue, virtual currency, and level progression.
The partnership brings mutual benefits to both parties; Huawei can continue growing its ecosystem with more platform partners and further support the success of its game developers. Simultaneously, GameAnalytics can leverage Huawei's technological capabilities, enhancing its Android SDK to support OAID (Huawei Ads Kit) across all mobile devices. This enables HMS ecosystem game developers to integrate with the platform, to unlock deeper analysis and continue to grow their games.
Used by nearly 100,000 developers and over 63,000 studios worldwide, GameAnalytics supports around 100,000 active games, providing necessary data to help mobile developers achieve their growth goals. GameAnalytics can be easily integrated in less than 15 minutes, and its core analytics tool has and will continue to remain free.
How to get started
If you’re a Huawei game developer, or want to become one and use GameAnalytics, here’s what you need to do:
1 Download and integrate our Android SDK. Head to our documentation to learn how.
2 Integrate Huawei Ads SDK by following these steps.
3 And you’re done! The Android SDK can now automatically detect if the OAID is available to be used.
Please note, if both GAID and OAID are available on a player’s device, then GAID will be used as a primary device ID, though both will be collected and available in features like the Player Warehouse, Event Export, or Raw Export. If you have any questions, just get in touch with our friendly support team.
Find more here:https://gameanalytics.com/product-updates/gameanalytics-partners-with-huawei
For more information, please visit Huawei Partner Site and GameAnalytics website SDK integration guide
Click to expand...
Click to collapse
The emergence of AR technology has allowed us to interact with our devices in a new and unexpected way. With regard to smart device development, from PCs to mobile phones and beyond, the process has been dramatically simplified. Interactions have been streamlined to the point where only slides and taps are required, and even children as young as 2 or 3 can use devices.
Rather than having to rely on tools like keyboards, mouse devices, and touchscreens, we can now control devices in a refreshingly natural and easy way. Traditional interactions with smart devices have tended to be cumbersome and unintuitive, and there is a hunger for new engaging methods, particularly among young people. Many developers have taken heed of this, building practical but exhilarating AR features into their apps. For example, during live streams, or when shooting videos or images, AR-based apps allow users to add stickers and special effects with newfound ease, simply by striking a pose; in smart home scenarios, users can use specific gestures to turn smart home appliances on and off, or switch settings, all without any screen operations required; or when dancing using a video game console, the dancer can raise a palm to pause or resume the game at any time, or swipe left or right to switch between settings, without having to touch the console itself.
So what is the technology behind these groundbreaking interactions between human and devices?
HMS Core AR Engine is a preferred choice among AR app developers. Its SDK provides AR-based capabilities that streamline the development process. This SDK is able to recognize specific gestures with a high level of accuracy, output the recognition result, and provide the screen coordinates of the palm detection box, and both the left and right hands can be recognized. However, it is important to note that when there are multiple hands within an image, only the recognition results and coordinates from the hand that has been most clearly captured, with the highest degree of confidence, will be sent back to your app. You can switch freely between the front and rear cameras during the recognition.
Gesture recognition allows you to place virtual objects in the user's hand, and trigger certain statuses based on the changes to the hand gestures, providing a wealth of fun interactions within your AR app.
The hand skeleton tracking capability works by detecting and tracking the positions and postures of up to 21 hand joints in real time, and generating true-to-life hand skeleton models with attributes like fingertip endpoints and palm orientation, as well as the hand skeleton itself.
AR Engine detects the hand skeleton in a precise manner, allowing your app to superimpose virtual objects on the hand with a high degree of accuracy, including on the fingertips or palm. You can also perform a greater number of precise operations on virtual hands and objects, to enrich your AR app with fun new experiences and interactions.
Getting StartedPrepare the development environment as follows:
JDK: 1.8.211 or later
Android Studio: 3.0 or later
minSdkVersion: 26 or later
targetSdkVersion: 29 (recommended)
compileSdkVersion: 29 (recommended)
Gradle version: 6.1.1 or later (recommended)
Before getting started, make sure that the AR Engine APK is installed on the device. You can download it from AppGallery. Click here to learn on which devices you can test the demo.
Note that you will need to first register as a Huawei developer and verify your identity on HUAWEI Developers. Then, you will be able to integrate the AR Engine SDK via the Maven repository in Android Studio. Check which Gradle plugin version you are using, and configure the Maven repository address according to the specific version.
App Development1. Check whether AR Engine has been installed on the current device. Your app can run properly only on devices with AR Engine installed. If it is not installed, you need to prompt the user to download and install AR Engine, for example, by redirecting the user to AppGallery. The sample code is as follows:
Code:
boolean isInstallArEngineApk =AREnginesApk.isAREngineApkReady(this);
if (!isInstallArEngineApk) {
// ConnectAppMarketActivity.class is the activity for redirecting users to AppGallery.
startActivity(new Intent(this, com.huawei.arengine.demos.common.ConnectAppMarketActivity.class));
isRemindInstall = true;
}
2. Initialize an AR scene. AR Engine supports the following five scenes: motion tracking (ARWorldTrackingConfig), face tracking (ARFaceTrackingConfig), hand recognition (ARHandTrackingConfig), human body tracking (ARBodyTrackingConfig), and image recognition(ARImageTrackingConfig).
Call ARHandTrackingConfig to initialize the hand recognition scene.
Code:
mArSession = new ARSession(context);
ARHandTrackingConfig config = new ARHandTrackingconfig(mArSession);
3. You can set the front or rear camera as follows after obtaining an ARhandTrackingconfig object.
Code:
Config.setCameraLensFacing(ARConfigBase.CameraLensFacing.FRONT);
4. After obtaining config, configure it in ArSession, and start hand recognition.
Code:
mArSession.configure(config);
mArSession.resume();
5. Initialize the HandSkeletonLineDisplay class, which draws the hand skeleton based on the coordinates of the hand skeleton points.
Code:
Class HandSkeletonLineDisplay implements HandRelatedDisplay{
// Methods used in this class are as follows:
// Initialization method.
public void init(){
}
// Method for drawing the hand skeleton. When calling this method, you need to pass the ARHand object to obtain data.
public void onDrawFrame(Collection<ARHand> hands,){
// Call the getHandskeletonArray() method to obtain the coordinates of hand skeleton points.
Float[] handSkeletons = hand.getHandskeletonArray();
// Pass handSkeletons to the method for updating data in real time.
updateHandSkeletonsData(handSkeletons);
}
// Method for updating the hand skeleton point connection data. Call this method when any frame is updated.
public void updateHandSkeletonLinesData(){
// Method for creating and initializing the data stored in the buffer object.
GLES20.glBufferData(..., mVboSize, ...);
//Update the data in the buffer object.
GLES20.glBufferSubData(..., mPointsNum, ...);
}
}
6. Initialize the HandRenderManager class, which is used to render the data obtained from AR Engine.
Code:
Public class HandRenderManager implements GLSurfaceView.Renderer{
// Set the ARSession object to obtain the latest data in the onDrawFrame method.
Public void setArSession(){
}
}
7. Initialize the onDrawFrame() method in the HandRenderManager class.
Code:
Public void onDrawFrame(){
// In this method, call methods such as setCameraTextureName() and update() to update the calculation result of ArEngine.
// Call this API when the latest data is obtained.
mSession.setCameraTextureName();
ARFrame arFrame = mSession.update();
ARCamera arCamera = arFrame.getCamera();
// Obtain the tracking result returned during hand tracking.
Collection<ARHand> hands = mSession.getAllTrackables(ARHand.class);
// Pass the obtained hands object in a loop to the method for updating gesture recognition information cyclically for processing.
For(ARHand hand : hands){
updateMessageData(hand);
}
}
8. On the HandActivity page, set a render for SurfaceView.
Code:
mSurfaceView.setRenderer(mHandRenderManager);
Setting the rendering mode.
mSurfaceView.setRenderMode(GLEurfaceView.RENDERMODE_CONTINUOUSLY);
Physical controls and gesture-based interactions come with unique advantages and disadvantages. For example, gestures are unable to provide the tactile feedback provided by keys, especially crucial for shooting games, in which pulling the trigger is an essential operation; but in simulation games and social networking, gesture-based interactions provide a high level of versatility.
Gestures are unable to replace physical controls in situations that require tactile feedback, and physical controls are unable to naturally reproduce the effects of hand movements and complex hand gestures, but there is no doubt that gestures will become indispensable to future smart device interactions.
Many somatosensory games, smart home appliances, and camera-dependent games are now using AR to offer a diverse range of smart, convenient features. Common gestures include eye movements, pinches, taps, swipes, and shakes, which users can strike without having to learn additionally. These gestures are captured and identified by mobile devices, and used to implement specific functions for users. When developing an AR-based mobile app, you will need to first enable your app to identify these gestures. AR Engine helps by dramatically streamlining the development process. Integrate the SDK to equip your app with the capability to accurately identify common user gestures, and trigger corresponding operations. Try out the toolkit for yourself, to explore a treasure trove of powerful, interesting AR features.
References
AR Engine Development Guide
AR Engine Sample Code