HUAWEI ML Kit's Automatic Speech Recognition (ASR) Service: Tongue Twisters - Huawei Developers

Overview
When I try to perform voice commands on my devices, the device will often fail to recognize what I am trying to say, because of my poor pronunciation. For example, sometimes I can't distinguish between syllables, or make the "ch" and "sh" sounds, which have led to some frustrating experiences. I've always envied people who can enunciate well, and recite tongue twisters with ease, and have dreamed of the day when that could be me. By chance, I came across the game Tongue Twister, which integrates HUAWEI ML Kit's ASR service, and has changed my life for the better. Let's take a look at how the game works.
Application Scenarios
There are five levels in Tongue Twister, and as you'd expect, each level contains a tongue twister. The key for passing each level is ML Kit's ASR service. By integrating the service, the game is able to recognize the player's voice with a high degree of accuracy. Players are thus able to pass each level when they demonstrate clear enunciation. The service has proven itself to be highly useful in certain fields, enhancing recognition capabilities for product, movie, and music searches, as well as navigation services.
Now, let's look at what the game looks like in practice.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Piqued your interest? With the ASR service, why not create a tongue twister game of your own? Here's how...
Development Procedures
1. For details about how to set the authentication information for your app, please refer to Notes on Using Cloud Authentication Information.
2. Call an API to create a speech recognizer.
Code:
MLAsrRecognizer mSpeechRecognizer = MLAsrRecognizer.createAsrRecognizer(context);
3. Create a speech recognition result listener callback.
Code:
/**
* Use the callback to implement the MLAsrListener API and methods in the API.
*/
protected class SpeechRecognitionListener implements MLAsrListener {
@Override
public void onStartListening() {
// The recorder starts to receive speech.
}
@Override
public void onStartingOfSpeech() {
// The user starts to speak, that is, the speech recognizer detects that the user starts to speak.
}
@Override
public void onVoiceDataReceived(byte[] data, float energy, Bundle bundle) {
// Return the original PCM stream and audio power to the user.
}
@Override
public void onRecognizingResults(Bundle partialResults) {
// Receive the recognized text from MLAsrRecognizer.
}
@Override
public void onResults(Bundle results) {
// Text data of ASR.
}
}
@Override
public void onError(int error, String errorMessage) {
// If you don't add this, there will be no response after you cut the network
}
@Override
public void onState(int state, Bundle params) {
// Notify the app status change.
}
}
4. Bind the new result listener callback to the speech recognizer.
Code:
mSpeechRecognizer.setAsrListener(new SpeechRecognitionListener());
5. Set the recognition parameters and initiate speech recognition.
Code:
// Set parameters and start the audio device.
Intent mSpeechRecognizerIntent = new Intent(MLAsrConstants.ACTION_HMS_ASR_SPEECH);
mSpeechRecognizerIntent
// Set the language that can be recognized to English. If this parameter is not set,
// English is recognized by default. Example: "zh-CN": Chinese;"en-US": English;"fr-FR": French;"es-ES": Spanish;"de-DE": German;"it-IT": Italian.
.putExtra(MLAsrConstants.LANGUAGE, language)
// Set to return the recognition result along with the speech. If you ignore the setting, this mode is used by default. Options are as follows:
// MLAsrConstants.FEATURE_WORDFLUX: Recognizes and returns texts through onRecognizingResults.
// MLAsrConstants.FEATURE_ALLINONE: After the recognition is complete, texts are returned through onResults.
.putExtra(MLAsrConstants.FEATURE, MLAsrConstants.FEATURE_WORDFLUX);mSpeechRecognizer.startRecognizing(mSpeechRecognizerIntent);
6. Release resources when the recognition ends.
Code:
if (mSpeechRecognizer != null) {
mSpeechRecognizer.destroy();
mSpeechRecognizer = null;
}
Maven repository address
Code:
buildscript {
repositories {
maven { url 'https://developer.huawei.com/repo/' }
}
}
allprojects {
repositories {
maven { url 'https://developer.huawei.com/repo/' }
}
}
SDK import
Code:
dependencies {
// Automatic speech recognition Long voice SDK.
implementation 'com.huawei.hms:ml-computer-voice-realtimetranscription:2.0.3.300'
// Automatic speech recognition SDK.
implementation 'com.huawei.hms:ml-computer-voice-asr:2.0.3.300'
// Automatic speech recognition plugin.
implementation 'com.huawei.hms:ml-computer-voice-asr-plugin:2.0.3.300'
}
Manifest files
Code:
<manifest
...
<meta-data
android:name="com.huawei.hms.ml.DEPENDENCY"
android:value="ocr />
...
</manifest>
Permission
Code:
<uses-permission android:name="android.permission.RECORD_AUDIO" />
Dynamic permission application
Code:
private void requestCameraPermission() {
final String[] permissions = new String[]{Manifest.permission.RECORD_AUDIO};
if (!ActivityCompat.shouldShowRequestPermissionRationale(this,
Manifest.permission.RECORD_AUDIO)) { ActivityCompat.requestPermissions(this,
permissions,
TongueTwisterActivity.AUDIO_CODE);
return;
}
}
Summary
In addition to game applications, ML Kit's ASR service also takes effect in other scenarios, such as in shopping apps. The service is able to recognize a spoken product name or feature, and convert it into text to search for the product. For music apps, the service can likewise, recognize song and artist names. For navigation as well, the driver will naturally prefer to speak a destination rather than type it, and have it converted into text using ASR, to enjoy an optimally safe driving experience.
Learn More
For more information, please visit HUAWEI Developers.
For detailed instructions, please visit Development Guide.
You can join the HMS Core developer discussion by going to Reddit.
You can download the demo and sample code on GitHub.
To solve integration problems, please go to Stack Overflow.