Sound Event Detection using ML kit | JAVA - Huawei Developers

Introduction
Sound detection service can detect sound events. Automatic environmental sound classification is a growing area of research with real world applications.
Steps
1. Create App in Android
2. Configure App in AGC
3. Integrate the SDK in our new Android project
4. Integrate the dependencies
5. Sync project
Use case
This service we will use in day to day life, it will detect different types of sounds such as Baby crying, laugher, snoring, running water, alarm sounds, doorbell, etc.! Currently this service will detect only one sound at a time, so multiple sound detection is not supporting this service. Default interval at least 2 seconds for each sound detection.
ML Kit Configuration.
1. Login into AppGallery Connect, select MlKitSample in My Project list.
2. Enable Ml Kit, Choose My Projects > Project settings > Manage APIs
Integration
Create Application in Android Studio.
App level gradle dependencies.
Code:
apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'
Gradle dependencies
Code:
implementation 'com.huawei.hms:ml-speech-semantics-sounddect-sdk:2.0.3.300'
implementation 'com.huawei.hms:ml-speech-semantics-sounddect-model:2.0.3.300'
Root level gradle dependencies
Code:
maven {url 'https://developer.huawei.com/repo/'}
classpath 'com.huawei.agconnect:agcp:1.3.1.300'
Add the below permissions in Android Manifest file
Code:
<manifest xlmns:android...>
...
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
</manifest>
1. Create Instance for Sound Detection in onCreate.
Code:
MLSoundDector soundDector = MLSoundDector.createSoundDector();
2. Check Run time permissions.
Code:
private void getRuntimePermissions() {
List<String> allNeededPermissions = new ArrayList<>();
for (String permission : getRequiredPermissions()) {
if (!isPermissionGranted(this, permission)) {
allNeededPermissions.add(permission);
}
}
if (!allNeededPermissions.isEmpty()) {
ActivityCompat.requestPermissions(
this, allNeededPermissions.toArray(new String[0]), PERMISSION_REQUESTS);
}
}
private boolean allPermissionsGranted() {
for (String permission : getRequiredPermissions()) {
if (!isPermissionGranted(this, permission)) {
return false;
}
}
return true;
}
private static boolean isPermissionGranted(Context context, String permission) {
if (ContextCompat.checkSelfPermission(context, permission)
== PackageManager.PERMISSION_GRANTED) {
Log.i(TAG, "Permission granted: " + permission);
return true;
}
Log.i(TAG, "Permission NOT granted: " + permission);
return false;
}
private String[] getRequiredPermissions() {
try {
PackageInfo info = this.getPackageManager().getPackageInfo(this.getPackageName(), PackageManager.GET_PERMISSIONS);
String[] ps = info.requestedPermissions;
if (ps != null && ps.length > 0) {
return ps;
} else {
return new String[0];
}
} catch (RuntimeException e) {
throw e;
} catch (Exception e) {
return new String[0];
}
}
@Override
public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
if (requestCode != PERMISSION_REQUESTS) {
return;
}
boolean isNeedShowDiag = false;
for (int i = 0; i < permissions.length; i++) {
if ((permissions[i].equals(Manifest.permission.READ_EXTERNAL_STORAGE)
&& grantResults[i] != PackageManager.PERMISSION_GRANTED)
|| (permissions[i].equals(Manifest.permission.CAMERA)
&& permissions[i].equals(Manifest.permission.RECORD_AUDIO)
&& grantResults[i] != PackageManager.PERMISSION_GRANTED)) {
isNeedShowDiag = true;
}
}
if (isNeedShowDiag && !ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.CALL_PHONE)) {
AlertDialog dialog = new AlertDialog.Builder(this)
.setMessage(getString(R.string.camera_permission_rationale))
.setPositiveButton(getString(R.string.settings), new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog, int which) {
Intent intent = new Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS);
intent.setData(Uri.parse("package:" + getPackageName()));
startActivityForResult(intent, 200);
startActivity(intent);
}
})
.setNegativeButton(getString(R.string.cancel), new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog, int which) {
finish();
}
}).create();
dialog.show();
}
}
3. Create sound detection result callback, this callback will detect the sound results.
Code:
MLSoundDectListener listener = new MLSoundDectListener() {
@Override
public void onSoundSuccessResult(Bundle result) {
int soundType = result.getInt(MLSoundDector.RESULTS_RECOGNIZED);
String soundName = hmap.get(soundType);
textView.setText("Successfully sound has been detected : " + soundName);
}
@Override
public void onSoundFailResult(int errCode) {
textView.setText("Failure" + errCode);
}
};
soundDector.setSoundDectListener(listener);
soundDector.start(this);
4. Once sound detection obtained call notification service.
Code:
serviceIntent = new Intent(MainActivity.this, NotificationService.class);
serviceIntent.putExtra("response", soundName);
ContextCompat.startForegroundService(MainActivity.this, serviceIntent);
5. If you want to stop sound detection call onStop()
Code:
soundDector.stop();
6. Below are the sound type results.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Result
Conclusion
This article will help you to detect Real time streaming sounds, sound detection service will help you to notify sounds to users in daily life.
Thank you for reading and if you have enjoyed this article, I would suggest you implement this and provide your experience.
Reference
ML Kit – Sound Detection
Refer the URL