Search News with Voice (Search Kit — ML Kit(ASR)— Network Kit) - Huawei Developers

{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Hello everyone. In this article, I will try to talk about the uses of Huawei Search Kit, Huawei ML Kit and Huawei Network Kit. I have developed a demo app using these 3 kits to make it clearer.
What is Search Kit?​HUAWEI Search Kit fully opens Petal Search capabilities through the device-side SDK and cloud-side APIs, enabling ecosystem partners to quickly provide the optimal mobile app search experience.
What is Network Kit?​Network Kit is a basic network service suite. It incorporates Huawei’s experience in far-field network communications, and utilizes scenario-based RESTful APIs as well as file upload and download APIs. Therefore, Network Kit can provide you with easy-to-use device-cloud transmission channels featuring low latency, high throughput, and high security.
What is ML Kit — ASR?​Automatic speech recognition (ASR) can recognize speech not longer than 60s and convert the input speech into text in real time. This service uses industry-leading deep learning technologies to achieve a recognition accuracy of over 95%.
Development Steps​1. Integration
First of all, we need to create an app on AppGallery Connect and add related details about HMS Core to our project. You can access the article about that steps from the link below.
Android | Integrating Your Apps With Huawei HMS Core
Hi, this article explains you how to integrate with HMS (Huawei Mobile Services) and making AppGallery Connect Console project settings.
medium.com
2. Adding Dependencies
After HMS Core is integrated into the project and the Search Kit and ML Kit are activated through the console, the required library should added to the build.gradle file in the app directory as follows. The project’s minSdkVersion value should be 24. For this, the minSdkVersion value in the same file should be updated to 24.
Java:
...
defaultConfig {
applicationId "com.myapps.searchappwithml"
minSdkVersion 24
targetSdkVersion 30
versionCode 1
versionName "1.0"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
...
dependencies {
...
implementation 'com.huawei.agconnect:agconnect-core:1.5.0.300'
implementation 'com.huawei.hms:network-embedded:5.0.1.301'
implementation 'com.huawei.hms:searchkit:5.0.4.303'
implementation 'com.huawei.hms:ml-computer-voice-asr-plugin:2.2.0.300'
...
}
3. Adding Permissions
Java:
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
4. Application Class
When the application is started, we need to initialize the Kits in the Application class. Then we need to specify the Application class to the “android: name” tag in the manifest file.
Java:
@HiltAndroidApp
class SearchApplication : Application(){
override fun onCreate() {
super.onCreate()
initNetworkKit()
initSearchKit()
initMLKit()
}
private fun initNetworkKit(){
NetworkKit.init(
applicationContext,
object : NetworkKit.Callback() {
override fun onResult(result: Boolean) {
if (result) {
Log.i(NETWORK_KIT_TAG, "init success")
} else {
Log.i(NETWORK_KIT_TAG, "init failed")
}
}
})
}
private fun initSearchKit(){
SearchKitInstance.init(this, APP_ID)
CoroutineScope(Dispatchers.IO).launch {
SearchKitInstance.instance.refreshToken()
}
}
private fun initMLKit() {
MLApplication.getInstance().apiKey = API_KEY
}
}
5. Getting Access Token
We need to use Access Token to send requests to Search Kit. I used the Network Kit to request the Access Token. Its use is very similar to services that perform other Network operations.
As with other Network Services, there are Annotations such as POST, FormUrlEncoded, Headers, Field.
Java:
interface AccessTokenService {
@POST("oauth2/v3/token")
@FormUrlEncoded
@Headers("Content-Type:application/x-www-form-urlencoded", "charset:UTF-8")
fun createAccessToken(
@Field("grant_type") grant_type: String,
@Field("client_secret") client_secret: String,
@Field("client_id") client_id: String
) : Submit<String>
}
We need to create our request structure using the RestClient class.
Java:
@Module
@InstallIn(ApplicationComponent::class)
class ApplicationModule {
companion object{
private const val TIMEOUT: Int = 500000
private var restClient: RestClient? = null
fun getClient() : RestClient {
val httpClient = HttpClient.Builder()
.connectTimeout(TIMEOUT)
.writeTimeout(TIMEOUT)
.readTimeout(TIMEOUT)
.build()
if (restClient == null) {
restClient = RestClient.Builder()
.baseUrl("https://oauth-login.cloud.huawei.com/")
.httpClient(httpClient)
.build()
}
return restClient!!
}
}
}
Finally, by sending the request, we reach the AccessToken.
Java:
data class AccessTokenModel (
var access_token : String,
var expires_in : Int,
var token_type : String
)
...
fun SearchKitInstance.refreshToken() {
ApplicationModule.getClient().create(AccessTokenService::class.java)
.createAccessToken(
GRANT_TYPE,
CLIENT_SECRET,
CLIENT_ID
)
.enqueue(object : Callback<String>() {
override fun onFailure(call: Submit<String>, t: Throwable) {
Log.d(ACCESS_TOKEN_TAG, "getAccessTokenErr " + t.message)
}
override fun onResponse(
call: Submit<String>,
response: Response<String>
) {
val convertedResponse =
Gson().fromJson(response.body, AccessTokenModel::class.java)
setInstanceCredential(convertedResponse.access_token)
}
})
}
6. ML Kit (ASR) — Search Kit
Since we are using ML Kit (ASR), we first need to get microphone permission from the user. Then we start ML Kit (ASR) with the help of a button and get a text from the user. By sending this text to the function we created for the Search Kit, we reach the data we will show on the screen.
Here I used the Search Kit’s Web search feature. Of course, News, Image, Video search features can be used according to need.
Java:
@AndroidEntryPoint
class MainActivity : AppCompatActivity() {
private lateinit var binding: MainBinding
private val adapter: ResultAdapter = ResultAdapter()
private var isPermissionGranted: Boolean = false
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = MainBinding.inflate(layoutInflater)
setContentView(binding.root)
binding.button.setOnClickListener {
if (isPermissionGranted) {
startASR()
}
}
binding.recycler.adapter = adapter
val permission = arrayOf(Manifest.permission.INTERNET, Manifest.permission.RECORD_AUDIO)
ActivityCompat.requestPermissions(this, permission,MIC_PERMISSION)
}
override fun onRequestPermissionsResult(
requestCode: Int,
permissions: Array<out String>,
grantResults: IntArray
) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults)
when (requestCode) {
MIC_PERMISSION -> {
// If request is cancelled, the result arrays are empty.
if (grantResults.isNotEmpty()
&& grantResults[0] == PackageManager.PERMISSION_GRANTED
&& grantResults[1] == PackageManager.PERMISSION_GRANTED) {
// permission was granted
Toast.makeText(this, "Permission granted", Toast.LENGTH_SHORT).show()
isPermissionGranted = true
} else {
// permission denied,
Toast.makeText(this, "Permission denied", Toast.LENGTH_SHORT).show()
}
return
}
}
}
private fun startASR() {
val intent = Intent(this, MLAsrCaptureActivity::class.java)
.putExtra(MLAsrCaptureConstants.LANGUAGE, "en-US")
.putExtra(MLAsrCaptureConstants.FEATURE, MLAsrCaptureConstants.FEATURE_WORDFLUX)
startActivityForResult(intent, ASR_REQUEST_CODE)
}
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (requestCode == ASR_REQUEST_CODE) {
when (resultCode) {
MLAsrCaptureConstants.ASR_SUCCESS -> if (data != null) {
val bundle = data.extras
if (bundle != null && bundle.containsKey(MLAsrCaptureConstants.ASR_RESULT)) {
val text = bundle.getString(MLAsrCaptureConstants.ASR_RESULT).toString()
performSearch(text)
}
}
MLAsrCaptureConstants.ASR_FAILURE -> if (data != null) {
val bundle = data.extras
if (bundle != null && bundle.containsKey(MLAsrCaptureConstants.ASR_ERROR_CODE)) {
val errorCode = bundle.getInt(MLAsrCaptureConstants.ASR_ERROR_CODE)
Toast.makeText(this, "Error Code $errorCode", Toast.LENGTH_LONG).show()
}
if (bundle != null && bundle.containsKey(MLAsrCaptureConstants.ASR_ERROR_MESSAGE)) {
val errorMsg = bundle.getString(MLAsrCaptureConstants.ASR_ERROR_MESSAGE)
Toast.makeText(this, "Error Code $errorMsg", Toast.LENGTH_LONG).show()
}
}
else -> {
Toast.makeText(this, "Failed to get data", Toast.LENGTH_LONG).show()
}
}
}
}
private fun performSearch(query: String) {
CoroutineScope(Dispatchers.IO).launch {
val searchKitInstance = SearchKitInstance.instance
val webSearchRequest = WebSearchRequest().apply {
setQ(query)
setLang(loadLang())
setSregion(loadRegion())
setPs(5)
setPn(1)
}
val response = searchKitInstance.webSearcher.search(webSearchRequest)
displayResults(response.data)
}
}
private fun displayResults(data: List<WebItem>) {
runOnUiThread {
adapter.items.apply {
clear()
addAll(data)
}
adapter.notifyDataSetChanged()
}
}
}
Output​
Conclusion​By using these 3 kits effortlessly, you can increase the quality of your application in a short time. I hope this article was useful to you. See you in other articles
References​Network Kit: https://developer.huawei.com/consum...s-V5/network-introduction-0000001050440045-V5
ML Kit: https://developer.huawei.com/consum...s-V5/service-introduction-0000001050040017-V5
Search Kit: https://developer.huawei.com/consum...re-Guides-V5/introduction-0000001055591730-V5