ML Kit: Text Recognition Development Procedure - Huawei Developers

Text recognition from images on the device
1. Create the text analyzer MLTextAnalyzer to recognize text in images. You can set MLLocalTextSetting to specify languages that can be recognized. If you do not set the languages, only Latin-based languages can be recognized by default.
Code:
// Method 1: Use default parameter settings to configure the on-device text analyzer. Only Latin-based languages can be recognized.
MLTextAnalyzer analyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer();
// Method 2: Use the customized parameter MLLocalTextSetting to configure the text analyzer on the device.
MLLocalTextSetting setting = new MLLocalTextSetting.Factory()
.setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
// Specify languages that can be recognized.
.setLanguage("en")
.create();
MLTextAnalyzer analyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting);
2. Create an MLFrame using android.graphics.Bitmap. JPG, JPEG, PNG, and BMP images are supported. It is recommended that the length-width ratio range from 1:2 to 2:1.
Code:
// Create an MLFrame object using the bitmap, which is the image data in bitmap format.
MLFrame frame = MLFrame.fromBitmap(bitmap);
3. Pass the MLFrame object to the asyncAnalyseFrame method for text recognition.
Code:
Task<MLText> task = analyzer.asyncAnalyseFrame(frame);
task.addOnSuccessListener(new OnSuccessListener<MLText>() {
@Override
public void onSuccess(MLText text) {
// Recognition success.
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Recognition failure.
}
});
The sample code uses the asynchronous call mode. Local text recognition also supports synchronous call. The recognition result is specified by the MLText.Block array.
Code:
Context context = getApplicationContext();
MLTextAnalyzer analyzer = new MLTextAnalyzer.Factory(context).create();
SparseArray<MLText.Block> blocks = analyzer.analyseFrame(frame);
4. After the recognition is complete, stop the analyzer to release recognition resources.
Code:
if (analyzer != null) {
analyzer.close();
}
Text recognition from images on the cloud
1. Creat an analyzer. The recommended way is to using MLRemoteTextSetting. In this way, you can specify to-be-recognized languages for more accurate text recognition.
Code:
// Method 1: Use customized parameter settings.
MLRemoteTextSetting setting = new MLRemoteTextSetting.Factory()
//Set the on-cloud text detection mode.
// MLRemoteTextSetting.OCR_COMPACT_SCENE: dense text recognition
// MLRemoteTextSetting.OCR_LOOSE_SCENE: sparse text recognition
.setTextDensityScene(MLRemoteTextSetting.OCR_LOOSE_SCENE)
// Specify the languages that can be recognized, which should comply with ISO 639-1.
.setLanguageList(new ArrayList<String>(){{this.add("zh"); this.add("en");}})
// Set the format of the returned text border box.
// MLRemoteTextSetting.NGON: Return the coordinates of the four vertices of the quadrilateral.
// MLRemoteTextSetting.ARC: Return the vertices of a polygon border in an arc. The coordinates of up to 72 vertices can be returned.
.setBorderType(MLRemoteTextSetting.ARC)
.create();
MLTextAnalyzer analyzer = MLAnalyzerFactory.getInstance().getRemoteTextAnalyzer(setting);
// Method 2: Use the default parameter settings to automatically detect languages for text recognition. This method is applicable to sparse text scenarios. The format of the returned text box is MLRemoteTextSetting.NGON.
MLTextAnalyzer analyzer = MLAnalyzerFactory.getInstance().getRemoteTextAnalyzer();
2. Create an MLFrame using the bitmap. JPG, JPEG, PNG, and BMP images are supported.
Code:
MLFrame frame = MLFrame.fromBitmap(bitmap);
3. Pass the MLFrame object to the asyncAnalyseFrame method for text recognition.
Code:
Task<MLText> task = analyzer.asyncAnalyseFrame(frame);
task.addOnSuccessListener(new OnSuccessListener<MLText>() {
@Override
public void onSuccess(MLText text) {
// Recognition success.
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(Exception e) {
// Recognition failure.
}
});
4. After the recognition is complete, stop the analyzer to release recognition resources.
Code:
try {
if (analyzer != null) {
ananlzer.close;
}
} catch (IOException e) {
// Exception handling.
}
Text recognition from camera streams on the device
Your app can process camera streams, convert camera frames into the MLFrame object, and recognize text using the local static image recognition method. If the synchronous recognition API is called, your app can also use the LensEngine class built in the SDK to locally detect text in camera streams and create and initialize a LensEngine object. For details, please refer to Sample Code.
1. Create an analyzer.
Code:
MLTextAnalyzer analyzer = new MLTextAnalyzer.Factory(context).create();
2. Create the OcrDetectorProcessor class for processing detection results. This class provides the MLAnalyzer.MLTransactor<T> API, which uses the transactResult method to obtain the detection results and implement specific services.
Code:
public class OcrDetectorProcessor implements MLAnalyzer.MLTransactor<MLText.Block> {
@Override
public void transactResult(MLAnalyzer.Result<MLText.Block> results) {
SparseArray<MLText.Block> items = results.getAnalyseList();
// Determine detection result processing as required. Note that only the detection results are processed.
// Other detection-related APIs provided by HUAWEI ML Kit cannot be called.
}
@Override
public void destroy() {
// Callback method used to release resources when the detection ends.
}
}
3. Set the detection result processor to bind the analyzer to the result processor.
Code:
analyzer.setTransactor(new OcrDetectorProcessor());
4. Call the synchronous API to use the built-in LensEngine of the SDK to create an object, register the analyzer, and initialize camera parameters.
Code:
LensEngine lensEngine = new LensEngine.Creator(getApplicationContext(),analyzer)
.setLensType(LensEngine.BACK_LENS)
.applyDisplayDimension(1440, 1080)
.applyFps(30.0f)
.enableAutomaticFocus(true)
.create();
5. Call the run method to start the camera and read camera streams for recognition.
Code:
// Implement other logic of the SurfaceView control by yourself.
SurfaceView mSurfaceView = findViewById(R.id.surface_view);
try {
lensEngine.run(mSurfaceView.getHolder());
} catch (IOException e) {
// Exception handling logic.
}
6. After the recognition is complete, stop the analyzer to release recognition resources.
Code:
if (analyzer != null) {
try {
analyzer.close();
} catch (IOException e) {
// Exception handling.
}
}
if (lensEngine != null) {
lensEngine.release();
}
In camera stream detection, when MLAnalyzer.MLTransactor<T> is inherited to process detection results, if your app needs to stop detection after a specific result is detected and continue detection after the result is processed, please refer to Development for Multi Detections in Camera Stream Detection Mode.