HiAi Image Super Resolution - Improve Low Resolution Images - Huawei Developers

This article is originally from HUAWEI Developer Forum
Forum link: https://forums.developer.huawei.com/forumPortal/en/home​
HiAi Image Super Resolution
Upscales an image or reduces image noise and improves image details without changing the resolution.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Upscales an image or reduces image noise and improves image details without changing the resolution.
Base on AI deep learning of CV (Computer Vision)
Utilize Huawei NPU (Neural Processing Unit), 50X faster than CPU
1X & 3X super-resolution furnish images with clearer effect, reducing JPEG compression noise
You can check the offical documentation about HiAi Image super resolution.
Huawei continuous investment on NPU technology
Huawei Phones support HiAI
Software: Huawei EMUI 9.0 & above
Hardware: CPU 970,810, 820,985,990
Codelab
https://developer.huawei.com/consumer/en/codelab/HiAIImageSuperresolution/index.html#0
You can also follow the codelab to implement the HiAi image resolution with the help DevEco IDE plugin in Android Studio.
Project: (HiAi Image Super Resolution)
In this article we are going to make project in which we can implement HiAi Image Super Resolution to improve low resolution image quality which is used in most of the application as a thumbnail images.
1. Implementation:
Download the vision-oversea-release.aar package in the Huawei AI Engine SDKs from the Huawei developer community.
Copy the downloaded vision-oversea-release.aar package to the app/libs directory of the project.
Add the following code to build.gradle in the APP directory of the project, and add vision-release.aar to the project. Dependency on the Gson library must be added, because the conversion of parameters and results between the JavaScript Object Notation (JSON) and Java classes inside vision-release.aar depends on the Gson library.
Code:
repositories {
flatDir {
dirs 'libs'
}
}
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar'])
implementation(name: 'vision-oversea-release', ext: 'aar')
implementation 'com.google.code.gson:gson:2.8.6'
}
2. Assets:
In this section we adding some low resolution images in "assets/material/image_super_resolution" directory, to further fetch images for local direction for optimization.
3. Design ListView:
In this section we are design ListView in our layouts to show original images and optimized images.
activity_main.xml
Code:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="vertical"
tools:context=".MainActivity">
<LinearLayout
android:id="@+id/linearLayout"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginTop="1pt"
android:layout_marginBottom="5pt"
android:orientation="horizontal"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent">
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_weight="1"
android:gravity="center"
android:text="Original Image"
android:textSize="24sp" />
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_weight="1"
android:gravity="center"
android:text="Improved Image"
android:textSize="24sp" />
</LinearLayout>
<ListView
android:id="@+id/item_listView"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:dividerHeight="3pt"
/>
</LinearLayout>
items.xml
Code:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:orientation="horizontal"
android:layout_width="match_parent"
android:layout_height="match_parent"
>
<ImageView
android:id="@+id/imgOriginal"
android:layout_width="100dp"
android:layout_height="100dp"
app:srcCompat="@drawable/noimage"
android:layout_gravity="start"
android:layout_weight="1"
/>
<TextView
android:id="@+id/imgTitle"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text=" "
android:layout_gravity="center_horizontal"
android:layout_weight="0"
android:textAlignment="center"
/>
<ImageView
android:id="@+id/imgConverted"
android:layout_width="100dp"
android:layout_height="100dp"
app:srcCompat="@drawable/noimage"
android:layout_gravity="end"
android:layout_weight="1"
app:layout_constraintDimensionRatio="h,4:3"
/>
</LinearLayout>
4. Coding: (Adapter, HiAi Image Super Resolution )
Util Class:
　　We make a util class AssetsFileUtil, to get all the images from local assets directory, get single BitMap image.
Code:
public class AssetsFileUtil {
public static Bitmap getBitmapByFilePath(Context context, String filePath){
try{
AssetManager assetManager = context.getAssets();
InputStream is = assetManager.open(filePath);
Bitmap bitmap = BitmapFactory.decodeStream(is);
return bitmap;
}catch (Exception e){
e.printStackTrace();
return null;
}
}
public static List<Bitmap> getBitmapListByDirPath(Context context, String dirPath){
List<Bitmap> list = new ArrayList<Bitmap>();
try{
AssetManager assetManager = context.getResources().getAssets();
String[] photos = assetManager.list(dirPath);
for(String photo : photos){
if(isFile(photo)){
Bitmap bitmap = getBitmapByFilePath(context,dirPath + "/" + photo);
list.add(bitmap);
}else {
List<Bitmap> childList = getBitmapListByDirPath(context,dirPath + "/" + photo);
list.addAll(childList);
}
}
}catch (Exception e){
e.printStackTrace();
}
return list;
}
public static List<String> getFileNameListByDirPath(Context context, String dirPath){
List<String> list = new ArrayList<String>();
try{
AssetManager assetManager = context.getResources().getAssets();
String[] photos = assetManager.list(dirPath);
for(String photo : photos){
if(isFile(photo)){
list.add(dirPath + "/" + photo);
}else {
List<String> childList = getFileNameListByDirPath(context,dirPath + "/" + photo);
list.addAll(childList);
}
}
}catch (Exception e){
e.printStackTrace();
}
return list;
}
public static boolean isFile(String fileName){
if(fileName.contains(".")){
return true;
}else {
return false;
}
}
}
MainActivity Class:
　　In this class we are getting local images and attaching images list to our Adapter
Code:
public class MainActivity extends AppCompatActivity {
private String mDirPath;
private ArrayList<Item> itemList;
private List<String> imageList;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
itemList = new ArrayList<Item>();
getLocalImages();
// Setting Adapter and listview
ItemAdapter itemAdapter = new ItemAdapter(getApplicationContext(), R.layout.items, itemList);
ListView listView = findViewById(R.id.item_listView);
listView.setAdapter(itemAdapter);
}
public void getLocalImages(){
mDirPath ="material/image_super_resolution";
imageList = AssetsFileUtil.getFileNameListByDirPath(this,mDirPath);
for(int i=0; i<imageList.size();i++){
itemList.add(new Item(imageList.get(i), " ", imageList.get(i)));
}
}
}
Item Class:
　　Prepare item data class.
Code:
public class Item {
private String imgOriginal;
private String imgTitle;
private String imgConverted;
public Item(String imgOriginal, String imgTitle, String imgConverted) {
this.imgOriginal = imgOriginal;
this.imgTitle = imgTitle;
this.imgConverted = imgConverted;
}
public String getImgOriginal() {
return imgOriginal;
}
public void setImgOriginal(String imgOriginal) {
this.imgOriginal = imgOriginal;
}
public String getImgTitle() {
return imgTitle;
}
public void setImgTitle(String imgTitle) {
this.imgTitle = imgTitle;
}
public String getImgConverted() {
return imgConverted;
}
public void setImgConverted(String imgConverted) {
this.imgConverted = imgConverted;
}
}
ItemAdapter Class:
In this class we are binding our images array list with our layout ImageView and implement HiAi Image Resolution to optimize image resolution.
ItemAdapter class extends ArrayAdapter with the type of Item class.
Code:
public class ItemAdapter extends ArrayAdapter<Item>
Define some constants
Code:
private ArrayList<Item> itemList;
private final static int SUPERRESOLUTION_RESULT = 110;
private Bitmap bitmapOriginal;
private Bitmap bitmapConverted;
ImageView imgOriginal;
ImageView imgConverted;
private String TAG = "ItemAdapter";
Prepare constructor for the Adpater class
Code:
public ItemAdapter(@NonNull Context context, int resource, @NonNull ArrayList<Item> itemList) {
super(context, resource, itemList);
this.itemList = itemList;
}
Define initHiAi function to check service connected or disconnected
Code:
/**
* init HiAI interface
*/
private void initHiAI() {
/** Initialize with the VisionBase static class and asynchronously get the connection of the service */
VisionBase.init(getContext(), new ConnectionCallback() {
@Override
public void onServiceConnect() {
/** This callback method is invoked when the service connection is successful; you can do the initialization of the detector class, mark the service connection status, and so on */
}
@Override
public void onServiceDisconnect() {
/** When the service is disconnected, this callback method is called; you can choose to reconnect the service here, or to handle the exception*/
}
});
}
Define setHiAi function to perform HiAi operation on Original image and generate the optimized Bitmap.
Code:
/**
* Capability Interfaces
*
* @return
*/
private void setHiAi() {
/** Define class detector, the context of this project is the input parameter */
ImageSuperResolution superResolution = new ImageSuperResolution(getContext());
/** Define the frame, put the bitmap that needs to detect the image into the frame*/
Frame frame = new Frame();
/** BitmapFactory.decodeFile input resource file path*/
// Bitmap bitmap = BitmapFactory.decodeFile(null);
frame.setBitmap(bitmapOriginal);
/** Define and set super-resolution parameters*/
SuperResolutionConfiguration paras = new SuperResolutionConfiguration(
SuperResolutionConfiguration.SISR_SCALE_3X,
SuperResolutionConfiguration.SISR_QUALITY_HIGH);
superResolution.setSuperResolutionConfiguration(paras);
/** Run super-resolution and get result of processing */
ImageResult result = superResolution.doSuperResolution(frame, null);
/** After the results are processed to get bitmap*/
Bitmap bmp = result.getBitmap();
/** Note: The result and the Bitmap in the result must be NULL, but also to determine whether the returned error code is 0 (0 means no error)*/
this.bitmapConverted = bmp;
handler.sendEmptyMessage(SUPERRESOLUTION_RESULT);
}
Define Handler if the optimization completed attach the optimized Bitmap to image.
Code:
private Handler handler = new Handler() {
@Override
public void handleMessage(Message msg) {
super.handleMessage(msg);
switch (msg.what) {
case SUPERRESOLUTION_RESULT:
if (bitmapConverted != null) {
imgConverted.setImageBitmap(bitmapConverted);
} else { // Set the original image
imgConverted.setImageBitmap(bitmapOriginal);
// toast("High Resolution image");
}
break;
}
}
};
Override the getView method attached orginial image to image view and process the original image to attached optimized image.
Code:
@NonNull
@Override
public View getView(int position, @Nullable View convertView, @NonNull ViewGroup parent) {
initHiAI();
int itemIndex = position;
if(convertView == null){
convertView = LayoutInflater.from(getContext()).inflate(R.layout.items,parent, false);
}
imgOriginal = convertView.findViewById(R.id.imgOriginal);
TextView imgTitle = convertView.findViewById(R.id.imgTitle);
imgConverted = convertView.findViewById(R.id.imgConverted);
bitmapOriginal = AssetsFileUtil.getBitmapByFilePath(imgOriginal.getContext(), itemList.get(itemIndex).getImgConverted());
imgOriginal.setImageBitmap(bitmapOriginal);
bitmapConverted =AssetsFileUtil.getBitmapByFilePath(imgConverted.getContext(), itemList.get(itemIndex).getImgOriginal());
imgConverted.setImageBitmap(bitmapConverted);
int height = bitmapOriginal.getHeight();
int width = bitmapOriginal.getWidth();
Log.e(TAG, "width:" + width + ";height:" + height);
if (width <= 800 && height <= 600) {
new Thread() {
@Override
public void run() {
setHiAi();
}
}.start();
} else {
toast("Width and height of the image cannot exceed 800*600");
}
imgTitle.setText(itemList.get(itemIndex).getImgTitle());
return convertView;
}
public void toast(String text) {
Toast.makeText(getContext(), text, Toast.LENGTH_SHORT).show();
}
Coding section has been complete here, now run you project and check the output of the Image Optimization by using HiAi Image Super Resolution.
5. Result