OFFICAL TWRP RELEASED, this thread is no longer active. 3-30-2018
https://twrp.me/motorola/motorolamotox4.html
---
UNOFFICIAL BUILDS -- USE AT YOUR OWN RISK AND KNOW HOW TO GET YOURSELF OUT OF TROUBLE IF IT ARISES. I ASSUME NO RESPONSIBILITY FOR YOUR BROKEN THINGS.
UPDATED 01-11-2018
There are now 7.1 based and 8.0 based builds. Obviously, be careful to select the proper download. While I don't think flashing/booting the wrong one would permanently brick the device, let's not find out
For now I will not be attempting to make data decryption work. It is REQUIRED that you unencrypt your device by formatting userdata, so back up your stuff first.
IMPORTANT NOTES: TWRP for OREO is a bit of a pain as it currently requires manually editing your fstab. I have tried to automate this process, but it breaks stuff. So just be aware before you begin that it is a rather time consuming process.
If you make any change to your boot partition after flashing SuperSU, you will need to reflash it or you will get a bootloop.
8.0 OREO TWRP INSTALLATION:
OREO SEEMS VERY FINICKY AND DOES NOT LIKE CHANGES TO ITS FILESYSTEM -- BE PREPARED TO REFLASH STOCK.
0. FLASH OREO FACTORY IMAGE (may work otherwise, but we should be starting from fresh stock here)
1. Download FASTBOOT BOOTABLE TWRP for 8.0/OREO below
2. Download SuperSU 2.82 SR5 below
3. Move SuperSU to external SD or USB OTG
4. From bootloader, fasboot BOOT TWRP
5. Flash SuperSU (note: do not format /data now... not necessary and will cause errors on boot)
6. Reboot system
7. With any root file editor/text editor (Amaze, Total Commander, etc) open /system/vendor/etc/fstab.qcom as a text file for editing.
8. At the end of the /data partition entry, delete "fileencryption=ice" and replace it with "encryptable=footer".
9. Save fstab.qcom (and make sure it is actually saved properly!)
10. Reboot to bootloader and fasboot BOOT TWRP
11. Go to Wipe, hit the FORMAT DATA button, and type "yes" to format /data. This will erase your data, obviously:silly:
12. Reboot system (should now be unencrypted, verify in Settings>Security or by booting TWRP and checking /data with File Manager.
If you later choose to flash TWRP (not the bootable we used here!), you may need to flash SuperSU again to avoid bootloops.
7.1.1 NOUGAT TWRP INSTALLATION:
1. Download current TWRP for 7.1.1 build below
2. Download SuperSU 2.85 SR5 (https://forum.xda-developers.com/apps/supersu/2014-09-02-supersu-v2-05-t2868133
3. Move SuperSU to your external Micro SD card.
4. Fastboot flash the TWRP image.
5. Reboot to TWRP.
6. FORMAT data (not wipe...use the "FORMAT DATA" button and type "yes". OBVIOUSLY THIS WILL ERASE YOUR DATA)
7. Install SuperSU 2.85 SR5
8. Reboot to system (it WILL bootloop a couple times...don't panic!) and confirm that device is unencrypted by checking that SETTINGS>SECURITY>ENCRYPTION now prompts "encrypt" (don't do it).
DOWNLOADS:
TWRP FOR 7.1.1 (Nougat) DOWNLOAD: https://drive.google.com/open?id=1Et-AQgCNx7WDAwzihlI51euUa2ixKHEP
TWRP FOR 8.0 (Oreo) DOWNLOAD: https://drive.google.com/open?id=1WcVS_3rloF7jxPulj_jKxfsp3zy5pB5N
FASTBOOT BOOTABLE TWRP IMAGE (OREO BASED): https://drive.google.com/open?id=12ClviqtEjtflB63UQ1CZQNKEqkprBO0u **For temporary TWRP boot using "fastboot boot". Do not flash or you will be stuck in recovery!**
DEVICE TREE: https://github.com/mightysween/android_device_motorola_payton (NEEDS TO BE UPDATED WITH OREO BRANCH)
changelog:
BETA4
-reverted to 3.2.0 for current build (release candidate rebased to 3.2.1).
-fixed "format data" button
-finalized fstab for OTG/SD/INTERNAL mounting
-target is now UNENCRYPTED devices only (/data decrypt will not be fixed)
BETA3
-Rebased to TWRP 3.2.1
-USB OTG working
-all partitions mounting correctly
-considered working except for decrypt and MTP/ADB
BETA2
-fixed internal storage mount
BETA1
- updated source to TWRP 3.2.0
ALPHA3
- Fixed USB mounting (adb/mtp still nonfunctional) BROKEN IN BETA1
ALPHA2
- SD Card fixed
ALPHA1
- /system is now properly mounted.
- now plays nice with our working root method.
- ramdisk is patched to prevent first boot encryption once /data is decrypted (now requires flashable ZIP)
NOT WORKING:
adb/mtp/sideload
/data decryption (abandoned -- /data access requires unencryption)
CREDITS: @kraatus90 for kernel fix, @Chainfire for SuperSU, @jcadduono for no-verity-opt-encrypt scripts.
---
Thanks so much for all your work!
hi,
you said this is really unstable and could brick the device easily. however, you seem to be testing and experimenting with your device a lot, so i'd like to ask if you have any particual unbrick method that you use when something goes wrong.. like, a via fastboot flashable image or something simmilar...?
thanks for your work!
Thanks for you work. If you have any unbrick methods pls tell.appreciated your work ?
I am not going to provide step by step "unbrick" methods, because until the partitioning is properly set up, TWRP has potential access to things that can not be fixed.
Again, this is still highly experimental.
All that said, my entire process to protect any device remains the same: Have a backup for every partition you will be testing, make as few changes as possible at a time, test boot images before flashing (fastboot boot), and test restore methods frequently (flashing back to stock or backups), and don't do anything unless you are highly certain of the outcome.
By those standards, there is nothing to gain by installing TWRP right now, as its basic function (install/backup/restore) is not set up yet.
Found the BoardConfig flag to enable FBE (TW_INCLUDE_CRYPTO_FBE := true) but do not have the lib it is dependent on (libe4crypt) and I don't see it anywhere yet...
For reference (not sure this is most current, but it demonstrates the process)
https://github.com/nijel8/TWRP/commit/bd7492de28963b7e74e8e5d3f17ec9a5a287d9c3
I have confirmed that FBE support is present in the source, dependent on this missing module... so need to figure out where/how to enable it.
It is possible that this entire process is specific to only certain devices (i.e. Pixel, Nexus). If this is the case, we may be stuck at this point for awhile.
Obvious workaround is to not be encrypted to begin with -- but that isn't a "solution".
mightysween said:
It is possible that this entire process is specific to only certain devices (i.e. Pixel, Nexus). If this is the case, we may be stuck at this point for awhile.
Click to expand...
Click to collapse
This appears to be the case, unfortunately. Seems that the TWRP FBE support was built specifically for the Google implementation of FBE which was merged into kernel sources for Nexus and Pixel. Not even using the qseecomd I assumed it was... will remove on next build.
Info on FBE:
https://source.android.com/security/encryption/file-based
Will be testing options to disable forced encryption, and if necessary dm-verity...
Anyone who wants to dig through kernel for related flags and props, it would be greatly appreciated!
Hey, where did the big "format data" button go in TWRP? Is that optional on compile... can't find a flag for it...
Making good progress this morning.
Seem to have a build with properly decrypted /system, and working SD Card. I also have patched the boot.img to disable forced encryption on the first boot. But now, I can not find a safe way to fully format (not "wipe") the /data partition. As mentioned in the previous post, the "FORMAT DATA" button is missing. The fastboot command "fastboot format userdata" returns an error that it does not support RAW format.
Need to figure out why this is happening... and once I do, I believe I can reformat /data without encryption and then will have an almost fully working TWRP build. Obviously, the ideal solution would be to have TWRP work out of the gate with an encrypted /data, but until then this is going to be our best option.
Will post an updated test build in the OP soon.... needs further testing before I would recommend non-expert users to try it.
Getting very close now!
UPDATE: ADDED NEW BUILD TO OP
---
Also, just occurred to me that the ramdisk will need to be patched every time, so now that I have SD card support will be testing some of the existing flashable ZIPs out there that are designed specificially to prevent first-boot encryption and/or disable dm-verity.
---
mightysween said:
Hey, where did the big "format data" button go in TWRP? Is that optional on compile... can't find a flag for it...
Click to expand...
Click to collapse
This is really the only hold up... I changed the partition from 'Advanced Wipe", but as expected, it was still encrypted on boot as it doesn't actually format the footer where encryption is stored. I can't figure out where that darn "FORMAT DATA" button ran off to, and that is exactly what we need here.
mightysween said:
This is really the only hold up... I changed the partition from 'Advanced Wipe", but as expected, it was still encrypted on boot as it doesn't actually format the footer where encryption is stored. I can't figure out where that darn "FORMAT DATA" button ran off to, and that is exactly what we need here.
Click to expand...
Click to collapse
I don't know how to help except to say that using TWRP 3.1.0-MOD_1 with my XT1254 (DROID Turbo), when you go to wipe it has two buttons --- one on the left for Advanced Wipe and Format Data on the right.
johnjingle said:
I don't know how to help except to say that using TWRP 3.1.0-MOD_1 with my XT1254 (DROID Turbo), when you go to wipe it has two buttons --- one on the left for Advanced Wipe and Format Data on the right.
Click to expand...
Click to collapse
I have 3.1.1-0 (same version I am building here) on several other devices, and the button is there. It has to be triggered by something during compile, but I can't figure it out. Driving me nuts
Looking through TWRP source, and can find actions for every other button (wipe, backup, restore, install, etc) but not for format. Hmm.
I posted over on an old but semi-active TWRP flags thread, maybe someone will have some insight.
Wondering if I make a stock boot image without the encryption tag in fstab, and then wipe data and reboot... then flash my image. That may make it so the data partition is never encrypted in the first place and allow TWRP to work.
But that is an ugly, non-user friendly fix. Why can't we just format /data?
---
Final update for today... this seems to be a compile issue, which is a good thing. I tried to manually decrypt /data from the TWRP command line, and got this:
No crypto support was compiled into this build.
Click to expand...
Click to collapse
So, I must be missing something in boardconfig still... and maybe need to set up a small proprietary vendor folder with the necessary libs
mightysween said:
Final update for today... this seems to be a compile issue, which is a good thing. I tried to manually decrypt /data from the TWRP command line, and got this:
So, I must be missing something in boardconfig still... and maybe need to set up a small proprietary vendor folder with the necessary libs
Click to expand...
Click to collapse
Thanks for doing all of this! I wish I had the time and knowledge to help.
Had a few PM's checking on TWRP status, so an update.
The good news is that the X4 is using Qualcomm based decryption for /data... the bad news is that most if it seems to be closed source. This will take some time for me to figure out, but I have already made some progress by sifting through logs.
Right now, I am trying to find a device with similar decrypt scheme to have some more guidance on the process.
mightysween said:
Had a few PM's checking on TWRP status, so an update.
The good news is that the X4 is using Qualcomm based decryption for /data... the bad news is that most if it seems to be closed source. This will take some time for me to figure out, but I have already made some progress by sifting through logs.
Right now, I am trying to find a device with similar decrypt scheme to have some more guidance on the process.
Click to expand...
Click to collapse
dont know all about these things but maybe xiaomi mi a1 twrp can help as it is also using same a/b partition. and twrp is already there for it
vivek638 said:
dont know all about these things but maybe xiaomi mi a1 twrp can help as it is also using same a/b partition. and twrp is already there for it
Click to expand...
Click to collapse
Thanks, the Mi A1 is one of the devices I have been comparing to, and has been quite helpful.
mightysween said:
Thanks, the Mi A1 is one of the devices I have been comparing to, and has been quite helpful.
Click to expand...
Click to collapse
Keep searching. wish i could have helped but dont know anything about compiling n all..
I'm not much of a programmer, but I can look through the files. Is there anything in particular we're searching for?
Hi everyone and Happy New Year,
I am trying to open ROM_0 file created with SP Flash tool. I have tried ROM explorer 0.9.1, I have tried various option converting with simg2img and opening with 7zip but nothing has worked so far.
The file is about 100GB and it is a SP Flash tool backup of my userdata on which I have a lot of images which i need to save.
I was using Dot OS 5.2 general image and a message popped up about trying Android 12 and I have clicked on it just to get rid of it but I assume it has triggered a download. My phone crashed yesterday evening when I started the cmera app and once restarted it was in a boot loop mode stuck on the dot os logo.
So far I have tried various options unsuccessful - I have reflashed the image which I originally flashed, I have set the partitions active - a and b and reverted to the initial active one which was "a".
I have also flashed system.img (with the treble general image) but still it is in a boot loop mode.
I have just decided to flash back the super.img image from the stock and guess what - still stuck.
Flashed the stock boot.img again thinking there might be an issue with the kernel but that didn't help.
I understand that it is the case of fully flashing back the stock ROM which will lock the bootloader and delete all my userdata in order to have the phone back.
However the phone IS NOT important, the ONLY IMPORTANT thing are the images in the userdata.
I have created the backup of it straight after the boot loop appeared. Tried to read here on XDA but it is not clear what format is that file and how I can access the data on it.
Looked for a recovery partition but there is none. Potentially hidden as you can get into stock recovery via fastbootd. But the options there are only to wipe the partitions/reset.
The phone is Umidigi Bison Pro and I have been having all but troubles with it.
Any help greatly appreciated it.
Regards
s80_gad said:
Hi everyone and Happy New Year,
I am trying to open ROM_0 file created with SP Flash tool. I have tried ROM explorer 0.9.1, I have tried various option converting with simg2img and opening with 7zip but nothing has worked so far.
The file is about 100GB and it is a SP Flash tool backup of my userdata on which I have a lot of images which i need to save.
I was using Dot OS 5.2 general image and a message popped up about trying Android 12 and I have clicked on it just to get rid of it but I assume it has triggered a download. My phone crashed yesterday evening when I started the cmera app and once restarted it was in a boot loop mode stuck on the dot os logo.
So far I have tried various options unsuccessful - I have reflashed the image which I originally flashed, I have set the partitions active - a and b and reverted to the initial active one which was "a".
I have also flashed system.img (with the treble general image) but still it is in a boot loop mode.
I have just decided to flash back the super.img image from the stock and guess what - still stuck.
Flashed the stock boot.img again thinking there might be an issue with the kernel but that didn't help.
I understand that it is the case of fully flashing back the stock ROM which will lock the bootloader and delete all my userdata in order to have the phone back.
However the phone IS NOT important, the ONLY IMPORTANT thing are the images in the userdata.
I have created the backup of it straight after the boot loop appeared. Tried to read here on XDA but it is not clear what format is that file and how I can access the data on it.
Looked for a recovery partition but there is none. Potentially hidden as you can get into stock recovery via fastbootd. But the options there are only to wipe the partitions/reset.
The phone is Umidigi Bison Pro and I have been having all but troubles with it.
Any help greatly appreciated it.
Regards
Click to expand...
Click to collapse
May I'm wrong, but I guess that if you didn't give it an extension then the file doesn't have a format; when you make a backup of a partition using SP Flash tool you should give it an extension, for example userdata_backup.img will work, in some devices, for some partition the .bin extension is used.
And to restore the device to a working state without losing data you could flash the stock ROM unchecking the userdata partition and using Download only option won't re-lock your bootloader.
If actually your userdata was not overwritten you still can try a second attempt to preserve it using mtk-client, search for it in GitHub, also consider what I stated about re-flash your original ROM preserving the userdata partition.
Thanks SubwayChamp, I appreciate your comment.
I have tried .img, .bin, ext4 etc but cannot open it - I am not sure if there is another application that can convert it in a readable format or maybe if we can mount it and access the files.
I had the impression that if you flash the stock rom the bootloader is locked and you loose everything.
But thanks for your advice - I will flash everything apart from the userdata partition which is last in the order anyway. Should I select or deselect the preloader partition- will that make a difference?
Regards
Just flashed the full stock rom without the userdata partition - still stuck on the logo in a boot loop . I really need to open the userdata backup file from SP flash tool as I feel I have to do a full reset/wipe.
Any other suggestions about explorer for the sp flash dump file, please?
Regards
s80_gad said:
Just flashed the full stock rom without the userdata partition - still stuck on the logo in a boot loop . I really need to open the userdata backup file from SP flash tool as I feel I have to do a full reset/wipe.
Any other suggestions about explorer for the sp flash dump file, please?
Regards
Click to expand...
Click to collapse
No, I didn't say to change the extension now and try it in various format, unfortunately I feel that if you didn't give you the extension at the time to make a backup then the file is unreadable, what I mean is that when you make the dump through SP Flash tool you have to give to the file a name and an extension, not letting it as is offered by SP Flash tool, for example you did see the name ROM_0 or similar, but you have to give it a name and an extension, in this case userdata_backup.img would work.
Did you check mtk-client?, you can read (dump) the userdata partition through this CLI tool, and after that you can restore it at any time.
Using the download option (only) you never re-lock your bootloader.
But wait a minute, keep in mind that your device is A/b, so you have to double-try all the things, for example, if you want to flash a specific partition like boot you have to be sure in which partition you are right now BUT unfortunately you don't know which partition is the working one, so better use fastboot to flash the missed partition, target to both slots.
And what about the option to get to a custom recovery? (I guess you had it previously to flash CR Droid) either taking a backup of userdata or re-flashing the same CR Droid that was functional previously.
Thanks SubwayChamp for your reply.
So I will try to dump the userdata again then - I still haven't touched it so I hope the partition and the data on it is fine.
I assume it is that mtkclient you are referring to. Will see if I can get some time today to try the live cd first as I am on Windows at this moment.
So my device is indeed A/B - the system is on "a" and I have flashed dot os using fastbootd and overwriting the system.img within the super.img. It worked fine for about 20 days until that crash (I only assume it is due to the update - nothing else has happened that could create trouble).
Also tried to set the b partition active but didn't help so switched back to "a".
Unfortunately there is no recovery partition, from what I learned the recovery is within the boot img. I have tried to load temporary unofficial twrp - fastboot boot twrp.img - and the first step is ok, but then it crashes. so no luck to load custom recovery even temporary in order to save the userdata on sdcard.
Tried to get to the contents trough adb shell but while some directories are listed, I get access denied to the userdata - I think maybe the links are broken?
I will try with the mtk to see if I can back it up - and what I'll do is I'll flash the full stock rom including the userdata and potentially will try to flash the old userdata through fastboot or sp flash or mtk.
TBH I don't understand why the phone is still in a bootloop - can't be only because I haven't cleared the userdata?
Regards
s80_gad said:
Thanks SubwayChamp for your reply.
So I will try to dump the userdata again then - I still haven't touched it so I hope the partition and the data on it is fine.
I assume it is that mtkclient you are referring to. Will see if I can get some time today to try the live cd first as I am on Windows at this moment.
Click to expand...
Click to collapse
It works on Windows though.
s80_gad said:
So my device is indeed A/B - the system is on "a" and I have flashed dot os using fastbootd and overwriting the system.img within the super.img. It worked fine for about 20 days until that crash (I only assume it is due to the update - nothing else has happened that could create trouble).
Click to expand...
Click to collapse
The issue was originated due to the lack of the other system files that also occupy this space; vendor, odm, product (may vary depending on the device), can be fixed flashing the super.img using fastbootd again.
s80_gad said:
Also tried to set the b partition active but didn't help so switched back to "a".
Unfortunately there is no recovery partition, from what I learned the recovery is within the boot img. I have tried to load temporary unofficial twrp - fastboot boot twrp.img - and the first step is ok, but then it crashes. so no luck to load custom recovery even temporary in order to save the userdata on sdcard.
Click to expand...
Click to collapse
Yes, this device doesn't have a dedicated recovery partition, but it is placed in a tiny portion of the boot image (usually the ramdisk) you can try by flashing the TWRP image onto the boot partition (flashing, not booting only) then boot to it, do the stuff you need through TWRP, from there you could solve the bootloop. To can boot to Android again you should need to flash a boot image.
s80_gad said:
Tried to get to the contents trough adb shell but while some directories are listed, I get access denied to the userdata - I think maybe the links are broken?
Click to expand...
Click to collapse
No, it's encrypted.
s80_gad said:
I will try with the mtk to see if I can back it up - and what I'll do is I'll flash the full stock rom including the userdata and potentially will try to flash the old userdata through fastboot or sp flash or mtk.
TBH I don't understand why the phone is still in a bootloop - can't be only because I haven't cleared the userdata?
Regards
Click to expand...
Click to collapse
When you flashed a system image onto the super partition the other partitions that are set dynamically didn't find a place to be recreated or couldn't play its role, added to this, a different system image that which is contained in the super image can differ in sizes either logical and/or dynamical (virtual sized).
SubwayChamp said:
The issue was originated due to the lack of the other system files that also occupy this space; vendor, odm, product (may vary depending on the device), can be fixed flashing the super.img using fastbootd again.
Click to expand...
Click to collapse
Flashed already the original stock rom super. img and everything else apart from userdata - it doesn't work.
see below
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
SubwayChamp said:
Yes, this device doesn't have a dedicated recovery partition, but it is placed in a tiny portion of the boot image (usually the ramdisk) you can try by flashing the TWRP image onto the boot partition (flashing, not booting only) then boot to it, do the stuff you need through TWRP, from there you could solve the bootloop. To can boot to Android again you should need to flash a boot image.
Click to expand...
Click to collapse
Tried to flash it - it just restarts the phone straight away - in fact replaced it with sp flash tool as well which recognises only the "a" partition and flashes it there.
SubwayChamp said:
No, it's encrypted.
Click to expand...
Click to collapse
I see
SubwayChamp said:
When you flashed a system image onto the super partition the other partitions that are set dynamically didn't find a place to be recreated or couldn't play its role, added to this, a different system image that which is contained in the super image can differ in sizes either logical and/or dynamical (virtual sized).
Click to expand...
Click to collapse
I am guessing this is why I have to reflash the whole rom incl userdata in order to make the phone usable.
What I'll do is I'll try to dump userdata with mtk and then will reflash everything with the stock rom ()hopefully the phone will boot) and then will flash the dumped userdata with mtk. Hopefully that will work.
I'll see if I can somehow mount the mtk .bin file to see if I can get to the contents of it
Will have to use the live dvd as I have win 7 and python 3.9 cannot run on win 7.
EDIT: Can't start anything through the live dvd - is there any workaround for win 7 or is there a direct executable file which I can get to start the mtkclient?
Regards
Hello,
I also have an Umidigi Bison Pro that I am going to use as a daily driver. (It's a pity that it's unpopular it would be a great device for modding, it's cheap, rugged and has source code availability of the official ROM and kernel). I created a Telegram group about this phone if you want to join is https://t.me/UmidigiBisonPro
About your problem you can read this guide (it describes how to backup and extract from the file created by SP Flash Tool even the partitions that not visible such as the b slots) https://www.hovatek.com/forum/thread-21970.html
To give you an idea on my Bison Pro a total of 52 partitions were extracted.
If you have the full backup from before the bootloop (before the upgrade, when it was still working) my advice is to restore all partitions.
I consider myself a novice regarding modding but it is likely that after the upgrade the userdata partition is no longer readable.
I have read that you should not update the GSI ROMs but repeat the whole flash sequence.
I also recommend removing the forced encryption of the userdata partition (you can do this when rooting) to avoid exactly these problems where you have the partition backup but not the decryption key.
s80_gad said:
Flashed already the original stock rom super. img and everything else apart from userdata - it doesn't work.
see below
View attachment 5499133
Tried to flash it - it just restarts the phone straight away - in fact replaced it with sp flash tool as well which recognises only the "a" partition and flashes it there.
I see
I am guessing this is why I have to reflash the whole rom incl userdata in order to make the phone usable.
What I'll do is I'll try to dump userdata with mtk and then will reflash everything with the stock rom ()hopefully the phone will boot) and then will flash the dumped userdata with mtk. Hopefully that will work.
I'll see if I can somehow mount the mtk .bin file to see if I can get to the contents of it
Will have to use the live dvd as I have win 7 and python 3.9 cannot run on win 7.
EDIT: Can't start anything through the live dvd - is there any workaround for win 7 or is there a direct executable file which I can get to start the mtkclient?
Regards
Click to expand...
Click to collapse
Sorry for delay, I didn't receive any notification on this (or I didn't notice it), I hope you sorted out your issue, if not, let me know.
SubwayChamp said:
Sorry for delay, I didn't receive any notification on this (or I didn't notice it), I hope you sorted out your issue, if not, let me know.
Click to expand...
Click to collapse
I didn't received notification too on your message and I found out on profile account that the notification for new message on a thread are default disabled.
I recently had some problems and experimented with partitions.
Reducing the possible cases I think the decryption key for the userdata partition might be in these partitions: super , misc , nvdata , nvcfg , md_udc
and I noticed that if one of them is corrupted/different version the dm-verity check fails (in my case it is written on the screen) and it was necessary to reflash all partitions except userdata (I don't know if there is a faster combination, from the few tests done in this case I didn't find any)
Do you have more information about where the decryption key might be between those partitions?
I have made a brief description of the role of all the partitions encountered but I still don't know some of them:
boot_para
gz_a (/ gz_b)
md_udc
otp
spmfw_a (/ spmfw_b)
sspm_a (/ sspm_b)
teksunhw_a (/ teksunhw_b)
Werve said:
I didn't received notification too on your message and I found out on profile account that the notification for new message on a thread are default disabled.
I recently had some problems and experimented with partitions.
Reducing the possible cases I think the decryption key for the userdata partition might be in these partitions: super , misc , nvdata , nvcfg , md_udc
and I noticed that if one of them is corrupted/different version the dm-verity check fails (in my case it is written on the screen) and it was necessary to reflash all partitions except userdata (I don't know if there is a faster combination, from the few tests done in this case I didn't find any)
Do you have more information about where the decryption key might be between those partitions?
I have made a brief description of the role of all the partitions encountered but I still don't know some of them:
boot_para
gz_a (/ gz_b)
md_udc
otp
spmfw_a (/ spmfw_b)
sspm_a (/ sspm_b)
teksunhw_a (/ teksunhw_b)
Click to expand...
Click to collapse
Why do you think userdata has a decryption key? Unless the user set it in a backup done through a custom recovery or through the device itself, I don't think so, may I'm wrong, but which is your scenario?
SubwayChamp said:
Why do you think userdata has a decryption key? Unless the user set it in a backup done through a custom recovery or through the device itself, I don't think so, may I'm wrong, but which is your scenario?
Click to expand...
Click to collapse
Since the userdata partition is now usually encrypted either with FBE or FDE but once the system loads the files are readable and moveable even externally then it is clear that somehow the data has been decrypted precisely using the relevant decryption key, AES encryption usually.
So if the user has not specified any key this must be derived from the information already in the partitions from the factory.
Then by restoring the right combination of partitions the system can boot correctly by decrypting the userdata partition. Hence the tests and the report I wrote in my last post.
At the moment I was able to remove the forced encryption of the userdata partition by modifying super (specifically fstab present in the /vendor sub partition) but I would like to achieve the same systemless modification using Magisk (to be OTA compatible). Unfortunately, the options to remove dm-verity and forceencrypt have been hidden in the latest versions of Magisk to avoid problems with inexperienced uses.
Since I don't have a custom recovery on the Umidigi Bison Pro I can't force flag those options in the .magisk file so I have to find another way.
Werve said:
Since the userdata partition is now usually encrypted either with FBE or FDE but once the system loads the files are readable and moveable even externally then it is clear that somehow the data has been decrypted precisely using the relevant decryption key, AES encryption usually.
So if the user has not specified any key this must be derived from the information already in the partitions from the factory.
Then by restoring the right combination of partitions the system can boot correctly by decrypting the userdata partition. Hence the tests and the report I wrote in my last post.
At the moment I was able to remove the forced encryption of the userdata partition by modifying syper (specifically fstab present in the /vendor sub partition) but I would like to achieve the same systemless modification using Magisk (to be OTA compatible). Unfortunately, the options to remove dm-verity and forceencrypt have been hidden in the latest versions of Magisk to avoid problems with inexperienced uses.
Since I don't have a custom recovery on the Umidigi Bison Pro I can't force flag those options in the .magisk file so I have to find another way
Click to expand...
Click to collapse
Well, what I said is a different thing, the other user had a different interest than this. They did want to access to some data from a backup in a non-booting device, I referred to that, the userdata image backed up doesn't have an encryption by default, unless the user set one through a custom recovery, suppose that someone did take a backup from the userdata partition, this userdata image can be opened/readable for anyone with minimum skills and the appropriate tool.
In regard to your issue, I don't think, the userdata partition has any kind of restrictions to take OTA updates, most likely this resides in the bootloader, kernel or even a "silent/hidden" partition with no more functions than that.
As a side note, you should check some custom recoveries, specially in Xiaomi devices that easily allow taking OTA updates, for example I always can take OTA, when I use Orange Fox recovery, although I'm not interested, so I make updates manually, to be sure that all run fine.
SubwayChamp said:
Well, what I said is a different thing, the other user had a different interest than this. They did want to access to some data from a backup in a non-booting device, I referred to that, the userdata image backed up doesn't have an encryption by default, unless the user set one through a custom recovery, suppose that someone did take a backup from the userdata partition, this userdata image can be opened/readable for anyone with minimum skills and the appropriate tool.
In regard to your issue, I don't think, the userdata partition has any kind of restrictions to take OTA updates, most likely this resides in the bootloader, kernel or even a "silent/hidden" partition with no more functions than that.
As a side note, you should check some custom recoveries, specially in Xiaomi devices that easily allow taking OTA updates, for example I always can take OTA, when I use Orange Fox recovery, although I'm not interested, so I make updates manually, to be sure that all run fine.
Click to expand...
Click to collapse
The methodology I was referring to that is not OTA supported is to modify the super partition (the dynamic partition that from Android 8? contains system, vendor, product--for Project Treble) to disable the forced encryption of the userdata partition. In my case FBE (File Based Encryption) Android 11 encryption.
Even having disabled the dm-verity if you apply an OTA update the super partition is replaced with the one that does not have the modification to remove the forced encryption and from the tests I have done this refuses to read unencrypted partitions and asks to do a factory reset.
So, the userdata partition makes the OTA update problematic (it doesn't block it, but you lose your personal data).
I am sure that instead of modifying the super partition to disable encryption you can achieve the same result via Magisk and a modified boot partition.
Unfortunately despite many trials due to my inexperience with Magisk I could not do it.
I wanted to do all this to avoid problems as described in the case of this thread that is, have the userdata partition intact but not the rest to be able to describe it. But seems I must let the encryption and do a backup after every OTA update.
Werve said:
The methodology I was referring to that is not OTA supported is to modify the super partition (the dynamic partition that from Android 8? contains system, vendor, product--for Project Treble) to disable the forced encryption of the userdata partition. In my case FBE (File Based Encryption) Android 11 encryption.
Even having disabled the dm-verity if you apply an OTA update the super partition is replaced with the one that does not have the modification to remove the forced encryption and from the tests I have done this refuses to read unencrypted partitions and asks to do a factory reset.
So, the userdata partition makes the OTA update problematic (it doesn't block it, but you lose your personal data).
I am sure that instead of modifying the super partition to disable encryption you can achieve the same result via Magisk and a modified boot partition.
Unfortunately despite many trials due to my inexperience with Magisk I could not do it.
I wanted to do all this to avoid problems as described in the case of this thread that is, have the userdata partition intact but not the rest to be able to describe it. But seems I must let the encryption and do a backup after every OTA update.
Click to expand...
Click to collapse
If you want to apply an OEM vendor stock update then it is a restriction from the OEM itself, and if you want to apply a GSI based update, it's a different approach, not sure if the restriction is FBE related or if the userdata is encrypted or not but probably related to AVB.
There are some tools/scripts you should search for, that can unpack and repack super partition, maybe you find something in the ODM or product image, this is assuming that the super partition it is the culprit.
Just know that it's a nonsense that an order (script) to restore a specific partition, be placed just there, but in other partition.
You should check what the OTA update contains, try to catch the OTA update through some ADB script, then unpack it, and see inside.
Also, you can try backing up every partition, and restoring them one by one, seeing if it boots.
SubwayChamp said:
If you want to apply an OEM vendor stock update then it is a restriction from the OEM itself, and if you want to apply a GSI based update, it's a different approach, not sure if the restriction is FBE related or if the userdata is encrypted or not but probably related to AVB.
There are some tools/scripts you should search for, that can unpack and repack super partition, maybe you find something in the ODM or product image, this is assuming that the super partition it is the culprit.
Just know that it's a nonsense that an order (script) to restore a specific partition, be placed just there, but in other partition.
You should check what the OTA update contains, try to catch the OTA update through some ADB script, then unpack it, and see inside.
Also, you can try backing up every partition, and restoring them one by one, seeing if it boots.
Click to expand...
Click to collapse
I have already done these tests, not with an OTA update but with a different version of the firmware for all partitions, and set out the conclusions.
Obviously it's an OEM restriction since it left the forced FBE encryption on and the way it was created (so I guess also from AOSP) it refuses to read the userdata partition if it doesn't find it encrypted.
I have tried to install Lineage but so far I have not had any luck. I was able to get the install to work after modifying a file to remove the "assert" lines, but once the install is completed, I am stuck in a boot loop and am no longer able to use TWRP from the phone as the touchscreen isn't working after booting into recovery.
It seems that it is possible to restore a backup in TWRP from the command line, but the twrp command line guide isn't real helpful for me.
restore PARTITION-TO-RESTORE BACKUP-NAME :
restore PARTITION-TO-RESTORE from existant BACKUP-NAME:
PARTITION-TO-RESTORE, multiples selection allowed:
S: system,
D: data,
C: cache,
R: recovery,
B: boot,
A: and-sec,
E: sd-ext,
M: skip md5 check.
BACKUP-NAME: could be full path or name of twrp backup in current twrp backup folder.
Backup can't be encrypted.
So I am guessing that I have to mount the microsd first and then provide the path to the backup in the format that TWRP requires to restore that backup. Anyone have any input on how this could be done?
Since the touch doesn't work in TWRP, it's likely that the ROM.zip wasn't good for this model either.
Read old archives that better explain the use of commands in twrp.
OpenRecoveryScript - RootzWiki
web.archive.org
OpenRecoveryScript | TeamWin
OpenRecoveryScript
web.archive.org
ze7zez said:
Since the touch doesn't work in TWRP, it's likely that the ROM.zip wasn't good for this model either.
Read old archives that better explain the use of commands in twrp.
OpenRecoveryScript - RootzWiki
web.archive.org
OpenRecoveryScript | TeamWin
OpenRecoveryScript
web.archive.org
Click to expand...
Click to collapse
Thanks for the reply. I just found another way to get TWRP working by attempting to use Huanwei's multi download software and even though it ultimately failed, I was still able to use TWRP from the phone again. Unfortunately even though the phone is supposed to be a supported model, I still can't use anything bu the stock firmware as trying to install other ROM's just results in Error 7. I just recently purchased it, but I think there is something different about it so I can't install any other ROMS.
It would be a lot less guessing if there was an example like in the older documentation...
restore /sdcard/TWRP/BACKUPS/*serialno*/myrom (Would restore all partitions found in "myrom")
restore /sdcard/TWRP/BACKUPS/*serialno*/myrom DM (Would restore only the data partition in "myrom" and skip MD5 checking)
so I could have done resotre /sdcard/TWRP/BACKUPS/*/*
with no extra switches.