A long time ago while installing TWRP & rooting the tablet using the DirtyCOW exploit, I lost my entire user partition due to it being encrypted.
Currently am trying to switch my root from SuperSU to Magisk Systemless.
I don't want the possibility to lose my user data or /system modifications if anything goes wrong. I made a lot of edits here and there to tweak the tablet to my liking, and would hate to have everything lost.
To preserve its current state, I want be create a full TWRP backup.
I would like to make sure, that the TWRP backup is a full NAND backup that will let me revert to the current state no matter what I do to the tablet. Eg, it acts like a true system snapshot.
- My /data partition is showing up as 0bytes (Unknown Errror 150)
- I read that a /system backup is worthless due to dm-verity. For a working backup you need a block exact system-image backup.
Anyone have a guide on how to create a non worthless backup>? I am on TWRP 3.0.2-0 (ancient I know)
Related
I decided to write this guide to describe the method I’ve found to be most useful to restore my data after I've updated my ROM. I’ve discovered this method through trial-and-error for myself, and I thought it could be useful for others who are looking for an easy way to transfer all your apps, settings etc. after doing a clean ROM update.
Prerequisites to use this method
You must have root access
You must have a custom recovery (like CWM or TWRP)
You must have X-posed framework installed (http://forum.xda-developers.com/showthread.php?t=1574401)
You must have this X-posed mod that disables Play Store encryption (http://www.modaco.com/topic/361776-xmod-disable-play-store-encryption-mntasec/)
Principles behind this method
The idea behind this method is to restore your entire data partition in an easy way when you’ve done a clean ROM update, so that you get the same benefit as doing a dirty flash, except you’re doing it clean instead of dirty (some custom ROMs do not support dirty flashing).
To do this, you must ensure that all your data is actually on the data partition - this is where the X-posed mod comes into play. Normally, paid apps are put in a special encrypted folder (/mnt/asec), and thus not part of the normal /data partition, and therefore also not backed up with a backup of the data partition, but instead backup up as part of the system partition (at least this is what I think, somebody correct me if I’m wrong).
With the X-posed mod mentioned above, all apps go into /data/app instead, and are thus being backed up with the data partition. This is really handy, because you then have all your user apps in one place, instead of several different folders. This again enables you to restore only your data partition, and by that restoring all your apps, settings etc.
This also means that if you previously haven’t used this X-posed mod, you have to uninstall and reinstall all your paid apps with the X-posed mod active, so that they’re installed to /data/app instead of /mnt/asec (I’ve had to do this a couple of times).
I’ve only tried this method when updating a ROM to a later version of the same ROM (like a new build of CM), but it might also work when switching between ROMs, although this is totally untested for my part (I imagine there could be some invalid system settings getting restored, but it would be worth a try if anybody is up for it).
Step-by-step guide
Disable any set-on-boot settings that could be problematic with a new ROM/Kernel (for instance Kernel settings, themes etc).
Backup your apps and data with your backup app (Titanium or the like). This is not strictly necessary as part of the procedure, but acts as a second backup, just in case.
Boot into your custom recovery and make a Nandroid backup of your entire phone. You should do this before every ROM update anyway, and it is especially important here, as you’re later going to restore the data partition from this backup.
Now while in custom recovery, clean flash your new ROM update (factory reset/cache wipe first) - flash any kernel or Gapps as well.
Boot into Android and setup Gapps (if Gapps asks to restore your phone, you can answer no - why use the time and bandwidth to download apps again, when you can use this method?). You now have a freshly installed new system with basic Gapps settings, done as you would with any other ROM update. And now comes the good part...
Boot back into your custom recovery, and clear caches again. From backup/restore, do an advanced restore to restore only your data partition.
Boot back into Android. You will now get the screen “Android is upgrading”, while it initializes all you previous apps.
When this is done, you should basically have your old system back with the all your previous apps and system settings, including launcher, themes etc. (X-posed mods may require an extra reboot though). Some specific system settings might have been lost in translation, but I believe it’s one of the easiest way to get all your user data back, and it’s been working like a charm for me.
You could also dirty flash if your ROM supports it, but sometimes there is an update that requires a clean flash, and then this method will give you basically the same end result.
I hope somebody besides me finds this useful, as it’s been my preferred way to restore my data between updates. In this way, I only keep my app backups as a secondary backup to the data partition. Any feedback on this method is very welcome - all I can say is that it’s been working flawlessly for me so far.
:: AM ::
Reserve Q&A
ameinild said:
I decided to write this guide to describe the method I’ve found to be most useful to restore my data after I've updated my ROM. I’ve discovered this method through trial-and-error for myself, and I thought it could be useful for others who are looking for an easy way to transfer all your apps, settings etc. after doing a clean ROM update.
Prerequisites to use this method
You must have root access
You must have a custom recovery (like CWM or TWRP)
You must have X-posed framework installed (http://forum.xda-developers.com/showthread.php?t=1574401)
You must have this X-posed mod that disables Play Store encryption (http://www.modaco.com/topic/361776-xmod-disable-play-store-encryption-mntasec/)
Principles behind this method
The idea behind this method is to restore your entire data partition in an easy way when you’ve done a clean ROM update, so that you get the same benefit as doing a dirty flash, except you’re doing it clean instead of dirty (some custom ROMs do not support dirty flashing).
To do this, you must ensure that all your data is actually on the data partition - this is where the X-posed mod comes into play. Normally, paid apps are put in a special encrypted folder (/mnt/asec), and thus not part of the normal /data partition, and therefore also not backed up with a backup of the data partition, but instead backup up as part of the system partition (at least this is what I think, somebody correct me if I’m wrong).
With the X-posed mod mentioned above, all apps go into /data/app instead, and are thus being backed up with the data partition. This is really handy, because you then have all your user apps in one place, instead of several different folders. This again enables you to restore only your data partition, and by that restoring all your apps, settings etc.
This also means that if you previously haven’t used this X-posed mod, you have to uninstall and reinstall all your paid apps with the X-posed mod active, so that they’re installed to /data/app instead of /mnt/asec (I’ve had to do this a couple of times).
I’ve only tried this method when updating a ROM to a later version of the same ROM (like a new build of CM), but it might also work when switching between ROMs, although this is totally untested for my part (I imagine there could be some invalid system settings getting restored, but it would be worth a try if anybody is up for it).
Step-by-step guide
Disable any set-on-boot settings that could be problematic with a new ROM/Kernel (for instance Kernel settings, themes etc).
Backup your apps and data with your backup app (Titanium or the like). This is not strictly necessary as part of the procedure, but acts as a second backup, just in case.
Boot into your custom recovery and make a Nandroid backup of your entire phone. You should do this before every ROM update anyway, and it is especially important here, as you’re later going to restore the data partition from this backup.
Now while in custom recovery, clean flash your new ROM update (factory reset/cache wipe first) - flash any kernel or Gapps as well.
Boot into Android and setup Gapps (if Gapps asks to restore your phone, you can answer no - why use the time and bandwidth to download apps again, when you can use this method?). You now have a freshly installed new system with basic Gapps settings, done as you would with any other ROM update. And now comes the good part...
Boot back into your custom recovery, and clear caches again. From backup/restore, do an advanced restore to restore only your data partition.
Boot back into Android. You will now get the screen “Android is upgrading”, while it initializes all you previous apps.
When this is done, you should basically have your old system back with the all your previous apps and system settings, including launcher, themes etc. (X-posed mods may require an extra reboot though). Some specific system settings might have been lost in translation, but I believe it’s one of the easiest way to get all your user data back, and it’s been working like a charm for me.
You could also dirty flash if your ROM supports it, but sometimes there is an update that requires a clean flash, and then this method will give you basically the same end result.
I hope somebody besides me finds this useful, as it’s been my preferred way to restore my data between updates. In this way, I only keep my app backups as a secondary backup to the data partition. Any feedback on this method is very welcome - all I can say is that it’s been working flawlessly for me so far.
:: AM ::
Click to expand...
Click to collapse
awesome it worked for me without xposed modules cause i didnt had any paid apps. My data partition was encrypted so every time i was going to recovery i had to put password to decrypt it. But i created backup of data partition after decrypting it. Then copied all thhe internal file to the pc after that clean flashed rom with DFE , magisk , gapps. After booting into rom setting it to like new then go to my orange fox recovery clear dalvik and caches restore data partition. wollah got my encrypted data partition to decrypted partition without getting die from reinstalling apps and data.
Hi! So, I have a newer Android 10 device with no TWRP support. I think my question is not model specific, so I'm posting generally, but if it matters, it's a OnePlus 7T Pro, bootloader unlocked of course.
Anyway, since there's no good custom recoveries for Android Q (none that I know of that can decrypt /data), I was trying to be safe and using a hacked (adb enabled) stock recovery to make backups from the command line. I have a nice tarball created in recovery mode with "tar cvf - /data/". It looks like it was made properly, I can unpack the files on back onto the device in recovery mode, and it's not corrupt or anything.
The problem is that naively trying to restore it back to my /data partition (e.g. tar xvf) after a wipe causes two problems:
1) after rebooting to recovery, the /data partition just looks encrypted again. If it was working properly, stock recovery can decrypt /data
2) normal system boot just bootloops.
If anyone has tried manually saving and restoring /data from a tarball, I'd appreciate any tips on getting it to work.
My best guess is that the ext4 encryption keys are the culprit - since after a factory reset the device creates new encryption keys, I think I need to figure out how to restore 99% of my backup without overwriting the fresh encryption keys that were created after wiping. I think I need to combine the new encryption keys with the other files in my tarball backup. If I can get past that, I might have to worry about things like SELinux xattrs as well, but one thing at a time.
If any experienced Android hackers know how filesystem encryption works in Android 10, I would appreciate a map of which files should not be overwritten. For example, I'm thinking I should probably avoid restoring some of these files:
Code:
/data/unencrypted/key
/data/unencrypted/key/encrypted_key
/data/unencrypted/key/keymaster_key_blob
/data/misc/vold/user_keys
/data/misc/vold/user_keys/de/0/keymaster_key_blob
/data/misc/vold/user_keys/de/0/encrypted_key
/data/misc/vold/user_keys/ce/0/current/keymaster_key_blob
/data/misc/vold/user_keys/ce/0/current/encrypted_key
/data/misc/systemkeys
/data/misc/keystore
/data/misc/keychain
/data/system/recoverablekeystore.db
[edit] spoke too soon
I ended up doing something janky and wrote a little shell script to repack the files from /data/data and /data/app into files that Titanium Backup could understand. I also figured out how to restore my SMS messages from /data/user_de. That was enough to recover the things I cared most about.
Still begs the question of what's the best method of doing backup and restores of /data on Android 10 when you don't have custom recovery but you do have Magisk, root, and stock recovery with adb. I want to keep hacking and playing with this phone so gotta figure something out.
I like Titanium Backup, have used it for many years, but the maintainance hasn't really kept up with Android development, theres a lot of gotchas like it hangs trying to restore multi APK packages. I still want to figure out something more seamless like a nandroid backup solution even if its from the command line. At least until TWRP eventually supports Android 10.
try backing up data partition in a external storage live usb or sd card
DrWowe said:
I ended up doing something janky and wrote a little shell script to repack the files from /data/data and /data/app into files that Titanium Backup could understand. I also figured out how to restore my SMS messages from /data/user_de. That was enough to recover the things I cared most about.
Still begs the question of what's the best method of doing backup and restores of /data on Android 10 when you don't have custom recovery but you do have Magisk, root, and stock recovery with adb. I want to keep hacking and playing with this phone so gotta figure something out.
I like Titanium Backup, have used it for many years, but the maintainance hasn't really kept up with Android development, theres a lot of gotchas like it hangs trying to restore multi APK packages. I still want to figure out something more seamless like a nandroid backup solution even if its from the command line. At least until TWRP eventually supports Android 10.
Click to expand...
Click to collapse
try backing up data partition in a external storage live usb or sd card, first you for encryption, then restore from external source after formatting phone storage
I am looking for a full backup solution for my rooted Pixel 5a.
Unfortunately, TWRP is not (yet) available, so I can't do nandroid backups.
Also, solutions such as Swift Backup don't backup everything (e.g., device settings, downloaded data).
Can 'rsync' be used to get a full and restorable backup?
If I am not concerned with the static (and non-user-writable system elements), would it be enough to backup:
/sdcard
/data
/cache
/persist
The notion would be that if I wanted to restore the phone at any point, I would just use 'rsync' to restore the previously transferred data.
Any limitations/challenges to this backup method?
If I wanted to extend this to also back up the static system itself, what would I need to backup and what would I need to do to restore?
Thanks
Hi. I hope this is the proper place to post this. I've been away from playing with phones for almost 5 years (Couldn't even remember my password here!) due to being very busy with my business. So I feel like a total noob again!
I have successfully unlocked the bootloader and rooted my Oneplus Nord N10 5g using TWRP and Magisk. Now, before I go any further I want to take a Nandroid backup. Do they still even call it that?
In the unlocking and rooting process there were dire warnings about not formatting certain partitions. I would have to refer back to my notes to see which ones but that is irrelevant to my question right now.
I boot to TWRP, go to backup and I have the following options: Metadata, Boot, Recovery, Data, Modem, EFS, and Super. Is it safe to backup and restore all of these? I have no intentions at this time of installing a custom kernel, eventually some different rom's maybe. I'm just looking for a complete backup where I can restore the entire phone in the event of a total fail.
Thanks, and it's great to be back playing with phones again!
So it seems a lot changed with Android 11. The partition Super is now a partition that contains other partitions if I read correctly. If I include all partitions in the backup, would it be safe to restore them all? Or is the Super partition the container for all the other ones? And if so, is that what I need to backup for a full nandroid backup? Really need to figure this out before I flash any other rom's to it.
TL;DR: somewhere in /data, it is written that Android should encrypt some partitions including /data on next reboot, but my ROM doesn't support decrypting. I managed to make a backup of /data with twrp, but when I remove encryption, restore the backup, and reboot, the ROM tries to encrypt (which it only does if I restore my data backup so somewhere on /data it sees that it was rebooting to encrypt), so what should I delete from /data after restoring it to cancel the planned encryption?
(Clarification: I don't think this is encryption enforced, because I can format and boot to Android without encryption happening, it's only if I restore my backed up data that encryption happens on boot.)
Some background:
So I made the mistake of starting the Android full disk encryption on a custom ROM (non official AOKP 7.1.2, J120F/DS, SC8830) and since I installed the ROM a year ago I forgot that it doesn't support encryption and clicked encrypt.
After the encryption, it got stuck on a black screen, adb working but /data, /dev and something else I forgot what were showing as "tmpfs", and to make matters worse, twrp for j120f was too old and doesn't support encryption, so what I did was use the tool from Hovatek and the working twrp 3.0.2 to make a working twrp 3.2.1 for my phone (sc8830), and it worked, and twrp decrypted it and I made a backup with adb backup --twrp data. I removed encryption by formatting and restored it and checked that it's no longer encrypted by rebooting to twrp.
Now the issue is, rebooting to the system again starts the encryption (coz the ROM encrypts but can't decrypt) so somewhere in /data it's saved that it has rebooted to encrypt. I tried deleting these:
/data/system/gatekeeper.password.key
/data/system/gatekeeper.patern.key
/data/system/locksettings.db
/data/system/locksettings.db-shm
/data/system/locksettings.db-wal
but still the encryption started after reboot. So it must be saved in some other file that is included in my backup. Can you please tell me where so I can delete it and restore my backup? Thank you very much!