Information on backing up AND RESTORING with TWRP - Xiaomi Mi 9 Guides, News, & Discussion

My phone ran into problems after I upgraded Magisk (I was on Canary build). No problem I figured, I will just restore from my TWRP backup.
... only issue was, I have never restored from a TWRP backup. And it turned out to be not as straightforward as I thought. I ended up learning a lot about the process to be able to successfully backup AND RESTORE. So I thought I might write something here to make it easier for you to do this, so you don't have to learn it the hard way like I did.
First I think most of us know data storage is not backed up by default. The partitions selected by TWRP for you (i.e. system, data, vendor, and boot) are sufficient to save an image of your OS plus the installed apps, but not app data (specifically /data/media and everything underneath, so actually this is dependent on how each app stores its data, some or all of an app's data may be stored in /data/data and not /data/media. /data/data IS backed up by TWRP.), and the excluded portion contains your own personal data as well, i.e. the internal storage.
Simply put, if you want a so-called 'full backup', or a complete snapshot of your phone, you will want to do something more than simply a TWRP backup. See below.
Another thing to note is that the location you are backing up to (and I recommend external storage like a USB-OTG device) may *not* have enough space. And while backing up you may not even realize that you have run out of space (as there is no warning particularly with the tar command which I will detail below) therefore your backup may become corrupt ! Take extra precaution to validate the integrity of your backup, and make sure there is always enough space to create the backup. TWRP *does* warn you about lack of space though.
Last but not least, if you have a password (either pattern or numeric) to lock your device, after restoring it is gone. So first thing after restore is to go into settings to setup the password once again. Interestingly the fingerprint data is still there. Once you set up a password the fingerprints will reattach to it. For the apps that have fingerprint login registered, you will need to re-register as the app thinks you have updated your password/fingerprint.
If you don't set up your password after restoring, in TWRP it will just go straight into the main menu as if no password has been set, so basically anyone who knows how TWRP works will be able to access your data if they get a hold of your phone.
Oh one more thing. The format of the backup media (e.g. usb-otg) has to be ex-FAT. Very important. I got badly burned because I had FAT32. And like I said the backup process did NOT warn me at all that this could be a potential issue. Not the tar command anyway.
OK here is the HOW on BACKing UP:
Backup using TWRP should be easy enough. It is documented just above everywhere (restoring is another story however). Just make sure you mount your USB-OTG before starting your backup, and pick USB-OTG as your destination. Let TWRP choose the partitions to be backed up.
This will create a TWRP folder with subfolders, one of which containing the actual backup files.
On to your personal data - in TWRP (while your USB is still mounted), go to Advanced, Terminal, then issue this command: "tar -cpvzf /usb_otg/data_media_backup.tar.gz /data/media"
This will create a .tar.gz file. This is your own data (and like I said some apps write to this space too so it is also app backup).
Validate the integrity of the .tar.gz file with this command: "tar -tf /usb_otg/data_media_backup.tar.gz". If you see no error message then you are good.
Unmount your usb finger, and take it out before leaving TWRP. Otherwise the usb finger may need repairing when you access it in say Windows. And that's always a scary thing.
If you need to archive the backup somewhere in your PC etc, copy 2 things from your USB: The TWRP folder, or at least the subfolder that contains your TWRP backup files (copy the entire folder, or better yet the entire folder structure); and the .tar.gz file. This forms your complete backup image.
Here is the HOW on RESTORING:
Flash MIUI (or whatever your OS is), with matching version as the one in your backup image. i.e. you have a vanilla enviroment that is the same version as the one in your backup.
If for some reason you can't boot into your system afterwards, go into recovery (can be built in recovery, not necessarily TWRP), then wipe your DATA partition clean. Now you should be able to boot.
Boot up the vanilla system. When the first spalsh screen is shown ("welcome to the OS, let's start setting things up", or something like that) just long press the power button to shutdown or restart your phone. Then go into bootloader (lower volume + power buttons for my phone).
Install TWRP, reboot to recovery / TWRP, then do your TWRP restore. Don't forget to mount your otg-usb first ! if the folder structure is there in your usb, or even if you just have that one folder with the backup files underneath, the restore function will show that folder for you to pick. Just pick it then proceed with the restore.
in TWRP (while your USB is still mounted), go to Advanced, Terminal, then issue this command: "rm -r /data/media/0/*" (this is why step 3 was necessary)
if there are other folders underneath /data/media, repeat the rm command once for each of these folders
"cd /"
"tar -xvzf /usb_otg/data_media_backup.tar.gz" <-- this will restore your data !!
Reboot. Voala you have your backed-up system in action once again. Nothing in your phone is different than before the backup, except for the pattern password which is now gone. And you better set it up again right away.
Hope this helps !!

Related

[Q]can't execute non-system apps after restoring from recovery backup

My phone is LU3700, which is one of three korean versions of O1, and I don't think it differs a lot from P500 in this situation.
The recovery is ClockworkMod recovery v3.0.2.4
I backed up three times, and deleted former two backups because I was lack of space in SDcard.
All the downloaded apps and updated system apps such as market or helper won't be executed.
When I try to execute a problematic app, it says "Application is not installed on your phone"
In the settings->applications->manage applications, the list of apps is still there, and for each app Data/Cache are also there but Application isn't(0.00B)
I guess it could be just a simple problem like permission or configuration?
Even if all the applications are really gone, I can re-install them so it's not so big deal. I just want to save the time to install them.
Is there anything I could do in this situation?
P.S. Maybe there was too small space in the SD card when the third backup was made...I think I saw a message saying the backup was completed but I'm not so sure.
Oh, it IS a big deal since I can't use even market or helper(i frequently use it)...if not it's not possible to restore, could anyone teach me how to backup those data?
this happened to me too. i think that the part of the app that is located on the sd card got deleted (if i remember correctly, these "parts" should be located on /sdcard/.android_secure, which, for some reason, does not show its contents on the phone but only on the computer, at least for me). you can always backup your apps using titanium backup and i recommend keeping only one backup at a time so you dont get confused between each one and also for the sake of space. or, instead of using titanium backup for your apps, you can make a nand+android secure backup in recovery, which also does a backup of your app data (i think)
you can confirm all of this by plugging in your phone to the computer (usb mass storage) and check the /sdcard/.android_secure folder
edit: and as for trying to restore, if theres nothing in your android_secure folder, then you have to start from scratch (unless one of your nand backups was a nand+android_secure backup)
I'm afraid that's not the case. it is not only App2SD apps that aren't working right now. they used to work without SD before.
Flash the rom again. Start everything again.

Unable to restore Helium backup!

I used the Helium backup app to backup all my apps before flashing my S3 with the stock ROM. I copied my entire internal memory as it is, and pasted it after flashing. This of course included the crucial 'Carbon' folder with all my backups made my Helium.
BUT now I installed helium and it refuses to detect those backups. In 'Restore and Sync' it simply says "No backups were found on your internal storage".
I know this is probably best asked to the Helium developer but I reckon it's a very famous app so someone might be able to suggest a solution here. I've already tried rebooting.
Thanks!
cyanide911 said:
I used the Helium backup app to backup all my apps before flashing my S3 with the stock ROM. I copied my entire internal memory as it is, and pasted it after flashing. This of course included the crucial 'Carbon' folder with all my backups made my Helium.
BUT now I installed helium and it refuses to detect those backups. In 'Restore and Sync' it simply says "No backups were found on your internal storage".
I know this is probably best asked to the Helium developer but I reckon it's a very famous app so someone might be able to suggest a solution here. I've already tried rebooting.
Thanks!
Click to expand...
Click to collapse
Had the same problem, wanted to root my phone, but unlocking the bootloader wipes everything, so used Helium to backup all my apps. Backed up to both internal storage and dropbox, copied "carbon" folder from internal storage to computer, unlocked bootloader, copied back, and helium wouldn't see backups from "carbon" folder. Will, however, find backups from dropbox, but without premium version, cannot restore them.
I did find a workaround, but it is not the most pleasant way of doing things. If you open the menu options and tap PC Download, it will display a local ip address and port (i.e. http://192.168.1.xxx:5000 or 10.1.10.xxx:5000). This is your phones ip address on your network and may appear different depending on your router and/or network setup, and the 5000 is the port to access helium web server on phone. Navigate to that ip address through your web browser, including http:// and it will open a web interface for backing up and restoring. See attached screenshot. Click on the browse button under Restore and navigate to the carbon folder that you backed up. In each folder will be 3 files, xxx.png, xxx.ab, and .nomedia (xxx is the name of the file i.e. com.adobe.reader). Choose the file with .ab and click open. The web interface will ask if you want to restore, click yes. Repeat for all items to be restored. If the web interface says file invalid, make sure the server is still running on your phone. Should show a cloud in the notification bar and say Helium server is running. If the server stops, just restart it from menu->pc download in the helium app.
This is not the most elegant or efficient way of doing this, but if all your data is in those backups and you really want them, this is the only solution I have found so far. I hope this helps.
Edit: Was able to restore first two apps before posting, but after continuing, remaining apps would not restore. May still work, but even though I get restore complete, the apps don't show up. Will require further investigation.
kind of solution
i have the same problem i tried to unlock nexus 10 and made a backup and (manualy) copied the carbon folder to my PC and restored after unlocking
and as you know it wont detect the backups..
i think it has to do with a .json file that is either not created when the backup is done or not carried over the copy.
(i know about this .json file, as i had a previous backup from whatsapp which had this txt file which has info about the backup)
so when i tried to restore all my apps i only got that whatsapp detected (as an old version)
i mailed Koush( the developer) and he said that manual moving is not supported and the way to go is using the IP server to extract the backups to PC.. (ie doing the backups with the Web tool and not in the app itself)
one thing i can think of (cheating the app. installing the app, doing a backup of the app and replacing the .ab file for the one in the lost backup.. ) i will try that and post back
---------- Post added at 03:28 PM ---------- Previous post was at 03:13 PM ----------
ok i had an old PLUME (twitter client) backup with data,
what i did and seemed to work (for this test )
is to delete data on installed PLUME
backup plume without data. (a plume folder gets created on the carbon folder)
replaced the .ab from this backup with the old backup's .ab
now helium restored the old file with the previous backup data restored as well
i hope this works for other apps. i will try with my nimble quest game which i have a backup and is the only thing no recoverable i had in helium everything else is account based on the cloud
so the way to go here could (until koush changes this)
install the affected apps
backup them
navigate to the carbon folder and replace the .ab's with the old counterparts (which have data)
enjoy
I also had same problem, and i try use the solution below, but it can NOT work for me.
My device is ASUS Padfone 2, without root.
After try many method, I success finally!!
Here is my step, hope can help other people like me:
1. check the backup folder name. i.e jp.co.johospace.jorte, copy it to your PC.
2. use google search this key word "jp.co.johospace.jorte", you can find out what is this app, download and install it from play store.
3. After install finished, open your Helium and backup this app, i.e. "jorte"
4. After backup finished, check Helium restore, you can see "jorte" was backuped.
5. close Heilum from setting/apps, then open file explorer, find a folder named "jp.co.johospace.jorte" in the ../sdcard/carbon/
6. open the folder, you can see 3 files: 1.._jp.co.johospace.jorte.png 2..nomedia 3..jp.co.johospace.jorte.ab
copy jp.co.johospace.jorte.ab from your PC to android /sdcard/carbon/jp.co.johospace.jorte and replace it
7. open Helium again, and restore jorte, you can see data was restored!
it's probably possible to generate a set of manifests for it. the json ones don't seem to have any special data (as in no binary or signature type things)
e.g. for Lux the app
{"enabled":true,"system":false,"flags":545832516,"packageName":"com.vito.lux","versionCode":158,"label":"Lux","versionName":"1.58","locked":false,"date":1375397585745,"apk":true,"backup":true}
cut down to
{"enabled":true,"packageName":"com.vito.lux","label":"Lux","locked":false,"date":1375398819080,"apk":true,"backup":true}
still restores ok
I'm not sure how much metadata is necessary for it to work but that still restores (and the date is moot anyway).
It's possible if you mess up the enabled or system attributes or if the flags attributes are useful hints to the restore command to set package properties - it may make you have to wipe your device (or replace a newer version of the app or install data for an older version, etc). but for your everyday apps for a recent backup - you could probably make a quick powershell or whatever script to iterate over a set of subfolders and generate some dummy manifests.
Code:
$subfolders = Get-Item -Path 'E:\carbon\*'
foreach ($f in $subfolders){
if($f.Attributes.ToString() -eq "Directory")
{
$filecont="{`"enabled`":true,`"packageName`":`""+$f.name+"`",`"label`":`"" + $f.Name + "`",`"locked`":false,`"date`":1375398819080,`"apk`":true,`"backup`":true}"
$fname= $f.FullName + "\" + $f.Name + ".json"
[System.IO.File]::WriteAllText($fname, $filecont)
}
}
(I'd probably suggest using the manifests from backup restores for google system apps and contacts/sms/launcher, etc instead of these though)
anyway, wish the dev added clear non skippable instructions to the app (first run), and the desktop dl page
Maybe they're there and im stupid.
gierso said:
kind of solution
so the way to go here could (until koush changes this)
install the affected apps
backup them
navigate to the carbon folder and replace the .ab's with the old counterparts (which have data)
enjoy
Click to expand...
Click to collapse
thanks for this workaround. this feels like a huge gap in the functionality of a backup app. if there is a forced need to use the cloud, the user should at least be made aware of it.
not sure how much I can legitimately complain about a free app, but definitely not interested in the premium version based on this.
It's simple !
I found out a way. What you do is, 1st backup all your apps using Carbon to your Internal or External storage. Copy all the files in the carbon folder where you have backed up your data. Once you finish wiping your data, install all the apps again and backup using carbon. Now,replace these backup files with the actual backups ! Since you have backed up after re-installing your ROM , carbon won't say it couldn't find any backups.
Cheers
- Sent from Galaxy S3
chinnu189 said:
I found out a way. What you do is, 1st backup all your apps using Carbon to your Internal or External storage. Copy all the files in the carbon folder where you have backed up your data. Once you finish wiping your data, install all the apps again and backup using carbon. Now,replace these backup files with the actual backups ! Since you have backed up after re-installing your ROM , carbon won't say it couldn't find any backups.
Cheers
- Sent from Galaxy S3
Click to expand...
Click to collapse
it's not simple. it's annoying and longwinded. and defeats about half the point of automated backup apps.
the Carbon/Helium app is horrible.
While this went ok I had to reset my device today for another reason.
So before doing that, I backed up using Carbon, and transferred the data via the the web interface as the dev suggested
but was unable to restore the packages as uploading the zip back fails.
(the server on the n4 just stops after the upload. Moreover - there's no logs of any kind for the user as to why it fails. and no upload progress.).
Dunno if the dev will reply what the problem with the android web server component is but for now since i needed the backup urgently
I ended up having to extract the zip and convert the backup into individual manifests/copy the zip contents into app packages folders
Here's the powershell code to do this if anyone else has a similar problem (use at your own risk, I only tested it once on my device)
Code:
$backuppath='E:\_MediaServerE\backup\' #extract your backup.zip contents here (those are the .ab, png files in the zip you downloaded + a backup.json maniest the app generated for all the packages)
$jsonFile=$backuppath+"backup.json"
$jsonText=[System.IO.File]::ReadAllText($jsonFile)
$files = Get-Item -Path ($backuppath + "*")
[System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
$ser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$jsonPackageDefs = $ser.DeserializeObject($jsonText)
function getPackageDef([string]$pname, [Object]$packagesIn){
$ret = ""
foreach($p in $packagesIn.packages){
if($p.packageName -eq $pname){
$ret = $ser.Serialize($p)
break;
}
}
return $ret
}
#for each file copy it to a subfolder with the package name, and find the json manifest in the manifest collection
#and write it to the app folder
foreach ($f in $files){
if($f.Attributes.ToString() -eq "Directory"){
#skip
}
elseif($f.basename[0] -eq "." -or ($f.basename+$f.Extension) -eq "backup.json"){
#skip
}
else{
$bn=$f.BaseName
$bn
if([System.IO.Directory]::Exists(($backuppath+$f.BaseName))-eq $false){
[System.IO.Directory]::CreateDirectory(($backuppath+$f.BaseName));
}
if([System.IO.File]::Exists($backuppath+$f.BaseName+"\"+$f.BaseName+$f.Extension)){
[System.IO.File]::Delete($backuppath+$f.BaseName+"\"+$f.BaseName+$f.Extension)
}
if([System.IO.File]::Exists($backuppath+$f.BaseName+"\"+$f.BaseName +".json")){
[System.IO.File]::Delete($backuppath+$f.BaseName+"\"+$f.BaseName +".json")
}
$f.CopyTo(($backuppath+$f.BaseName+"\"+ $f.BaseName + $f.Extension))
$packageDetailsText=getPackageDef $f.BaseName $jsonPackageDefs;
$jsonFileNameOut=$backuppath+$f.BaseName + "\" + $f.BaseName + ".json"
[System.IO.File]::WriteAllText($jsonFileNameOut, $packageDetailsText)
}
}
I then copied the results to internal storage\carbon and was able to restore.
I hate the carbon app it's probably the worst experience I had backing up my device. It's cost me hours of worrying and coding to restore my data and i paid for that crap... There is hardly any documentation to follow and the dev doesn't even warn to not copy files on the app page (that's for the previous issue just above, this one is more about the 'proper' way breaking) Moreover it failed to backup the com.ea.games.nfs13_row game correctly due to its size so I lost the save data for it.
This is not an experience people should have with a backup app.
He should just provide a secondary way to transfer things using the files in the carbon folder (and/or a zip of the backup) instead of the web server bit that breaks, and takes forever. (and yes, I realize the web server generates the zip from the carbon folder and adds the manifest for the download, but if you can't upload it back for restoring properly - what's the point)
oh and keep in mind - you're storing unencrypted tar equivalents of your phone app data - so take care about where you keep the zip or the ab files. (the ab files are tars with a slight header change) (not that Carbon asks you to provide any means of protecting them, so more of an unrelated aside)
No idea, what I posted worked for me !
SMS not returning?
chinnu189 said:
No idea, what I posted worked for me !
Click to expand...
Click to collapse
Everything else seemed to work (thanks for this solution, definitely helpful to at least confirm data). Anyways - the one thing I can't seem to figure out is how to restore text messages (com.android.providers.telephony.ab) - I've tried to do it as a whole app using the PC interface, tried copying the folder as a replacement via Android File Transfer to the 'carbon' folder, etc.
When I try to restore via helium server, it quickly shows me everything as restored but I don't see the messages. When I try to do it manually (e.g. force copy), I don't see an option under "Restore and Sync" to actually select messaging.
Am I missing something stupid / obvious? The raw data seems to be there (for example, call logs were uploaded just fine, text messages seem to have an issue).
--Dan
So I came up with a solution based on what works for others. Why this works when it didn't previously is beyond me, but hopefully someone else can use it if individual parts of their Helium backup don't work properly (in theory it should work for any app / SQL dataset). The whole point of Helium is that we're running non-rooted phones and so Titanium Backup or the like aren't applicable. For reference, I'm running an N4 to OSX 10.7.x
1. Use the Helium Server to create a full backup.
2. Wipe phone or whatever you need to do (I had to do a factory reset for Google since it's an N4 and my proximity sensor is dead).
3. Install the apps you had previously via Play Store (pain in the a**).
4. Run Helium server and upload the backup data.
5. Update apps as needed (should be auto but who knows for individual stuff).
6. Check app data in apps. Texts are obviously easy to see for me. Hopefully everything restored nicely for you here.
***Now for the bit for people who got corrupted restores.
7. Unzip the backup file on your computer, select the .ab files that correspond to the missing / corrupted restores and then zip those into a new file on your computer (for step 9). Keep these files handy in unzipped form as well for step 10.
8. Perform a new Helium backup of those sections onto the internal SD card (yes it will be blank, worry not).
9. (Optional step): Run Helium server and upload that new zip file (with the stuff that didn't make it) alone. Try restore. Unlikely to work but worth a shot.
10. Running Android File Transfer, find the SD card, find the folder 'carbon' and then find the corresponding sub-folders that have the modules that were corrupted. Replace the SD card backups of the broken modules with the good ones from your computer.
11. Run Helium restore again, but only for the sections you want (I swear, for me Messaging didn't show up the first few times but after step 10 it did).
12. Wait, hoepfully it works well. I would reboot after running the restore to a) kill Helium and b) confirm functionality.
FWIW, my telephony.ab was 1.8MB and took SIX hours to restore. Seriously. Yeah yeah, that's a lot of texts but six hours? I've since added "SMS Backup & Restore" to my phone and will probably just use that, map out my phone layout and functionality and just manually replace apps from now on since most data is in the cloud. You lose some stuff, but prima facie it's boat tons easier.
Hopefully that helps someone else who had the same issues.
Best,
--Dan
app is just crap
i lose the most important data.some apps restored,but not the precious i wanted.
thanks helium
Garbage app, just installed it, backed up then when i tried to restore i get the same errors all because i copied the carbon folder to my pc. Stupid
digitallysick said:
Garbage app, just installed it, backed up then when i tried to restore i get the same errors all because i copied the carbon folder to my pc. Stupid
Click to expand...
Click to collapse
Yeah.. I wanted to give a try.. but its real garbage.
thanks chinnu189
your solution worked great for me and only took 2 minutes. messages were restored too. now i can finish Final Fantasy :victory:
It worked for me - JUST BUY THE GOD DAMNED APP
Yes its $5.00 but you know what, I bet you've spent that on a burger, fries and drink in the past month.
I spent an hour trying to do it the free way, made a backup to internal, made a backup to pc using the server, yadda yadda yadda, then ended up on here looking for help because it wasn't working properly.
Finally went fck it here's my five bucks and I backed up to the cloud and restored from the cloud. It just works as advertised and perfectly at that.
Even in high school my time was worth $5 an hour. I just wasn't ready to acknowledge it then. Trust me, it's worth every cent. Buy Helium Premium.
</Advertisement>
No seriously I am a long time Titanium Pro user and Helium allowed me to move my nexus 4 data to my nexus 5 without having to go through the hassles of root. Well worth it for that alone.
Ok, hope this helps somebody. Just did the whole copy the folder instead of use the webserver business. I tried the replace methods above to no avail.
What did work is backing up ONE app to internal and replacing the *.ab file with the backup data into the 'carbon' folder, overwriting the one that has basically a blank app.
If I tried to restore multiple apps at the same time, it didn't work.
This is on a nexus 5, by the way. Now on to the rest of my apps...
the webserver method is a) slow b) keeps dropping out. the app is nothing but a headache.
as for the cloud backup one - depends on the service and whether you can specify the file to be encrypted prior to the transfer.
when doing backups using PC download mode (thru the URL port 5000), the device must be in standby, ie. Screen ON and UNLOCKED. its the only way the backups gets created good and can be restored.
Extracting Helium ab files
I backup up using Helium, and was unable to restore, so I had a small adventure recovering my files (eventually successful). These are some brief notes.
I had used the PC option to get the backup.zip file on my PC (I was going to reflash my Nexus 7 "grouper", 8GB). However, Helium refused to re-install the 2.5GB file back to the tablet. (Wait for upload via WiFi, then "failed".)
I could extract the particular 1.8GB ab file I wanted from the backup-zip that Helium made. However, "adb restore ..." would not work either.
After searching on "adb extractor" I found that the perl scripts (http://forum.xda-developers.com/showthread.php?t=1730309) were able to get me an extracted file in nominal "tar" format. I say nominal, because none of the tar programs I have would touch it---bad blocks etc. (I tried the Java adb extractor first, but did not pursue it further because of the tar issues.)
It turns out that the Helium "ab-tar" file was ordered like this:
2 bytes of something.
5 bytes of something.
Tar header block.
Tar data/header blocks...
5 bytes of something (starting at location 0x10007).
Tar data/header blocks...
5 bytes of something (at location 0x1800C).
Etc.
I.e., every 0x8000-byte chunk was prefixed by 5 bytes of stuff. Except for the second chunk. And the whole file was prefixed by 2 bytes of stuff.
Not only that, but the tar header checksums had been divided by 8 (i.e., last octal digit replaced with '\0').
After converting to a regular tar file, I was able to restore the files to their place. I now have a tar-reconstructing program that I will send to anyone who requests it.
Sigh. Backup should not be this hard.

[Solved] Qtions about backup / restore nandroid feature on twrp recovery mass deploy

Hi guys,
I've got 60 phones to setup, and that's just the biginning. I looked for some solutions to mass deploy them, the fastest way possible. I of course already know titanium backup and applications like that, but i would prefer not to root phone...
I was pretty confident about a nandroid backup : flashing the recovery of the phone is really quick, and i would just have to backup on card or on usb on the go on the master phone, and to restore the backup on all the phone remaining. But it didn't worked as expected, it happens that on each phone, twrp is generating backup in a folder named something like "420khg66g6cca200" and which is different on each phone, and when you put the card in another phone, this one doesn't see the main backup.
I of course tried to sneak arround and generate a backup on the second phone, which i replaced with the content of the first one, but the phone doesn't even boot anymore.
I used twrp because i dind't find another recovery avaiable for the phone, which is by the way a samsung Xcover 3 also known as SM-G388F.
I tried to backup only system, system image, and data and to restore it and the phones is stuck on boot screens.
I tried to backup again with boot partition, same thing happen'd. There's something i don't understand tho, there's some warning about /system partition which can't be mounted when i look around on the recovery of the slave phone.
I'm on stock rom, no root. I looked for similar post (really), but there's mostly guys which make backup / restore on the same phone, and i didn't find my answer.
My questions are : is there someone's who did this before ? Why the phone doesn't boot anymore ? Is there something that's making the backup unrecoverable on another phone ?
In this case, is there anyway to bypass that ?
Thank you by advance.
So...
We've done more tests, and when we only select the data partition, it does what we want.
So what I've done to deploy 50 phone :
- Prepared my "master" phone with all my apps and parameters
- Gone in TWRP to backup, saved the folder inside TWRP/BACKUPS/cde0654364576****/ (the generated folder), so saved the folder inside UNTOUCHED on my computer (for unknow reason, if i use it once to restore a phone, i can't use it again on another phone, so keep a copy somewhere)
- Boot to twrp on the slave phone, generate a backup to also generate the name folder
- Boot on system phone, delete what's inside the folder, and copy / past the previously generated backup from the master phone
- Boot again in twrp end restore. Reboot and that's done, you're slave phone should be the same as the master.
If someone find this thread, don't hesitate to ask more details.

How to recover data from "adb backup --twrp" archive? Been through hell and back

How to recover data from "adb backup --twrp" archive? Been through hell and back
I have two backups I made using "adb backup --twrp" while my OnePlus one phone was in TWRP recovery 3.2.1-0 running LineageOS with a recent build (don't know exactly which one, but it was from Feb/March 2018). Each of the backups are about 6.0GB so they appear to be complete backups. I am trying to recover one app and its data (Signal/Textsecure) but the rest would be nice too. The backup files have .ab extensions. I am fairly android savvy and have been working on this for days, your help is VERY MUCH APPRECIATED! I have access to Windows and Linux.
I have tried the following:
-"adb restore filename.ab" while in TWRP recovery. It appears to restore (progress bar goes, it reports success, etc), but no apps are restored. I have tried this both with totally wiped partitions/system/data and with a fresh install of LineageOS. As far as I can tell, nothing happens. When doing a restore after a full wipe and trying to restart, TWRP warns that no OS is installed.
-Restoring using the restore menu in TWRP. Immediately upon clicking on the backup, the phone restarts and TWRP opens again. Not sure what's going on there.
-Opening the backups in Titanium Backup, Nandroid manager, and a few other apps. None of them worked. Titanium backup crashes instantly in recover from ADB, doesn't recognize the backups in nandroid recovery mode.
-Opening the .ab files in an archive manager, all of which report it isn't a valid tar archive. One time I was able to get a list of directories, and things looked pretty right, so I know at least some of the data should be there. Running variants of this command with different skip values, none of which produces a valid tar file dd if=file.ab bs=512 skip=1 of=test.tar
Questions:
- Is there a way to extract data from these backups in Linux or Windows outside of Android? From what I read, TWRP adds some stuff in the headers which makes them invalid tar files, and that doing "adb backup --twrp" is different from a regular TWRP backup.
- Why is TWRP crashing? Is there a log I can look at for this and how? Maybe i can find a way to diagnose it that way.
- Where does Signal store message data? I have a flat-file backup that may have this information depending on where it's stored.
- Can I somehow convert these backups to a different format that might be better-tolerated by TWRP or other apps?
- Any good utilities for reading a corrupted tar file? I think all this utility would need to do is skip through the junk at the start.

How to restore a /data partition on Android Q from tarball after factory reset?

Hi! So, I have a newer Android 10 device with no TWRP support. I think my question is not model specific, so I'm posting generally, but if it matters, it's a OnePlus 7T Pro, bootloader unlocked of course.
Anyway, since there's no good custom recoveries for Android Q (none that I know of that can decrypt /data), I was trying to be safe and using a hacked (adb enabled) stock recovery to make backups from the command line. I have a nice tarball created in recovery mode with "tar cvf - /data/". It looks like it was made properly, I can unpack the files on back onto the device in recovery mode, and it's not corrupt or anything.
The problem is that naively trying to restore it back to my /data partition (e.g. tar xvf) after a wipe causes two problems:
1) after rebooting to recovery, the /data partition just looks encrypted again. If it was working properly, stock recovery can decrypt /data
2) normal system boot just bootloops.
If anyone has tried manually saving and restoring /data from a tarball, I'd appreciate any tips on getting it to work.
My best guess is that the ext4 encryption keys are the culprit - since after a factory reset the device creates new encryption keys, I think I need to figure out how to restore 99% of my backup without overwriting the fresh encryption keys that were created after wiping. I think I need to combine the new encryption keys with the other files in my tarball backup. If I can get past that, I might have to worry about things like SELinux xattrs as well, but one thing at a time.
If any experienced Android hackers know how filesystem encryption works in Android 10, I would appreciate a map of which files should not be overwritten. For example, I'm thinking I should probably avoid restoring some of these files:
Code:
/data/unencrypted/key
/data/unencrypted/key/encrypted_key
/data/unencrypted/key/keymaster_key_blob
/data/misc/vold/user_keys
/data/misc/vold/user_keys/de/0/keymaster_key_blob
/data/misc/vold/user_keys/de/0/encrypted_key
/data/misc/vold/user_keys/ce/0/current/keymaster_key_blob
/data/misc/vold/user_keys/ce/0/current/encrypted_key
/data/misc/systemkeys
/data/misc/keystore
/data/misc/keychain
/data/system/recoverablekeystore.db
[edit] spoke too soon
I ended up doing something janky and wrote a little shell script to repack the files from /data/data and /data/app into files that Titanium Backup could understand. I also figured out how to restore my SMS messages from /data/user_de. That was enough to recover the things I cared most about.
Still begs the question of what's the best method of doing backup and restores of /data on Android 10 when you don't have custom recovery but you do have Magisk, root, and stock recovery with adb. I want to keep hacking and playing with this phone so gotta figure something out.
I like Titanium Backup, have used it for many years, but the maintainance hasn't really kept up with Android development, theres a lot of gotchas like it hangs trying to restore multi APK packages. I still want to figure out something more seamless like a nandroid backup solution even if its from the command line. At least until TWRP eventually supports Android 10.
try backing up data partition in a external storage live usb or sd card
DrWowe said:
I ended up doing something janky and wrote a little shell script to repack the files from /data/data and /data/app into files that Titanium Backup could understand. I also figured out how to restore my SMS messages from /data/user_de. That was enough to recover the things I cared most about.
Still begs the question of what's the best method of doing backup and restores of /data on Android 10 when you don't have custom recovery but you do have Magisk, root, and stock recovery with adb. I want to keep hacking and playing with this phone so gotta figure something out.
I like Titanium Backup, have used it for many years, but the maintainance hasn't really kept up with Android development, theres a lot of gotchas like it hangs trying to restore multi APK packages. I still want to figure out something more seamless like a nandroid backup solution even if its from the command line. At least until TWRP eventually supports Android 10.
Click to expand...
Click to collapse
try backing up data partition in a external storage live usb or sd card, first you for encryption, then restore from external source after formatting phone storage

Categories

Resources