Related
This thread is for discussing features and improvements of this rooting method
For questions and problems read the [HOWTO] thread
Project site online: archos-gen8-sde-rooting
wdl1908 said:
chulri what about creating a 1Gb file on the /mnt/storage and formatting that as ext3 copying all the original /data to it and then mounting that with a loop interface on /data.
Click to expand...
Click to collapse
That's what I tried previously (before the /data thing), but I had no luck and it's a big issue because android tries to unmount /mnt/storage when you connect your archos device to the computer but that's not possible because the lock of the mounted rw-file makes umounting of /mnt/storage impossible and I have to mount the rw-file before /mnt/storage gets mounted, that's another issue which must be resolved.
chulri said:
That's what I tried previously (before the /data thing), but I had no luck and it's a big issue because android tries to unmount /mnt/storage when you connect your archos device to the computer but that's not possible because the lock of the mounted rw-file makes umounting of /mnt/storage impossible and I have to mount the rw-file before /mnt/storage gets mounted, that's another issue which must be resolved.
Click to expand...
Click to collapse
Yeah I came to the same conclusion as you, that what I suggested would bork the usb mount option to the PC.
Another thing I realized is that the official firmware upgrades could probably update files on the data partition. So moving the whole partition is not an option as that would break the upgrade process.
I've been looking at splitting the storage partition in several parts with parted I found an arm binary at http://plugapps.com/arm/ maybe these can be included in the initramfs.
I've also been analyzing my data partition
Code:
# du -s /data/*
112003 app
70503 dalvik-cache
40084 data
4622 test
The test directory is the place where the google market is installed via arctools or gappsinstaller.
So if it's possible to split the storage partition in several part we could move these dirs to it's own partition. This would not be optimal a good solution would be to move the complete data partition over but this needs a bit of thinking how to handle upgrades.
wdl1908 said:
So if it's possible to split the storage partition in several part we could move these dirs to it's own partition. This would not be optimal a good solution would be to move the complete data partition over but this needs a bit of thinking how to handle upgrades.
Click to expand...
Click to collapse
We could shrink the internal storage and append partition(s) after it. I'll give it a try, as long as my usb port is broken I have more time to focus on this here
chulri said:
We could shrink the internal storage and append partition(s) after it. I'll give it a try, as long as my usb port is broken I have more time to focus on this here
Click to expand...
Click to collapse
I've been trying to cross compile e2fsprogs and parted but I can't seem to get it.
Code:
e2fsprogs-1.41.14$ cross ./configure --host=arm-linux-uclibcgnueabi --build=i686-linux
Completes without errors but the make does not complete.
Code:
gen_uuid.c:(.text+0x418): undefined reference to `__aeabi_read_tp'
../../lib/libuuid.a(gen_uuid.o):gen_uuid.c:(.text+0x788): more undefined references to `__aeabi_read_tp' follow
I've tried to use the pre-compiled packages but it seems they don't work or i'm missing something.
fdisk is already included in initramfs thus no need for a parted binary.
edit: but to minimize data loss we need a resize2fs binary to resize the fat/ext3 partition
mkfs.ext3 (for the rw partition) and fsck are included too in the initramfs by archos
chulri said:
fdisk is already included in initramfs thus no need for a parted binary.
edit: but to minimize data loss we need a resize2fs binary to resize the fat/ext3 partition
mkfs.ext3 (for the rw partition) and fsck are included too in the initramfs by archos
Click to expand...
Click to collapse
Yep resize2fs is part of e2fsprogs. I've been working on the packages in the buildroot there seems to be a lot of errors but I finally succeeded in building the e2fsprogs package. but riseze2fs is not included. I need to check the config for that package maybe there is an option missing.
To get the buildroot working properly you need to copy the file
Code:
cp local/g8_arm/g8_arm.config .config
remove the line
Code:
package/apdf/Config.in
from .config.cmd
remove the line
Code:
depends on BR2_EXT_UCLIBC_VERSION_0_9_30_1
from toolchain/uClibc/Config.in
then in the buildroot directory execute
Code:
make menuconfig
enable the e2fsprogs in Package selection -> Harware handling
also you have to remove the --disable-resizer from the e2fsprogs.mk file else the resizer is not build.
I can't believe this build package from archos is very up-to-date it seems very strange that all these bugs are in there how where they ever building a good firmware.
but that builds dynamic linked binaries, doesn't it? but we need a static build, don't we?
chulri said:
but that builds dynamic linked binaries, doesn't it? but we need a static build, don't we?
Click to expand...
Click to collapse
Yeah stupid me it needs to run in the initramfs and that does not contain any libraries. Let me check to see if it's possible to link it statically.
and because it has to be statically linked, maybe we better just take the newest e2fsprogs (btw.: does this support FAT resizing?!) and compile it without that buildroot stuff (except for the toolchain of course, we need that to crosscompile)
chulri said:
and because it has to be statically linked, maybe we better just take the newest e2fsprogs (btw.: does this support FAT resizing?!) and compile it without that buildroot stuff (except for the toolchain of course, we need that to crosscompile)
Click to expand...
Click to collapse
Well I tried that and failed. That's why I wanted to do it in the buildroot. I'll try again later need to create a clean environment and do some diffs after I fixed all the stuff that's wrong.
wdl1908 said:
Well I tried that and failed. That's why I wanted to do it in the buildroot. I'll try again later need to create a clean environment and do some diffs after I fixed all the stuff that's wrong.
Click to expand...
Click to collapse
I think I got it.
Add
Code:
export BOARD=g8_arm
To your .bashrc then in the buildroot directory do a make it will take a while as it needs to build everything. This is just a precaution as i think the statically linked resize2fs needs some linking with uclib libraries.
After that is finished do a
Code:
make e2fsprogs LDFLAGS=-static
in the buildroot directory. You should find the resize2fs binary in the directory buildroot/project_build_arm/uclibc/root/sbin
Code:
$ file resize2fs
resize2fs: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, not stripped
resize2fs does not resize vfat so we probably need parted and some extra utils
How to compile parted with buildroot.
I found the attached files on some forum
e3fsprogs.mk is a replacement for the existing file.
Config.in parted.mk and parted-001-ui.cast.patch need to be placed in the directory buildroot/package/parted
then execute the following commands
Code:
make e2fsprogs LDFLAGS=-static
make e2fsprogs-libs
make parted LDFLAGS=-static
you can find the statically linked parted in buildroot/build_arm/parted-2.3/parted
and this is what i tried.
Code:
# [B]parted /dev/block/mmcblk1[/B]
GNU Parted 2.3
Using /dev/block/mmcblk1
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) [B]print[/B]
print
Model: MMC MMC08G (sd/mmc)
Disk /dev/block/mmcblk1: 7466MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 8192B 7466MB 7466MB primary fat32 lba
(parted) [B]resize[/B]
resize
WARNING: you are attempting to use ./parted to operate on (resize) a file system.
./parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use ./parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition number? [B]1[/B]
1
Start? [8192B]?
End? [7466MB]? [B]6466MB[/B]
6466MB
(parted) [B]check[/B]
check
WARNING: you are attempting to use ./parted to operate on (check) a file system.
./parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs. We recommend
you use ./parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition number? [B]1[/B]
1
(parted) [B]quit[/B]
quit
Information: You may need to update /etc/fstab.
#[B]fdisk /dev/block/mmcblk1[/B]
The number of cylinders for this disk is set to 227840.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): [B]p[/B]
Disk /dev/block/mmcblk1: 7465 MB, 7465861120 bytes
4 heads, 16 sectors/track, 227840 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
Device Boot Start End Blocks Id System
/dev/block/mmcblk1p1 1 197327 6314445+ c Win95 FAT32 (LBA)
Command (m for help): [B]n[/B]
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): [B]2[/B]
First cylinder (197327-227840, default 197327): Using default value 197327
Last cylinder or +size or +sizeM or +sizeK (197327-227840, default 227840): Using default value 227840
Command (m for help): [B]p[/B]
Disk /dev/block/mmcblk1: 7465 MB, 7465861120 bytes
4 heads, 16 sectors/track, 227840 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
Device Boot Start End Blocks Id System
/dev/block/mmcblk1p1 1 197327 6314445+ c Win95 FAT32 (LBA)
/dev/block/mmcblk1p2 197327 227840 976426+ 83 Linux
Command (m for help): [B]w[/B]
The partition table has been altered!
Calling ioctl() to re-read partition table
# [B]mkfs.ext3 -v -I 128 /dev/block/mmcblk1p2[/B]
mke2fs 1.40.9 (27-Apr-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61184 inodes, 244106 blocks
12205 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=251658240
8 block groups
32768 blocks per group, 32768 fragments per group
7648 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
# [B]tune2fs.static -c -1 -i 0 -m 1 /dev/block/mmcblk1p2[/B]
tune2fs 1.40.9 (27-Apr-2008)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 1% (2441 blocks)
I leave the scripting to you but with these utils it should work perfectly to resize the partition and create the second partition.
I copied the mkfs.ext3, fdisk and tune2fs.static from the recovery initramfs
I started hacking around and I came to the conclusion that it would be better to change the initramfs to mount /data from mmcblk1p2 if that partition exists and not if it's not existing and move the whole partition resizing, partition creating, partition deleting (if you wan't to revert without dataloss) and again resizing into an app. so the user has more control over what he's doing and see's if something is failing and not just get's a bootloop or some fancy log file.
edit: /data is mounted by /init.rc script, all of the above can be done by an app, incl. modifying /init.rc script. no need for any special initramfs, yay!
everybody who has +rw rooting installed will be able to use that app. I'm starting development...
edit2: project page online: http://code.google.com/p/archos-gen8-sde-rooting/
stay tuned
First test app: http://code.google.com/p/archos-gen8-sde-rooting/downloads/detail?name=AppDataResizer_v0.1.apk
Release notes:
initial test version v0.1:
- parted binary added
- test button lists partitions of mmcblk1 device
Click to expand...
Click to collapse
note: 250 GB version of the A70 is currently not supported.
chulri said:
First test app: http://code.google.com/p/archos-gen8-sde-rooting/downloads/detail?name=AppDataResizer_v0.1.apk
Release notes:
note: 250 GB version of the A70 is currently not supported.
Click to expand...
Click to collapse
Nice. Yep I was thinking about the transition from standard to custom and also came to the conclusion it had to be done outside the boot process scripts.
Edit: Should this app be installable via the usual way or should it be a system app? (Copied to /system/app)
What I was thinking was split the process in 3 steps.
Step1: Resize storage partition, Add new-data partition and format.
Step2: Copy existing /data to /new-data
Step3: Enable/Disable new-data
Maybe a step2a: To run after upgrade of firmware to check things that have changed.
The step1 requires a reboot as the partitioning should be done in the initramfs if you do that when apps are running you're going to have a hell of a time getting the storage partition unmounted (I know I had the problem when testing the parted binary)
Step2 can be done without any problem when storage is mounted and Step3 requires a reboot after the init.rc is changed.
wdl1908 said:
Nice. Yep I was thinking about the transition from standard to custom and also came to the conclusion it had to be done outside the boot process scripts.
Edit: Should this app be installable via the usual way or should it be a system app? (Copied to /system/app)
Click to expand...
Click to collapse
usual way (download and install) (or maybe I include it in the initramfs (like the Superuser.apk) and copy it to /system/app, but I don't like modifying initramfs any further, no need for 100 different versions )
wdl1908 said:
What I was thinking was split the process in 3 steps.
Step1: Resize storage partition, Add new-data partition and format.
Step2: Copy existing /data to /new-data
Step3: Enable/Disable new-data
Maybe a step2a: To run after upgrade of firmware to check things that have changed.
Click to expand...
Click to collapse
great, that were my plans too.
wdl1908 said:
The step1 requires a reboot as the partitioning should be done in the initramfs if you do that when apps are running you're going to have a hell of a time getting the storage partition unmounted (I know I had the problem when testing the parted binary)
Click to expand...
Click to collapse
No API to unmount /sdcard/? I think I got one: IMountService it's not a public API but android.os.FileUtils isn't public either and it's working great. I think IMountService is the API that the popup, which pops up when you connect your android device to the computer, uses. I think, no need to worry because android handles everything pretty well when you connect your device to the computer, isn't it?
wdl1908 said:
Step2 can be done without any problem when storage is mounted and Step3 requires a reboot after the init.rc is changed.
Click to expand...
Click to collapse
agreed
chulri said:
usual way (download and install) (or maybe I include it in the initramfs (like the Superuser.apk) and copy it to /system/app, but I don't like modifying initramfs any further, no need for 100 different versions )
Click to expand...
Click to collapse
I tried to install it but it won't install. logcat gives something like
Pckage chrulri.gen8.AppDataResizer has no certificated at entry res/layout/main.xml
Yep I agree no need for different versions of the initramfs
chulri said:
No API to unmount /sdcard/?
Click to expand...
Click to collapse
The problem is not only the mount of /mnt/storage if the user has used move2sd there are a lot of other mounts present that also uses that partition.
maybe the API will do. Needs to be tested.
One other remark. Let the AppDataResizer check for the unionfs directory so that you can be sure you'r running on the correct initramfs.
So after a failed attempt to upgrade from CyanogenMod 10.1.3 to 10.2, I was unable to access /data or /sdcard because both systems were encrypted. I ended up having to factory reset my phone because it refused to co-operate or let me access my files. However, before I did that, I was able to run
Code:
adb shell "dd if=/dev/block/mmcblk0p2" > data.img
and
Code:
adb shell "dd if=/dev/block/mmcblk0p3" > sdcard.img
, which appears to have copied the raw partition images from the phone (at least, they're the right sizes).
According to my reading, Android (and, I'm inferring, CyanogenMod) encrypts filesystems using dm-crypt, with a AES-CBC ESSIV:SHA256 cipher, with the key being derived from the password using PBKDF2. Knowing the precious little I do about encrypted file systems, my guess is that if I configure the image in cryptsetup to create a drive mapping, I can mount the mapped drive and recover the data from the images.
According to /fstab.herring on my ahem, fresh, install of Android, the /data partition is in ext4 format whereas the /sdcard partition is vFAT. So, once I've gotten through the encryption on the partition images, they should mount normally, right?
I know that dm-crypt accepts plain, LUKS, LoopAES and TrueCrypt device formats. I'm inferring from the PBKDF2 extension that Android goes the LUKS route for encrypting. Is this conclusion correct?
Could someone explain whether it's possible to decrypt a dumped android image? I'm really hoping that the cypher information is stored on the file system and not on some key file that I nuked in the factory reset. If it can, in theory, be decrypted, am I using the right tools to approach the matter? If so, I'll continue fiddling with cryptsetup and mount, but no sense in wasting time if it's an impossible task.
Never did get a response to this question, so I'll try it again, but start with a simpler question:
If someone dds an Android (specifically Cyanogenmod 10.x) partition to an img file, is there any way to read that image from, say a Linux laptop? I dumped the contents of the /system partition using
Code:
adb shell "dd if=/dev/block/mmcblk0p1" > system.img
I expected system.img to be a normal ext4 partition. However, attempting to loopback mount it with
Code:
sudo mount -t ext4 -o loop,ro system.img ~/android/system
Gave me errors about corrupt group descriptors, bad magic numbers and other maladies indicative of a thoroughly corrupted file system. I'm assuming that:
/data has the same ext4 partition structure as /system; and
The process to mount /storage would be no different to mounting /system with the exception that the former uses vFAT as its file system
However, as my Android is currently working normally (well, as well as one can hope for Android to work), I know I don't have a corrupted file system.
So what's going on? Does Android use a special version of ext4 that other Linuxes don't recognise? Am I not dd-ing correctly? Is there a block-size issue I ignored to my peril?
Borden Rhodes said:
So after a failed attempt to upgrade from CyanogenMod 10.1.3 to 10.2, I was unable to access /data or /sdcard because both systems were encrypted. I ended up having to factory reset my phone because it refused to co-operate or let me access my files. However, before I did that, I was able to run
Code:
adb shell "dd if=/dev/block/mmcblk0p2" > data.img
and
Code:
adb shell "dd if=/dev/block/mmcblk0p3" > sdcard.img
, which appears to have copied the raw partition images from the phone (at least, they're the right sizes).
According to my reading, Android (and, I'm inferring, CyanogenMod) encrypts filesystems using dm-crypt, with a AES-CBC ESSIV:SHA256 cipher, with the key being derived from the password using PBKDF2. Knowing the precious little I do about encrypted file systems, my guess is that if I configure the image in cryptsetup to create a drive mapping, I can mount the mapped drive and recover the data from the images.
According to /fstab.herring on my ahem, fresh, install of Android, the /data partition is in ext4 format whereas the /sdcard partition is vFAT. So, once I've gotten through the encryption on the partition images, they should mount normally, right?
I know that dm-crypt accepts plain, LUKS, LoopAES and TrueCrypt device formats. I'm inferring from the PBKDF2 extension that Android goes the LUKS route for encrypting. Is this conclusion correct?
Could someone explain whether it's possible to decrypt a dumped android image? I'm really hoping that the cypher information is stored on the file system and not on some key file that I nuked in the factory reset. If it can, in theory, be decrypted, am I using the right tools to approach the matter? If so, I'll continue fiddling with cryptsetup and mount, but no sense in wasting time if it's an impossible task.
Click to expand...
Click to collapse
Can you give the result of the "file sdcard.img" and "file data.img" commands?
You are quite right. With regular LUKS container/partition, you would do (being root) the following. With the following commands, you can create a container named "safe", setup it, then format its content in ext3 and mount the partition:
Code:
dd if=/dev/zero bs=1M count=50 of=safe
losetup /dev/loop0 safe
cryptsetup luksFormat -c aes -h sha256 /dev/loop0
cryptsetup luksOpen /dev/loop0 safe
mkfs.ext3 /dev/mapper/safe
(losetup /dev/loop0 safe)
(cryptsetup luksOpen /dev/loop0 safe)
mkdir mnt
mount -t ext3 /dev/mapper/safe mnt
//HERE: do whatever you want in your mounted encrypted filesystem
umount mnt
cryptsetup luksClose safe
losetup -d /dev/loop0
For details, you can go there: http://blog.theglu.org/index.php/20...-couteau-suisse-du-chiffrement-de-partitions/
Sorry, the article is in French but you can translate it if you need to.
Here, using "hexdump", you can see the "safe" file has a LUKS magic at the beginning. And doing a "file safe" command, you can check it detects it as a "LUKS encrypted file".
If doing "file" on your .img files does not give you the same result, you may not be able to directly use the "cryptsetup" command and need to adapt it.
Finally: usually in Android the header containing the key is stored on another partition so you may have lost it when wiping your phone, sorry.
---------- Post added at 02:44 PM ---------- Previous post was at 02:41 PM ----------
Borden Rhodes said:
Never did get a response to this question, so I'll try it again, but start with a simpler question:
If someone dds an Android (specifically Cyanogenmod 10.x) partition to an img file, is there any way to read that image from, say a Linux laptop? I dumped the contents of the /system partition using
Code:
adb shell "dd if=/dev/block/mmcblk0p1" > system.img
I expected system.img to be a normal ext4 partition. However, attempting to loopback mount it with
Code:
sudo mount -t ext4 -o loop,ro system.img ~/android/system
Gave me errors about corrupt group descriptors, bad magic numbers and other maladies indicative of a thoroughly corrupted file system. I'm assuming that:
/data has the same ext4 partition structure as /system; and
The process to mount /storage would be no different to mounting /system with the exception that the former uses vFAT as its file system
However, as my Android is currently working normally (well, as well as one can hope for Android to work), I know I don't have a corrupted file system.
So what's going on? Does Android use a special version of ext4 that other Linuxes don't recognise? Am I not dd-ing correctly? Is there a block-size issue I ignored to my peril?
Click to expand...
Click to collapse
Can you give the result of the "file system.img" command?
Thanks, saidlike, for your reply:
saidelike said:
Can you give the result of the "file sdcard.img"...
Click to expand...
Click to collapse
sdcardPartitionDump.img: data
saidelike said:
... and "file data.img" commands?
Click to expand...
Click to collapse
data.img: data
saidelike said:
Can you give the result of the "file system.img" command?
Click to expand...
Click to collapse
system.img: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
Again, attempting to run
Code:
mount -t ext4 -o loop systemimg mountpoint/
yields
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
Click to expand...
Click to collapse
Ignoring the results of data.img and sdcard.img for the time being, the fresh dump of the system partition shows that it's an EXT4 filesystem, but that it's heavily corrupted. fsck.ext4 on that partition basically asks me to fix every single inode, so it's not a simple unclean journal issue. Therefore, is it fair to conclude that CyanogenMod (and maybe AOSP too) have modified the ext4 partiiton type?
@Borden Rhodes
Maybe, my reply is too late, but you could try to make the same experiment with backup of your current data.
If you get the same results as with the old pre-wipe backup, then you still have a hope.
Hello
I have mounted system.img with linux the format i think it ext4 but the file is not sparse so i was successful using the following command.
sudo mount -t ext4 system.img ~/firmware2 -o loop
When the system image is mounted i have made some edits to the live image then i have unmounted the image and transferd the image back to my windows computer and ran the system.img through ext4 unpacker and extunpacker shows that my changes havent been made and and the file is the same as before however when i remount the system.img on my linux pc it shows the changes.
Im really stuck here and if anyone could help that would be great.
I am only new here but when i possible i will contribute and repay favours.
ps with the unmount i use visual unmount but have also tried unmounting with commands i used are umount /firmware2
re
I have sorted this issue now and managed to fix.
I have TWRP installation file in img format ready to be loaded with
Code:
fastboot boot twrp-unofficial.img
.
I don't have the source and want to have a look at what is packaged with it.
Is it possible to mount this img file on linux or extract the contents somehow?
Code:
file twrp-unofficial.img
twrp-unofficial.img: Android bootimg, kernel, ramdisk, page size: 2048, cmdline (console=null androidboot.console=ttyHSL0 androidboot.hardware=q)
I tried with simg2img but failed
Code:
simg2img twrp-unofficial.img twrp.raw
Invalid sparse file format at header magi
Failed to read sparse file
Furius ISO mount creates a mount point but it is an empty directory.
? How to recover deleted files on rooted Android without USB Debug & PC connection?
Hello.
I have removed some important files in my DCIM folder on internal memory of my Android device. The USB socket of the phone is broken so I couldn't use any recovery software that using USB debug mode but I have Team Win I want to use the terminal of Team Win to make an image and copy it to SD Card of my Android device. I know that there is a command dd but how to use it in a proper way to make a full image of the partition including also free space.
Thank you very much in advance!
Yes, dd command could be used simmilar to this:
open terminal, cd to external SD folder
dd if=/dev/block/block/bootdevice/by-name/userdata of=data.img
or if you know number of partition
dd if=/dev/block/mmcblk0p18 of=data.img (p18 is on Huawei LDN, image size is that same as partition size 16GB/32GB/64GB..etc, so for bigger then 32GB need to use NTFS sdcard or exFAT sdcard and TWRP also has to support NTFS or exFAT).
Or edit etc/*.fstab and repack twrp. You can back up files from /data for now (as ext4 or f2fs). Just add line to back up full image of /data (as emmc).
If /data has ext4 filesystem it can easilly mount/unpack/scan/rip image. But if /data has f2fs ... got not cure.
Example:
/data f2fs /dev/block/bootdevice/by-name/userdata flags=length=-16384;backup=1;settingsstorage;encryptable=footer;
/data_image emmc /dev/block/mmcblk0p55 flags=display="Data Image";backup=1;flashimg;
adeii said:
Yes, dd command could be used simmilar to this:
open terminal, cd to external SD folder
dd if=/dev/block/block/bootdevice/by-name/userdata of=data.img
or if you know number of partition
dd if=/dev/block/mmcblk0p18 of=data.img (p18 is on Huawei LDN, image size is that same as partition size 16GB/32GB/64GB..etc, so for bigger then 32GB need to use NTFS sdcard or exFAT sdcard and TWRP also has to support NTFS or exFAT).
Or edit etc/*.fstab and repack twrp. You can back up files from /data for now (as ext4 or f2fs). Just add line to back up full image of /data (as emmc).
If /data has ext4 filesystem it can easilly mount/unpack/scan/rip image. But if /data has f2fs ... got not cure.
Example:
/data f2fs /dev/block/bootdevice/by-name/userdata flags=length=-16384;backup=1;settingsstorage;encryptable=footer;
/data_image emmc /dev/block/mmcblk0p55 flags=display="Data Image";backup=1;flashimg;
Click to expand...
Click to collapse
Thank you very much for you point to point reply!
Finally, I choose to use that option with some modifications because vfat doesn't support files larger than 4GB.
Code:
dd if=/dev/block/bootdevice/by-name/userdata conv=noerror,sync bs=100M | gzip -c | split -b1000000000 - mybackup.img.gz
I have mounted this *.img partition using OSFMount for Windows. But after the scanning process (I was using R-Studio that supports ext4 file system I found my deleted files in the tree structure /media/0/DCIM/Camera but all of the deleted files has 0 bytes size and have 2 flags: deleted, wiped.
I couldn't understand how that happened. I mean I didn't use my phone after deleting files at all. I also mounted this *.img as raw disk in Active Undelete but the result is actually the same all of the deleted files have 0 bytes file size.
Is that a bug of the program? Or I have made an image using wrong command? Or Android 9 actually wiping files after deletion?
The files have been accidentally deleted by AirDroid-web app but I don't think so that this app is wiping deleted files it doesn't make sense...
RaTr said:
Thank you very much for you point to point reply!
Click to expand...
Click to collapse
You are welcome. Thank you are for note about 4GB file size limit on vfat/fat32, will save us from a lot of headache.
---------- Post added at 10:03 AM ---------- Previous post was at 09:49 AM ----------
RaTr said:
actually the same all of the deleted files have 0 bytes file size.
Click to expand...
Click to collapse
Maybe to try DiskDigger on phone if it is rooted to scan internal sd?
PhotoRec for Windows/Linux: https://www.cgsecurity.org/wiki/PhotoRec_Step_By_Step
There are few programs to try on your image on GNU/Linux like extundelete, ext4magic, AnalyzeEXT, ext3grep ...
Source: https://askubuntu.com/questions/217606/undelete-files-on-ext4
I am trying to use ext4magic to recover deleted files. But I need a copy of the journal when I am trying to use command
Code:
debugfs -R "dump <8> /var/tmp/home.journal" /dev/mapper/home
I see that Team win terminal do not have this command how to add it or how to make a copy of the journal of my ext4 partition where I am trying to recover my files.
adeii said:
Maybe to try DiskDigger on phone if it is rooted to scan internal sd?
Click to expand...
Click to collapse
I have tried to use DiskDigger. When I am putting a filter to show only deleted files it doesn't show anything. Which is pretty strange.
adeii said:
There are few programs to try on your image on GNU/Linux like extundelete, ext4magic, AnalyzeEXT, ext3grep ...
Source: https://askubuntu.com/questions/217606/undelete-files-on-ext4
Click to expand...
Click to collapse
Next program that I have tried was TestDisk but looks like it doesn´t support ext4 file system.
Thank you for the advice about PhotoRec. It has support of ext4 system. But it won't help. So, there are still some options. I will try to use extundelete as the next one.
If you have an image of EXT4, then you can use 7-Zip Archiver to read all the files inside it.
jwoegerbauer said:
If you have an image of EXT4, then you can use 7-Zip Archiver to read all the files inside it.
Click to expand...
Click to collapse
7-Zip will show deleted files also?..
I have tried 2 more utilities:
1. ext4magic without external journal file. It has done the job but I couldn't find any files that I need.
2. extundelete that program restored less files and also no files that I want to recover.
One more strange thing:
I have installed R-Studio and opened my image there. I fount full list of my deleted files, but all of the records of my deleted files have 2 flags: deleted, wiped and it shows me that the size of that specific files is 0 bytes.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
I have checked other files from different dates, they can be recovered, there is no flag wiped and also I can see the size, in some of them there is a flag cross-link, but I think it is normal, that means part of that file is already overwritten by another one.
RaTr said:
I have tried 2 more utilities:
1. ext4magic without external journal file. It has done the job but I couldn't find any files that I need.
2. extundelete that program restored less files and also no files that I want to recover.
One more strange thing:
I have installed R-Studio and opened my image there. I fount full list of my deleted files, but all of the records of my deleted files have 2 flags: deleted, wiped and it shows me that the size of that specific files is 0 bytes.
I have checked other files from different dates, they can be recovered, there is no flag wiped and also I can see the size, in some of them there is a flag cross-link, but I think it is normal, that means part of that file is already overwritten by another one.
Click to expand...
Click to collapse
I am stuck in the same situation bro
See it here I described the issue very similar to yours. Any luck in trying to recover that `ext4` data
RaTr said:
I found my deleted files in the tree structure /media/0/DCIM/Camera but all of the deleted files has 0 bytes size and have 2 flags: deleted, wiped.
Click to expand...
Click to collapse
expected.
emmc/ufs flash storage is handled different from hard disk drive. there is FTL controller with own firmware that is wear-leveling whole storage all the time. not to mention files are fragmented.
file system sends TRIM on each deletion of file. note the discard mount flag for userdata partition.
Android 4.3 Update Brings TRIM to All Nexus Devices
www.anandtech.com
Hi, I tried dd command, and it returned
Code:
dd: data.img: Read-only file system
But ./adb pull /dev/block/mmcblk0p57 57.img worked, and created a NDIF image.
Why dd did not work?
And which format of image would dd create?
Thanks.
you're trying to write into phones / rootdir. you cannot dump partition into phone itself. external MicroSD card or OTG pendrive is required. but you could redirect to stdout into remote file (note the quotes make the difference where > redirection is executed)
Code:
adb root
adb shell 'dd if=/dev/block/mmcblk0p57 bs=1m status=none' > data.img
open data.img file with HxD editor and have a look into first bytes. search for magic 53 ef at offset 0x438 to confirm it's ext4 image.
dd is useful in case no usb connection available (topic of thread). the result is same as adb pull. you can increase speed with block size (bs= default 512 bytes) up to 1 MB.
Note: on FDE encrypted phone one can't pull userdata directly. instead pull whatever is mounted /data (like /dev/block/dm-0)
aIecxs said:
you're trying to write into phones / rootdir. you cannot dump partition into phone itself. external MicroSD card or OTG pendrive is required. but you could redirect to stdout into remote file (note the quotes make the difference where > redirection is executed)
Code:
adb root
adb shell 'dd if=/dev/block/mmcblk0p57 bs=1m status=none' > data.img
open data.img file with HxD editor and have a look into first bytes. search for magic 53 ef at offset 0x438 to confirm it's ext4 image.
dd is useful in case no usb connection available (topic of thread). the result is same as adb pull. you can increase speed with block size (bs= default 512 bytes) up to 1 MB.
Note: on FDE encrypted phone one can't pull userdata directly. instead pull whatever is mounted /data (like /dev/block/dm-0)
Click to expand...
Click to collapse
Thanks for your reply.
What would be the differences between
adb shell 'dd if=/dev/block/mmcblk0p57 bs=1m status=none' > data.img
and
the method described in this message: https://stackoverflow.com/a/41214172 ?
Thanks.
streaming over netcat avoids unwanted characters using stty raw. on macOS probably result is no difference (after unpacking gzip).
you don't need this as you wrote adb pull worked (which is the easiest method)
aIecxs said:
streaming over netcat avoids unwanted characters using stty raw. on macOS probably result is no difference (after unpacking gzip).
you don't need this as you wrote adb pull worked (which is the easiest method)
Click to expand...
Click to collapse
Thanks.
I searched Hex in 57dd.img created by adb shell 'dd if=/dev/block/mmcblk0p57 bs=1m status=none' > 57dd.img.
It showed
Is the 53EF on line 1080 in the image above is the one you mentioned "at offset 0x438"?
right, that is ext4 magic at offset/byte hex (0x438)16 = (1080)10 dec
I have a slightly different but related question /problem. My apologies if this is not the right place to post.
On my Samsung A10e, TWRP (and other recoveries) gives me a tarfork 255 error when trying to backup userdata. Normally this should be 32GB in size from Samsung specs.
I have used two alternatives :
* adb pull /dev/block/by-name/userdata data.img which creates a 26G file on my linux PC, that I can mount and inspect. When I run filesystem check it throws out errors, probably due to some of the data being encrypted - I am running AOSP Android 11 gsi. Perhaps the same errors that prevents TWRP from working ?
* alternatively I have put in a clean 32GB SD card, then via adb shell run dd if=/dev/block/by-name/userdata of=/dev/block/mmcblk1p1 to copy the full userdata partition over to the SD card. Once removed from the phone and put in a card reader, again this can be inspected on my linux PC, and gives the same filesystem check errors. I also ran dd if=/dev/external_sd of=data2.img on my linux PC to create a similar image file as adb pull but it is now the full 32GB that I would have intially expected.
So why the size difference between adb pull and dd ?? Does adb pull actually get everything - in other words if I try and restore with adb push will the phone recover to previous state and boot ?
I was hoping to then reduce the size of my data copy to 16GB since on my phone I am only using 12GB of the 32GB, but file errors are preventing me at the moment.
I was thinking about wiping data and then with arm32 parted and adb shell creating a 16GB userdata partition and an additional 16GB user shadow partition - the latter only to be used locally via dd to do backup and restore and avoid TWRP errors.
that is very lightly due to encryption issues....esp Magisk is known to break encryption.
if you can decrypt at TWRP level and then proceed to backup user data via ADB shell to memory card that is your best bed and then wipe the partition and then restore your app data individually for apps that you need e.g. contacts db for contact an SMS. I strongly doubt if you will be able to restore that entire partition and boot to that original partition successfully.... it may have become what what we can call "encryption tainted"
excuse my typos