So after a failed attempt to upgrade from CyanogenMod 10.1.3 to 10.2, I was unable to access /data or /sdcard because both systems were encrypted. I ended up having to factory reset my phone because it refused to co-operate or let me access my files. However, before I did that, I was able to run
Code:
adb shell "dd if=/dev/block/mmcblk0p2" > data.img
and
Code:
adb shell "dd if=/dev/block/mmcblk0p3" > sdcard.img
, which appears to have copied the raw partition images from the phone (at least, they're the right sizes).
According to my reading, Android (and, I'm inferring, CyanogenMod) encrypts filesystems using dm-crypt, with a AES-CBC ESSIV:SHA256 cipher, with the key being derived from the password using PBKDF2. Knowing the precious little I do about encrypted file systems, my guess is that if I configure the image in cryptsetup to create a drive mapping, I can mount the mapped drive and recover the data from the images.
According to /fstab.herring on my ahem, fresh, install of Android, the /data partition is in ext4 format whereas the /sdcard partition is vFAT. So, once I've gotten through the encryption on the partition images, they should mount normally, right?
I know that dm-crypt accepts plain, LUKS, LoopAES and TrueCrypt device formats. I'm inferring from the PBKDF2 extension that Android goes the LUKS route for encrypting. Is this conclusion correct?
Could someone explain whether it's possible to decrypt a dumped android image? I'm really hoping that the cypher information is stored on the file system and not on some key file that I nuked in the factory reset. If it can, in theory, be decrypted, am I using the right tools to approach the matter? If so, I'll continue fiddling with cryptsetup and mount, but no sense in wasting time if it's an impossible task.
Never did get a response to this question, so I'll try it again, but start with a simpler question:
If someone dds an Android (specifically Cyanogenmod 10.x) partition to an img file, is there any way to read that image from, say a Linux laptop? I dumped the contents of the /system partition using
Code:
adb shell "dd if=/dev/block/mmcblk0p1" > system.img
I expected system.img to be a normal ext4 partition. However, attempting to loopback mount it with
Code:
sudo mount -t ext4 -o loop,ro system.img ~/android/system
Gave me errors about corrupt group descriptors, bad magic numbers and other maladies indicative of a thoroughly corrupted file system. I'm assuming that:
/data has the same ext4 partition structure as /system; and
The process to mount /storage would be no different to mounting /system with the exception that the former uses vFAT as its file system
However, as my Android is currently working normally (well, as well as one can hope for Android to work), I know I don't have a corrupted file system.
So what's going on? Does Android use a special version of ext4 that other Linuxes don't recognise? Am I not dd-ing correctly? Is there a block-size issue I ignored to my peril?
Borden Rhodes said:
So after a failed attempt to upgrade from CyanogenMod 10.1.3 to 10.2, I was unable to access /data or /sdcard because both systems were encrypted. I ended up having to factory reset my phone because it refused to co-operate or let me access my files. However, before I did that, I was able to run
Code:
adb shell "dd if=/dev/block/mmcblk0p2" > data.img
and
Code:
adb shell "dd if=/dev/block/mmcblk0p3" > sdcard.img
, which appears to have copied the raw partition images from the phone (at least, they're the right sizes).
According to my reading, Android (and, I'm inferring, CyanogenMod) encrypts filesystems using dm-crypt, with a AES-CBC ESSIV:SHA256 cipher, with the key being derived from the password using PBKDF2. Knowing the precious little I do about encrypted file systems, my guess is that if I configure the image in cryptsetup to create a drive mapping, I can mount the mapped drive and recover the data from the images.
According to /fstab.herring on my ahem, fresh, install of Android, the /data partition is in ext4 format whereas the /sdcard partition is vFAT. So, once I've gotten through the encryption on the partition images, they should mount normally, right?
I know that dm-crypt accepts plain, LUKS, LoopAES and TrueCrypt device formats. I'm inferring from the PBKDF2 extension that Android goes the LUKS route for encrypting. Is this conclusion correct?
Could someone explain whether it's possible to decrypt a dumped android image? I'm really hoping that the cypher information is stored on the file system and not on some key file that I nuked in the factory reset. If it can, in theory, be decrypted, am I using the right tools to approach the matter? If so, I'll continue fiddling with cryptsetup and mount, but no sense in wasting time if it's an impossible task.
Click to expand...
Click to collapse
Can you give the result of the "file sdcard.img" and "file data.img" commands?
You are quite right. With regular LUKS container/partition, you would do (being root) the following. With the following commands, you can create a container named "safe", setup it, then format its content in ext3 and mount the partition:
Code:
dd if=/dev/zero bs=1M count=50 of=safe
losetup /dev/loop0 safe
cryptsetup luksFormat -c aes -h sha256 /dev/loop0
cryptsetup luksOpen /dev/loop0 safe
mkfs.ext3 /dev/mapper/safe
(losetup /dev/loop0 safe)
(cryptsetup luksOpen /dev/loop0 safe)
mkdir mnt
mount -t ext3 /dev/mapper/safe mnt
//HERE: do whatever you want in your mounted encrypted filesystem
umount mnt
cryptsetup luksClose safe
losetup -d /dev/loop0
For details, you can go there: http://blog.theglu.org/index.php/20...-couteau-suisse-du-chiffrement-de-partitions/
Sorry, the article is in French but you can translate it if you need to.
Here, using "hexdump", you can see the "safe" file has a LUKS magic at the beginning. And doing a "file safe" command, you can check it detects it as a "LUKS encrypted file".
If doing "file" on your .img files does not give you the same result, you may not be able to directly use the "cryptsetup" command and need to adapt it.
Finally: usually in Android the header containing the key is stored on another partition so you may have lost it when wiping your phone, sorry.
---------- Post added at 02:44 PM ---------- Previous post was at 02:41 PM ----------
Borden Rhodes said:
Never did get a response to this question, so I'll try it again, but start with a simpler question:
If someone dds an Android (specifically Cyanogenmod 10.x) partition to an img file, is there any way to read that image from, say a Linux laptop? I dumped the contents of the /system partition using
Code:
adb shell "dd if=/dev/block/mmcblk0p1" > system.img
I expected system.img to be a normal ext4 partition. However, attempting to loopback mount it with
Code:
sudo mount -t ext4 -o loop,ro system.img ~/android/system
Gave me errors about corrupt group descriptors, bad magic numbers and other maladies indicative of a thoroughly corrupted file system. I'm assuming that:
/data has the same ext4 partition structure as /system; and
The process to mount /storage would be no different to mounting /system with the exception that the former uses vFAT as its file system
However, as my Android is currently working normally (well, as well as one can hope for Android to work), I know I don't have a corrupted file system.
So what's going on? Does Android use a special version of ext4 that other Linuxes don't recognise? Am I not dd-ing correctly? Is there a block-size issue I ignored to my peril?
Click to expand...
Click to collapse
Can you give the result of the "file system.img" command?
Thanks, saidlike, for your reply:
saidelike said:
Can you give the result of the "file sdcard.img"...
Click to expand...
Click to collapse
sdcardPartitionDump.img: data
saidelike said:
... and "file data.img" commands?
Click to expand...
Click to collapse
data.img: data
saidelike said:
Can you give the result of the "file system.img" command?
Click to expand...
Click to collapse
system.img: Linux rev 1.0 ext4 filesystem data, UUID=57f8f4bc-abf4-655f-bf67-946fc0f9f25b (needs journal recovery) (extents) (large files)
Again, attempting to run
Code:
mount -t ext4 -o loop systemimg mountpoint/
yields
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
Click to expand...
Click to collapse
Ignoring the results of data.img and sdcard.img for the time being, the fresh dump of the system partition shows that it's an EXT4 filesystem, but that it's heavily corrupted. fsck.ext4 on that partition basically asks me to fix every single inode, so it's not a simple unclean journal issue. Therefore, is it fair to conclude that CyanogenMod (and maybe AOSP too) have modified the ext4 partiiton type?
@Borden Rhodes
Maybe, my reply is too late, but you could try to make the same experiment with backup of your current data.
If you get the same results as with the old pre-wipe backup, then you still have a hope.
Hi, I'd like to ask a general question about device boot. I'd like to mount /system, /data, and /cache partitions in selected locations based on detection of sdcard in the device.
How could I add an if ... then ... else statement to init.rc before that /system, /data, and /cache partitions are mounted? Any ideas?
Solved
I've found a way to perform this task. It's possible to place if ... then ... else statement into a shell script and put the shell script into ramdisk root directory of boot.img. Then, just run the script by busybox ash command from init.rc (with busybox located into ramdisk root directory).
cristian_c said:
I've found a way to perform this task. It's possible to place if ... then ... else statement into a shell script and put the shell script into ramdisk root directory of boot.img. Then, just run the script by busybox ash command from init.rc (with busybox located into ramdisk root directory).
Click to expand...
Click to collapse
Hey... I managed to mount /dev/block/mmcblk1p2 as my /data partition by editing my boot.img ramdisk(the 3. fstab files and the .rc files... i replaced [email protected] with dev/block/mmcblk1p2 wherever i saw them)
So i was trying to use an if statement in some of the .rc files to check if /dev/block/mmcblk1p2 was mounted on /data and if not then mount /dev/block/mmcblk0p9 (my usrdata partition) on /data... Soon i realised that this isn't exactly bash...
Please explain how you managed to get a .sh to run from your .rc files( or do you only need it in the init.rc, and remove all the /data mounting lines from them and do it all from the script???)
I don't always have access to a pc so if my sdcard somehow dies i want my phone to boot using the internal /data partition otherwise i will bootloop until i flash the original boot.img
nullbyte001 said:
Hey... I managed to mount /dev/block/mmcblk1p2 as my /data partition by editing my boot.img ramdisk(the 3. fstab files and the .rc files... i replaced [email protected] with dev/block/mmcblk1p2 wherever i saw them)
So i was trying to use an if statement in some of the .rc files to check if /dev/block/mmcblk1p2 was mounted on /data and if not then mount /dev/block/mmcblk0p9 (my usrdata partition) on /data... Soon i realised that this isn't exactly bash...
Please explain how you managed to get a .sh to run from your .rc files( or do you only need it in the init.rc, and remove all the /data mounting lines from them and do it all from the script???)
I don't always have access to a pc so if my sdcard somehow dies i want my phone to boot using the internal /data partition otherwise i will bootloop until i flash the original boot.img
Click to expand...
Click to collapse
I've found on fs_property:ro.mount.fs=EXT4 in init.rc. In that section, I've added the following iine:
Code:
exec /busybox ash /mount_partitions.sh
[of course, I've placed busybox arm compiled binary (the version provided by busybox android app should work) and a mount_partitions.sh script (created by myself) into boot image ramdisk (I mean / main directory, the same where init.rc is located) ]
You could also need to give permissions to busybox and to .sh script. You could also need to remount / in read-write mode, in case of issues with the above command.
ive created a boot.img maybe someone can try and report back - in theory it should speed up /data /system and /cache partitions by the fstab flags set.
also uploaded the default boot.img from oxygen 4-7-5 (please make sure you're on 4-7-5) maybe someone can help fix this if it doesnt work.
please note this may wipe all storage / etc etc take all necessary precautions (external backup etc) until we get it working 100% as i havent tested
this work is open to anyone who'd like to use it.
Cliffs
- ive modified boot.img to try and disable verity and force encrypt - also enabling / disabling certain IO functions to improve throughput
- i need someone to go into twrp > flash the modifiedboot.img then format /data and check if it boots (i.e boots into OS if it does please report back)
-please make all necessary backups incase it doesnt work
-attached stock boot.img incase it doesnt work (please make external backup too)
here are my attempts:
Modified boot.img:
https://drive.google.com/open?id=154jrUAQ1oq7Fpjw7PgBKCJ_E6yBCxvaQ
stock boot.img
https://drive.google.com/open?id=17JU_OiEGdGm_9K1TcYlFtEkiiTJmODve
please report back if it works or not (i.e does it boot into OS or hang?)
background
my previous android phones ive created a perfect boot image whereby i edit the fstab in the boot.img
to disable force encryption and apply other android attributed in the fstab.
however i do not yet have my 1+5T as of yet so i can not try the edits as the boot.img is alittle different from my previous android phones.
The stock boot.img for the oneplus 5T (extracted from 4-7-5)
has two fstab files
the first fstab.qcom, and the second fstab_nodata.qcom
both these fstab are alittle different.
fstab.qcom looks like this:
Code:
#<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags>
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1,discard,errors=panic wait,verify
/dev/block/bootdevice/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr wait,check,forceencrypt=footer,resize
/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=panic wait,check,fileencryption=ice,resize
#/devices/soc/c0a4900.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/dev/block/zram0 none swap defaults zramsize=536870912
/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
/dev/block/bootdevice/by-name/modem /firmware vfat ro,shortname=lower,uid=1000,gid=1000,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait
/dev/block/bootdevice/by-name/bluetooth /bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait
/dev/block/bootdevice/by-name/cache /cache ext4 rw,nosuid,nodev,barrier=1 wait,check
/devices/soc/a800000.ssusb/a800000.dwc3/xhci-hcd.0.auto/usb* auto auto defaults voldmanaged=usbotg:auto
and fstab_nodata.qcom looks like this:
Code:
#<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags>
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1,discard wait,verify
#/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=panic wait,check,forceencrypt=footer,resize
tmpfs /data tmpfs defaults defaults
#/devices/soc/c0a4900.sdhci/mmc_host* /storage/sdcard1 vfat nosuid,nodev wait,voldmanaged=sdcard1:auto,encryptable=footer
/dev/block/zram0 none swap defaults zramsize=536870912
/dev/block/bootdevice/by-name/misc /misc emmc defaults defaults
/dev/block/bootdevice/by-name/modem /firmware vfat ro,shortname=lower,uid=1000,gid=1000,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait
/dev/block/bootdevice/by-name/bluetooth /bt_firmware vfat ro,shortname=lower,uid=1002,gid=3002,dmask=227,fmask=337,context=u:object_r:bt_firmware_file:s0 wait
/dev/block/bootdevice/by-name/cache /cache ext4 rw,nosuid,nodev,barrier=1 wait
/devices/soc/a800000.ssusb/a800000.dwc3/xhci-hcd.0.auto/usb* auto auto defaults voldmanaged=usbotg:auto
as you can see these two files are alittle different - previous android devices have only had one fstab file.
so my first question why are there two fstab files? - anywho ive added my modifications to both where applicable.
secondly the modified fstab id like to introduce is the following:
Code:
/dev/block/bootdevice/by-name/system /system ext4 ro,noatime,noauto_da_alloc,nodev,nodiratime,barrier=0,data=writeback,nobh wait
/dev/block/bootdevice/by-name/userdata /data f2fs nosuid,nodev,noatime,nodiratime,discard,inline_data,inline_xattr wait,check,encryptable,resize
/dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,nodiratime,barrier=0,data=writeback,noauto_da_alloc,discard,nobh wait,check,encryptable,resize
/dev/block/bootdevice/by-name/cache /cache ext4 rw,nosuid,nodev,noatime,noauto_da_alloc,nodiratime,barrier=0,data=writeback,nobh wait,check
virtyx said:
Hi guys
my previous android phones ive created a perfect boot image whereby i edit the fstab in the boot.img
to disable force encryption and apply other android attributed in the fstab.
however i do not yet have my 1+5T as of yet so i can not try the edits as the boot.img is alittle different from my previous android phones.
The stock boot.img for the oneplus 5T (extracted from 4-7-5)
has two fstab files
the first fstab.qcom, and the second fstab_nodata.qcom
both these fstab are alittle different.
fstab.qcom looks like this:
and fstab_nodata.qcom looks like this:
as you can see these two files are alittle different - previous android devices have only had one fstab file.
so my first question why are there two fstab files? - anywho ive added my modifications to both where applicable.
secondly the modified fstab id like to introduce is the following:
ive created a boot.img maybe someone can try and report back - in theory it should speed up /data /system and /cache partitions by the fstab flags set.
also uploaded the default boot.img from oxygen 4-7-5 (please make sure you're on 4-7-5) maybe someone can help fix this if it doesnt work.
please note this may wipe all storage / etc etc take all necessary precautions (external backup etc) until we get it working 100% as i havent tested
this work is open to anyone who'd like to use it.
here are my attempts:
Modified boot.img:
https://drive.google.com/open?id=1BYH4J3Du9VBRyxM1hyTQhs_hKNLgOU0W
stock boot.img
https://drive.google.com/open?id=1IpByGusbuWXKOXGKamHHEwax3aSnsvfm
please report back if it works or not (i.e does it boot into OS or hang?)
Click to expand...
Click to collapse
You also need to make the kernel compatible, without verity. Otherwise won't boot
jgcaap said:
You also need to make the kernel compatible, without verity. Otherwise won't boot
Click to expand...
Click to collapse
please read OP again as ive done this
by default force encryption on /data is enabled, and dm-verify is enabled...boot.img needs to be modified to disable. also /data will have to be formatted to flash ROMS.
with dm-verify enabled, any changes to /system will cause no boot
to modify boot.img:
Code:
open /ramdisk/fstab.qcom
change:
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait,verify
/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=continue wait,check,formattable,forceencrypt=/dev/block/bootdevice/by-name/extra
to:
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait
/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=continue wait,check,formattable,encryptable=/dev/block/bootdevice/by-name/extra
virtyx said:
If we flash the boot.Img then another kernel may it work? @jgcaap how would we get it to work?
Never needed to do that on a previous android
Click to expand...
Click to collapse
You need to compile kernel that will allow to boot without the verity and change the ram disk.
benny3 said:
by default force encryption on /data is enabled, and dm-verify is enabled...boot.img needs to be modified to disable. also /data will have to be formatted to flash ROMS.
with dm-verify enabled, any changes to /system will cause no boot
to modify boot.img:
Code:
open /ramdisk/fstab.qcom
change:
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait,verify
/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=continue wait,check,formattable,forceencrypt=/dev/block/bootdevice/by-name/extra
to:
/dev/block/bootdevice/by-name/system /system ext4 ro,barrier=1 wait
/dev/block/bootdevice/by-name/userdata /data ext4 nosuid,nodev,barrier=1,noauto_da_alloc,discard,errors=continue wait,check,formattable,encryptable=/dev/block/bootdevice/by-name/extra
Click to expand...
Click to collapse
which is why ive asked users to format /data before flashing the boot.img which should disable verity and forceencrypt
jgcaap said:
You need to compile kernel that will allow to boot without the verity and change the ram disk.
Click to expand...
Click to collapse
this is what im doing...
so if someone is brave enough to test this (as i dont have a 1+5T yet)
format /data then using mtp trasnfer the modifiedboot.img to the phone, then flash modified boot.img
and see if you can boot into OS if you can please report back.
updated the modifiedboot.img to omit discard flag - should slightly improve performance but requires a cron job to manually trim partitions, automatic trim can cause hangs upon large file deletion, plus i dont find the need to run trim on the block layer every time a file is deleted, i would rather run it daily with a cron job.
Isn't this what the disable force encryption and disable no verity zip is for? To flash over stock boot image and allow boot on a decrypted device by disabling Force encryption and no verity.?
yung40oz84 said:
Isn't this what the disable force encryption and disable no verity zip is for? To flash over stock boot image and allow boot on a decrypted device by disabling Force encryption and no verity.?
Click to expand...
Click to collapse
yes, those zips only remove verity and force encryption
this boot img includes other tweaks not found elsewhere (ie, data writeback, barrier=0, noatime, nodiratime, noautoalloc, disabling auto-trim, etc) which all theoretically should improve throughput while reducing overhead
virtyx said:
yes, those zips only remove verity and force encryption
this boot img includes other tweaks not found elsewhere (ie, data writeback, barrier=0, noatime, nodiratime, noautoalloc, disabling auto-trim, etc) which all theoretically should improve throughput while reducing overhead
Click to expand...
Click to collapse
Doesn't the auto-trim flag keeps the NAND flash healthy and prevents data corruption? ?
ground-zero said:
Doesn't the auto-trim flag keeps the NAND flash healthy and prevents data corruption? ?
Click to expand...
Click to collapse
No, doesnt prevent corruption.
we dont need the trim command to run every time a file is deleted that just causes unnecessary overhead, its better if we run the trim command daily via a cron job.
(imagine deleting files and running the trim command everytime we do so, its not needed, the OS does adequate garbage collection and the Trim command gets issued automatically from android 5 so its not needed to add it in the fstab)
for your reading
https://wiki.archlinux.org/index.php/Solid_State_Drives#Periodic_TRIM