[Q] Read raw block range with dd/cat? - Android Q&A, Help & Troubleshooting

Hi,
I used to know a way to dump a raw range of data (i.e. specifying start/end in hex address) from a block device which I used to do from data recovery days, but I can't remember what it is. I have been googling for about an hour and it's driving me nuts! Can anybody help?
FYI, I am trying to grab data from an unknown range of data on the nand layout for the Xperia Play but this is a general linux/busybox question. For details on what I'm doing check here.
Thanks so much in advance.
EDIT: Nevermind, I've discovered that the range I'm trying to read is a protected area. Mods please close if possible.

cat is more like a parser, dd is capable of dealing with raw data.
http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/
http://linux.die.net/man/1/dd

Yeah I figured it would be done with dd, though I can't find any device that represents the "entire" mtd/nand - only mtd# for existing partitions exposed by the kernel. If i could find a "root mtd" device I could use skip and count parameters of dd to read what I want.
Regardless, I don't really need help with this specifically anymore - my problem seems to be specific to the Xperia Play. I am basically trying to resize the partitions (which I did previously on the X10) and I have exposed an unknown ~100MB+ that goes between userdata and cache, but I can't read or write to it at all no matter what I do.
I think what I'm trying to get is a protected area for DRM or something which I want to shift (so I can give space from cache to userdata). I/we need to make kernel or bootloader changes for the device.
Thanks for the help anyway.

I have a Samsung and Samsung is the probably the only one brand that adopt a different partition system for mtd; but remember that dd just copies everything, free space included, if with dd you are copying a filesystem with a total of 100MB and only 40MB are in use, you end up having a 100MB image file with dd.

Yeah I know it's a raw by-sector mirror/dump tool. Well what I did was edit the kernel to only create one entire partition taking the complete nand storage and then tried to dd from that, it works for a long time then once it hits this special "protected" area around ~800MB offset it spams a lot of "I/O Error" messages but doesn't fill these with zero's or anything (using conv=noerror), then once it passes the protected area it successfully dumps the rest (which is where the cache partition for the zeus would go).

OK, I have another question now. I found that this unknown 133MB has about 53MB of data in there, somewhere in the middle, which grabs fine. But the resulting file is not 133MB so I don't know the offset. Can I use dd or another tool to grab this partition while filling I/O errors with zero's? I have googled a lot and couldn't find anything.
Nevermind *facepalm* I use conv=noerror,sync. http://www.mkssoftware.com/docs/man1/dd.1.asp

Related

Partitions - how many, where are they, etc.

Partitions are something we usually don't need to mess with - probably shouldn't mess with. That said - to understand how things work, it would be nice to peek around and look. I have enough experience with linux to be dangerous but not enough to answer my question on the gtab. I know there are multiple partititons - clockwork mod can wipe some. I see when I do nvflash that a bunch of partitions are restored. When we repartition with clockwork we set the size of only one partition to 2040 and the swap partition to 0 - what about all the others? Now, if I use terminal emulator or can get up to what I think is the root directory. In linux I could use fdisk -l and it would show me the various partitions. It doesn't produce any output here. I'm sure I just don't understand the structure so I'm not phrasing the command properly. There is also probably another place that would clue me in as to what partitions are mounted where but I don't know it. Anyone have any ideas for me?

[Q] How to test flash memory?

Hello,
How to test internal nonvolatile storage (filesystem space) accessible to running apps for being successfully writable and readable afterwards? And have some numerical results, not having to guess what is going on from behavioural events. I mean some already available app or some console command. I do not mean writing code myself.
This post is related to my other question posted here, but I decided to to separate this one for clarity.
I consider this not trivial (maybe in error) because it appears to me, that Android filesystem must be RAM cached and written back to NVram only when needed (which may in light use without power cycling not happen for months), and simple read and write to file does not reflect NVram state: in the moment of writing and reading back it occurs almost certainly to and from volatile RAM.

[Q] Sharp 003SH 005 SH root success - SIM unlock help

I live in Japan and after more than 6 months I have successfully and permanently rooted both my Sharp 003 SH Galapagos and the 005SH Galapagos (Softbank not Docomo). My next concern is how to SIM unlock. I have been reading the posts about hacking the nv_bin file. I have searched through all of the the files (Root FTP thank you!) but there was no such file. I am happy to send along any screenshots or data files if that helps.
Thanks in advance.
Search Sharp 003SH Root Success and Sharp 005SH Root success on Youtube for more info
Can't really help you. Don't know anything about it. But I would like to know how you ended up rooting this phone of ours.
Its not a file on the filesystem. The sim locking in these phones is in the radio image; which can be accessed when you use the custom build kernel thats in the latest rootkit (I assume thats what you are using).
See the 2ch root/ROM thread for more details, but basically it is done through ADB, manually backing up the "_modem" partition; stripping the spare/ECC bytes and then extracting the radio OS using QualcommDumpAnalyser
I have managed to extract this image, but no idea where to go from there. None of the other device info seems to apply to this (HTC, Samsung, LG, any other Android that has had its sim-lock discovered in the radio)
Advice i got from the guys on 2ch: "Qualcomm's NAND code is neither difficult, nor unique, so if you know what you are looking for its not hard"
003SH 005SH Sim unlock
Thanks very much for giving me a new direction. I'll get started on it right away and let you know how it progresses.
It just sucks that the guys who know how to unlock it are staying quiet, saying its "taboo"
FYI, stripping the Spare/ECC bytes can be done manually (i wrote a C program to do it), but there is an option in the RevSkills app to do it all for you - i recommend doing that.
Of course we face another issue once we find the actual unlock - recalculating the ECC bytes after making the change; the only way to access the radio is with raw data access.
P.S. hope you have warranty on your phones - this is very likely to brick at least one phone until we get it right
---------- Post added at 12:30 PM ---------- Previous post was at 12:24 PM ----------
In the spirit of open cooperation, here are the instructions i was given, translated and simplified
In ADB Shell, type su to get the # prompt, then:
cat /proc/mtd <Enter>
Confirm that you have the "_modem" partition available. If not, you need to reflash with the custom build kernel
Dump the image to file with the following command:
dump_image -r -D -F _modem /sdcard/backupimages/modem.img
Access this with anything as "raw dump" and all blocks will get read as ECC error, so definitely dont do this
ECC positioning is different to Linux, so take care
The following maps out how 512bytes of data and 10 bytes of ECC info are stored in a 528 byte block:
0000 - 01CF (0-463): Data
01D0 - 01D1 (464-465): Unused (0xff)
01D2 - 0201 (466-513): Data
0202 - 020B (514-523): ECC
020C - 020F (524-527): Unused (0xff)
Use RevSkills application to extract the data portions:
Menu⇒Calculators/Generators⇒Android MTD Nand remove Spare and ECC
Extract all of the Data only portions out of the raw dump, and then use QualcommDumpAnalyser to read it and split up the various parts. I did notice that i wasnt able to get the AMSS block out with QualcommDumpAnalyser - i copied that out manually by calculating the byte positions shown in QDA.
003SH bootloader key sequence?
Eternalardor,
I'd be happy to swap information. Perhaps you could shed some light on the question of the bootloader for the Sharp 003SH and 005SH? There seems to be no discernible key sequence (Power+home+Volume up etc.) to access the bootloader. I feel like I've tried them all. Can you tell me this critical piece of information?
Is a form of the USB Jig necessary to access it?
Looking forward to your response.
003SH SIM unlock
Dominik,
Here are the results of the original /proc/mtd (before rooting)
boot
cache
misc
recovery
ipl
system
persist
log
battlog
calllog
ldb
userdata
I don't see the _modem partition. Should I?
I have also included a screenshot of the results showing size. I have most of them backed up as .img files too.
FYI: .img backed up sizes. Perhaps this will help you to ponder where the _modem partition may have gone. Maybe it's been renamed?
boot 11,264KB
cache 3,072KB
misc 1,024KB
recovery 11,264KB
ipl 15,360KB
system 419,840KB
persist 30,720KB
ldb 45,056KB
userdata 405,120KB
There is no bootloader menu AFAIK. If you install the custom kernel, you will have the option of a quasi-recovery mode, by pressing the home button between 7-12 seconds after the Galapagos logo is seen (or was that the Softbank logo)
Anyway, looking at the screenshots, it seems you do not have the custom kernel.
How did you achieve root on your phone?
To do this, you need to use the "003sh_005sh_dm009sh-rootkit" from at least 5/27 (recommend _0614); which is available on the 2ch forums. This includes 2 possible ways of achieving root:
1. A modified standard kernel (boot image), which, when flashed gives you regular root access
2. A custom compiled kernel, which has full root, a bunch of power profiles, and heaps more features (inc that quasi recovery), as well as access to the "_modem" image.
Judging from your youtube videos, you speak some Japanese, so the Japanese menus in the rootkit shouldnt be much trouble.
http://www1.axfc.net/uploader/Si/so/142435
This is what i used.
Go here for help/instructions http://anago.2ch.net/test/read.cgi/android/1337845757/
And dont even think about typing in English on there, or you will be ignored and/or told to go away
This all looks familiar. I have been using the root kit (5/27) to get where I am now - step by blessed step. It was pretty straight forward BUT I have never seen the option to write to the system partition. It is in all the instructions but the only option I have with respect to the system partition is to back it up. I'm confused as to why it doesn't seem to show up for me. I am using a Japanese machine so all the characters are displayed and I can read the instructions but I can't find help anywhere as to why I don't have that particular (and critical) option. I can see a lot of new and cool options in the 6/14 release. I'm excited and would like to get it installed.
I'll let you know how it goes. Thanks for your help .... keep it coming!
And another thing
Could you explain a little more about "having" the custom kernel? Using the root kit, I wrote to the Recovery partition then the Boot partition then rebooted from the Recovery partition and all seemed well. As I said above, I have never been able to write to the System partition despite it appearing in all the instructions. I suspect that is what is holding me back from the latest and greatest custom kernel. Still, I am enjoying all the same functionality that everyone else seems to be enjoying in root. What am I missing?
Eep, you wrote to the boot partition before trying the recovery? Brave!
The steps should be:
Write image to recovery partition;
Then reboot to recovery partition (from the menu) and confirm it all works without errors.
Then write image to boot partition
And then turn off the phone, and reboot (the last part is only my instructions - you could just select "reboot to boot partition" from the menu)
You are doing this on your 005SH right? It should be the same for the 003SH, but i only have the 005SH. In the rootkit there is 2 options when you say "burn custom image":
1 カスタムビルドrootedカーネル(リカバリーキット機能付き)
2 S4080 標準rootedカーネル(簡易リカバリー機能付き)
Q 中止してメインメニューへ戻る
You must do the first one, the CUSTOM rooted kernel, to get any of the really cool features. The second option is only if you just want root access for a particular app or something. AFAIK the second option doesnt even disable MIYABI LSM, which prevents you from mounting the system dir as R/W
But either way, writing to the System dir is not important for what we are doing. You need the Custom kernel, which gives you access to the "_modem"
Edit, i just noticed in your screenshots above, you didnt even get root in ADB shell?
Type
ADB Shell<Enter>
Then type
su<enter>
The cursor should change to a #, this means root. You may get a prompt on the phone from Superuser asking you to give root access to "shell". Once you have this try the cat /proc/mtd again
jcroot003sh,
can you tell me how to root 003sh?
Use the link i provided in my previous post
http://forum.xda-developers.com/showpost.php?p=27989085&postcount=8
You can use a translator if you dont understand Japanese, but the general instructions are in the post above yours
I translated it for a friend, but that is at work, so wont be able to put it up until monday.
DominikB said:
Use the link i provided in my previous post
http://forum.xda-developers.com/showpost.php?p=27989085&postcount=8
You can use a translator if you dont understand Japanese, but the general instructions are in the post above yours
I translated it for a friend, but that is at work, so wont be able to put it up until monday.
Click to expand...
Click to collapse
Thank you for your replying. I will wait for your translated version. You are really a good person.
Progress
I have successfully found and dumped the "_modem" image. Exactly as you stated - forgot the "su" command in ADB. Thanks. The next problem is editing out the code. I am way above my head here so I will do some research before bugging you for a step-by-step for that.
Also, the bootloader worked. I didn't realize how to do it until I read the notes in the 6/14 release. I successfully put a previously dead phone back on it's feet EXACTLY to the point of my current phone simply by backing up and then restoring partitions through the bootloader. Very slick and easy.
Will get to work. I'll be in contact soon with my progress on the SIM unlock.
I have spent a bit of time looking at it, it certainly isnt easy (Certainly isnt a "lock=yes" section). I assume the actual locking portion is encrypted/compressed/or just compiled, because it would be too easy otherwise (be happy to be proven wrong). For starters, i cannot even find my IMEI number in the dump file... I think that this dump only includes the radio code, not the NV RAM which contains the IMEI and SIM Lock status. If that is the case then the solution should be to change the portion of the radio code that queries the NV RAM, so that it doesnt care if the SIM lock is supposed to be applied.
Extracting the spare/ECC bits out should be done with the RevSkills app; extracting the relevant portions, that is a bit of a cludge; QualcommDumpAnalyser can show the start/end positions, but doesnt extract the AMSS part (AFAIK thats where the code will be). You need to use a hex editor to cut that part out manually... And i am still not 100% sure what the block size is on this NAND.
Good luck!
And if there *are* any experienced hackers out there willing to help out, i can offer some monetary help (as will a few of my fellow Japanese smartphone owning friends) as this will be valuable for not just these 2 phones (there is an army of 007SH owners waiting on this unlock)
Shall we give the 007/009 a shot?
I can see mountains of the 007SH on the auction (mostly pink). Perhaps I should pick one up and take it for a spin. I am happy to try to do something to help out for all the help I am receiving.
Or perhaps the 009SH?
How hard would it be to crack the 007? The 009SH looks like it is supported in the latest release kit.
Thoughts?
Currently, the 003/005SH are going to be the easiest, because they have the custom kernel which allows access to the "_modem" image. To do it on the 007SH we need to build a custom kernel (compiled from the sources available on the ktai-dev site), and add the modem access code (this is in the src directory of the rootkit). Not impossible, but i dont have a Linux machine to compile the sources.
However i think that the code will be fairly universal. Once we find it on the 005SH we will know what we are looking for on the 007SH as well. That will make many people happy
Anyway, my 005SH is under warranty/anshin plan so i dont mind if it gets bricked (especially now that we can take nand backups).
First things first though - examining the 005SH modem image. Does anyone know whether the NAND is a 16kb or 128kb block size? Or is it something completely different?
P.S. The DM009SH is just the Disney Mobile version of the 003SH
Linux machine no problem
I have a Linux server running 24/7 so compiling the kernel is easy. Don't let that be the holdup. I'll keep working on the 003SH _modem image.
DominikB,
I can't open this site [anago.2ch.net/test/read.cgi/smartphone/1319287551/] on channel2 for free. This site had been moved to the past-log storehouse. So.... I even can't look at Japanese version for rooting 003sh. It is very helpful if you can show me the steps for rooting 003sh.

SHOstock3 device encryption

I've been running SHOstock3 for a few days to get comfortable with it. Tonight, I decided to encrypt the device. It rebooted, encrypted itself, then rebooted again and asked me for the password. For over half an hour now, it's been playing the SHOstock3 boot animation over and over again. The SAMSUNG screen doesn't show up between loops.
Is that normal behavior? Should I just give it more time?
The power button was able to turn it off. After restarting, it would ask for the password and do the same thing. I should point out that entering the wrong password would make it ask again, so it was working "properly". I decided it was toast and tried wiping it. However, it still asked for the password. Repeatedly entering the wrong password to force a wipe didn't work properly either. It still remembered that it had a password, but forgot what it was.
To fix it, I had to go back to stock Jelly Bean (flash stock Gingerbread then use Kies to upgrade; Gingerbread doesn't know about encryption). When the newly flashed Jelly Bean asked for a password, but as soon as I entered something, it rebooted. I presume that it wiped whatever encryption information was left because it rebooted properly.
I'm still trying to decide where to go from here. I keep work stuff on my phone, so encryption is fairly important to me.
I found this information regarding encryption on Android:
http://source.android.com/tech/encryption/android_crypto_implementation.html
It's for Honeycomb, but I'm going to assume that it hasn't changed significantly. It looks like all the encryption information is stored at the end of the /data partition. However, it's not part of the filesystem itself. If init can't mount /data, it assumes that it's encrypted and takes appropriate action.
As such, I would assume that completely erasing the entire /data partition would take care of it. Note that the /data partition needs to be erased, not just the filesystem. Based on what I've read, I think that the /data partition needs to be wiped/erased/formatted in such a way that the last 16KB of the partition is erased. After that, a new filesystem would need to be created to keep it from asking for a non-existent password.
So, does anyone know what the wipes actually do in recovery?
A couple of observations.
I don't think it is advisable to work at this level of the file system while making assumptions. In my view, you make two very questionable assumptions in your remarks.
I don't have any information on the workings of wipe and format in recovery. You can, however, work with eMMC blocks using Linux commands. For instance, if you use the dd command to make a copy of the data partition, you will get the whole partition, not just the file system. You could then use reverse engineering to see what is contained in the last 16 kb of the partition. This would require a skill set that is certainly way beyond me, and I suspect beyond you. You could also use dd to write to just the last 16 kb as well.
Well, at this point, I'm not really trying to find a "solution", I'm just trying to understand why it's so hard to wipe the phone after it's been encrypted. The only reliable method I've found is to put on the stock firmware, then repeatedly enter the wrong password until it wipes itself.
I was poking around in the jeboo github (SHOstock3 uses the jeboo kernel) to see if I could figure out what's going on. I found the following line in fstab.smdk4210:
Code:
/dev/block/mmcblk0p10 /data ext4 noatime,nosuid,nodev,discard,noauto_da_alloc,journal_async_commit,errors=panic wait,check,encryptable=/efs/metadata
I'm currently running stock 4.1.2 and I found the same file with that line. After doing some research, I found that the encryptable flag tells the system to allow encryption for that particular filesystem. Its argument says were to keep the encryption metadata. In this case, it's kept in /efs/metadata. That file exists on my encrypted stock JB system and the file happens to be exactly 16KB. The first part of the file is plain-text and it appears to be encryption related. After further research, I found that "footer" is an acceptable value for encryptable. In that case, it stores the metadata in the last 16KB of the partition (but the filesystem can't extend into it for obvious reasons).
Given the behavior I've seen, my guess is that if init sees /efs/metadata, it asks for the password. This would explain how wiping /data would cause the system to still remember the password. Even if you were to erase everything in /data, /efs/metadata would still exist. I also suspect that certain methods of "wiping" /data don't actually do so because they attempt a check before doing the wipe. I'm far from an Android expert, most standard methods of checking a filesystem in linux would fail if said filesystem were encrypted.
So, I think I've figured out why wiping an encrypted phone is so hard, but I still haven't figured out why SHOstock3 doesn't boot after it encrypts the phone.
Jebo knows a lot about the kernel. You could probably get into a meaningful discussion with him on encryption. I don't know if he has a chat channel of his own, but he is probably in Shoman94's chat channel quite a lot. You can find that in the OP of the SHOstock3 thread.

[HELP!] Recovering NANDroid backup files from raw clone of formatted internal storage

I had been flashing a rom onto my phone (RR 8.0) and customarily had taken a TWRP nandroid backup of the existing rom (RR 7.1.2). Then when I went to wipe data, cache, system, I had accidentally checked the internal storage too and wiped it as a result. At this point all I could do was flash the rom, then magisk root and busybox, then I cloned internal storage (which I identified using cat /proc/partitions to be block sda18) using ADB and Cygwin (with the netcat package) thus:
Code:
TERM-1
------
adb forward tcp:5555 tcp:5555
adb shell
su
/system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd if=/dev/block/sda18
TERM-2
------
adb forward tcp:5555 tcp:5555
nc 127.0.0.1 5555 | pv -i 0.5 > sda18.dd
Couple hours later I end up with ~23.5 GB of raw partition dump (all 24,649,728 blocks of sda18 copied, I checked). Now most file recovery software (Testdisk, PhotoRec, [email protected] products) I've tried seem to check for file signatures/extensions, naturally TWRP's backups aren't among them.
TL;DR: All I need at this point is one file in particular: data.ext4.win?? – the backup of the data partition, hope I got that extension right – by a data recovery software that can hunt those very few backup files from a cloned internal storage image.
(No, partition recovery is no good.)
Clearly these files are still out there among the sectors because all that I'd done to internal storage was a format. Having come so far I find it hard to believe there isn't software that can find TWRP backup files due to its variable filetype extension. I would assume the file signature is that of TAR files, but I'm not sure.
Recovering these backups is important to me because trying to recover files from the data partition, for example, ends up returning nondescript, unlabeled data by the thousands, no way to know which file is which (looking at you, Photorec). Restoring a backup would return the data partition exactly the way it was before it was wiped out. The rest of what used to be the contents of my internal storage are not important to me. I have over 2,000 contacts that were not backed up to any online or offline service, which most definitely resides in that backup file. Please help me with some legitimate solution or suggestion rather than to say it won't be possible – no reason to believe so when I have the cloned image of the internal storage – at the very least a data recovery software that can help me get those deleted backups out ASAP (my business relies critically on those contacts and chat app histories). Time is of the essence here, I humbly request the community's help urgently.
PS I realize this question has more to do with forensic data recovery but I'll still give it a shot in here. Hope this post isn't too vague in stating what I want done. I'm sure this would massively help others in the future too, given how common such an occurrence is.
The problem is not only did you format it but when the system was setting up it also altered it. So the data maybe gone and anything that could find it is beyond the skills found on this forum these days.
zelendel said:
The problem is not only did you format it but when the system was setting up it also altered it. So the data maybe gone and anything that could find it is beyond the skills found on this forum these days.
Click to expand...
Click to collapse
I don't deny that. But assuming data written is sequential only 30-50 MB would've been written since. And the backups were taken when the internal storage already had around 8-9 GB of other data. Which I believe should mean that there's still a chance that the backup files weren't overwritten.
Although now that you mention it I'm beginning to have my doubts, considering wear leveling patterns for NAND flash memory...
PS Photorec identifies twrp backups as tar files with wrong extension but that doesn't seem to recover anything anyway. I guess you're right :'(

Categories

Resources