Related
I read with great interest the article on XDA Memory Management. I am pretty sure I understood everything, however, there seems to be one thing missing. I understand how to get the address of the Translation Tabe Base from CP15:R1(31..14), however, it would seem to me that would be a physical address. Since I'd like to peek through the TTB, I really need to know the virtual address. In the example given, the TTB had a physical address of 0xc00b0000, and the author stated that he read the MMU tables at 0xac0b0000. When you look at the resulting tables, you can see how this works, however, this is a little bit like putting the cart before the horse, ie; *how* does one know the virtual address of the physical address that the MMU table starts at? I am trying to look at a system that is not an XDA, thus I have a different MMU map, ie; my base tabel begins at physical address 0xa0010000.
Hope someone can shed some light on this.
Cheers!
ok, now I see a way for this to work.... VirtualAlloc, VirtualCopy
Is that how the author found the mapping, or is there another way?
rcp said:
ok, now I see a way for this to work.... VirtualAlloc, VirtualCopy
Is that how the author found the mapping, or is there another way?
Click to expand...
Click to collapse
The author of that bit of information is on vacation right now, and will be back in a week or so. But maybe XDA developer entity W4XY or one of the other entities can help you as well. I'll check..
Hi all,
I've learned that the StarTrek is able to boot the current kernel from the Linwizard project without any modifications. I've posted a video of this at http://vimeo.com/2114567. Before you get too excited, however, there's a few things to keep in mind:
The buttons aren't mapped correctly by default.
You need a networked computer to log in and do anything fun, for now.
If you're wondering about Android, the answer is "not yet".
I will do my best to address the important things here; what works, what needs tweaks to work, and (most of all) what needs attention. I will also, over the next day or two, try to post my findings regarding the button GPIOs collected from HaRET, kernel modifications necessary to fix a few things. etc.
Consider this post a call for help. I'm hoping this will encourage others to contribute toward more complete linux support on the phone. I personally am driven by the opportunity to dual-boot Android on the device, but I have much to learn and can't do this alone. So, if you have kernel experience and a StarTrek, here's your motivation to get hacking! (If you'd like to help, but don't have a device to test kernels on, I'm often in #xda-devs.)
Status (Let me know if there's anything missing here.)
Internal LCD - YES - Seems to work great.
External LCD - ? - Shows the device's default image. Not sure how to control this in linux.
Backlight - ? - Unsure. As far as I can tell, Linux will retain the brightness level being used when HaRET finished loading the kernel.
USB Networking - YES - Works like a charm.
Audio - ? - No idea. I don't know how to test this.
Cellular - ? - No idea, but I don't think it will necessarily come easily.
Buttons/Keypad - SOME - By default, the keys and buttons are mapped either incorrectly or not at all. The phone keypad and some of the special buttons type random keystrokes, since we are using the mappings for the Wizard. I have a modified board-htcwizard.c with correct mappings for the StarTrek I'll post really soon, as well as a list of the GPIO readings for many of the buttons. Unfortunately, the Up/Down/Left/Right/Select/End buttons function differently than the other, matrix-based keys. I have yet to discover how to configure them.
Camera - ? - No idea, but they don't even have this working on the Wizard (Linwizard's primary target device).
Light Sensor - ? - I wouldn't know where to begin. It barely works in WM, anyway.
MicroSD - YES - I can mount the MicroSD with no issues, full read/write. Even managed to open my photos in the GPE photo gallery on the device (launched over telnet).
X[\B] - YES - I am able to start X, by calling startx over the telnet connection. However, X will try to run a touch screen calibration before doing anything else, and this is a bit of a problem for us! Fortunately, on newer Linwizard versions, you can disable this by passing tslib=0 to the kernel (in the defaults.txt file you give to HaRET). GPE also seems to be running perfectly after X starts.
Problems - After a few idle minutes in linux, the screen likes to black itself out automatically, as per the design of Linwizard. Unfortunately, on the Star Trek, there seems to be no way to make the screen wake up. Telnet still works in this situation.
I promise to keep this post updated, and get my data together and uploaded soon. Please share your own results and tricks as they come along, and join me (and some talented people) in #xda-xevs and #htc-linux on irc.freenode.net. Happy hacking!
Attached:
arch/arm/mach-omap1/board-htcwizard.c - Contains key matrix mappings modified for StarTrek. Be sure to remove the ".txt" from the file extension before using.
default.txt - Used with HaRET to make Linux boot. The only thing unusual in here is the TSLIB=0 option that's passed to the kernel, telling linux to skip touchscreen-related tasks.
great work! but you need some advanced startrek owners to test linux - that's a big problem
Fantastic news! I' would really try it out. Unfortunatly my startrek's connector is damaged. I can charge the device, but I cannot connect with the pc. If I can use the micro sd card to install it(I think is possible) I will try it.
Thanks for the great job.
dancer_69 said:
Fantastic news! I' would really try it out. Unfortunatly my startrek's connector is damaged. I can charge the device, but I cannot connect with the pc. If I can use the micro sd card to install it(I think is possible) I will try it.
Thanks for the great job.
Click to expand...
Click to collapse
You can, by dropping Haret+default.txt+zImage+initrd onto the SD card. Unfortunately, without a USB connection, you won't be able to telnet into the device and run commands.
very interesting! Ive been waiting to hear this kind of news Unfortunately, I still use my StarTrek as my primary phone, so I can't test it out until it moves along a little further. I'll be sure to keep an eye on this thread, as this is good news!
gigawatts said:
very interesting! Ive been waiting to hear this kind of news Unfortunately, I still use my StarTrek as my primary phone, so I can't test it out until it moves along a little further. I'll be sure to keep an eye on this thread, as this is good news!
Click to expand...
Click to collapse
Linux actually boots directly from a Windows Mobile program called HaRET; When you're done playing with Linux, you can just reset the phone and you're back to normal. No permanent changes.
awesome! I'll give it a go right now, thanks!
BHSPitMonkey said:
You can, by dropping Haret+default.txt+zImage+initrd onto the SD card. Unfortunately, without a USB connection, you won't be able to telnet into the device and run commands.
Click to expand...
Click to collapse
I already did it, and works. Thanks anyway.
Why not combine the effort to get Android on it?
Kwen said:
Why not combine the effort to get Android on it?
Click to expand...
Click to collapse
Are you volunteering your expertise on the subject?
The point of this post was to accomplish just that: to attract the interest of more people who can contribute. I'm in the process of setting up a repository of changes right now, so the patches I've made so far are kind of scattered about my hard drive for the moment. If you're interested, you can try catch me in #htc-linux or #xda-devs on irc.freenode.com. Otherwise, I'm trying to keep the post at the top of this thread up-to-date.
Unfortunatly Android cant work, it needs 128 ram.
http://source.android.com/release-features
BHSPitMonkey said:
Cellular - ? - No idea, but I don't think it will necessarily come easily.
Click to expand...
Click to collapse
from linwizard wiki page :
What we need
Right now we need people with experience with GSM and WiFI to help get linwizard connectivity up!
Click to expand...
Click to collapse
about two years ago , another linux project ran that called Xanadux. why you don't use from Xanadux ?
sorry for my english !
I have linux booting
it up booting to the point correctly then sit conclusion cannot one yet start?
sorry for my bad english my german are better
Oh.......I think HTC Phone can do that ADIDAS ' AD "impossible is nothing"
Been a few months since ive been on the scene, any news here? My Cingular 3125 is nearing the end of its life, and I figured I can play a bit more with it
P.S. Whats new BHSP?
autologin on linwizard
hi
http://wlcrck.wl.funpic.org/wlcrck-elf-linux/autologin
to solve the problem i develop a simple aplication to automatic login but dont have any security autologin in root mode
to use this have to make a couple of things
you need decompress the inirtd and add this app to /bin and set suid and modify
/etc/inittab file and compress the initrd again.
change this line
tty1::respawn:/sbin/getty 38400 tty1
for
tty1::respawn:/sbin/getty -l /bin/autlogin 38400 tty1
sorry for my english
thanks for the code of htc board
I've been hankering to start porting a game I've been working on(originally was going to be for Zune HD but they haven't released the damn 3D SDK. Video of it running in windows is available on youtube, where my username is marcusmaximus04) to android but was hesitant due to the lack of OpenGL ES2.0 support in the Android SDK. However, looking through the file system on my nexus one, I spotted the file libGLESv2.so, which appears to be the shared object file for exactly this!
Obviously, I still can't use the normal SDK to use this, but in theory, one could use the NDK to use OpenGL ES 2.0 features(shaders, etc) through that shared object file.
The purpose of this post is three-fold. One I want to know if anyone has any background with using OpenGL through the NDK or knows of any tutorials for doing so(I'm not entirely sure how that would work...). Two, I wanted to bring this to the attention of all the devs out there who might be holding off from using ES 2.0 features. And three, I wanted to let everyone else know that such a thing is possible, just 'cause it's exciting.
At any rate, even if I can't find any tutorials I'll probably start plugging away at getting this to work and this should make for an exciting couple weeks/months.
Since we have this pretty well working(even on non-rooted phones) I'll post a sample apk. It doesn't look like much, but it does prove that we're using OpenGL ES 2.0(this will only work on the Nexus One). I'll also post the libGLESv2_adreno.so shared library in the same zip file. Enjoy .
From http://developer.android.com/guide/topics/graphics/opengl.html:
note that though Android does include some basic support for OpenGL ES 1.1, the support is not complete, and should not be relied upon at this time.
Click to expand...
Click to collapse
so official 2.0 support might be a bit off :/
kozm0naut said:
From http://developer.android.com/guide/topics/graphics/opengl.html:
so official 2.0 support might be a bit off :/
Click to expand...
Click to collapse
Possibly, but it does still support it in the NDK(really, this hardcore graphics stuff should be done in the NDK anyway). Even there it's not really official, but they aren't likely to change the API calls since those are already set in stone by khronos.
Hi Marcus,
to get started, grab the OpenGL ES 2.0 headers from www DOT khronos.org/registry/gles/ and copy the library file from your phone to the NDK installation dir (build/platforms/android-4/arch-arm/usr/lib/). Next you might inspect the libGLESv2.so with objdump to see what functions it provides:
build/prebuilt/linux-x86/arm-eabi-4.2.1/bin/arm-eabi-objdump -t libGLESv2.so
It should have the 2.0 functions like glCompileShader(), glCreateProgram(), etc.
Finally, add the following line to the Android.mk file in the jni/ directory of your Android project:
LOCAL_LDLIBS += -Lbuild/platforms/android-4/arch-arm/usr/lib -lGLESv2
If you're lucky, that should allow you to compile and link OpenGL ES 2.0 code. I cannot try this right now but will as soon as I get my Nexus One on monday
I haven't done any OpenGL ES coding with the NDK, but it should be straightforward if you've written OpenGL (ES) code before. This might be of use: code DOT google.com/p/glesquake/
robert-qfh said:
Hi Marcus,
to get started, grab the OpenGL ES 2.0 headers from www DOT khronos.org/registry/gles/ and copy the library file from your phone to the NDK installation dir (build/platforms/android-4/arch-arm/usr/lib/). Next you might inspect the libGLESv2.so with objdump to see what functions it provides:
build/prebuilt/linux-x86/arm-eabi-4.2.1/bin/arm-eabi-objdump -t libGLESv2.so
It should have the 2.0 functions like glCompileShader(), glCreateProgram(), etc.
Finally, add the following line to the Android.mk file in the jni/ directory of your Android project:
LOCAL_LDLIBS += -Lbuild/platforms/android-4/arch-arm/usr/lib -lGLESv2
If you're lucky, that should allow you to compile and link OpenGL ES 2.0 code. I cannot try this right now but will as soon as I get my Nexus One on monday
I haven't done any OpenGL ES coding with the NDK, but it should be straightforward if you've written OpenGL (ES) code before. This might be of use: code DOT google.com/p/glesquake/
Click to expand...
Click to collapse
Alright! Thanks for the help. I've done a lot of opengl work before and a little bit of opengl es when i had my g1 but haven't ever used the ndk, so that'll get me started.
EDIT: Sweet, I've dumped the list of functions available and it looks like all the appropriate shader functions are there. I'm attaching the full list output from my computer. I actually had to use arm-eabi-strings, since the objs one showed no objects. Regardless, it's looking good on the OpenGL ES2.0 front, and I've already looked into running opengl code from the NDK which is perfectly possible and ok.
ANOTHER EDIT: Haven't quite gotten it to build yet, but I got it to load gl2.h without any complaints. Now I just have a crapload of errors because the sample project I was using(from the ndk samples, san-angeles) was written for OpenGL ES 1.0 and all the things now done in shaders no longer exist(GL_LIGHTING, GL_MATERIAL, etc). Regardless, I'm considering this a success so far, since all that is expected.
YET ANOTHER EDIT: It runs! I'm not displaying anything yet(so it's not really a good test), but I got the shared library to build using OpenGL ES 2.0 and it runs on my phone. Gonna implement a simple program to display a triangle or something just to prove it works and will provide source code/binaries here(also, if anyone has a Droid or knows someone who does, it'd be great if we could test out to make sure this works on the Droid as well)
I just wanted to drop in and say I'm very interested in what y'all are doing I'm of very good at coding. All I know is Java but ill definantly be following yall and keep up the good work
Decided to post a reply, rather than just adding to that one post.
Well, I encountered a fairly large roadblock:
Code:
E/libEGL ( 3366): called unimplemented OpenGL ES API
That's what I see now every time I call one of the functions dealing with ES 2.0. I'm hoping there's another library I can use that'll make that go away, but we might not be as easy as I thought to do this.
That's bad news indeed. I was hoping that everything is in place and Google just didn't release a new NDK with ES 2.0 support yet, but if the libGLESv20.so is merely a stub, we're pretty much out of luck. I'd probably disassemble the file to see if there's actually some code besides "return unimplemented" in there, but even if there was, I'd have no idea on how to continue...
I'm wondering how they render the fancy rain drop effect on the wallpaper. Looks like a perfect fit for a 2.0 fragment shader, however there are a lot of other ways to create that effect.
robert-qfh said:
That's bad news indeed. I was hoping that everything is in place and Google just didn't release a new NDK with ES 2.0 support yet, but if the libGLESv20.so is merely a stub, we're pretty much out of luck. I'd probably disassemble the file to see if there's actually some code besides "return unimplemented" in there, but even if there was, I'd have no idea on how to continue...
I'm wondering how they render the fancy rain drop effect on the wallpaper. Looks like a perfect fit for a 2.0 fragment shader, however there are a lot of other ways to create that effect.
Click to expand...
Click to collapse
Well I managed to view the assembly code, but can't get it into any form that I can/want to read(I know how to read assembly but unless you spend hours with it, it's damn near impossible to decode anything. Plus, I haven't looked at ARM assembly before). Know of a good program to convert it into C?
At any rate, I'll attach the disassembler dump(ran it using arm-eabi-objdump with flags -DS). From looking at it briefly, all the implemented functions look like they might be stubs since they're pretty bare bones and have almost all the same instructions(with a slight modification to one single instruction). But it's hard to tell with assembly. Let me know if you can make anything of it. I had to attach the file in a .zip by the way because it was too big.
SUCCESS!!!! So, it looks like those functions were stubs. Google seems to have done it intentionally for whatever reason. They took the .so that actually worked(libGLESv2_adreno200.so) and shoved it under /system/lib/egl, which the ndk doesn't seem able to access(gives you a runtime error if you linked against it). However, If you just copy that file to /system/lib (yes, this requires root unfortunately) then you can use it. I successfully drew a triangle using shaders .
I'll try to get it to attach that xxx_adreno200.so file to the .apk so it can be used without messing around with such things(and work without root). If I can, then I'll throw it on here so everyone can see. Right now it's just a triangle, but tomorrow: the world.
EDIT: Alright, I attached the .apk(it's in the zip file, xda still doesn't allow posting .apks). I tried to attach the libGLESv2_adreno200.so library to it, so it wouldn't require moving the local one but it just wouldn't accept it. So, at least for the mean time, if you want to actually run that app without having it crash you'll have to do an adb remount and then the following in the terminal(or adb shell) as root:
Code:
cd /system/lib/egl
cp libGLESv2_adreno200.so ../
Once you do that, when you run the app it should display a yellowish triangle on a blue background(it was from the second tutorial from the OpenGL ES 2.0 sdk for PowerVX). Like I said, it's not much right now, but it is using custom shaders to display that triangle.
ANOTHER EDIT: I decided to change the test program a little. All I changed was the fragment and vertex shaders, which I made to change the color of the pixels based on where they are on the triangle. This should show(to those of you who know OpenGL) that it really is using custom shaders, since it's one triangle and it's clearly not using fixed-function pipeline shading .
Awesome work on reverse-engineering this! Keep us posted.
Indeed, keep at it Marcus! Very interested to see how this progresses.
Well done, Marcus! I'm glad you found that working library.
You should be able to embed the library search path in your own library by using the rpath directive while linking your program. See "man ld" for more information. You need to get the NDK to call the compiler like this:
gcc ... -Wl,-rpath=/system/lib/egl
That should allow you to use that library without copying it around, and therefore make it work on non-root phones as well.
I guess all you need to do is add the directive to the LOCAL_LDLIBS variable in Android.mk, or maybe it has LOCAL_LDFLAGS, which would be more appropriate.
robert-qfh said:
Well done, Marcus! I'm glad you found that working library.
You should be able to embed the library search path in your own library by using the rpath directive while linking your program. See "man ld" for more information. You need to get the NDK to call the compiler like this:
gcc ... -Wl,-rpath=/system/lib/egl
That should allow you to use that library without copying it around, and therefore make it work on non-root phones as well.
I guess all you need to do is add the directive to the LOCAL_LDLIBS variable in Android.mk, or maybe it has LOCAL_LDFLAGS, which would be more appropriate.
Click to expand...
Click to collapse
Ya, I tried adding it to LOCAL_LDLIBS, but that just seemed to make it use that path for the local library on my computer when building it. It still failed to find that library on my phone when I ran it. I'll try poking around a little more since that would obviously be ideal.
Alright. Are you sure you used rpath and not rpath-link?
BTW, there's a libGLESv2.so in the 2.1 emulator image. Did you try using that? I wonder if the emulator supports ES 2.0...
robert-qfh said:
Alright. Are you sure you used rpath and not rpath-link?
BTW, there's a libGLESv2.so in the 2.1 emulator image. Did you try using that? I wonder if the emulator supports ES 2.0...
Click to expand...
Click to collapse
I didn't touch either actually, just changed the name of the library and the path it had. I haven't done a lot of make files beyond the most basic operations so a little study might be necessary.
Haven't tried the emulator image one, but the one from the phone just had those stubs. It would be really strange if the emulator had the actual functionality and the phone didn't.
I think you're right that if there's hope then it lies in the makefile, so I'll keep looking into that.
EDIT: Just tried adding --rpath=/system/lib/egl everywhere I could possibly think of and all with no change.
Hmm I can't try this yet, but it looks like it uses the property java.library.path to determine where to look for shared libraries. Maybe I can have it look in /system/lib/egl too?
Ack, tried it, didn't do anything. Still can't find that library.
So ya, I'm giving up on this front for now. Google obviously has some of this ready but doesn't want us using it for now, so I'll just wait for them to give us access unless I get some new info. In the meantime, I'll continue the work I've already started on porting that game from the video I linked to. At least we now have the tools to test this stuff.
[email protected] said:
Hmm I can't try this yet, but it looks like it uses the property java.library.path to determine where to look for shared libraries. Maybe I can have it look in /system/lib/egl too?
Ack, tried it, didn't do anything. Still can't find that library.
So ya, I'm giving up on this front for now. Google obviously has some of this ready but doesn't want us using it for now, so I'll just wait for them to give us access unless I get some new info. In the meantime, I'll continue the work I've already started on porting that game from the video I linked to. At least we now have the tools to test this stuff.
Click to expand...
Click to collapse
thats depressing i was checking this thread like 10 times a day waiting for new info to popup lol call me obsessed but i respect your work
Yet another idea to try. On Linux you can specify the library search path using the environment variable LD_LIBRARY_PATH, e.g. if you run a program like this
LD_LIBRARY_PATH=/some/path /path/to/program
then the linker will search that path for libraries.
Maybe it's possible to set that variable from within the Java code, but before the JNI library is loaded:
static {
// Set environment variable LD_LIBRARY_PATH
System.loadLibrary("your_jni_lib");
}
Finally another idea, so we cannot link our library to the OpenGL ES library. But we could probably dlopen() it from within the JNI library and grab all the function pointers using dlsym(). I will get my Nexus One in a few hours, and ES 2.0 will be one of the first things I'm going to look into
EDIT: Marcus, would you mind posting the source code of your ES 2.0 triangle demo application? Basically all I need is the gl....() stuff. I haven't done too much Shader stuff so far, so that would save me a lot of time.
robert-qfh said:
Yet another idea to try. On Linux you can specify the library search path using the environment variable LD_LIBRARY_PATH, e.g. if you run a program like this
LD_LIBRARY_PATH=/some/path /path/to/program
then the linker will search that path for libraries.
Maybe it's possible to set that variable from within the Java code, but before the JNI library is loaded:
static {
// Set environment variable LD_LIBRARY_PATH
System.loadLibrary("your_jni_lib");
}
Finally another idea, so we cannot link our library to the OpenGL ES library. But we could probably dlopen() it from within the JNI library and grab all the function pointers using dlsym(). I will get my Nexus One in a few hours, and ES 2.0 will be one of the first things I'm going to look into
EDIT: Marcus, would you mind posting the source code of your ES 2.0 triangle demo application? Basically all I need is the gl....() stuff. I haven't done too much Shader stuff so far, so that would save me a lot of time.
Click to expand...
Click to collapse
Sure. I'll post it when I get home from work tonight. I don't have anything proprietary in there yet.
For the setting of the environment variable, from my understanding that's basically what System.SetProperty("java.library.path"... is supposed to do. But it still couldn't find the library and didn't seem to have any effect.
The dlopen bit might work. I was thinking of trying that but haven't had any time. I'm not really too sure on how these shared libraries are used, whether once they're open any other program can find and use them or whether the path to them has to be given explicitly at link time for the program that wants to use them and just generally having one loaded won't help.
EDIT: By the way, @pr0cl1v1ty: Don't worry, I'm not going to stop digging into ES 2.0 entirely. I'm just not going to be pushing as much for getting it to work on non-rooted phones. I've got bigger fish to fry.
Hey guys,
I've done some dev on *NIX before, including writing ways to sign my own packages using encrypted hashes and the like. Does anyone know the method that HTC is using to sign the zip files?
The reason that I ask is because I'm interested in trying my hand at reverse-engineering the signature. I am sure some of you guys have already done some work in that area, and I'd rather not repeat someone else's effort if y'all have already taken steps to break the signature. My CSI teacher told me to never start from scratch if someone else has already done good work. It's insulting to them, and makes more work for you.
Where are you folks at with breaking the signature? Is the method known (i.e., is it based on files inside the zip, is based on the bits of the zip, is an additional hash or added metadata, etc)? I would really appreciate any feedback on this if you have the time.
EDIT: For those of you who are leakers or users et. al. DO NOT get any hopes up about this thread. I'm just getting started and this idea could fizzle within minutes of you reading this particular sentence. Anyone posting, please focus on practical suggestions or comments such as sickbox's initial comment below. Thank you!
I've wondered about this since the beginning.
I understand just how complex signing can be (to some degree, I'm not a math guy but I understand scale).
My thought though is we can utilize several tools to make this process possible - though I have no idea how to implement most of this to make this possible. Maybe I'm nuts, but here goes...?
- We now have what, three or four different HTC signed images in the wild with another on the way (OTA). Would it be possible when trying to reverse the sig to utlize the differences between the packages to narrow the cope a bit?
And next
- Using GPUs to process data like this has been shown to be exponentially more efficient and effective than CPUs. What would it take to use some of our awesome GPU power ( a la CUDA) to attempt this task?
Lastly:
- Can we break up the processing required among several of us to speed things further?
I know this has probably been thought of before and discarded for good reason, but I guess the more ideas the merrier.
I'm no CS guy, but I would love to help! I'm one of those unfortunate leakers but rather than whine I'm looking for ways to help. Reversing the HTC key would make life sooo easy. Who knows, maybe they'll use the same key on the next few phones?
Pretty much why I'm asking NOW is because I have enough packages for me to examine and compare and test against. I'm not the best or the most experienced at it, but this kind of thing is fun for me and fits into my spare time. When I have spare time.
It's not the signing we need to know how to do its the cryptographic key that they use to sign their packages that we need. The private key changed with the last bootloader so even if we cracked the key before the couple hundred years it would have taken us to crack the one used for 1.5 we would have to do it again now for 2.1 stuff.
Just look around for test signing and such and you can find the test key that people use to sign stuff as well as the method used to sign the package.
As far as getting the key... you will have to know someone from HTC who would risk their job to get you a copy of their private key.
Greetings Sickbox,
I guess my intention isn't clear. I want to be able to sign packages regardless of what key HTC uses. We have a signature, and we have keyhole. I've noted that the behavior on my Eris is that the signed packages check out just fine each time no matter what version I'm trying to flash (obviously, cannot downgrade, I know, but trying to downgrade still passes the signature and it is the version that fails). So what I would like to do is reverse engineer the signature not necessarily to find the key, but to discover how to create keys. I have 4 different packages, and two test keys that I can examine.
I'm only wanting to know if someone knows how the packages are signed so that I can eliminate looking at all the signing methods. In my research so far, I haven't been able to google, bing, or yahoo anyone who knows what method is used to sign the HTC official packages.
That help, Sickbox? Thanks for your input, I really appreciate it.
So the intent is to reverse engineer the key correct? Then we can sign whatever we want...
Or are you trying something else?
Just want to see if we're on the same page.
1234567ten
I don't necessarily want to reverse engineer the specific key that HTC used to sign any one package, but rather the template for the keys. A prime example of this kind of key decryption would be DeCSS written by DVDJon. He quit trying to reverse engineer the keys used to encrypt DVDs and reverse engineered the decryption of DVD signatures.
I'm not using technical terms for the following, but basically when you sign or encrypt something, the key used is not found within the package or signature, nor is it in the program used to verify the signature or decrypt the package.
If I can do nothing with the signatures of the Eris roms, it's no waste to me. I have fun with this because I want to design an open source DRM system someday. *Sigh* dreams.
Try these. I'm still not sure if I fully understand your question but this as much as I could come up with.
Found by searching "android signing" & "android sign rom" on google if you wanna see what else might come up.
http://developer.android.com/guide/publishing/app-signing.html
http://androidforums.com/developer-101/8665-how-signing-roms.html
sickbox said:
Try these. I'm still not sure if I fully understand your question but this as much as I could come up with.
Found by searching "android signing" & "android sign rom" on google if you wanna see what else might come up.
developer.android.com/guide/publishing/app-signing.html
androidforums.com/developer-101/8665-how-signing-roms.html
Click to expand...
Click to collapse
Hmm... maybe I was being too specific when looking for "htc sign rom" and "eris htc sign rom," etc. I'll see what I can cull from those broader searches. Thanks for the tip, sickbox.
np
34567ten
You might have noticed that there is a little bit of confusion in the posts here when "signing" is brought up; there are two completely different signing methods in use.
The first applies to applications (.apk bundles), "update.zip" files (which could be used with Amon_RA's recovery), and OTA-delivered update files. The distinguishing feature of these .zip files are: 1. They have a META-INF folder in them with two Manifest files and a RSA public key file, and 2. there is nothing "unusual" about the zip file itself. (The contents of the zip file are signed, but the whole zip file is not.)
The second type is the "rom.zip" files buried inside the MR1/MR2 " RUU" updates. These files, when renamed to PB00IMG.ZIP, can be used with the bootloader to update the phone. The distinguishing feature of this type of file is that: it has a mystery blob of binary data at the front of the zip file - 256 bytes. The rest of the file is an ordinary .zip file, and if you unpack it you will find that there are no manifests, no META-INF file, and no public key certs. (In this case, the entire zip is signed, but none of the individual content files are.) I think it is this second type of signing you were referring to in your posts, but honestly I am not certain.
The first form of signing is perfomed with a java tool called "jarsigner", and its behavior is well understood: it creates the first manifest file by computing SHA-1 hashes for every file to be included in the .zip archive. Then, it creates a second manifest file which shadows the first one, and for each SHA-1 hash value, it "signs" them using the signer's private key. In this 2nd file, it also computes the hash for the complete (1st) manifest file, and signs that hash. In any event, what I mean by " well understood" is that this is just a standard use of RSA public key cryptography, using widely deployed Sun Java tools. Break it and you will have made quite a name for yourself.
Now, as for the 2nd type of file - rom.zip/PB00IMG.ZIP, I have not seen anyone (yet) describe the format of that MIC (Message Integrity Check) 256-byte blob. I poked at it a little, but certainly not exhaustively.
If you want to add to the knowledge here, try and discover what the "format" of that MIC is. I suspect that even if you do that, you will find that the sig uses exactly the same PK tools that are already built in to the bootloader - from the standpoint of practicality, it really doesn't make any sense why HTC would " roll their own" when they already went to the effort of coding RSA tools into their botloader(s).
bftb0
Hey bftb0,
You answered my question PERFECTLY. Nobody I've spoken with elsewhere has yet brought up the RSA encryption that's already built into it. You're probably only second guy to mention it, beyond some dude in an IRC somewhere (and I think he was drunk at the time).
Knowing that it is just additional bits on the zip, has anyone thought off hacking it off and paring it to another zip in an attempt to "sign" the zip (I've done this successfully with various signed ISOs)? Also, the public key could be arrived at, given two factors, 1. The same key was used for all Eris 2.1 packages; and 2. The "blobs" of data can be sufficiently compared and I have enough computing power.
Thank God I may be getting an intel I7.
Or I'll just borrow my friend's PC.
I hope I'm not just blowing steam, because it would suck to get working on this and then find it's impossible. But they say that about a lot of key encryption schemes. LIKE RSA on Blu-Ray.
Thank you so much bftb0
Don't read too much "encouragement" into my post; I responded in order to shed some light on the way that HTC is doing things, and that's about all.
If you think about it carefully, you will understand that the manifest-signing operation gives you hundreds, if not thousands, of individual plaintext/crypt-text pairs that are all signed with the same private key. That doesn't mean that a known-plaintext attack is easy, though.
The EFF commissioned a project a couple of years back where they built custom hardware that would brute-force key searches for short keys- 256 bit keys IIRC. The machine they built was a parallel processor built from fpgas/DSPs, and it could recover keys in a few days. Their budget for that was in the low 100,000s. Offhand, I don't know what key length HTC is using, but I doubt it is 256 bits.
I don't recommend you spend any cycles trying to brute force a key recovery.
bftb0
bftb0 said:
Don't read too much "encouragement" into my post; I responded in order to shed some light on the way that HTC is doing things, and that's about all.
I don't recommend you spend any cycles trying to brute force a key recovery.
Click to expand...
Click to collapse
Dude, I was so encouraged that I want to rip open my PS3 and put it to work RIGHT NOW.
Not really. I'm too lazy-assed to spend much time brute forcing it. I'd rather pick it apart and see if there's anyway to mimic the signature. Your advice that it may be RSA based is more exciting in that it helps me know what I may be dealing with. I hope to pick at the binary data appended to the signed roms either tomorrow or next weekend.
And thats what I appreciate.
I'm not sure what you're trying to do. You either brute force the private key, using various bits of super-math (e.g. elliptic curve cryptography?), or you give up and move on -- perhaps looking at patching the subsystem responsible for validating signatures (dangerous for production use).
There are no "mimicing" possibilities and swapping blobs/zips around is just silly. You should spend your time elsewhere, like reading up on how public-key cryptography works.
Thanks WithinRafael,
I think some of what I've written above shows I'm researching public-key cryptography. I really appreciate your thoughts, and it became clear by the end of sunday that the signature is specific to the package. Without me doing much work, mostly research. RSA is a load of work and I do not want to mess with trying to crack that.
I recently became interested in trying to turn S-off. Someone recently gained RW access to NVRAM, and I'm hoping this weekend to move on as you mentioned. Thanks for the good discussion, guys!
with a pen....duh j/k
Anyone have a supercomputer? ...lets brute force it.
Is there a way to check if a rom passes the signature test without trying to load into the phone? Can we check if the signature passes on a computer?
If so we could sign it with all possible keys and see which one passes.
Is this frowned upon and shouldn't even be discussed? or would it just take too long to do? ... or is it just not possible to check the signature on a computer?
... or all of the above?
DarthMowzy said:
Anyone have a supercomputer? ...lets brute force it.
Is there a way to check if a rom passes the signature test without trying to load into the phone? Can we check if the signature passes on a computer?
If so we could sign it with all possible keys and see which one passes.
Is this frowned upon and shouldn't even be discussed? or would it just take too long to do? ... or is it just not possible to check the signature on a computer?
... or all of the above?
Click to expand...
Click to collapse
We can check the signatures based on what is stored in the Manifest file inside the PB00IMG.zip file.
It is possible to brute-force it but it would take years to do so it isn't really worth the effort.
Here I am providing a link for a modified version of the 20110721 rootfs.img. But before anything else, allow me to explain exactly what was modified and why. And also, I'm running FRX07 with the default kernel it came with and the newest rootfs, OC'd to 614.4MHz (although OC doesn't play much factor in this tweak)
During the portion of the boot cycle when it loads the rootfs, at some point the read_ahead_kb (cache on the SD card) is set to 2,048KB as default with the newest rootfs. For higher-end SD cards, this is a good, wholesome number to work with..
With me, I'm running on a 2GB class 2, and using the program SD Booster (free), I was able to switch the read_ahead_kb while running Android. I tried multiple values and noticed the best results with 256KB.
Before with the 2048KB setting, the boot animation would cease a few times for probably a good 6-7 seconds, was terribly laggy throughout, and ultimately, from my experience, the performance of Android was impacted negatively. The system would literally halt while performing basic operations such as opening the Settings menu... although not all the time, still pretty annoying :\
By making this change in the rootfs.img, I noticed that the boot animation was MUCH MUCH smoother. I did not get the aforementioned problem at all using this rootfs.img with my SD card. And I also noticed a nice decrease in boot time, PLUS a pretty hefty boost in performance and stability after the boot even more than when manually changing the value with SD Booster!
I know not all people are running on premium SD cards and that is pretty much the whole point of this thread: to provide people who DON'T have the best with something that could help with their performance.
And for the record, I'm no dev. I just know some bits and pieces and like to share my experiences.
Please post feedback if it helps you!! It makes me feel better lol
ALSO: Before I even thought to post this, I made sure I uninstalled SD Booster to make sure it didn't apply any settings on boot, rebooted the phone into Android, reinstalled SD Booster, and noticed that it was no longer set to 2,048KB as default doesn't hurt to double-check!!
Since you're talking about a 1-line change, it would seem to me to have been far more efficient just to post the single line you changed, rather than make people download a multi-megabyte rootfs img file all over again.
highlandsun said:
Since you're talking about a 1-line change, it would seem to me to have been far more efficient just to post the single line you changed, rather than make people download a multi-megabyte rootfs img file all over again.
Click to expand...
Click to collapse
Good point.. although most people like swapping rootfs.img's and making things easy, plus I already had the rootfs ready for myself.. I figured why not.
But you're right, it would be helpful to know either way, so this is exactly what I did. I opened the init file within the rootfs.img (after mounting in Ubuntu) and searched for "read_ahead_kb" and found it. At the front of the line read: "echo 2048". That's the command that enables the 2,048KB SD cache during boot (the rest of the line I assume is to apply that setting to the SD card? Not sure..). I changed that value to 256KB, making that: "echo 256". Saved the file, unmounted the rootfs, and voila.
maff1989 said:
Good point.. although most people like swapping rootfs.img's and making things easy, plus I already had the rootfs ready for myself.. I figured why not.
But you're right, it would be helpful to know either way, so this is exactly what I did. I opened the init file within the rootfs.img (after mounting in Ubuntu) and searched for "read_ahead_kb" and found it. At the front of the line read: "echo 2048". That's the command that enables the 2,048KB SD cache during boot (the rest of the line I assume is to apply that setting to the SD card? Not sure..). I changed that value to 256KB, making that: "echo 256". Saved the file, unmounted the rootfs, and voila.
Click to expand...
Click to collapse
Interesting. I don't think my card would be considered 'premium', but it is a class4 card.
I'll see if this change makes any difference. IIRC this value was bumped in the newest build to provide better performance, not worse! I guess that's the tradeoff - better performance for some, worse for others... oh well. Thanks for your work!
arrrghhh said:
Interesting. I don't think my card would be considered 'premium', but it is a class4 card.
I'll see if this change makes any difference. IIRC this value was bumped in the newest build to provide better performance, not worse! I guess that's the tradeoff - better performance for some, worse for others... oh well. Thanks for your work!
Click to expand...
Click to collapse
You're very welcome I know the 256kb setting may not work better for all.. but I think this setting is key to how well Android performs on different cards.
Also, I DID notice that OC'ing is much more stable using this setting for my SD.. currently running at 768MHz and minimal instabilities.
Sent from my MSM using XDA App
I appreciate you posting the process you used to figure out optimal settings. I'm toggling between sd tools and sd booster as we speak. I'm still trying different settings. If my settings are different, that'll give me an excuse to figure out that virtualbox build I started on a few months ago.
EDIT: hmmm, I'm not sure sd tools is going to be a useful bench tool. 2 different runs on the same settings vary pretty wildly. I think the meter animations might be interfering with the testing. How did you decide which setting performed best while using sd booster? Were you just going with what felt best?
fortunz said:
I appreciate you posting the process you used to figure out optimal settings. I'm toggling between sd tools and sd booster as we speak. I'm still trying different settings. If my settings are different, that'll give me an excuse to figure out that virtualbox build I started on a few months ago.
EDIT: hmmm, I'm not sure sd tools is going to be a useful bench tool. 2 different runs on the same settings vary pretty wildly. I think the meter animations might be interfering with the testing. How did you decide which setting performed best while using sd booster? Were you just going with what felt best?
Click to expand...
Click to collapse
Personally, I never benchmarked my results.. I would change the kb setting in SD Booster to different values and resume normal operation of the phone (opening programs, checking for any abnormal "halts" during opening/closing animations and general program operation).
Basically, you're right; I found the best setting for my card based on trial-and-error and what felt right, then applied that setting in the rootfs to further increase performance over applying this setting AFTER boot.
Whichever setting feels right to you will most likely yield the best results.
Sent from my MSM using XDA App
maff1989 said:
Good point.. although most people like swapping rootfs.img's and making things easy, plus I already had the rootfs ready for myself.. I figured why not.
But you're right, it would be helpful to know either way, so this is exactly what I did. I opened the init file within the rootfs.img (after mounting in Ubuntu) and searched for "read_ahead_kb" and found it. At the front of the line read: "echo 2048". That's the command that enables the 2,048KB SD cache during boot (the rest of the line I assume is to apply that setting to the SD card? Not sure..). I changed that value to 256KB, making that: "echo 256". Saved the file, unmounted the rootfs, and voila.
Click to expand...
Click to collapse
Should I use DroidExplorer after booting the phone and then should I be looking for a file simply called "init" with no extension - or how do I do it ? It's just that I now see that a new rootfs 20110724 has just been released and since I'm sure it won't be the last, I'd like to know how best to edit these myself
C
championc said:
Should I use DroidExplorer after booting the phone and then should I be looking for a file simply called "init" with no extension - or how do I do it ? It's just that I now see that a new rootfs 20110724 has just been released and since I'm sure it won't be the last, I'd like to know how best to edit these myself
C
Click to expand...
Click to collapse
Yes, the init file will be located in the "/" directory (where you find the /system and /data folders), and in that file, seach for "read_ahead_kb", and once you find that line, go to the bbeginning of that same line and change "echo 2048" to whichever value works best with your SD card.
Personally, I would use a program on the phone such as Root Explorer to modify the file, save, and reboot to have the effects instantly. That would simulate the same process I used to modify the rootfs in Ubuntu.
Sent from my MSM using XDA App
Gee, why all this complicated workaround for something easily done in one line inside froyo.user.conf...
echo "256" > /sys/devices/.....
-- Starfox
Starfox said:
Gee, why all this complicated workaround for something easily done in one line inside froyo.user.conf...
echo "256" > /sys/devices/.....
-- Starfox
Click to expand...
Click to collapse
Um, it's not anymore complicated... Although my wording might have made it seem that way. The only difference was mounting the rootfs.img and changing the value in the init instead of the froyo.user.conf.
maff1989 said:
Um, it's not anymore complicated... Although my wording might have made it seem that way. The only difference was mounting the rootfs.img and changing the value in the init instead of the froyo.user.conf.
Click to expand...
Click to collapse
Well... the user.conf method probably is easier, considering you won't have to change it with any rootfs updates - you can swap rootfs', so long as that entry is in the user.conf file it will always get reapplied.
arrrghhh said:
Well... the user.conf method probably is easier, considering you won't have to change it with any rootfs updates - you can swap rootfs', so long as that entry is in the user.conf file it will always get reapplied.
Click to expand...
Click to collapse
Yeah, that's true, since you put it that way lol. I was simply looking for the most direct way to apply the change, and the modding the rootfs seemed like the logical choice. Tbh I didn't know the user.conf was that significant.. besides applying some user-based settings and whatnot.
Boy I have a LOT to learn...
maff1989 said:
Yeah, that's true, since you put it that way lol. I was simply looking for the most direct way to apply the change, and the modding the rootfs seemed like the logical choice. Tbh I didn't know the user.conf was that significant.. besides applying some user-based settings and whatnot.
Boy I have a LOT to learn...
Click to expand...
Click to collapse
You and me both! I still have quite a bit to learn about how Android has been pieced together on our phones. Doesn't help that I don't really know jack about Android itself .
The user.conf file is quite powerful, assuming you don't need stuff applied too early in the boot process...
arrrghhh said:
You and me both! I still have quite a bit to learn about how Android has been pieced together on our phones. Doesn't help that I don't really know jack about Android itself .
The user.conf file is quite powerful, assuming you don't need stuff applied too early in the boot process...
Click to expand...
Click to collapse
For me, Android as an operating system is easy to understand, since it operates similarly to any computer system. But the fine details like how the rootfs, initrd.gz, kernel, and (most recently) the user.conf work puzzles me like no other. +1 for a Dev Learning Thread!
And yeah, the initrd.gz is what is loaded first, and to my knowledge it's that portion of the boot process that handles the Apps2Ext2/SD function... Fun little tidbit of info
You just need to study init (the file) to find out where everything is located, then adjust what you put into froyo.user.conf. Anything that is done before userinit.sh parses *.user.conf can be manipulated directly, anything after you need to get creative with sed or similar.
Initrd is the INITial Ram Disk. Basically, the kernel when loaded does not have enough to load the whole root/etc. so the initrd contains enough tools to get the kernel to recognize where the rootfs is and load it, then pivotroots with the real root then exec init.
App2sd has nothing to do with initrd. /sdcard/.android_secure, if you look in a real Android handset, is a world-unreadable/inaccessible directory. If you look closer, you notice it's been bind-mounted with those permission to "hide" the content. If you umount it, you have files named *.asec, which is a encrypted (to my knowledge) mini-fs containing the files for each program moved to SD. When Android boots it loopmounts each file to a dir in /mnt/asec or something. I don't think XDAndroid has the necessary stuff that goes through .android_secure yet.
-- Starfox
Starfox said:
You just need to study init (the file) to find out where everything is located, then adjust what you put into froyo.user.conf. Anything that is done before userinit.sh parses *.user.conf can be manipulated directly, anything after you need to get creative with sed or similar.
Initrd is the INITial Ram Disk. Basically, the kernel when loaded does not have enough to load the whole root/etc. so the initrd contains enough tools to get the kernel to recognize where the rootfs is and load it, then pivotroots with the real root then exec init.
App2sd has nothing to do with initrd. /sdcard/.android_secure, if you look in a real Android handset, is a world-unreadable/inaccessible directory. If you look closer, you notice it's been bind-mounted with those permission to "hide" the content. If you umount it, you have files named *.asec, which is a encrypted (to my knowledge) mini-fs containing the files for each program moved to SD. When Android boots it loopmounts each file to a dir in /mnt/asec or something. I don't think XDAndroid has the necessary stuff that goes through .android_secure yet.
-- Starfox
Click to expand...
Click to collapse
Thank you for the information, it's very enlightening
I mentioned Apps2SD because I owned an HD2 and was able to modify the initrd.gz to remove the Apps2SD functionality in a NAND Android ROM.. But thank you again for helping clarify any misunderstandings.
Also, if you have any links/bookmarks to online resources regarding these topics, I'd love for you to share cuz I'd hate to ask questions that could be answered with the right research. Plus, I'm sure you have better things to do with your time than to answer all the questions I have lol
Found this link here on XDA:
Increase Read Cache
I'm using SD Tools and SD Booster to change the cache and test the speed.
Got a class 6 card. Still playing around
großa said:
Found this link here on XDA:
Increase Read Cache
I'm using SD Tools and SD Booster to change the cache and test the speed.
Got a class 6 card. Still playing around
Click to expand...
Click to collapse
I have seen that link a while back; never could get that method to work, though I'm not saying that it doesn't.
From my experience, the difference between using SD Booster and applying the change to the rootfs/user.conf is that the read_ahead_kb, during boot, will impact the remainder of the boot process and furthermore, impact overall performance. That's why, for me, SD Booster simply wasn't cutting it. Although, it was VERY useful in helping me determine which value to place in the rootfs to get the best results.
I've been trying to find a more reliable objective benching tool since sdtools results seem almost random for me. That thread above suggests timing the opening of the gallery, but when I do, not all of the lag appears related to reading the contents of the card -- some of it seems tied up in animations. Anyone have any other ideas?
I actually might try overclocking temporarily if I can't find another card speed bench that less animated. I think the problem with me and sdtools is related to speedometer graphics. It hangs for 15-45 seconds at the beginning of the test while the needle makes its way painfully slowly to the first burst of speed read before continuing the test. Running multiple tests on 2048, 1024, 512 and 256 as set in sd booster, there was almost as much variation at each setting as there was between all the settings.