[Guide] Shi1tROM Tutorial Firmware - The beginning - AT&T Samsung Galaxy S II SGH-I777

Long ago, I said I'd start working on a high-level guide to creating firmware for one's own use (or possibly even eventually releasing to the wild). The idea was to make a high-level guide suitable for those who are comfortable with a Linux commandline but who might not be familiar with Android userland and tools available for working with firmwares.
So with this - I'm going to go through the steps I went through to create the deodexed XWKL1 firmware I've been using for the past two weeks, documenting it with notes as I go along about unusual "gotchas" that aren't necessarily in the documentation for the tools used.
Once I've gotten to the point where I have created a usable firmware, I'll post it here for consumption. Then I'll figure out what to do next - I've been lazy and happy with just using JKay mods lately.
Up next (after I reserve some posts): Deodexing

---

Obtaining firmware and deodexing
The best place to find I9100 stock firmware images for deodexing is, in my opinion, Intratech's thread in the I9100 Original Android Development forum. This thread is at:
http://forum.xda-developers.com/showthread.php?t=1075278
For the purposes of this guide, I'm originally going to use the XWKL1 download.
The next thing you need (in addition to a Linux box and the knowledge of how to use it...) is dsixda's Android Kitchen. While kitchens are generally frowned upon by firmware maintainers, deodexing manually is a massive pain in the ass and usually using dsixda's kitchen for this purposes is considered OK. To paraphrase gtg465x: "Deodexing with it is OK, but anything more and the ROM gods will frown upon you". Some of the other features of the kitchen like preintegrating busybox and rooting have issues - namely that they often inject old versions of busybox/su.
While the kitchen officially is supported on Windows, I cannot help you with any Windows-specific issues - you're on your own.
To get dsixda's kitchen, start with his GS2 thread at:
http://forum.xda-developers.com/showthread.php?t=1227549
I am going to use the latest version as of the time of this writing, which is 0.188
Extract the kitchen somewhere in your filesystem - I will refer to the directory unzipping the kitchen creates as $KITCHEN_ROOT
Extract factoryfs.img from the stock firmware TAR you downloaded earlier - you don't need anything else. You may need to fix its permissions by making it readable:
Code:
chmod a+r factoryfs.img
Place factoryfs.img in:
Code:
$KITCHEN_ROOT/original_update/
Start the kitchen by going to $KITCHEN_ROOT and running the menu:
Code:
./menu
Choose 1) Setup working folder from ROM
Accepting all of the defaults should work. It'll complain about not finding any kernel or cache - but this is OK. Neither of these are really needed as Hellraiser will take care of these later. The kitchen will ask for root permissions (sudo password) to extract the firmware contents. Say yes when it asks about a fake boot.img
Next, we deodex. Choose 0) Advanced Options
Choose 11) Deodex
Choose b) to deodex both /system/app and /system/framework
This will take a while. Go do something else. For example, I'm going to take a shower. (You don't need that much time - but I need a shower. )
Take a careful look at the last few lines of results here. If the deodex job worked perfectly, you should see no failure reports and 0 odex files in /system/framework or /system/app. However, for I9100 XWKL1, three .odex files will remain in /system/app, and you will see the following error:
Code:
Could not deodex the following (you can try to deodex these files again):
Maps.odex Phonesky.odex VoiceSearch.odex
Sometimes a second deodex pass will fix these, but not for these three files for XWKL1. You need to find a different way to fix them. Now I've heard that some advanced options like API Level mangling can fix these errors, but we're going to go with a simpler and easier approach for now.
We are now going to enter our last steps of the kitchen before we switch to manual maintenance of the firmware:
Return back to the main menu of the kitchen.
Choose option 99) Build ROM from working folder
Go ahead and use the defaults EXCEPT when it asks you to sign the firmware - note that this will zipalign all files (this is why zipalign scripts in init.d are rarely useful - on any decent firmware everything should already be zipaligned.)
Say no when it says to sign the firmware - since you're going to be making manual changes to the results, the signature will be broken.
You will now have a ZIP file in the following location:
Code:
$KITCHEN_ROOT/OUTPUT_ZIP/
DO NOT FLASH THIS FILE. IT STILL HAS SERIOUS ISSUES THAT REQUIRE REPAIR.
On to the next step... We take this firmware that is high in fiber and digest it a bit ourselfs, preparing to plop it out.

---

Fixing failed deodexes and debloating
If you were lucky, the firmware fully deodexed properly. However, often, it won't. For example, as covered in the previous post, with XWKL1, three files won't deodex. We will go through the process of manually dealing with these files in this post.
First, take the ZIP you obtained from the previous step, and unzip it to its own working directory somewhere. I will refer to it from now on as $FIRMWARE_ROOT .
You will find three APKs with .odex files in the following location for XWKL1:
Code:
$FIRMWARE_ROOT/system/app
Remove all three APKs and associated .odex files - these files are not useful to us. We now have to deal with the removed files.
Phonesky.odex
When I first saw this, I thought "WTF? I've never seen that before." - Time to Google it. It turns out that at some point, Google renamed the Market APK. It used to be Vending.apk, now it's Phonesky.apk - if we don't do something about this, the firmware will have no Market! So it must be fixed.
The good news is - The Market gets auto-updated frequently. As a result, we can obtain a replacement, AND that replacement is likely to be newer! What we need here is a device that is already running with a functioning Market.
Connect to your phone with ADB, and start navigating through directories. Downloaded apps are in the following location:
Code:
/data/app
Look for a Market APK there. (I need to take a few minutes to nandroid back to a different firmware - as mine currently has the latest market in /system, I don't have one in /data/app to pull.) OK, I'm on my UCKK6 Nandroid now, so, the Market is at the following location for me:
Code:
/data/app/com.android.vending-1.apk
Let's pull that into the right location in our firmware:
Code:
cd $FIRMWARE_ROOT/system/app
adb pull /data/app/com.android.vending-1.apk Phonesky.apk
OK, we've fixed the Market AND integrated the latest update!
Next up, VoiceSearch.apk:
While it is less critical than the Market, most users won't like a firmware without Google Voice Search. As this never gets auto updated and is not Market-downloadable, it's harder to find a working copy. Try deodexing other firmware bases (such as XXKI3) to get this file, or pull it from a firmware that has it properly deodexed. I grabbed mine from VillainROM 3.0 (Thanks Pulser!) initially, I'm going to try a different candidate before posting up my firmware this time around. This time, I'm gonna grab it from XWLA4. (Yes, after I finish with this guide based on XWKL1, I will be moving to XWLA4 to play with it. Also, yes - this file does deodex properly on LA4.)
Last, Maps.apk:
This is downloadable from the Market, so the safest thing to do is just remove it. However, you can use the same trick as with Phonesky.apk to grab the latest and greatest from an updated running system.
Code:
cd $FIRMWARE_ROOT/system/app
adb pull /data/app/com.google.android.apps.maps-1.apk Maps.apk
Note: If you outright remove Maps, you should also remove Street.apk - however if you are replacing Maps, you can leave Street.apk - it is often not in /data/app since it's updated far less frequently.
Removing Bloat:
Next, let's remove bloat. pulser_g2 has a great reference for what is safe to remove from I9100 firmwares at http://forum.xda-developers.com/showthread.php?t=1069924
First, disable OTA updates by removing:
Code:
rm wssyncmlnps.apk
rm syncmldm.apk
rm syncmlds.apk
rm fotaclient.apk
FM Radio is NFG on the I777, so kill it:
Code:
rm FmRadio.apk
SNS (Samsung's own social media integration) is a battery hog. Kill it, remove it all:
Code:
rm Sns*
rm SevenEngine.apk
Nuke the Hubs:
Code:
rm GameHub.apk MusicHub_10.apk ReadersHub.apk SocialHub.apk
Per ryude's post at http://forum.xda-developers.com/showpost.php?p=22719913&postcount=22, you can also:
Code:
rm $FIRMWARE_ROOT/system/lib/libnggame.so
Nuke the extra clocks - plenty of good ones on the Market:
Code:
rm DualClock.apk AnalogClock.apk DigitalClock.apk
Nuke Samsung's annoying panning tutorial:
Code:
rm PanningTryActually.apk
Nuke some other apps I never use, including all Samsung apps and widgets:
Code:
rm Kobo.apk Zinio.apk BuddiesNow.apk GenieWidget.apk PressReader.apk Samsung*
Live wallpapers are evil device-slowing battery eaters:
Code:
rm LiveWallpapers*
rm Microbesgl.apk
Some weird wifi file transfer stuff that is useless and starts a background service:
Code:
rm FTM.apk FTS.apk
Flashplayer can be installed from the Market:
Code:
rm install_flash_player.apk
Swype is 28 ****ing megabytes, I don't use it, and as I understand it, it sometimes conflicts with trying to update to newer versions. I was going to leave this in until I saw its size:
Code:
rm Swype.apk
rm $FIRMWARE_ROOT/system/lib/libSwypeCore.so
Some people will remove some of the DRM stuff such as SisoDrmProvider.apk - however the last time I tried to debloat DRM, I had some issues with even non-DRM videos. (Video player would hang on exit.)
At this point, we've fixed all of our failed deodexes and debloated. Now that we've digested that high-fiber kitchen product a bit, we can plop out our first flashable firmware image.
Code:
cd $FIRMWARE_ROOT
zip -r -9 ../PowerDump_XWKL1.zip *
This ZIP is actually flashable - if you wipe, flash this, then flash Hellraiser, you should have a working firmware. (going to test this now...) Note that with Hellraiser 0.3.1, you'll need to update to a newer Daily Driver (1/30/2012 or later) to get working Bluetooth HID. I'm going to put up 0.4.0 with DD 2/15-C later today. Note: If you want to flash one of JKay's themes, you must boot the firmware without the theme/mod at least once before flashing JKay. You may also want to flash one of ChainsDD's superuser packages - the firmware is not yet rooted.
Tested and uploaded - at this point in the process, this PowerDump will be your result - http://dev-host.org/I6C
Remember, it'll be a little rough, since you were digesting a high-fiber source and your system probably wasn't used to it, but it's still a good PowerDump.

---

Integrating Hellraiser and Flashing a Kernel
Integrating Hellraiser:
Actually, this one is fairly easy. We'll cover integrating the system aspects of it here, the remaining functions of Hellraiser (flashing kernel and modem) will be covered separately (kernel) or are irrelevant (modem - Hellraiser only flashes a modem because many I9100 firmwares include it - many people including myself believe modems should always be separate.)
Take either a Hellraiser ZIP file, or clone my github repo for Hellraiser at https://github.com/Entropy512/hellraiser_i777 - I strongly recommend looking through the commit history there to see what various changes were done and why. Some of them are just bringing in kernel updates, so you'll need to look through the kernel commit history around the data of kernel release to figure out important changes (like wifi tethering fixes in DD 12/21/2011 and BTHID fixes in 1/30/2012)
The root of this ZIP or the repo will be referred to as $HELLRAISER_ROOT - $FIRMWARE_ROOT is the same as in the previous post.
Copy all files from system/ in Hellraiser over the ones already in your deodexed firmware from the previous post.
Code:
cd $HELLRAISER_ROOT/system/
cp -aR * $FIRMWARE_ROOT/system/
The -a after the cp command means to "archive" (preserve as much file permissions and metadata as possible - although this probably doesn't really matter as the updater-script handles permissions), -R means to copy recursively (copy all directory contents, including subdirectories).
That's it - the system portion of Hellraiser 0.3.1 is complete. Hellraiser 0.4.0 will be a bit more complex when released, we'll get into this in a later post.
Adding a Kernel:
Flashing a kernel isn't too hard. Let's look at the updater-script that my Daily Driver releases use:
Code:
ui_print("Entropy's Daily Driver");
ui_print("2/18/2012 Standard Release - Part B");
show_progress(0.100000, 0);
ui_print("Flashing Kernel");
show_progress(1.000000,5);
assert(package_extract_file("zImage", "/tmp/zImage"),
write_raw_image("/tmp/zImage", "/dev/block/mmcblk0p5"),
delete("/tmp/zImage"));
ui_print("Thank designgears of Cognition X2 for the CWM kernel flashing example for I777");
ui_print("No more redbend_ua makes Entropy happy.");
Most of this is just user interface stuff - my show_progress lines are probably not quite right, but the worst case is they'll make the bar in CWM do funny things, no real damage.
Same for ui_print lines - they just print stuff in CWM. The real meat is this:
Code:
assert(package_extract_file("zImage", "/tmp/zImage"),
write_raw_image("/tmp/zImage", "/dev/block/mmcblk0p5"),
delete("/tmp/zImage"));
Place the zImage of the kernel you want to include in $FIRMWARE_ROOT/ and then add the above lines to your updater-script. Boom - your firmware will now flash a kernel. Maybe add some lines saying that you are flashing a kernel when you start - it's up to you.
Naming your firmware:
You can rename what your firmware displays in Settings->About Phone
Edit $FIRMWARE_ROOT/system/build.prop and change ro.build.display.id to what you want your firmware's name to be.

---

Fixing wifi region issues
Most I9100 firmware bases have the wifi region codes set for Europe. This can be overridden in Settings for 2.4 GHz wireless networks, but not 5 GHz wireless networks. The end result is that without modifications, I9100 firmware bases can only see 1-2 United States 5 GHz channels. Hellraiser does not handle this at the moment - I'm working on making Hellraiser 0.4.0 handle this issue.
Edit $FIRMWARE_ROOT/system/etc/wifi/nvram_net.txt and change the ccode from GB to ALL.
Change the ccode from GB to US in $FIRMWARE_ROOT/system/etc/wifi/nvram_net.txt_murata
At this point, 5 GHz wifi should work in the USA.

---

Adding Superuser (rooting your firmware)
This one is also simple. We're going to integrate the contents of ChainsDD's latest Superuser packages from his site: http://androidsu.com/superuser/
Grab the latest package, and unzip it.
Take system/app/Superuser.apk and put it in $FIRMWARE_ROOT/system/app/Superuser.apk
Take system/bin/su and put it at $FIRMWARE_ROOT/system/bin/su
Now edit your updater-script, first locate the following in your existing script:
Code:
set_perm_recursive(0, 2000, 0755, 0755, "/system/xbin");
Now, after this, take the following lines from ChainsDD's Superuser ZIP and add them:
Code:
set_perm(0, 0, 06755, "/system/bin/su");
symlink("/system/bin/su", "/system/xbin/su");
This sets the su binary to suid root, and symlinks it to /system/xbin
Everything else needed is already handled by our updater-script, since Superuser.apk needs no special permissions.

---

Integrating busybox and bash
This is going to be fairly basic right now - When you integrated Hellraiser earlier, it included bash and busybox, however, busybox isn't installed by the updater-script.
This will change that, it's straight out of the Hellraiser package, add it to your updater-script just before unmounting /system:
Code:
set_perm(0, 0, 0755, "/system/xbin/busybox");
ui_print("Installing busybox update...");
run_program("/system/xbin/busybox", "--install", "-s", "/system/xbin");
Note that this busybox binary was not compiled properly, and as a result has DNS issues. Later on I'll include info on compiling it properly - but for 95%+ of functions it works as it is.
The end result is at: http://dev-host.org/XCB

---

Wanted to be the first to say I can't wait as I've been following you since I got here! Your my inspiration
Sent from my SGH-I777

---

....rubs hands together!
Sent from my GT-I9100 using XDA App

---

I am soooo looking forward to this!!!

---

Keeping my eye on this.

---

Sigh... wish people would start moving their uploads since multiupload doesn't work!

---

c0ldburn3r said:
Sigh... wish people would start moving their uploads since multiupload doesn't work!
Click to expand...
Click to collapse
Yea I was just noticed that one to.

---

For glory!!!!!
I've used Ubuntu in a VM to create a Webos Doctor back when I had a Palm Pre, this looks... Fun.
Can't wait, this should be awesome.

---

Miget be looking to cook something up in a few weeks.
I already got a name too, Jersey Shore Rom. That's the area of NJ Im from.
Stay tuned!
Thanks for the tutorial, Entropy.

---

Am I the only person who thought the title said **** Rom? Lol
Sent from my GT-I9100 using Tapatalk

---

Good stuff Entropy! I'll be the first to admit I am a linux noob.. but I installed Ubuntu a few weeks ago and have been playing around.. learning a little here and there. Looking forward to this and learning more about android and linux! Thanks

---

MysticKing32 said:
Am I the only person who thought the title said **** Rom? Lol
Sent from my GT-I9100 using Tapatalk
Click to expand...
Click to collapse
It does say it
I'm so stoked for this!
Sent from my SAMSUNG-SGH-I777 using Tapatalk

---

but Entropy... why do all this when you could just take someone elses rom from the i9100 forums, hellraise it, then post it here in our forums just like everyone else has been doing lately???

---

task650 said:
but Entropy... why do all this when you could just take someone elses rom from the i9100 forums, hellraise it, then post it here in our forums just like everyone else has been doing lately???
Click to expand...
Click to collapse
LOL. Well, I usually just ran VillainROM, but I found a nasty bug in XXKI3. Actually, my experimentation with KL1 or LA4 may lead to VillainROM 3.1 (I eval the raw base, Pulser perfects it.)