Related
If anyone is interested, here is a patch which merges Dell's 309 release with the CodeAurora gingerbread kernel for the Streak. (It's not really 7zipped, but I didn't realize bz2 wasn't a legal extension here). It compiles but I'm wrestling with an error creating the System.map file.
If anyone knows someone at Dell such that we can get a newer drop than 309, I'd be happy to merge that instead.
Gingerbread kernel progress
I've managed to push my changes up to github.
https://github.com/coredog64
There are two repositories:
Dell-Streak-Device-Tree, which I cloned from an existing Froyo version by Mistadmin (credit where credit is due ). The only significant changes are the removal of some prebuilt modules which wouldn't work with the new kernel and massaging of make files to eliminate LOCAL_MODULE_TAGS errors on build.
streak-android, which is the CodeAurora kernel sources with Dell's alpha level Gingerbread changes merged in. As DJ_Steve mentioned in another thread, the 4329 drivers out of CodeAurora don't compile, so I replaced them with the latest and greatest from CyanogenMod.
The only other significant change was to add tun.ko and cifs.ko to the list of modules being built.
The tree builds (yay!) but it only builds the image for a generic device. It does build a kernel but not an image for the Streak.
If someone has some time to take a look and give me some pointers I would appreciate it.
Hi.
Just sharing this for reference for them who wants to:
- Build a linux mainline up-to-date kernel (3.10.y). Builds fine with some reverts and no other kernel commits. Updated straight on M kernel (3.10.40). (branch 'mainline'). Easy to maintain and update yourself when linux mainline is updated.
- Build with interactive governor from 3.18 kernel. (branch 'interactive')
- Build with an up to date wireless driver from: https://android.googlesource.com/kernel/common/+log/bcmdhd-3.10. (branch 'wireless')
All three sorted in respective branches at https://github.com/PerLycke/kernel_moto_shamu
And all of those branches are independent, all applied on pure M (r26) source, so merge away!
Master branch is the branch I'm using personally, so that's not a reference. Or sort off. If you like what I like (very stockish).
Cheers
pemell
@pemell
This is great! Thanks for sharing your work! I too tried to merge all the way to .96 and ended up with no sound lol. Regardless, I will be working on basing my new Shamu kernel off of your work. It looks great! Thanks again!
Introduction
Hello all, I am bringing you this thread as a jumping off point to compiling kernels and working with upstream Linux. I will include a guide, some links, and some terms that will help you get started with modifying kernel source. This is also a good reference point for existing developers as I have consolidated all upstream patches into one repo. Let's get down into it!
What in the world is upstream Linux?
When an OEM sets up a device, they will pick a stable longterm branch from the Linux kernel to base their modifications around (drivers and such). In the case of Shamu, they picked 3.10.40. Currently, the Linux kernel's 3.10 is updated to 3.10.105, as you can see on kernel.org. This means that Google is "missing" versions 3.10.41 to 3.10.105. Now, why does this matter? Well, the way that the Linux kernel runs its stable branches, the only things that get merged into there are bug fixes and security updates. That's it, there are no wonky features or unstable patches. The only way you get a patch into a stable branch is by having it be in the mainline branch first. Some developers have an aversion to adding upstream because they claim it is excessive and not necessary and they are partially right since not all the patches that come in are relevant to our architecture (arm). However, upstream Linux is not unstable and by adding each version one at a time, you can verify this. Being up to date is good since you keep yourself protected from bugs and security issues that crop up. Google has been better about doing this lately with their monthly security updates but it never hurts to take matters into your own hands.
I suggest watching one of these talks given by Greg Kroah-Hartman, it is really interesting to see how the process goes:
https://www.youtube.com/watch?v=SPY0LyTU53w | https://www.youtube.com/watch?v=L2SED6sewRw
What did you do?
All I did was fetch the latest kernel.org patches from here and cherry pick them on top of the latest kernel source from Google.
I merge these patches by cherry-picking each version individually (3.10.40 to 3.10.41, 3.10.41 to 3.10.42, etc), that way I can verify that the kernel compiles fine and that there are no merge conflicts. Google will sometimes pick certain commits from upstream that are of a higher importance than others which can result in conflicts if you try to pick it again. Additionally, upstream might fix a bug one way and Google has done it another (which is not really good, Google should be pushing their fixes back to upstream so everything stays in sync).
What do I do with this?
I have created two repos below: one with the latest N security update branch with the latest upstream patches picked up on top of it (the shamu-upstream branch) and another one with a plain AnyKernel source for you to modify (shamu-stock is the most basic, shamu-decrypt contains an fstab file that will disable both forced encryption and dm-verity). You are free to fork these or base other branches on them, that's the whole point of this post. I have verified that all the patches contain no major detectable issues. I would like some credit if you do use it but it's not required since the kernel is licensed under GPL Another reason I offer this is I have seen a lot of developers picking in upstream in patch sets, so you get one single commit for an upstream version. This is detrimental as you start to add your own patches as you cannot fully tell what was modified and for what reason without the individual commits. It might look cleaner but you don't get full history which hurts you in the long run.
Links
Kernel source: https://github.com/nathanchance/shamu/tree/upstream-7.1.1
AnyKernel source: https://github.com/nathanchance/AnyKernel2/tree/shamu-stock-decrypt
Toolchain source: https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/
How to compile
This will be a quick, step by step guide on how to compile this kernel from source. By using this process, you can start to make modifications to the kernel source and make a flashable zip.
Clone the kernel source, the AnyKernel source, and toolchain source
Code:
cd ~
mkdir Kernel && cd Kernel
git clone https://github.com/nathanchance/shamu.git source
git clone https://github.com/nathanchance/AnyKernel2.git anykernel
git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8 AOSP-4.8
Explanation:
First command: make sure you are in your home directory (or whatever directory you want to hold the kernel folder we are about to make.
Second command: make a Kernel folder and move into it
Third command: clone the kernel source from my repo into a folder named source
Fourth command: clone the AnyKernel source from my repo into a folder named anykernel
Fifth command: clone the Google 4.8 toolchain into a folder named AOSP-4.8
AnyKernel is the name of the zip we are going to make which allows the kernel to be flashed over any ROM.
A toolchain is a set of compiler tools that allow us to compile the kernel on any computer architecture.
Make sure you are on the correct branches
Code:
cd source && git checkout upstream-7.1.1
cd ../anykernel && git checkout shamu-stock-decrypt
Explanation:
First command: move into the source directory and checkout the branch "upstream-7.1.1", which is the necessary branch for this device.
Second command: move into the AnyKernel directory and checkout the branch "shamu-decrypt". If you don't want to disable forced encryption, use the "shamu-stock" branch.
Tell the compiler what you are are compiling
Code:
export CROSS_COMPILE=${HOME}/Kernel/AOSP-4.8/bin/arm-eabi-
export ARCH=arm && export SUBARCH=arm
make clean && make mrproper
make shamu_defconfig
Explanation:
First command: point the compiler to the location of your toolchain. If you have done anything different with the folder locations, you will need to modify the "${HOME}/Kernel/AOSP-4.8" part.
Second command: tell the compiler which architecture we are compiling for. In this case, our device is an arm device.
Third command: clean out any compiled files and remove our previous defconfig.
Fourth command: tell the compiler which options we want in the kernel using the shamu_defconfig.
A defconfig is a file that will tell the compiler which features we want in the kernel. No computer ever uses all of the options in the kernel since there are different drivers for various devices.
Make the kernel!
Code:
make -j$(nproc --all)
Explanation:
make tells the compiler to make the kernel (duh) and the -j$(nproc --all) tells the compiler to use the maximum number of cores your computer has available
Make the AnyKernel zip
Code:
cp -v arch/arm/boot/zImage-dtb ../anykernel
cd ../anykernel
zip -r9 stock-upstream.zip * -x README stock-upstream.zip
Explanation:
First command: copies the completed kernel (zImage-dtb) into the AnyKernel folder
Second command: moves us into the AnyKernel folder
Third command: makes the zip file (named stock-upstream.zip) in the AnyKernel folder.
If you ever want to do this again, run these commands and go straight to step 3:
Code:
cd source && git clean -fxd && git pull
cd anykernel && git clean -fxd && git pull
NOTE: This thread is aimed to be a breeding ground for kernel development, a place to jump off if you will. I am happy to answer how to questions about building kernels or flashing the one I have provided but this is not supposed to be a general Q&A thread. Please use another thread or create your own in Q&A if you need assistance.
Nice write up.
One suggestion though. Instead of using the current version as the branch name, you may want to just name them for the device, like "Shamu-Upstream" then tag the versions once you've gotten to the version commit. This will prevent you from having to update the thread for this command anytime that you pull in more upstream changes. It also makes it alot cleaner/easier if someone for whatever reason wanted to checkout a specific patch version as they could just checkout the tag rather than resetting back to the commit.
For lack of a better explanation, tags are sort of like sub-branches within a branch.
Very new to kernel building and AOSP in general. Followed the guide but I'm getting stuck when make mrproper is ran. Build stops and i get the error : unknown target 'mrproper' then the build fails. Any help?
The Flash said:
Another reason I offer this is I have seen a lot of developers picking in upstream in patch sets, so you get one single commit for an upstream version. This is detrimental as you start to add your own patches as you cannot fully tell what was modified and for what reason without the individual commits. It might look cleaner but you don't get full history which hurts you in the long run.
Click to expand...
Click to collapse
That's true. Some devs are lazy to commit each patch and bring up from 3.10.40 to 3.10.103 which comprises nearly 3000 patches.
neobuddy89 said:
That's true. Some devs are lazy to commit each patch and bring up from 3.10.40 to 3.10.103 which comprises nearly 3000 patches.
Click to expand...
Click to collapse
Exactly lol. It's also annoying for people looking at your source because if you had to fix anything, it is going to be lumped in with that enormous patch.
Root Wizard said:
Very new to kernel building and AOSP in general. Followed the guide but I'm getting stuck when make mrproper is ran. Build stops and i get the error : unknown target 'mrproper' then the build fails. Any help?
Click to expand...
Click to collapse
can I see a full terminal window/log showing you entering all the commands?
The Flash said:
can I see a full terminal window/log showing you entering all the commands?
Click to expand...
Click to collapse
Just logged off for the night. Will give a full report back tomorrow with pictures. Basically all the commands go good up to 'make mrproper' . 'make clean' runs fine but the 'make mrproper' goes through incuding all the aosp files for build, then ninja enters build directory, then the error 'unknown target 'mrproper' ' comes up, and the build fails.
Root Wizard said:
Just logged off for the night. Will give a full report back tomorrow with pictures. Basically all the commands go good up to 'make mrproper' . 'make clean' runs fine but the 'make mrproper' goes through incuding all the aosp files for build, then ninja enters build directory, then the error 'unknown target 'mrproper' ' comes up, and the build fails.
Click to expand...
Click to collapse
Oh you're building this into a ROM? Alright, that might make things different but yeah show me pictures tomorrow and I'll try to get you up and running.
The Flash said:
Oh you're building this into a ROM? Alright, that might make things different but yeah show me pictures tomorrow and I'll try to get you up and running.
Click to expand...
Click to collapse
Ah I guess I should have metioned that I was building an aosp rom. Thanks for your quick assistance. Will get back tomorrow.
If you're building into a rom, you just need to change the existing kernel source to this one in the xml for this device.
Yoinx said:
If you're building into a rom, you just need to change the existing kernel source to this one in the xml for this device.
Click to expand...
Click to collapse
Can you explain how to do this? First time building with aosp, not very familiar which xml file or what to change to apply the upstream patch.
Root Wizard said:
Can you explain how to do this? First time building with aosp, not very familiar which xml file or what to change to apply the upstream patch.
Click to expand...
Click to collapse
For shamu in aosp, the kernel is pulled with the main aosp source by that manifest. https://github.com/android/platform_manifest/blob/master/default.xml#L65
You world need to substitute this repo with the new one, and delete your existing kernel source. Then next time you sync the repo tool will pull it down.
On other roms it gets pulled in during the breakfast/brunch process. I can't remember exactly where that comes from, but I think that is in the vendor directory for the device, maybe.
How to modify rom manifests and such though is a little outside of this thread's topic. If you need more info I'd recommend opening a thread in the Q&A forum unless this is something that The flash wants to cover in this thread.
There are a few other caveats with doing it in the rom as well. The defconfig name would need to match what the rom build expects and you would need to go in and checkout the correct branch.
Yoinx said:
For shamu in aosp, the kernel is pulled with the main aosp source by that manifest. https://github.com/android/platform_manifest/blob/master/default.xml#L65
You world need to substitute this repo with the new one, and delete your existing kernel source. Then next time you sync the repo tool will pull it down.
On other roms it gets pulled in during the breakfast/brunch process. I can't remember exactly where that comes from, but I think that is in the vendor directory for the device, maybe.
How to modify rom manifests and such though is a little outside of this thread's topic. If you need more info I'd recommend opening a thread in the Q&A forum unless this is something that The flash wants to cover in this thread.
There are a few other caveats with doing it in the rom as well. The defconfig name would need to match what the rom build expects and you would need to go in and checkout the correct branch.
Click to expand...
Click to collapse
Ok, thanks for your help.
For those of you who have forked my kernel, you will need to revert this patch: https://github.com/nathanchance/and...mmit/0b17006c16bba286ecc9d7bacd75a3902a26835e
It causes issues with Wi-Fi and Chrome.
The Flash said:
For those of you who have forked my kernel, you will need to revert this patch: https://github.com/nathanchance/and...mmit/0b17006c16bba286ecc9d7bacd75a3902a26835e
It causes issues with Wi-Fi and Chrome.
Click to expand...
Click to collapse
It does? I have been using Chrome Beta (on wifi) since last night fine with that patch. Maybe its just a matter of time before things start failing. Thanks anyway, will keep a heads up.
Root Wizard said:
It does? I have been using Chrome Beta (on wifi) since last night fine with that patch. Maybe its just a matter of time before things start failing. Thanks anyway, will keep a heads up.
Click to expand...
Click to collapse
If you toggle Wi-Fi, start flipping through pages in Chrome, and toggle Wi-Fi again, it will reboot. Oddest issue I have ever dealt with.
Sent from my Nexus 6P using XDA Labs
The Flash said:
If you toggle Wi-Fi, start flipping through pages in Chrome, and toggle Wi-Fi again, it will reboot. Oddest issue I have ever dealt with.
Click to expand...
Click to collapse
Interesting, will look into this. I also have been getting reboots from searching forums in xda labs app. Logcat says a ton of "PowerHAL_touch_boost failed to send: Connection refused" before reboots. Is this what is happening to you?
Root Wizard said:
Interesting, will look into this. I also have been getting reboots from searching forums in xda labs app. Logcat says a ton of "PowerHAL_touch_boost failed to send: Connection refused" before reboots. Is this what is happening to you?
Click to expand...
Click to collapse
No I haven't had that happen but that error shouldn't cause reboots.
The Flash said:
No I haven't had that happen but that error shouldn't cause reboots.
Click to expand...
Click to collapse
Huh, odd
Github and the kernel in the OP are updated to 3.10.104.
Introduction
Hello everyone, this is a thread to introduce both users and kernel developers to the concept of linux-stable as well as give developers some tips and a tree to either merge into their own, use as a base, or just as a reference. Feel free to ask questions and enjoy!
What is it?
linux-stable is, as the name implies, the stable branch of the Linux kernel, the base of Android. The phone could not run without the Linux kernel (at least not without reworking a lot of stuff). The Android kernels are based on the longterm stable trees:
Longterm
There are usually several "longterm maintenance" kernel releases provided for the purposes of backporting bugfixes for older kernel trees. Only important bugfixes are applied to such kernels and they don't usually see very frequent releases, especially for older trees.
Click to expand...
Click to collapse
Source: https://www.kernel.org/category/releases.html
All Linux development happens on the master branch, which is governed by Linus Torvalds. When issues are discovered there, the fixes are applied then backported to these various stable trees for consumption. It is not uncommon for a fix to need to go back a few years.
There is a LOT more information available in the notes repo in the android-linux-stable organization if you care to learn more in-depth: https://github.com/android-linux-stable/notes
What does this mean for me?
If you are a developer, this means you should be merging these changes into your own tree. These are vetted, stable fixes to real world problems and they are being handed out for free. It does not take long to get up to date (as you can just merge this tree directly into your own or do it yourself using the tree as a reference) and once you are up to date, there is usually a release once every two weeks, give or take. I provide a rebuttal to a lot of various complaints here. If you still feel like there is a good reason not to do this, please let me know, I'll be happy to try and debate on it!
If you are a user, it means that you should be looking for and using kernels that have these fixes, as it shows the developers care for your security and stability. The current version for this device is 4.4.78 on OOS and 4.4.100 on Lineage while the current version upstream is 4.4.162 so all you need to do is go into Settings > About phone and look at the kernel version to know if you are up to date.
How do I use?
If you are a developer, the reference tree is located in the android-linux-stable organization: https://github.com/android-linux-stable/op5
There are three branches, one for OxygenOS 8.0 stable, one for OyxgenOS 8.1 beta, and one for Lineage 15.1.
This can either be merged into your existing kernel tree if you have one or be used as a fresh base. You do not need my permission to use it nor do you need to give me credit (although it would be appreciated).
If you are a user, use a kernel that has the changes added in!
Getting notified about updates
There are a few ways to get notified of linux-stable updates:
The linux-kernel-announce mailing list: http://vger.kernel.org/vger-lists.html#linux-kernel-announce
The android-linux-stable Telegram channel: https://t.me/alsupdates
Subscribe to this thread
Follow me on Google+ or Twitter
Getting help
If you have any issues with getting these changes into your tree or want to ask a question, there are a few different ways to do it:
Post in this thread
Join the linux-stable support chat on Telegram: https://t.me/joinchat/C1UAJ1EMSX31PCFdwLnOSg
File an issue either in the android-linux-stable notes repo or the android-linux-stable repo for this device
When requesting help, please give some solid details as to what you are struggling with, as I am happy to provide assistant and clarity but not to do something for you (unless I screwed up).
The OP has been updated, including some more information for users as well as some more ways of getting help and notified of updates! Enjoy
Very Nice.!!! :fingers-crossed:
nathanchance said:
The OP has been updated, including some more information for users as well as some more ways of getting help and notified of updates! Enjoy
Click to expand...
Click to collapse
Thanks for everything Sir.
You really are techno-Jesus, here to teach the ways of kernel perfection. I love your work, dedication, and care for your kernel users. Flash4lyfe.
Yes, please flame me for being a fanboy. You can never show enough appreciation for our beloved xda developers (especially the ones that don't treat users like ****).
Many thanks for your work and initiative @nathanchance :good: appreciated :highfive:
Currently giving a try to make a kernel version based from linux .116 of your op5 branch and looks like I have the following error :
Code:
fs/f2fs/f2fs.h:136:20: error: static declaration of 'inode_nohighmem' follows non-static declaration
Any idea ? I was thinking of a "KBUILD_CFLAGS" addition into Makefile (or something like that), but I'm probably wrong I guess.
FYI I'm using stock 4.9 android google toolchain
AndroidGX said:
Many thanks for your work and initiative @nathanchance :good: appreciated :highfive:
Currently giving a try to make a kernel version based from linux .116 of your op5 branch and looks like I have the following error :
Code:
fs/f2fs/f2fs.h:136:20: error: static declaration of 'inode_nohighmem' follows non-static declaration
Any idea ? I was thinking of a "KBUILD_CFLAGS" addition into Makefile (or something like that), but I'm probably wrong I guess.
FYI I'm using stock 4.9 android google toolchain
Click to expand...
Click to collapse
Wow I am so sorry about that, I must not have built 4.4.116 I removed all of the upstream f2fs code in my own kernel tree so I didn't encounter this error. I have pushed the fix that popped up in kernel/common, should be able to pull and build successfully now :good: :highfive:
nathanchance said:
Wow I am so sorry about that, I must not have built 4.4.116 I removed all of the upstream f2fs code in my own kernel tree so I didn't encounter this error. I have pushed the fix that popped up in kernel/common, should be able to pull and build successfully now :good: :highfive:
Click to expand...
Click to collapse
No worries
Thanks, it works like a charm now (currently trying my kernel on .116), again good job :highfive:
Cheers'
4.4.117 has been merged in.
4.4.118 has been merged in (conflict notes).
4.4.119 and OOS 5.0.4 have been merged in.
4.4.120 has been merged in (conflict notes).
thanks for making this, trying to build lineage and I keep getting the following error:
Code:
drivers/bluetooth/btfm_slim.c:26:23: fatal error: btfm_slim.h: No such file or directory
Any ideas?
---------- Post added at 10:47 PM ---------- Previous post was at 10:20 PM ----------
lol, found one of your comments on another post that helped
Use an out folder.
mkdir out
make O=out mata_defconfig
make O=out -j$(nproc --all)
Click to expand...
Click to collapse
new problem, the kernel compiles into an Image.gz, how would I go about getting a zImage for use in AnyKernel
aeppacher said:
thanks for making this, trying to build lineage and I keep getting the following error:
Any ideas?
---------- Post added at 10:47 PM ---------- Previous post was at 10:20 PM ----------
lol, found one of your comments on another post that helped
new problem, the kernel compiles into an Image.gz, how would I go about getting a zImage for use in AnyKernel
Click to expand...
Click to collapse
That is the zImage (you'll actually want to use Image.gz-dtb), you can just add it to the AnyKernel2 zip as is, the backend handles that name.
Sent from my Google Pixel 2 XL using XDA Labs
nathanchance said:
That is the zImage (you'll actually want to use Image.gz-dtb), you can just add it to the AnyKernel2 zip as is, the backend handles that name.
Sent from my Google Pixel 2 XL using XDA Labs
Click to expand...
Click to collapse
It booted perfectly, thanks so much! I would use the thanks button but it doesn't seem to be working :/
4.4.121 has been merged in.
I will have a new tree based on the 8.1 sources out tonight.
Sent from my Google Pixel 2 XL using XDA Labs
I have added a new branch for the 8.1 beta source; I will continue to update both 8.0 and 8.1 until 8.1 goes stable.
https://github.com/android-linux-stable/op5/commits/oneplus/QC8998_O_8.1
4.4.122 has been merged in.
4.4.123 has been merged in.
Introduction
I am writing this guide as I was unable to find a well-documented thread over XDA (there are just a lot of them). In case there is anything that concerns this guide, feel free to ask in this thread. I expect you to know the basics of Linux, GIT, Android. Please look over some guides in the forum in case you are not familiar with them. In case you want to contribute to this, feel free to do so. I welcome everyone's contribution to be it fixing my derp, pointing a mistake or adding a new section.
Links to some nice documentations:
Git Documentation
Linux Kernel Newbies
How to take logs in Android
How to get an Android kernel up to date with linux-stable
Checking Current CAF Tag of the Kernel
We will be using best_kernel for this purpose, whose aim is to compare your current kernel source code with different CAF tags and select the best possible match using git diff.
Clone/Download your kernel repo and commit your changes(if not already committed) as best_kernel depends upon git diff to compare changes.
Depending upon your kernel version, add required CAF kernel repo as remote and fetch it. best_kernel will use this fetched data to compare changes.
After fetch is done, run best_kernel. It should run automatically if you have placed dotfiles in the $BIN directory and have the necessary dependencies installed.
best_kernel will compare changes and will provide results based on that. However, depending on the number of tags, it will take time. A lot of time, so go and grab a cup of coffee till then.
A Dummy Example:
Downloading stock kernel source code from OEM’s website
Code:
wget http://nokiaphones-opensource.azureedge.net/download/phones/Nokia6.1_V2.22J.tar.bz2 && tar xvjf Nokia6.1_V2.22J.tar.bz2 && cd kernel
Adding and committing all files due to lack of a .git dir.
Code:
git init && git add --all && git commit -m "Initial Commit"
As the kernel source code version is 4.4, I will add 4.4 remotes from CAF and fetch it.
Code:
git remote add caf https://source.codeaurora.org/quic/la/kernel/msm-4.4/ && git fetch caf
After fetch, I will check for the current CAF tag using best_kernel. As my device platform is sdm660 and CAF has been releasing sdm660 tags as sdm660.0, I will be using it as an argument in best_kernel. This will ensure that best_kernel doesn’t waste time checking CAF Tags of other platforms and will speed up the process considerably.
Code:
best_kernel "*sdm660.0"
RESULT will be something like this:
Code:
Best match TAG: LA.UM.6.2.r1-06900-sdm660.0 Lines changed: 177924
What is CAF/Code Aurora | CodeAurora Android Releases Page | CAF Kernel Sources
Creating/Importing GIT History for Kernel Source
A lot of OEMs release kernel sources in a compressed format excluding the .git dir which results in no git history. Creating or it's better to say to import one is simple and easy.
Download and check the current CAF tag of your Android kernel source.
Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base while checking out to the specific version of your kernel source you are going to copy-paste. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.
Based on that CAF tag, download the respective kernel repository from CAF and checkout for that specific tag.
Copy and paste your downloaded kernel source (with no history) over the CAF one (which has the history of that CAF tag).
Commit the changes.
You have accomplished creating/importing history.
A Dummy Example:
My current CAF tag is LA.UM.6.2.r1-06900-sdm660.0.
Therefore, cloning kernel source from CAF and checking out for this tag while creating a new branch named rebase.
Code:
git clone https://source.codeaurora.org/quic/la/kernel/msm-4.4/
cd msm-4.4/
git checkout tags/LA.UM.6.2.r1-06900-sdm660.0 -b rebase
Now, I will copy-paste the kernel source that I downloaded from OEM's website over the CAF one. Assuming the downloaded kernel is in ~/Downloads/kernel and CAF kernel source is in ~/msm-4.4 directories.
Code:
#Renaming the dir so, that it will get merged while copying due to similar names
mv kernel/ msm-4.4/
cp -avr msm-4.4/ ~/msm-4.4
cd ~/msm-4.4
git status
git commit -m "Import Nokia Kernel Source"
Keep in mind that a .gitignore is generated which can and will ignore files during commit which were not getting ignored earlier due to its absence. In case during compilation you encounter any error stating about missing file, add it forcefully and commit again.
Remember, its always good to split that one big commit into small parts. It's helpful when you start rebasing your kernel usually when new CAF Tag for new android versions come out. Solving a few conflicts ina lot of commit is much better than solving a load of them in one. It also gives you a chance to easily diff on websites like GitHub so that you can figure what's a particular line is doing.
Doing that is also easy. Just reset head for a particular folder like driver/touchscreen and commit them separately. Check git rebase documentation which will help you with this.
Upstreaming your Kernel
Merging upstream versions is a really good thing. It brings in security patches, stability, new features and much more. The usual and best way is to visit Linux Kernel Source and check if there are upstream updates available. If yes, start merging them in your kernel source one by one.
Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.
A Dummy Example:
Assuming my current kernel version is v4.4.205, I will fetch v4.4.206 from Linux Kernel Source and merge that. I can also fetch and merge different branches from CAF Source which are usually upstreamed with Linux-stable (for example, aosp/upstream-linux-4.4.y branch)
Code:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git v4.4.206
git merge FETCH_HEAD
Fix conflicts if you get some. What to keep and what not is a question that depends upon the device to device. Usually, you should compare local changes and upstream and try to figure out what could be the best way to fix it. I will advise to check the history of that file in upstream and see which commit caused that conflict and why. You can also check similar merges by other same kernel versions to see if they had the same conflict and how they fixed it.
I will also advise merging AOSP's kernel/common and new CAF upstream tags (Only QCOM devices should merge new CAF tags) regularly when available. The reason is pretty simple. There are commits that get merged in AOSP like which improves performance or other stuff which doesn't get merged in Linux Kernel Source. This way you can get the benefit from all sources. This, however, is not pretty easy and might result in a huge number of conflicts considering if OEM has modified the source a lot from upstream.
CAF also hosts f2fs and other file system changes on their servers. Merge it if you want and use it.
Compiling Standalone Kernel
Compiling a kernel standalone requires you to have a compatible toolchain like GCC or Clang. The best way to figure out which one suits your need is testing and debugging. The standard AOSP GCC toolchains are recommended for devices with old kernels, to begin with, while newer ones can use clang with backports or upstream merges. In case you want to compile with clang, checking out android-kernel-clang is a good way, to begin with.
Compiling standalone kernels can give a lot of missing headers errors/warnings. This is due to not compiling it in a ROM environment. This can be easily fixed either by correcting paths of those header files being included or creating an out dir manually and using it as output. You can also use this commit to do that automatically.
Clone your kernel source and desired toolchain(s). Export arch and cross compiler arguments. Choose the desired defconfig to make and compile.
A Dummy Example:
I will clone the kernel source and toolchain in the same dir and export the following arguments in kernel source root dir.
Code:
export ARCH=arm64
export CROSS_COMPILE=/media/hdd/aayush/kernel/aarch64-elf-gcc/bin/aarch64-elf-
make nokia_defconfig
make -j$(nproc --all)
Kernel conflicts might occur. You can search on GitHub to check if someone has fixed that already, otherwise fix the issues yourself. Comparing code with upstream repositories is also a way to figure out the differences. Choosing defconfig can be a bit tricky if you are doing bringups. I recommend pulling the current defconfig from your device itself. Its usually by the name of config.gz in /proc/ of your device.
Links to some cross compiler
GNU Toolchain by ARM
DragonTC
GNU and Linaro
Working with TWRP from Scratch
This post is supposed to help you with building TWRP from scratch for the first time if you are a newbie. There are some good resources already available for that over the web, specially XDA. I am linking some good ones below that you can read for more info on it.
How to compile TWRP Touch Recovery
The aim of a device tree is to contain the device-specific code for the build you are doing. In this case, as we are building TWRP which is a custom recovery, only flags relating to building TWRP and a normal recovery are enough to build it. Adding system specific codes like partition sizes for the system, vendor, boot and other doesn't contribute anything. An example tree can be begonia's tree hosted on TeamWin's org maintained by me.
Now coming to the device you want to build TWRP for, you will the following :
- A working kernel
- Device-specific binaries/libraries etc required for features like encryption which heavily differs on OEM implementation
AB devices need an extra commit in their kernel to make 'fastboot boot' command work: init: initramfs: disable do_skip_initramfs . Additionally if your device ships a separate DTBO image you will also need the following flag and the DTBO image: begonia: Build DTBO image into recovery
Newer devices are nowadays coming with 'fastboot boot' command disabled. Nokia 8.1 is an example of it. To make it work again, you might need to tweak bootloader which can prove to be risky.
Now coming to creating a device tree for TWRP. The very first step is to look out if someone else has booted TWRP on a similar SOC, OEM or same device. It can be used as a reference to fix similar/OEM related bugs and hacks quickly. In case, one doesn't exist, just pick a latest one from Teamwin's Github org for the android version you are compiling for.
Reserved (2)
Wew many thanks,
Thanks
Have just started exploring xda chef central.
I cam across this guide, really ahlepful Hope you complete it soon.
Updated the thread. Fixed formatting, links and added a new section.
How To Unofficially Unlock Bootloader For LG G8
We all know LG G8 is a android device. It's open source. But I recently bought a LG G8 only but it's locked to sprint sim . No any sim work. I can't unlock bootloader. I can't root. I can't flag firmware or install custom recovery. Anyone know here , how to totally remove firmware and but another firmware ? Why we can't build a method to do that?
TheImpulson said:
Updated the thread. Fixed formatting, links and added a new section.
Click to expand...
Click to collapse
Can you create a guide for, "How to Rebase an Android Kernel". Like from Android 9 to Android 10?
thanks for this guide bro
While using best caf kernel, Prefix of the caf tag should start with a -
For example, "*- sdm660.0"
@theimpulson
Hey there,
Am getting this error del.dog/megupebinu.txt
While trying execute best_kernel script
This is helpful, thanks mate
@theimpulson I had a question: (Before reading this guide) I had already downloaded stock Samsung sources and git init push the thing to my Github. So will anything bad happen if I did it this way or do I have to rebase it with CAF?
Also what are the benefits of these so-called CAF based kernels everywhere? Improved performance? Cuz I really wanted to compile my own kernel specifically for performance with WSL-2 (shameless plug)
Thanks in advance and sorry if the @ bothered you,
Ishaq
NullCode said:
@theimpulson I had a question: (Before reading this guide) I had already downloaded stock Samsung sources and git init push the thing to my Github. So will anything bad happen if I did it this way or do I have to rebase it with CAF?
Also what are the benefits of these so-called CAF based kernels everywhere? Improved performance? Cuz I really wanted to compile my own kernel specifically for performance with WSL-2 (shameless plug)
Thanks in advance and sorry if the @ bothered you,
Ishaq
Click to expand...
Click to collapse
The existing sources are fine, no issues if you are fine without any git history.
SOC specific optimizations, bug fixes etc. Nothing more. So, yeah, if you want SOC specific optimizations, performance improvements, you will want to merge latest CAF tags for your kernel.
theimpulson said:
The existing sources are fine, no issues if you are fine without any git history.
SOC specific optimizations, bug fixes etc. Nothing more. So, yeah, if you want SOC specific optimizations, performance improvements, you will want to merge latest CAF tags for your kernel.
Click to expand...
Click to collapse
thanks a lot for the reply! Another thing, because of your suggestion I tried to Import the git history (and i succeeded). Then I wanted to upload these sources to my own Github, which I can't figure out how to do. It only gives Github repo for the place where it came from (msm-3.18). Help me please
Edit: The guides which I found for merging CAF tags say that my base also has to be CAF (which it isn't) and this guide here basically does not work