[DEV] CM7 Kernel performance comparisons (stock vs OC) - Nook Color Android Development

So, for all us CM7 users, there have recently been alot of questions as to why there is such a disparity between the stock CM7 kernel, and the 1.1Ghz kernel, given both are made by the same person (dalingrin). Really, there are two questions -
1) Why is the quadrant score different between the kernels?
2) How does this equate to real-world use?
To help answer #1, i went ahead and purchased a copy of Quadrant Advanced. The advanced version lets me run the bench offline (helpful at work ), and also shows each piece of the score (the important part, as seen in the results). this breakdown shows where the difference is.
But to answer #2, I have to go well beyond Quadrant, and look at many different benches. I tried to find a variety of both system and 3D benches in a hope to uncover any problems anywhere. If there is a more widespread problem, it may be uncovered in other benchmarks. So, without further ado, the test system:
CM7- nightly 27, running on eMMC
Stock CM7 Kernel, 925Mhz, Performance Governor
OC CM7 Kernel, 1000Mhz, Performance Governor
OC CM7 Kernel, 1100Mhz, Performance Governor
I kept the gov on performance, to help rule out any differences between governors. Performance runs the CPU at full speed all the time, so it keeps the benches comparable. For every CPU speed/kernel change, i rebooted the system, and ran each bench once in the order listed. And the results!
Stock kernel
CPU @ 925, Performance gov,
Quadrant (First run only):
Total: 1536
CPU: 2504
Mem: 1080
I/O: 3629
2D: 188
3D: 278
Linpack:
12.078Mflops
NenaMark:
16.7 Fps
Benchmark PI (https://market.android.com/details?id=gr.androiddev.BenchmarkP):
Pi found in 1636ms
Antutu System benchmark (https://market.android.com/details?id=com.antutu.ABenchMark):
Total Score: 1675
Memory: 407
CPU Integer: 578
CPU Float: 129
2D Graphics: 100
3D Graphics: 276
Database IO: 10
SD Card Write: 5.0 MB/s
SD Card Read: 12.5 MB/s
An3DBench (https://market.android.com/details?id=com.threed.jpct.bench):
Fillrate ST/MT: 6.21/6.22 MP/s
High object count: 27.03 Fps
Multiple Lights: 40.19 Fps
High polygon count: 19.97 Fps
Keyframe animation: 39.97 Fps
Game level: 30.04 Fps
Total score: 4278
3/16 Overclock Kernel
CPU @1000Mhz, Performance gov
Quadrant (First run only):
Total: 960
CPU: 2693
Mem: 1099
I/O: 522
2D: 202
3D: 286
Linpack:
12.983Mflops
NenaMark:
17.0 Fps
Benchmark PI:
Pi found in 1627ms
Antutu System benchmark :
Total Score: 1832
Memory: 445
CPU Integer: 631
CPU Float: 144
2D Graphics: 109
3D Graphics: 302
Database IO: 20
SD Card Write: 5.8 MB/s
SD Card Read: 12.3 MB/s
An3DBench:
Fillrate ST/MT: 6.23/6.19 MP/sec
High object count: 30.46 fps
Multiple Lights: 39.96 fps
High polygon count: 20.16 fps
Keyframe animation: 40.40 fps
Game level: 30.43 fps
Total score: 4397
CPU @1100Mhz, Performance gov
Quadrant (First run only):
Total: 1001
CPU: 2833
Mem: 1085
I/O: 566
2D: 213
3D: 306
Linpack:
MFlops: 13.917
NenaMark:
16.8 Fps
Benchmark PI:
Pi found in 1460
Antutu System benchmark:
**Would not run at 1100**
Total Score:
Memory:
CPU Integer:
CPU Float:
2D Graphics:
3D Graphics:
Database IO:
SD Card Write:
SD Card Read:
An3DBench :
Fillrate ST/MT: 5.89/6.01
High object count: 17.53 fps
Multiple Lights: 40.22 fps
High polygon count: 20.13 fps
Keyframe animation: 40.37 fps
Game level: 30.44
Total score: 4054
The results speak alot, i think, and yet they don't. The big difference, is that the IO score on Quadrant tanks on the OC kernel, but is fine/better on every other test. Specifically, i noticed that file system writes takes much longer on the OC kernel, than the stock. 3D performance makes obvious gains with increasing clock speed, and other CPU / IO benches show no problem either.
The antutu bench failing at 1.1 is very odd, since my system has never shown any instability at this speed. It crashes almost immediately , where are 1.0Ghz makes it through just fine. Could it be my system is instable? Possibly...
just for the heck of it, i set the gov to interactive, and here is what i got (1100Mhz, OC kernel, Interactive Gov):
Antutu System benchmark:
Total Score: 1089
Memory: 481
CPU Integer: 701
CPU Float: 154
2D Graphics: 101
3D Graphics: 209
Database IO: 10
SD Card Write: 4.9 MB/s
SD Card Read: 10.4 MB/s
who knows....
Thus, I am left with this question: Is the Quadrant bench testing an IO function that no other bench i tried is, or is it testing something in a way no other bench does, and just doesn't like this kernel? Obivously, SOMETHING is going on, becuase the problem is measurable and repeatable. The kernel change showing the problem alludes to a possible issue, but other benches say that the likelyhood of noticing it is minimal.
That said, our device isn't the only one that seems like it has a problem with IO scores: http://androidforums.com/samsung-captivate/136969-quadrant-scores.html
Hopefully, this is a starting point for people, and might even help a dev or two pinpoint what might be happening. I am no expert, but am willing to help where i can.

---

Data Formatting
Thanks for the bench scores. Hopefully its a starting point to understand the issue.
Here is a better looking version of your numbers :
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}

---

Wow.. many thanks for the awesome graph; it really makes things so much neater!

---

Also the guesses that it may be a problem with quadrant may pan out. I always thought the OC kernel seemed snappier as far as user interaction (especially launching the applications list w/ animations.)

---

chisleu said:
Also the guesses that it may be a problem with quadrant may pan out. I always thought the OC kernel seemed snappier as far as user interaction (especially launching the applications list w/ animations.)
Click to expand...
Click to collapse
The user interaction improvement is due to increased CPU clock and 2D performance. IO matters when you install or load something.

---

amtrakcn said:
The user interaction improvement is due to increased CPU clock and 2D performance. IO matters when you install or load something.
Click to expand...
Click to collapse
Except those numbers are inaccurate.

---

chisleu said:
Except those numbers are inaccurate.
Click to expand...
Click to collapse
The performance increase I've experianced with the o/c kernal leads me to agree with your point about the quad benchmark numbers being off the mark. That said, I was wondering if your statement is based on a deeper understanding of what is causing the low io quadrant numbers, and if it is, that you would be willing to share your thoughts. Thanks.

---

I think the statement really is worth looking into - the performance variance should be explored; just because quadrant is the only bench that shows and issue, doesn't mean there isn't one in the system...

---

Divine_Madcat, just wanted to say I appreciated the way you analyzed the issue and presented your findings. I learned alot from your approach. Enjoying your post. I'll send a thanks your way next time I sign in from my web browser.
Sent from my SGH-I897 using XDA App

---

vizographic said:
The performance increase I've experianced with the o/c kernal leads me to agree with your point about the quad benchmark numbers being off the mark. That said, I was wondering if your statement is based on a deeper understanding of what is causing the low io quadrant numbers, and if it is, that you would be willing to share your thoughts. Thanks.
Click to expand...
Click to collapse
I am not making a statement based on personal knowledge, but simply parroting something the guy who manages the kernal builds said.

---

So which should we use?

---

evilmerlin said:
So which should we use?
Click to expand...
Click to collapse
The OC Kernel. The OP was documenting something in detail to try to help out. There is a weird issue causing the OC kernel to show up slower than the stock in one benchmark. It's faster in all other benchmarks. There is probably something wrong with the benchmark.

---

chisleu said:
The OC Kernel. The OP was documenting something in detail to try to help out. There is a weird issue causing the OC kernel to show up slower than the stock in one benchmark. It's faster in all other benchmarks. There is probably something wrong with the benchmark.
Click to expand...
Click to collapse
Thanks for all the feedback guys. Chisleu, i would say you are correct, that it probably is the bench. Yet, there is just this small nagging part of me that wonder if quadrant isn't using something nothing else is, and found a hidden problem. Needless to say, i am not done looking at all this yet.

---

Divine_Madcat said:
Thanks for all the feedback guys. Chisleu, i would say you are correct, that it probably is the bench. Yet, there is just this small nagging part of me that wonder if quadrant isn't using something nothing else is, and found a hidden problem. Needless to say, i am not done looking at all this yet.
Click to expand...
Click to collapse
It is worth investigating though I must say it is low on my priority list. I don't put much weight in Quadrant and especially their I/O tests. Their I/O tests are known to be especially flaky.
When I get a chance, I will go through and remove the tweaks that are not in common with the CM7 kernel to see what is causing it. Unless someone beats me to it. *hint hint*

---

Thank you, Mr. Divine_Madcat! Hopefully you will continue your highly valuable benchmarking work with every significant CM7 nightly and RC, and Froyo/HC, to show the progress and better appreciate the work of our devs.
Quadrant marks peculiarities are, yes, puzzling. And they are not just in their absolute values, but the scatter of these between consequent benchmarkings.
Also, I know it's not the opportune time, but just to get into an understanding of a baseline FPS for OpenGL ES HW acceleration (or lack thereof), it might be worth the effort to do Neocore, at least on CM7 builds.
Thank you.

---

dalingrin said:
It is worth investigating though I must say it is low on my priority list. I don't put much weight in Quadrant and especially their I/O tests. Their I/O tests are known to be especially flaky.
When I get a chance, I will go through and remove the tweaks that are not in common with the CM7 kernel to see what is causing it. Unless someone beats me to it. *hint hint*
Click to expand...
Click to collapse
I got confused going into the kernel code. It looked like you guys only changed 8-10 lines of code from the B&N release. The last CVSystem I've used was CVS. heh. This new fangled "git" thingie is blowing my mind.
EDIT: NM... I wasn't seeing all the commits. Now I get it. Do we have to make config/menuconfig/whatever to setup the kernel, or are all the flags ready to go?
EDIT: Man I have some catching up to do. I remember when menuconfig was hot ****. The last kernel I built was 2.2.something IIRC.
Can't find the .config. Surely it's not hidden?

---

i know that quadrant puts a big emphasis on i/o score. just going from ext3 to ext4 on a archos 101 gave ~800-1000 pts.
scores have been around 2900 on quadrant for a device that feels slower than a galaxy tab.

---

I seem to have more touchscreen lag/miscalibration when using the oc kernal. It's only really apparent when i'm typing on the keyboard. I was using the stock kernal for about a week with no real issues. Is this something anyone else is experiencing? I was going to flash back to stock, but if it seems isolated, and i'll just flash the new nightly and the OC kernal on top of it again.

---

xwint3rxmut3x said:
I seem to have more touchscreen lag/miscalibration when using the oc kernal. It's only really apparent when i'm typing on the keyboard. I was using the stock kernal for about a week with no real issues. Is this something anyone else is experiencing? I was going to flash back to stock, but if it seems isolated, and i'll just flash the new nightly and the OC kernal on top of it again.
Click to expand...
Click to collapse
search for the touchscreen calibration. it's like 1 su/adb command.

---

chisleu said:
I am not making a statement based on personal knowledge, but simply parroting something the guy who manages the kernal builds said.
Click to expand...
Click to collapse
Thanks for the clarification. I take it you are referring to dalingrin. I think I recall the issue being addressed in a thread but I can't remember exactly what was said or where it was brought up. Do you have the threads post number by chance, since any observations on his part are worthy of serious consideration. Just hoping to learn something new here. Thanks in advance.
Web page refresh showed post by chisleu which quoted dalingrin on the io issue. If this was the post You were referring to then please ignore the above request.