I know that 3.0 is mostly UI improvements but I'm wondering if Google updated the JIT or made any other performance improvements with Gingerbread.
I know there was a CES presentation and we have a gingerbread ROM already... does anyone know if there were speed improvements made?
Oh I got 3.0 and gingerbread confused. I guess I'd be interested in hearing about both! lol
edit:
JVM speed. For Java developers, 2.3 adds a number of speedups, most notably a concurrent garbage collector. According to Google garbage collection pauses will be under 3ms, which is small enough not to be noticed in a 30fps or even 60fps game. New JIT optimizations make Dalvik code run even faster than before.
I remember when 2.2 was coming out we were able to use the JIT on our 2.1 ROMs... will we be able to backport 2.3's JIT?
FWIW, I compared CELBv3.9 to KGBv8 using Linpack @ min=max=748Mhz
(governor set to performance in both cases, but that shouldn't have any effect when min=max)
If you recall, that particular benchmark was/is pretty sensitive to whether JIT is operational.
The results? Over about 10 repeated trials each,
CELBv3.9: 4.75 - 5.05*
KGBv8: 4.68 - 5.28
There is an observable, but modest increase in the peak (maximum) value of the benchmark score in KGBv8 - roughly a 5% increase. But at the same time, note that the "scatter" in the benchmark actually worsened. I take that to mean that in real-world performance, it is unlikely that you will be able to notice a perceptible difference. Hard to know, really - synthetic benchmarks are pretty crappy predictors of the "pleasure of use" for an arbitrary application mix.
* Peak quadrant score was 429 for CELBv3.9 @ 748Mhz - quadrant currently fails in the 3D graphics tests in KGBv8
Thank you bftb0, very interesting results.
I think it's fair to attribute that scatter to the lack of optimization currently in place with that ROM. It is nice to see that it is capable of more though. Certainly when you use that ROM it's easy to see that the stable 2.2 ROMs are still going to give you better real world performance.
And idea as to whether we can backport the 2.3 JIT?
so maybe this is a newbie question but I was wondering what everyone uses as a benchmark or if the modding community as a whole finds benchmarking archaic or silly. I downloaded cf-bench and passmark performance last night, any feedback on these two benchmarks, or recommendations on new ones, or even reasonable condemnation of benchmarking in general is welcome.
JohnnyKushi said:
so maybe this is a newbie question but I was wondering what everyone uses as a benchmark or if the modding community as a whole finds benchmarking archaic or silly. I downloaded cf-bench and passmark performance last night, any feedback on these two benchmarks, or recommendations on new ones, or even reasonable condemnation of benchmarking in general is welcome.
Click to expand...
Click to collapse
Benchmarks take the human factor out of the equation, and what we really want is a snappy response and smooth transitions (the human experience)... neither of which can be benchmarked.
I'm partial to Quadrant Standard myself.
I love the aesthetics of Quadrant more than most others but like the functionality of AnTuTu. I know the results really don't translate to everyday performance all the time with these apps but find myself using them nonetheless. Again, the cool graphics is probably why
All benchmarks are different, and none are completely indicative of real world performance
What do benchmarks actually measure? I always make allkinds of tests to the phones i have just for fun and to compare to others i have understood that a bigger score is always better..?
Benchmark test with antutu shows for me
10372
I have a s3 16gb and using the latest stock ics..
Sent from my GT-I9300 using xda app-developers app
CPU tests i guess is based on really long loops and depends on time that loop take you get result. RAM tests I think write big file direct to memory and depend of time get you speed of RAM memory in MBps or GBps...
On Antutu I have about 14,9K-15K with Paranoid and Siayah kernel. On Quadrant i have about 5,7K
Sent from my GT-I9300 using xda app-developers app
e-peen.
Seriously, different benchmarks measure different types of system performance by subjecting different elements of the device to different stress and performance tests. CPU calculstions, frames per second, javascript performance - they're genuinely useful in some situations, such as for devs to test out different approaches and ideas to see if one works fundamentally better than the other, and if done properly it can be useful to compare raw performance of one device to another, but they've cottoned on in the wider community because some people like to benchmark their phones and post those benchmarks proudly.
It's all a bit ridiculous, especially if - indeed - they don't know what they're benchmarking, or they're using out of date and largely useless benchmarks like quadrsnt, or they're just doing it for the bragging rights. It's relatively easy to push your phone and have it score well in assorted benchmarking tools if you really want to, it's just unlikely to be genuinely useful in that state - for example, setting the CPU governor to 'performance' had the effect of boosting it up to the highest available clock speed, and of raping your battery life. These are mobile devixes, not gaming PCs, and are designed to operate accordingly.
Personally, I ignore people's benchmark posts, because if they're posting benchmarks they're either brsgging, complaining about something absurd - "Your kernel doesn't perform as well as someone else's when in this unrealistic benchmarking scenario! It may best optimised for daily use, but now I can't hold my precious benchmarking score above everyone else's heads!" or "This new software update sucks because I don't get as high as score in Wankermark Extreme! [twenty sad faces]."
Hope that helps.
Sent from my GT-I9300 using xda premium
onslaught86 said:
e-peen.
Seriously, different benchmarks measure different types of system performance by subjecting different elements of the device to different stress and performance tests. CPU calculstions, frames per second, javascript performance - they're genuinely useful in some situations, such as for devs to test out different approaches and ideas to see if one works fundamentally better than the other, and if done properly it can be useful to compare raw performance of one device to another, but they've cottoned on in the wider community because some people like to benchmark their phones and post those benchmarks proudly.
It's all a bit ridiculous, especially if - indeed - they don't know what they're benchmarking, or they're using out of date and largely useless benchmarks like quadrsnt, or they're just doing it for the bragging rights. It's relatively easy to push your phone and have it score well in assorted benchmarking tools if you really want to, it's just unlikely to be genuinely useful in that state - for example, setting the CPU governor to 'performance' had the effect of boosting it up to the highest available clock speed, and of raping your battery life. These are mobile devixes, not gaming PCs, and are designed to operate accordingly.
Personally, I ignore people's benchmark posts, because if they're posting benchmarks they're either brsgging, complaining about something absurd - "Your kernel doesn't perform as well as someone else's when in this unrealistic benchmarking scenario! It may best optimised for daily use, but now I can't hold my precious benchmarking score above everyone else's heads!" or "This new software update sucks because I don't get as high as score in Wankermark Extreme! [twenty sad faces]."
Hope that helps.
Sent from my GT-I9300 using xda premium
Click to expand...
Click to collapse
200% agree but it wont stop the what is your quadrant score posts .
jje
it wont stop the what is your quadrant score posts .
Click to expand...
Click to collapse
Anyone good with Photoshop? Create a graph that reads 999'999'999'999'999'999 for your phone of choice and paste it under other people's claim.
It's neither more useful nor more useless than what they do
Benchmarks can be useful for yourself to check very basic performance before and after making changes (rom, kernel etc)
Sent via TCP/IP
gsw5700 said:
Benchmarks can be useful for yourself to check very basic performance before and after making changes (rom, kernel etc)
Sent via TCP/IP
Click to expand...
Click to collapse
Possible but run ten benchmarks one after the other and you get different results .
But as a method of a basic test to see if firmware has brought something extra to your phone yes .
Two areas of concern for users are always one use how the phone responds scrolls etc and that's subjective .Two is battery life and none of these benchmarks actually run a standard test that could be used to give a basic battery consumption figure over say a two hour period . But even that's not a good indicator of individuals battery usage .
jje
Hi guys,
I read many post that high benchmark score is just for fun or something like that. What matter is your daily usage, if it is smooth and responsive enough, then you can ignore benchmark result.
I am not completely agree with that. For me, benchmark score shows how efficient your ROM + kernel works.
If you get low score and not feel laggy, it means that you use your phone light enough, so even poor score doesn't matter for you.
For ex, when you open an excel file with a small data and simple calculation, it won't make much different whether you get lower/higher score in benchmark, it will open instantly. But when you open large excel file with complex calculation, then you will see the different.
Another example is when you run 3d benchmark with Antutu, with some ROM + kernel, it got around 25-30fps, another got 50-55fps. For gaming or app that use high def video a lot, surely, this will make a different.
Now, this is my experiences
1) On some ROM + kernel, I got Antutu's score around 22-23K, OC to 2496MHz
With this ROM + kernel, for daily usage, I don't feel any lag
2) On another ROM + kernel, I got Antutu's score around 32-35K, without OC
Since it has better score, I set it UC to 1.7Mhz and UV 100mAV and don't feel lag at all
So with a better score, I can UC and UV without effecting performance and the result is better battery life and if needed for heavy app, I can set it back to normal clock/volt to boost performance.
Regarding of this, choosing a ROM + kernel is not only based on performance, and I really-really appreciate ROM and kernel developer, they give ROM, kernel and times for free, hat's off for them, so
don't ask me with ROM + kernel that give best result
And then when you change just the kernel, what do you do?
Stock kernel, stock rom.. Antutu = 33K
Other kernel, stock rom... Antutu = 29K
Both at completely default settings. They both feel the same.
According to your theory, I should be able to UC and UV the stock kernel more then the other kernel, because it has a higher benchmark. But in practice, I have brought the other kernel down to 1.7 with no change in feel for actual usage. Bringing the stock kernel down to the same settings, there is more lag (as much as 3 seconds) opening some programs. Especially things that use the internet.
So the kernel that has the lower score can be underclocked more then the ne with the higher score. So the results are the opposite of what was predicted.
Things like changing TCP congestion to westwood have an effect on the responsiveness of some programs that isn't tested in the benchmark programs.
benchmarks really depend on the kernel...
I also experimented with different kernels.
I think benchmarks are for bragging right. some manufacturers have been caught fudging their benchmark scores.
So for me its not really that important.........
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
souleman said:
Things like changing TCP congestion to westwood have an effect on the responsiveness of some programs that isn't tested in the benchmark programs.
Click to expand...
Click to collapse
Hi,
TCP congestion have nothing to do with "responsiveness of some programs" like you said, nothing like this (nor speed, smoothess) is improved when you change the TCP congestion...
It's only about network stuff... http://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm.
So how the TCP congestion could affect the responsiveness of some programs???
For sure also nothing improved in benches like Antutu, Quadrant, etc... when you change the TCP congestion...
viking37 said:
Hi,
TCP congestion have nothing to do with "responsiveness of some programs" like you said, nothing like this (nor speed, smoothess) is improved when you change the TCP congestion... For sure also nothing improved in benches like Antutu, Quadrant, etc... when you change the TCP congestion...
It's only about network stuff... http://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm.
So how the TCP congestion could affect the responsiveness of some programs???
Click to expand...
Click to collapse
How many programs on your phone use an internet connection? Like any messenger program, many games, etc etc.. How many sync to an online server? Like gmail, facebook, etc.
So changing the TCP congestion algorithm can help programs that use the Internet because they are able to access the online information better.
Ok.
You should have explained this in your first sentence, this is confusing
It's not what I (we) call "responsiveness" of programs, but maybe it's just me :angel:
I said that because I've even read from some users that TCP congestion improves speed and smoothness in general UI
Debate closed for me.
Speeds don't always = smoothness. Some roms are built for pure speed that take away from day to day memory pools to help push graphics.
Numbers are a good reference for overall phone speeds compared to others but take it with a grain of salt.
htcm7 said:
Hi guys,
I read many post that high benchmark score is just for fun or something like that. What matter is your daily usage, if it is smooth and responsive enough, then you can ignore benchmark result.
I am not completely agree with that. For me, benchmark score shows how efficient your ROM + kernel works.
If you get low score and not feel laggy, it means that you use your phone light enough, so even poor score doesn't matter for you.
For ex, when you open an excel file with a small data and simple calculation, it won't make much different whether you get lower/higher score in benchmark, it will open instantly. But when you open large excel file with complex calculation, then you will see the different.
Another example is when you run 3d benchmark with Antutu, with some ROM + kernel, it got around 25-30fps, another got 50-55fps. For gaming or app that use high def video a lot, surely, this will make a different.
Now, this is my experiences
1) On some ROM + kernel, I got Antutu's score around 22-23K, OC to 2496MHz
With this ROM + kernel, for daily usage, I don't feel any lag
2) On another ROM + kernel, I got Antutu's score around 32-35K, without OC
Since it has better score, I set it UC to 1.7Mhz and UV 100mAV and don't feel lag at all
So with a better score, I can UC and UV without effecting performance and the result is better battery life and if needed for heavy app, I can set it back to normal clock/volt to boost performance.
Regarding of this, choosing a ROM + kernel is not only based on performance, and I really-really appreciate ROM and kernel developer, they give ROM, kernel and times for free, hat's off for them, so
don't ask me with ROM + kernel that give best result
Click to expand...
Click to collapse
there is no any advantage .. this bench crap thing is just gimmick makes children happy when they have higher score than others..
Using your theory that means many high end android phones a year or two ago should have been much smoother than the iPhone which clearly was not the case
Sent from my LG-D802 using xda app-developers app
shadehh said:
Using your theory that means many high end android phones a year or two ago should have been much smoother than the iPhone which clearly was not the case
Sent from my LG-D802 using xda app-developers app
Click to expand...
Click to collapse
that is not theory ! that is my state !
u need to start read more 'bout benchmarks tho .. it is app (program) for stupid people naive and pathetic .. well smart people making money of stupid people
http://www.anandtech.com/show/7384/state-of-cheating-in-android-benchmarks
http://blog.gsmarena.com/almost-all-android-makers-cheat-on-benchmarks/
It's only my opinion (maybe read closely to get whole point)...
viking37 said:
Hi,
Instead of always say "benchmark means nothing", "benchmark are useless", "benchmark are not real life" (obviously), bla bla bla... Maybe the OP could know why his score are low or why there is too much difference?
Like for some other guys if almost everyone have, it's an example, about 35 000 in Antutu and one guy post because he have only 20 000, I think he could know why, what is wrong, etc..., right?
But no, each time there is always the same kind of "ready repplies" like above...
Benches can be useful if you know how to run them (and yes there is a manner to run a bench like CPU temp at the beginning, with data active or not, after reboot or not, etc...) and how you interprete them (the most important, at least for me), even Quadrant can be useful for a quick bench but it's just my opinion (ok Quadrant seems to have more variations than Antutu)... And i don't manipulate them (like with the FPS unlock, I mean the debug.egl... build.prop edit) after it's for fun too.
And if I tweak my device with VM, governor, i/o scheduler settings for example (and it is not manipulating, not for me, it's some real "tweaks" that you can judge too in real use, in general UI, user experience (improvement or not) and not only in benchs) I see in first in general UI and after I look in bench to see...
Yes don't get these benches as a scientist approach or the best accurate result or a result to garantie you a general user experience (bad or good), for example my main goal is about general UI (smoothess, speed, responsiveness) and after I play with benchs because it's always fun...
But the tendance, here on XDA, is to ban benches with always the same words and the same mind (because xx says benchs are useless, xy repeat the same thing without knowing anything or xx have a bad/low score so benchs are useless, etc, etc, etc...), it bothers me... (it's not directed to you in particular).
And in general the guys who post a result of bench post a screenshot of the overall score and nothing more... The most important is to know how you got this score (governor settings, rom, kernel, "tweaks", etc...) and the detailed score is also important.
A bench makes sense with explications and details on how and why you have this or that score.
I agree that a screenshot without information means nothing..., I'm the first to say that in different bench threads but people don't care and continue to post a screenshot only, it's absolutely ridiculous, but well...
It's like the battery stats or screen on time (and we have at least to thrads for that...), it's a kind of benchmark, but here people agree more with these (?) and same thing: 2 or 3 screenshots without any information (settings or usage) means nothing...
In preparation of car engine you run benchs (for the raw power) and you test also your car on street to see the comportement in general not only on a bench and also ther is some conditions to respect before run a bench like for a phone. It's the same thing for a device if you like tweak it, you tweak your device, you run bench to see the result, you use it in day to day use to see if there is an improvement or not (general UI, battery).
It's the same thing for a sportsman, the 100 m, marathon, etc... it's a kind of bench, you don't use it always in "real life" nor it suggests your real state but people don't shout against, right? But benches for phones, welcome to the drama... it's pathetic.
"It shows benchmark apps are no different than a random number generator...", false, I posted some Antutu scores in the benchmark thread (testing i/o scheduler) and I can say you I have accurate results (just keep in mind I was testing different i/o schedulers so the overall score varies a little) in 35 800-36100 range, nothing with 2/4 000 points difference. I can show you accurate results in Antutu with always the same settings like 36 000-36 100 range..., is it a random number generator? No, I don't think... You will always have a little variation, if it's a variation of 5 000 points then something is wrong (thermal throttling mostly, an app auto update, who knows?).
Things to retain: detail your scores, explain your setup and yes don't get the result as a condition on how your phone runs in day to day use, that being said I don't know where is there a problem with benchs on phone... and sometimes fun is nice...
Ok, end, to the OP it's probably due to thermal throttling, but you give very little information... so hard to say exactly but it seems you ran Antutu one after another?
You need to let your phone cool down between each run...
Sorry for this post but I had something to say... and yes I assume totally
Click to expand...
Click to collapse
I'll have one or two things to add but it will be even too long (even some kernel (Elite)devs uses benches to test some stuff, even some Elite devs who claims that benches are useless develops bench apps, etc...), and the general idea should be here I hope...
And stop insulting with "stupid people" please...
I have the LS 980 and just flashed OpTimuS DE 1.5 with Furnace kernel. I just ran Antutu for the first time and got 20396. The chart shows the LG G2 over 30k. So I ran a second time and got 9907. This isn't right.
The only benchmark you really need is you. Only you know what you want from your phone, how you use it and where and which part of it you need the most.
This benchmark pissing contest is more and more boring annoying with every passing day.