Hi, just wanted to check if one can measure speed differences between ROMs, so I started to bench.
Since it's quite time consuming, I need you guys if you're also interested in results, please post them.
Here is the protocol to use (please tell
if you used another one/modified it):
Pre-requisite: Backup, hard-reset, let the extended rom installation and the following soft-reset happen.
Reboot and note down the time it takes to reach:
Windows Mobile loading logo
Today page loaded
Full memory recovered (check in Settings / Memory)
Disable "auto-shutdown if not used", every today plug-ins, and htc taskmanager if present.
Put the phone in flight mode, wait 10 seconds, shutdown your device. Turn it on after another 10s.
Go to Settings/Memory, note Rom and Ram informations. Wait 1 minute, and note the new Ram information.
Connect to ActiveSync in guest mode. Install & launch SPB Benchmark (free for personnal use) : test activesync
Optionnal: Test battery (very interesting to know): no display, mp3 playback ; then maximum backlight, GPRS/Bluetooth/WiFi.
Disconnect from activesync and test storage card. Reboot and launch main test.
(disable built-in applications tests, because they often fail)
View results and report, hard-reset again & enjoy your rom
Please contribute & share your results, either for fresh ROMs, (active) synched ones or fully installed
Below are summarized results for latest ROMs, after hard-reset, today plug-ins disabled:
Crystal v2:
27.98 Free Ram
50s Today loading
1:40 Memory fully available
271.77 Spb Benchmark index
3773.54 ActiveSync index
TNT v5, after HR (Today plug-ins disabled)
29.30 Free Ram
33s Today loading
1:35 Memory fully available
3730.53 ActiveSync index
PDAViet BetaRom:
271.40 Spb Benchmark index
3622.41 ActiveSync index
PDAViet 4.0.0.0.5b ?
269.28 Spb Benchmark index
Here's a first shoot (updated):
.Crystal v2, after HR (Today plug-ins disabled)
Free Rom 39.42
Free Ram 27.98
Boot time:
Welcomehead 25
Today 50
Completed 1:40
Spb Benchmark index 271.77
CPU index 946.84
File system index 113.59
Graphics index 2929.54
ActiveSync index 3773.54
----------------------------------------------
. Crystal v2, fully installed, with cpuscaler o/c (BS default settings, no boost)
Free Rom 11
Free Ram 23.70
while synching 20
once synched 21.80
Boot time:
welcomehead 26
Today appears 53
Today fully loaded 1:13
Memory fully recovered 1:40
Spb Benchmark index 333.15
CPU index 1227.94
File system index 137.73
Graphics index 3844.05
ActiveSync index 2132.93
----------------------------------------------
. TNT v5, after HR (Today plug-ins disabled)
Free Rom 39.67
Free Ram 29.30
Boot time:
Welcomehead 20
Today 33
Completed 1:35
Spb Benchmark index ?
CPU index ?
File system index ?
Graphics index ?
ActiveSync index 3730.53
----------------------------------------------
. TNT v5, fully installed, with cpuscaler o/c (BS default settings, no boost)
Free Rom 15
Free Ram 25
Spb Benchmark index 344.87
CPU index 1267.58
File system index 142.70
Graphics index 3912.60
ActiveSync index 3718.54
Those are other benchs to compare with:
. PDAViet BetaRom
Spb Benchmark index 271.40
CPU index 952.90
File system index 113.25
Graphics index 2987.79
ActiveSync index 3622.41
----------------------------------------------
. PDAViet 4.0.0.0.5b ?
Spb Benchmark index 269.28
CPU index 954.15
File system index 112.20
Graphics index 2917.14
----------------------------------------------
(same) +Wisbar Advance
Spb Benchmark index 240.33
CPU index 910.67
File system index 110.25
Graphics index 2800.08
Great thread Indeed ! This is what I call a scientific comparison good job man. I really wanted to know these myself too. Thanks
it means that TNT V5 is the fastest and has a best graphic performanse?
farzin_silver said:
it means that TNT V5 is the fastest and has a best graphic performanse?
Click to expand...
Click to collapse
The device was overclocked
I'm testing crystal o/c right now and will publish results tonight.
Then I'll wait other members to post their results, whatever rom they have
Updated with Crystal v2 o/c results
My shoot:
.Gullum VII, after HR (Today plug-ins disabled)
Normal clockspeed(195)
Boot time(in seconds):
Welcomehead 21
Today 37
Completed:58
Upload with activesync: 2845ms
download with activesync: 1631ms
Spb Benchmark index 266.87
CPU index 946.76
File system index 111.19
Graphics index 2868.54
ActiveSync index 3841.85
.Gullum VII, Today plug-ins disabled Only battery status on
O/c to 260
Spb Benchmark index 360.42
CPU index 1314.19
File system index 149.41
Graphics index 3865.06
Thanks a lot for reporting, I hope more members will follow.
Please also not that boot time is only *complete* when the memory needed to start the OS has been released. That means you must go to memory control panel applet and wait for memory to increase by around 10 to 15 Mo. My guess is WM6 only releases memory after a timeout. If this is confirmed, every ROM should behave the same and should only be completely operational after about 1:40 (100s).
It seems *to me* that TNT v5 is the fastest and most stable rom I've tested right now, but Crystal v2 looks like it has a better memory management, I guess because of the new way drivers are implemented. It also has a few freezes. Hope we can get a rom that combine those advantages together soon, Gullum, hma4 and others, do you hear me ?
Here are some more benchmark tools to play with:
jbenchmark
3D Benchmark
PocketPC Benchmark
CE Performance
Gx Mark
While I don't have enough time right now, I may release later a complete Benchmark package with automations.
If you'd like to test & report various ROMs using such a package, please let me know
the free ram for crystal is wrong after you soft reset, the device will unload the dlls after few minuets. then the free available memory will become something around 27 Mg.
edit : I saw your last post Apparently you know what you are doing
about TNT rom being faster and more stable, well i'm not sure what we can improve. from my perspective the only thing different is the OEMDrivers. I will make a rom with same oemdrivers and test it later.
Thanks for your answer, I may remove the "most stable" statement later, as I need to test Crystal a bit more. But while I never got any pb with TNT, I got a few freezes since I installed Crystal. But I didn't do a "clean" 3rd-party softwares installation yet, have only put it under heavy loads and tests, so maybe the same would have happen with TNT.
Have a few more tests to complete tonight, then I'll HR and make a clean install, so that I can really test it for a few days.
Then, I'll report back
About speed, both are great. Regarding benchmark results, it looks like tnt is (a bit) faster at executing apps, but that (little) difference is of course not of a big importance. I was waiting for huge OS response-time improvements with the new way you've built that ROM, but I couldn't notice it. I may check that later, too.
It seems that the memory is handled much much better with that ROM. Though I didn't do any particular test, that's just an impression.
Is it because of the new OS build, or because of the new way you implemented drivers, I don't know, maybe you know ?
Regarding what you can improve, well I don't know what's involved in startup times, but I can clearly see BIG BIG differences between ROMs (at least until today is displayed and device usable), so I guess that could be a way to test drivers or the benefits of removing useless OS parts. Just a way to investigate, not a real issue, cause who still cares about startup times anymore if we have a stable ROM ?
updated testing protocol with battery usage, first results may come within a week, please consider to also contribute, we need a few samples
I think boot time is not that important as I don't reset my handphone unless there is problem or low on RAM that can't be free (mostly 1 month once). I'm more concern on usage performance and smooth scrolling of the navigation.
My comment here is after using Crystal v2 for one week, it is definitely much faster compare to pdaviet 4.0.0.6 and any WM5 rom. I try out my friend O2 Atom and Crystal v2 screen navigation is as smooth as O2 Atom. Overall Crystal v2 ROM does help to renew the feel of my handphone.
pierreye said:
I think boot time is not that important as I don't reset my handphone unless there is problem or low on RAM that can't be free (mostly 1 month once). I'm more concern on usage performance and smooth scrolling of the navigation.
My comment here is after using Crystal v2 for one week, it is definitely much faster compare to pdaviet 4.0.0.6 and any WM5 rom. I try out my friend O2 Atom and Crystal v2 screen navigation is as smooth as O2 Atom. Overall Crystal v2 ROM does help to renew the feel of my handphone.
Click to expand...
Click to collapse
Yeah you're right, you all want to get the most reliable WM6 rom I guess, and so do I, that's why this thread is important. I don't really want to get more speed out of a ROM, I'll rather get a more reliable one, too. But, now, how would you compare them ? One could start a "bug requests" thread, but I think it should fit just in here. At least for bugs which aren't solved for now, that way we could get rid of them, at least if someone take care of 'em Please report
Another ROM performance thread I just found out, very interesting.
Maybe its an idea to make a wiki for this? Because its changing by day?
qtek s200, prophet 4.0:
spb benchmark index: 326.99
cpu index: 908.04
fs index: 143.67
graphics index: 2825.59
nice thread. update into directory
hope other can share their roms experience here
Very nice thread !
Related
Hey all,
Just to preface, this is a discussion, not a complaint. I simply want to figure out the cause and effects of different pagepool settings.
I'm not sure about other devices, but from my experience, it seems like the prophet is very "limited" in RAM... One of the biggest pains is not being able to load third party input methods like calligrapher, or htc phone pad (when it's not installed in the ROM).
From information I've gathered here and there, it seems like input methods use a "different" part of the RAM that's shared with the today screen plugins. I'm not entirely sure if this relates to the pagepool, but I have a feeling it is.
So in my quest to figure things out, I've realized the following:
- In the shipped ROMs from the manufacturer, this issue was never a problem. What was the pagepool setting of these ROMs? 12mb? 8mb? 4mb? From the old WM5 threads, it seems like they were set to 12mb pagepool. is this correct?
- In the current set of roms, the cooks like to use 8mb pagepool because the speed is a little better. These roms definitely seem to have the input method problems. ie. pdaviet, pdamobiz, etc...
- Someone mentioned a 4mb pagepool is better for ram with a little sacrifice on speed. I haven't actually tried using a 4mb pagepool rom to see if it has the input method issue. has anyone here experienced it with 4mb pagepool? (maybe those running gullum's 4mb pagepool rom)?
- so in the end, what is the affect of changing the pagepool size in relation to speed, available program ram, third part input methods/today plugins? I'm really confused! Is there even a relation?
mr_yellow said:
Hey all,
Just to preface, this is a discussion, not a complaint. I simply want to figure out the cause and effects of different pagepool settings.
I'm not sure about other devices, but from my experience, it seems like the prophet is very "limited" in RAM... One of the biggest pains is not being able to load third party input methods like calligrapher, or htc phone pad (when it's not installed in the ROM).
From information I've gathered here and there, it seems like input methods use a "different" part of the RAM that's shared with the today screen plugins. I'm not entirely sure if this relates to the pagepool, but I have a feeling it is.
So in my quest to figure things out, I've realized the following:
- In the shipped ROMs from the manufacturer, this issue was never a problem. What was the pagepool setting of these ROMs? 12mb? 8mb? 4mb? From the old WM5 threads, it seems like they were set to 12mb pagepool. is this correct?
- In the current set of roms, the cooks like to use 8mb pagepool because the speed is a little better. These roms definitely seem to have the input method problems. ie. pdaviet, pdamobiz, etc...
- Someone mentioned a 4mb pagepool is better for ram with a little sacrifice on speed. I haven't actually tried using a 4mb pagepool rom to see if it has the input method issue. has anyone here experienced it with 4mb pagepool? (maybe those running gullum's 4mb pagepool rom)?
- so in the end, what is the affect of changing the pagepool size in relation to speed, available program ram, third part input methods/today plugins? I'm really confused! Is there even a relation?
Click to expand...
Click to collapse
Well i used the pdaviet 4.0.0.6 touch then converted it to 4mb pagepool and then used it again here r the changes i observed
-the third party input thing started working gr8 in the beginning for some days but for some weird reason i cannot again switch to blimkeys or any other input again now.
-the speed of windows is a bit slow not too much but opening windows folder now takes a lot of time.
-i have a lot of ram now 25mb in the starting compared to 8mb pagepool which used to give me 18mb.
-it starts a bit faster the home pluggin take less time to come.
applications like games r more faster............
thats all ive observed will try 6mb pagepool too sometime..........
Hello mr_yellow
have you tried to google page pool ?! trust me it helps.
to make it easier , have a look at here :
http://blogs.msdn.com/windowsmobile/archive/2005/11/17/494177.aspx
Room cooks didn't reduce the rom to make the rom faster! Theoretically the Roms with larger page pool are faster. They reduce the page pool to increase the available program ram.
Hope it helped
I've tried.. and failed... with all the different devices, it's hard to figure it all out. The article you linked was *very* helpful though...
So pagepool is basically the "cache" between the storage ram and program ram... That's what I thought. Interpreting that, the size of the page pool also seems to limit the amount of running application. ie. if page pool is only 4 megs, and 1 application needs 3 megs to run, if a second app that needs 2 megs to run starts, will the OS to stop loading the app since there's not enough memory?
So.. do *all* applications share the page pool? in other words, do all the third party input methods like calligrapher and blimkeys and HTC Phone pad load into the pagepool before executing? Same with PIM apps, internet explorer, games, etc....
Doesn't that mean we should increase our page pool size to 12 megs?
Also, to change the pagepool of a rom, i know it's a simple hex edit... Are there any other changes to the ROM we have to make?
Thanks
Sorry to dig up this thread again, so I've flashed to PDAVIEt 40005b cross which has 12 mb pagepool and I don't see any difference regarding the problems with loading 3rd party input methods. I'm trying to load PocketCM Keyboard and I just can't without turning off all my today plugins. and even then, after a few hours of uptime, it'll stop loading all together and I'm stuck with letter/block recognizer, keyboard, and transcriber (somehow transcriber always loads)...
So my workaround so far is to disable most today plugins, and use SIPChange utility that will automatically load any desired input method after reset. You can find the SIPChange utility on the forums.
If anyone else has any good workaround/solutions, please post them!
I have noticed similar issues around phonepad/blimkeys etc
I have just flashed to pdaviet 4.0.0.0.6
again same. blim keys works for a while then dont anymore
sorry about my silly question but
I have a jamin g4 device now tell me when a rom relesed , I should download witch version : 4M , 8M , Or 12M
darvish_boy said:
sorry about my silly question but
I have a jamin g4 device now tell me when a rom relesed , I should download witch version : 4M , 8M , Or 12M
Click to expand...
Click to collapse
If there was one "best" page pool size, I think the rom cookers would have figured it out by now. It depends on what's pre-installed in the rom and what the cooker wants, speed, or ram.
personally, i've tried roms with 8mb and 12mb pagepools, and functionally, there is no difference that I can tell. What you should do is take your favourite rom and edit it so it has different page pool sizes and flash it and try it out yourself and report back here what you find.
THanks,
mr_yellow said:
If there was one "best" page pool size, I think the rom cookers would have figured it out by now. It depends on what's pre-installed in the rom and what the cooker wants, speed, or ram.
personally, i've tried roms with 8mb and 12mb pagepools, and functionally, there is no difference that I can tell. What you should do is take your favourite rom and edit it so it has different page pool sizes and flash it and try it out yourself and report back here what you find.
THanks,
Click to expand...
Click to collapse
thanks for ur answer
Well I finally have my new device and it looks like i might have a little time this weekend to make an update and fix some minor bugs in my rom. please post suggestions here. All suggestions need to be in my friday midnight EST. thanks
How about making these changes to your last ROM?
- 16mb page pool to see if it goes a little faster.
- Do not auto download the quick gps data during the setting phase, since the device does not have time and data connection yet at that phase.
- update the large menu text cab so that it works. I had to look for the cab elsewhere last time.'
- Focus on speed.
- Keep it ultra vanilla. This is your number one selling point for me. I don't want all third party application to be chosen by myself and not seeing duplicated application/old versions/unused application.
- Make sure it is spl1.2. Do not want to lose memory for useless applications.
- If you intent to use newer built (Are you thinking of Touch Pro?), make sure it still supports Gsen application, and has random access for contact application. I personally love these two features.
- Make sure it has no htc touchflo or Ftouchflo. Have Ftouchflosl trial version if you wish. Else leave it for user to decide.
- Again, leaveb out realVGA. Some of us don't use it anymore.
- Continue to leave out HTC Home, and have just one theme. HTC Home They are all add ons.
- An important bug to fix is its inabikity to delete POP3 mails. Currently I can only delete non-POP3 mail, but for my pop3 mail, I'm forced to sign into the web-mail to delete them to prevent mail-box overflow. This is a pain.
Frankly, I'm still so happy with your last ROM, although I feel the speed could be further improved. So if there is only one change, let there be just speed improvement.
Thanks irus. I look forward to your fantastic ROM.
I'm now using Michy's rom whose main merits are speed and stability, but I like your Gsen function very much. I hope you can integrate the above-mentioned features in your new rom.
I would suggest no changing from your latest rom which I think is perfect, only bugfixes.
Changing could be for a newer release while I think your last one deserves just a trimming to be a reference rom. (I can't download full messages from the mail program???)
I would say fix the known bugs, increase speed if possible. After releasing that version, you can spend sometime looking at other possibilities (lite version, other builds or base, new features, etc...).
The lite / vanilla builds never actually worked in the long run for me because it always seemed something was missing which caused other applications not to work properly. They always seemed fast and potentially stable for a few weeks. After installing my apps and heavy usage, it always seemed to have gone down hill. Nonetheless, a lot PPL here love them.
It is also very important that auto rotate, gsen, and other features continue to work because I believe a lot of PPL stuck with this base because that stuff work well with on this ROM.
I never tried your rom mainly because I had the impression it was missing MMS. I know it is fairly easy to install myself, but I hadn't time to look for a proper version.
If your rom contains MMS, then forget about this post
Yes! also mms. I would consider the missing of it nearly a bug to be fixed as it is a standard feature of the athena missing... It is true that most of us don't use mms, but just in case...
ok these are all good suggestions, i will keep them in mind
I currently use your FULL ROM/3.5 and find it to be excellent. I hope you will NOT include any real VGA additions. Every one of them I have tried so far causes display problems with my "must use" apps.
Thanks for all your hard work.
I've tested All Athena ROM available.
I am using now the Sogarth base, isn't the fastest, isn't the most complete, but it is stable, has gsensor, big htc home, audio player doesn't stop working in suspend mode, index bar in contacts, battery life is great WITH BT enabled and some diamond enhancements.
The probem is ... it has to be tweaked
.net35 AND net2.0 to have compatibility with older and newest applications (modaco netcf config), some registry tweaks to improve the speed , michy's sound enhancement are much better, athena sensor service has to be stopped, IMHO are needed:
1) a newest remote desktop,
2) wifirouter (ori ICS control) ,
3) schaps advanced config ,
4) moblue,
5) total commander,
6) irus fsplayer,
7) pocketrar,
8) omnia d3d drivers,
9) Gsen,
10) wkTask (HTC Xbutton removed),
11) cyberon voice dial
12) I need mms too
I don't need realvga, the applications I like could use VGA resolution without realvga! Last, but not least, having and athena ROM CLOSER to diamond ROM should be easy to have compatibility with diamond and raphael applications (the most "developed" HTC model)
I strongly suggest a newest Sogarth, not "Extreme" but improved one (SPL 3.5). and "Extreme Sogarth" lite SPL 1.2 and 3.5 tweaked for maximum speed
irus said:
Well I finally have my new device and it looks like i might have a little time this weekend to make an update and fix some minor bugs in my rom. please post suggestions here. All suggestions need to be in my friday midnight EST. thanks
Click to expand...
Click to collapse
I have forgotten if you use the extrom.
If you don't , could you ask michyprima for his ap to access this .
It is called "Addins"
there's a new base rom popped up in the general forum..
Scratch that I can't find it now..
lakeman said:
I have forgotten if you use the extrom.
If you don't , could you ask michyprima for his ap to access this .
It is called "Addins"
Click to expand...
Click to collapse
I think that app is good but only if you have the loop problem as all my roms are uc compliant, which is the same thing just launches from the sd card not ext rom.
my 2cents: irus in your knew rom maybe you can focus on a new build and speed speed speed.
lennie said:
my 2cents: irus in your knew rom maybe you can focus on a new build and speed speed speed.
Click to expand...
Click to collapse
well i like this build and i don't have the time for a full update but i will def focus on speed speed speed
build 20721 aku 1.4 is out r u planning for this
irus said:
I think that app is good but only if you have the loop problem as all my roms are uc compliant, which is the same thing just launches from the sd card not ext rom.
Click to expand...
Click to collapse
I haven't used the uc app.
does it allow you to choose what you install , or does it install everything in the list?
lakeman said:
I haven't used the uc app.
does it allow you to choose what you install , or does it install everything in the list?
Click to expand...
Click to collapse
it installs everything in the sdconfig.txt file but if you don't want something just delete that line of text. when i release my next version i will put a template of sdconfig.txt but this will only work for people without the loop problem.
Suggessions
Hi Irus,
What about the following:
1- Fnger touch
2- More diamnod drivers, graphics & functions like in ap5
3- check this link and sea if we can avail such functions http://www.touchxperience.com/
Cheers
wsayed said:
Hi Irus,
What about the following:
1- Fnger touch
2- More diamnod drivers, graphics & functions like in ap5
3- check this link and sea if we can avail such functions http://www.touchxperience.com/
Cheers
Click to expand...
Click to collapse
sea = the ocean: "look at the ship way out there in the sea"
see = look: "did you see that?"
hope this helped
I know this has been mentioned in several posts but nothing I have seen compiling a bunch of peoples opinions into one post..So here goes:
Out of the 6.1 and 6.5 ROMs what do you see as being the fastest ROM - fastest as in responsiveness to screen taps, scrolling, animations etc.. And also fast as in how well will it play movies (I know this is a very subjective one as results will vary with how the movie has been encoded).
My results have been:
Fastest - Michy's 6.5 Clean Edition; It seems to scroll very well, load opera fast and when i load up SPB 3 on it the animations to screen swipes are smother that any other ROM. Issues are batt life seems to be a lot less than other 6.1 ROM's. I also notice the Bluetooth seems to not cut out as much with this ROM so that's a plus.
Downside is there are a few bugs still in this ROM (I know Mitchy is and has worked hard to iron them out..and many thanks to him for that!!)
Movies seem to play (for me) better with this ROM as well. the same movie will have less lag and sound sync issues with this ROM..almost like there is a better driver in this ROM for graphics? Also I did not add any codec bundles or anything...just as the ROM came and added CorePlayer..
I still find myself using the Mitchy's Clean 6.1 ROM just due to almost no bugs and longer batt life...almost as fast if you keep as much RAM free as you can with it but still not as fast as his 6.5 ROM I currently have about 62 MB with everything installed that i like to run on it.
Please post your take on what you find is faster for you and your setup.
My setup:
Mitchy 6.1 Clean ROM
Programs Added:
SPB 3
SoftMaker office 2008 - loaded on mem card
Memaid
Calendar Touch
Resco Radio
WMWifiRouter
C-ApiSRO (XM and Sirius player)
Core Player
Google maps
Opera
MSVC
WinMobileTorrent
Numpty (great game!)
After Soft reset:
free Storage - 95.64
free Program - 61.25
for the 6.5 ROM the RAM was about 14MB less with all the same loaded..would love to get all this onto a very clean 6.5 ROM and see how much better the performance can be!
Just wish I had the know-how to cook ROM's!
Please post your results too
I know these forums are getting less and less traffic but I thought I would bring up a thread that would spark some discussion on best configuration.
Let me first thank all the cooks that have contributed personal time into developing some great ROMs, without them the Athena would be only a shadow of the capable phone it is now. Let me also say I'm not a programmer at all and the following observations are just from my personal experience with some conjecture thrown in.
I think the development of ROMs have almost reached its apex in that I don't think we will get any more performance out of these devices until someone can implement the use of the ATI graphics chip. The reason I say this is because of the ROM I have settled on. After testing EVERY ROM (6.1 and 6.5)that is available on this forum I have settled for Michy RO clean 6.1 ROM. Out of all of them this one gives me the most available program memory (70 to 71 MB) and smoothest responce. In my experience because of the lack of the graphics drivers the more free program mem there is the smoother this device seems to run - same true with how well multiple programs will run. My current configuration is:
Michy RO 6.1 clean ROM (all items unchecked in "Today" so totally blank screen save for the task bar. gsensor turned off - thought with it on i see only a small decrease in program mem of about 250KB and still no lag. email set up and running)
SPB 3.5 Mobile Shell loaded (3.5 runs faster with smaller footprint than 3.0 and has a great new widgets implementation) - weather set up to auto update - works great btw!!
Small tweaks to the task bar icons (no effect on performance..just visual enhancements)
A handful of programs installed:
Pandora, Opera, google maps, WMtorrents, wmwifi, XM&Sirius radio player.
everything else is already installed with the ROM or from mobile shell like Adobe reader, MS office etc...
Performance is great, smooth transitions and fast as anything I have tested to date...carousel spins around smooth as butter with 0 lag. Program Memory is at 61MB (with SPB MS loaded and over 70MB with nothing loaded) and I have had 0 issues with bugs. I have found Opera 9.7 build 35541 to be the best running browser..its very fast (especially with turbo turned on) and almost gives me the same experience as the iphone one (my wife's phone). Only down side is that i cant take advantage of the hardware acceleration that SPB 3.5 mobile shell can use for extra eye candy.
Keep in mind i keep my phone in portrait mode almost all the time. I have found no matter what ROM is used if you go into landscape mode everything lags again because of the lack of hardware acceleration.
As far as the newer 6.5 ROM's, i have tried them all as well but as good as they are they still are not nearly as fast as my setup and wind up using more program mem then my configuration.
Please contribute to this thread if you feel moved to do so. Id like to see if any others have found a better setup that give great response and as much or more program memory than I have been able to set up.
Thanks...
This thread is for the submission and viewing of ROM benchmarks. There are two sections of this post: One for the Official ROM Benchmarks, and one for user submitted benchmarks. This will help us get a more accurate representation of the 'Real' benchmark.
'Official' Benchmarks
-----------------------------------------------------------------------------------------------------
These benchmarks are taken by Scabes24. He fallows a more strict set of rules and posts a more exact representation of ROM speeds. Please use this in conjunction with the User Submitted benchmarks to compare ROMs for speed.
This won't be updated constantly,(Will be updated Monthly) so we encourage you to submit your benchmarks in order to keep things fresh around here.
Click here to view the 'Official' benchmarks
User Submitted Benchmarks
-----------------------------------------------------------------------------------------------------
These Benchmarks are submitted by other XDA-Developers users for us to read. These Benchmarks will be up to date (Hopefully), and the rom date will be on the benchmark submission. These spreadsheets will be STRICTLY Moderated so no bogus gets in there. Remember, some of these benchmarks may not be 100% accurate, because it may differ from device to device (but most likely won't). If you need help, just shoot me a PM.
Please Note: The numbers in this spreadsheet are the time in Milliseconds, and not a 'Score'. The lower the number, the better.
Spreadsheet Link (Results)
IF YOU SUBMIT A BENCHMARK, PLEASE "TEST" THE ROM WITHOUT ANY MODS!
Submitting a Benchmark
1. Flash a new ROM and do a hard reset.
2. Go into airplane mode
3. Disable switching screen off and turning the topaz off
4. DO NOT CHANGE ANY SETTINGS!
5. Install SPB Benchmark (Download HERE, and it IS freeware!)
6. Reboot (soft-reset)
7. Run SPB Benchmark : Select the "Main Test" option, and disable external software tests like word and IE or file explorer (whole category)
8. Submit results using this form.
Changes for 'Official' Benchmarks
Code:
[SIZE=6][COLOR=red][B]Changelog[/B][/COLOR][/SIZE]
[B][SIZE=4][COLOR=green][U][COLOR=Black]Changes 9/6/10[/COLOR][/U][/COLOR][/SIZE][/B] [B][COLOR=Red]*[/COLOR]Will be updating over the next few days[COLOR=Red]*[/COLOR][/B]
•If anyone would like any [B]stock/shipped ROMs[/B] tested let me know.
[B][SIZE=4][COLOR=Green][U][COLOR=Black]Changes 7/30/10[/COLOR][/U][/COLOR][/SIZE][/B]
•Added [B]Energy CHT[/B] - [B]DinikGlass[/B] 23673/21911 (July 17) - 23121 (July 21)
•Added [B]Energy[/B] [B]Standard Sense[/B] 23673/21911 (July 17)
•Added [B]Energy[/B] [B]Titanium[/B] and [B]Sense 2.1[/B] 23569/21911 July 14
•Added [B]Energy[/B] [B]Standard Reference[/B] 23673/21911 (July 17)
•Added [B]Energy[/B] [B]GTX[/B] 23121 (July 21)
•Added [B]NhatHoa 4.0[/B] 23128 (July 18)
•Added [B]Dutty's HG R15[/B] 23569/21905 (May 31)
•Added [B]Dutty's Nu Era V1 Redemption[/B] 23128 (July 23)
•Added [B]LouMeiYin Yin.09[/B] 23568/21905 (June 11)
•Added [B]QBUS22[/B] 23569/21909 (July 12)
•Added [B]Melissa v0.3A[/B] 21887 (July 8)
•Added [B]MaryOne[/B] 23128/23569/21907 (July 15)
•Added [B]Tess Prime V[/B] 21907 (June 30)
•Added [B]NullPointer Evo v3.5[/B] 23569/21907 (June 28)
•Added [B]Cheetah 4.3r[/B] 21901 (July 20)
•Added [B]Cancer TIR-2[/B] (July 2) and [B]TIR-3[/B] (July 10)
•Added [B]Avatar IV[/B] 23128 (July 19) and 21911 (July 20 - in progress)
•Added [B]TopazNihon[/B] 21911 (July 19) (in progress)
•Added [B]Googy ROM v3.4[/B] 219xx (June 14) (w/ RAM Cab)
•Added [B]Column[/B] - '[B]Average[/B]' of columns H-Z ([B]NOT an overall score[/B])
•Added [B]Column[/B] - '[B]Sum[/B]' of columns H-Z ([B]NOT an overall score[/B])
•Added [B]Column[/B] - '[B]Overall Score[/B]' - (still calculating)
•Added [B]Column[/B] - '[B]Free RAM on first boot[/B]'
•Added [B]Column[/B] - '[B]Free Storage on first boot[/B]'
And it would be great to have some kind of spreadsheet formula to calculate the rating of each test (average function, maybe?). Just for us to know which ROM is the fastest, overall.
This initiative is great!
Guys, a benchmark is a benchmark, you must try the ROM for a week and decide for yourself.
I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.
Scabes24 said:
I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.
Click to expand...
Click to collapse
you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score
bnm7bnm said:
you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score
Click to expand...
Click to collapse
Yeah I was trying to figure out how to determine what the 'best/max' score would be, going with the current highest/fastest score for each column (or even groups of columns) should work. As far as overall I was thinkin of seperating it into 2 or 3 scores. one for read/write/copy speed and maybe another for graphics.. but you are right, I would need to know which ones effect performance. Also I'm not quite sure how much of an affect the different (21xxx vs 23xxx) builds have on the stats.
I put the average and sum of the columns (except free RAM and storage) for a few energy ROMs, Dutty's Redemption, and Cheetah to see if it they were comparable and accurate to performance but havn't looked at them enough yet, most likely meaningless.
ThaDeanesta said:
I will do this. First, however, we need benchmarks to make the score. I will handle everything =].
Click to expand...
Click to collapse
Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).
mondilv said:
Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).
Click to expand...
Click to collapse
Yes, this is true. There is a special formula for this. Piertrucci knows this, so it is a good idea to ask him instead of doing the same work twice.
i have scabes24 doing everything on the same device
Hello!
When you do a benchmark with SPB Benchmark this generates an XML file that can be uploaded on the website of SPB and compare with other devices. It also lets us do a custom table of devices and gives the% of one over the other in each test.
So if you still have all generated XML can be loaded into SPB under section Benchmark>Visualice Test Results(on the left column) and will be much easier to see which is better.
I'm Spanish my good english mercy to Google
LLKS said:
Hello!
When you do a benchmark with SPB Benchmark this generates an XML file that can be uploaded on the website of SPB and compare with other devices. It also lets us do a custom table of devices and gives the% of one over the other in each test.
So if you still have all generated XML can be loaded into SPB under section Benchmark>Visualice Test Results(on the left column) and will be much easier to see which is better.
I'm Spanish my good english mercy to Google
Click to expand...
Click to collapse
good idea.
I actually do have most of the XMLs, ill take a look. I was also thinkin about that a while back.
Hi!
I was lookin how the sistem rates and I think that the results all are in milisecond. Then the lower of all sums is the speedy ROM.
BUT I think som results have more weight in the result.
For example Arkball test measures ms between. An speedy ROM has a resul of 3 and a slow one has 12. But this is nothing wen compared to the time spend in list 2000 files (2-3 thousand)
If we want to compare the scores with a formula i think we have to determine how make som values weigth in more than others.
Sorry for my english
LLKS said:
Hi!
I was lookin how the sistem rates and I think that the results all are in milisecond. Then the lower of all sums is the speedy ROM.
BUT I think som results have more weight in the result.
For example Arkball test measures ms between. An speedy ROM has a resul of 3 and a slow one has 12. But this is nothing wen compared to the time spend in list 2000 files (2-3 thousand)
If we want to compare the scores with a formula i think we have to determine how make som values weigth in more than others.
Sorry for my english
Click to expand...
Click to collapse
yes, the user of xda, pietrucci had the formula, but i think it is possible to make a new one, with only logic and ponderation of all the factors, or the most important at least.
Yeah I tried contacting pietrucci a while ago with no response.
We discussed this before, the simple way would be to mark the highest and lowest scores for each column and place them on a grading scale from 1-100 (1-X). Could be done without the XML docs. Then graph it ..or just grade each ROM with a number.
But yeah we would have to figure out which tests are relevant to speed. I personally don't know enough about each specific test to figure that out. Maybe ThaDeanesta would know a bit more.
Scabes24 said:
Yeah I tried contacting pietrucci a while ago with no response.
We discussed this before, the simple way would be to mark the highest and lowest scores for each column and place them on a grading scale from 1-100 (1-X). Could be done without the XML docs. Then graph it ..or just grade each ROM with a number.
But yeah we would have to figure out which tests are relevant to speed. I personally don't know enough about each specific test to figure that out. Maybe ThaDeanesta would know a bit more.
Click to expand...
Click to collapse
yes of course.
well i think one of the importants are the "copy files" times? for example...
i hope all the people post what they think is relevant to try t make a simple formula that helps in a single view of the numbers to decide. maybe not accurate, but more simple than now.
i know all the data are important but it is kaos ....
well, i have decided my rom for long time ago so, it is basically for helping people here
lmemperador said:
yes of course.
well i think one of the importants are the "copy files" times? for example...
i hope all the people post what they think is relevant to try t make a simple formula that helps in a single view of the numbers to decide. maybe not accurate, but more simple than now.
i know all the data are important but it is kaos ....
well, i have decided my rom for long time ago so, it is basically for helping people here
Click to expand...
Click to collapse
Yeah other users input would be helpful. I'm going to finish updating the latest releases and then I'll mark the highs and lows. Then we can move foward from there
Scabes24 said:
Yeah other users input would be helpful. I'm going to finish updating the latest releases and then I'll mark the highs and lows. Then we can move foward from there
Click to expand...
Click to collapse
yes of course. the people may be invloved in this
*****************************************
i have an idea, can we make a survey to let the people vote for the prefered rom, like this user did in this thread to decide the build?¿?
http://forum.xda-developers.com/showthread.php?t=722602
at the top (view poll results)
also it can be done by google forms
you know, all the peolple said that numbers are numbers but the overall experience with a rom is a fact. so.... they could vote for this.
as we try to improve the benchmarks view.
hope this helps
Things i think are important and why:
Free RAM: If you try to surf the web without memory will go slow. Also Sense will work slow
Directory list of 2000 files: Because if you enter explorer with a slow ROM in this test and you want enter Windows directory it freeze the sistem. Is a shame for that ROM.
File Explorer large Folder List: Same as above.
Arkaball frames per second: For thos who want gaming and may be ¿video?. It depends on the tech used in the game (hardware acceleration¿?)
Things to determine:
Read-Write-Copy: To determine how important it must first determine where copying, whether in RAM or SD.
CPU Test: I think these aren't important because results are very pretty the same.
These are my thoughts.... may be wrong