MsgQueue Map File Microsoft Status Message SMS Protocol - MDA, XDA, 1010 General

Hello all,
i have around 10 Files with the name above in my XDA some of them with a size of 260k and some with 1,65Mb. What are they good for or where are they from.
I can´t delete them and I don´t see them in the memory usage.
regards
miba

Huge MsgQueue.. files in Windows directory
I have the T-Mobile Pocket PC with just the built-in 32MB of RAM (256MB SD card on order...)
In my case there are a bunch of MsgQueue-blah-blah files in the Windows directory, 7 of which are 1.65MB each. Plus a 3.83MB file called "rsupgrade.cp64". That's about 15MB.
Unfortunately I don't have an answer, only an additional question... hopefully someone can answer us both.
Are these eating into my RAM or am I seeing files from ROM? I suspect RAM since they change periodically. Are they critical or can I delete them? If the latter, is there a way to limit their size or prevent them from reappearing?
T-Mobile Customer Care level-3 support was relatively clueless as usual, but they did suggest that I could do the following to determine how critical those files were:
1. Perform a backup
2. Do a hard reset
3. Check and see if the offending files went away
4. Restore from backup
If the files went away, then supposedly you should be able to delete them yourself if/when they reappear later. But realize that's just the CC guy's theory, and he didn't sound too sure of himself. I'm sure the regulars in this forum could provide a much more intelligent analysis.
Any help?
Thanks.

(some?) T-Mobile MDA's have an rsupgrade as part of ROM, as they perform a phone upgrade as part of a ROM upgrade. Which has really surprised a few people with the 'wrong' phone (900/1800 as opposed to 900/1900) as they then had to struggle to get phone functionality back.
You can try deleting the file, it no longer serves a purpose after the phone is upgraded. But if it won't delete, I would suspect it's in ROM, and in that case it's not bothering you.

Odd
Emboldended by your advice (Peter Poelman) I prepared to delete the rsupgrade file by first backing up through ActiveSync. However, after doing so, the offending file was gone. Stranger yet, I don't think I reclaimed any memory as a result... (hadn't recorded "before" amount yet - was going to do that after backup. But I think I would notice a 4M difference)
Well, either way, that's one down... Could you (or someone) comment on the MsgQueue* files? They all seem to be SMS related. How MicroSoft could justify allocating nearly *half* of the unit's available ram for some SMS caches is beyond me... Well, for M$ I guess it's not surprising, given the alleged unholy alliance between M$ and the RAM industry.

these files are actually in ROM, not RAM but it's impossible for the end user to know that from the properties of the file. Likewise, according to microsoft, those files are not reporting what is actually being used, it may say 1.7megs but it's not really. Per MS, do not try to delete these files as doing so would render the device broken. I think they have to do with the messaging portion of the PPCPE.

Thanks
Useful info, thanks.

Related

Why is there no CLEAN/LEAN/STABLE builds?!

Sorry to make a whole new thread about this, but just curious if anyone else has been having this 'issue'
ive flashed quite a few of the roms on the forums, and for some strange reason, i cant seem to find a rom which just WORKS
the main thing i use my handset for is email and SMS messaging, in any given month easily 4000+ SMS' will be sent and the same number received.
this for some reason appears to 'break' a lot of the rom's
the phone slow's riiiight down and when exiting one conversation to go to the next, it hangs every single time, when i go to select the messaging application, it just hangs, or doesnt even go in at all.
i've used the Energy roms, the Chrome roms, Da_G's clean rom, NATF's roms
i've been through pretty much all of the rom's on here and they ALL break once the SMS count gets in the 3000+ region, and i appreciate its simple to just 'delete' the messages, but this isnt particularly practical, at the moment i'm having to hard reset every month or so and this really takes the piss
can anyone point me in the direction of a nice lean rom that can handle an obscene amount of SMS' and emails?
with thanks
I guess no one send and receive 4000+ SMS' per month except you
that's why nobody tried to check with this issue
but the phone should be able to handle it! i mean, one of my older nokia's can handle the 4000+ with ease
The microsoft SMS program just cannot handle that many of messages. Either back them up (remove from inbox) periodically, or delete them periodically.
you may want to disable sms conversation (threading).
just out of curiosity, why do you need 4000+ sms a month ?
one question... where are your messages stored? in phone memory or on the storage card?
Pampilius86 said:
one question... where are your messages stored? in phone memory or on the storage card?
Click to expand...
Click to collapse
for the email, the attachments are stored on storage card
otherwise, i never knew we could change where SMS's are stored?!
the 4000+ sms's are just what i use tbh, thats on a good month tho, usually hovers around that mark however
i remember back in the wizard/hermes time clean rom's were all the rage, now, since we've got the devices with huge amounts of ram/rom all we're seeing is bloated rom's!
bursucul said:
you may want to disable sms conversation (threading).
just out of curiosity, why do you need 4000+ sms a month ?
Click to expand...
Click to collapse
the thing is, i use the threading facility a LOT, back before wm6/6.1 i actually had an application i would install on my ppc to have the SMS' threaded
but to be honest, why should the tp not be able to handle that many SMS's without slowing down?!
duke_stix said:
Sorry to make a whole new thread about this, but just curious if anyone else has been having this 'issue'
ive flashed quite a few of the roms on the forums, and for some strange reason, i cant seem to find a rom which just WORKS
the main thing i use my handset for is email and SMS messaging, in any given month easily 4000+ SMS' will be sent and the same number received.
this for some reason appears to 'break' a lot of the rom's
the phone slow's riiiight down and when exiting one conversation to go to the next, it hangs every single time, when i go to select the messaging application, it just hangs, or doesnt even go in at all.
i've used the Energy roms, the Chrome roms, Da_G's clean rom, NATF's roms
i've been through pretty much all of the rom's on here and they ALL break once the SMS count gets in the 3000+ region, and i appreciate its simple to just 'delete' the messages, but this isnt particularly practical, at the moment i'm having to hard reset every month or so and this really takes the piss
can anyone point me in the direction of a nice lean rom that can handle an obscene amount of SMS' and emails?
with thanks
Click to expand...
Click to collapse
I dont think this would be the ROMs fault. You see the more messages you store in your fone the less memory and stability it has. Instead of having old messages in your fone why not delete them? Talk about practicality, whats practical about having old messages in ur fone just to take up excess memory? So someone texts u one question, u answer it and have no intentions of messaging them back, yet its more practical to keep these messages in ur fone and then complain that u have to hard reset instead of deleting a few messages due to having all these redundant mesages that u really dont need? Seems to me an intelligent person would rather delete a few messages rather than hard resetting their device and lose everything. Maybe u can make a fresh ROM with no extras and nothing more than a fone and an email device. I mean, why have a PDA if all ur gonna do is text and email? Seems to me u can save a few hundred bux and just get a cheap lil fone that has no customization whatsoever to leave all the room for messaging? Sorry if I sound like I have an attitude, honestly I dont, I just think that the chefs u mentioned have awesome ROMs that keep getting better and better yet u diss them cuz ud rather hard resest and lose everything rather than simply deleting a few hundred messages that u dont need anyway.
It's like if you don't clean your house regularly and just throw your rubbish everywhere, then you complain that you're living in a pig sty and your house fills up with rubbish so fast you can't move around the house as quickly as you used to. Then after a month, you need to fully renovate the whole interior of the house again so that you have space to walk...
My suggestions :
1) start learning some housekeeping(make that little effort to delete the sms yourself)
2) get a bigger house (new phone with more ram and faster CPU)
3) get a cleaner to come in to clean your rubbish once in awhile (there might be some sms program out there that can auto delete your sms on a regular basis)
4) or reorganise the layout of your house so that it's more efficient (forget about sms threading and use the conventional format, I'm sure it'll speed things up a bit since it doesn't have to load the entire conversation everytime)
duke_stix said:
but the phone should be able to handle it! i mean, one of my older nokia's can handle the 4000+ with ease
Click to expand...
Click to collapse
Is your older nokia using sms threading and displaying the whole history of the conversation everytime you enter the inbox?
I highly doubt it.. so you've got your answer there already.
I used to be a project manager, I would send about 500 emails a day at work easily. After a month of working it would take FOREVER to open outlook. Thats because it has to load the entire contents of 10k + emails. Its the same with your phone. Back up and delete or don't complain. Flash a stock ROM and open your windows folder, and then flash a custom ROM and do the same. you will see the speed increase there. Also, if you leave SMS open and just hit end key it wont have to reload everything every time you open a message.
I have the same problem that your talking about since i average about 7-12K a month of sms.. Like everyone said the only way to deal with it is just to clear your inbox or put it to unthreaded.. Our phones just werent meant to hold that much sms i suppose..
duke_stix said:
the main thing i use my handset for is email and SMS messaging, in any given month easily 4000+ SMS' will be sent and the same number received.
this for some reason appears to 'break' a lot of the rom's
the phone slow's riiiight down and when exiting one conversation to go to the next, it hangs every single time, when i go to select the messaging application, it just hangs, or doesnt even go in at all.
Click to expand...
Click to collapse
I believe most of the problem here is that you don't precisely understand what's going on behind-the-scenes. Your CPU/processor is only capable of running a single instruction at once- and thus only process a finite amount of data per unit time.
When designing an algorithm (in this case, the algorithm that reads SMS/E-mail database entries and displays them onscreen), tasks which apply to more than one or two items are usually completed by either iteration or recursion. For a task such as enumerating SMS messages onscreen, the process would be iterative; for example, if you were to break it down into simple steps, those steps might be:
1. Read the current SMS message from the database.
2. Display it on screen.
3. If there's another text message after this, repeat this process for it.
Step 1 itself actually is composed of several detailed steps, which take the time to perform a second algorithm, which tries to locate the current SMS message in the database, usually based on a unique identifier and a hashing algorithm. This takes time, but is faster than the alternative- which is to check each SMS and ask "is this the one I want?". Instead of having to check each message, the device usually only has to check a few until it finds the right location.
You can think of this as the device automatically categorizing messages into virtual 'boxes'- you have to spend a hell of a lot less time digging through a box to find 1 document out of 10 then by searching through a much larger heap of 4000 documents.
Step 3 is also composed of several steps, as the device has to figure out which SMS message is really next. As the user is capable of configuring the way in which the messages are sorted (by date, sender, etc.), the message that's next in the database is probably not the next message that's stored in the database. Time has to be spend finding the message, usually by yet another algorithm.
The problem is, each of these steps, and each of their sub-steps, and each of the sub-steps required to run those (all the way down to the machine code level) take time, and each have to be executed for every message the application would like to display.
Thread-view further complicates the matter by creating additional relations between the messages, and requiring the OS to look for even more posts during each step.
And this process isn't all that's going on at once- the operating system is trying to do something called TDM, or Time-Division Multiplexing (Multitasking), which allow you to run more than one program at once on a single CPU (which in turn can only do one thing at once). This basically lets the individual processes take individual turns so quickly that they seem simultaneous.
Because of all this, computer scientists rate their algorithms by degradation. We actually have a measure called Big-O (asymptotic) notation that tells you how well an algorithm handles load.
A good algorithm generally has performance of O(N) or better. This means that for every N elements (in your case, e-mail), it takes approximately N iterations to complete. Note the assumption that each "base iteration" executes in a (roughly) fixed time.
Assuming the WM message app uses a sensible algorithm (and it would be difficult for it not to), we can assume its efficiency is approximately O(N)- as it is simply O(N) for non-threaded, and it is O((N/S) * S) for threaded, which simplifies to O(N). This means that for every 4000 messages, it will need to iterate 4000 times.
Considering the application can't store 4000-messages worth of data, it spends a lot of time during each iteration moving messages in and out of memory.
Given all of this, the Windows Mobile message parsing algorithm is the cause of your 'slowness' and 'hangups'. These periods of non-responsiveness are simply WM trying to run through the algorithm for all of the huge quantities of messages.
Hence, the problem is that Windows Mobile simply wasn't designed to bear the load you were forcing it to bear. This means the problem isn't related to the implementation in any given ROM.
can anyone point me in the direction of a nice lean rom that can handle an obscene amount of SMS' and emails?
with thanks
Click to expand...
Click to collapse
A ROM isn't going to be your solution- every ROM uses the same core Windows Mobile messaging applications.
It is possible that a combination of a data structure and message parsing algorithm (with the addition of hash and cache optimized for the way you specifically use messaging) could handle all of these messages without any considerable degradation; but it would run far less efficiently on small amount of messages than the current WM scheme. (And think: how many users have as many messages as you?)
Since no one really needs as many messages as you seem to, consider your options:
1) Implement a database optimized for large amount of messages. Implement a program that hooks all messages and places them in this database instead of the WM one. Write your own message reading/writing application and use in place of the WM one. Note that no one will do this for you. It's not a public interest: if you want it, you'll have to write it. We can help you- but we're not here to do the work for you.
2) Try a third party SMS reader/writer like Vito SMS-Chat. I don't think these will fare much better (in fact, they may fare worse), but if they implement any database of their own and any type of localized caching (especially of recent messages), they may work a bit better.
3) Remove your older SMS messages. This is probably the best option. If you don't need those messages anymore, a Microsoft applet called InboxExtender adds buttons to delete all messages (and to mark them all as read.)
4) Don't use threaded mode. This will decrease the time each parsing iteration takes.
---
This isn't a bug; nor is it a glitch, nor is it WM being crappy. It's much the same as if you tell photoshop to open an 8GB file on your desktop. Your computer may slow to a halt and take forever doing it- but the cause isn't that your computer is crappy, but rather that you're trying to open such a huge file.
Wow! Talk about a detailed explanation. Great explanation ktemkin, u obviously know what ur talkin about. Im actually a part time computer programmer. The biggest problem I see/hear about is people sayng their systems are getting way too slow. No surprise that the biggest reason for this is the used space in their computers hard drive for out-of-date programs, used cache space, ...basically all of the things theyve used before and just never cleaned out. I install 1 program, free their computer up of its junked up memory and update their programs. On this I get credit to simply sit on my butt and delete a few things, lol. Id suppose regardless of the device/computer its all the same. IE, the more room u have to work with, the faster the device's responsiveness is.
Best advice for these PDAs and smartphones for emails is dont leave all of your emails in your inbox folder. Make different folders, then as u go through your emails simply move the selected ones to the selected folders. Then DO NOT sync all of your folders to your device. Only sync the inbox to your device. Then if u ever need a message, u will know which folder its in, then sync that 1 folder, download the message(s), then after ur done, remove that folder from your sync list. When u organize your emails this way u will have a much enjoyable and longer life experience with your device. Hope this helps.
panthersdzynes said:
I dont think this would be the ROMs fault. You see the more messages you store in your fone the less memory and stability it has. Instead of having old messages in your fone why not delete them? Talk about practicality, whats practical about having old messages in ur fone just to take up excess memory? So someone texts u one question, u answer it and have no intentions of messaging them back, yet its more practical to keep these messages in ur fone and then complain that u have to hard reset instead of deleting a few messages due to having all these redundant mesages that u really dont need? Seems to me an intelligent person would rather delete a few messages rather than hard resetting their device and lose everything. Maybe u can make a fresh ROM with no extras and nothing more than a fone and an email device. I mean, why have a PDA if all ur gonna do is text and email? Seems to me u can save a few hundred bux and just get a cheap lil fone that has no customization whatsoever to leave all the room for messaging? Sorry if I sound like I have an attitude, honestly I dont, I just think that the chefs u mentioned have awesome ROMs that keep getting better and better yet u diss them cuz ud rather hard resest and lose everything rather than simply deleting a few hundred messages that u dont need anyway.
Click to expand...
Click to collapse
The fact of the matter is, that a lot of the messages on my handset DO need to remain on there for at least the remainder of a fortnight following receipt of said message.
the only 'qualm' i was having was that my older wizard and hermes devices seem to have managed that number of SMS' and more perfectly fine, but my Raphael begins to struggle.
lukesky said:
It's like if you don't clean your house regularly and just throw your rubbish everywhere, then you complain that you're living in a pig sty and your house fills up with rubbish so fast you can't move around the house as quickly as you used to. Then after a month, you need to fully renovate the whole interior of the house again so that you have space to walk...
My suggestions :
1) start learning some housekeeping(make that little effort to delete the sms yourself)
2) get a bigger house (new phone with more ram and faster CPU)
3) get a cleaner to come in to clean your rubbish once in awhile (there might be some sms program out there that can auto delete your sms on a regular basis)
4) or reorganise the layout of your house so that it's more efficient (forget about sms threading and use the conventional format, I'm sure it'll speed things up a bit since it doesn't have to load the entire conversation everytime)
Click to expand...
Click to collapse
I have already tried disabling the SMS threading and although it speeds it up a little, i then also lose track of what message has come from whom.
i understand that 'doing a bit of house keeping' is what i should be doing, however, i barely have time to do said house keeping, and when i do attempt to delete a large volume of messages in one go the phone hangs on me again!
panthersdzynes said:
Wow! Talk about a detailed explanation. Great explanation ktemkin, u obviously know what ur talkin about. Im actually a part time computer programmer. The biggest problem I see/hear about is people sayng their systems are getting way too slow. No surprise that the biggest reason for this is the used space in their computers hard drive for out-of-date programs, used cache space, ...basically all of the things theyve used before and just never cleaned out. I install 1 program, free their computer up of its junked up memory and update their programs. On this I get credit to simply sit on my butt and delete a few things, lol. Id suppose regardless of the device/computer its all the same. IE, the more room u have to work with, the faster the device's responsiveness is.
Best advice for these PDAs and smartphones for emails is dont leave all of your emails in your inbox folder. Make different folders, then as u go through your emails simply move the selected ones to the selected folders. Then DO NOT sync all of your folders to your device. Only sync the inbox to your device. Then if u ever need a message, u will know which folder its in, then sync that 1 folder, download the message(s), then after ur done, remove that folder from your sync list. When u organize your emails this way u will have a much enjoyable and longer life experience with your device. Hope this helps.
Click to expand...
Click to collapse
my emails are already pretty much organised, and i've only got it syncing the last 7 days for me, emails to be honest are fine its moreso the SMS side of things.
I was not complaining nor targetting any particular chef, i was merely voicing my concern that a device as powerful as the raphael seems to baulk at the prospect of a few thousand messages when the older wizard and hermes seem to manage perfectly fine.
i'm not exactly expecting an instantaneous loading of my inbox, i'm not entirely thick, i appreciate that having such a large number of SMS' will inevitably slow the handset down, however, i do not see why the handset should HANG when i try to open the messaging application, or why, when i try to go and open the messaging application it just doesnt register that i've asked it to open the application, just stops and i have to 'tap' it a few times before it opens.
duke_stix said:
the main thing i use my handset for is email and SMS messaging, in any given month easily 4000+ SMS' will be sent and the same number received.
Click to expand...
Click to collapse
4000+ sms' send AND receive???
so that means:
8000 per month / 30 days in a month = 266,667 per day
266,667 per day / 960 minutes (16 hours * 60) awake a day = 0,278
so say you are 16 hours awake on a day than you send/receive a sms every 15 seconds?? don't you have a real life?
TheWeird1 said:
4000+ sms' send AND receive???
so that means:
8000 per month / 30 days in a month = 266,667 per day
266,667 per day / 960 minutes (16 hours * 60) awake a day = 0,278
so say you are 16 hours awake on a day than you send/receive a sms every 15 seconds?? don't you have a real life?
Click to expand...
Click to collapse
not all of the messages that are sent and received are single SMS' long!
majority, if not all will be 3/4+ messages long and the replies can be twice as long
duke_stix said:
The fact of the matter is, that a lot of the messages on my handset DO need to remain on there for at least the remainder of a fortnight following receipt of said message.
the only 'qualm' i was having was that my older wizard and hermes devices seem to have managed that number of SMS' and more perfectly fine, but my Raphael begins to struggle.
I have already tried disabling the SMS threading and although it speeds it up a little, i then also lose track of what message has come from whom.
i understand that 'doing a bit of house keeping' is what i should be doing, however, i barely have time to do said house keeping, and when i do attempt to delete a large volume of messages in one go the phone hangs on me again!
Click to expand...
Click to collapse
1 thing's for sure, the wizard and hermes are definitely non-threaded sms. But it's hard to compare apple to apple. Did you have 4000 in your Inbox and Sent folders on your Wizard and Hermes too?
I don't think the phone has hung, it's probably doing the processing for you. Have a little patience and let it sit there for awhile and do it's thing and see if it's really hanged.
duke_stix said:
my emails are already pretty much organised, and i've only got it syncing the last 7 days for me, emails to be honest are fine its moreso the SMS side of things.
I was not complaining nor targetting any particular chef, i was merely voicing my concern that a device as powerful as the raphael seems to baulk at the prospect of a few thousand messages when the older wizard and hermes seem to manage perfectly fine.
i'm not exactly expecting an instantaneous loading of my inbox, i'm not entirely thick, i appreciate that having such a large number of SMS' will inevitably slow the handset down, however, i do not see why the handset should HANG when i try to open the messaging application, or why, when i try to go and open the messaging application it just doesnt register that i've asked it to open the application, just stops and i have to 'tap' it a few times before it opens.
Click to expand...
Click to collapse
Maybe you should try Da_G's test ROM.. the feedback is that it's very fast. If that doesn't work for you, it's time to get a new phone.. maybe consider one of the 1GHZ models...
i have 8000 plus messages on my tp n it doesn't slow down
thing is, i don't use threaded sms. can't get used to it cos i've been using wm since wm5 n it doesn't have it. i guess i got used to the older stuff

SMS Database

I know this question has been asked before, but never answered and I was hoping that things might have changed by now.
I was wondering if there was any way whatsoever to get WM6.5.3 to store the SMS database on the micro-sd card, thereby not forcing me to keep trimming my sms messages as frequently as I have to.
I have a Tilt 2, running the latest NRGY ROM.
Thanks
~Z
I am not so sure about your phone, but I am positive you could find out on the respective thread. I know for my Excalibur, there is a 'hack' (mod really) that allows for a larger number of SMS to store ... I don't think it re-routes it to the SD card though.
The issue is primarily that the SMS database file is the same as the Phonebook, Recent Calls, and VoiceMail records(?)...I honestly forget. I did some digging into this a while ago, but stopped. Its rather inconvenient since its all partially encrypted, or at least not that easy to just parse through.
I have been contemplating starting a project (which I have already started conceptualizing) that will replace the dumb CE MAPI once and for all. It would be a suite of programs [Replacement SMS/MMS/EMail/etc. Transport DLL, Replacement tmail.exe, Replacement database] ... with end-users in mind, such that data is easy to obtain. I would probably store it as XML or something similar. If I do go through with this, I will be sure to consider storing to Storage Card AND/OR main memory

[Q] Hidden folders through Android MTP

Hey !
I've searched the web as well as the forum but i didn't find any solution and it seems i'm not the only one wanting to do that.
Is it possible to see hidden folders of internal sdcard through MTP ?
I really talk about hidden folder like ".android_secure", not the photos that aren't yet scanned by the media scanner service.
In my windows environment i already set the view all hidden files/folders as well as view system files.
If i have to change folders permissions through terminal what are the correct permissions i should put ?
Thanks for the help !!
Am i the only having that problem ?
Do you guys see the hidden folders through mtp connexion ?
If you'd bother reading a bit before asking questions, you might have the answer already.
http://en.wikipedia.org/wiki/Media_Transfer_Protocol
MTP and PTP specifically overcome this issue by making the unit of managed storage a local file rather than an entire (possibly very large) unit of mass storage at the block level. In this way, MTP works like a transactional file system - either the entire file is written/read or nothing.
More or less, it is designed to keep hidden what should be hidden stick to USB storage or FTP/SCP.
PS: can't believe we're using a transfer protocol designed by Microsoft in a Linux-based system mastered by Google. The horror, the pain... no wonder it doesn't work on my Mac!
VAXXi said:
If you'd bother reading a bit before asking questions, you might have the answer already.
http://en.wikipedia.org/wiki/Media_Transfer_Protocol
MTP and PTP specifically overcome this issue by making the unit of managed storage a local file rather than an entire (possibly very large) unit of mass storage at the block level. In this way, MTP works like a transactional file system - either the entire file is written/read or nothing.
More or less, it is designed to keep hidden what should be hidden stick to USB storage or FTP/SCP.
PS: can't believe we're using a transfer protocol designed by Microsoft in a Linux-based system mastered by Google. The horror, the pain... no wonder it doesn't work on my Mac!
Click to expand...
Click to collapse
Thx for the link ! I knew what mtp is but its always good to refresh things up
I read the whole article and unfortunately it doesn't answer my question.
I understood that its the same for all of us meaning that we are not able to see hidden folders.
But the main question is if there is a way to bypass this behavior ? Where is this defined exactly ?
It seems according to the article that its not on the file permission layer ? Should it be at partition level ? How can mtp know what to share with the host ?
It's not a "behaviour" per se, but more/less a database. Remember that media scanner program which wakes up every time you change something on your SD card, be it internal or external ? it simply keeps a "database" of files which are presented to the MTP client (your computer) and hides away the phone's internal folders which are not supposed to be seen by the end-user.
Theoretically, this is done in order to have the iPhone-like behaviour: seeing the entire device memory as one big unified storage which you can fill as you want (no more of those partition size limitations which brought us the app2sd hacks). In my opinion, it's more or less just a small step done in order to "secure" phones for DRM protected content.
Studios and media companies require devices to have such measures of protection in order to allow you to sell media content (like movies, books, etc) and since Google wants to push its business further with the Play Store, it needs to cave in to the studios' demands. In the future, it will probably even become illegal (DMCA-style) for me to give you the information below
Back to the technical part of our show now.
At the first boot in the life of the device, Media Scanner will look under /system/media and index everything there. After that, it will scan everything under /mnt/sdcard (hence your external SD card too, if you have one, as it's mounted under /mnt/sdcard/external_sd). On some devices, you have an "internal.db" file and one "external-123xyz.db" file; that's a unique ID of the SD card which was scanned. The idea here is you might have 2-3 SD cards which you swap often, and it's not nice to do a full rescan each time you change the SD card. These databases are in /dbdata/databases/com.android.providers.media (on my phone, there is only one "external.db" for example).
So what you're looking for is a way to populate this database with all the files found by the Media Scanner. But Media Scanner doesn't want you to see the hidden folders you mentioned above. So, you get an application which doesn't ignore them, like Rescan SD Card! or SDrescan.
Happy now ?
Thank you very much i really enjoyed reading the explanation as it answers completely my questions and it makes sense !
Ill give a try to the apps but i'm also curious to investigate on those files
Cheers
PS. Is this a disguised way to support non open source protocols ?
Well, you could poke around inside the database with sqlite if you want. But don't really see what's so interesting about them.
It is just another protocol which is supported, which happens to be designed by Microsoft (and probably licensed/paid by Google to be used in Android). I understand the technical explanation behind this decision, but I'm also wary that someday UMS will be disabled by default and enabling it will disable content purchasing for that device (just like having a rooted phone now disables some "sensitive" apps, like banking and online TV).
As a system admin its interesting to know whats happening on my system
As a hobby its interesting for my personal knowledge
And from a development point of view it gives me some ideas for maybe future apps
You say that its just another protocol which is supported but to my understanding there are not so many of them ! So i would say that Google was kind of forced to use the MTP method, first from a hardware point of view and second to be compliant with the rest of the world.
Is there any other protocol that could suite their needs ? Since USB mass storage is not usable on some devices and MTP is well spead.
.HiddenAndroid folders in Win - Here Yesterday, Gone Tomorrow
<Win 7 and GN2>
What's curious and a bit frustrating to me is that, yesterday, when I mounted my device ALL of the hidden (.folders and .files) were visible and searchable. Today, when I went to explore some more, all are gone. I understand that I can use a 3rd party app, just can't figure out why it was visible yesterday and not today.
Hmmph
Did you ever resolve this issue? I really hate MTP for several reasons but this is one of the reasons why! I need to back up all my directories on the SDCard because some apps store them as hidden files in hidden directories. Furthermore, I came across the Play Store bug that requires me to delete a "temp.asec" file in the ".android_secure" directory, but of course it is not visible.
In a post further up it was suggested to use a "Rescan" app to force the DB to include hidden files/directories, but I have tried no less than 4 of these apps, and all they do is trigger the built-in android media scan, which is the problem in the first place!
So has anyone ever been able to access hidden files and directories using MTP?
EDIT: I have an HTC phone that actually shows hidden files and directories, so they have obviously implemented their own media scan. The problem I am currently having is on a Samsung phone (Epic 4G touch)
It may depend on the implementation. For example, using stock rom for the phone doesn't show some files and folders, for example folders starting with dot. But if you use Neatrom Lite it will show all files and folders.
Flash forward about 6 years and Samsung still does not show hidden folders/files when viewing the phone contents in Windows File Explorer, but HTC does.
The reason this is still a problem for me, is that I want to backup the contents of a particular directory (WhatsApp) which contains some hidden folders, so I can restore it anytime on a new phone or the same phone.
I recently switched from HTC to Samsung and alas, it seems the problem with this implementation still exists. Anyone found a workaround to this to allow File Explorer to see the hidden folders?
Update: It appears that hidden .nomedia files (and probably others) appear under regular folders, so the problem is limited to hidden folders themselves

[Q] Problem with Colornote App (Please read and help me)....

I am having some SERIOUS difficulties in lifting these Colornote app files BACK to some readable form after I was forced to do a restore of my phone. It's causing me TOO MUCH STRESS. Someone recommended your blog and that you might be able to help with this.
This problem has been DOGGING me for awhile now. Obviously, I can only spend so much time troubleshooting this on my own limited time because I have to work often and dedicate my precious time on other things. I know it's Proprietary software. But I simply REFUSE to believe that someone out there couldn't put this in a readable format; either on PC or back on my phone where they originated. There's someone out there that could look at this and say, "Piece of cake, here's what you do....". No one has said that so far though.
The makers could though, but they either don't know or refuse to help me. I also contacted them on their Facebook page, only to have them ignore me completely (which is really infuriating). I DO KNOW that others experienced this same problem that im having in the past.
I was forced to do a master reset on my phone and basically the files are not going back to where they originated from (The Colornote App). As you can see per the image I attached here, the files are STILL ON THE PHONE, but this stupid Colornote is not locating them or something! It says that it's "Unable to find application to perform this action" when I try to access the files. It's really bizarre. So these files are basically stuck in limbo. The file types are in 2 forms: IDX and DAT. I contacted Samsung and they they were absolutely worthless for any type of support and said they couldn't help because it's a 3rd party app. Same thing with Verizon.
If there's anyone that can please help me out with either transferring/resurrecting these files in a readable form, I would so greatly appreciate it!

Moving Pics to new phone while preserving the dates?

Oops, seem to have cross posted due to having two tabs open, cant make sense of the changed layout!
Mods please feel free to delete this thread if no one has answered!
Hi all!
An oldie but a goldie!
I've changed from S6 32g, to S6Eedge 64g since I was running out of space.
Now I need to transfer 4000 photos (sorted by date) to the new phone. A direct copy from PC converts all the created dates to the current date (ie when transfer is done) which as you can imagine screws everything up royally!
Ive tried and failed in many many many ways including:
Samsung Smart Switch
FTP
Syncing programs from pc to phone
ADB push
WEBdav
Hail marys, and praying to every god out there
They all bugger up the dates...........
The ONLY ONLY way Ive found out that works is by zipping up the original files, transferring to new phone, and unzipping locally with Total Commander. ONLY Total Commander seems to be able to unzip and preserve the dates, all the other file managers and extracting programs (I tried 10-15) out there can NOT manage this. Total Commander probably works cuz im rooted, and it can use this.
HOWEVER this only worked with smaller directories, but my main pics folder is a 8 gig compressed zip, and when I try the same way Total Commander only manages to unzip the same 700 files and stops (anyone else come across this limitation?). So the only way I can manage is to zip up the original files in 7-8 different zip files and unzip locally individually, which is a big hassle.
Ive googled many threads on this problem, most of them were unresolved and some solutions refer to previous android versions or software and dont work anymore.
I'm hoping that you all dont spend four days setting up a new phone everytime ( home screen and app settings is a whole different drama! Thank you Titanium backup! iOS really has the jump on Andoird when it comes to changing phones...)
The only thing I can think of which I havent tried yet is USB OTG, since I dont have the cable. Anyone can confirm whether this method keeps the dates unchnaged?
So in your experience, whats the best way of doing this? How do you guys do it?
To be honest for 4 thousand photos swapping back and forth between devices you are better off using a could storage. Especially if you want to minimise the risk of loosing them all if something goes wrong. All zip folders will extract with the original date it. You could try and torrent you DCIM folder and language download it to your computer. Torrents download with their file dates intact.
Thanks for your reply!
IMHO this cloud thing is actually whats causing so many of the problems in hardware design these days in general.
Why is it so hard for companies to realise that when you are two floors underground, or roaming or even in middle of central london, net connection is not guaranteed......
Let alone the speed, convenience, security issues etc etc etc
As to the loss risk, I back them up on my home pcs regularly with a sync application, which ironically enough works perfectly well thank god!
So for me cloud is a no go....
As to your other suggestion, sounds great
Can I torrent just on the local net? ie direct from phone to pc?
Any apps you can reccomend? will save me trawling through a hundred crap apps in GPlay.
Thanks again for your time, its appreciated!
PS re what you said about all zips extracting correctly. Can you try on your device? Zip up a few small old files on your pc, transfer the zip to phone, extract there, and check the dates.
All except one of the ones I have tried do it, as mentioned above.
PPS Gave your torrent suggestion a go, no joy. All the transferred files had the new date ;-(.
I tried the the utorrent android app...
Update and for future help for people who will come across this issue.
I managed it!
I used Total Commander on the phone through the LAN plugin function, to access my backup folder on the pc, select all files and copy to local phone directory.
You need to initiate the copy from the PHONE on Total Commander, ie pull the files onto the phone, dont copy them by drag and drop from PC.
It took a while but it was done!
Dates and timestamps all preserved and correct!

Categories

Resources