[Q] How to fix Streak5 'ARM9 has crashed' reboot problem? - Streak 5 Q&A, Help & Troubleshooting

Hi!
I am currently working on a CM 7.2 port for the Dell Streak 5, which I want to share soon (I can't post this in the dev forum because I don't have enough posts written just yet). Currently I am trying to find a fix or a workaround for the following problem:
Sometimes, the Streak immediately reboots and shows the Dell logo. It just shows the following kernel logs (also written to LogCat):
Code:
07-03 19:13:09.348 W/PrintK ( 83): ARM9 has CRASHED
07-03 19:13:09.348 W/PrintK ( 83): smd_diag, smem: DIAG ''
07-03 19:13:09.348 W/PrintK ( 83): ARM9 has CRASHED
07-03 19:13:09.348 W/PrintK ( 83): smd_diag, smem: DIAG ''
07-03 19:13:09.348 W/PrintK ( 83): ARM9 has CRASHED
07-03 19:13:09.348 W/PrintK ( 83): smd_diag, smem: DIAG ''
07-03 19:13:09.348 W/PrintK ( 83): ARM9 has CRASHED
07-03 19:13:09.348 W/PrintK ( 83): smd_diag, smem: DIAG ''
07-03 19:13:10.430 W/PrintK ( 83): ARM9 has CRASHED
07-03 19:13:10.430 W/PrintK ( 83): smd_diag, smem: DIAG ''
07-03 19:13:10.430 W/PrintK ( 83): ARM9 has CRASHED
07-0
When I searched the kernel source, it seems that the following lines in arch/arm/mach-msm/smd.c are responsible for this log output (ommitting the parts that are not compiled):
Code:
#if defined (CONFIG_QSD_ARM9_CRASH_FUNCTION)
void smd_diag(void)
{
char *x;
x = smem_find(ID_DIAG_ERR_MSG, SZ_DIAG_ERR_MSG);
if (x != 0)
{
SMD_CRASH_LOG( "%s, smem: DIAG '%s'\n", __func__, x);
queue_work(crash_work_queue, &smd_crash_worker);
}
}
#else /* CONFIG_QSD_ARM9_CRASH_FUNCTION */
/* ommitted (not compiled) */
#endif /* CONFIG_QSD_ARM9_CRASH_FUNCTION */
static void handle_modem_crash(void)
{
SMD_CRASH_LOG( "ARM9 has CRASHED\n");
smd_diag();
/* hard reboot if possible FIXME
if (msm_reset_hook)
msm_reset_hook();
*/
/* in this case the modem or watchdog should reboot us */
#ifndef CONFIG_QSD_ARM9_CRASH_FUNCTION /* return imediately for kernel to do something */
for (;;)
#endif /* CONFIG_QSD_ARM9_CRASH_FUNCTION */
;
}
It seems that the modem is crashing and forces the device to reboot. The severe problem is that sometimes the device ends up rebooting again and again, suddenly stuck in the Dell logo - so maybe the reboot performed by the modem is not correct. But in the optimal case the device should just reset the modem, and not reboot the entire device.
As the radio drivers are only available in binary form (of course) and the device already is EOL, I really need help to find a good workaround. Does anybody have a solution to this problem?
I discovered that there is a kernel driver available in arch/arm/mach-msm/reset_modem.c that sounds interesting. It should be able to reset the modem by writing the appropriate command to a kernel device, and which is compiled as a kernel module in the default Dell Streak kernel config. Has anybody tried this out yet?
Thanks in advance & regards
Marc
PS: I am using the Phoenix kernel by the DSC Team, added with some patches by _n0p_.

---

Hi Marc!
How severe this problem is?

---

Hi _n0p_!
It's very severe to me - I have a couple of these devices here, and it's really critical if they turn off during operation (and some of them don't even come back up again afterwards).
It would be better if the radio was reset on failure - I can afford that the devices do not have a radio connection for a couple of seconds. Or at least this is what I hope.

---

There's a "System Type -> Reset Modem Driver" module option in kernel.
May sound dumb, but maybe it needs it?

---

_n0p_ said:
There's a "System Type -> Reset Modem Driver" module option in kernel.
May sound dumb, but maybe it needs it?
Click to expand...
Click to collapse
I recompiled the kernel with this module built-in, but from what I read from the source code it's just providing a device /dev/reset_modem to enable users to reset the modem from userspace (as root). However, this will not be possible entirely if the kernel itself is rebooting already, leaving no chance to any userspace commands. I believe this has to be fixed in the kernel itself. Right now I am looking at the method smsm_reset_modem in the file smd.c which sounds quite interesting. Maybe I can deactivate the code for rebooting the device there and instead run code to reset the modem. Interestingly, setting CONFIG_QSD_ARM9_CRASH_FUNCTION to 'n' in the config leads to kernel compilation errors - so the Streak was never intented to resolve modem problems the better way...

---

By the way, a few minutes ago I was able to log a modem restart during the reboot loop with LogCat:
pastebin.com/9U3XPxdK (yay, I can not post links yet :/ )
This looks really ugly - but it could help during debugging the problem. It would be much nicer to know what actually forces the modem to restart... (is it possible to get the stack trace during runtime, maybe by using a special macro?)

---

cant you check on the build based on the cm7.2 that already at the dev section? if it is not all, at least some of it would help...

---

deysmacro said:
cant you check on the build based on the cm7.2 that already at the dev section? if it is not all, at least some of it would help...
Click to expand...
Click to collapse
I don't know exactly what triggers the problem - maybe the devices run fine for a long time, but crash when using GPS + network in some odd way, or when the network type switched from 2G to 3G or back - I really can't tell.
As for CM 7.2, I had to build from the sources (both Kernel and CM) in order to do fixes to the ROM etc. Maybe I could try out the CM 7.2 build, but even if the problem would (not) exist there, too, I would be back to the start.
At least yesterday I tried out the reset_modem kernel module and tried all commands I could find in the source on the device /dev/reset_modem - it just crashed the modem and the device all the time, instead of one command which was effectively doing nothing. So back to the start... :-(

---

man... that does sucks... well... i wish you all the best nonetheless... :good:

---

Where's an interesting module param:
msm_smd_debug_mask
--
Also, according to:
Code:
#define MODEM_CRASH_NOTIFY_USER
extern void qi2ccapkybd_set_led(unsigned int light);
#define MODEM_CRASH_NOTIFY_USER_0 do {qi2ccapkybd_set_led(0x0);} while(0)
#define MODEM_CRASH_NOTIFY_USER_1 do {qi2ccapkybd_set_led(0xFFFFFF);} while(0)
It should blink leds on going to reboot.
--
Marc, thank you! While looking at SMD i found the code that blinks leds on AMSS mismatch (finally!)

---

_n0p_ said:
Marc, thank you! While looking at SMD i found the code that blinks leds on AMSS mismatch (finally!)
Click to expand...
Click to collapse
wow...! a discovery... ! :victory:

---

This actually leads me to think that we now free of "Blinking LEDs problem" and maybe, just maybe changing AMSS (BB) might help with modem reset problem.
I'm currently using 366 (manics told that it has good voice quality).

---

_n0p_ said:
This actually leads me to think that we now free of "Blinking LEDs problem" and maybe, just maybe changing AMSS (BB) might help with modem reset problem.
I'm currently using 366 (manics told that it has good voice quality).
Click to expand...
Click to collapse
last time i downgraded from rom based on 407 to 366 and although the blinking led happens.. but phone continue to work like normal... :laugh:
and yeah... 366 does have a good voice quality... and not mention excellent cell tower reception... it is proven since i have been in the building where reception is pretty much poor.. but with 366... somehow it almost always full...
but since nobody make a custom ROM based on 366, i had to sadly use other ROM...

---

As it's a bit offtopic, let's continue here:
http://forum.xda-developers.com/showpost.php?p=28318630&postcount=4374

---

Hey _n0p_!
Yeah, that seems to be the code that leads to blinking LED sensor keys on wrong AMSS!
As for the AMSS/DSP1 partitions, I currently use the newest from the original Streak 407 firmware, which I flashed via 'fastboot flash amss amss.mbn' and 'fastboot flash dsp1 dsp1.mbn'. Maybe that leads to the problems.
So you would recommend downgrading to amss 366? Although it does not match the kernel version and Android version? Isn't this a bad idea in general?
And for the DSP1, which version do you recommend - also from 366? Or the complete 366 update.pkg?
I am currently testing the image I flashed after initially flashing the 407 package the correct way - maybe this fixes the problems. Otherwise I will try 366.
By the way: Is there a good overview available what all the other files do, e.g. fsbl.mbn, osbl.mbn, dbl.mbn and appsboot.mbn?
From what I know, DT.img is the bootloader, amss the baseband software and dsp1 the DSP software for audio. And I know it's dangerous to flash the files individually, as I was able to completely brick a device by just flashing one of these special partitions...

---

Well, i'm using 366 right now. Too early to tell a conclusion, but it seems like default audio output is lower then in 4062 BB.
So far no glitches.
And i'm using only AMSS.
--
AFAIK DT.mbn is firmware updater, but I might be wrong.
Also DSP also works for video processing as well.
--
Almost forgot: it seems simple to disable bb error reboot.
Maybe it's worth giving a test?

---

_n0p_ said:
Almost forgot: it seems simple to disable bb error reboot.
Maybe it's worth giving a test?
Click to expand...
Click to collapse
Do you mean disabling CONFIG_QSD_ARM9_CRASH_FUNCTION in the kernel config? I did this, recompiled (had to fix a small include error after that, too) and tried it out - so far it crashed again after some time of usage.
BTW, how did you disable the AMSS version check? I guess you disabled the option CONFIG_QSD_OEM_RPC_VERSION_CHECK - but it's good to know.

---

I thought of simply trimming that rpc probe proc.
And you are right, that's the conf value plus needs ifdefing undeclared version check (in this case), variable.

---

Marc, it seems like
#if 0
//#ifdef CONFIG_BUILDTYPE_SHIP
SMD_CRASH_LOG( "%s: restart system\n", __func__);
queue_delayed_work(restart_work_queue, &smd_crash_restart_worker,
MODEM_CRASH_RESTART_TIMEOUT);
#endif /* CONFIG_BUILDTYPE_SHIP */
would reboot device only if modem in reset state and camera button is pressed.

---

Hey n0p!
_n0p_ said:
Marc, it seems like
#if 0
//#ifdef CONFIG_BUILDTYPE_SHIP
SMD_CRASH_LOG( "%s: restart system\n", __func__);
queue_delayed_work(restart_work_queue, &smd_crash_restart_worker,
MODEM_CRASH_RESTART_TIMEOUT);
#endif /* CONFIG_BUILDTYPE_SHIP */
would reboot device only if modem in reset state and camera button is pressed.
Click to expand...
Click to collapse
I tried your code, as well as some other fixes. It always led to a kernel loop and the modem didn't come back, so in the end I just included a solution to detect the problem and continue to reboot the device.
From what I learned in the last few weeks, it seems the complete ARM processor goes down with the modem. In the end, it seems to be a hardware problem, because I noticed the following facts:
Even if I completely re-flash a device that shows an error, it still occurs - especially when the network is used.
Some devices do not show this problem and can run for a long time. Other devices that had this problem always continue to have this problem. Even after re-flashing and even using different amss/dsp1 versions and on the original software.
What's more: I also had the problem on a new device that never had a custom ROM on it. Yeah, using the original firmware.
So in the end everyone should be better off to throw away his device if this problem occurs.