Hardware :: Random Boot Kernel Panic With MPT Fusion SCSI?

I have an IBM server with a: "SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)", meaning kernel module CONFIG_FUSION. This is the real hardware, not emulated by vmware. With recent kernels (2.6.30-35) the boot results in one of three situations:

1.) Most common: kernel panic, way too long stack dump to see anything useful but did have a terminal briefly to debug this and its just not being able to communicate with the scsi subsystems, google for errors found nothing of use.

2.) Sometimes: A hang waiting for the scsi disks after the controller fails to initialize properly, does not pass in a couple of hours.

3.) Very rarely: A long wait for the scsi controller to initialize but a succesful boot afterwards. ACPI subsystems are taking all CPU power. This is the state the machine is at now, working somewhat.

I can boot the machine with a Gentoo live install disk (kernel 2.6.29 I think) and under that version the controller initializes instantly and there are no problems whatsover using the disks so the hardware seems to be solid!

There were some MPT changes in 2.6.25rc5 so thats the latest kernel I've tried. The errors during boot change to a bit more verbose but thats about it. Unfortunately I don't have a serial terminal with me anymore so cannot capture these errors but it was something about an "unexpected doorbell" with 2.6.24 and nothing memorable with .25. Long story short tho, googling the errors provides no solutions and the little I find points to the controller being too slow to initialize somehow.

I'm running out of ideas here and the server is burning fans like theres no tomorrow (ACPI taking everything it can) so is my only option to downgrade to 2.6.29 (and downgrade udev, lvm2, so on and on and on)? I can't believe I'm the only one for whom this has been broken for several minor versions.

Kernels tried:

- 2.6.29-gentoo (live disk) - works, blazing fast initialization
- 2.6.32-gentoo-r7 - random boot failures and successes
- 2.6.34-gentoo-r1 - same
- 2.6.35-rc5 vanilla - same, better errors, zero useful google results.

I'm not very keen on longshot attempts since it can take a couple of hours of panic-loops to get the machine booted up again and the server functions cannot be transferred to another machine. Also, I'm a bit hesitant to mention this, but the boot has only ever succeeded when a serial cable with something at the other end is connected. At first it was the testing serial terminal and now its just a connection to a UPS.

Attached: bootlog of succesfull-ish boot with 2.6.34 (had to cut some out to make the sizelimit):

Code:

Linux version 2.6.34-gentoo-r1 (root@livecd) (gcc version 4.3.4 (Gentoo Hardened 4.3.4 p1.1, pie-10.1.5) ) #1 SMP Sun Jul 18 18:54:48 EEST 2010
BIOS-provided physical RAM map:

BIOS-e820: 0000000000000000 - 0000000000099c00 (usable)
BIOS-e820: 0000000000099c00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)

[code]....