Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.846500] Oops: 0000 [#1] SMP

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.847000] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-adapter/i2c-0/name

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730] Stack:

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  ffffffffa03f8f40 ffffffffa003a5a0 00000000fffffff4 00007f322d0b1010

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  ffff88021c9bfb98 ffffffff804199dc ffff880224c83060 ffff880224cd9cc8

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730] Call Trace:

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffffa03d4e1f>] scst_add+0x44/0x2ad [scst]

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff804199dc>] class_interface_register+0x8a/0xc0

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe000>] ? init_scst+0x0/0x905 [scst]

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffffa002091d>] scsi_register_interface+0x16/0x18 [scsi_mod]

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe5a7>] init_scst+0x5a7/0x905 [scst]

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff80525c4b>] ? schedule_timeout+0x27/0xb9

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff80239787>] ? __enqueue_rt_entity+0x16a/0x1f6

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802cd6e4>] ? add_partial+0x61/0x6b

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe000>] ? init_scst+0x0/0x905 [scst]

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff8020a062>] do_one_initcall+0x62/0x16b

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802d2d9d>] ? __percpu_depopulate_mask+0x31/0x58

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802cdeb1>] ? __slab_free+0x13/0x2c9

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802d2f3c>] ? percpu_free+0x2a/0x2e

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802d2f3c>] ? percpu_free+0x2a/0x2e

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff80525fc4>] ? mutex_unlock+0x9/0x15

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff8028018e>] ? stop_machine_destroy+0x4b/0x4d

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff8026fca1>] ? load_module+0x14e4/0x15b2

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff80525faa>] ? mutex_lock+0x1f/0x30

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff803820d1>] ? __up_read+0x92/0x9a

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff80260d7a>] ? up_read+0xe/0x10

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff8026176e>] ? __blocking_notifier_call_chain+0x56/0x65

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff8026fe1b>] sys_init_module+0xac/0x1ca

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730]  [<ffffffff802113ca>] system_call_fastpath+0x16/0x1b

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730] Code: 14 08 72 04 0f 0b eb fe 4d 29 c1 4d 01 c2 49 8d 41 ff 4c 39 c9 4c 89 d7 fc 48 0f 43 c8 48 89 d0 f3 a4 c6
07 00 c9 c3 55 48 89 e5 <0f> b6 17 89 d0 2a 06 48 ff c6 84 c0 75 07 48 ff c7 84 d2 75 eb

Message from syslogd@Storage03 at Mon Jul 12 17:55:23 2010 ...
Storage03 kernel: [ 4502.848730] CR2: 0000000000000000

###################################
####### /var/log/messages #########
###################################
Jul 12 17:55:23 Storage03 kernel: [ 4502.844994] BUG: unable to handle kernel NULL pointer dereference at (null)
Jul 12 17:55:23 Storage03 kernel: [ 4502.845619] IP: [<ffffffff8038286f>] strcmp+0x4/0x1e
Jul 12 17:55:23 Storage03 kernel: [ 4502.845965] PGD 21c8ae067 PUD 21c8ad067 PMD 0
Jul 12 17:55:23 Storage03 kernel: [ 4502.846500] Oops: 0000 [#1] SMP
Jul 12 17:55:23 Storage03 kernel: [ 4502.847000] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-adapter/i2c-0/name
Jul 12 17:55:23 Storage03 kernel: [ 4502.847543] CPU 1
Jul 12 17:55:23 Storage03 kernel: [ 4502.847871] Modules linked in: scst(+) nfsd lockd nfs_acl auth_rpcgss autofs4 sunrpc aoe binfmt_misc xfs
exportfs dm_multipath scsi_dh dm_snapshot dm_mod sbs sbshc battery ac bonding ipv6 i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support joydev sg tg3
libphy shpchp e752x_edac edac_core floppy button scsi_wait_scan ext3 jbd ips raid10 raid456 async_memcpy async_xor xor async_tx ehci_hcd uhci_hcd
sata_vsc sata_via sata_uli sata_sx4 sata_svw sata_sis pata_sis sata_sil sata_sil24 sata_qstor sata_promise sata_nv sata_mv sata_inic162x ata_piix
ahci libata sd_mod crc_t10dif scsi_mod rtc_cmos rtc_core rtc_lib
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] Pid: 4129, comm: modprobe Not tainted 2.6.29.6-0.24.smp.gcc3.4.x86_64 #1 eserver xSeries 346
-[884025Y]-
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RIP: 0010:[<ffffffff8038286f>]  [<ffffffff8038286f>] strcmp+0x4/0x1e
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RSP: 0018:ffff88021c9bfb18  EFLAGS: 00010246
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RAX: ffffffffa015e200 RBX: 0000000000000000 RCX: ffff880224cd9b10
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RDX: 0000000000000000 RSI: ffffffffa03eaabf RDI: 0000000000000000
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RBP: ffff88021c9bfb18 R08: ffff8802248a5514 R09: ffff8802248a5514
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] R10: ffff8802248a5514 R11: ffffffffa03f8f40 R12: ffff880224cd9920
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] R13: ffff880224cd9800 R14: 00007f322d0b1010 R15: 0000000000000000
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] FS:  00007f322d1946e0(0000) GS:ffff8802267fd000(0000) knlGS:0000000000000000
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] CR2: 0000000000000000 CR3: 000000021c9b6000 CR4: 00000000000006e0
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] Process modprobe (pid: 4129, threadinfo ffff88021c9be000, task ffff88021c8e16f0)
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] Stack:
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  ffff88021c9bfb58 ffffffffa03d4e1f ffffffffa003a5a0 ffffffffa003a5a0
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  ffffffffa03f8f40 ffffffffa003a5a0 00000000fffffff4 00007f322d0b1010
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  ffff88021c9bfb98 ffffffff804199dc ffff880224c83060 ffff880224cd9cc8
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] Call Trace:
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffffa03d4e1f>] scst_add+0x44/0x2ad [scst]
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff804199dc>] class_interface_register+0x8a/0xc0
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe000>] ? init_scst+0x0/0x905 [scst]
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffffa002091d>] scsi_register_interface+0x16/0x18 [scsi_mod]
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe5a7>] init_scst+0x5a7/0x905 [scst]
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff80525c4b>] ? schedule_timeout+0x27/0xb9
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff80239787>] ? __enqueue_rt_entity+0x16a/0x1f6
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802cd6e4>] ? add_partial+0x61/0x6b
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffffa03fe000>] ? init_scst+0x0/0x905 [scst]
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff8020a062>] do_one_initcall+0x62/0x16b
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802d2d9d>] ? __percpu_depopulate_mask+0x31/0x58
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802cdeb1>] ? __slab_free+0x13/0x2c9
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802d2f3c>] ? percpu_free+0x2a/0x2e
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802d2f3c>] ? percpu_free+0x2a/0x2e
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff80525fc4>] ? mutex_unlock+0x9/0x15
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff8028018e>] ? stop_machine_destroy+0x4b/0x4d
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff8026fca1>] ? load_module+0x14e4/0x15b2
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff80525faa>] ? mutex_lock+0x1f/0x30
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff803820d1>] ? __up_read+0x92/0x9a
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff80260d7a>] ? up_read+0xe/0x10
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff8026176e>] ? __blocking_notifier_call_chain+0x56/0x65
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff8026fe1b>] sys_init_module+0xac/0x1ca
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  [<ffffffff802113ca>] system_call_fastpath+0x16/0x1b
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] Code: 14 08 72 04 0f 0b eb fe 4d 29 c1 4d 01 c2 49 8d 41 ff 4c 39 c9 4c 89 d7 fc 48 0f 43 c8 48
89 d0 f3 a4 c6 07 00 c9 c3 55 48 89 e5 <0f> b6 17 89 d0 2a 06 48 ff c6 84 c0 75 07 48 ff c7 84 d2 75 eb
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] RIP  [<ffffffff8038286f>] strcmp+0x4/0x1e
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730]  RSP <ffff88021c9bfb18>
Jul 12 17:55:23 Storage03 kernel: [ 4502.848730] CR2: 0000000000000000
Jul 12 17:55:23 Storage03 kernel: [ 4502.882728] ---[ end trace d06ff612b807a72e ]---