2012-07-24T17:31:11.928Z cpu3:191989)WARNING: UserObj: 675: Failed to crossdup fd 8, /dev/ptyp0 type CHAR: Busy
2012-07-24T17:31:11.928Z cpu3:191989)WARNING: UserObj: 675: Failed to crossdup fd 10, /dev/ptyp0 type CHAR: Busy
2012-07-24T17:31:11.928Z cpu1:191991)WARNING: UserLinux: 1340: unsupported: (void)
2012-07-24T17:32:21.538Z cpu1:2049)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177139 microseconds to 3637169 microseconds.
2012-07-24T17:32:23.136Z cpu1:121479)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 3637169 microseconds to 722304 microseconds.
2012-07-24T17:32:26.562Z cpu0:2052)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177178 microseconds to 3881406 microseconds.
2012-07-24T17:32:43.797Z cpu2:121481)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177271 microseconds to 8444857 microseconds.
2012-07-24T17:32:44.050Z cpu1:120958)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 8444857 microseconds to 1679669 microseconds.
2012-07-24T17:33:44.438Z cpu1:119821)WARNING: NMP: nmpDeviceTaskMgmt:2210:Attempt to issue lun reset on device naa.600144f0c52383000000500928190001. This will clear any SCSI-2 reservations on the device.
2012-07-24T17:33:46.930Z cpu3:2126)VSCSI: 2763: Retry 0 on handle 8209 still in progress after 2 seconds
2012-07-24T17:33:47.631Z cpu3:121479)WARNING: VSCSI: 3418: handle 8209(vscsi0:0):WaitForCIF: Issuing reset;  number of CIF:17
2012-07-24T17:33:47.631Z cpu3:121479)WARNING: VSCSI: 2401: handle 8209(vscsi0:0):Ignoring double reset
2012-07-24T17:33:53.820Z cpu3:119821)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x41240078e5c0, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0. Act:EVAL
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:33:53.820Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x41240078e5c0) 0x2a, CmdSN 0x80000031 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177258 microseconds to 3716504 microseconds.
2012-07-24T17:33:53.820Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x412400769600) 0x2a, CmdSN 0x8000001d from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177425 microseconds to 9443825 microseconds.
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177425 microseconds to 27309713 microseconds.
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177710 microseconds to 58368737 microseconds.
2012-07-24T17:33:53.820Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 177996 microseconds to 120744428 microseconds.
2012-07-24T17:33:53.820Z cpu3:119821)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x4124007bdec0, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0. Act:EVAL
2012-07-24T17:33:53.820Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x4124007bdec0) 0x2a, CmdSN 0x80000008 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:33:53.820Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x412400760d80) 0x2a, CmdSN 0x8000006b from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-07-24T17:33:53.827Z cpu2:121479)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x412400793d40, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0. Act:NONE
2012-07-24T17:33:53.827Z cpu2:121479)ScsiDeviceIO: 2309: Cmd(0x412400793d40) 0x2a, CmdSN 0x8000005d from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0.
2012-07-24T17:33:53.827Z cpu2:121479)ScsiCore: 1460: Power-on Reset occurred on naa.600144f0c52383000000500928190001
2012-07-24T17:33:54.372Z cpu2:121481)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 120744428 microseconds to 24135374 microseconds.
2012-07-24T17:33:55.183Z cpu3:121479)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 24135374 microseconds to 4794122 microseconds.
2012-07-24T17:35:08.440Z cpu0:119821)WARNING: NMP: nmpDeviceTaskMgmt:2210:Attempt to issue lun reset on device naa.600144f0c52383000000500928190001. This will clear any SCSI-2 reservations on the device.
2012-07-24T17:35:08.440Z cpu0:119821)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x4124007cafc0, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x8 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0. Act:EVAL
2012-07-24T17:35:08.440Z cpu0:119821)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:35:08.440Z cpu0:119821)ScsiDeviceIO: 2309: Cmd(0x4124007cafc0) 0x2a, CmdSN 0x8000007b from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_ConnReceiveAtomic: vmhba38:CH:0 T:0 CN:0: Failed to receive data: Connection closed by peer
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_ConnReceiveAtomic: Sess [ISID: 00023d000001 TARGET: iqn.1986-03.com.sun:02:97a66d96-4de8-417c-dc9b-de72b28ff0e8 TPGT: 1 TSIH: 0]
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_ConnReceiveAtomic: Conn [CID: 0 L: 192.168.30.1:53368 R: 192.168.30.2:3260]
2012-07-24T17:35:09.240Z cpu3:2668)iscsi_vmk: iscsivmk_ConnRxNotifyFailure: vmhba38:CH:0 T:0 CN:0: Connection rx notifying failure: Failed to Receive. State=Online
2012-07-24T17:35:09.240Z cpu3:2668)iscsi_vmk: iscsivmk_ConnRxNotifyFailure: Sess [ISID: 00023d000001 TARGET: iqn.1986-03.com.sun:02:97a66d96-4de8-417c-dc9b-de72b28ff0e8 TPGT: 1 TSIH: 0]
2012-07-24T17:35:09.240Z cpu3:2668)iscsi_vmk: iscsivmk_ConnRxNotifyFailure: Conn [CID: 0 L: 192.168.30.1:53368 R: 192.168.30.2:3260]
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_StopConnection: vmhba38:CH:0 T:0 CN:0: iSCSI connection is being marked "OFFLINE" (Event:6)
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_StopConnection: Sess [ISID: 00023d000001 TARGET: iqn.1986-03.com.sun:02:97a66d96-4de8-417c-dc9b-de72b28ff0e8 TPGT: 1 TSIH: 0]
2012-07-24T17:35:09.240Z cpu3:2668)WARNING: iscsi_vmk: iscsivmk_StopConnection: Conn [CID: 0 L: 192.168.30.1:53368 R: 192.168.30.2:3260]
2012-07-24T17:35:09.240Z cpu3:119821)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:35:09.240Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 178462 microseconds to 11718981 microseconds.
2012-07-24T17:35:09.241Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 178621 microseconds to 31560137 microseconds.
2012-07-24T17:35:09.241Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 178768 microseconds to 70297858 microseconds.
2012-07-24T17:35:09.241Z cpu0:2668)WARNING: iscsi_vmk: iscsivmk_TaskMgmtIssue: vmhba38:CH:0 T:0 L:0 : Task mgmt "Logical Unit Reset" with itt=0x6db4eb (refITT=0xffffffff) timed out.
2012-07-24T17:35:09.241Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x412400795640) 0x2a, CmdSN 0x80000028 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:09.241Z cpu3:119821)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x41240079eec0, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x8 D:0x40 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-07-24T17:35:09.241Z cpu3:119821)ScsiDeviceIO: 2309: Cmd(0x41240079eec0) 0x2a, CmdSN 0x8000002a from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x8 D:0x40 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-07-24T17:35:09.241Z cpu3:119821)WARNING: ScsiDeviceIO: 1218: Device naa.600144f0c52383000000500928190001 performance has deteriorated. I/O latency increased from average value of 179134 microseconds to 156577520 microseconds.
2012-07-24T17:35:09.243Z cpu0:2626)ScsiDeviceIO: 2309: Cmd(0x412400779940) 0x2a, CmdSN 0x8000006a from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:09.243Z cpu0:2626)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x41240077a240, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0. Act:EVAL
2012-07-24T17:35:09.243Z cpu0:2626)ScsiDeviceIO: 2309: Cmd(0x41240077a240) 0x2a, CmdSN 0x80000054 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2012-07-24T17:35:09.243Z cpu0:2626)ScsiDeviceIO: 2309: Cmd(0x412400786b80) 0x2a, CmdSN 0x8000006e from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:09.243Z cpu0:2626)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x412400769100, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0. Act:EVAL
2012-07-24T17:35:09.243Z cpu0:2626)ScsiDeviceIO: 2309: Cmd(0x412400769100) 0x2a, CmdSN 0x80000057 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:09.668Z cpu3:120958)ScsiDeviceIO: 2309: Cmd(0x4124007a6540) 0x2a, CmdSN 0x8000004f from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x20 0x0.
2012-07-24T17:35:10.068Z cpu0:121481)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:35:10.068Z cpu0:121481)ScsiDeviceIO: 2309: Cmd(0x4124007b5b40) 0x2a, CmdSN 0x80000065 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:10.468Z cpu2:121481)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x41240077cb40, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0. Act:EVAL
2012-07-24T17:35:10.468Z cpu2:121481)ScsiDeviceIO: 2309: Cmd(0x41240077cb40) 0x2a, CmdSN 0x80000027 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:10.868Z cpu0:121481)ScsiDeviceIO: 2309: Cmd(0x4124007b5b40) 0x2a, CmdSN 0x80000063 from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:11.268Z cpu0:121479)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:35:11.268Z cpu0:121479)ScsiDeviceIO: 2309: Cmd(0x4124007b5b40) 0x2a, CmdSN 0x8000005f from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:11.668Z cpu0:121481)ScsiDeviceIO: 2309: Cmd(0x4124007b5b40) 0x2a, CmdSN 0x8000000d from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:11.995Z cpu1:2668)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x410017a59150 network resource pool netsched.pools.persist.iscsi associated
2012-07-24T17:35:11.995Z cpu1:2668)iscsi_vmk: iscsivmk_ConnNetRegister: socket 0x410017a59150 network tracker id 1 tracker.iSCSI.192.168.30.2 associated
2012-07-24T17:35:12.068Z cpu1:121481)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.600144f0c52383000000500928190001" state in doubt; requested fast path state update...
2012-07-24T17:35:12.068Z cpu1:121481)ScsiDeviceIO: 2309: Cmd(0x4124007899c0) 0x2a, CmdSN 0x8000006c from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:12.468Z cpu2:121479)ScsiDeviceIO: 2309: Cmd(0x41240077cb40) 0x2a, CmdSN 0x8000002f from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x5 0x24 0x0.
2012-07-24T17:35:12.497Z cpu1:2668)WARNING: iscsi_vmk: iscsivmk_StartConnection: vmhba38:CH:0 T:0 CN:0: iSCSI connection is being marked "ONLINE"
2012-07-24T17:35:12.497Z cpu1:2668)WARNING: iscsi_vmk: iscsivmk_StartConnection: Sess [ISID: 00023d000001 TARGET: iqn.1986-03.com.sun:02:97a66d96-4de8-417c-dc9b-de72b28ff0e8 TPGT: 1 TSIH: 0]
2012-07-24T17:35:12.497Z cpu1:2668)WARNING: iscsi_vmk: iscsivmk_StartConnection: Conn [CID: 0 L: 192.168.30.1:61163 R: 192.168.30.2:3260]
2012-07-24T17:35:12.894Z cpu0:2052)NMP: nmp_ThrottleLogForDevice:2318: Cmd 0x2a (0x412400774680, 121479) to dev "naa.600144f0c52383000000500928190001" on path "vmhba38:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0. Act:NONE
2012-07-24T17:35:12.894Z cpu0:2052)ScsiDeviceIO: 2309: Cmd(0x412400774680) 0x2a, CmdSN 0x8000005a from world 121479 to dev "naa.600144f0c52383000000500928190001" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x6 0x29 0x0.
2012-07-24T17:35:12.894Z cpu0:2052)ScsiCore: 1460: Power-on Reset occurred on naa.600144f0c52383000000500928190001
2012-07-24T17:35:13.120Z cpu3:121481)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 156577520 microseconds to 30845451 microseconds.
2012-07-24T17:35:13.419Z cpu1:120958)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 30845451 microseconds to 6138355 microseconds.
2012-07-24T17:45:04.991Z cpu1:3729)ScsiDeviceIO: 1198: Device naa.600144f0c52383000000500928190001 performance has improved. I/O latency reduced from 6138355 microseconds to 1207617 microseconds.