Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- -41> 2015-02-20 16:55:13.839832 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x24396360 con 0x1d041600
- -40> 2015-02-20 16:55:18.239482 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -39> 2015-02-20 16:55:18.239510 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -38> 2015-02-20 16:55:18.239512 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -37> 2015-02-20 16:55:23.239576 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -36> 2015-02-20 16:55:23.239591 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -35> 2015-02-20 16:55:23.239593 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -34> 2015-02-20 16:55:23.839974 7f7935f19700 10 monclient: tick
- -33> 2015-02-20 16:55:23.839991 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:54:53.839990)
- -32> 2015-02-20 16:55:23.840000 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:23.840000; renew after: 2015-02-20 16:55:04.473023) -- yes
- -31> 2015-02-20 16:55:23.840005 7f7935f19700 10 monclient: renew_subs
- -30> 2015-02-20 16:55:23.840008 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
- -29> 2015-02-20 16:55:23.840018 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e03480 con 0x1d041600
- -28> 2015-02-20 16:55:28.239657 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -27> 2015-02-20 16:55:28.239684 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -26> 2015-02-20 16:55:28.239692 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -25> 2015-02-20 16:55:33.239755 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -24> 2015-02-20 16:55:33.239770 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -23> 2015-02-20 16:55:33.239772 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -22> 2015-02-20 16:55:33.840101 7f7935f19700 10 monclient: tick
- -21> 2015-02-20 16:55:33.840151 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:55:03.840150)
- -20> 2015-02-20 16:55:33.840164 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:33.840164; renew after: 2015-02-20 16:55:04.473023) -- yes
- -19> 2015-02-20 16:55:33.840169 7f7935f19700 10 monclient: renew_subs
- -18> 2015-02-20 16:55:33.840173 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
- -17> 2015-02-20 16:55:33.840183 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e00000 con 0x1d041600
- -16> 2015-02-20 16:55:38.239854 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -15> 2015-02-20 16:55:38.239874 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -14> 2015-02-20 16:55:38.239878 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -13> 2015-02-20 16:55:43.239947 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -12> 2015-02-20 16:55:43.239965 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -11> 2015-02-20 16:55:43.239968 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -10> 2015-02-20 16:55:43.840269 7f7935f19700 10 monclient: tick
- -9> 2015-02-20 16:55:43.840287 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:55:13.840286)
- -8> 2015-02-20 16:55:43.840299 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:43.840298; renew after: 2015-02-20 16:55:04.473023) -- yes
- -7> 2015-02-20 16:55:43.840306 7f7935f19700 10 monclient: renew_subs
- -6> 2015-02-20 16:55:43.840311 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
- -5> 2015-02-20 16:55:43.840320 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e07260 con 0x1d041600
- -4> 2015-02-20 16:55:48.240035 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
- -3> 2015-02-20 16:55:48.240053 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
- -2> 2015-02-20 16:55:48.240059 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
- -1> 2015-02-20 16:55:48.240060 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had suicide timed out after 150
- 0> 2015-02-20 16:55:48.257218 7f794b46f700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f794b46f700 time 2015-02-20 16:55:48.241593
- common/HeartbeatMap.cc: 79: FAILED assert(0 == "hit suicide timeout")
- ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
- 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0xb7ac55]
- 2: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
- 3: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
- 4: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
- 5: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
- 6: (()+0x7df3) [0x7f794e243df3]
- 7: (clone()+0x6d) [0x7f794cd261ad]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- logging levels ---
- 0/ 5 none
- 0/ 1 lockdep
- 0/ 1 context
- 1/ 1 crush
- 1/ 5 mds
- 1/ 5 mds_balancer
- 1/ 5 mds_locker
- 1/ 5 mds_log
- 1/ 5 mds_log_expire
- 1/ 5 mds_migrator
- 0/ 1 buffer
- 0/ 1 timer
- 0/ 1 filer
- 0/ 1 striper
- 0/ 1 objecter
- 0/ 5 rados
- 0/ 5 rbd
- 0/ 5 rbd_replay
- 0/ 5 journaler
- 0/ 5 objectcacher
- 0/ 5 client
- 0/ 5 osd
- 0/ 5 optracker
- 0/ 5 objclass
- 1/ 3 filestore
- 1/ 3 keyvaluestore
- 1/ 3 journal
- 0/ 5 ms
- 1/ 5 mon
- 0/10 monc
- 1/ 5 paxos
- 0/ 5 tp
- 1/ 5 auth
- 1/ 5 crypto
- 1/ 1 finisher
- 1/ 5 heartbeatmap
- 1/ 5 perfcounter
- 1/ 5 rgw
- 1/10 civetweb
- 1/ 5 javaclient
- 1/ 5 asok
- 1/ 1 throttle
- 0/ 0 refs
- -2/-2 (syslog threshold)
- -1/-1 (stderr threshold)
- max_recent 10000
- max_new 1000
- log_file /var/log/ceph/ceph-osd.34.log
- --- end dump of recent events ---
- 2015-02-20 16:55:48.306557 7f794b46f700 -1 *** Caught signal (Aborted) **
- in thread 7f794b46f700
- ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
- 1: /usr/bin/ceph-osd() [0xa88332]
- 2: (()+0xf130) [0x7f794e24b130]
- 3: (gsignal()+0x39) [0x7f794cc655c9]
- 4: (abort()+0x148) [0x7f794cc66cd8]
- 5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f794d5699d5]
- 6: (()+0x5e946) [0x7f794d567946]
- 7: (()+0x5e973) [0x7f794d567973]
- 8: (()+0x5eb9f) [0x7f794d567b9f]
- 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x27a) [0xb7ae4a]
- 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
- 11: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
- 12: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
- 13: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
- 14: (()+0x7df3) [0x7f794e243df3]
- 15: (clone()+0x6d) [0x7f794cd261ad]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- begin dump of recent events ---
- 0> 2015-02-20 16:55:48.306557 7f794b46f700 -1 *** Caught signal (Aborted) **
- in thread 7f794b46f700
- ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
- 1: /usr/bin/ceph-osd() [0xa88332]
- 2: (()+0xf130) [0x7f794e24b130]
- 3: (gsignal()+0x39) [0x7f794cc655c9]
- 4: (abort()+0x148) [0x7f794cc66cd8]
- 5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f794d5699d5]
- 6: (()+0x5e946) [0x7f794d567946]
- 7: (()+0x5e973) [0x7f794d567973]
- 8: (()+0x5eb9f) [0x7f794d567b9f]
- 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x27a) [0xb7ae4a]
- 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
- 11: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
- 12: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
- 13: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
- 14: (()+0x7df3) [0x7f794e243df3]
- 15: (clone()+0x6d) [0x7f794cd261ad]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- logging levels ---
- 0/ 5 none
- 0/ 1 lockdep
- 0/ 1 context
- 1/ 1 crush
- 1/ 5 mds
- 1/ 5 mds_balancer
- 1/ 5 mds_locker
- 1/ 5 mds_log
- 1/ 5 mds_log_expire
- 1/ 5 mds_migrator
- 0/ 1 buffer
- 0/ 1 timer
- 0/ 1 filer
- 0/ 1 striper
- 0/ 1 objecter
- 0/ 5 rados
- 0/ 5 rbd
- 0/ 5 rbd_replay
- 0/ 5 journaler
- 0/ 5 objectcacher
- 0/ 5 client
- 0/ 5 osd
- 0/ 5 optracker
- 0/ 5 objclass
- 1/ 3 filestore
- 1/ 3 keyvaluestore
- 1/ 3 journal
- 0/ 5 ms
- 1/ 5 mon
- 0/10 monc
- 1/ 5 paxos
- 0/ 5 tp
- 1/ 5 auth
- 1/ 5 crypto
- 1/ 1 finisher
- 1/ 5 heartbeatmap
- 1/ 5 perfcounter
- 1/ 5 rgw
- 1/10 civetweb
- 1/ 5 javaclient
- 1/ 5 asok
- 1/ 1 throttle
- 0/ 0 refs
- -2/-2 (syslog threshold)
- -1/-1 (stderr threshold)
- max_recent 10000
- max_new 1000
- log_file /var/log/ceph/ceph-osd.34.log
- --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement