Advertisement
Guest User

Untitled

a guest
Feb 20th, 2015
330
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.18 KB | None | 0 0
  1. -41> 2015-02-20 16:55:13.839832 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x24396360 con 0x1d041600
  2. -40> 2015-02-20 16:55:18.239482 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  3. -39> 2015-02-20 16:55:18.239510 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  4. -38> 2015-02-20 16:55:18.239512 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  5. -37> 2015-02-20 16:55:23.239576 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  6. -36> 2015-02-20 16:55:23.239591 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  7. -35> 2015-02-20 16:55:23.239593 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  8. -34> 2015-02-20 16:55:23.839974 7f7935f19700 10 monclient: tick
  9. -33> 2015-02-20 16:55:23.839991 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:54:53.839990)
  10. -32> 2015-02-20 16:55:23.840000 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:23.840000; renew after: 2015-02-20 16:55:04.473023) -- yes
  11. -31> 2015-02-20 16:55:23.840005 7f7935f19700 10 monclient: renew_subs
  12. -30> 2015-02-20 16:55:23.840008 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
  13. -29> 2015-02-20 16:55:23.840018 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e03480 con 0x1d041600
  14. -28> 2015-02-20 16:55:28.239657 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  15. -27> 2015-02-20 16:55:28.239684 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  16. -26> 2015-02-20 16:55:28.239692 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  17. -25> 2015-02-20 16:55:33.239755 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  18. -24> 2015-02-20 16:55:33.239770 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  19. -23> 2015-02-20 16:55:33.239772 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  20. -22> 2015-02-20 16:55:33.840101 7f7935f19700 10 monclient: tick
  21. -21> 2015-02-20 16:55:33.840151 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:55:03.840150)
  22. -20> 2015-02-20 16:55:33.840164 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:33.840164; renew after: 2015-02-20 16:55:04.473023) -- yes
  23. -19> 2015-02-20 16:55:33.840169 7f7935f19700 10 monclient: renew_subs
  24. -18> 2015-02-20 16:55:33.840173 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
  25. -17> 2015-02-20 16:55:33.840183 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e00000 con 0x1d041600
  26. -16> 2015-02-20 16:55:38.239854 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  27. -15> 2015-02-20 16:55:38.239874 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  28. -14> 2015-02-20 16:55:38.239878 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  29. -13> 2015-02-20 16:55:43.239947 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  30. -12> 2015-02-20 16:55:43.239965 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  31. -11> 2015-02-20 16:55:43.239968 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  32. -10> 2015-02-20 16:55:43.840269 7f7935f19700 10 monclient: tick
  33. -9> 2015-02-20 16:55:43.840287 7f7935f19700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2015-02-20 16:55:13.840286)
  34. -8> 2015-02-20 16:55:43.840299 7f7935f19700 10 monclient: renew subs? (now: 2015-02-20 16:55:43.840298; renew after: 2015-02-20 16:55:04.473023) -- yes
  35. -7> 2015-02-20 16:55:43.840306 7f7935f19700 10 monclient: renew_subs
  36. -6> 2015-02-20 16:55:43.840311 7f7935f19700 10 monclient: _send_mon_message to mon.ceph0-mon2 at 10.1.8.42:6789/0
  37. -5> 2015-02-20 16:55:43.840320 7f7935f19700 1 -- 10.1.8.52:6830/25483 --> 10.1.8.42:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=4556}) v2 -- ?+0 0x20e07260 con 0x1d041600
  38. -4> 2015-02-20 16:55:48.240035 7f794b46f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f792ef0b700' had timed out after 30
  39. -3> 2015-02-20 16:55:48.240053 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f792f70c700' had timed out after 15
  40. -2> 2015-02-20 16:55:48.240059 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had timed out after 15
  41. -1> 2015-02-20 16:55:48.240060 7f794b46f700 1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f7931710700' had suicide timed out after 150
  42. 0> 2015-02-20 16:55:48.257218 7f794b46f700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f794b46f700 time 2015-02-20 16:55:48.241593
  43. common/HeartbeatMap.cc: 79: FAILED assert(0 == "hit suicide timeout")
  44.  
  45. ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
  46. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0xb7ac55]
  47. 2: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
  48. 3: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
  49. 4: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
  50. 5: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
  51. 6: (()+0x7df3) [0x7f794e243df3]
  52. 7: (clone()+0x6d) [0x7f794cd261ad]
  53. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  54.  
  55. --- logging levels ---
  56. 0/ 5 none
  57. 0/ 1 lockdep
  58. 0/ 1 context
  59. 1/ 1 crush
  60. 1/ 5 mds
  61. 1/ 5 mds_balancer
  62. 1/ 5 mds_locker
  63. 1/ 5 mds_log
  64. 1/ 5 mds_log_expire
  65. 1/ 5 mds_migrator
  66. 0/ 1 buffer
  67. 0/ 1 timer
  68. 0/ 1 filer
  69. 0/ 1 striper
  70. 0/ 1 objecter
  71. 0/ 5 rados
  72. 0/ 5 rbd
  73. 0/ 5 rbd_replay
  74. 0/ 5 journaler
  75. 0/ 5 objectcacher
  76. 0/ 5 client
  77. 0/ 5 osd
  78. 0/ 5 optracker
  79. 0/ 5 objclass
  80. 1/ 3 filestore
  81. 1/ 3 keyvaluestore
  82. 1/ 3 journal
  83. 0/ 5 ms
  84. 1/ 5 mon
  85. 0/10 monc
  86. 1/ 5 paxos
  87. 0/ 5 tp
  88. 1/ 5 auth
  89. 1/ 5 crypto
  90. 1/ 1 finisher
  91. 1/ 5 heartbeatmap
  92. 1/ 5 perfcounter
  93. 1/ 5 rgw
  94. 1/10 civetweb
  95. 1/ 5 javaclient
  96. 1/ 5 asok
  97. 1/ 1 throttle
  98. 0/ 0 refs
  99. -2/-2 (syslog threshold)
  100. -1/-1 (stderr threshold)
  101. max_recent 10000
  102. max_new 1000
  103. log_file /var/log/ceph/ceph-osd.34.log
  104. --- end dump of recent events ---
  105. 2015-02-20 16:55:48.306557 7f794b46f700 -1 *** Caught signal (Aborted) **
  106. in thread 7f794b46f700
  107.  
  108. ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
  109. 1: /usr/bin/ceph-osd() [0xa88332]
  110. 2: (()+0xf130) [0x7f794e24b130]
  111. 3: (gsignal()+0x39) [0x7f794cc655c9]
  112. 4: (abort()+0x148) [0x7f794cc66cd8]
  113. 5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f794d5699d5]
  114. 6: (()+0x5e946) [0x7f794d567946]
  115. 7: (()+0x5e973) [0x7f794d567973]
  116. 8: (()+0x5eb9f) [0x7f794d567b9f]
  117. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x27a) [0xb7ae4a]
  118. 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
  119. 11: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
  120. 12: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
  121. 13: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
  122. 14: (()+0x7df3) [0x7f794e243df3]
  123. 15: (clone()+0x6d) [0x7f794cd261ad]
  124. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  125.  
  126. --- begin dump of recent events ---
  127. 0> 2015-02-20 16:55:48.306557 7f794b46f700 -1 *** Caught signal (Aborted) **
  128. in thread 7f794b46f700
  129.  
  130. ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
  131. 1: /usr/bin/ceph-osd() [0xa88332]
  132. 2: (()+0xf130) [0x7f794e24b130]
  133. 3: (gsignal()+0x39) [0x7f794cc655c9]
  134. 4: (abort()+0x148) [0x7f794cc66cd8]
  135. 5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f794d5699d5]
  136. 6: (()+0x5e946) [0x7f794d567946]
  137. 7: (()+0x5e973) [0x7f794d567973]
  138. 8: (()+0x5eb9f) [0x7f794d567b9f]
  139. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x27a) [0xb7ae4a]
  140. 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x2d9) [0xab31d9]
  141. 11: (ceph::HeartbeatMap::is_healthy()+0xde) [0xab3ace]
  142. 12: (ceph::HeartbeatMap::check_touch_file()+0x2c) [0xab41ec]
  143. 13: (CephContextServiceThread::entry()+0x15b) [0xb8f78b]
  144. 14: (()+0x7df3) [0x7f794e243df3]
  145. 15: (clone()+0x6d) [0x7f794cd261ad]
  146. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  147.  
  148. --- logging levels ---
  149. 0/ 5 none
  150. 0/ 1 lockdep
  151. 0/ 1 context
  152. 1/ 1 crush
  153. 1/ 5 mds
  154. 1/ 5 mds_balancer
  155. 1/ 5 mds_locker
  156. 1/ 5 mds_log
  157. 1/ 5 mds_log_expire
  158. 1/ 5 mds_migrator
  159. 0/ 1 buffer
  160. 0/ 1 timer
  161. 0/ 1 filer
  162. 0/ 1 striper
  163. 0/ 1 objecter
  164. 0/ 5 rados
  165. 0/ 5 rbd
  166. 0/ 5 rbd_replay
  167. 0/ 5 journaler
  168. 0/ 5 objectcacher
  169. 0/ 5 client
  170. 0/ 5 osd
  171. 0/ 5 optracker
  172. 0/ 5 objclass
  173. 1/ 3 filestore
  174. 1/ 3 keyvaluestore
  175. 1/ 3 journal
  176. 0/ 5 ms
  177. 1/ 5 mon
  178. 0/10 monc
  179. 1/ 5 paxos
  180. 0/ 5 tp
  181. 1/ 5 auth
  182. 1/ 5 crypto
  183. 1/ 1 finisher
  184. 1/ 5 heartbeatmap
  185. 1/ 5 perfcounter
  186. 1/ 5 rgw
  187. 1/10 civetweb
  188. 1/ 5 javaclient
  189. 1/ 5 asok
  190. 1/ 1 throttle
  191. 0/ 0 refs
  192. -2/-2 (syslog threshold)
  193. -1/-1 (stderr threshold)
  194. max_recent 10000
  195. max_new 1000
  196. log_file /var/log/ceph/ceph-osd.34.log
  197. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement