API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Jul 26th, 2013
101
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 15.47 KB | None | 0 0
raw download clone embed print report
  1. -83> 2013-07-26 17:46:42.594011 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  2. -82> 2013-07-26 17:46:43.720158 7f79e4eca700 10 monclient: tick
  3. -81> 2013-07-26 17:46:43.720177 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:46:13.720176)
  4. -80> 2013-07-26 17:46:43.720216 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:46:43.720216; renew after: 2013-07-26 17:47:37.719372) -- no
  5. -79> 2013-07-26 17:46:44.322830 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  6. -78> 2013-07-26 17:46:47.594097 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  7. -77> 2013-07-26 17:46:47.594117 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  8. -76> 2013-07-26 17:46:47.594119 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  9. -75> 2013-07-26 17:46:47.594122 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  10. -74> 2013-07-26 17:46:50.222960 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  11. -73> 2013-07-26 17:46:52.594208 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  12. -72> 2013-07-26 17:46:52.594228 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  13. -71> 2013-07-26 17:46:52.594231 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  14. -70> 2013-07-26 17:46:52.594232 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  15. -69> 2013-07-26 17:46:53.720300 7f79e4eca700 10 monclient: tick
  16. -68> 2013-07-26 17:46:53.720321 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:46:23.720319)
  17. -67> 2013-07-26 17:46:53.720357 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:46:53.720357; renew after: 2013-07-26 17:47:37.719372) -- no
  18. -66> 2013-07-26 17:46:54.923076 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  19. -65> 2013-07-26 17:46:57.223192 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  20. -64> 2013-07-26 17:46:57.594315 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  21. -63> 2013-07-26 17:46:57.594334 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  22. -62> 2013-07-26 17:46:57.594338 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  23. -61> 2013-07-26 17:46:57.594339 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  24. -60> 2013-07-26 17:47:01.323325 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  25. -59> 2013-07-26 17:47:02.423490 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  26. -58> 2013-07-26 17:47:02.594417 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  27. -57> 2013-07-26 17:47:02.594436 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  28. -56> 2013-07-26 17:47:02.594441 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  29. -55> 2013-07-26 17:47:02.594442 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  30. -54> 2013-07-26 17:47:03.720453 7f79e4eca700 10 monclient: tick
  31. -53> 2013-07-26 17:47:03.720472 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:46:33.720471)
  32. -52> 2013-07-26 17:47:03.720483 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:47:03.720483; renew after: 2013-07-26 17:47:37.719372) -- no
  33. -51> 2013-07-26 17:47:04.123614 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26444 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  34. -50> 2013-07-26 17:47:07.594537 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  35. -49> 2013-07-26 17:47:07.594557 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  36. -48> 2013-07-26 17:47:07.594559 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  37. -47> 2013-07-26 17:47:07.594560 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  38. -46> 2013-07-26 17:47:10.023753 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26412 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  39. -45> 2013-07-26 17:47:12.323892 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26412 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  40. -44> 2013-07-26 17:47:12.594647 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  41. -43> 2013-07-26 17:47:12.594667 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  42. -42> 2013-07-26 17:47:12.594669 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  43. -41> 2013-07-26 17:47:12.594671 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  44. -40> 2013-07-26 17:47:13.424007 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26412 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  45. -39> 2013-07-26 17:47:13.720605 7f79e4eca700 10 monclient: tick
  46. -38> 2013-07-26 17:47:13.720627 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:46:43.720625)
  47. -37> 2013-07-26 17:47:13.720663 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:47:13.720663; renew after: 2013-07-26 17:47:37.719372) -- no
  48. -36> 2013-07-26 17:47:17.594754 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  49. -35> 2013-07-26 17:47:17.594774 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  50. -34> 2013-07-26 17:47:17.594779 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  51. -33> 2013-07-26 17:47:17.594780 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  52. -32> 2013-07-26 17:47:19.324169 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  53. -31> 2013-07-26 17:47:22.224304 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  54. -30> 2013-07-26 17:47:22.594865 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  55. -29> 2013-07-26 17:47:22.594884 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  56. -28> 2013-07-26 17:47:22.594887 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  57. -27> 2013-07-26 17:47:22.594888 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  58. -26> 2013-07-26 17:47:23.720761 7f79e4eca700 10 monclient: tick
  59. -25> 2013-07-26 17:47:23.720781 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:46:53.720779)
  60. -24> 2013-07-26 17:47:23.720817 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:47:23.720817; renew after: 2013-07-26 17:47:37.719372) -- no
  61. -23> 2013-07-26 17:47:26.924422 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  62. -22> 2013-07-26 17:47:27.594974 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  63. -21> 2013-07-26 17:47:27.594994 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  64. -20> 2013-07-26 17:47:27.594998 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  65. -19> 2013-07-26 17:47:27.594999 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  66. -18> 2013-07-26 17:47:32.224551 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  67. -17> 2013-07-26 17:47:32.595084 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  68. -16> 2013-07-26 17:47:32.595103 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  69. -15> 2013-07-26 17:47:32.595105 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  70. -14> 2013-07-26 17:47:32.595106 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  71. -13> 2013-07-26 17:47:33.720908 7f79e4eca700 10 monclient: tick
  72. -12> 2013-07-26 17:47:33.720928 7f79e4eca700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2013-07-26 17:47:03.720926)
  73. -11> 2013-07-26 17:47:33.720964 7f79e4eca700 10 monclient: renew subs? (now: 2013-07-26 17:47:33.720964; renew after: 2013-07-26 17:47:37.719372) -- no
  74. -10> 2013-07-26 17:47:35.124641 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  75. -9> 2013-07-26 17:47:37.595199 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  76. -8> 2013-07-26 17:47:37.595218 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e36c7700' had timed out after 15
  77. -7> 2013-07-26 17:47:37.595220 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e3ec8700' had timed out after 15
  78. -6> 2013-07-26 17:47:37.595223 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e2ec6700' had timed out after 15
  79. -5> 2013-07-26 17:47:38.624793 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  80. -4> 2013-07-26 17:47:39.124913 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  81. -3> 2013-07-26 17:47:41.425053 7f79df6bf700 5 osd.0 1551 heartbeat: osd_stat(26396 MB used, 20459 GB avail, 20485 GB total, peers []/[])
  82. -2> 2013-07-26 17:47:42.595306 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had timed out after 15
  83. -1> 2013-07-26 17:47:42.595326 7f79f1ee4700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f79e26c5700' had suicide timed out after 150
  84. 0> 2013-07-26 17:47:42.596351 7f79f1ee4700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f79f1ee4700 time 2013-07-26 17:47:42.595373
  85. common/HeartbeatMap.cc: 79: FAILED assert(0 == "hit suicide timeout")
  86.  
  87. ceph version 0.61.7 (8f010aff684e820ecc837c25ac77c7a05d7191ff)
  88. 1: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x107) [0x7ab187]
  89. 2: (ceph::HeartbeatMap::is_healthy()+0xa7) [0x7abb37]
  90. 3: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x7ac0e3]
  91. 4: (CephContextServiceThread::entry()+0x55) [0x84f2b5]
  92. 5: (()+0x7e9a) [0x7f79f4eede9a]
  93. 6: (clone()+0x6d) [0x7f79f376bccd]
  94. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  95.  
  96. --- logging levels ---
  97. 0/ 5 none
  98. 0/ 1 lockdep
  99. 0/ 1 context
  100. 1/ 1 crush
  101. 1/ 5 mds
  102. 1/ 5 mds_balancer
  103. 1/ 5 mds_locker
  104. 1/ 5 mds_log
  105. 1/ 5 mds_log_expire
  106. 1/ 5 mds_migrator
  107. 0/ 1 buffer
  108. 0/ 1 timer
  109. 0/ 1 filer
  110. 0/ 1 striper
  111. 0/ 1 objecter
  112. 0/ 5 rados
  113. 0/ 5 rbd
  114. 0/ 5 journaler
  115. 0/ 5 objectcacher
  116. 0/ 5 client
  117. 0/ 5 osd
  118. 0/ 5 optracker
  119. 0/ 5 objclass
  120. 1/ 3 filestore
  121. 1/ 3 journal
  122. 0/ 5 ms
  123. 1/ 5 mon
  124. 0/10 monc
  125. 0/ 5 paxos
  126. 0/ 5 tp
  127. 1/ 5 auth
  128. 1/ 5 crypto
  129. 1/ 1 finisher
  130. 1/ 5 heartbeatmap
  131. 1/ 5 perfcounter
  132. 1/ 5 rgw
  133. 1/ 5 hadoop
  134. 1/ 5 javaclient
  135. 1/ 5 asok
  136. 1/ 1 throttle
  137. -2/-2 (syslog threshold)
  138. -1/-1 (stderr threshold)
  139. max_recent 10000
  140. max_new 1000
  141. log_file /var/log/ceph/ceph-osd.0.log
  142. --- end dump of recent events ---
  143. 2013-07-26 17:47:42.599717 7f79f1ee4700 -1 *** Caught signal (Aborted) **
  144. in thread 7f79f1ee4700
  145.  
  146. ceph version 0.61.7 (8f010aff684e820ecc837c25ac77c7a05d7191ff)
  147. 1: /usr/bin/ceph-osd() [0x79219a]
  148. 2: (()+0xfcb0) [0x7f79f4ef5cb0]
  149. 3: (gsignal()+0x35) [0x7f79f36ae425]
  150. 4: (abort()+0x17b) [0x7f79f36b1b8b]
  151. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f79f400069d]
  152. 6: (()+0xb5846) [0x7f79f3ffe846]
  153. 7: (()+0xb5873) [0x7f79f3ffe873]
  154. 8: (()+0xb596e) [0x7f79f3ffe96e]
  155. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x84303f]
  156. 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x107) [0x7ab187]
  157. 11: (ceph::HeartbeatMap::is_healthy()+0xa7) [0x7abb37]
  158. 12: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x7ac0e3]
  159. 13: (CephContextServiceThread::entry()+0x55) [0x84f2b5]
  160. 14: (()+0x7e9a) [0x7f79f4eede9a]
  161. 15: (clone()+0x6d) [0x7f79f376bccd]
  162. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  163.  
  164. --- begin dump of recent events ---
  165. 0> 2013-07-26 17:47:42.599717 7f79f1ee4700 -1 *** Caught signal (Aborted) **
  166. in thread 7f79f1ee4700
  167.  
  168. ceph version 0.61.7 (8f010aff684e820ecc837c25ac77c7a05d7191ff)
  169. 1: /usr/bin/ceph-osd() [0x79219a]
  170. 2: (()+0xfcb0) [0x7f79f4ef5cb0]
  171. 3: (gsignal()+0x35) [0x7f79f36ae425]
  172. 4: (abort()+0x17b) [0x7f79f36b1b8b]
  173. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f79f400069d]
  174. 6: (()+0xb5846) [0x7f79f3ffe846]
  175. 7: (()+0xb5873) [0x7f79f3ffe873]
  176. 8: (()+0xb596e) [0x7f79f3ffe96e]
  177. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x84303f]
  178. 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x107) [0x7ab187]
  179. 11: (ceph::HeartbeatMap::is_healthy()+0xa7) [0x7abb37]
  180. 12: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x7ac0e3]
  181. 13: (CephContextServiceThread::entry()+0x55) [0x84f2b5]
  182. 14: (()+0x7e9a) [0x7f79f4eede9a]
  183. 15: (clone()+0x6d) [0x7f79f376bccd]
  184. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  185.  
  186. --- logging levels ---
  187. 0/ 5 none
  188. 0/ 1 lockdep
  189. 0/ 1 context
  190. 1/ 1 crush
  191. 1/ 5 mds
  192. 1/ 5 mds_balancer
  193. 1/ 5 mds_locker
  194. 1/ 5 mds_log
  195. 1/ 5 mds_log_expire
  196. 1/ 5 mds_migrator
  197. 0/ 1 buffer
  198. 0/ 1 timer
  199. 0/ 1 filer
  200. 0/ 1 striper
  201. 0/ 1 objecter
  202. 0/ 5 rados
  203. 0/ 5 rbd
  204. 0/ 5 journaler
  205. 0/ 5 objectcacher
  206. 0/ 5 client
  207. 0/ 5 osd
  208. 0/ 5 optracker
  209. 0/ 5 objclass
  210. 1/ 3 filestore
  211. 1/ 3 journal
  212. 0/ 5 ms
  213. 1/ 5 mon
  214. 0/10 monc
  215. 0/ 5 paxos
  216. 0/ 5 tp
  217. 1/ 5 auth
  218. 1/ 5 crypto
  219. 1/ 1 finisher
  220. 1/ 5 heartbeatmap
  221. 1/ 5 perfcounter
  222. 1/ 5 rgw
  223. 1/ 5 hadoop
  224. 1/ 5 javaclient
  225. 1/ 5 asok
  226. 1/ 1 throttle
  227. -2/-2 (syslog threshold)
  228. -1/-1 (stderr threshold)
  229. max_recent 10000
  230. max_new 1000
  231. log_file /var/log/ceph/ceph-osd.0.log
  232. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!