API tools faq
paste
Advertisement
Guest User

Ceph MDS crash

a guest
May 27th, 2013
109
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 6.21 KB | None | 0 0
raw download clone embed print report
  1. 2013-05-26 15:40:35.746584 7fba9ca0c700 0 -- 192.168.178.102:6800/1136 >> 192.168.178.46:0/2653615883 pipe(0x1f3f9000 sd=27 :680
  2. 0 s=0 pgs=0 cs=0 l=0).accept connect_seq 0 vs existing 1 state standby
  3. 2013-05-26 15:40:35.746642 7fba9ca0c700 0 -- 192.168.178.102:6800/1136 >> 192.168.178.46:0/2653615883 pipe(0x1f3f9000 sd=27 :680
  4. 0 s=0 pgs=0 cs=0 l=0).accept peer reset, then tried to connect to us, replacing
  5. 2013-05-26 15:40:35.835756 7fbaa1f29700 -1 mds/SessionMap.h: In function 'virtual Session::~Session()' thread 7fbaa1f29700 time 2
  6. 013-05-26 15:40:35.781770
  7. mds/SessionMap.h: 197: FAILED assert(!item_session_list.is_on_list())
  8.  
  9. ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
  10. 1: (Session::~Session()+0x196) [0x4dc0c6]
  11. 2: (Session::~Session()+0x9) [0x4dc269]
  12. 3: (Connection::~Connection()+0x96) [0x7a06b6]
  13. 4: (Pipe::~Pipe()+0x136) [0x83a496]
  14. 5: (Pipe::~Pipe()+0x9) [0x83a559]
  15. 6: (SimpleMessenger::reaper()+0x773) [0x79a943]
  16. 7: (SimpleMessenger::reaper_entry()+0x90) [0x79bfa0]
  17. 8: (SimpleMessenger::ReaperThread::entry()+0xd) [0x7a099d]
  18. 9: (()+0x7e9a) [0x7fbaa5b15e9a]
  19. 10: (clone()+0x6d) [0x7fbaa49b6ccd]
  20. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  21.  
  22. --- begin dump of recent events ---
  23. -10000> 2013-05-26 13:46:25.040813 7fba9ec20700 5 mds.0.bal mds.0 epoch 49967 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0,
  24. cpu 0.03>
  25. -9999> 2013-05-26 13:46:26.763032 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  26. -9998> 2013-05-26 13:46:26.763088 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
  27. :active seq 124966 v51) v2 -- ?+0 0x3c93b00 con 0x158e840
  28. -9997> 2013-05-26 13:46:26.771387 7fbaa0f27700 1 -- 192.168.178.102:6800/1136 <== mon.1 192.168.178.101:6789/0 128346 ==== mdsb
  29. eacon(5501/a up:active seq 124966 v51) v2 ==== 103+0+0 (3729436356 0 0) 0x246f1600 con 0x158e840
  30. -9996> 2013-05-26 13:46:29.781176 7fba9ff25700 10 monclient: tick
  31. -9995> 2013-05-26 13:46:29.781225 7fba9ff25700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 20
  32. 13-05-26 13:45:59.781222)
  33. -9994> 2013-05-26 13:46:29.781256 7fba9ff25700 10 monclient: renew subs? (now: 2013-05-26 13:46:29.781255; renew after: 2013-05-
  34. 26 13:47:09.777394) -- no
  35. -9993> 2013-05-26 13:46:30.040851 7fba9ec20700 2 mds.0.cache check_memory_usage total 965264, rss 785920, heap 20584, malloc 49
  36. 3375 mmap 0, baseline 16460, buffers 0, max 1048576, 0 / 99955 inodes have caps, 0 caps, 0 caps per inode
  37. -9992> 2013-05-26 13:46:30.763401 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  38. -9991> 2013-05-26 13:46:30.763457 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
  39. :active seq 124967 v51) v2 -- ?+0 0x3c94b80 con 0x158e840
  40. -9990> 2013-05-26 13:46:30.771843 7fbaa0f27700 1 -- 192.168.178.102:6800/1136 <== mon.1 192.168.178.101:6789/0 128347 ==== mdsb
  41. eacon(5501/a up:active seq 124967 v51) v2 ==== 103+0+0 (2808337315 0 0) 0x246f1340 con 0x158e840
  42. -9989> 2013-05-26 13:46:34.763772 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  43. -9988> 2013-05-26 13:46:34.763827 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
  44. :active seq 124968 v51) v2 -- ?+0 0x3c948c0 con 0x158e840
  45. -9987> 2013-05-26 13:46:34.772070 7fbaa0f27700 1 -- 192.168.178.102:6800/1136 <== mon.1 192.168.178.101:6789/0 128348 ==== mdsb
  46. eacon(5501/a up:active seq 124968 v51) v2 ==== 103+0+0 (170829644 0 0) 0x246f1080 con 0x158e840
  47. -9986> 2013-05-26 13:46:35.041118 7fba9ec20700 2 mds.0.cache check_memory_usage total 965264, rss 785920, heap 20584, malloc 49
  48. 3376 mmap 0, baseline 16460, buffers 0, max 1048576, 0 / 99955 inodes have caps, 0 caps, 0 caps per inode
  49. -9985> 2013-05-26 13:46:35.041312 7fba9ec20700 5 mds.0.bal mds.0 epoch 49968 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0,
  50. cpu 0.02>
  51. -9984> 2013-05-26 13:46:38.764140 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  52. -9983> 2013-05-26 13:46:38.764196 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
  53. :active seq 124969 v51) v2 -- ?+0 0x255b2b80 con 0x158e840
  54. -9982> 2013-05-26 13:46:38.772413 7fbaa0f27700 1 -- 192.168.178.102:6800/1136 <== mon.1 192.168.178.101:6789/0 128349 ==== mdsb
  55. eacon(5501/a up:active seq 124969 v51) v2 ==== 103+0+0 (1929894443 0 0) 0x246f0dc0 con 0x158e840
  56. -9981> 2013-05-26 13:46:39.781573 7fba9ff25700 10 monclient: tick
  57. -9980> 2013-05-26 13:46:39.781622 7fba9ff25700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 20
  58. 13-05-26 13:46:09.781619)
  59. -9979> 2013-05-26 13:46:39.781655 7fba9ff25700 10 monclient: renew subs? (now: 2013-05-26 13:46:39.781655; renew after: 2013-05-
  60. 26 13:47:09.777394) -- no
  61. -9978> 2013-05-26 13:46:40.041353 7fba9ec20700 2 mds.0.cache check_memory_usage total 965264, rss 785920, heap 20584, malloc 49
  62. 3376 mmap 0, baseline 16460, buffers 0, max 1048576, 0 / 99955 inodes have caps, 0 caps, 0 caps per inode
  63. -9977> 2013-05-26 13:46:42.764516 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  64. -9976> 2013-05-26 13:46:42.764572 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
  65. :active seq 124970 v51) v2 -- ?+0 0x255b2600 con 0x158e840
  66. -9975> 2013-05-26 13:46:42.772893 7fbaa0f27700 1 -- 192.168.178.102:6800/1136 <== mon.1 192.168.178.101:6789/0 128350 ==== mdsb
  67. eacon(5501/a up:active seq 124970 v51) v2 ==== 103+0+0 (4168899970 0 0) 0x246f0b00 con 0x158e840
  68. -9974> 2013-05-26 13:46:45.041604 7fba9ec20700 2 mds.0.cache check_memory_usage total 965264, rss 785920, heap 20584, malloc 49
  69. 3376 mmap 0, baseline 16460, buffers 0, max 1048576, 0 / 99955 inodes have caps, 0 caps, 0 caps per inode
  70. -9973> 2013-05-26 13:46:45.041799 7fba9ec20700 5 mds.0.bal mds.0 epoch 49969 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0,
  71. cpu 0.02>
  72. -9972> 2013-05-26 13:46:46.764890 7fba9ec20700 10 monclient: _send_mon_message to mon.b at 192.168.178.101:6789/0
  73. -9971> 2013-05-26 13:46:46.764945 7fba9ec20700 1 -- 192.168.178.102:6800/1136 --> 192.168.178.101:6789/0 -- mdsbeacon(5501/a up
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!