Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- 2012-09-19 14:49:02.627890 7fdffd6ec700 -1 mon.0@0(leader) e1 *** Got Signal Terminated ***
- 2012-09-19 14:49:02.627917 7fdffd6ec700 1 mon.0@0(leader) e1 shutdown
- 2012-09-19 14:49:09.378872 7fc09a99b780 1 store(/var/ceph/mon0) mount
- 2012-09-19 14:49:09.379083 7fc09a99b780 0 ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30), process ceph-mon, pid 17420
- 2012-09-19 14:49:09.380112 7fc09a99b780 1 mon.0@-1(probing) e1 init fsid 167d0267-4fde-440c-a624-3c82dba448a6
- 2012-09-19 14:49:09.392701 7fc09a997700 0 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x1ffa240 sd=18 pgs=0 cs=0 l=0).accept we reset (peer sent cseq 2), sending RESETSESSION
- 2012-09-19 14:49:09.745380 7fc0958a8700 0 -- 10.87.1.87:6789/0 >> 10.87.1.88:6789/0 pipe(0x58cd440 sd=17 pgs=0 cs=0 l=0).accept we reset (peer sent cseq 2), sending RESETSESSION
- 2012-09-19 14:49:09.782084 7fc09a99b780 0 mon.0@-1(probing) e1 my rank is now 0 (was -1)
- 2012-09-19 14:49:09.783022 7fc0957a7700 0 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58cdb00 sd=17 pgs=595 cs=1 l=0).fault initiating reconnect
- 2012-09-19 14:49:09.783346 7fc0956a6700 -1 msg/SimpleMessenger.cc: In function 'void SimpleMessenger::Pipe::register_pipe()' thread 7fc0956a6700 time 2012-09-19 14:49:09.782424
- msg/SimpleMessenger.cc: 1295: FAILED assert(msgr->rank_pipe.count(peer_addr) == 0)
- ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30)
- 1: (SimpleMessenger::Pipe::register_pipe()+0x25e) [0x5ff9ae]
- 2: (SimpleMessenger::connect_rank(entity_addr_t const&, int, Connection*)+0x240) [0x609fe0]
- 3: (SimpleMessenger::submit_message(Message*, Connection*, entity_addr_t const&, int, bool)+0xaae) [0x60ac8e]
- 4: (SimpleMessenger::_send_message(Message*, Connection*, bool)+0xbe) [0x60b08e]
- 5: (Monitor::handle_probe_probe(MMonProbe*)+0x4b5) [0x48f225]
- 6: (Monitor::handle_probe(MMonProbe*)+0x25b) [0x4940bb]
- 7: (Monitor::_ms_dispatch(Message*)+0x12b3) [0x49d3b3]
- 8: (Monitor::ms_dispatch(Message*)+0x89) [0x4ac959]
- 9: (SimpleMessenger::DispatchQueue::entry()+0x493) [0x608b83]
- 10: (SimpleMessenger::dispatch_entry()+0x24) [0x609d34]
- 11: (SimpleMessenger::DispatchThread::entry()+0xd) [0x5d920d]
- 12: (()+0x7d0c) [0x7fc09a580d0c]
- 13: (clone()+0x6d) [0x7fc098ca5bed]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- begin dump of recent events ---
- -56> 2012-09-19 14:49:09.375924 7fc09a99b780 5 asok(0x1ffc000) register_command perfcounters_dump hook 0x1fe6010
- -55> 2012-09-19 14:49:09.375949 7fc09a99b780 5 asok(0x1ffc000) register_command 1 hook 0x1fe6010
- -54> 2012-09-19 14:49:09.375958 7fc09a99b780 5 asok(0x1ffc000) register_command perf dump hook 0x1fe6010
- -54> 2012-09-19 14:49:09.375966 7fc09a99b780 5 asok(0x1ffc000) register_command perfcounters_schema hook 0x1fe6010
- 2012-09-19 14:49:09.783476 7fc0945a3700 0 -53> 2012-09-19 14:49:09.375970 7fc09a99b780 5 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58ccfc0 sd=20 pgs=596 cs=1 l=0).fault with nothing to send, going to standbyasok(0x1ffc000) register_command 2 hook 0x1fe6010
- -53> 2012-09-19 14:49:09.375972 7fc09a99b780 5 asok(0x1ffc000) register_command perf schema hook 0x1fe6010
- -52> 2012-09-19 14:49:09.375979 7fc09a99b780 5 asok(0x1ffc000) register_command config show hook 0x1fe6010
- -51> 2012-09-19 14:49:09.375986 7fc09a99b780 5 asok(0x1ffc000) register_command config set hook 0x1fe6010
- -50> 2012-09-19 14:49:09.375990 7fc09a99b780 5 asok(0x1ffc000) register_command log flush hook 0x1fe6010
- -49> 2012-09-19 14:49:09.375996 7fc09a99b780 5 asok(0x1ffc000) register_command log dump hook 0x1fe6010
- -48> 2012-09-19 14:49:09.375998 7fc09a99b780 5 asok(0x1ffc000) register_command log reopen hook 0x1fe6010
- -47> 2012-09-19 14:49:09.378872 7fc09a99b780 1 store(/var/ceph/mon0) mount
- -46> 2012-09-19 14:49:09.379083 7fc09a99b780 0 ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30), process ceph-mon, pid 17420
- -45> 2012-09-19 14:49:09.379158 7fc09a99b780 1 -- 10.87.1.87:6789/0 accepter.bind my_inst.addr is 10.87.1.87:6789/0 need_addr=0
- -44> 2012-09-19 14:49:09.379649 7fc09a99b780 1 finished global_init_daemonize
- -43> 2012-09-19 14:49:09.379847 7fc09a99b780 5 asok(0x1ffc000) init /var/run/ceph/ceph-mon.0.asok
- -42> 2012-09-19 14:49:09.379868 7fc09a99b780 5 asok(0x1ffc000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
- -41> 2012-09-19 14:49:09.379907 7fc09a99b780 5 asok(0x1ffc000) register_command 0 hook 0x1fe4038
- -40> 2012-09-19 14:49:09.379913 7fc09a99b780 5 asok(0x1ffc000) register_command version hook 0x1fe4038
- -39> 2012-09-19 14:49:09.379920 7fc09a99b780 5 asok(0x1ffc000) register_command git_version hook 0x1fe4038
- -38> 2012-09-19 14:49:09.379931 7fc09a99b780 5 asok(0x1ffc000) register_command help hook 0x1fe6050
- -37> 2012-09-19 14:49:09.379961 7fc0979ad700 5 asok(0x1ffc000) entry start
- -36> 2012-09-19 14:49:09.379984 7fc09a99b780 1 -- 10.87.1.87:6789/0 messenger.start
- -35> 2012-09-19 14:49:09.380007 7fc09a99b780 1 -- 10.87.1.87:6789/0 accepter.start
- -34> 2012-09-19 14:49:09.380112 7fc09a99b780 1 mon.0@-1(probing) e1 init fsid 167d0267-4fde-440c-a624-3c82dba448a6
- -33> 2012-09-19 14:49:09.392566 7fc09a997700 1 -- 10.87.1.87:6789/0 >> :/0 pipe(0x1ffa240 sd=18 pgs=0 cs=0 l=0).accept sd=18
- -32> 2012-09-19 14:49:09.392701 7fc09a997700 0 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x1ffa240 sd=18 pgs=0 cs=0 l=0).accept we reset (peer sent cseq 2), sending RESETSESSION
- -31> 2012-09-19 14:49:09.745137 7fc09a997700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) get 74 (0 -> 74)
- -30> 2012-09-19 14:49:09.745253 7fc0958a8700 1 -- 10.87.1.87:6789/0 >> :/0 pipe(0x58cd440 sd=17 pgs=0 cs=0 l=0).accept sd=17
- -29> 2012-09-19 14:49:09.745380 7fc0958a8700 0 -- 10.87.1.87:6789/0 >> 10.87.1.88:6789/0 pipe(0x58cd440 sd=17 pgs=0 cs=0 l=0).accept we reset (peer sent cseq 2), sending RESETSESSION
- -29> 2012-09-19 14:49:09.754039 7fc09a997700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) get 501 (74 -> 575)
- -28> 2012-09-19 14:49:09.772365 7fc0958a8700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) get 501 (575 -> 1076)
- -27> 2012-09-19 14:49:09.780342 7fc09a99b780 4 mon.0@-1(probing).mds e17 new map
- -26> 2012-09-19 14:49:09.781999 7fc09a99b780 2 auth: KeyRing::load: loaded key file /var/ceph/mon0/keyring
- -25> 2012-09-19 14:49:09.782007 7fc09a99b780 5 asok(0x1ffc000) register_command mon_status hook 0x3806960
- -24> 2012-09-19 14:49:09.782014 7fc09a99b780 5 asok(0x1ffc000) register_command quorum_status hook 0x3806960
- -23> 2012-09-19 14:49:09.782020 7fc09a99b780 5 asok(0x1ffc000) register_command add_bootstrap_peer_hint hook 0x3806960
- -22> 2012-09-19 14:49:09.782084 7fc09a99b780 0 mon.0@-1(probing) e1 my rank is now 0 (was -1)
- -21> 2012-09-19 14:49:09.782068 7fc0956a6700 1 -- 10.87.1.87:6789/0 <== mon.2 10.87.1.104:6789/0 1 ==== mon_probe(probe 167d0267-4fde-440c-a624-3c82dba448a6 name 2) v2 ==== 74+0+0 (1321215285 0 0) 0x55a0000 con 0x200af00
- -20> 2012-09-19 14:49:09.782092 7fc09a99b780 1 -- 10.87.1.87:6789/0 mark_down_all
- -19> 2012-09-19 14:49:09.782098 7fc09a99b780 5 -- 10.87.1.87:6789/0 mark_down_all 10.87.1.104:6789/0 0x1ffa240
- -18> 2012-09-19 14:49:09.782130 7fc09a99b780 5 -- 10.87.1.87:6789/0 mark_down_all 10.87.1.88:6789/0 0x58cd440
- -17> 2012-09-19 14:49:09.782131 7fc09a997700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x1ffa240 sd=18 pgs=591 cs=1 l=0).reader couldn't read tag, Success
- -16> 2012-09-19 14:49:09.782159 7fc09a99b780 1 -- 10.87.1.87:6789/0 --> mon.1 10.87.1.88:6789/0 -- mon_probe(probe 167d0267-4fde-440c-a624-3c82dba448a6 name 0) v2 -- ?+0 0x55a0900
- -15> 2012-09-19 14:49:09.782149 7fc0958a8700 2 -- 10.87.1.87:6789/0 >> 10.87.1.88:6789/0 pipe(0x58cd440 sd=17 pgs=932 cs=1 l=0).reader couldn't read tag, Success
- -14> 2012-09-19 14:49:09.782170 7fc09a997700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x1ffa240 sd=18 pgs=591 cs=1 l=0).fault 0: Success
- -13> 2012-09-19 14:49:09.782183 7fc0958a8700 2 -- 10.87.1.87:6789/0 >> 10.87.1.88:6789/0 pipe(0x58cd440 sd=17 pgs=932 cs=1 l=0).fault 0: Success
- -12> 2012-09-19 14:49:09.782216 7fc09a99b780 1 -- 10.87.1.87:6789/0 --> mon.2 10.87.1.104:6789/0 -- mon_probe(probe 167d0267-4fde-440c-a624-3c82dba448a6 name 0) v2 -- ?+0 0x55a0600
- -11> 2012-09-19 14:49:09.782230 7fc0969ab700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) put 501 (0x906448 -> 575)
- -10> 2012-09-19 14:49:09.782289 7fc0969ab700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) put 501 (0x906448 -> 74) -9> 2012-09-19 14:49:09.782349 7fc0956a6700 5 mon.0@0(probing) e1 setting monitor caps on this connection
- -8> 2012-09-19 14:49:09.782381 7fc0956a6700 1 -- 10.87.1.87:6789/0 --> 10.87.1.104:6789/0 -- mon_probe(reply 167d0267-4fde-440c-a624-3c82dba448a6 name 0 versions {auth=35,logm=10529,mdsmap=17,monmap=1,osdmap=25,pgmap=10025}) v2 -- ?+0 0x55a0c00 con 0x200af00
- -7> 2012-09-19 14:49:09.782974 7fc0958a8700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) get 646 (74 -> 720)
- -6> 2012-09-19 14:49:09.782960 7fc0957a7700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58cdb00 sd=17 pgs=595 cs=1 l=0).reader couldn't read tag, Success
- -5> 2012-09-19 14:49:09.783006 7fc0957a7700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58cdb00 sd=17 pgs=595 cs=1 l=0).fault 0: Success
- -4> 2012-09-19 14:49:09.783022 7fc0957a7700 0 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58cdb00 sd=17 pgs=595 cs=1 l=0).fault initiating reconnect
- -3> 2012-09-19 14:49:09.783346 7fc0956a6700 -1 msg/SimpleMessenger.cc: In function 'void SimpleMessenger::Pipe::register_pipe()' thread 7fc0956a6700 time 2012-09-19 14:49:09.782424
- msg/SimpleMessenger.cc: 1295: FAILED assert(msgr->rank_pipe.count(peer_addr) == 0)
- ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30)
- 1: (SimpleMessenger::Pipe::register_pipe()+0x25e) [0x5ff9ae]
- 2: (SimpleMessenger::connect_rank(entity_addr_t const&, int, Connection*)+0x240) [0x609fe0]
- 3: (SimpleMessenger::submit_message(Message*, Connection*, entity_addr_t const&, int, bool)+0xaae) [0x60ac8e]
- 4: (SimpleMessenger::_send_message(Message*, Connection*, bool)+0xbe) [0x60b08e]
- 5: (Monitor::handle_probe_probe(MMonProbe*)+0x4b5) [0x48f225]
- 6: (Monitor::handle_probe(MMonProbe*)+0x25b) [0x4940bb]
- 7: (Monitor::_ms_dispatch(Message*)+0x12b3) [0x49d3b3]
- 8: (Monitor::ms_dispatch(Message*)+0x89) [0x4ac959]
- 9: (SimpleMessenger::DispatchQueue::entry()+0x493) [0x608b83]
- 10: (SimpleMessenger::dispatch_entry()+0x24) [0x609d34]
- 11: (SimpleMessenger::DispatchThread::entry()+0xd) [0x5d920d]
- 12: (()+0x7d0c) [0x7fc09a580d0c]
- 13: (clone()+0x6d) [0x7fc098ca5bed]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- -2> 2012-09-19 14:49:09.783426 7fc0945a3700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58ccfc0 sd=20 pgs=596 cs=1 l=0).reader couldn't read tag, Success
- -1> 2012-09-19 14:49:09.783460 7fc0945a3700 2 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58ccfc0 sd=20 pgs=596 cs=1 l=0).fault 0: Success
- 0> 2012-09-19 14:49:09.783476 7fc0945a3700 0 -- 10.87.1.87:6789/0 >> 10.87.1.104:6789/0 pipe(0x58ccfc0 sd=20 pgs=596 cs=1 l=0).fault with nothing to send, going to standby
- 1> 2012-09-19 14:49:09.783773 7fc0944a2700 5 throttle(msgr_dispatch_throttler-mon 0x204a378) get 646 (720 -> 1366)
- --- logging levels ---
- 0/ 5 none
- 0/ 5 lockdep
- 0/ 5 context
- 1/ 5 crush
- 1/ 5 mds 1/ 5 mds_balancer
- 1/ 5 mds_locker
- 1/ 5 mds_log
- 1/ 5 mds_log_expire
- 1/ 5 mds_migrator
- 0/ 0 buffer
- 0/ 5 timer
- 0/ 5 filer
- 0/ 0 objecter
- 0/ 5 rados
- 0/ 5 rbd
- 0/ 5 journaler 0/ 5 objectcacher
- 0/ 5 client 0/ 5 osd
- 0/ 5 optracker
- 0/ 5 objclass
- 1/ 5 filestore
- 1/ 5 journal
- 0/ 5 ms
- 1/ 5 mon
- 1/ 5 mon
- 0/ 5 monc
- 0/ 5 paxos
- 0/ 5 tp
- 1/ 5 auth
- 1/ 5 finisher
- 1/ 5 heartbeatmap
- 1/ 5 perfcounter
- 1/ 5 rgw
- 1/ 5 hadoop
- 1/ 5 asok
- 1/ 5 throttle
- -2/-2 (syslog threshold)
- -1/-1 (stderr threshold)
- max_recent 10000
- max_new 1000000
- log_file /var/log/ceph/mon.0.log
- --- end dump of recent events ---
- 2012-09-19 14:49:09.785595 7fc0956a6700 -1 *** Caught signal (Aborted) **
- in thread 7fc0956a6700
- ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30)
- 1: /usr/bin/ceph-mon() [0x55b8c4]
- 2: (()+0x10420) [0x7fc09a589420]
- 3: (gsignal()+0x35) [0x7fc098bfeb45]
- 4: (abort()+0x186) [0x7fc098bfffc6]
- 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fc0994afd0d]
- 6: (()+0xbcf26) [0x7fc0994adf26] 7: (()+0xbcf53) [0x7fc0994adf53]
- 8: (()+0xbd05e) [0x7fc0994ae05e]
- 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7a2) [0x62a6f2]
- 10: (SimpleMessenger::Pipe::register_pipe()+0x25e) [0x5ff9ae]
- 11: (SimpleMessenger::connect_rank(entity_addr_t const&, int, Connection*)+0x240) [0x609fe0]
- 12: (SimpleMessenger::submit_message(Message*, Connection*, entity_addr_t const&, int, bool)+0xaae) [0x60ac8e]
- 13: (SimpleMessenger::_send_message(Message*, Connection*, bool)+0xbe) [0x60b08e]
- 14: (Monitor::handle_probe_probe(MMonProbe*)+0x4b5) [0x48f225]
- 15: (Monitor::handle_probe(MMonProbe*)+0x25b) [0x4940bb]
- 16: (Monitor::_ms_dispatch(Message*)+0x12b3) [0x49d3b3]
- 17: (Monitor::ms_dispatch(Message*)+0x89) [0x4ac959]
- 18: (SimpleMessenger::DispatchQueue::entry()+0x493) [0x608b83] 19: (SimpleMessenger::dispatch_entry()+0x24) [0x609d34]
- 20: (SimpleMessenger::DispatchThread::entry()+0xd) [0x5d920d] 21: (()+0x7d0c) [0x7fc09a580d0c]
- 22: (clone()+0x6d) [0x7fc098ca5bed]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- begin dump of recent events ---
- 1> 2012-09-19 14:49:09.785595 7fc0956a6700 -1 *** Caught signal (Aborted) **
- in thread 7fc0956a6700
- ceph version 0.51 (commit:c03ca95d235c9a072dcd8a77ad5274a52e93ae30)
- 1: /usr/bin/ceph-mon() [0x55b8c4]
- 2: (()+0x10420) [0x7fc09a589420]
- 3: (gsignal()+0x35) [0x7fc098bfeb45]
- 4: (abort()+0x186) [0x7fc098bfffc6]
- 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fc0994afd0d]
- 6: (()+0xbcf26) [0x7fc0994adf26]
- 7: (()+0xbcf53) [0x7fc0994adf53]
- 8: (()+0xbd05e) [0x7fc0994ae05e]
- 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7a2) [0x62a6f2]
- 10: (SimpleMessenger::Pipe::register_pipe()+0x25e) [0x5ff9ae]
- 11: (SimpleMessenger::connect_rank(entity_addr_t const&, int, Connection*)+0x240) [0x609fe0]
- 12: (SimpleMessenger::submit_message(Message*, Connection*, entity_addr_t const&, int, bool)+0xaae) [0x60ac8e]
- 13: (SimpleMessenger::_send_message(Message*, Connection*, bool)+0xbe) [0x60b08e]
- 14: (Monitor::handle_probe_probe(MMonProbe*)+0x4b5) [0x48f225]
- 15: (Monitor::handle_probe(MMonProbe*)+0x25b) [0x4940bb]
- 16: (Monitor::_ms_dispatch(Message*)+0x12b3) [0x49d3b3]
- 17: (Monitor::ms_dispatch(Message*)+0x89) [0x4ac959]
- 18: (SimpleMessenger::DispatchQueue::entry()+0x493) [0x608b83]
- 19: (SimpleMessenger::dispatch_entry()+0x24) [0x609d34]
- 20: (SimpleMessenger::DispatchThread::entry()+0xd) [0x5d920d]
- 21: (()+0x7d0c) [0x7fc09a580d0c]
- 22: (clone()+0x6d) [0x7fc098ca5bed]
- NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
- --- logging levels --- 0/ 5 none
- 0/ 5 lockdep
- 0/ 5 context
- 1/ 5 crush
- 1/ 5 mds
- 1/ 5 mds_balancer
- 1/ 5 mds_locker
- 1/ 5 mds_log
- 1/ 5 mds_log_expire
- 1/ 5 mds_migrator
- 0/ 0 buffer
- 0/ 5 timer 0/ 5 filer
- 0/ 0 objecter 0/ 5 rados
- 0/ 5 rbd
- 0/ 5 journaler
- 0/ 5 objectcacher
- 0/ 5 client
- 0/ 5 osd
- 0/ 5 optracker
- 0/ 5 osd
- 0/ 5 optracker
- 0/ 5 objclass
- 1/ 5 filestore
- 1/ 5 journal
- 0/ 5 ms
- 1/ 5 mon
- 0/ 5 monc
- 0/ 5 paxos
- 0/ 5 tp
- 1/ 5 auth
- 1/ 5 finisher
- 1/ 5 heartbeatmap
- 1/ 5 perfcounter
- 1/ 5 rgw
- 1/ 5 hadoop
- 1/ 5 asok
- 1/ 5 throttle
- -2/-2 (syslog threshold)
- -1/-1 (stderr threshold)
- max_recent 10000
- max_new 1000000
- log_file /var/log/ceph/mon.0.log
- --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment