Advertisement
gregmark

Ceph Mon Debacle: node 14

Mar 18th, 2015
233
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.52 KB | None | 0 0
  1. 2015-03-18 16:09:16.955474 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.925738 mon.1 192.168.241.114:6789/0 45 : [INF] osd.69 out (down for 4660.975554)
  2. 2015-03-18 16:09:16.955484 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.925825 mon.1 192.168.241.114:6789/0 46 : [INF] osd.72 out (down for 4660.975554)
  3. 2015-03-18 16:09:16.955494 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.925908 mon.1 192.168.241.114:6789/0 47 : [INF] osd.74 out (down for 4660.975553)
  4. 2015-03-18 16:09:16.955504 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.925994 mon.1 192.168.241.114:6789/0 48 : [INF] osd.80 out (down for 4660.975553)
  5. 2015-03-18 16:09:16.955516 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926096 mon.1 192.168.241.114:6789/0 49 : [INF] osd.97 out (down for 4660.975553)
  6. 2015-03-18 16:09:16.955525 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926185 mon.1 192.168.241.114:6789/0 50 : [INF] osd.98 out (down for 4660.975552)
  7. 2015-03-18 16:09:16.955536 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926275 mon.1 192.168.241.114:6789/0 51 : [INF] osd.121 out (down for 4660.975552)
  8. 2015-03-18 16:09:16.955546 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926369 mon.1 192.168.241.114:6789/0 52 : [INF] osd.122 out (down for 4660.975552)
  9. 2015-03-18 16:09:16.955557 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926461 mon.1 192.168.241.114:6789/0 53 : [INF] osd.123 out (down for 4660.975551)
  10. 2015-03-18 16:09:16.955567 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926590 mon.1 192.168.241.114:6789/0 54 : [INF] osd.124 out (down for 4660.975551)
  11. 2015-03-18 16:09:16.955578 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926676 mon.1 192.168.241.114:6789/0 55 : [INF] osd.125 out (down for 4660.975551)
  12. 2015-03-18 16:09:16.955588 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926845 mon.1 192.168.241.114:6789/0 56 : [INF] osd.126 out (down for 4660.975551)
  13. 2015-03-18 16:09:16.955598 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.926939 mon.1 192.168.241.114:6789/0 57 : [INF] osd.127 out (down for 4660.975550)
  14. 2015-03-18 16:09:16.955608 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927031 mon.1 192.168.241.114:6789/0 58 : [INF] osd.128 out (down for 4660.975550)
  15. 2015-03-18 16:09:16.955618 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927123 mon.1 192.168.241.114:6789/0 59 : [INF] osd.129 out (down for 4660.975550)
  16. 2015-03-18 16:09:16.955629 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927238 mon.1 192.168.241.114:6789/0 60 : [INF] osd.130 out (down for 4660.975550)
  17. 2015-03-18 16:09:16.955640 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927331 mon.1 192.168.241.114:6789/0 61 : [INF] osd.131 out (down for 4660.975549)
  18. 2015-03-18 16:09:16.955651 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927433 mon.1 192.168.241.114:6789/0 62 : [INF] osd.132 out (down for 4660.975549)
  19. 2015-03-18 16:09:16.955662 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927523 mon.1 192.168.241.114:6789/0 63 : [INF] osd.134 out (down for 4660.975549)
  20. 2015-03-18 16:09:16.955672 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927617 mon.1 192.168.241.114:6789/0 64 : [INF] osd.135 out (down for 4660.975549)
  21. 2015-03-18 16:09:16.955715 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927712 mon.1 192.168.241.114:6789/0 65 : [INF] osd.136 out (down for 4660.975549)
  22. 2015-03-18 16:09:16.955725 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927800 mon.1 192.168.241.114:6789/0 66 : [INF] osd.137 out (down for 4660.975548)
  23. 2015-03-18 16:09:16.955736 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927888 mon.1 192.168.241.114:6789/0 67 : [INF] osd.138 out (down for 4660.975548)
  24. 2015-03-18 16:09:16.955747 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.927973 mon.1 192.168.241.114:6789/0 68 : [INF] osd.140 out (down for 4660.975548)
  25. 2015-03-18 16:09:16.955758 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928093 mon.1 192.168.241.114:6789/0 69 : [INF] osd.141 out (down for 4660.975548)
  26. 2015-03-18 16:09:16.955769 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928178 mon.1 192.168.241.114:6789/0 70 : [INF] osd.142 out (down for 4660.975547)
  27. 2015-03-18 16:09:16.955780 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928302 mon.1 192.168.241.114:6789/0 71 : [INF] osd.146 out (down for 4660.975547)
  28. 2015-03-18 16:09:16.955790 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928397 mon.1 192.168.241.114:6789/0 72 : [INF] osd.151 out (down for 4660.975547)
  29. 2015-03-18 16:09:16.955800 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928490 mon.1 192.168.241.114:6789/0 73 : [INF] osd.172 out (down for 4660.975546)
  30. 2015-03-18 16:09:16.955810 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928580 mon.1 192.168.241.114:6789/0 74 : [INF] osd.173 out (down for 4660.975546)
  31. 2015-03-18 16:09:16.955822 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928670 mon.1 192.168.241.114:6789/0 75 : [INF] osd.174 out (down for 4660.975546)
  32. 2015-03-18 16:09:16.955832 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928793 mon.1 192.168.241.114:6789/0 76 : [INF] osd.175 out (down for 4660.975546)
  33. 2015-03-18 16:09:16.955843 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928885 mon.1 192.168.241.114:6789/0 77 : [INF] osd.176 out (down for 4660.975545)
  34. 2015-03-18 16:09:16.955853 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.928973 mon.1 192.168.241.114:6789/0 78 : [INF] osd.177 out (down for 4660.975545)
  35. 2015-03-18 16:09:16.955863 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.929099 mon.1 192.168.241.114:6789/0 79 : [INF] osd.178 out (down for 4660.975545)
  36. 2015-03-18 16:09:16.955874 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.929188 mon.1 192.168.241.114:6789/0 80 : [INF] osd.181 out (down for 4660.975545)
  37. 2015-03-18 16:09:16.955885 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.929276 mon.1 192.168.241.114:6789/0 81 : [INF] osd.184 out (down for 4660.975544)
  38. 2015-03-18 16:09:16.955895 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:28.929360 mon.1 192.168.241.114:6789/0 82 : [INF] osd.185 out (down for 4660.975544)
  39. 2015-03-18 16:09:16.955905 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).log v35564904 update_from_paxos applying incremental log 35564904 2015-03-17 17:27:38.805201 osd.113 192.168.241.102:6800/4645 30 : [INF] 3.1ff9 restarting backfill on osd.116 from (0'0,0'0] MAX to 316542'5653826
  40. 2015-03-18 16:09:16.955984 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).log v35564904 check_subs
  41. 2015-03-18 16:09:16.956231 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos
  42. 2015-03-18 16:09:16.956258 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos version 31470 keys ver 0 latest 31469
  43. 2015-03-18 16:09:16.956306 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos loading summary e 31469
  44. 2015-03-18 16:09:16.956310 7f8c405d87c0 7 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos latest length 20003
  45. 2015-03-18 16:09:16.957143 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos key server version 31469
  46. 2015-03-18 16:09:16.957187 7f8c405d87c0 20 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos walking through version 31470 len 649
  47. 2015-03-18 16:09:16.957207 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).auth v31470 update_from_paxos() last_allocated_id=4119496 max_global_id=4119496
  48. 2015-03-18 16:09:16.957214 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).pg v50881041 map_pg_creates to 0 pgs
  49. 2015-03-18 16:09:16.957227 7f8c405d87c0 10 mon.kvm-cs-sn-14i@-1(probing).health(0) init
  50. 2015-03-18 16:09:16.957630 7f8c405d87c0 -1 compacting monitor store ...
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement