API tools faq
paste
Guest User

Untitled

a guest
May 6th, 2016
192
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 177.47 KB | None | 0 0
raw download clone embed print report
  1. 2016-05-03 11:20:22.632447 7fed05941940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 6008
  2. 2016-05-03 11:20:22.838902 7fed05941940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  3. 2016-05-03 11:20:22.866946 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  4. 2016-05-03 11:20:22.866957 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  5. 2016-05-03 11:20:22.866973 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  6. 2016-05-03 11:20:22.890166 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  7. 2016-05-03 11:20:22.890232 7fed05941940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  8. 2016-05-03 11:20:23.156715 7fed05941940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  9. 2016-05-03 11:20:24.231692 7fed05941940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  10. 2016-05-03 11:20:24.450076 7fed05941940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  11. 2016-05-03 11:20:24.451041 7fed05941940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  12. 2016-05-03 11:20:24.544574 7fed05941940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  13. 2016-05-03 11:20:24.547080 7fed05941940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  14. 2016-05-03 11:20:24.627185 7fed05941940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  15. 2016-05-03 11:20:24.627199 7fed05941940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  16. 2016-05-03 11:20:24.627203 7fed05941940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  17. 2016-05-03 11:20:29.403237 7fed05941940 0 osd.51 160621 load_pgs
  18. 2016-05-03 11:21:26.515922 7fed05941940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7fed05941940 time 2016-05-03 11:21:26.512073
  19. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  20.  
  21. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  22. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7fed054377eb]
  23. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fed04f71e8d]
  24. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fed04fee55a]
  25. 4: (OSD::load_pgs()+0x61d) [0x7fed04eac58d]
  26. 5: (OSD::init()+0xe74) [0x7fed04ebc694]
  27. 6: (main()+0x2954) [0x7fed04e41474]
  28. 7: (__libc_start_main()+0xf5) [0x7fed01cc3ec5]
  29. 8: (()+0x2f82b7) [0x7fed04e712b7]
  30. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  31.  
  32. --- begin dump of recent events ---
  33. -217> 2016-05-03 11:20:22.627542 7fed05941940 5 asok(0x7fed08caa000) register_command perfcounters_dump hook 0x7fed08c2a050
  34. -216> 2016-05-03 11:20:22.627579 7fed05941940 5 asok(0x7fed08caa000) register_command 1 hook 0x7fed08c2a050
  35. -215> 2016-05-03 11:20:22.627584 7fed05941940 5 asok(0x7fed08caa000) register_command perf dump hook 0x7fed08c2a050
  36. -214> 2016-05-03 11:20:22.627591 7fed05941940 5 asok(0x7fed08caa000) register_command perfcounters_schema hook 0x7fed08c2a050
  37. -213> 2016-05-03 11:20:22.627593 7fed05941940 5 asok(0x7fed08caa000) register_command 2 hook 0x7fed08c2a050
  38. -212> 2016-05-03 11:20:22.627597 7fed05941940 5 asok(0x7fed08caa000) register_command perf schema hook 0x7fed08c2a050
  39. -211> 2016-05-03 11:20:22.627601 7fed05941940 5 asok(0x7fed08caa000) register_command perf reset hook 0x7fed08c2a050
  40. -210> 2016-05-03 11:20:22.627603 7fed05941940 5 asok(0x7fed08caa000) register_command config show hook 0x7fed08c2a050
  41. -209> 2016-05-03 11:20:22.627605 7fed05941940 5 asok(0x7fed08caa000) register_command config set hook 0x7fed08c2a050
  42. -208> 2016-05-03 11:20:22.627607 7fed05941940 5 asok(0x7fed08caa000) register_command config get hook 0x7fed08c2a050
  43. -207> 2016-05-03 11:20:22.627609 7fed05941940 5 asok(0x7fed08caa000) register_command config diff hook 0x7fed08c2a050
  44. -206> 2016-05-03 11:20:22.627611 7fed05941940 5 asok(0x7fed08caa000) register_command log flush hook 0x7fed08c2a050
  45. -205> 2016-05-03 11:20:22.627613 7fed05941940 5 asok(0x7fed08caa000) register_command log dump hook 0x7fed08c2a050
  46. -204> 2016-05-03 11:20:22.627615 7fed05941940 5 asok(0x7fed08caa000) register_command log reopen hook 0x7fed08c2a050
  47. -203> 2016-05-03 11:20:22.630218 7fed05941940 0 set uid:gid to 64045:64045
  48. -202> 2016-05-03 11:20:22.632447 7fed05941940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 6008
  49. -201> 2016-05-03 11:20:22.637977 7fed05941940 1 accepter.accepter.bind my_inst.addr is [::]:6808/6008 need_addr=1
  50. -200> 2016-05-03 11:20:22.638005 7fed05941940 1 accepter.accepter.bind my_inst.addr is [::]:6809/6008 need_addr=1
  51. -199> 2016-05-03 11:20:22.638041 7fed05941940 1 accepter.accepter.bind my_inst.addr is [::]:6810/6008 need_addr=1
  52. -198> 2016-05-03 11:20:22.638065 7fed05941940 1 accepter.accepter.bind my_inst.addr is [::]:6811/6008 need_addr=1
  53. -197> 2016-05-03 11:20:22.658367 7fed05941940 5 asok(0x7fed08caa000) init /var/run/ceph/ceph-osd.51.asok
  54. -196> 2016-05-03 11:20:22.658381 7fed05941940 5 asok(0x7fed08caa000) bind_and_listen /var/run/ceph/ceph-osd.51.asok
  55. -195> 2016-05-03 11:20:22.658406 7fed05941940 5 asok(0x7fed08caa000) register_command 0 hook 0x7fed08c260b0
  56. -194> 2016-05-03 11:20:22.658414 7fed05941940 5 asok(0x7fed08caa000) register_command version hook 0x7fed08c260b0
  57. -193> 2016-05-03 11:20:22.658427 7fed05941940 5 asok(0x7fed08caa000) register_command git_version hook 0x7fed08c260b0
  58. -192> 2016-05-03 11:20:22.658437 7fed05941940 5 asok(0x7fed08caa000) register_command help hook 0x7fed08c2a230
  59. -191> 2016-05-03 11:20:22.658447 7fed05941940 5 asok(0x7fed08caa000) register_command get_command_descriptions hook 0x7fed08c2a220
  60. -190> 2016-05-03 11:20:22.658579 7fecfec51700 5 asok(0x7fed08caa000) entry start
  61. -189> 2016-05-03 11:20:22.664297 7fed05941940 10 monclient(hunting): build_initial_monmap
  62. -188> 2016-05-03 11:20:22.832505 7fed05941940 5 adding auth protocol: cephx
  63. -187> 2016-05-03 11:20:22.832519 7fed05941940 5 adding auth protocol: cephx
  64. -186> 2016-05-03 11:20:22.836910 7fed05941940 5 asok(0x7fed08caa000) register_command objecter_requests hook 0x7fed08c2a270
  65. -185> 2016-05-03 11:20:22.836975 7fed05941940 1 -- [::]:6808/6008 messenger.start
  66. -184> 2016-05-03 11:20:22.837005 7fed05941940 1 -- :/0 messenger.start
  67. -183> 2016-05-03 11:20:22.837032 7fed05941940 1 -- [::]:6811/6008 messenger.start
  68. -182> 2016-05-03 11:20:22.837054 7fed05941940 1 -- [::]:6810/6008 messenger.start
  69. -181> 2016-05-03 11:20:22.837075 7fed05941940 1 -- [::]:6809/6008 messenger.start
  70. -180> 2016-05-03 11:20:22.837100 7fed05941940 1 -- :/0 messenger.start
  71. -179> 2016-05-03 11:20:22.837171 7fed05941940 2 osd.51 0 mounting /var/lib/ceph/osd/ceph-51 /var/lib/ceph/osd/ceph-51/journal
  72. -178> 2016-05-03 11:20:22.838902 7fed05941940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  73. -177> 2016-05-03 11:20:22.866946 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  74. -176> 2016-05-03 11:20:22.866957 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  75. -175> 2016-05-03 11:20:22.866973 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  76. -174> 2016-05-03 11:20:22.890166 7fed05941940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  77. -173> 2016-05-03 11:20:22.890232 7fed05941940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  78. -172> 2016-05-03 11:20:23.156715 7fed05941940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  79. -171> 2016-05-03 11:20:24.231631 7fed05941940 2 journal open /var/lib/ceph/osd/ceph-51/journal fsid 3ca75ccf-1c42-4282-9504-c07f2cee91b3 fs_op_seq 21461862
  80. -170> 2016-05-03 11:20:24.231692 7fed05941940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  81. -169> 2016-05-03 11:20:24.231830 7fed05941940 2 journal open advancing committed_seq 21461799 to fs op_seq 21461862
  82. -168> 2016-05-03 11:20:24.241222 7fed05941940 2 journal read_entry 256061440 : seq 21461800 3021 bytes
  83. -167> 2016-05-03 11:20:24.249639 7fed05941940 2 journal read_entry 256065536 : seq 21461801 2981 bytes
  84. -166> 2016-05-03 11:20:24.249707 7fed05941940 2 journal read_entry 256069632 : seq 21461802 2864 bytes
  85. -165> 2016-05-03 11:20:24.249730 7fed05941940 2 journal read_entry 256073728 : seq 21461803 2788 bytes
  86. -164> 2016-05-03 11:20:24.249748 7fed05941940 2 journal read_entry 256077824 : seq 21461804 3252 bytes
  87. -163> 2016-05-03 11:20:24.258095 7fed05941940 2 journal read_entry 256081920 : seq 21461805 3072 bytes
  88. -162> 2016-05-03 11:20:24.258113 7fed05941940 2 journal read_entry 256086016 : seq 21461806 3136 bytes
  89. -161> 2016-05-03 11:20:24.258121 7fed05941940 2 journal read_entry 256090112 : seq 21461807 2178 bytes
  90. -160> 2016-05-03 11:20:24.258126 7fed05941940 2 journal read_entry 256094208 : seq 21461808 2146 bytes
  91. -159> 2016-05-03 11:20:24.258131 7fed05941940 2 journal read_entry 256098304 : seq 21461809 2417 bytes
  92. -158> 2016-05-03 11:20:24.258138 7fed05941940 2 journal read_entry 256102400 : seq 21461810 2291 bytes
  93. -157> 2016-05-03 11:20:24.258150 7fed05941940 2 journal read_entry 256106496 : seq 21461811 2417 bytes
  94. -156> 2016-05-03 11:20:24.258157 7fed05941940 2 journal read_entry 256110592 : seq 21461812 2417 bytes
  95. -155> 2016-05-03 11:20:24.258462 7fed05941940 2 journal read_entry 256114688 : seq 21461813 2499 bytes
  96. -154> 2016-05-03 11:20:24.258473 7fed05941940 2 journal read_entry 256118784 : seq 21461814 3140 bytes
  97. -153> 2016-05-03 11:20:24.258479 7fed05941940 2 journal read_entry 256122880 : seq 21461815 2385 bytes
  98. -152> 2016-05-03 11:20:24.258484 7fed05941940 2 journal read_entry 256126976 : seq 21461816 2385 bytes
  99. -151> 2016-05-03 11:20:24.258490 7fed05941940 2 journal read_entry 256131072 : seq 21461817 2483 bytes
  100. -150> 2016-05-03 11:20:24.258496 7fed05941940 2 journal read_entry 256135168 : seq 21461818 2275 bytes
  101. -149> 2016-05-03 11:20:24.258510 7fed05941940 2 journal read_entry 256139264 : seq 21461819 2889 bytes
  102. -148> 2016-05-03 11:20:24.258518 7fed05941940 2 journal read_entry 256143360 : seq 21461820 3144 bytes
  103. -147> 2016-05-03 11:20:24.258524 7fed05941940 2 journal read_entry 256147456 : seq 21461821 3156 bytes
  104. -146> 2016-05-03 11:20:24.258531 7fed05941940 2 journal read_entry 256151552 : seq 21461822 2146 bytes
  105. -145> 2016-05-03 11:20:24.258538 7fed05941940 2 journal read_entry 256155648 : seq 21461823 2889 bytes
  106. -144> 2016-05-03 11:20:24.258545 7fed05941940 2 journal read_entry 256159744 : seq 21461824 2412 bytes
  107. -143> 2016-05-03 11:20:24.258551 7fed05941940 2 journal read_entry 256163840 : seq 21461825 2291 bytes
  108. -142> 2016-05-03 11:20:24.258559 7fed05941940 2 journal read_entry 256167936 : seq 21461826 3446 bytes
  109. -141> 2016-05-03 11:20:24.258566 7fed05941940 2 journal read_entry 256172032 : seq 21461827 3201 bytes
  110. -140> 2016-05-03 11:20:24.258573 7fed05941940 2 journal read_entry 256176128 : seq 21461828 2417 bytes
  111. -139> 2016-05-03 11:20:24.267237 7fed05941940 2 journal read_entry 256180224 : seq 21461829 3050 bytes
  112. -138> 2016-05-03 11:20:24.267254 7fed05941940 2 journal read_entry 256184320 : seq 21461830 2146 bytes
  113. -137> 2016-05-03 11:20:24.267261 7fed05941940 2 journal read_entry 256188416 : seq 21461831 3012 bytes
  114. -136> 2016-05-03 11:20:24.267266 7fed05941940 2 journal read_entry 256192512 : seq 21461832 3160 bytes
  115. -135> 2016-05-03 11:20:24.267271 7fed05941940 2 journal read_entry 256196608 : seq 21461833 2781 bytes
  116. -134> 2016-05-03 11:20:24.267276 7fed05941940 2 journal read_entry 256200704 : seq 21461834 2385 bytes
  117. -133> 2016-05-03 11:20:24.267289 7fed05941940 2 journal read_entry 256204800 : seq 21461835 2385 bytes
  118. -132> 2016-05-03 11:20:24.267297 7fed05941940 2 journal read_entry 256208896 : seq 21461836 2389 bytes
  119. -131> 2016-05-03 11:20:24.267306 7fed05941940 2 journal read_entry 256212992 : seq 21461837 2385 bytes
  120. -130> 2016-05-03 11:20:24.267313 7fed05941940 2 journal read_entry 256217088 : seq 21461838 2275 bytes
  121. -129> 2016-05-03 11:20:24.267320 7fed05941940 2 journal read_entry 256221184 : seq 21461839 2417 bytes
  122. -128> 2016-05-03 11:20:24.267326 7fed05941940 2 journal read_entry 256225280 : seq 21461840 2385 bytes
  123. -127> 2016-05-03 11:20:24.267333 7fed05941940 2 journal read_entry 256229376 : seq 21461841 2417 bytes
  124. -126> 2016-05-03 11:20:24.267341 7fed05941940 2 journal read_entry 256233472 : seq 21461842 2990 bytes
  125. -125> 2016-05-03 11:20:24.267348 7fed05941940 2 journal read_entry 256237568 : seq 21461843 3028 bytes
  126. -124> 2016-05-03 11:20:24.267354 7fed05941940 2 journal read_entry 256241664 : seq 21461844 2889 bytes
  127. -123> 2016-05-03 11:20:24.275918 7fed05941940 2 journal read_entry 256245760 : seq 21461845 2990 bytes
  128. -122> 2016-05-03 11:20:24.275936 7fed05941940 2 journal read_entry 256249856 : seq 21461846 2788 bytes
  129. -121> 2016-05-03 11:20:24.275942 7fed05941940 2 journal read_entry 256253952 : seq 21461847 2990 bytes
  130. -120> 2016-05-03 11:20:24.275947 7fed05941940 2 journal read_entry 256258048 : seq 21461848 3036 bytes
  131. -119> 2016-05-03 11:20:24.275952 7fed05941940 2 journal read_entry 256262144 : seq 21461849 2788 bytes
  132. -118> 2016-05-03 11:20:24.275957 7fed05941940 2 journal read_entry 256266240 : seq 21461850 2904 bytes
  133. -117> 2016-05-03 11:20:24.275974 7fed05941940 2 journal read_entry 256270336 : seq 21461851 2840 bytes
  134. -116> 2016-05-03 11:20:24.275982 7fed05941940 2 journal read_entry 256274432 : seq 21461852 2990 bytes
  135. -115> 2016-05-03 11:20:24.275989 7fed05941940 2 journal read_entry 256278528 : seq 21461853 2856 bytes
  136. -114> 2016-05-03 11:20:24.443590 7fed05941940 2 journal read_entry 256282624 : seq 21461854 4196824 bytes
  137. -113> 2016-05-03 11:20:24.443623 7fed05941940 2 journal read_entry 260485120 : seq 21461855 2840 bytes
  138. -112> 2016-05-03 11:20:24.443636 7fed05941940 2 journal read_entry 260489216 : seq 21461856 2990 bytes
  139. -111> 2016-05-03 11:20:24.448290 7fed05941940 2 journal read_entry 260493312 : seq 21461857 869132 bytes
  140. -110> 2016-05-03 11:20:24.448311 7fed05941940 2 journal read_entry 261365760 : seq 21461858 7957 bytes
  141. -109> 2016-05-03 11:20:24.448320 7fed05941940 2 journal read_entry 261373952 : seq 21461859 2438 bytes
  142. -108> 2016-05-03 11:20:24.448328 7fed05941940 2 journal read_entry 261378048 : seq 21461860 2990 bytes
  143. -107> 2016-05-03 11:20:24.448335 7fed05941940 2 journal read_entry 261382144 : seq 21461861 2856 bytes
  144. -106> 2016-05-03 11:20:24.448342 7fed05941940 2 journal read_entry 261386240 : seq 21461862 2964 bytes
  145. -105> 2016-05-03 11:20:24.448350 7fed05941940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  146. -104> 2016-05-03 11:20:24.448359 7fed05941940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  147. -103> 2016-05-03 11:20:24.448362 7fed05941940 3 journal journal_replay: end of journal, done.
  148. -102> 2016-05-03 11:20:24.450076 7fed05941940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  149. -101> 2016-05-03 11:20:24.451041 7fed05941940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  150. -100> 2016-05-03 11:20:24.451061 7fed05941940 2 osd.51 0 boot
  151. -99> 2016-05-03 11:20:24.542704 7fed05941940 1 <cls> cls/timeindex/cls_timeindex.cc:259: Loaded timeindex class!
  152. -98> 2016-05-03 11:20:24.542812 7fed05941940 1 <cls> cls/log/cls_log.cc:313: Loaded log class!
  153. -97> 2016-05-03 11:20:24.544448 7fed05941940 1 <cls> cls/refcount/cls_refcount.cc:232: Loaded refcount class!
  154. -96> 2016-05-03 11:20:24.544574 7fed05941940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  155. -95> 2016-05-03 11:20:24.546310 7fed05941940 1 <cls> cls/rgw/cls_rgw.cc:3059: Loaded rgw class!
  156. -94> 2016-05-03 11:20:24.546435 7fed05941940 1 <cls> cls/version/cls_version.cc:228: Loaded version class!
  157. -93> 2016-05-03 11:20:24.546802 7fed05941940 1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class!
  158. -92> 2016-05-03 11:20:24.546988 7fed05941940 1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class!
  159. -91> 2016-05-03 11:20:24.547080 7fed05941940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  160. -90> 2016-05-03 11:20:24.547168 7fed05941940 1 <cls> cls/user/cls_user.cc:367: Loaded user class!
  161. -89> 2016-05-03 11:20:24.627185 7fed05941940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  162. -88> 2016-05-03 11:20:24.627199 7fed05941940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  163. -87> 2016-05-03 11:20:24.627203 7fed05941940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  164. -86> 2016-05-03 11:20:29.403237 7fed05941940 0 osd.51 160621 load_pgs
  165. -85> 2016-05-03 11:20:31.663054 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.6(unlocked)] enter Initial
  166. -84> 2016-05-03 11:20:31.663365 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000311 0 0.000000
  167. -83> 2016-05-03 11:20:31.663391 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  168. -82> 2016-05-03 11:20:31.663490 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.1a(unlocked)] enter Initial
  169. -81> 2016-05-03 11:20:31.663677 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000186 0 0.000000
  170. -80> 2016-05-03 11:20:31.663690 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  171. -79> 2016-05-03 11:20:31.663783 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.22(unlocked)] enter Initial
  172. -78> 2016-05-03 11:20:31.663942 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.000159 0 0.000000
  173. -77> 2016-05-03 11:20:31.663955 7fed05941940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  174. -76> 2016-05-03 11:20:31.734006 7fed05941940 5 osd.51 pg_epoch: 160581 pg[1.36(unlocked)] enter Initial
  175. -75> 2016-05-03 11:20:31.758294 7fed05941940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024288 0 0.000000
  176. -74> 2016-05-03 11:20:31.758351 7fed05941940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  177. -73> 2016-05-03 11:20:31.909216 7fed05941940 5 osd.51 pg_epoch: 160599 pg[1.3f(unlocked)] enter Initial
  178. -72> 2016-05-03 11:20:31.934920 7fed05941940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025704 0 0.000000
  179. -71> 2016-05-03 11:20:31.934946 7fed05941940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  180. -70> 2016-05-03 11:20:31.982878 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.4(unlocked)] enter Initial
  181. -69> 2016-05-03 11:20:32.012885 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.030007 0 0.000000
  182. -68> 2016-05-03 11:20:32.012914 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  183. -67> 2016-05-03 11:20:32.013054 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.d(unlocked)] enter Initial
  184. -66> 2016-05-03 11:20:32.040476 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.027421 0 0.000000
  185. -65> 2016-05-03 11:20:32.040501 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  186. -64> 2016-05-03 11:20:32.040674 7fed05941940 5 osd.51 pg_epoch: 160621 pg[2.13(unlocked)] enter Initial
  187. -63> 2016-05-03 11:20:32.068818 7fed05941940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.028143 0 0.000000
  188. -62> 2016-05-03 11:20:32.068850 7fed05941940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  189. -61> 2016-05-03 11:20:32.069061 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.14(unlocked)] enter Initial
  190. -60> 2016-05-03 11:20:32.095003 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.025942 0 0.000000
  191. -59> 2016-05-03 11:20:32.095029 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  192. -58> 2016-05-03 11:20:32.095180 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1b(unlocked)] enter Initial
  193. -57> 2016-05-03 11:20:32.113131 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.017951 0 0.000000
  194. -56> 2016-05-03 11:20:32.113155 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  195. -55> 2016-05-03 11:20:32.113312 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.32(unlocked)] enter Initial
  196. -54> 2016-05-03 11:20:40.259387 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] exit Initial 8.146074 0 0.000000
  197. -53> 2016-05-03 11:20:40.259412 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] enter Reset
  198. -52> 2016-05-03 11:20:40.259585 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.3b(unlocked)] enter Initial
  199. -51> 2016-05-03 11:20:47.706141 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] exit Initial 7.446555 0 0.000000
  200. -50> 2016-05-03 11:20:47.706169 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] enter Reset
  201. -49> 2016-05-03 11:20:47.706447 7fed05941940 5 osd.51 pg_epoch: 160580 pg[2.48(unlocked)] enter Initial
  202. -48> 2016-05-03 11:20:47.706626 7fed05941940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000178 0 0.000000
  203. -47> 2016-05-03 11:20:47.706639 7fed05941940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  204. -46> 2016-05-03 11:20:47.706763 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.50(unlocked)] enter Initial
  205. -45> 2016-05-03 11:20:47.748720 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.041957 0 0.000000
  206. -44> 2016-05-03 11:20:47.748743 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  207. -43> 2016-05-03 11:20:47.748898 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.51(unlocked)] enter Initial
  208. -42> 2016-05-03 11:20:47.767661 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.018762 0 0.000000
  209. -41> 2016-05-03 11:20:47.767680 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  210. -40> 2016-05-03 11:20:47.767824 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.72(unlocked)] enter Initial
  211. -39> 2016-05-03 11:20:47.789692 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021867 0 0.000000
  212. -38> 2016-05-03 11:20:47.789713 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  213. -37> 2016-05-03 11:20:47.789843 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.b7(unlocked)] enter Initial
  214. -36> 2016-05-03 11:20:47.821583 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.031740 0 0.000000
  215. -35> 2016-05-03 11:20:47.821607 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  216. -34> 2016-05-03 11:20:47.840188 7fed05941940 5 osd.51 pg_epoch: 160595 pg[2.cf(unlocked)] enter Initial
  217. -33> 2016-05-03 11:20:47.892696 7fed05941940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.052507 0 0.000000
  218. -32> 2016-05-03 11:20:47.892724 7fed05941940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  219. -31> 2016-05-03 11:20:47.892888 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.10e(unlocked)] enter Initial
  220. -30> 2016-05-03 11:20:47.914613 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021724 0 0.000000
  221. -29> 2016-05-03 11:20:47.914648 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  222. -28> 2016-05-03 11:20:47.914805 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.111(unlocked)] enter Initial
  223. -27> 2016-05-03 11:20:59.321542 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] exit Initial 11.406737 0 0.000000
  224. -26> 2016-05-03 11:20:59.321567 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] enter Reset
  225. -25> 2016-05-03 11:20:59.321720 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.16f(unlocked)] enter Initial
  226. -24> 2016-05-03 11:21:12.676267 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] exit Initial 13.354546 0 0.000000
  227. -23> 2016-05-03 11:21:12.676294 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] enter Reset
  228. -22> 2016-05-03 11:21:12.686114 7fed05941940 5 osd.51 pg_epoch: 160607 pg[2.186(unlocked)] enter Initial
  229. -21> 2016-05-03 11:21:12.726365 7fed05941940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.040250 0 0.000000
  230. -20> 2016-05-03 11:21:12.726386 7fed05941940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  231. -19> 2016-05-03 11:21:12.726538 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.188(unlocked)] enter Initial
  232. -18> 2016-05-03 11:21:12.768336 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.041798 0 0.000000
  233. -17> 2016-05-03 11:21:12.768355 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  234. -16> 2016-05-03 11:21:12.768532 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1a2(unlocked)] enter Initial
  235. -15> 2016-05-03 11:21:12.791190 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.022657 0 0.000000
  236. -14> 2016-05-03 11:21:12.791209 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  237. -13> 2016-05-03 11:21:12.791334 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1aa(unlocked)] enter Initial
  238. -12> 2016-05-03 11:21:12.832242 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.040908 0 0.000000
  239. -11> 2016-05-03 11:21:12.832263 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  240. -10> 2016-05-03 11:21:12.832399 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1bb(unlocked)] enter Initial
  241. -9> 2016-05-03 11:21:12.868735 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.036336 0 0.000000
  242. -8> 2016-05-03 11:21:12.868754 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  243. -7> 2016-05-03 11:21:12.868898 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1e8(unlocked)] enter Initial
  244. -6> 2016-05-03 11:21:12.895902 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.027003 0 0.000000
  245. -5> 2016-05-03 11:21:12.895920 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  246. -4> 2016-05-03 11:21:12.896050 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1f2(unlocked)] enter Initial
  247. -3> 2016-05-03 11:21:12.926431 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.030381 0 0.000000
  248. -2> 2016-05-03 11:21:12.926450 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  249. -1> 2016-05-03 11:21:12.926591 7fed05941940 5 osd.51 pg_epoch: 160608 pg[2.1fd(unlocked)] enter Initial
  250. 0> 2016-05-03 11:21:26.515922 7fed05941940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7fed05941940 time 2016-05-03 11:21:26.512073
  251. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  252.  
  253. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  254. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7fed054377eb]
  255. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fed04f71e8d]
  256. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fed04fee55a]
  257. 4: (OSD::load_pgs()+0x61d) [0x7fed04eac58d]
  258. 5: (OSD::init()+0xe74) [0x7fed04ebc694]
  259. 6: (main()+0x2954) [0x7fed04e41474]
  260. 7: (__libc_start_main()+0xf5) [0x7fed01cc3ec5]
  261. 8: (()+0x2f82b7) [0x7fed04e712b7]
  262. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  263.  
  264. --- logging levels ---
  265. 0/ 5 none
  266. 0/ 1 lockdep
  267. 0/ 1 context
  268. 1/ 1 crush
  269. 1/ 5 mds
  270. 1/ 5 mds_balancer
  271. 1/ 5 mds_locker
  272. 1/ 5 mds_log
  273. 1/ 5 mds_log_expire
  274. 1/ 5 mds_migrator
  275. 0/ 1 buffer
  276. 0/ 1 timer
  277. 0/ 1 filer
  278. 0/ 1 striper
  279. 0/ 1 objecter
  280. 0/ 5 rados
  281. 0/ 5 rbd
  282. 0/ 5 rbd_replay
  283. 0/ 5 journaler
  284. 0/ 5 objectcacher
  285. 0/ 5 client
  286. 0/ 5 osd
  287. 0/ 5 optracker
  288. 0/ 5 objclass
  289. 1/ 3 filestore
  290. 1/ 3 keyvaluestore
  291. 1/ 3 journal
  292. 0/ 5 ms
  293. 1/ 5 mon
  294. 0/10 monc
  295. 1/ 5 paxos
  296. 0/ 5 tp
  297. 1/ 5 auth
  298. 1/ 5 crypto
  299. 1/ 1 finisher
  300. 1/ 5 heartbeatmap
  301. 1/ 5 perfcounter
  302. 1/ 5 rgw
  303. 1/10 civetweb
  304. 1/ 5 javaclient
  305. 1/ 5 asok
  306. 1/ 1 throttle
  307. 0/ 0 refs
  308. 1/ 5 xio
  309. 1/ 5 compressor
  310. 1/ 5 newstore
  311. -2/-2 (syslog threshold)
  312. -1/-1 (stderr threshold)
  313. max_recent 10000
  314. max_new 1000
  315. log_file /var/log/ceph/ceph-osd.51.log
  316. --- end dump of recent events ---
  317. 2016-05-03 11:21:26.649990 7fed05941940 -1 *** Caught signal (Aborted) **
  318. in thread 7fed05941940
  319.  
  320. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  321. 1: (()+0x7d1aca) [0x7fed0534aaca]
  322. 2: (()+0x10340) [0x7fed03a5d340]
  323. 3: (gsignal()+0x39) [0x7fed01cd8cc9]
  324. 4: (abort()+0x148) [0x7fed01cdc0d8]
  325. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fed025e3535]
  326. 6: (()+0x5e6d6) [0x7fed025e16d6]
  327. 7: (()+0x5e703) [0x7fed025e1703]
  328. 8: (()+0x5e922) [0x7fed025e1922]
  329. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7fed054379d8]
  330. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fed04f71e8d]
  331. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fed04fee55a]
  332. 12: (OSD::load_pgs()+0x61d) [0x7fed04eac58d]
  333. 13: (OSD::init()+0xe74) [0x7fed04ebc694]
  334. 14: (main()+0x2954) [0x7fed04e41474]
  335. 15: (__libc_start_main()+0xf5) [0x7fed01cc3ec5]
  336. 16: (()+0x2f82b7) [0x7fed04e712b7]
  337. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  338.  
  339. --- begin dump of recent events ---
  340. 0> 2016-05-03 11:21:26.649990 7fed05941940 -1 *** Caught signal (Aborted) **
  341. in thread 7fed05941940
  342.  
  343. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  344. 1: (()+0x7d1aca) [0x7fed0534aaca]
  345. 2: (()+0x10340) [0x7fed03a5d340]
  346. 3: (gsignal()+0x39) [0x7fed01cd8cc9]
  347. 4: (abort()+0x148) [0x7fed01cdc0d8]
  348. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fed025e3535]
  349. 6: (()+0x5e6d6) [0x7fed025e16d6]
  350. 7: (()+0x5e703) [0x7fed025e1703]
  351. 8: (()+0x5e922) [0x7fed025e1922]
  352. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7fed054379d8]
  353. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fed04f71e8d]
  354. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fed04fee55a]
  355. 12: (OSD::load_pgs()+0x61d) [0x7fed04eac58d]
  356. 13: (OSD::init()+0xe74) [0x7fed04ebc694]
  357. 14: (main()+0x2954) [0x7fed04e41474]
  358. 15: (__libc_start_main()+0xf5) [0x7fed01cc3ec5]
  359. 16: (()+0x2f82b7) [0x7fed04e712b7]
  360. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  361.  
  362. --- logging levels ---
  363. 0/ 5 none
  364. 0/ 1 lockdep
  365. 0/ 1 context
  366. 1/ 1 crush
  367. 1/ 5 mds
  368. 1/ 5 mds_balancer
  369. 1/ 5 mds_locker
  370. 1/ 5 mds_log
  371. 1/ 5 mds_log_expire
  372. 1/ 5 mds_migrator
  373. 0/ 1 buffer
  374. 0/ 1 timer
  375. 0/ 1 filer
  376. 0/ 1 striper
  377. 0/ 1 objecter
  378. 0/ 5 rados
  379. 0/ 5 rbd
  380. 0/ 5 rbd_replay
  381. 0/ 5 journaler
  382. 0/ 5 objectcacher
  383. 0/ 5 client
  384. 0/ 5 osd
  385. 0/ 5 optracker
  386. 0/ 5 objclass
  387. 1/ 3 filestore
  388. 1/ 3 keyvaluestore
  389. 1/ 3 journal
  390. 0/ 5 ms
  391. 1/ 5 mon
  392. 0/10 monc
  393. 1/ 5 paxos
  394. 0/ 5 tp
  395. 1/ 5 auth
  396. 1/ 5 crypto
  397. 1/ 1 finisher
  398. 1/ 5 heartbeatmap
  399. 1/ 5 perfcounter
  400. 1/ 5 rgw
  401. 1/10 civetweb
  402. 1/ 5 javaclient
  403. 1/ 5 asok
  404. 1/ 1 throttle
  405. 0/ 0 refs
  406. 1/ 5 xio
  407. 1/ 5 compressor
  408. 1/ 5 newstore
  409. -2/-2 (syslog threshold)
  410. -1/-1 (stderr threshold)
  411. max_recent 10000
  412. max_new 1000
  413. log_file /var/log/ceph/ceph-osd.51.log
  414. --- end dump of recent events ---
  415. 2016-05-03 11:21:27.372810 7fa68bb75940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14789
  416. 2016-05-03 11:21:27.386237 7fa68bb75940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  417. 2016-05-03 11:21:27.386680 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  418. 2016-05-03 11:21:27.386688 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  419. 2016-05-03 11:21:27.386709 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  420. 2016-05-03 11:21:27.414976 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  421. 2016-05-03 11:21:27.415050 7fa68bb75940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  422. 2016-05-03 11:21:27.516147 7fa68bb75940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  423. 2016-05-03 11:21:27.518512 7fa68bb75940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  424. 2016-05-03 11:21:27.566425 7fa68bb75940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  425. 2016-05-03 11:21:27.567494 7fa68bb75940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  426. 2016-05-03 11:21:27.571240 7fa68bb75940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  427. 2016-05-03 11:21:27.574430 7fa68bb75940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  428. 2016-05-03 11:21:27.574849 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  429. 2016-05-03 11:21:27.574860 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  430. 2016-05-03 11:21:27.574865 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  431. 2016-05-03 11:21:27.657473 7fa68bb75940 0 osd.51 160621 load_pgs
  432. 2016-05-03 11:21:28.463067 7fa68bb75940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7fa68bb75940 time 2016-05-03 11:21:28.460685
  433. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  434.  
  435. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  436. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7fa68b66b7eb]
  437. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fa68b1a5e8d]
  438. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fa68b22255a]
  439. 4: (OSD::load_pgs()+0x61d) [0x7fa68b0e058d]
  440. 5: (OSD::init()+0xe74) [0x7fa68b0f0694]
  441. 6: (main()+0x2954) [0x7fa68b075474]
  442. 7: (__libc_start_main()+0xf5) [0x7fa687ef7ec5]
  443. 8: (()+0x2f82b7) [0x7fa68b0a52b7]
  444. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  445.  
  446. --- begin dump of recent events ---
  447. -217> 2016-05-03 11:21:27.369444 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command perfcounters_dump hook 0x7fa68f24e050
  448. -216> 2016-05-03 11:21:27.369485 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command 1 hook 0x7fa68f24e050
  449. -215> 2016-05-03 11:21:27.369493 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command perf dump hook 0x7fa68f24e050
  450. -214> 2016-05-03 11:21:27.369496 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command perfcounters_schema hook 0x7fa68f24e050
  451. -213> 2016-05-03 11:21:27.369504 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command 2 hook 0x7fa68f24e050
  452. -212> 2016-05-03 11:21:27.369507 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command perf schema hook 0x7fa68f24e050
  453. -211> 2016-05-03 11:21:27.369514 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command perf reset hook 0x7fa68f24e050
  454. -210> 2016-05-03 11:21:27.369517 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command config show hook 0x7fa68f24e050
  455. -209> 2016-05-03 11:21:27.369524 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command config set hook 0x7fa68f24e050
  456. -208> 2016-05-03 11:21:27.369527 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command config get hook 0x7fa68f24e050
  457. -207> 2016-05-03 11:21:27.369534 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command config diff hook 0x7fa68f24e050
  458. -206> 2016-05-03 11:21:27.369536 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command log flush hook 0x7fa68f24e050
  459. -205> 2016-05-03 11:21:27.369539 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command log dump hook 0x7fa68f24e050
  460. -204> 2016-05-03 11:21:27.369546 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command log reopen hook 0x7fa68f24e050
  461. -203> 2016-05-03 11:21:27.372727 7fa68bb75940 0 set uid:gid to 64045:64045
  462. -202> 2016-05-03 11:21:27.372810 7fa68bb75940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14789
  463. -201> 2016-05-03 11:21:27.373390 7fa68bb75940 1 accepter.accepter.bind my_inst.addr is [::]:6808/14789 need_addr=1
  464. -200> 2016-05-03 11:21:27.373416 7fa68bb75940 1 accepter.accepter.bind my_inst.addr is [::]:6809/14789 need_addr=1
  465. -199> 2016-05-03 11:21:27.373432 7fa68bb75940 1 accepter.accepter.bind my_inst.addr is [::]:6810/14789 need_addr=1
  466. -198> 2016-05-03 11:21:27.373448 7fa68bb75940 1 accepter.accepter.bind my_inst.addr is [::]:6811/14789 need_addr=1
  467. -197> 2016-05-03 11:21:27.375331 7fa68bb75940 5 asok(0x7fa68f2ce000) init /var/run/ceph/ceph-osd.51.asok
  468. -196> 2016-05-03 11:21:27.375343 7fa68bb75940 5 asok(0x7fa68f2ce000) bind_and_listen /var/run/ceph/ceph-osd.51.asok
  469. -195> 2016-05-03 11:21:27.375446 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command 0 hook 0x7fa68f24a0b0
  470. -194> 2016-05-03 11:21:27.375454 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command version hook 0x7fa68f24a0b0
  471. -193> 2016-05-03 11:21:27.375457 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command git_version hook 0x7fa68f24a0b0
  472. -192> 2016-05-03 11:21:27.375462 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command help hook 0x7fa68f24e230
  473. -191> 2016-05-03 11:21:27.375465 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command get_command_descriptions hook 0x7fa68f24e220
  474. -190> 2016-05-03 11:21:27.375506 7fa684e85700 5 asok(0x7fa68f2ce000) entry start
  475. -189> 2016-05-03 11:21:27.375553 7fa68bb75940 10 monclient(hunting): build_initial_monmap
  476. -188> 2016-05-03 11:21:27.385759 7fa68bb75940 5 adding auth protocol: cephx
  477. -187> 2016-05-03 11:21:27.385767 7fa68bb75940 5 adding auth protocol: cephx
  478. -186> 2016-05-03 11:21:27.385855 7fa68bb75940 5 asok(0x7fa68f2ce000) register_command objecter_requests hook 0x7fa68f24e270
  479. -185> 2016-05-03 11:21:27.385918 7fa68bb75940 1 -- [::]:6808/14789 messenger.start
  480. -184> 2016-05-03 11:21:27.385955 7fa68bb75940 1 -- :/0 messenger.start
  481. -183> 2016-05-03 11:21:27.385978 7fa68bb75940 1 -- [::]:6811/14789 messenger.start
  482. -182> 2016-05-03 11:21:27.386002 7fa68bb75940 1 -- [::]:6810/14789 messenger.start
  483. -181> 2016-05-03 11:21:27.386039 7fa68bb75940 1 -- [::]:6809/14789 messenger.start
  484. -180> 2016-05-03 11:21:27.386082 7fa68bb75940 1 -- :/0 messenger.start
  485. -179> 2016-05-03 11:21:27.386168 7fa68bb75940 2 osd.51 0 mounting /var/lib/ceph/osd/ceph-51 /var/lib/ceph/osd/ceph-51/journal
  486. -178> 2016-05-03 11:21:27.386237 7fa68bb75940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  487. -177> 2016-05-03 11:21:27.386680 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  488. -176> 2016-05-03 11:21:27.386688 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  489. -175> 2016-05-03 11:21:27.386709 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  490. -174> 2016-05-03 11:21:27.414976 7fa68bb75940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  491. -173> 2016-05-03 11:21:27.415050 7fa68bb75940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  492. -172> 2016-05-03 11:21:27.516147 7fa68bb75940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  493. -171> 2016-05-03 11:21:27.518434 7fa68bb75940 2 journal open /var/lib/ceph/osd/ceph-51/journal fsid 3ca75ccf-1c42-4282-9504-c07f2cee91b3 fs_op_seq 21461862
  494. -170> 2016-05-03 11:21:27.518512 7fa68bb75940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  495. -169> 2016-05-03 11:21:27.518720 7fa68bb75940 2 journal open advancing committed_seq 21461799 to fs op_seq 21461862
  496. -168> 2016-05-03 11:21:27.532204 7fa68bb75940 2 journal read_entry 256061440 : seq 21461800 3021 bytes
  497. -167> 2016-05-03 11:21:27.532425 7fa68bb75940 2 journal read_entry 256065536 : seq 21461801 2981 bytes
  498. -166> 2016-05-03 11:21:27.532484 7fa68bb75940 2 journal read_entry 256069632 : seq 21461802 2864 bytes
  499. -165> 2016-05-03 11:21:27.532507 7fa68bb75940 2 journal read_entry 256073728 : seq 21461803 2788 bytes
  500. -164> 2016-05-03 11:21:27.532528 7fa68bb75940 2 journal read_entry 256077824 : seq 21461804 3252 bytes
  501. -163> 2016-05-03 11:21:27.532683 7fa68bb75940 2 journal read_entry 256081920 : seq 21461805 3072 bytes
  502. -162> 2016-05-03 11:21:27.532695 7fa68bb75940 2 journal read_entry 256086016 : seq 21461806 3136 bytes
  503. -161> 2016-05-03 11:21:27.532705 7fa68bb75940 2 journal read_entry 256090112 : seq 21461807 2178 bytes
  504. -160> 2016-05-03 11:21:27.532713 7fa68bb75940 2 journal read_entry 256094208 : seq 21461808 2146 bytes
  505. -159> 2016-05-03 11:21:27.532721 7fa68bb75940 2 journal read_entry 256098304 : seq 21461809 2417 bytes
  506. -158> 2016-05-03 11:21:27.532729 7fa68bb75940 2 journal read_entry 256102400 : seq 21461810 2291 bytes
  507. -157> 2016-05-03 11:21:27.532737 7fa68bb75940 2 journal read_entry 256106496 : seq 21461811 2417 bytes
  508. -156> 2016-05-03 11:21:27.532744 7fa68bb75940 2 journal read_entry 256110592 : seq 21461812 2417 bytes
  509. -155> 2016-05-03 11:21:27.532947 7fa68bb75940 2 journal read_entry 256114688 : seq 21461813 2499 bytes
  510. -154> 2016-05-03 11:21:27.532958 7fa68bb75940 2 journal read_entry 256118784 : seq 21461814 3140 bytes
  511. -153> 2016-05-03 11:21:27.532968 7fa68bb75940 2 journal read_entry 256122880 : seq 21461815 2385 bytes
  512. -152> 2016-05-03 11:21:27.532976 7fa68bb75940 2 journal read_entry 256126976 : seq 21461816 2385 bytes
  513. -151> 2016-05-03 11:21:27.532983 7fa68bb75940 2 journal read_entry 256131072 : seq 21461817 2483 bytes
  514. -150> 2016-05-03 11:21:27.532992 7fa68bb75940 2 journal read_entry 256135168 : seq 21461818 2275 bytes
  515. -149> 2016-05-03 11:21:27.533000 7fa68bb75940 2 journal read_entry 256139264 : seq 21461819 2889 bytes
  516. -148> 2016-05-03 11:21:27.533008 7fa68bb75940 2 journal read_entry 256143360 : seq 21461820 3144 bytes
  517. -147> 2016-05-03 11:21:27.533016 7fa68bb75940 2 journal read_entry 256147456 : seq 21461821 3156 bytes
  518. -146> 2016-05-03 11:21:27.533023 7fa68bb75940 2 journal read_entry 256151552 : seq 21461822 2146 bytes
  519. -145> 2016-05-03 11:21:27.533031 7fa68bb75940 2 journal read_entry 256155648 : seq 21461823 2889 bytes
  520. -144> 2016-05-03 11:21:27.533038 7fa68bb75940 2 journal read_entry 256159744 : seq 21461824 2412 bytes
  521. -143> 2016-05-03 11:21:27.533046 7fa68bb75940 2 journal read_entry 256163840 : seq 21461825 2291 bytes
  522. -142> 2016-05-03 11:21:27.533055 7fa68bb75940 2 journal read_entry 256167936 : seq 21461826 3446 bytes
  523. -141> 2016-05-03 11:21:27.533062 7fa68bb75940 2 journal read_entry 256172032 : seq 21461827 3201 bytes
  524. -140> 2016-05-03 11:21:27.533070 7fa68bb75940 2 journal read_entry 256176128 : seq 21461828 2417 bytes
  525. -139> 2016-05-03 11:21:27.533298 7fa68bb75940 2 journal read_entry 256180224 : seq 21461829 3050 bytes
  526. -138> 2016-05-03 11:21:27.533309 7fa68bb75940 2 journal read_entry 256184320 : seq 21461830 2146 bytes
  527. -137> 2016-05-03 11:21:27.533317 7fa68bb75940 2 journal read_entry 256188416 : seq 21461831 3012 bytes
  528. -136> 2016-05-03 11:21:27.533325 7fa68bb75940 2 journal read_entry 256192512 : seq 21461832 3160 bytes
  529. -135> 2016-05-03 11:21:27.533332 7fa68bb75940 2 journal read_entry 256196608 : seq 21461833 2781 bytes
  530. -134> 2016-05-03 11:21:27.533340 7fa68bb75940 2 journal read_entry 256200704 : seq 21461834 2385 bytes
  531. -133> 2016-05-03 11:21:27.533348 7fa68bb75940 2 journal read_entry 256204800 : seq 21461835 2385 bytes
  532. -132> 2016-05-03 11:21:27.533356 7fa68bb75940 2 journal read_entry 256208896 : seq 21461836 2389 bytes
  533. -131> 2016-05-03 11:21:27.533363 7fa68bb75940 2 journal read_entry 256212992 : seq 21461837 2385 bytes
  534. -130> 2016-05-03 11:21:27.533371 7fa68bb75940 2 journal read_entry 256217088 : seq 21461838 2275 bytes
  535. -129> 2016-05-03 11:21:27.533379 7fa68bb75940 2 journal read_entry 256221184 : seq 21461839 2417 bytes
  536. -128> 2016-05-03 11:21:27.533386 7fa68bb75940 2 journal read_entry 256225280 : seq 21461840 2385 bytes
  537. -127> 2016-05-03 11:21:27.533394 7fa68bb75940 2 journal read_entry 256229376 : seq 21461841 2417 bytes
  538. -126> 2016-05-03 11:21:27.533402 7fa68bb75940 2 journal read_entry 256233472 : seq 21461842 2990 bytes
  539. -125> 2016-05-03 11:21:27.533411 7fa68bb75940 2 journal read_entry 256237568 : seq 21461843 3028 bytes
  540. -124> 2016-05-03 11:21:27.533418 7fa68bb75940 2 journal read_entry 256241664 : seq 21461844 2889 bytes
  541. -123> 2016-05-03 11:21:27.533628 7fa68bb75940 2 journal read_entry 256245760 : seq 21461845 2990 bytes
  542. -122> 2016-05-03 11:21:27.533640 7fa68bb75940 2 journal read_entry 256249856 : seq 21461846 2788 bytes
  543. -121> 2016-05-03 11:21:27.533648 7fa68bb75940 2 journal read_entry 256253952 : seq 21461847 2990 bytes
  544. -120> 2016-05-03 11:21:27.533656 7fa68bb75940 2 journal read_entry 256258048 : seq 21461848 3036 bytes
  545. -119> 2016-05-03 11:21:27.533663 7fa68bb75940 2 journal read_entry 256262144 : seq 21461849 2788 bytes
  546. -118> 2016-05-03 11:21:27.533671 7fa68bb75940 2 journal read_entry 256266240 : seq 21461850 2904 bytes
  547. -117> 2016-05-03 11:21:27.533679 7fa68bb75940 2 journal read_entry 256270336 : seq 21461851 2840 bytes
  548. -116> 2016-05-03 11:21:27.533686 7fa68bb75940 2 journal read_entry 256274432 : seq 21461852 2990 bytes
  549. -115> 2016-05-03 11:21:27.533694 7fa68bb75940 2 journal read_entry 256278528 : seq 21461853 2856 bytes
  550. -114> 2016-05-03 11:21:27.559448 7fa68bb75940 2 journal read_entry 256282624 : seq 21461854 4196824 bytes
  551. -113> 2016-05-03 11:21:27.559474 7fa68bb75940 2 journal read_entry 260485120 : seq 21461855 2840 bytes
  552. -112> 2016-05-03 11:21:27.559485 7fa68bb75940 2 journal read_entry 260489216 : seq 21461856 2990 bytes
  553. -111> 2016-05-03 11:21:27.564350 7fa68bb75940 2 journal read_entry 260493312 : seq 21461857 869132 bytes
  554. -110> 2016-05-03 11:21:27.564369 7fa68bb75940 2 journal read_entry 261365760 : seq 21461858 7957 bytes
  555. -109> 2016-05-03 11:21:27.564378 7fa68bb75940 2 journal read_entry 261373952 : seq 21461859 2438 bytes
  556. -108> 2016-05-03 11:21:27.564387 7fa68bb75940 2 journal read_entry 261378048 : seq 21461860 2990 bytes
  557. -107> 2016-05-03 11:21:27.564395 7fa68bb75940 2 journal read_entry 261382144 : seq 21461861 2856 bytes
  558. -106> 2016-05-03 11:21:27.564403 7fa68bb75940 2 journal read_entry 261386240 : seq 21461862 2964 bytes
  559. -105> 2016-05-03 11:21:27.564412 7fa68bb75940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  560. -104> 2016-05-03 11:21:27.564421 7fa68bb75940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  561. -103> 2016-05-03 11:21:27.564424 7fa68bb75940 3 journal journal_replay: end of journal, done.
  562. -102> 2016-05-03 11:21:27.566425 7fa68bb75940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  563. -101> 2016-05-03 11:21:27.567494 7fa68bb75940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  564. -100> 2016-05-03 11:21:27.567518 7fa68bb75940 2 osd.51 0 boot
  565. -99> 2016-05-03 11:21:27.568172 7fa68bb75940 1 <cls> cls/timeindex/cls_timeindex.cc:259: Loaded timeindex class!
  566. -98> 2016-05-03 11:21:27.568298 7fa68bb75940 1 <cls> cls/log/cls_log.cc:313: Loaded log class!
  567. -97> 2016-05-03 11:21:27.571095 7fa68bb75940 1 <cls> cls/refcount/cls_refcount.cc:232: Loaded refcount class!
  568. -96> 2016-05-03 11:21:27.571240 7fa68bb75940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  569. -95> 2016-05-03 11:21:27.573531 7fa68bb75940 1 <cls> cls/rgw/cls_rgw.cc:3059: Loaded rgw class!
  570. -94> 2016-05-03 11:21:27.573663 7fa68bb75940 1 <cls> cls/version/cls_version.cc:228: Loaded version class!
  571. -93> 2016-05-03 11:21:27.574091 7fa68bb75940 1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class!
  572. -92> 2016-05-03 11:21:27.574336 7fa68bb75940 1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class!
  573. -91> 2016-05-03 11:21:27.574430 7fa68bb75940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  574. -90> 2016-05-03 11:21:27.574537 7fa68bb75940 1 <cls> cls/user/cls_user.cc:367: Loaded user class!
  575. -89> 2016-05-03 11:21:27.574849 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  576. -88> 2016-05-03 11:21:27.574860 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  577. -87> 2016-05-03 11:21:27.574865 7fa68bb75940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  578. -86> 2016-05-03 11:21:27.657473 7fa68bb75940 0 osd.51 160621 load_pgs
  579. -85> 2016-05-03 11:21:27.665862 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.6(unlocked)] enter Initial
  580. -84> 2016-05-03 11:21:27.666195 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000333 0 0.000000
  581. -83> 2016-05-03 11:21:27.666220 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  582. -82> 2016-05-03 11:21:27.666309 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.1a(unlocked)] enter Initial
  583. -81> 2016-05-03 11:21:27.666505 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000195 0 0.000000
  584. -80> 2016-05-03 11:21:27.666519 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  585. -79> 2016-05-03 11:21:27.666601 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.22(unlocked)] enter Initial
  586. -78> 2016-05-03 11:21:27.666768 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.000167 0 0.000000
  587. -77> 2016-05-03 11:21:27.666782 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  588. -76> 2016-05-03 11:21:27.667196 7fa68bb75940 5 osd.51 pg_epoch: 160581 pg[1.36(unlocked)] enter Initial
  589. -75> 2016-05-03 11:21:27.668449 7fa68bb75940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001253 0 0.000000
  590. -74> 2016-05-03 11:21:27.668466 7fa68bb75940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  591. -73> 2016-05-03 11:21:27.668866 7fa68bb75940 5 osd.51 pg_epoch: 160599 pg[1.3f(unlocked)] enter Initial
  592. -72> 2016-05-03 11:21:27.670161 7fa68bb75940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001294 0 0.000000
  593. -71> 2016-05-03 11:21:27.670184 7fa68bb75940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  594. -70> 2016-05-03 11:21:27.670606 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.4(unlocked)] enter Initial
  595. -69> 2016-05-03 11:21:27.693922 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.023316 0 0.000000
  596. -68> 2016-05-03 11:21:27.693946 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  597. -67> 2016-05-03 11:21:27.694102 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.d(unlocked)] enter Initial
  598. -66> 2016-05-03 11:21:27.714258 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020156 0 0.000000
  599. -65> 2016-05-03 11:21:27.714280 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  600. -64> 2016-05-03 11:21:27.714470 7fa68bb75940 5 osd.51 pg_epoch: 160621 pg[2.13(unlocked)] enter Initial
  601. -63> 2016-05-03 11:21:27.739463 7fa68bb75940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024992 0 0.000000
  602. -62> 2016-05-03 11:21:27.739484 7fa68bb75940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  603. -61> 2016-05-03 11:21:27.739631 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.14(unlocked)] enter Initial
  604. -60> 2016-05-03 11:21:27.760269 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020638 0 0.000000
  605. -59> 2016-05-03 11:21:27.760292 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  606. -58> 2016-05-03 11:21:27.760429 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1b(unlocked)] enter Initial
  607. -57> 2016-05-03 11:21:27.780255 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019826 0 0.000000
  608. -56> 2016-05-03 11:21:27.780276 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  609. -55> 2016-05-03 11:21:27.780428 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.32(unlocked)] enter Initial
  610. -54> 2016-05-03 11:21:27.839090 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] exit Initial 0.058662 0 0.000000
  611. -53> 2016-05-03 11:21:27.839118 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] enter Reset
  612. -52> 2016-05-03 11:21:27.839277 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.3b(unlocked)] enter Initial
  613. -51> 2016-05-03 11:21:27.894480 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] exit Initial 0.055202 0 0.000000
  614. -50> 2016-05-03 11:21:27.894502 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] enter Reset
  615. -49> 2016-05-03 11:21:27.894707 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[2.48(unlocked)] enter Initial
  616. -48> 2016-05-03 11:21:27.894883 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000176 0 0.000000
  617. -47> 2016-05-03 11:21:27.894898 7fa68bb75940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  618. -46> 2016-05-03 11:21:27.895019 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.50(unlocked)] enter Initial
  619. -45> 2016-05-03 11:21:27.918714 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.023695 0 0.000000
  620. -44> 2016-05-03 11:21:27.918736 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  621. -43> 2016-05-03 11:21:27.918875 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.51(unlocked)] enter Initial
  622. -42> 2016-05-03 11:21:27.940515 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021640 0 0.000000
  623. -41> 2016-05-03 11:21:27.940537 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  624. -40> 2016-05-03 11:21:27.940683 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.72(unlocked)] enter Initial
  625. -39> 2016-05-03 11:21:27.962667 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021983 0 0.000000
  626. -38> 2016-05-03 11:21:27.962688 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  627. -37> 2016-05-03 11:21:27.962815 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.b7(unlocked)] enter Initial
  628. -36> 2016-05-03 11:21:27.985501 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.022686 0 0.000000
  629. -35> 2016-05-03 11:21:27.985523 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  630. -34> 2016-05-03 11:21:27.986152 7fa68bb75940 5 osd.51 pg_epoch: 160595 pg[2.cf(unlocked)] enter Initial
  631. -33> 2016-05-03 11:21:28.015773 7fa68bb75940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.029620 0 0.000000
  632. -32> 2016-05-03 11:21:28.015798 7fa68bb75940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  633. -31> 2016-05-03 11:21:28.015957 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.10e(unlocked)] enter Initial
  634. -30> 2016-05-03 11:21:28.036443 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020485 0 0.000000
  635. -29> 2016-05-03 11:21:28.036465 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  636. -28> 2016-05-03 11:21:28.036618 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.111(unlocked)] enter Initial
  637. -27> 2016-05-03 11:21:28.107435 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] exit Initial 0.070817 0 0.000000
  638. -26> 2016-05-03 11:21:28.107462 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] enter Reset
  639. -25> 2016-05-03 11:21:28.107615 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.16f(unlocked)] enter Initial
  640. -24> 2016-05-03 11:21:28.182308 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] exit Initial 0.074693 0 0.000000
  641. -23> 2016-05-03 11:21:28.182335 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] enter Reset
  642. -22> 2016-05-03 11:21:28.182956 7fa68bb75940 5 osd.51 pg_epoch: 160607 pg[2.186(unlocked)] enter Initial
  643. -21> 2016-05-03 11:21:28.208911 7fa68bb75940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025955 0 0.000000
  644. -20> 2016-05-03 11:21:28.208931 7fa68bb75940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  645. -19> 2016-05-03 11:21:28.209069 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.188(unlocked)] enter Initial
  646. -18> 2016-05-03 11:21:28.235346 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.026277 0 0.000000
  647. -17> 2016-05-03 11:21:28.235369 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  648. -16> 2016-05-03 11:21:28.235539 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1a2(unlocked)] enter Initial
  649. -15> 2016-05-03 11:21:28.256112 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020572 0 0.000000
  650. -14> 2016-05-03 11:21:28.256134 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  651. -13> 2016-05-03 11:21:28.256258 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1aa(unlocked)] enter Initial
  652. -12> 2016-05-03 11:21:28.281595 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025337 0 0.000000
  653. -11> 2016-05-03 11:21:28.281620 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  654. -10> 2016-05-03 11:21:28.281765 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1bb(unlocked)] enter Initial
  655. -9> 2016-05-03 11:21:28.307953 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.026187 0 0.000000
  656. -8> 2016-05-03 11:21:28.307977 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  657. -7> 2016-05-03 11:21:28.308126 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1e8(unlocked)] enter Initial
  658. -6> 2016-05-03 11:21:28.348322 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.040195 0 0.000000
  659. -5> 2016-05-03 11:21:28.348370 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  660. -4> 2016-05-03 11:21:28.348626 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1f2(unlocked)] enter Initial
  661. -3> 2016-05-03 11:21:28.388240 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.039614 0 0.000000
  662. -2> 2016-05-03 11:21:28.388267 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  663. -1> 2016-05-03 11:21:28.388459 7fa68bb75940 5 osd.51 pg_epoch: 160608 pg[2.1fd(unlocked)] enter Initial
  664. 0> 2016-05-03 11:21:28.463067 7fa68bb75940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7fa68bb75940 time 2016-05-03 11:21:28.460685
  665. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  666.  
  667. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  668. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7fa68b66b7eb]
  669. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fa68b1a5e8d]
  670. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fa68b22255a]
  671. 4: (OSD::load_pgs()+0x61d) [0x7fa68b0e058d]
  672. 5: (OSD::init()+0xe74) [0x7fa68b0f0694]
  673. 6: (main()+0x2954) [0x7fa68b075474]
  674. 7: (__libc_start_main()+0xf5) [0x7fa687ef7ec5]
  675. 8: (()+0x2f82b7) [0x7fa68b0a52b7]
  676. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  677.  
  678. --- logging levels ---
  679. 0/ 5 none
  680. 0/ 1 lockdep
  681. 0/ 1 context
  682. 1/ 1 crush
  683. 1/ 5 mds
  684. 1/ 5 mds_balancer
  685. 1/ 5 mds_locker
  686. 1/ 5 mds_log
  687. 1/ 5 mds_log_expire
  688. 1/ 5 mds_migrator
  689. 0/ 1 buffer
  690. 0/ 1 timer
  691. 0/ 1 filer
  692. 0/ 1 striper
  693. 0/ 1 objecter
  694. 0/ 5 rados
  695. 0/ 5 rbd
  696. 0/ 5 rbd_replay
  697. 0/ 5 journaler
  698. 0/ 5 objectcacher
  699. 0/ 5 client
  700. 0/ 5 osd
  701. 0/ 5 optracker
  702. 0/ 5 objclass
  703. 1/ 3 filestore
  704. 1/ 3 keyvaluestore
  705. 1/ 3 journal
  706. 0/ 5 ms
  707. 1/ 5 mon
  708. 0/10 monc
  709. 1/ 5 paxos
  710. 0/ 5 tp
  711. 1/ 5 auth
  712. 1/ 5 crypto
  713. 1/ 1 finisher
  714. 1/ 5 heartbeatmap
  715. 1/ 5 perfcounter
  716. 1/ 5 rgw
  717. 1/10 civetweb
  718. 1/ 5 javaclient
  719. 1/ 5 asok
  720. 1/ 1 throttle
  721. 0/ 0 refs
  722. 1/ 5 xio
  723. 1/ 5 compressor
  724. 1/ 5 newstore
  725. -2/-2 (syslog threshold)
  726. -1/-1 (stderr threshold)
  727. max_recent 10000
  728. max_new 1000
  729. log_file /var/log/ceph/ceph-osd.51.log
  730. --- end dump of recent events ---
  731. 2016-05-03 11:21:28.467923 7fa68bb75940 -1 *** Caught signal (Aborted) **
  732. in thread 7fa68bb75940
  733.  
  734. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  735. 1: (()+0x7d1aca) [0x7fa68b57eaca]
  736. 2: (()+0x10340) [0x7fa689c91340]
  737. 3: (gsignal()+0x39) [0x7fa687f0ccc9]
  738. 4: (abort()+0x148) [0x7fa687f100d8]
  739. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa688817535]
  740. 6: (()+0x5e6d6) [0x7fa6888156d6]
  741. 7: (()+0x5e703) [0x7fa688815703]
  742. 8: (()+0x5e922) [0x7fa688815922]
  743. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7fa68b66b9d8]
  744. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fa68b1a5e8d]
  745. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fa68b22255a]
  746. 12: (OSD::load_pgs()+0x61d) [0x7fa68b0e058d]
  747. 13: (OSD::init()+0xe74) [0x7fa68b0f0694]
  748. 14: (main()+0x2954) [0x7fa68b075474]
  749. 15: (__libc_start_main()+0xf5) [0x7fa687ef7ec5]
  750. 16: (()+0x2f82b7) [0x7fa68b0a52b7]
  751. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  752.  
  753. --- begin dump of recent events ---
  754. 0> 2016-05-03 11:21:28.467923 7fa68bb75940 -1 *** Caught signal (Aborted) **
  755. in thread 7fa68bb75940
  756.  
  757. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  758. 1: (()+0x7d1aca) [0x7fa68b57eaca]
  759. 2: (()+0x10340) [0x7fa689c91340]
  760. 3: (gsignal()+0x39) [0x7fa687f0ccc9]
  761. 4: (abort()+0x148) [0x7fa687f100d8]
  762. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa688817535]
  763. 6: (()+0x5e6d6) [0x7fa6888156d6]
  764. 7: (()+0x5e703) [0x7fa688815703]
  765. 8: (()+0x5e922) [0x7fa688815922]
  766. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7fa68b66b9d8]
  767. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7fa68b1a5e8d]
  768. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7fa68b22255a]
  769. 12: (OSD::load_pgs()+0x61d) [0x7fa68b0e058d]
  770. 13: (OSD::init()+0xe74) [0x7fa68b0f0694]
  771. 14: (main()+0x2954) [0x7fa68b075474]
  772. 15: (__libc_start_main()+0xf5) [0x7fa687ef7ec5]
  773. 16: (()+0x2f82b7) [0x7fa68b0a52b7]
  774. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  775.  
  776. --- logging levels ---
  777. 0/ 5 none
  778. 0/ 1 lockdep
  779. 0/ 1 context
  780. 1/ 1 crush
  781. 1/ 5 mds
  782. 1/ 5 mds_balancer
  783. 1/ 5 mds_locker
  784. 1/ 5 mds_log
  785. 1/ 5 mds_log_expire
  786. 1/ 5 mds_migrator
  787. 0/ 1 buffer
  788. 0/ 1 timer
  789. 0/ 1 filer
  790. 0/ 1 striper
  791. 0/ 1 objecter
  792. 0/ 5 rados
  793. 0/ 5 rbd
  794. 0/ 5 rbd_replay
  795. 0/ 5 journaler
  796. 0/ 5 objectcacher
  797. 0/ 5 client
  798. 0/ 5 osd
  799. 0/ 5 optracker
  800. 0/ 5 objclass
  801. 1/ 3 filestore
  802. 1/ 3 keyvaluestore
  803. 1/ 3 journal
  804. 0/ 5 ms
  805. 1/ 5 mon
  806. 0/10 monc
  807. 1/ 5 paxos
  808. 0/ 5 tp
  809. 1/ 5 auth
  810. 1/ 5 crypto
  811. 1/ 1 finisher
  812. 1/ 5 heartbeatmap
  813. 1/ 5 perfcounter
  814. 1/ 5 rgw
  815. 1/10 civetweb
  816. 1/ 5 javaclient
  817. 1/ 5 asok
  818. 1/ 1 throttle
  819. 0/ 0 refs
  820. 1/ 5 xio
  821. 1/ 5 compressor
  822. 1/ 5 newstore
  823. -2/-2 (syslog threshold)
  824. -1/-1 (stderr threshold)
  825. max_recent 10000
  826. max_new 1000
  827. log_file /var/log/ceph/ceph-osd.51.log
  828. --- end dump of recent events ---
  829. 2016-05-03 11:21:28.996176 7f3072eb2940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14872
  830. 2016-05-03 11:21:29.008866 7f3072eb2940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  831. 2016-05-03 11:21:29.009294 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  832. 2016-05-03 11:21:29.009302 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  833. 2016-05-03 11:21:29.009321 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  834. 2016-05-03 11:21:29.032052 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  835. 2016-05-03 11:21:29.032134 7f3072eb2940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  836. 2016-05-03 11:21:29.124895 7f3072eb2940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  837. 2016-05-03 11:21:29.127357 7f3072eb2940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  838. 2016-05-03 11:21:29.180465 7f3072eb2940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  839. 2016-05-03 11:21:29.181518 7f3072eb2940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  840. 2016-05-03 11:21:29.185220 7f3072eb2940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  841. 2016-05-03 11:21:29.188396 7f3072eb2940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  842. 2016-05-03 11:21:29.188830 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  843. 2016-05-03 11:21:29.188840 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  844. 2016-05-03 11:21:29.188846 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  845. 2016-05-03 11:21:29.284009 7f3072eb2940 0 osd.51 160621 load_pgs
  846. 2016-05-03 11:21:30.051924 7f3072eb2940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7f3072eb2940 time 2016-05-03 11:21:30.049490
  847. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  848.  
  849. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  850. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f30729a87eb]
  851. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7f30724e2e8d]
  852. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7f307255f55a]
  853. 4: (OSD::load_pgs()+0x61d) [0x7f307241d58d]
  854. 5: (OSD::init()+0xe74) [0x7f307242d694]
  855. 6: (main()+0x2954) [0x7f30723b2474]
  856. 7: (__libc_start_main()+0xf5) [0x7f306f234ec5]
  857. 8: (()+0x2f82b7) [0x7f30723e22b7]
  858. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  859.  
  860. --- begin dump of recent events ---
  861. -217> 2016-05-03 11:21:28.993030 7f3072eb2940 5 asok(0x7f3077184000) register_command perfcounters_dump hook 0x7f3077104050
  862. -216> 2016-05-03 11:21:28.993076 7f3072eb2940 5 asok(0x7f3077184000) register_command 1 hook 0x7f3077104050
  863. -215> 2016-05-03 11:21:28.993085 7f3072eb2940 5 asok(0x7f3077184000) register_command perf dump hook 0x7f3077104050
  864. -214> 2016-05-03 11:21:28.993089 7f3072eb2940 5 asok(0x7f3077184000) register_command perfcounters_schema hook 0x7f3077104050
  865. -213> 2016-05-03 11:21:28.993097 7f3072eb2940 5 asok(0x7f3077184000) register_command 2 hook 0x7f3077104050
  866. -212> 2016-05-03 11:21:28.993100 7f3072eb2940 5 asok(0x7f3077184000) register_command perf schema hook 0x7f3077104050
  867. -211> 2016-05-03 11:21:28.993103 7f3072eb2940 5 asok(0x7f3077184000) register_command perf reset hook 0x7f3077104050
  868. -210> 2016-05-03 11:21:28.993109 7f3072eb2940 5 asok(0x7f3077184000) register_command config show hook 0x7f3077104050
  869. -209> 2016-05-03 11:21:28.993112 7f3072eb2940 5 asok(0x7f3077184000) register_command config set hook 0x7f3077104050
  870. -208> 2016-05-03 11:21:28.993115 7f3072eb2940 5 asok(0x7f3077184000) register_command config get hook 0x7f3077104050
  871. -207> 2016-05-03 11:21:28.993122 7f3072eb2940 5 asok(0x7f3077184000) register_command config diff hook 0x7f3077104050
  872. -206> 2016-05-03 11:21:28.993125 7f3072eb2940 5 asok(0x7f3077184000) register_command log flush hook 0x7f3077104050
  873. -205> 2016-05-03 11:21:28.993128 7f3072eb2940 5 asok(0x7f3077184000) register_command log dump hook 0x7f3077104050
  874. -204> 2016-05-03 11:21:28.993135 7f3072eb2940 5 asok(0x7f3077184000) register_command log reopen hook 0x7f3077104050
  875. -203> 2016-05-03 11:21:28.996094 7f3072eb2940 0 set uid:gid to 64045:64045
  876. -202> 2016-05-03 11:21:28.996176 7f3072eb2940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14872
  877. -201> 2016-05-03 11:21:28.996757 7f3072eb2940 1 accepter.accepter.bind my_inst.addr is [::]:6808/14872 need_addr=1
  878. -200> 2016-05-03 11:21:28.996782 7f3072eb2940 1 accepter.accepter.bind my_inst.addr is [::]:6809/14872 need_addr=1
  879. -199> 2016-05-03 11:21:28.996797 7f3072eb2940 1 accepter.accepter.bind my_inst.addr is [::]:6810/14872 need_addr=1
  880. -198> 2016-05-03 11:21:28.996811 7f3072eb2940 1 accepter.accepter.bind my_inst.addr is [::]:6811/14872 need_addr=1
  881. -197> 2016-05-03 11:21:28.998554 7f3072eb2940 5 asok(0x7f3077184000) init /var/run/ceph/ceph-osd.51.asok
  882. -196> 2016-05-03 11:21:28.998566 7f3072eb2940 5 asok(0x7f3077184000) bind_and_listen /var/run/ceph/ceph-osd.51.asok
  883. -195> 2016-05-03 11:21:28.998660 7f3072eb2940 5 asok(0x7f3077184000) register_command 0 hook 0x7f30771000b0
  884. -194> 2016-05-03 11:21:28.998667 7f3072eb2940 5 asok(0x7f3077184000) register_command version hook 0x7f30771000b0
  885. -193> 2016-05-03 11:21:28.998670 7f3072eb2940 5 asok(0x7f3077184000) register_command git_version hook 0x7f30771000b0
  886. -192> 2016-05-03 11:21:28.998675 7f3072eb2940 5 asok(0x7f3077184000) register_command help hook 0x7f3077104230
  887. -191> 2016-05-03 11:21:28.998680 7f3072eb2940 5 asok(0x7f3077184000) register_command get_command_descriptions hook 0x7f3077104220
  888. -190> 2016-05-03 11:21:28.998719 7f306c1c2700 5 asok(0x7f3077184000) entry start
  889. -189> 2016-05-03 11:21:28.998762 7f3072eb2940 10 monclient(hunting): build_initial_monmap
  890. -188> 2016-05-03 11:21:29.008431 7f3072eb2940 5 adding auth protocol: cephx
  891. -187> 2016-05-03 11:21:29.008440 7f3072eb2940 5 adding auth protocol: cephx
  892. -186> 2016-05-03 11:21:29.008527 7f3072eb2940 5 asok(0x7f3077184000) register_command objecter_requests hook 0x7f3077104270
  893. -185> 2016-05-03 11:21:29.008588 7f3072eb2940 1 -- [::]:6808/14872 messenger.start
  894. -184> 2016-05-03 11:21:29.008620 7f3072eb2940 1 -- :/0 messenger.start
  895. -183> 2016-05-03 11:21:29.008642 7f3072eb2940 1 -- [::]:6811/14872 messenger.start
  896. -182> 2016-05-03 11:21:29.008664 7f3072eb2940 1 -- [::]:6810/14872 messenger.start
  897. -181> 2016-05-03 11:21:29.008685 7f3072eb2940 1 -- [::]:6809/14872 messenger.start
  898. -180> 2016-05-03 11:21:29.008707 7f3072eb2940 1 -- :/0 messenger.start
  899. -179> 2016-05-03 11:21:29.008789 7f3072eb2940 2 osd.51 0 mounting /var/lib/ceph/osd/ceph-51 /var/lib/ceph/osd/ceph-51/journal
  900. -178> 2016-05-03 11:21:29.008866 7f3072eb2940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  901. -177> 2016-05-03 11:21:29.009294 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  902. -176> 2016-05-03 11:21:29.009302 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  903. -175> 2016-05-03 11:21:29.009321 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  904. -174> 2016-05-03 11:21:29.032052 7f3072eb2940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  905. -173> 2016-05-03 11:21:29.032134 7f3072eb2940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  906. -172> 2016-05-03 11:21:29.124895 7f3072eb2940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  907. -171> 2016-05-03 11:21:29.127280 7f3072eb2940 2 journal open /var/lib/ceph/osd/ceph-51/journal fsid 3ca75ccf-1c42-4282-9504-c07f2cee91b3 fs_op_seq 21461862
  908. -170> 2016-05-03 11:21:29.127357 7f3072eb2940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  909. -169> 2016-05-03 11:21:29.140136 7f3072eb2940 2 journal open advancing committed_seq 21461799 to fs op_seq 21461862
  910. -168> 2016-05-03 11:21:29.148862 7f3072eb2940 2 journal read_entry 256061440 : seq 21461800 3021 bytes
  911. -167> 2016-05-03 11:21:29.149076 7f3072eb2940 2 journal read_entry 256065536 : seq 21461801 2981 bytes
  912. -166> 2016-05-03 11:21:29.149131 7f3072eb2940 2 journal read_entry 256069632 : seq 21461802 2864 bytes
  913. -165> 2016-05-03 11:21:29.149156 7f3072eb2940 2 journal read_entry 256073728 : seq 21461803 2788 bytes
  914. -164> 2016-05-03 11:21:29.149176 7f3072eb2940 2 journal read_entry 256077824 : seq 21461804 3252 bytes
  915. -163> 2016-05-03 11:21:29.149321 7f3072eb2940 2 journal read_entry 256081920 : seq 21461805 3072 bytes
  916. -162> 2016-05-03 11:21:29.149333 7f3072eb2940 2 journal read_entry 256086016 : seq 21461806 3136 bytes
  917. -161> 2016-05-03 11:21:29.149344 7f3072eb2940 2 journal read_entry 256090112 : seq 21461807 2178 bytes
  918. -160> 2016-05-03 11:21:29.149352 7f3072eb2940 2 journal read_entry 256094208 : seq 21461808 2146 bytes
  919. -159> 2016-05-03 11:21:29.149361 7f3072eb2940 2 journal read_entry 256098304 : seq 21461809 2417 bytes
  920. -158> 2016-05-03 11:21:29.149369 7f3072eb2940 2 journal read_entry 256102400 : seq 21461810 2291 bytes
  921. -157> 2016-05-03 11:21:29.149385 7f3072eb2940 2 journal read_entry 256106496 : seq 21461811 2417 bytes
  922. -156> 2016-05-03 11:21:29.149393 7f3072eb2940 2 journal read_entry 256110592 : seq 21461812 2417 bytes
  923. -155> 2016-05-03 11:21:29.149593 7f3072eb2940 2 journal read_entry 256114688 : seq 21461813 2499 bytes
  924. -154> 2016-05-03 11:21:29.149605 7f3072eb2940 2 journal read_entry 256118784 : seq 21461814 3140 bytes
  925. -153> 2016-05-03 11:21:29.149614 7f3072eb2940 2 journal read_entry 256122880 : seq 21461815 2385 bytes
  926. -152> 2016-05-03 11:21:29.149623 7f3072eb2940 2 journal read_entry 256126976 : seq 21461816 2385 bytes
  927. -151> 2016-05-03 11:21:29.149631 7f3072eb2940 2 journal read_entry 256131072 : seq 21461817 2483 bytes
  928. -150> 2016-05-03 11:21:29.149641 7f3072eb2940 2 journal read_entry 256135168 : seq 21461818 2275 bytes
  929. -149> 2016-05-03 11:21:29.149649 7f3072eb2940 2 journal read_entry 256139264 : seq 21461819 2889 bytes
  930. -148> 2016-05-03 11:21:29.149657 7f3072eb2940 2 journal read_entry 256143360 : seq 21461820 3144 bytes
  931. -147> 2016-05-03 11:21:29.149665 7f3072eb2940 2 journal read_entry 256147456 : seq 21461821 3156 bytes
  932. -146> 2016-05-03 11:21:29.149673 7f3072eb2940 2 journal read_entry 256151552 : seq 21461822 2146 bytes
  933. -145> 2016-05-03 11:21:29.149681 7f3072eb2940 2 journal read_entry 256155648 : seq 21461823 2889 bytes
  934. -144> 2016-05-03 11:21:29.149689 7f3072eb2940 2 journal read_entry 256159744 : seq 21461824 2412 bytes
  935. -143> 2016-05-03 11:21:29.149696 7f3072eb2940 2 journal read_entry 256163840 : seq 21461825 2291 bytes
  936. -142> 2016-05-03 11:21:29.149705 7f3072eb2940 2 journal read_entry 256167936 : seq 21461826 3446 bytes
  937. -141> 2016-05-03 11:21:29.149713 7f3072eb2940 2 journal read_entry 256172032 : seq 21461827 3201 bytes
  938. -140> 2016-05-03 11:21:29.149720 7f3072eb2940 2 journal read_entry 256176128 : seq 21461828 2417 bytes
  939. -139> 2016-05-03 11:21:29.149924 7f3072eb2940 2 journal read_entry 256180224 : seq 21461829 3050 bytes
  940. -138> 2016-05-03 11:21:29.149936 7f3072eb2940 2 journal read_entry 256184320 : seq 21461830 2146 bytes
  941. -137> 2016-05-03 11:21:29.149945 7f3072eb2940 2 journal read_entry 256188416 : seq 21461831 3012 bytes
  942. -136> 2016-05-03 11:21:29.149953 7f3072eb2940 2 journal read_entry 256192512 : seq 21461832 3160 bytes
  943. -135> 2016-05-03 11:21:29.149961 7f3072eb2940 2 journal read_entry 256196608 : seq 21461833 2781 bytes
  944. -134> 2016-05-03 11:21:29.149969 7f3072eb2940 2 journal read_entry 256200704 : seq 21461834 2385 bytes
  945. -133> 2016-05-03 11:21:29.149977 7f3072eb2940 2 journal read_entry 256204800 : seq 21461835 2385 bytes
  946. -132> 2016-05-03 11:21:29.149985 7f3072eb2940 2 journal read_entry 256208896 : seq 21461836 2389 bytes
  947. -131> 2016-05-03 11:21:29.149993 7f3072eb2940 2 journal read_entry 256212992 : seq 21461837 2385 bytes
  948. -130> 2016-05-03 11:21:29.150001 7f3072eb2940 2 journal read_entry 256217088 : seq 21461838 2275 bytes
  949. -129> 2016-05-03 11:21:29.150008 7f3072eb2940 2 journal read_entry 256221184 : seq 21461839 2417 bytes
  950. -128> 2016-05-03 11:21:29.150033 7f3072eb2940 2 journal read_entry 256225280 : seq 21461840 2385 bytes
  951. -127> 2016-05-03 11:21:29.150043 7f3072eb2940 2 journal read_entry 256229376 : seq 21461841 2417 bytes
  952. -126> 2016-05-03 11:21:29.150052 7f3072eb2940 2 journal read_entry 256233472 : seq 21461842 2990 bytes
  953. -125> 2016-05-03 11:21:29.150068 7f3072eb2940 2 journal read_entry 256237568 : seq 21461843 3028 bytes
  954. -124> 2016-05-03 11:21:29.150077 7f3072eb2940 2 journal read_entry 256241664 : seq 21461844 2889 bytes
  955. -123> 2016-05-03 11:21:29.150265 7f3072eb2940 2 journal read_entry 256245760 : seq 21461845 2990 bytes
  956. -122> 2016-05-03 11:21:29.150276 7f3072eb2940 2 journal read_entry 256249856 : seq 21461846 2788 bytes
  957. -121> 2016-05-03 11:21:29.150285 7f3072eb2940 2 journal read_entry 256253952 : seq 21461847 2990 bytes
  958. -120> 2016-05-03 11:21:29.150292 7f3072eb2940 2 journal read_entry 256258048 : seq 21461848 3036 bytes
  959. -119> 2016-05-03 11:21:29.150300 7f3072eb2940 2 journal read_entry 256262144 : seq 21461849 2788 bytes
  960. -118> 2016-05-03 11:21:29.150308 7f3072eb2940 2 journal read_entry 256266240 : seq 21461850 2904 bytes
  961. -117> 2016-05-03 11:21:29.150316 7f3072eb2940 2 journal read_entry 256270336 : seq 21461851 2840 bytes
  962. -116> 2016-05-03 11:21:29.150324 7f3072eb2940 2 journal read_entry 256274432 : seq 21461852 2990 bytes
  963. -115> 2016-05-03 11:21:29.150332 7f3072eb2940 2 journal read_entry 256278528 : seq 21461853 2856 bytes
  964. -114> 2016-05-03 11:21:29.176160 7f3072eb2940 2 journal read_entry 256282624 : seq 21461854 4196824 bytes
  965. -113> 2016-05-03 11:21:29.176183 7f3072eb2940 2 journal read_entry 260485120 : seq 21461855 2840 bytes
  966. -112> 2016-05-03 11:21:29.176191 7f3072eb2940 2 journal read_entry 260489216 : seq 21461856 2990 bytes
  967. -111> 2016-05-03 11:21:29.178390 7f3072eb2940 2 journal read_entry 260493312 : seq 21461857 869132 bytes
  968. -110> 2016-05-03 11:21:29.178414 7f3072eb2940 2 journal read_entry 261365760 : seq 21461858 7957 bytes
  969. -109> 2016-05-03 11:21:29.178459 7f3072eb2940 2 journal read_entry 261373952 : seq 21461859 2438 bytes
  970. -108> 2016-05-03 11:21:29.178467 7f3072eb2940 2 journal read_entry 261378048 : seq 21461860 2990 bytes
  971. -107> 2016-05-03 11:21:29.178476 7f3072eb2940 2 journal read_entry 261382144 : seq 21461861 2856 bytes
  972. -106> 2016-05-03 11:21:29.178484 7f3072eb2940 2 journal read_entry 261386240 : seq 21461862 2964 bytes
  973. -105> 2016-05-03 11:21:29.178493 7f3072eb2940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  974. -104> 2016-05-03 11:21:29.178502 7f3072eb2940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  975. -103> 2016-05-03 11:21:29.178506 7f3072eb2940 3 journal journal_replay: end of journal, done.
  976. -102> 2016-05-03 11:21:29.180465 7f3072eb2940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  977. -101> 2016-05-03 11:21:29.181518 7f3072eb2940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  978. -100> 2016-05-03 11:21:29.181540 7f3072eb2940 2 osd.51 0 boot
  979. -99> 2016-05-03 11:21:29.182213 7f3072eb2940 1 <cls> cls/timeindex/cls_timeindex.cc:259: Loaded timeindex class!
  980. -98> 2016-05-03 11:21:29.182349 7f3072eb2940 1 <cls> cls/log/cls_log.cc:313: Loaded log class!
  981. -97> 2016-05-03 11:21:29.185077 7f3072eb2940 1 <cls> cls/refcount/cls_refcount.cc:232: Loaded refcount class!
  982. -96> 2016-05-03 11:21:29.185220 7f3072eb2940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  983. -95> 2016-05-03 11:21:29.187543 7f3072eb2940 1 <cls> cls/rgw/cls_rgw.cc:3059: Loaded rgw class!
  984. -94> 2016-05-03 11:21:29.187666 7f3072eb2940 1 <cls> cls/version/cls_version.cc:228: Loaded version class!
  985. -93> 2016-05-03 11:21:29.188068 7f3072eb2940 1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class!
  986. -92> 2016-05-03 11:21:29.188302 7f3072eb2940 1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class!
  987. -91> 2016-05-03 11:21:29.188396 7f3072eb2940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  988. -90> 2016-05-03 11:21:29.188510 7f3072eb2940 1 <cls> cls/user/cls_user.cc:367: Loaded user class!
  989. -89> 2016-05-03 11:21:29.188830 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  990. -88> 2016-05-03 11:21:29.188840 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  991. -87> 2016-05-03 11:21:29.188846 7f3072eb2940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  992. -86> 2016-05-03 11:21:29.284009 7f3072eb2940 0 osd.51 160621 load_pgs
  993. -85> 2016-05-03 11:21:29.292177 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.6(unlocked)] enter Initial
  994. -84> 2016-05-03 11:21:29.292477 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000299 0 0.000000
  995. -83> 2016-05-03 11:21:29.292501 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  996. -82> 2016-05-03 11:21:29.292586 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.1a(unlocked)] enter Initial
  997. -81> 2016-05-03 11:21:29.292782 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000195 0 0.000000
  998. -80> 2016-05-03 11:21:29.292796 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  999. -79> 2016-05-03 11:21:29.292880 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.22(unlocked)] enter Initial
  1000. -78> 2016-05-03 11:21:29.293044 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.000164 0 0.000000
  1001. -77> 2016-05-03 11:21:29.293058 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1002. -76> 2016-05-03 11:21:29.293477 7f3072eb2940 5 osd.51 pg_epoch: 160581 pg[1.36(unlocked)] enter Initial
  1003. -75> 2016-05-03 11:21:29.294746 7f3072eb2940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001268 0 0.000000
  1004. -74> 2016-05-03 11:21:29.294762 7f3072eb2940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1005. -73> 2016-05-03 11:21:29.295165 7f3072eb2940 5 osd.51 pg_epoch: 160599 pg[1.3f(unlocked)] enter Initial
  1006. -72> 2016-05-03 11:21:29.296457 7f3072eb2940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001292 0 0.000000
  1007. -71> 2016-05-03 11:21:29.296473 7f3072eb2940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1008. -70> 2016-05-03 11:21:29.296895 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.4(unlocked)] enter Initial
  1009. -69> 2016-05-03 11:21:29.319826 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.022930 0 0.000000
  1010. -68> 2016-05-03 11:21:29.319847 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1011. -67> 2016-05-03 11:21:29.319985 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.d(unlocked)] enter Initial
  1012. -66> 2016-05-03 11:21:29.339957 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019971 0 0.000000
  1013. -65> 2016-05-03 11:21:29.339977 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1014. -64> 2016-05-03 11:21:29.340133 7f3072eb2940 5 osd.51 pg_epoch: 160621 pg[2.13(unlocked)] enter Initial
  1015. -63> 2016-05-03 11:21:29.364794 7f3072eb2940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024661 0 0.000000
  1016. -62> 2016-05-03 11:21:29.364814 7f3072eb2940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1017. -61> 2016-05-03 11:21:29.364956 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.14(unlocked)] enter Initial
  1018. -60> 2016-05-03 11:21:29.385396 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020439 0 0.000000
  1019. -59> 2016-05-03 11:21:29.385416 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1020. -58> 2016-05-03 11:21:29.385546 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1b(unlocked)] enter Initial
  1021. -57> 2016-05-03 11:21:29.405048 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019502 0 0.000000
  1022. -56> 2016-05-03 11:21:29.405068 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1023. -55> 2016-05-03 11:21:29.405223 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.32(unlocked)] enter Initial
  1024. -54> 2016-05-03 11:21:29.461815 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] exit Initial 0.056591 0 0.000000
  1025. -53> 2016-05-03 11:21:29.461836 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] enter Reset
  1026. -52> 2016-05-03 11:21:29.461971 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.3b(unlocked)] enter Initial
  1027. -51> 2016-05-03 11:21:29.516973 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] exit Initial 0.055002 0 0.000000
  1028. -50> 2016-05-03 11:21:29.516993 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] enter Reset
  1029. -49> 2016-05-03 11:21:29.517181 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[2.48(unlocked)] enter Initial
  1030. -48> 2016-05-03 11:21:29.517352 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000171 0 0.000000
  1031. -47> 2016-05-03 11:21:29.517366 7f3072eb2940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1032. -46> 2016-05-03 11:21:29.517491 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.50(unlocked)] enter Initial
  1033. -45> 2016-05-03 11:21:29.540955 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.023463 0 0.000000
  1034. -44> 2016-05-03 11:21:29.540975 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1035. -43> 2016-05-03 11:21:29.541124 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.51(unlocked)] enter Initial
  1036. -42> 2016-05-03 11:21:29.562267 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021143 0 0.000000
  1037. -41> 2016-05-03 11:21:29.562287 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1038. -40> 2016-05-03 11:21:29.562429 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.72(unlocked)] enter Initial
  1039. -39> 2016-05-03 11:21:29.584215 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021786 0 0.000000
  1040. -38> 2016-05-03 11:21:29.584236 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1041. -37> 2016-05-03 11:21:29.584365 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.b7(unlocked)] enter Initial
  1042. -36> 2016-05-03 11:21:29.606815 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.022449 0 0.000000
  1043. -35> 2016-05-03 11:21:29.606835 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1044. -34> 2016-05-03 11:21:29.607482 7f3072eb2940 5 osd.51 pg_epoch: 160595 pg[2.cf(unlocked)] enter Initial
  1045. -33> 2016-05-03 11:21:29.636475 7f3072eb2940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.028993 0 0.000000
  1046. -32> 2016-05-03 11:21:29.636496 7f3072eb2940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1047. -31> 2016-05-03 11:21:29.636637 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.10e(unlocked)] enter Initial
  1048. -30> 2016-05-03 11:21:29.656415 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019778 0 0.000000
  1049. -29> 2016-05-03 11:21:29.656435 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1050. -28> 2016-05-03 11:21:29.656582 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.111(unlocked)] enter Initial
  1051. -27> 2016-05-03 11:21:29.725927 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] exit Initial 0.069345 0 0.000000
  1052. -26> 2016-05-03 11:21:29.725949 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] enter Reset
  1053. -25> 2016-05-03 11:21:29.726127 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.16f(unlocked)] enter Initial
  1054. -24> 2016-05-03 11:21:29.799774 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] exit Initial 0.073647 0 0.000000
  1055. -23> 2016-05-03 11:21:29.799794 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] enter Reset
  1056. -22> 2016-05-03 11:21:29.800416 7f3072eb2940 5 osd.51 pg_epoch: 160607 pg[2.186(unlocked)] enter Initial
  1057. -21> 2016-05-03 11:21:29.826134 7f3072eb2940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025717 0 0.000000
  1058. -20> 2016-05-03 11:21:29.826153 7f3072eb2940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1059. -19> 2016-05-03 11:21:29.826293 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.188(unlocked)] enter Initial
  1060. -18> 2016-05-03 11:21:29.852037 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.025743 0 0.000000
  1061. -17> 2016-05-03 11:21:29.852057 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1062. -16> 2016-05-03 11:21:29.852242 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1a2(unlocked)] enter Initial
  1063. -15> 2016-05-03 11:21:29.872355 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020113 0 0.000000
  1064. -14> 2016-05-03 11:21:29.872374 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1065. -13> 2016-05-03 11:21:29.872503 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1aa(unlocked)] enter Initial
  1066. -12> 2016-05-03 11:21:29.896994 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024491 0 0.000000
  1067. -11> 2016-05-03 11:21:29.897014 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1068. -10> 2016-05-03 11:21:29.897153 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1bb(unlocked)] enter Initial
  1069. -9> 2016-05-03 11:21:29.922303 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025149 0 0.000000
  1070. -8> 2016-05-03 11:21:29.922322 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1071. -7> 2016-05-03 11:21:29.922460 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1e8(unlocked)] enter Initial
  1072. -6> 2016-05-03 11:21:29.954554 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.032094 0 0.000000
  1073. -5> 2016-05-03 11:21:29.954584 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1074. -4> 2016-05-03 11:21:29.954720 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1f2(unlocked)] enter Initial
  1075. -3> 2016-05-03 11:21:29.982947 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.028226 0 0.000000
  1076. -2> 2016-05-03 11:21:29.982966 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1077. -1> 2016-05-03 11:21:29.983109 7f3072eb2940 5 osd.51 pg_epoch: 160608 pg[2.1fd(unlocked)] enter Initial
  1078. 0> 2016-05-03 11:21:30.051924 7f3072eb2940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7f3072eb2940 time 2016-05-03 11:21:30.049490
  1079. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  1080.  
  1081. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1082. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f30729a87eb]
  1083. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7f30724e2e8d]
  1084. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7f307255f55a]
  1085. 4: (OSD::load_pgs()+0x61d) [0x7f307241d58d]
  1086. 5: (OSD::init()+0xe74) [0x7f307242d694]
  1087. 6: (main()+0x2954) [0x7f30723b2474]
  1088. 7: (__libc_start_main()+0xf5) [0x7f306f234ec5]
  1089. 8: (()+0x2f82b7) [0x7f30723e22b7]
  1090. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1091.  
  1092. --- logging levels ---
  1093. 0/ 5 none
  1094. 0/ 1 lockdep
  1095. 0/ 1 context
  1096. 1/ 1 crush
  1097. 1/ 5 mds
  1098. 1/ 5 mds_balancer
  1099. 1/ 5 mds_locker
  1100. 1/ 5 mds_log
  1101. 1/ 5 mds_log_expire
  1102. 1/ 5 mds_migrator
  1103. 0/ 1 buffer
  1104. 0/ 1 timer
  1105. 0/ 1 filer
  1106. 0/ 1 striper
  1107. 0/ 1 objecter
  1108. 0/ 5 rados
  1109. 0/ 5 rbd
  1110. 0/ 5 rbd_replay
  1111. 0/ 5 journaler
  1112. 0/ 5 objectcacher
  1113. 0/ 5 client
  1114. 0/ 5 osd
  1115. 0/ 5 optracker
  1116. 0/ 5 objclass
  1117. 1/ 3 filestore
  1118. 1/ 3 keyvaluestore
  1119. 1/ 3 journal
  1120. 0/ 5 ms
  1121. 1/ 5 mon
  1122. 0/10 monc
  1123. 1/ 5 paxos
  1124. 0/ 5 tp
  1125. 1/ 5 auth
  1126. 1/ 5 crypto
  1127. 1/ 1 finisher
  1128. 1/ 5 heartbeatmap
  1129. 1/ 5 perfcounter
  1130. 1/ 5 rgw
  1131. 1/10 civetweb
  1132. 1/ 5 javaclient
  1133. 1/ 5 asok
  1134. 1/ 1 throttle
  1135. 0/ 0 refs
  1136. 1/ 5 xio
  1137. 1/ 5 compressor
  1138. 1/ 5 newstore
  1139. -2/-2 (syslog threshold)
  1140. -1/-1 (stderr threshold)
  1141. max_recent 10000
  1142. max_new 1000
  1143. log_file /var/log/ceph/ceph-osd.51.log
  1144. --- end dump of recent events ---
  1145. 2016-05-03 11:21:30.056560 7f3072eb2940 -1 *** Caught signal (Aborted) **
  1146. in thread 7f3072eb2940
  1147.  
  1148. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1149. 1: (()+0x7d1aca) [0x7f30728bbaca]
  1150. 2: (()+0x10340) [0x7f3070fce340]
  1151. 3: (gsignal()+0x39) [0x7f306f249cc9]
  1152. 4: (abort()+0x148) [0x7f306f24d0d8]
  1153. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f306fb54535]
  1154. 6: (()+0x5e6d6) [0x7f306fb526d6]
  1155. 7: (()+0x5e703) [0x7f306fb52703]
  1156. 8: (()+0x5e922) [0x7f306fb52922]
  1157. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7f30729a89d8]
  1158. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7f30724e2e8d]
  1159. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7f307255f55a]
  1160. 12: (OSD::load_pgs()+0x61d) [0x7f307241d58d]
  1161. 13: (OSD::init()+0xe74) [0x7f307242d694]
  1162. 14: (main()+0x2954) [0x7f30723b2474]
  1163. 15: (__libc_start_main()+0xf5) [0x7f306f234ec5]
  1164. 16: (()+0x2f82b7) [0x7f30723e22b7]
  1165. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1166.  
  1167. --- begin dump of recent events ---
  1168. 0> 2016-05-03 11:21:30.056560 7f3072eb2940 -1 *** Caught signal (Aborted) **
  1169. in thread 7f3072eb2940
  1170.  
  1171. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1172. 1: (()+0x7d1aca) [0x7f30728bbaca]
  1173. 2: (()+0x10340) [0x7f3070fce340]
  1174. 3: (gsignal()+0x39) [0x7f306f249cc9]
  1175. 4: (abort()+0x148) [0x7f306f24d0d8]
  1176. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f306fb54535]
  1177. 6: (()+0x5e6d6) [0x7f306fb526d6]
  1178. 7: (()+0x5e703) [0x7f306fb52703]
  1179. 8: (()+0x5e922) [0x7f306fb52922]
  1180. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7f30729a89d8]
  1181. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7f30724e2e8d]
  1182. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7f307255f55a]
  1183. 12: (OSD::load_pgs()+0x61d) [0x7f307241d58d]
  1184. 13: (OSD::init()+0xe74) [0x7f307242d694]
  1185. 14: (main()+0x2954) [0x7f30723b2474]
  1186. 15: (__libc_start_main()+0xf5) [0x7f306f234ec5]
  1187. 16: (()+0x2f82b7) [0x7f30723e22b7]
  1188. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1189.  
  1190. --- logging levels ---
  1191. 0/ 5 none
  1192. 0/ 1 lockdep
  1193. 0/ 1 context
  1194. 1/ 1 crush
  1195. 1/ 5 mds
  1196. 1/ 5 mds_balancer
  1197. 1/ 5 mds_locker
  1198. 1/ 5 mds_log
  1199. 1/ 5 mds_log_expire
  1200. 1/ 5 mds_migrator
  1201. 0/ 1 buffer
  1202. 0/ 1 timer
  1203. 0/ 1 filer
  1204. 0/ 1 striper
  1205. 0/ 1 objecter
  1206. 0/ 5 rados
  1207. 0/ 5 rbd
  1208. 0/ 5 rbd_replay
  1209. 0/ 5 journaler
  1210. 0/ 5 objectcacher
  1211. 0/ 5 client
  1212. 0/ 5 osd
  1213. 0/ 5 optracker
  1214. 0/ 5 objclass
  1215. 1/ 3 filestore
  1216. 1/ 3 keyvaluestore
  1217. 1/ 3 journal
  1218. 0/ 5 ms
  1219. 1/ 5 mon
  1220. 0/10 monc
  1221. 1/ 5 paxos
  1222. 0/ 5 tp
  1223. 1/ 5 auth
  1224. 1/ 5 crypto
  1225. 1/ 1 finisher
  1226. 1/ 5 heartbeatmap
  1227. 1/ 5 perfcounter
  1228. 1/ 5 rgw
  1229. 1/10 civetweb
  1230. 1/ 5 javaclient
  1231. 1/ 5 asok
  1232. 1/ 1 throttle
  1233. 0/ 0 refs
  1234. 1/ 5 xio
  1235. 1/ 5 compressor
  1236. 1/ 5 newstore
  1237. -2/-2 (syslog threshold)
  1238. -1/-1 (stderr threshold)
  1239. max_recent 10000
  1240. max_new 1000
  1241. log_file /var/log/ceph/ceph-osd.51.log
  1242. --- end dump of recent events ---
  1243. 2016-05-03 11:21:30.590193 7ff588731940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14955
  1244. 2016-05-03 11:21:30.602832 7ff588731940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  1245. 2016-05-03 11:21:30.603279 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  1246. 2016-05-03 11:21:30.603287 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  1247. 2016-05-03 11:21:30.603311 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  1248. 2016-05-03 11:21:30.624153 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  1249. 2016-05-03 11:21:30.624241 7ff588731940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  1250. 2016-05-03 11:21:30.692122 7ff588731940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  1251. 2016-05-03 11:21:30.695019 7ff588731940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  1252. 2016-05-03 11:21:30.741961 7ff588731940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  1253. 2016-05-03 11:21:30.743520 7ff588731940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  1254. 2016-05-03 11:21:30.747950 7ff588731940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  1255. 2016-05-03 11:21:30.752236 7ff588731940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  1256. 2016-05-03 11:21:30.752747 7ff588731940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  1257. 2016-05-03 11:21:30.752760 7ff588731940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  1258. 2016-05-03 11:21:30.752767 7ff588731940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  1259. 2016-05-03 11:21:30.845415 7ff588731940 0 osd.51 160621 load_pgs
  1260. 2016-05-03 11:21:31.639208 7ff588731940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7ff588731940 time 2016-05-03 11:21:31.636815
  1261. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  1262.  
  1263. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1264. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7ff5882277eb]
  1265. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7ff587d61e8d]
  1266. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7ff587dde55a]
  1267. 4: (OSD::load_pgs()+0x61d) [0x7ff587c9c58d]
  1268. 5: (OSD::init()+0xe74) [0x7ff587cac694]
  1269. 6: (main()+0x2954) [0x7ff587c31474]
  1270. 7: (__libc_start_main()+0xf5) [0x7ff584ab3ec5]
  1271. 8: (()+0x2f82b7) [0x7ff587c612b7]
  1272. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1273.  
  1274. --- begin dump of recent events ---
  1275. -217> 2016-05-03 11:21:30.587109 7ff588731940 5 asok(0x7ff58c26e000) register_command perfcounters_dump hook 0x7ff58c1ee050
  1276. -216> 2016-05-03 11:21:30.587150 7ff588731940 5 asok(0x7ff58c26e000) register_command 1 hook 0x7ff58c1ee050
  1277. -215> 2016-05-03 11:21:30.587156 7ff588731940 5 asok(0x7ff58c26e000) register_command perf dump hook 0x7ff58c1ee050
  1278. -214> 2016-05-03 11:21:30.587160 7ff588731940 5 asok(0x7ff58c26e000) register_command perfcounters_schema hook 0x7ff58c1ee050
  1279. -213> 2016-05-03 11:21:30.587166 7ff588731940 5 asok(0x7ff58c26e000) register_command 2 hook 0x7ff58c1ee050
  1280. -212> 2016-05-03 11:21:30.587169 7ff588731940 5 asok(0x7ff58c26e000) register_command perf schema hook 0x7ff58c1ee050
  1281. -211> 2016-05-03 11:21:30.587172 7ff588731940 5 asok(0x7ff58c26e000) register_command perf reset hook 0x7ff58c1ee050
  1282. -210> 2016-05-03 11:21:30.587178 7ff588731940 5 asok(0x7ff58c26e000) register_command config show hook 0x7ff58c1ee050
  1283. -209> 2016-05-03 11:21:30.587181 7ff588731940 5 asok(0x7ff58c26e000) register_command config set hook 0x7ff58c1ee050
  1284. -208> 2016-05-03 11:21:30.587184 7ff588731940 5 asok(0x7ff58c26e000) register_command config get hook 0x7ff58c1ee050
  1285. -207> 2016-05-03 11:21:30.587190 7ff588731940 5 asok(0x7ff58c26e000) register_command config diff hook 0x7ff58c1ee050
  1286. -206> 2016-05-03 11:21:30.587193 7ff588731940 5 asok(0x7ff58c26e000) register_command log flush hook 0x7ff58c1ee050
  1287. -205> 2016-05-03 11:21:30.587196 7ff588731940 5 asok(0x7ff58c26e000) register_command log dump hook 0x7ff58c1ee050
  1288. -204> 2016-05-03 11:21:30.587200 7ff588731940 5 asok(0x7ff58c26e000) register_command log reopen hook 0x7ff58c1ee050
  1289. -203> 2016-05-03 11:21:30.590091 7ff588731940 0 set uid:gid to 64045:64045
  1290. -202> 2016-05-03 11:21:30.590193 7ff588731940 0 ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd), process ceph-osd, pid 14955
  1291. -201> 2016-05-03 11:21:30.590744 7ff588731940 1 accepter.accepter.bind my_inst.addr is [::]:6808/14955 need_addr=1
  1292. -200> 2016-05-03 11:21:30.590767 7ff588731940 1 accepter.accepter.bind my_inst.addr is [::]:6809/14955 need_addr=1
  1293. -199> 2016-05-03 11:21:30.590784 7ff588731940 1 accepter.accepter.bind my_inst.addr is [::]:6810/14955 need_addr=1
  1294. -198> 2016-05-03 11:21:30.590806 7ff588731940 1 accepter.accepter.bind my_inst.addr is [::]:6811/14955 need_addr=1
  1295. -197> 2016-05-03 11:21:30.592664 7ff588731940 5 asok(0x7ff58c26e000) init /var/run/ceph/ceph-osd.51.asok
  1296. -196> 2016-05-03 11:21:30.592676 7ff588731940 5 asok(0x7ff58c26e000) bind_and_listen /var/run/ceph/ceph-osd.51.asok
  1297. -195> 2016-05-03 11:21:30.592777 7ff588731940 5 asok(0x7ff58c26e000) register_command 0 hook 0x7ff58c1ea0b0
  1298. -194> 2016-05-03 11:21:30.592786 7ff588731940 5 asok(0x7ff58c26e000) register_command version hook 0x7ff58c1ea0b0
  1299. -193> 2016-05-03 11:21:30.592791 7ff588731940 5 asok(0x7ff58c26e000) register_command git_version hook 0x7ff58c1ea0b0
  1300. -192> 2016-05-03 11:21:30.592796 7ff588731940 5 asok(0x7ff58c26e000) register_command help hook 0x7ff58c1ee230
  1301. -191> 2016-05-03 11:21:30.592801 7ff588731940 5 asok(0x7ff58c26e000) register_command get_command_descriptions hook 0x7ff58c1ee220
  1302. -190> 2016-05-03 11:21:30.592880 7ff588731940 10 monclient(hunting): build_initial_monmap
  1303. -189> 2016-05-03 11:21:30.592892 7ff581a41700 5 asok(0x7ff58c26e000) entry start
  1304. -188> 2016-05-03 11:21:30.602383 7ff588731940 5 adding auth protocol: cephx
  1305. -187> 2016-05-03 11:21:30.602391 7ff588731940 5 adding auth protocol: cephx
  1306. -186> 2016-05-03 11:21:30.602494 7ff588731940 5 asok(0x7ff58c26e000) register_command objecter_requests hook 0x7ff58c1ee270
  1307. -185> 2016-05-03 11:21:30.602551 7ff588731940 1 -- [::]:6808/14955 messenger.start
  1308. -184> 2016-05-03 11:21:30.602589 7ff588731940 1 -- :/0 messenger.start
  1309. -183> 2016-05-03 11:21:30.602612 7ff588731940 1 -- [::]:6811/14955 messenger.start
  1310. -182> 2016-05-03 11:21:30.602634 7ff588731940 1 -- [::]:6810/14955 messenger.start
  1311. -181> 2016-05-03 11:21:30.602655 7ff588731940 1 -- [::]:6809/14955 messenger.start
  1312. -180> 2016-05-03 11:21:30.602678 7ff588731940 1 -- :/0 messenger.start
  1313. -179> 2016-05-03 11:21:30.602753 7ff588731940 2 osd.51 0 mounting /var/lib/ceph/osd/ceph-51 /var/lib/ceph/osd/ceph-51/journal
  1314. -178> 2016-05-03 11:21:30.602832 7ff588731940 0 filestore(/var/lib/ceph/osd/ceph-51) backend xfs (magic 0x58465342)
  1315. -177> 2016-05-03 11:21:30.603279 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
  1316. -176> 2016-05-03 11:21:30.603287 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
  1317. -175> 2016-05-03 11:21:30.603311 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: splice is supported
  1318. -174> 2016-05-03 11:21:30.624153 7ff588731940 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
  1319. -173> 2016-05-03 11:21:30.624241 7ff588731940 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-51) detect_features: extsize is supported and your kernel >= 3.5
  1320. -172> 2016-05-03 11:21:30.692122 7ff588731940 0 filestore(/var/lib/ceph/osd/ceph-51) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
  1321. -171> 2016-05-03 11:21:30.694928 7ff588731940 2 journal open /var/lib/ceph/osd/ceph-51/journal fsid 3ca75ccf-1c42-4282-9504-c07f2cee91b3 fs_op_seq 21461862
  1322. -170> 2016-05-03 11:21:30.695019 7ff588731940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  1323. -169> 2016-05-03 11:21:30.706710 7ff588731940 2 journal open advancing committed_seq 21461799 to fs op_seq 21461862
  1324. -168> 2016-05-03 11:21:30.715541 7ff588731940 2 journal read_entry 256061440 : seq 21461800 3021 bytes
  1325. -167> 2016-05-03 11:21:30.715796 7ff588731940 2 journal read_entry 256065536 : seq 21461801 2981 bytes
  1326. -166> 2016-05-03 11:21:30.715899 7ff588731940 2 journal read_entry 256069632 : seq 21461802 2864 bytes
  1327. -165> 2016-05-03 11:21:30.715931 7ff588731940 2 journal read_entry 256073728 : seq 21461803 2788 bytes
  1328. -164> 2016-05-03 11:21:30.715954 7ff588731940 2 journal read_entry 256077824 : seq 21461804 3252 bytes
  1329. -163> 2016-05-03 11:21:30.716103 7ff588731940 2 journal read_entry 256081920 : seq 21461805 3072 bytes
  1330. -162> 2016-05-03 11:21:30.716117 7ff588731940 2 journal read_entry 256086016 : seq 21461806 3136 bytes
  1331. -161> 2016-05-03 11:21:30.716130 7ff588731940 2 journal read_entry 256090112 : seq 21461807 2178 bytes
  1332. -160> 2016-05-03 11:21:30.716140 7ff588731940 2 journal read_entry 256094208 : seq 21461808 2146 bytes
  1333. -159> 2016-05-03 11:21:30.716151 7ff588731940 2 journal read_entry 256098304 : seq 21461809 2417 bytes
  1334. -158> 2016-05-03 11:21:30.716162 7ff588731940 2 journal read_entry 256102400 : seq 21461810 2291 bytes
  1335. -157> 2016-05-03 11:21:30.716173 7ff588731940 2 journal read_entry 256106496 : seq 21461811 2417 bytes
  1336. -156> 2016-05-03 11:21:30.716183 7ff588731940 2 journal read_entry 256110592 : seq 21461812 2417 bytes
  1337. -155> 2016-05-03 11:21:30.716478 7ff588731940 2 journal read_entry 256114688 : seq 21461813 2499 bytes
  1338. -154> 2016-05-03 11:21:30.716493 7ff588731940 2 journal read_entry 256118784 : seq 21461814 3140 bytes
  1339. -153> 2016-05-03 11:21:30.716504 7ff588731940 2 journal read_entry 256122880 : seq 21461815 2385 bytes
  1340. -152> 2016-05-03 11:21:30.716517 7ff588731940 2 journal read_entry 256126976 : seq 21461816 2385 bytes
  1341. -151> 2016-05-03 11:21:30.716528 7ff588731940 2 journal read_entry 256131072 : seq 21461817 2483 bytes
  1342. -150> 2016-05-03 11:21:30.716540 7ff588731940 2 journal read_entry 256135168 : seq 21461818 2275 bytes
  1343. -149> 2016-05-03 11:21:30.716551 7ff588731940 2 journal read_entry 256139264 : seq 21461819 2889 bytes
  1344. -148> 2016-05-03 11:21:30.716562 7ff588731940 2 journal read_entry 256143360 : seq 21461820 3144 bytes
  1345. -147> 2016-05-03 11:21:30.716572 7ff588731940 2 journal read_entry 256147456 : seq 21461821 3156 bytes
  1346. -146> 2016-05-03 11:21:30.716583 7ff588731940 2 journal read_entry 256151552 : seq 21461822 2146 bytes
  1347. -145> 2016-05-03 11:21:30.716594 7ff588731940 2 journal read_entry 256155648 : seq 21461823 2889 bytes
  1348. -144> 2016-05-03 11:21:30.716604 7ff588731940 2 journal read_entry 256159744 : seq 21461824 2412 bytes
  1349. -143> 2016-05-03 11:21:30.716615 7ff588731940 2 journal read_entry 256163840 : seq 21461825 2291 bytes
  1350. -142> 2016-05-03 11:21:30.716626 7ff588731940 2 journal read_entry 256167936 : seq 21461826 3446 bytes
  1351. -141> 2016-05-03 11:21:30.716636 7ff588731940 2 journal read_entry 256172032 : seq 21461827 3201 bytes
  1352. -140> 2016-05-03 11:21:30.716647 7ff588731940 2 journal read_entry 256176128 : seq 21461828 2417 bytes
  1353. -139> 2016-05-03 11:21:30.716974 7ff588731940 2 journal read_entry 256180224 : seq 21461829 3050 bytes
  1354. -138> 2016-05-03 11:21:30.716991 7ff588731940 2 journal read_entry 256184320 : seq 21461830 2146 bytes
  1355. -137> 2016-05-03 11:21:30.717002 7ff588731940 2 journal read_entry 256188416 : seq 21461831 3012 bytes
  1356. -136> 2016-05-03 11:21:30.717013 7ff588731940 2 journal read_entry 256192512 : seq 21461832 3160 bytes
  1357. -135> 2016-05-03 11:21:30.717024 7ff588731940 2 journal read_entry 256196608 : seq 21461833 2781 bytes
  1358. -134> 2016-05-03 11:21:30.717034 7ff588731940 2 journal read_entry 256200704 : seq 21461834 2385 bytes
  1359. -133> 2016-05-03 11:21:30.717045 7ff588731940 2 journal read_entry 256204800 : seq 21461835 2385 bytes
  1360. -132> 2016-05-03 11:21:30.717055 7ff588731940 2 journal read_entry 256208896 : seq 21461836 2389 bytes
  1361. -131> 2016-05-03 11:21:30.717065 7ff588731940 2 journal read_entry 256212992 : seq 21461837 2385 bytes
  1362. -130> 2016-05-03 11:21:30.717075 7ff588731940 2 journal read_entry 256217088 : seq 21461838 2275 bytes
  1363. -129> 2016-05-03 11:21:30.717085 7ff588731940 2 journal read_entry 256221184 : seq 21461839 2417 bytes
  1364. -128> 2016-05-03 11:21:30.717096 7ff588731940 2 journal read_entry 256225280 : seq 21461840 2385 bytes
  1365. -127> 2016-05-03 11:21:30.717106 7ff588731940 2 journal read_entry 256229376 : seq 21461841 2417 bytes
  1366. -126> 2016-05-03 11:21:30.717118 7ff588731940 2 journal read_entry 256233472 : seq 21461842 2990 bytes
  1367. -125> 2016-05-03 11:21:30.717128 7ff588731940 2 journal read_entry 256237568 : seq 21461843 3028 bytes
  1368. -124> 2016-05-03 11:21:30.717140 7ff588731940 2 journal read_entry 256241664 : seq 21461844 2889 bytes
  1369. -123> 2016-05-03 11:21:30.717151 7ff588731940 2 journal read_entry 256245760 : seq 21461845 2990 bytes
  1370. -122> 2016-05-03 11:21:30.717161 7ff588731940 2 journal read_entry 256249856 : seq 21461846 2788 bytes
  1371. -121> 2016-05-03 11:21:30.717172 7ff588731940 2 journal read_entry 256253952 : seq 21461847 2990 bytes
  1372. -120> 2016-05-03 11:21:30.717182 7ff588731940 2 journal read_entry 256258048 : seq 21461848 3036 bytes
  1373. -119> 2016-05-03 11:21:30.717193 7ff588731940 2 journal read_entry 256262144 : seq 21461849 2788 bytes
  1374. -118> 2016-05-03 11:21:30.717203 7ff588731940 2 journal read_entry 256266240 : seq 21461850 2904 bytes
  1375. -117> 2016-05-03 11:21:30.717261 7ff588731940 2 journal read_entry 256270336 : seq 21461851 2840 bytes
  1376. -116> 2016-05-03 11:21:30.717274 7ff588731940 2 journal read_entry 256274432 : seq 21461852 2990 bytes
  1377. -115> 2016-05-03 11:21:30.717285 7ff588731940 2 journal read_entry 256278528 : seq 21461853 2856 bytes
  1378. -114> 2016-05-03 11:21:30.736227 7ff588731940 2 journal read_entry 256282624 : seq 21461854 4196824 bytes
  1379. -113> 2016-05-03 11:21:30.736259 7ff588731940 2 journal read_entry 260485120 : seq 21461855 2840 bytes
  1380. -112> 2016-05-03 11:21:30.736271 7ff588731940 2 journal read_entry 260489216 : seq 21461856 2990 bytes
  1381. -111> 2016-05-03 11:21:30.738901 7ff588731940 2 journal read_entry 260493312 : seq 21461857 869132 bytes
  1382. -110> 2016-05-03 11:21:30.738927 7ff588731940 2 journal read_entry 261365760 : seq 21461858 7957 bytes
  1383. -109> 2016-05-03 11:21:30.738937 7ff588731940 2 journal read_entry 261373952 : seq 21461859 2438 bytes
  1384. -108> 2016-05-03 11:21:30.739008 7ff588731940 2 journal read_entry 261378048 : seq 21461860 2990 bytes
  1385. -107> 2016-05-03 11:21:30.739021 7ff588731940 2 journal read_entry 261382144 : seq 21461861 2856 bytes
  1386. -106> 2016-05-03 11:21:30.739033 7ff588731940 2 journal read_entry 261386240 : seq 21461862 2964 bytes
  1387. -105> 2016-05-03 11:21:30.739045 7ff588731940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  1388. -104> 2016-05-03 11:21:30.739056 7ff588731940 2 journal read_entry 261390336 : seq 21418404 7161 bytes
  1389. -103> 2016-05-03 11:21:30.739061 7ff588731940 3 journal journal_replay: end of journal, done.
  1390. -102> 2016-05-03 11:21:30.741961 7ff588731940 1 journal _open /var/lib/ceph/osd/ceph-51/journal fd 19: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1
  1391. -101> 2016-05-03 11:21:30.743520 7ff588731940 1 filestore(/var/lib/ceph/osd/ceph-51) upgrade
  1392. -100> 2016-05-03 11:21:30.743550 7ff588731940 2 osd.51 0 boot
  1393. -99> 2016-05-03 11:21:30.744357 7ff588731940 1 <cls> cls/timeindex/cls_timeindex.cc:259: Loaded timeindex class!
  1394. -98> 2016-05-03 11:21:30.744533 7ff588731940 1 <cls> cls/log/cls_log.cc:313: Loaded log class!
  1395. -97> 2016-05-03 11:21:30.747740 7ff588731940 1 <cls> cls/refcount/cls_refcount.cc:232: Loaded refcount class!
  1396. -96> 2016-05-03 11:21:30.747950 7ff588731940 0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
  1397. -95> 2016-05-03 11:21:30.751115 7ff588731940 1 <cls> cls/rgw/cls_rgw.cc:3059: Loaded rgw class!
  1398. -94> 2016-05-03 11:21:30.751281 7ff588731940 1 <cls> cls/version/cls_version.cc:228: Loaded version class!
  1399. -93> 2016-05-03 11:21:30.751794 7ff588731940 1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class!
  1400. -92> 2016-05-03 11:21:30.752103 7ff588731940 1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class!
  1401. -91> 2016-05-03 11:21:30.752236 7ff588731940 0 <cls> cls/cephfs/cls_cephfs.cc:136: loading cephfs_size_scan
  1402. -90> 2016-05-03 11:21:30.752380 7ff588731940 1 <cls> cls/user/cls_user.cc:367: Loaded user class!
  1403. -89> 2016-05-03 11:21:30.752747 7ff588731940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for clients
  1404. -88> 2016-05-03 11:21:30.752760 7ff588731940 0 osd.51 160621 crush map has features 283675107524608 was 8705, adjusting msgr requires for mons
  1405. -87> 2016-05-03 11:21:30.752767 7ff588731940 0 osd.51 160621 crush map has features 283675107524608, adjusting msgr requires for osds
  1406. -86> 2016-05-03 11:21:30.845415 7ff588731940 0 osd.51 160621 load_pgs
  1407. -85> 2016-05-03 11:21:30.853291 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.6(unlocked)] enter Initial
  1408. -84> 2016-05-03 11:21:30.853580 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000289 0 0.000000
  1409. -83> 2016-05-03 11:21:30.853604 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.6( empty local-les=160570 n=0 ec=1 les/c/f 160570/160580/154843 160563/160563/119449) [89,51,6] r=1 lpr=0 pi=119434-160562/118 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1410. -82> 2016-05-03 11:21:30.853686 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.1a(unlocked)] enter Initial
  1411. -81> 2016-05-03 11:21:30.853878 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000191 0 0.000000
  1412. -80> 2016-05-03 11:21:30.853892 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.1a( empty local-les=160564 n=0 ec=1 les/c/f 160564/160580/154843 160563/160563/160069) [46,51,22] r=1 lpr=0 pi=160060-160562/10 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1413. -79> 2016-05-03 11:21:30.853971 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.22(unlocked)] enter Initial
  1414. -78> 2016-05-03 11:21:30.854166 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.000195 0 0.000000
  1415. -77> 2016-05-03 11:21:30.854180 7ff588731940 5 osd.51 pg_epoch: 160580 pg[0.22( empty local-les=160580 n=0 ec=1 les/c/f 160580/160580/154843 160563/160563/160563) [51,42,69] r=0 lpr=0 crt=0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1416. -76> 2016-05-03 11:21:30.854581 7ff588731940 5 osd.51 pg_epoch: 160581 pg[1.36(unlocked)] enter Initial
  1417. -75> 2016-05-03 11:21:30.855807 7ff588731940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001226 0 0.000000
  1418. -74> 2016-05-03 11:21:30.855824 7ff588731940 5 osd.51 pg_epoch: 160581 pg[1.36( empty local-les=160580 n=0 ec=1 les/c/f 160580/160581/154843 160563/160563/155733) [13,87,51] r=2 lpr=0 pi=119482-160562/112 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1419. -73> 2016-05-03 11:21:30.856212 7ff588731940 5 osd.51 pg_epoch: 160599 pg[1.3f(unlocked)] enter Initial
  1420. -72> 2016-05-03 11:21:30.857473 7ff588731940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.001260 0 0.000000
  1421. -71> 2016-05-03 11:21:30.857490 7ff588731940 5 osd.51 pg_epoch: 160599 pg[1.3f( empty local-les=160584 n=0 ec=1 les/c/f 160584/160599/154843 160563/160563/119493) [22,34,51] r=2 lpr=0 pi=81477-160562/300 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1422. -70> 2016-05-03 11:21:30.857901 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.4(unlocked)] enter Initial
  1423. -69> 2016-05-03 11:21:30.880859 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.022958 0 0.000000
  1424. -68> 2016-05-03 11:21:30.880881 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.4( v 160608'617974 (158056'614808,160608'617974] local-les=160580 n=4189 ec=1 les/c/f 160580/160589/154843 160563/160563/159458) [26,51] r=1 lpr=0 pi=159447-160562/16 crt=160607'617968 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1425. -67> 2016-05-03 11:21:30.881004 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.d(unlocked)] enter Initial
  1426. -66> 2016-05-03 11:21:30.900629 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019624 0 0.000000
  1427. -65> 2016-05-03 11:21:30.900650 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.d( v 160608'529767 (157781'526547,160608'529767] local-les=160581 n=4201 ec=1 les/c/f 160581/160606/154843 160563/160573/160573) [51,32] r=0 lpr=0 crt=160607'529761 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1428. -64> 2016-05-03 11:21:30.900808 7ff588731940 5 osd.51 pg_epoch: 160621 pg[2.13(unlocked)] enter Initial
  1429. -63> 2016-05-03 11:21:30.925371 7ff588731940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024563 0 0.000000
  1430. -62> 2016-05-03 11:21:30.925391 7ff588731940 5 osd.51 pg_epoch: 160621 pg[2.13( v 160552'601389 (158092'597436,160552'601389] local-les=160580 n=4479 ec=1 les/c/f 160580/160554/154843 160621/160621/160621) [13] r=-1 lpr=0 pi=160476-160620/4 crt=160552'601387 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1431. -61> 2016-05-03 11:21:30.925528 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.14(unlocked)] enter Initial
  1432. -60> 2016-05-03 11:21:30.945701 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.020173 0 0.000000
  1433. -59> 2016-05-03 11:21:30.945722 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.14( v 160552'629248 (158932'625969,160552'629248] local-les=160581 n=4458 ec=1 les/c/f 160581/160477/154843 160563/160573/160573) [51,21] r=0 lpr=0 pi=160476-160572/5 crt=160551'629240 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1434. -58> 2016-05-03 11:21:30.945854 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1b(unlocked)] enter Initial
  1435. -57> 2016-05-03 11:21:30.965290 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019435 0 0.000000
  1436. -56> 2016-05-03 11:21:30.965311 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1b( v 160552'570218 (158741'567009,160552'570218] local-les=160581 n=4443 ec=1 les/c/f 160581/160477/154843 160563/160568/160568) [51,67] r=0 lpr=0 pi=160476-160567/5 crt=160552'570212 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1437. -55> 2016-05-03 11:21:30.965460 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.32(unlocked)] enter Initial
  1438. -54> 2016-05-03 11:21:31.021980 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] exit Initial 0.056519 0 0.000000
  1439. -53> 2016-05-03 11:21:31.022001 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.32( v 160551'558935 lc 0'0 (158615'555710,160551'558935] local-les=160582 n=4223 ec=1 les/c/f 160582/160477/154843 160563/160581/160581) [51,16] r=0 lpr=0 pi=160476-160580/7 crt=160551'558933 mlcod 0'0 inactive NIBBLEWISE m=4] enter Reset
  1440. -52> 2016-05-03 11:21:31.022145 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.3b(unlocked)] enter Initial
  1441. -51> 2016-05-03 11:21:31.076826 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] exit Initial 0.054681 0 0.000000
  1442. -50> 2016-05-03 11:21:31.076846 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.3b( v 160552'614978 lc 0'0 (158162'611752,160552'614978] local-les=160583 n=4461 ec=1 les/c/f 160583/160554/154843 160563/160581/160581) [51,14] r=0 lpr=0 pi=160476-160580/5 crt=160552'614976 mlcod 0'0 inactive NIBBLEWISE m=7] enter Reset
  1443. -49> 2016-05-03 11:21:31.077037 7ff588731940 5 osd.51 pg_epoch: 160580 pg[2.48(unlocked)] enter Initial
  1444. -48> 2016-05-03 11:21:31.077207 7ff588731940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.000170 0 0.000000
  1445. -47> 2016-05-03 11:21:31.077222 7ff588731940 5 osd.51 pg_epoch: 160580 pg[2.48( empty local-les=0 n=0 ec=1 les/c/f 160554/160554/154843 160567/160567/160567) [4,30] r=-1 lpr=0 pi=160553-160566/3 crt=0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1446. -46> 2016-05-03 11:21:31.077341 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.50(unlocked)] enter Initial
  1447. -45> 2016-05-03 11:21:31.101234 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.023893 0 0.000000
  1448. -44> 2016-05-03 11:21:31.101264 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.50( v 160608'624998 (157929'621625,160608'624998] local-les=160580 n=4276 ec=1 les/c/f 160580/160580/154843 160563/160566/159454) [34,51] r=1 lpr=0 pi=159447-160565/18 crt=160607'624992 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1449. -43> 2016-05-03 11:21:31.101447 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.51(unlocked)] enter Initial
  1450. -42> 2016-05-03 11:21:31.122704 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021257 0 0.000000
  1451. -41> 2016-05-03 11:21:31.122724 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.51( v 160608'528070 (157805'524720,160608'528070] local-les=160580 n=4282 ec=1 les/c/f 160580/160582/154843 160563/160563/160563) [51,66] r=0 lpr=0 crt=160606'528064 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1452. -40> 2016-05-03 11:21:31.122874 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.72(unlocked)] enter Initial
  1453. -39> 2016-05-03 11:21:31.144665 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.021790 0 0.000000
  1454. -38> 2016-05-03 11:21:31.144686 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.72( v 160552'516300 (158103'513081,160552'516300] local-les=160581 n=4188 ec=1 les/c/f 160581/160554/154843 160563/160564/160564) [51,47] r=0 lpr=0 pi=160476-160563/5 crt=160552'516298 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1455. -37> 2016-05-03 11:21:31.144821 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.b7(unlocked)] enter Initial
  1456. -36> 2016-05-03 11:21:31.167583 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.022761 0 0.000000
  1457. -35> 2016-05-03 11:21:31.167606 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.b7( v 160608'559635 (157924'556238,160608'559635] local-les=160580 n=4277 ec=1 les/c/f 160580/160580/154843 160563/160569/119493) [12,51] r=1 lpr=0 pi=119486-160568/112 crt=160607'559629 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1458. -34> 2016-05-03 11:21:31.168284 7ff588731940 5 osd.51 pg_epoch: 160595 pg[2.cf(unlocked)] enter Initial
  1459. -33> 2016-05-03 11:21:31.208211 7ff588731940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.039926 0 0.000000
  1460. -32> 2016-05-03 11:21:31.208251 7ff588731940 5 osd.51 pg_epoch: 160595 pg[2.cf( v 160552'646871 (158214'642833,160552'646871] local-les=160477 n=4504 ec=1 les/c/f 160554/160554/154843 160589/160589/160575) [51,27] r=0 lpr=0 pi=160476-160588/6 crt=160552'646865 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1461. -31> 2016-05-03 11:21:31.208487 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.10e(unlocked)] enter Initial
  1462. -30> 2016-05-03 11:21:31.236777 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.028290 0 0.000000
  1463. -29> 2016-05-03 11:21:31.236806 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.10e( v 160552'724744 (159785'721530,160552'724744] local-les=160581 n=8736 ec=1 les/c/f 160581/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'724738 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1464. -28> 2016-05-03 11:21:31.236997 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.111(unlocked)] enter Initial
  1465. -27> 2016-05-03 11:21:31.314744 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] exit Initial 0.077747 0 0.000000
  1466. -26> 2016-05-03 11:21:31.314775 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.111( v 160552'773280 lc 0'0 (159786'769993,160552'773280] local-les=160606 n=8714 ec=1 les/c/f 160606/160554/154843 160563/160589/160589) [51,47] r=0 lpr=0 pi=160476-160588/5 crt=160552'773278 mlcod 0'0 inactive NIBBLEWISE m=5] enter Reset
  1467. -25> 2016-05-03 11:21:31.314912 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.16f(unlocked)] enter Initial
  1468. -24> 2016-05-03 11:21:31.388515 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] exit Initial 0.073603 0 0.000000
  1469. -23> 2016-05-03 11:21:31.388537 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.16f( v 160552'753441 lc 0'0 (159762'750264,160552'753441] local-les=160605 n=9017 ec=1 les/c/f 160605/160554/154843 160591/160591/160573) [51,41] r=0 lpr=0 pi=160476-160590/6 crt=160552'753439 mlcod 0'0 inactive NIBBLEWISE m=6] enter Reset
  1470. -22> 2016-05-03 11:21:31.389152 7ff588731940 5 osd.51 pg_epoch: 160607 pg[2.186(unlocked)] enter Initial
  1471. -21> 2016-05-03 11:21:31.414567 7ff588731940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.025414 0 0.000000
  1472. -20> 2016-05-03 11:21:31.414588 7ff588731940 5 osd.51 pg_epoch: 160607 pg[2.186( v 160607'921912 (159779'918343,160607'921912] local-les=160580 n=8754 ec=1 les/c/f 160580/160580/154843 160563/160565/119493) [9,51] r=1 lpr=0 pi=119482-160564/112 crt=160607'921906 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1473. -19> 2016-05-03 11:21:31.414725 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.188(unlocked)] enter Initial
  1474. -18> 2016-05-03 11:21:31.440432 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.025707 0 0.000000
  1475. -17> 2016-05-03 11:21:31.440453 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.188( v 160608'821632 (159769'818618,160608'821632] local-les=160581 n=8874 ec=1 les/c/f 160581/160606/154843 160563/160569/160569) [51,65] r=0 lpr=0 crt=160607'821628 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1476. -16> 2016-05-03 11:21:31.440620 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1a2(unlocked)] enter Initial
  1477. -15> 2016-05-03 11:21:31.460476 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] exit Initial 0.019855 0 0.000000
  1478. -14> 2016-05-03 11:21:31.460496 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1a2( v 160552'880114 (159603'876851,160552'880114] local-les=160582 n=8990 ec=1 les/c/f 160582/160477/154843 160563/160563/160563) [51,92] r=0 lpr=0 pi=160476-160562/3 crt=160551'880108 lcod 0'0 mlcod 0'0 inactive NIBBLEWISE] enter Reset
  1479. -13> 2016-05-03 11:21:31.460616 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1aa(unlocked)] enter Initial
  1480. -12> 2016-05-03 11:21:31.484893 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024276 0 0.000000
  1481. -11> 2016-05-03 11:21:31.484913 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1aa( v 160608'868721 (159737'865371,160608'868721] local-les=160580 n=8888 ec=1 les/c/f 160580/160583/154843 160563/160565/155728) [61,51] r=1 lpr=0 pi=119433-160564/118 crt=160607'868715 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1482. -10> 2016-05-03 11:21:31.485043 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1bb(unlocked)] enter Initial
  1483. -9> 2016-05-03 11:21:31.509894 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.024851 0 0.000000
  1484. -8> 2016-05-03 11:21:31.509914 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1bb( v 160608'794616 (159603'791113,160608'794616] local-les=160580 n=8547 ec=1 les/c/f 160580/160580/154843 160563/160567/119449) [86,51] r=1 lpr=0 pi=119433-160566/118 crt=160607'794610 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1485. -7> 2016-05-03 11:21:31.510065 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1e8(unlocked)] enter Initial
  1486. -6> 2016-05-03 11:21:31.542008 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.031943 0 0.000000
  1487. -5> 2016-05-03 11:21:31.542045 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1e8( v 160608'854243 (159787'849411,160608'854243] local-les=160595 n=8747 ec=1 les/c/f 160595/160605/154843 160591/160591/160591) [41,51] r=1 lpr=0 pi=159447-160590/19 crt=160607'854237 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1488. -4> 2016-05-03 11:21:31.542193 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1f2(unlocked)] enter Initial
  1489. -3> 2016-05-03 11:21:31.570189 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] exit Initial 0.027996 0 0.000000
  1490. -2> 2016-05-03 11:21:31.570209 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1f2( v 160608'831572 (159787'828017,160608'831572] local-les=160580 n=8810 ec=1 les/c/f 160580/160580/154843 160563/160569/119448) [65,51] r=1 lpr=0 pi=119434-160568/114 crt=160607'831566 lcod 0'0 inactive NOTIFY NIBBLEWISE] enter Reset
  1491. -1> 2016-05-03 11:21:31.570347 7ff588731940 5 osd.51 pg_epoch: 160608 pg[2.1fd(unlocked)] enter Initial
  1492. 0> 2016-05-03 11:21:31.639208 7ff588731940 -1 osd/PGLog.cc: In function 'static void PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, const pg_info_t&, std::map<eversion_t, hobject_t>&, PGLog::IndexedLog&, pg_missing_t&, std::ostringstream&, std::set<std::basic_string<char> >*)' thread 7ff588731940 time 2016-05-03 11:21:31.636815
  1493. osd/PGLog.cc: 988: FAILED assert(oi.version == i->first)
  1494.  
  1495. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1496. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7ff5882277eb]
  1497. 2: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7ff587d61e8d]
  1498. 3: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7ff587dde55a]
  1499. 4: (OSD::load_pgs()+0x61d) [0x7ff587c9c58d]
  1500. 5: (OSD::init()+0xe74) [0x7ff587cac694]
  1501. 6: (main()+0x2954) [0x7ff587c31474]
  1502. 7: (__libc_start_main()+0xf5) [0x7ff584ab3ec5]
  1503. 8: (()+0x2f82b7) [0x7ff587c612b7]
  1504. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1505.  
  1506. --- logging levels ---
  1507. 0/ 5 none
  1508. 0/ 1 lockdep
  1509. 0/ 1 context
  1510. 1/ 1 crush
  1511. 1/ 5 mds
  1512. 1/ 5 mds_balancer
  1513. 1/ 5 mds_locker
  1514. 1/ 5 mds_log
  1515. 1/ 5 mds_log_expire
  1516. 1/ 5 mds_migrator
  1517. 0/ 1 buffer
  1518. 0/ 1 timer
  1519. 0/ 1 filer
  1520. 0/ 1 striper
  1521. 0/ 1 objecter
  1522. 0/ 5 rados
  1523. 0/ 5 rbd
  1524. 0/ 5 rbd_replay
  1525. 0/ 5 journaler
  1526. 0/ 5 objectcacher
  1527. 0/ 5 client
  1528. 0/ 5 osd
  1529. 0/ 5 optracker
  1530. 0/ 5 objclass
  1531. 1/ 3 filestore
  1532. 1/ 3 keyvaluestore
  1533. 1/ 3 journal
  1534. 0/ 5 ms
  1535. 1/ 5 mon
  1536. 0/10 monc
  1537. 1/ 5 paxos
  1538. 0/ 5 tp
  1539. 1/ 5 auth
  1540. 1/ 5 crypto
  1541. 1/ 1 finisher
  1542. 1/ 5 heartbeatmap
  1543. 1/ 5 perfcounter
  1544. 1/ 5 rgw
  1545. 1/10 civetweb
  1546. 1/ 5 javaclient
  1547. 1/ 5 asok
  1548. 1/ 1 throttle
  1549. 0/ 0 refs
  1550. 1/ 5 xio
  1551. 1/ 5 compressor
  1552. 1/ 5 newstore
  1553. -2/-2 (syslog threshold)
  1554. -1/-1 (stderr threshold)
  1555. max_recent 10000
  1556. max_new 1000
  1557. log_file /var/log/ceph/ceph-osd.51.log
  1558. --- end dump of recent events ---
  1559. 2016-05-03 11:21:31.783256 7ff588731940 -1 *** Caught signal (Aborted) **
  1560. in thread 7ff588731940
  1561.  
  1562. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1563. 1: (()+0x7d1aca) [0x7ff58813aaca]
  1564. 2: (()+0x10340) [0x7ff58684d340]
  1565. 3: (gsignal()+0x39) [0x7ff584ac8cc9]
  1566. 4: (abort()+0x148) [0x7ff584acc0d8]
  1567. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff5853d3535]
  1568. 6: (()+0x5e6d6) [0x7ff5853d16d6]
  1569. 7: (()+0x5e703) [0x7ff5853d1703]
  1570. 8: (()+0x5e922) [0x7ff5853d1922]
  1571. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7ff5882279d8]
  1572. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7ff587d61e8d]
  1573. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7ff587dde55a]
  1574. 12: (OSD::load_pgs()+0x61d) [0x7ff587c9c58d]
  1575. 13: (OSD::init()+0xe74) [0x7ff587cac694]
  1576. 14: (main()+0x2954) [0x7ff587c31474]
  1577. 15: (__libc_start_main()+0xf5) [0x7ff584ab3ec5]
  1578. 16: (()+0x2f82b7) [0x7ff587c612b7]
  1579. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1580.  
  1581. --- begin dump of recent events ---
  1582. 0> 2016-05-03 11:21:31.783256 7ff588731940 -1 *** Caught signal (Aborted) **
  1583. in thread 7ff588731940
  1584.  
  1585. ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd)
  1586. 1: (()+0x7d1aca) [0x7ff58813aaca]
  1587. 2: (()+0x10340) [0x7ff58684d340]
  1588. 3: (gsignal()+0x39) [0x7ff584ac8cc9]
  1589. 4: (abort()+0x148) [0x7ff584acc0d8]
  1590. 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff5853d3535]
  1591. 6: (()+0x5e6d6) [0x7ff5853d16d6]
  1592. 7: (()+0x5e703) [0x7ff5853d1703]
  1593. 8: (()+0x5e922) [0x7ff5853d1922]
  1594. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7ff5882279d8]
  1595. 10: (PGLog::read_log(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, std::map<eversion_t, hobject_t, std::less<eversion_t>, std::allocator<std::pair<eversion_t const, hobject_t> > >&, PGLog::IndexedLog&, pg_missing_t&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*)+0x107d) [0x7ff587d61e8d]
  1596. 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x4aa) [0x7ff587dde55a]
  1597. 12: (OSD::load_pgs()+0x61d) [0x7ff587c9c58d]
  1598. 13: (OSD::init()+0xe74) [0x7ff587cac694]
  1599. 14: (main()+0x2954) [0x7ff587c31474]
  1600. 15: (__libc_start_main()+0xf5) [0x7ff584ab3ec5]
  1601. 16: (()+0x2f82b7) [0x7ff587c612b7]
  1602. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1603.  
  1604. --- logging levels ---
  1605. 0/ 5 none
  1606. 0/ 1 lockdep
  1607. 0/ 1 context
  1608. 1/ 1 crush
  1609. 1/ 5 mds
  1610. 1/ 5 mds_balancer
  1611. 1/ 5 mds_locker
  1612. 1/ 5 mds_log
  1613. 1/ 5 mds_log_expire
  1614. 1/ 5 mds_migrator
  1615. 0/ 1 buffer
  1616. 0/ 1 timer
  1617. 0/ 1 filer
  1618. 0/ 1 striper
  1619. 0/ 1 objecter
  1620. 0/ 5 rados
  1621. 0/ 5 rbd
  1622. 0/ 5 rbd_replay
  1623. 0/ 5 journaler
  1624. 0/ 5 objectcacher
  1625. 0/ 5 client
  1626. 0/ 5 osd
  1627. 0/ 5 optracker
  1628. 0/ 5 objclass
  1629. 1/ 3 filestore
  1630. 1/ 3 keyvaluestore
  1631. 1/ 3 journal
  1632. 0/ 5 ms
  1633. 1/ 5 mon
  1634. 0/10 monc
  1635. 1/ 5 paxos
  1636. 0/ 5 tp
  1637. 1/ 5 auth
  1638. 1/ 5 crypto
  1639. 1/ 1 finisher
  1640. 1/ 5 heartbeatmap
  1641. 1/ 5 perfcounter
  1642. 1/ 5 rgw
  1643. 1/10 civetweb
  1644. 1/ 5 javaclient
  1645. 1/ 5 asok
  1646. 1/ 1 throttle
  1647. 0/ 0 refs
  1648. 1/ 5 xio
  1649. 1/ 5 compressor
  1650. 1/ 5 newstore
  1651. -2/-2 (syslog threshold)
  1652. -1/-1 (stderr threshold)
  1653. max_recent 10000
  1654. max_new 1000
  1655. log_file /var/log/ceph/ceph-osd.51.log
  1656. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!