API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Dec 1st, 2011
46
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 205.62 KB | None | 0 0
raw download clone embed print report
  1. 2011-12-01 11:56:15.017294 7f99a3e2b7a0 ceph version 0.38-255-g353ee00 (commit:353ee0004afb6b2c56b064904129c750cc68dd2a), process ceph-osd, pid 15332
  2. 2011-12-01 11:56:15.020585 7f99a3e2b7a0 filestore(/data/osd) mount FIEMAP ioctl is supported
  3. 2011-12-01 11:56:15.020641 7f99a3e2b7a0 filestore(/data/osd) mount did NOT detect btrfs
  4. 2011-12-01 11:56:15.020795 7f99a3e2b7a0 filestore(/data/osd) mount found snaps <>
  5. 2011-12-01 11:56:15.020862 7f99a3e2b7a0 filestore(/data/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  6. 2011-12-01 11:56:15.021266 7f99a3e2b7a0 journal _open /data/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes, directio = 1
  7. 2011-12-01 11:56:15.022086 7f99a3e2b7a0 journal read_entry 180224 : seq 21 529 bytes
  8. 2011-12-01 11:56:15.022193 7f99a3e2b7a0 journal _open /data/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes, directio = 1
  9. 2011-12-01 11:56:15.023984 7f99a3e2b7a0 FileStore is up to date.
  10. 2011-12-01 11:56:15.024569 7f99a3e2b7a0 journal close /data/osd.journal
  11. 2011-12-01 11:56:15.026922 7f99a3e2b7a0 osd.0 0 mounting /data/osd /data/osd.journal
  12. 2011-12-01 11:56:15.027828 7f99a3e2b7a0 filestore(/data/osd) mount FIEMAP ioctl is supported
  13. 2011-12-01 11:56:15.027865 7f99a3e2b7a0 filestore(/data/osd) mount did NOT detect btrfs
  14. 2011-12-01 11:56:15.027967 7f99a3e2b7a0 filestore(/data/osd) mount found snaps <>
  15. 2011-12-01 11:56:15.028003 7f99a3e2b7a0 filestore(/data/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  16. 2011-12-01 11:56:15.028301 7f99a3e2b7a0 journal _open /data/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes, directio = 1
  17. 2011-12-01 11:56:15.028345 7f99a3e2b7a0 journal read_entry 180224 : seq 21 529 bytes
  18. 2011-12-01 11:56:15.028378 7f99a3e2b7a0 journal _open /data/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes, directio = 1
  19. 2011-12-01 11:56:15.029240 7f99a3e2b7a0 osd.0 0 boot
  20. 2011-12-01 11:56:15.029522 7f99a3e2b7a0 osd.0 0 read_superblock sb(99ec31f1-3e67-4f35-9b24-6ca7d38d498b osd.0 e3 [1,3] lci=[2,3])
  21. 2011-12-01 11:56:15.029567 7f99a3e2b7a0 osd.0 0 get_map 3 - loading and decoding 0xcf2000
  22. 2011-12-01 11:56:15.029711 7f99a3e2b7a0 osd.0 0 add_map 3 0xcf2000
  23. 2011-12-01 11:56:15.029742 7f99a3e2b7a0 osd.0 3 clear_temp
  24. 2011-12-01 11:56:15.029844 7f99a3e2b7a0 osd.0 3 0 objects
  25. 2011-12-01 11:56:15.029882 7f99a3e2b7a0 osd.0 3 load_pgs
  26. 2011-12-01 11:56:15.029972 7f99a3e2b7a0 osd.0 3 _open_lock_pg 1.0p0
  27. 2011-12-01 11:56:15.030016 7f99a3e2b7a0 osd.0 3 _get_pool 1 0 -> 1
  28. 2011-12-01 11:56:15.030260 7f99a3e2b7a0 osd.0 0 pg[1.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  29. 2011-12-01 11:56:15.030325 7f99a3e2b7a0 osd.0 0 pg[1.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  30. 2011-12-01 11:56:15.030502 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  31. 2011-12-01 11:56:15.030594 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  32. 2011-12-01 11:56:15.030671 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  33. 2011-12-01 11:56:15.030733 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  34. 2011-12-01 11:56:15.030786 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000587 0 0.000000
  35. 2011-12-01 11:56:15.030844 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  36. 2011-12-01 11:56:15.030878 7f99a3e2b7a0 osd.0 3 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  37. 2011-12-01 11:56:15.030915 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  38. 2011-12-01 11:56:15.030940 7f99a3e2b7a0 osd.0 3 _open_lock_pg 2.0p0
  39. 2011-12-01 11:56:15.030963 7f99a3e2b7a0 osd.0 3 _get_pool 2 0 -> 1
  40. 2011-12-01 11:56:15.031030 7f99a3e2b7a0 osd.0 0 pg[2.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  41. 2011-12-01 11:56:15.031089 7f99a3e2b7a0 osd.0 0 pg[2.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  42. 2011-12-01 11:56:15.031209 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  43. 2011-12-01 11:56:15.031244 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  44. 2011-12-01 11:56:15.031286 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  45. 2011-12-01 11:56:15.031333 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  46. 2011-12-01 11:56:15.031421 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000347 0 0.000000
  47. 2011-12-01 11:56:15.031480 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  48. 2011-12-01 11:56:15.031519 7f99a3e2b7a0 osd.0 3 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  49. 2011-12-01 11:56:15.031565 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  50. 2011-12-01 11:56:15.031588 7f99a3e2b7a0 osd.0 3 load_pgs skipping non-pg temp
  51. 2011-12-01 11:56:15.031619 7f99a3e2b7a0 osd.0 3 _open_lock_pg 0.1p0
  52. 2011-12-01 11:56:15.031644 7f99a3e2b7a0 osd.0 3 _get_pool 0 0 -> 1
  53. 2011-12-01 11:56:15.031733 7f99a3e2b7a0 osd.0 0 pg[0.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  54. 2011-12-01 11:56:15.031777 7f99a3e2b7a0 osd.0 0 pg[0.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  55. 2011-12-01 11:56:15.031913 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  56. 2011-12-01 11:56:15.031946 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  57. 2011-12-01 11:56:15.031986 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  58. 2011-12-01 11:56:15.032037 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  59. 2011-12-01 11:56:15.032141 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000360 0 0.000000
  60. 2011-12-01 11:56:15.032198 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  61. 2011-12-01 11:56:15.032237 7f99a3e2b7a0 osd.0 3 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  62. 2011-12-01 11:56:15.032277 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  63. 2011-12-01 11:56:15.032297 7f99a3e2b7a0 osd.0 3 _open_lock_pg 0.0p0
  64. 2011-12-01 11:56:15.032318 7f99a3e2b7a0 osd.0 3 _get_pool 0 1 -> 2
  65. 2011-12-01 11:56:15.032398 7f99a3e2b7a0 osd.0 0 pg[0.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  66. 2011-12-01 11:56:15.032452 7f99a3e2b7a0 osd.0 0 pg[0.0p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  67. 2011-12-01 11:56:15.032644 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  68. 2011-12-01 11:56:15.032686 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  69. 2011-12-01 11:56:15.032775 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  70. 2011-12-01 11:56:15.032823 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  71. 2011-12-01 11:56:15.032871 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000471 0 0.000000
  72. 2011-12-01 11:56:15.033043 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  73. 2011-12-01 11:56:15.033087 7f99a3e2b7a0 osd.0 3 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  74. 2011-12-01 11:56:15.033178 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  75. 2011-12-01 11:56:15.033224 7f99a3e2b7a0 osd.0 3 _open_lock_pg 2.1p0
  76. 2011-12-01 11:56:15.033250 7f99a3e2b7a0 osd.0 3 _get_pool 2 1 -> 2
  77. 2011-12-01 11:56:15.033342 7f99a3e2b7a0 osd.0 0 pg[2.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  78. 2011-12-01 11:56:15.033393 7f99a3e2b7a0 osd.0 0 pg[2.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  79. 2011-12-01 11:56:15.033598 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  80. 2011-12-01 11:56:15.033628 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  81. 2011-12-01 11:56:15.033671 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  82. 2011-12-01 11:56:15.033721 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  83. 2011-12-01 11:56:15.033770 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000421 0 0.000000
  84. 2011-12-01 11:56:15.033828 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  85. 2011-12-01 11:56:15.033866 7f99a3e2b7a0 osd.0 3 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  86. 2011-12-01 11:56:15.033912 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  87. 2011-12-01 11:56:15.033942 7f99a3e2b7a0 osd.0 3 load_pgs skipping non-pg meta
  88. 2011-12-01 11:56:15.033964 7f99a3e2b7a0 osd.0 3 _open_lock_pg 1.1p0
  89. 2011-12-01 11:56:15.033985 7f99a3e2b7a0 osd.0 3 _get_pool 1 1 -> 2
  90. 2011-12-01 11:56:15.034115 7f99a3e2b7a0 osd.0 0 pg[1.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter Initial
  91. 2011-12-01 11:56:15.034166 7f99a3e2b7a0 osd.0 0 pg[1.1p0( DNE empty n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 mlcod 0'0 inactive] enter NotTrimming
  92. 2011-12-01 11:56:15.034313 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log 0~0
  93. 2011-12-01 11:56:15.034355 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=0 lpr=0 mlcod 0'0 inactive] read_log done
  94. 2011-12-01 11:56:15.034404 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] handle_backlog_loaded
  95. 2011-12-01 11:56:15.034455 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=0 mlcod 0'0 inactive] set_last_peering_reset 3
  96. 2011-12-01 11:56:15.034503 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] exit Initial 0.000436 0 0.000000
  97. 2011-12-01 11:56:15.034558 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] enter Reset
  98. 2011-12-01 11:56:15.034590 7f99a3e2b7a0 osd.0 3 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 3
  99. 2011-12-01 11:56:15.034639 7f99a3e2b7a0 osd.0 3 load_pgs loaded pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] log(0'0,0'0]
  100. 2011-12-01 11:56:15.034700 7f99a3e2b7a0 osd.0 3 load_pgs done
  101. 2011-12-01 11:56:15.034727 7f99a3e2b7a0 osd.0 3 superblock: i am osd.0
  102. 2011-12-01 11:56:15.034751 7f99a3e2b7a0 osd.0 3 create_logger
  103. 2011-12-01 11:56:15.036074 7f99a3e27700 osd.0 3 OSD::ms_get_authorizer type=mon
  104. 2011-12-01 11:56:15.036437 7f9997d32700 osd.0 3 ms_handle_connect on mon
  105. 2011-12-01 11:56:15.036464 7f9997d32700 osd.0 3 start_boot - have maps 1..3
  106. 2011-12-01 11:56:15.039326 7f999552d700 osd.0 3 _got_boot_version mon has osdmaps 1..3
  107. 2011-12-01 11:56:15.039400 7f999552d700 osd.0 3 send_boot
  108. 2011-12-01 11:56:15.039423 7f999552d700 osd.0 3 assuming cluster_addr ip matches client_addr
  109. 2011-12-01 11:56:15.039460 7f999552d700 osd.0 3 assuming hb_addr ip matches cluster_addr
  110. 2011-12-01 11:56:15.039491 7f999552d700 osd.0 3 client_addr 127.0.0.1:6801/15332, cluster_addr 127.0.0.1:6802/15332, hb addr 127.0.0.1:6803/15332
  111. 2011-12-01 11:56:15.039579 7f9992426700 osd.0 3 update_osd_stat osd_stat(167 MB used, 14182 MB avail, 15116 MB total, peers []/[])
  112. 2011-12-01 11:56:15.039606 7f9992426700 osd.0 3 heartbeat: osd_stat(167 MB used, 14182 MB avail, 15116 MB total, peers []/[])
  113. 2011-12-01 11:56:15.103597 7f9997d32700 osd.0 3 _dispatch 0xd00400 osd_map(3..4 src has 1..4) v2
  114. 2011-12-01 11:56:15.103649 7f9997d32700 osd.0 3 handle_osd_map epochs [3,4], i have 3, src has [1,4]
  115. 2011-12-01 11:56:15.103667 7f9997d32700 osd.0 3 locking handle_osd_map permissions
  116. 2011-12-01 11:56:15.103685 7f9997d32700 osd.0 3 handle_osd_map got inc map for epoch 4
  117. 2011-12-01 11:56:15.103712 7f9997d32700 osd.0 3 add_map_inc_bl 4 121 bytes
  118. 2011-12-01 11:56:15.103822 7f9997d32700 osd.0 3 add_map 4 0xcf2300
  119. 2011-12-01 11:56:15.103852 7f9997d32700 osd.0 3 add_map_bl 4 1250 bytes
  120. 2011-12-01 11:56:15.103886 7f9997d32700 osd.0 3 advance to epoch 4 (<= newest 4)
  121. 2011-12-01 11:56:15.103904 7f9997d32700 osd.0 4 advance_map epoch 4 6 pgs
  122. 2011-12-01 11:56:15.103924 7f9997d32700 osd.0 4 pool 0 removed snaps [], unchanged (snap_epoch = 0)
  123. 2011-12-01 11:56:15.103937 7f9997d32700 osd.0 4 pool 1 removed snaps [], unchanged (snap_epoch = 0)
  124. 2011-12-01 11:56:15.103949 7f9997d32700 osd.0 4 pool 2 removed snaps [], unchanged (snap_epoch = 0)
  125. 2011-12-01 11:56:15.103985 7f9997d32700 osd.0 4 Scanning pg pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  126. 2011-12-01 11:56:15.104011 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  127. 2011-12-01 11:56:15.104037 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  128. 2011-12-01 11:56:15.104072 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  129. 2011-12-01 11:56:15.104094 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  130. 2011-12-01 11:56:15.104129 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  131. 2011-12-01 11:56:15.104152 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  132. 2011-12-01 11:56:15.104184 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  133. 2011-12-01 11:56:15.104205 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  134. 2011-12-01 11:56:15.104241 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  135. 2011-12-01 11:56:15.104264 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  136. 2011-12-01 11:56:15.104300 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  137. 2011-12-01 11:56:15.104328 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  138. 2011-12-01 11:56:15.104370 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  139. 2011-12-01 11:56:15.104392 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  140. 2011-12-01 11:56:15.104437 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  141. 2011-12-01 11:56:15.104458 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  142. 2011-12-01 11:56:15.104489 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  143. 2011-12-01 11:56:15.104513 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  144. 2011-12-01 11:56:15.104545 7f9997d32700 osd.0 4 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  145. 2011-12-01 11:56:15.104572 7f9997d32700 osd.0 4 Scanning pg pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  146. 2011-12-01 11:56:15.104605 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  147. 2011-12-01 11:56:15.104625 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  148. 2011-12-01 11:56:15.104645 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  149. 2011-12-01 11:56:15.104664 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  150. 2011-12-01 11:56:15.104682 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  151. 2011-12-01 11:56:15.104702 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  152. 2011-12-01 11:56:15.104734 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  153. 2011-12-01 11:56:15.104750 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  154. 2011-12-01 11:56:15.104782 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  155. 2011-12-01 11:56:15.104801 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  156. 2011-12-01 11:56:15.104818 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  157. 2011-12-01 11:56:15.104851 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  158. 2011-12-01 11:56:15.104875 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  159. 2011-12-01 11:56:15.104967 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  160. 2011-12-01 11:56:15.104987 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  161. 2011-12-01 11:56:15.105004 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  162. 2011-12-01 11:56:15.105033 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  163. 2011-12-01 11:56:15.105051 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  164. 2011-12-01 11:56:15.105083 7f9997d32700 osd.0 4 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  165. 2011-12-01 11:56:15.105106 7f9997d32700 osd.0 4 Scanning pg pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  166. 2011-12-01 11:56:15.105127 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  167. 2011-12-01 11:56:15.105147 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  168. 2011-12-01 11:56:15.105166 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  169. 2011-12-01 11:56:15.105184 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  170. 2011-12-01 11:56:15.105222 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  171. 2011-12-01 11:56:15.105244 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  172. 2011-12-01 11:56:15.105276 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  173. 2011-12-01 11:56:15.105293 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  174. 2011-12-01 11:56:15.105324 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  175. 2011-12-01 11:56:15.105343 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  176. 2011-12-01 11:56:15.105360 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  177. 2011-12-01 11:56:15.105415 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  178. 2011-12-01 11:56:15.105456 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  179. 2011-12-01 11:56:15.105475 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  180. 2011-12-01 11:56:15.105506 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  181. 2011-12-01 11:56:15.105523 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  182. 2011-12-01 11:56:15.105540 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  183. 2011-12-01 11:56:15.105572 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  184. 2011-12-01 11:56:15.105590 7f9997d32700 osd.0 4 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  185. 2011-12-01 11:56:15.105626 7f9997d32700 osd.0 4 Scanning pg pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  186. 2011-12-01 11:56:15.105642 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  187. 2011-12-01 11:56:15.105719 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  188. 2011-12-01 11:56:15.105739 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  189. 2011-12-01 11:56:15.105757 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  190. 2011-12-01 11:56:15.105776 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  191. 2011-12-01 11:56:15.105809 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  192. 2011-12-01 11:56:15.105828 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  193. 2011-12-01 11:56:15.105857 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  194. 2011-12-01 11:56:15.105877 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  195. 2011-12-01 11:56:15.105895 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  196. 2011-12-01 11:56:15.105925 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  197. 2011-12-01 11:56:15.105945 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  198. 2011-12-01 11:56:15.105991 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  199. 2011-12-01 11:56:15.106010 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  200. 2011-12-01 11:56:15.106042 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  201. 2011-12-01 11:56:15.106059 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  202. 2011-12-01 11:56:15.106087 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  203. 2011-12-01 11:56:15.106111 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  204. 2011-12-01 11:56:15.106130 7f9997d32700 osd.0 4 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  205. 2011-12-01 11:56:15.106166 7f9997d32700 osd.0 4 Scanning pg pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  206. 2011-12-01 11:56:15.106186 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  207. 2011-12-01 11:56:15.106218 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  208. 2011-12-01 11:56:15.106238 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  209. 2011-12-01 11:56:15.106256 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  210. 2011-12-01 11:56:15.106287 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  211. 2011-12-01 11:56:15.106308 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  212. 2011-12-01 11:56:15.106339 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  213. 2011-12-01 11:56:15.106356 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  214. 2011-12-01 11:56:15.106375 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  215. 2011-12-01 11:56:15.106404 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  216. 2011-12-01 11:56:15.106422 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  217. 2011-12-01 11:56:15.106455 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  218. 2011-12-01 11:56:15.106479 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  219. 2011-12-01 11:56:15.106497 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  220. 2011-12-01 11:56:15.106516 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  221. 2011-12-01 11:56:15.106533 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  222. 2011-12-01 11:56:15.106551 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  223. 2011-12-01 11:56:15.106591 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  224. 2011-12-01 11:56:15.106610 7f9997d32700 osd.0 4 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  225. 2011-12-01 11:56:15.106654 7f9997d32700 osd.0 4 Scanning pg pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive]
  226. 2011-12-01 11:56:15.106676 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] handle_advance_map []/[]
  227. 2011-12-01 11:56:15.106715 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: Reset advmap
  228. 2011-12-01 11:56:15.106760 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] acting_up_affected newup [] newacting []
  229. 2011-12-01 11:56:15.106780 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] state<Reset>: up or acting affected, calling start_peering_interval again
  230. 2011-12-01 11:56:15.106801 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [0] r=0 lpr=3 mlcod 0'0 inactive] set_last_peering_reset 4
  231. 2011-12-01 11:56:15.106821 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 2/2/2) [] r=-1 lpr=4 inactive] noting past interval(2-3 [0]/[0] maybe_went_rw)
  232. 2011-12-01 11:56:15.106841 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] up [0] -> [], acting [0] -> [], role 0 -> -1
  233. 2011-12-01 11:56:15.106857 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] cancel_generate_backlog
  234. 2011-12-01 11:56:15.106888 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_primary_state
  235. 2011-12-01 11:56:15.106906 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_change
  236. 2011-12-01 11:56:15.106937 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] remove_watchers
  237. 2011-12-01 11:56:15.106957 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] exit NotTrimming
  238. 2011-12-01 11:56:15.106993 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] enter NotTrimming
  239. 2011-12-01 11:56:15.107012 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] clear_stats
  240. 2011-12-01 11:56:15.107030 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  241. 2011-12-01 11:56:15.107060 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] on_role_change
  242. 2011-12-01 11:56:15.107077 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 inactive] requeue_ops
  243. 2011-12-01 11:56:15.107096 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] cancel_recovery
  244. 2011-12-01 11:56:15.107126 7f9997d32700 osd.0 4 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] clear_recovery_state
  245. 2011-12-01 11:56:15.107179 7f9997d32700 osd.0 0 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  246. 2011-12-01 11:56:15.107208 7f9997d32700 osd.0 0 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  247. 2011-12-01 11:56:15.107252 7f9997d32700 osd.0 0 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  248. 2011-12-01 11:56:15.107288 7f9997d32700 osd.0 0 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  249. 2011-12-01 11:56:15.107324 7f9997d32700 osd.0 0 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  250. 2011-12-01 11:56:15.107367 7f9997d32700 osd.0 0 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  251. 2011-12-01 11:56:15.107397 7f9997d32700 osd.0 0 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  252. 2011-12-01 11:56:15.107437 7f9997d32700 osd.0 0 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  253. 2011-12-01 11:56:15.107481 7f9997d32700 osd.0 0 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  254. 2011-12-01 11:56:15.107508 7f9997d32700 osd.0 0 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  255. 2011-12-01 11:56:15.107552 7f9997d32700 osd.0 0 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info info 337
  256. 2011-12-01 11:56:15.107581 7f9997d32700 osd.0 0 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] write_info bigbl 38
  257. 2011-12-01 11:56:15.107603 7f9997d32700 osd.0 4 write_superblock sb(99ec31f1-3e67-4f35-9b24-6ca7d38d498b osd.0 e4 [1,4] lci=[2,3])
  258. 2011-12-01 11:56:15.111502 7f9997d32700 osd.0 4 trim_map_bl_cache up to 5
  259. 2011-12-01 11:56:15.111536 7f9997d32700 osd.0 4 trim_map_cache prior to 0
  260. 2011-12-01 11:56:15.111622 7f9997d32700 osd.0 4 start_boot - have maps 1..4
  261. 2011-12-01 11:56:15.111678 7f9997d32700 osd.0 4 unlocking map_in_progress
  262. 2011-12-01 11:56:15.112258 7f999552d700 osd.0 4 _got_boot_version mon has osdmaps 1..4
  263. 2011-12-01 11:56:15.112280 7f999552d700 osd.0 4 send_boot
  264. 2011-12-01 11:56:15.112315 7f999552d700 osd.0 4 client_addr 127.0.0.1:6801/15332, cluster_addr 127.0.0.1:6802/15332, hb addr 127.0.0.1:6803/15332
  265. 2011-12-01 11:56:16.039381 7f999ed40700 osd.0 4 tick
  266. 2011-12-01 11:56:16.039571 7f999ed40700 osd.0 4 scrub_should_schedule loadavg 0.01 < max 0.5 = yes
  267. 2011-12-01 11:56:16.039602 7f999ed40700 osd.0 4 sched_scrub
  268. 2011-12-01 11:56:16.039644 7f999ed40700 osd.0 4 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:16.039622 (86400 seconds ago)
  269. 2011-12-01 11:56:16.039664 7f999ed40700 osd.0 4 sched_scrub done
  270. 2011-12-01 11:56:16.039703 7f999ed40700 osd.0 4 do_mon_report
  271. 2011-12-01 11:56:16.039729 7f999ed40700 osd.0 4 send_alive up_thru currently 2 want 0
  272. 2011-12-01 11:56:16.039752 7f999ed40700 osd.0 4 send_pg_stats
  273. 2011-12-01 11:56:16.039775 7f999ed40700 osd.0 4 send_pg_stats - 0 pgs updated
  274. 2011-12-01 11:56:16.115601 7f9997d32700 osd.0 4 _dispatch 0xd00200 osd_map(4..5 src has 1..5) v2
  275. 2011-12-01 11:56:16.115642 7f9997d32700 osd.0 4 handle_osd_map epochs [4,5], i have 4, src has [1,5]
  276. 2011-12-01 11:56:16.115696 7f9997d32700 osd.0 4 locking handle_osd_map permissions
  277. 2011-12-01 11:56:16.115721 7f9997d32700 osd.0 4 handle_osd_map got inc map for epoch 5
  278. 2011-12-01 11:56:16.115751 7f9997d32700 osd.0 4 add_map_inc_bl 5 556 bytes
  279. 2011-12-01 11:56:16.115875 7f9997d32700 osd.0 4 add_map 5 0xd11300
  280. 2011-12-01 11:56:16.115921 7f9997d32700 osd.0 4 add_map_bl 5 1250 bytes
  281. 2011-12-01 11:56:16.115956 7f9997d32700 osd.0 4 advance to epoch 5 (<= newest 5)
  282. 2011-12-01 11:56:16.115983 7f9997d32700 osd.0 5 advance_map epoch 5 6 pgs
  283. 2011-12-01 11:56:16.116006 7f9997d32700 osd.0 5 up_epoch is 5
  284. 2011-12-01 11:56:16.116027 7f9997d32700 osd.0 5 boot_epoch is 5
  285. 2011-12-01 11:56:16.116048 7f9997d32700 osd.0 5 pool 0 removed snaps [], unchanged (snap_epoch = 0)
  286. 2011-12-01 11:56:16.116069 7f9997d32700 osd.0 5 pool 1 removed snaps [], unchanged (snap_epoch = 0)
  287. 2011-12-01 11:56:16.116091 7f9997d32700 osd.0 5 pool 2 removed snaps [], unchanged (snap_epoch = 0)
  288. 2011-12-01 11:56:16.116138 7f9997d32700 osd.0 5 Scanning pg pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  289. 2011-12-01 11:56:16.116196 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  290. 2011-12-01 11:56:16.116231 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  291. 2011-12-01 11:56:16.116263 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  292. 2011-12-01 11:56:16.116293 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  293. 2011-12-01 11:56:16.116349 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  294. 2011-12-01 11:56:16.116385 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  295. 2011-12-01 11:56:16.116418 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  296. 2011-12-01 11:56:16.116448 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  297. 2011-12-01 11:56:16.116504 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  298. 2011-12-01 11:56:16.116538 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  299. 2011-12-01 11:56:16.116587 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  300. 2011-12-01 11:56:16.116629 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  301. 2011-12-01 11:56:16.116671 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  302. 2011-12-01 11:56:16.116703 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  303. 2011-12-01 11:56:16.116735 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  304. 2011-12-01 11:56:16.116774 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  305. 2011-12-01 11:56:16.116833 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  306. 2011-12-01 11:56:16.116877 7f9997d32700 osd.0 5 Scanning pg pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  307. 2011-12-01 11:56:16.117034 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  308. 2011-12-01 11:56:16.117326 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  309. 2011-12-01 11:56:16.117362 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  310. 2011-12-01 11:56:16.117394 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  311. 2011-12-01 11:56:16.117449 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  312. 2011-12-01 11:56:16.117485 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  313. 2011-12-01 11:56:16.117518 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  314. 2011-12-01 11:56:16.117548 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  315. 2011-12-01 11:56:16.117582 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  316. 2011-12-01 11:56:16.117637 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  317. 2011-12-01 11:56:16.117671 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  318. 2011-12-01 11:56:16.117708 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  319. 2011-12-01 11:56:16.117772 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  320. 2011-12-01 11:56:16.117804 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  321. 2011-12-01 11:56:16.117836 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  322. 2011-12-01 11:56:16.117868 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  323. 2011-12-01 11:56:16.117899 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  324. 2011-12-01 11:56:16.117965 7f9997d32700 osd.0 5 Scanning pg pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  325. 2011-12-01 11:56:16.117998 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  326. 2011-12-01 11:56:16.118046 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  327. 2011-12-01 11:56:16.118105 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  328. 2011-12-01 11:56:16.118138 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  329. 2011-12-01 11:56:16.118170 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  330. 2011-12-01 11:56:16.118205 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  331. 2011-12-01 11:56:16.118260 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  332. 2011-12-01 11:56:16.118293 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  333. 2011-12-01 11:56:16.118326 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  334. 2011-12-01 11:56:16.118358 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  335. 2011-12-01 11:56:16.118396 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  336. 2011-12-01 11:56:16.118455 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  337. 2011-12-01 11:56:16.118497 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  338. 2011-12-01 11:56:16.118528 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  339. 2011-12-01 11:56:16.118558 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  340. 2011-12-01 11:56:16.118613 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  341. 2011-12-01 11:56:16.118644 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  342. 2011-12-01 11:56:16.118683 7f9997d32700 osd.0 5 Scanning pg pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  343. 2011-12-01 11:56:16.118717 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  344. 2011-12-01 11:56:16.118772 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  345. 2011-12-01 11:56:16.118804 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  346. 2011-12-01 11:56:16.118835 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  347. 2011-12-01 11:56:16.118866 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  348. 2011-12-01 11:56:16.118924 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  349. 2011-12-01 11:56:16.118958 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  350. 2011-12-01 11:56:16.118988 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  351. 2011-12-01 11:56:16.119021 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  352. 2011-12-01 11:56:16.119065 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  353. 2011-12-01 11:56:16.119123 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  354. 2011-12-01 11:56:16.119159 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  355. 2011-12-01 11:56:16.119200 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  356. 2011-12-01 11:56:16.119256 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  357. 2011-12-01 11:56:16.119286 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  358. 2011-12-01 11:56:16.119318 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  359. 2011-12-01 11:56:16.119349 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  360. 2011-12-01 11:56:16.119411 7f9997d32700 osd.0 5 Scanning pg pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  361. 2011-12-01 11:56:16.119444 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  362. 2011-12-01 11:56:16.119477 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  363. 2011-12-01 11:56:16.119509 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  364. 2011-12-01 11:56:16.119542 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  365. 2011-12-01 11:56:16.119573 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  366. 2011-12-01 11:56:16.119608 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  367. 2011-12-01 11:56:16.119641 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  368. 2011-12-01 11:56:16.119671 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  369. 2011-12-01 11:56:16.119727 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  370. 2011-12-01 11:56:16.119760 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  371. 2011-12-01 11:56:16.119792 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  372. 2011-12-01 11:56:16.119826 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  373. 2011-12-01 11:56:16.119896 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  374. 2011-12-01 11:56:16.119930 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  375. 2011-12-01 11:56:16.119960 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  376. 2011-12-01 11:56:16.119992 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  377. 2011-12-01 11:56:16.120023 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  378. 2011-12-01 11:56:16.120086 7f9997d32700 osd.0 5 Scanning pg pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray]
  379. 2011-12-01 11:56:16.120120 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] handle_advance_map [0]/[0]
  380. 2011-12-01 11:56:16.120155 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: Reset advmap
  381. 2011-12-01 11:56:16.120224 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] acting_up_affected newup [0] newacting [0]
  382. 2011-12-01 11:56:16.120257 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] state<Reset>: up or acting affected, calling start_peering_interval again
  383. 2011-12-01 11:56:16.120289 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [] r=-1 lpr=4 stray] set_last_peering_reset 5
  384. 2011-12-01 11:56:16.120323 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 4/4/4) [0] r=0 lpr=5 mlcod 0'0 !hml stray] noting past interval(4-4 []/[])
  385. 2011-12-01 11:56:16.120378 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] up [] -> [0], acting [] -> [0], role -1 -> 0
  386. 2011-12-01 11:56:16.120410 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray] cancel_generate_backlog
  387. 2011-12-01 11:56:16.120443 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] clear_primary_state
  388. 2011-12-01 11:56:16.120474 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_change
  389. 2011-12-01 11:56:16.120505 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] remove_watchers
  390. 2011-12-01 11:56:16.120563 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] exit NotTrimming
  391. 2011-12-01 11:56:16.120604 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] enter NotTrimming
  392. 2011-12-01 11:56:16.120635 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] on_role_change
  393. 2011-12-01 11:56:16.120665 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml stray+degraded] requeue_ops
  394. 2011-12-01 11:56:16.120721 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] cancel_recovery
  395. 2011-12-01 11:56:16.120753 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] clear_recovery_state
  396. 2011-12-01 11:56:16.120788 7f9997d32700 osd.0 5 state: booting -> active
  397. 2011-12-01 11:56:16.120820 7f9997d32700 osd.0 5 activate_map version 5
  398. 2011-12-01 11:56:16.120875 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  399. 2011-12-01 11:56:16.121106 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.086546 3 0.000187
  400. 2011-12-01 11:56:16.121171 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  401. 2011-12-01 11:56:16.121223 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  402. 2011-12-01 11:56:16.121286 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  403. 2011-12-01 11:56:16.121367 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000142 0 0.000000
  404. 2011-12-01 11:56:16.121419 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  405. 2011-12-01 11:56:16.121461 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  406. 2011-12-01 11:56:16.121537 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  407. 2011-12-01 11:56:16.121581 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  408. 2011-12-01 11:56:16.121639 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  409. 2011-12-01 11:56:16.121674 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  410. 2011-12-01 11:56:16.121709 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  411. 2011-12-01 11:56:16.121742 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  412. 2011-12-01 11:56:16.121765 7f9997d32700 osd.0 5 queue_want_up_thru now 5 (was 0), currently 2
  413. 2011-12-01 11:56:16.121794 7f9997d32700 osd.0 5 send_alive up_thru currently 2 want 5
  414. 2011-12-01 11:56:16.121835 7f9997d32700 osd.0 5 send_alive want 5
  415. 2011-12-01 11:56:16.121935 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000407 0 0.000000
  416. 2011-12-01 11:56:16.122017 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  417. 2011-12-01 11:56:16.122065 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  418. 2011-12-01 11:56:16.122103 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  419. 2011-12-01 11:56:16.122162 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  420. 2011-12-01 11:56:16.122196 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  421. 2011-12-01 11:56:16.122231 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  422. 2011-12-01 11:56:16.122269 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  423. 2011-12-01 11:56:16.122328 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  424. 2011-12-01 11:56:16.122374 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000357 0 0.000000
  425. 2011-12-01 11:56:16.122424 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  426. 2011-12-01 11:56:16.122486 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  427. 2011-12-01 11:56:16.122538 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000113 0 0.000000
  428. 2011-12-01 11:56:16.122589 7f9997d32700 osd.0 5 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  429. 2011-12-01 11:56:16.122635 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  430. 2011-12-01 11:56:16.122686 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.089648 3 0.000314
  431. 2011-12-01 11:56:16.122734 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  432. 2011-12-01 11:56:16.122799 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  433. 2011-12-01 11:56:16.122833 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  434. 2011-12-01 11:56:16.122878 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000080 0 0.000000
  435. 2011-12-01 11:56:16.122927 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  436. 2011-12-01 11:56:16.122993 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  437. 2011-12-01 11:56:16.123036 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  438. 2011-12-01 11:56:16.123069 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  439. 2011-12-01 11:56:16.123126 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  440. 2011-12-01 11:56:16.123161 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  441. 2011-12-01 11:56:16.123195 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  442. 2011-12-01 11:56:16.123228 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  443. 2011-12-01 11:56:16.123251 7f9997d32700 osd.0 5 queue_want_up_thru want 5 <= queued 5, currently 2
  444. 2011-12-01 11:56:16.123319 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000282 0 0.000000
  445. 2011-12-01 11:56:16.123370 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  446. 2011-12-01 11:56:16.123406 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  447. 2011-12-01 11:56:16.123464 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  448. 2011-12-01 11:56:16.123500 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  449. 2011-12-01 11:56:16.123533 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  450. 2011-12-01 11:56:16.123566 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  451. 2011-12-01 11:56:16.123624 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  452. 2011-12-01 11:56:16.123661 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  453. 2011-12-01 11:56:16.123706 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000334 0 0.000000
  454. 2011-12-01 11:56:16.123779 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  455. 2011-12-01 11:56:16.123832 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  456. 2011-12-01 11:56:16.123880 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000121 0 0.000000
  457. 2011-12-01 11:56:16.123951 7f9997d32700 osd.0 5 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  458. 2011-12-01 11:56:16.123999 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  459. 2011-12-01 11:56:16.124051 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.091855 3 0.000253
  460. 2011-12-01 11:56:16.124122 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  461. 2011-12-01 11:56:16.124167 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  462. 2011-12-01 11:56:16.124201 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  463. 2011-12-01 11:56:16.124278 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000080 0 0.000000
  464. 2011-12-01 11:56:16.124332 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  465. 2011-12-01 11:56:16.124375 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  466. 2011-12-01 11:56:16.124440 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  467. 2011-12-01 11:56:16.124476 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  468. 2011-12-01 11:56:16.124514 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  469. 2011-12-01 11:56:16.124547 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  470. 2011-12-01 11:56:16.124603 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  471. 2011-12-01 11:56:16.124637 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  472. 2011-12-01 11:56:16.124661 7f9997d32700 osd.0 5 queue_want_up_thru want 5 <= queued 5, currently 2
  473. 2011-12-01 11:56:16.124709 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000290 0 0.000000
  474. 2011-12-01 11:56:16.124763 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  475. 2011-12-01 11:56:16.124801 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  476. 2011-12-01 11:56:16.124848 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  477. 2011-12-01 11:56:16.125091 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  478. 2011-12-01 11:56:16.125152 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  479. 2011-12-01 11:56:16.125226 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  480. 2011-12-01 11:56:16.125262 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  481. 2011-12-01 11:56:16.125299 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  482. 2011-12-01 11:56:16.125344 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000581 0 0.000000
  483. 2011-12-01 11:56:16.125417 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  484. 2011-12-01 11:56:16.125451 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  485. 2011-12-01 11:56:16.125497 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000
  486. 2011-12-01 11:56:16.125549 7f9997d32700 osd.0 5 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  487. 2011-12-01 11:56:16.125593 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  488. 2011-12-01 11:56:16.125645 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.094802 3 0.000203
  489. 2011-12-01 11:56:16.125706 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  490. 2011-12-01 11:56:16.125749 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  491. 2011-12-01 11:56:16.125782 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  492. 2011-12-01 11:56:16.125827 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000077 0 0.000000
  493. 2011-12-01 11:56:16.125901 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  494. 2011-12-01 11:56:16.125945 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  495. 2011-12-01 11:56:16.125987 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  496. 2011-12-01 11:56:16.126043 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  497. 2011-12-01 11:56:16.126080 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  498. 2011-12-01 11:56:16.126114 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  499. 2011-12-01 11:56:16.126147 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  500. 2011-12-01 11:56:16.126202 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  501. 2011-12-01 11:56:16.126238 7f9997d32700 osd.0 5 queue_want_up_thru want 5 <= queued 5, currently 2
  502. 2011-12-01 11:56:16.126285 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000295 0 0.000000
  503. 2011-12-01 11:56:16.126358 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  504. 2011-12-01 11:56:16.126396 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  505. 2011-12-01 11:56:16.126432 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  506. 2011-12-01 11:56:16.126465 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  507. 2011-12-01 11:56:16.126522 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  508. 2011-12-01 11:56:16.126555 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  509. 2011-12-01 11:56:16.126587 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  510. 2011-12-01 11:56:16.126622 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  511. 2011-12-01 11:56:16.126689 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000330 0 0.000000
  512. 2011-12-01 11:56:16.126740 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  513. 2011-12-01 11:56:16.126772 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  514. 2011-12-01 11:56:16.126816 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000076 0 0.000000
  515. 2011-12-01 11:56:16.126888 7f9997d32700 osd.0 5 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  516. 2011-12-01 11:56:16.126932 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  517. 2011-12-01 11:56:16.126984 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.095506 3 0.000238
  518. 2011-12-01 11:56:16.127033 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  519. 2011-12-01 11:56:16.127075 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  520. 2011-12-01 11:56:16.127117 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  521. 2011-12-01 11:56:16.127167 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000090 0 0.000000
  522. 2011-12-01 11:56:16.127216 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  523. 2011-12-01 11:56:16.127258 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  524. 2011-12-01 11:56:16.127299 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  525. 2011-12-01 11:56:16.127367 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  526. 2011-12-01 11:56:16.127404 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  527. 2011-12-01 11:56:16.127438 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  528. 2011-12-01 11:56:16.127476 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  529. 2011-12-01 11:56:16.127510 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  530. 2011-12-01 11:56:16.127533 7f9997d32700 osd.0 5 queue_want_up_thru want 5 <= queued 5, currently 2
  531. 2011-12-01 11:56:16.127578 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000277 0 0.000000
  532. 2011-12-01 11:56:16.127633 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  533. 2011-12-01 11:56:16.127670 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  534. 2011-12-01 11:56:16.127706 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  535. 2011-12-01 11:56:16.127740 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  536. 2011-12-01 11:56:16.127773 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  537. 2011-12-01 11:56:16.127829 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  538. 2011-12-01 11:56:16.127864 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  539. 2011-12-01 11:56:16.127900 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  540. 2011-12-01 11:56:16.127943 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000313 0 0.000000
  541. 2011-12-01 11:56:16.128016 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  542. 2011-12-01 11:56:16.128050 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  543. 2011-12-01 11:56:16.128095 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000
  544. 2011-12-01 11:56:16.128166 7f9997d32700 osd.0 5 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  545. 2011-12-01 11:56:16.128212 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] handle_activate_map
  546. 2011-12-01 11:56:16.128263 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Reset 1.094439 3 0.000195
  547. 2011-12-01 11:56:16.128355 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started
  548. 2011-12-01 11:56:16.128399 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Start
  549. 2011-12-01 11:56:16.128432 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] state<Start>: transitioning to Primary
  550. 2011-12-01 11:56:16.128503 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] exit Start 0.000101 0 0.000000
  551. 2011-12-01 11:56:16.128554 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary
  552. 2011-12-01 11:56:16.128606 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded] enter Started/Primary/Peering
  553. 2011-12-01 11:56:16.128671 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetInfo
  554. 2011-12-01 11:56:16.128705 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] generate_past_intervals: already have past intervals back to 3
  555. 2011-12-01 11:56:16.128741 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(4-4 []/[])
  556. 2011-12-01 11:56:16.128796 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior interval(2-3 [0]/[0] maybe_went_rw)
  557. 2011-12-01 11:56:16.128831 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] PriorSet: build_prior final: probe 0 down blocked_by {}
  558. 2011-12-01 11:56:16.128867 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] up_thru 2 < same_since 5, must notify monitor
  559. 2011-12-01 11:56:16.129133 7f9997d32700 osd.0 5 queue_want_up_thru want 5 <= queued 5, currently 2
  560. 2011-12-01 11:56:16.129329 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetInfo 0.000650 0 0.000000
  561. 2011-12-01 11:56:16.129381 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetLog
  562. 2011-12-01 11:56:16.129403 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location newest_update 0'0 on osd.0
  563. 2011-12-01 11:56:16.129425 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting best_info is 2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) from osd.0
  564. 2011-12-01 11:56:16.129445 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] osd.0 (up) accepted 2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5)
  565. 2011-12-01 11:56:16.129465 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_acting want [0] (== acting)
  566. 2011-12-01 11:56:16.129485 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] choose_log_location
  567. 2011-12-01 11:56:16.129519 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: neither backlog nor master log needed, moving to GetMissing
  568. 2011-12-01 11:56:16.129541 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
  569. 2011-12-01 11:56:16.129567 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetLog 0.000187 0 0.000000
  570. 2011-12-01 11:56:16.129609 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/GetMissing
  571. 2011-12-01 11:56:16.129630 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering/GetMissing>: still need up_thru update before going active
  572. 2011-12-01 11:56:16.129657 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/GetMissing 0.000047 0 0.000000
  573. 2011-12-01 11:56:16.129700 7f9997d32700 osd.0 5 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Peering/WaitUpThru
  574. 2011-12-01 11:56:16.129732 7f9997d32700 osd.0 5 trim_map_cache prior to 3
  575. 2011-12-01 11:56:16.129749 7f9997d32700 osd.0 5 update_heartbeat_peers: hb to: {}
  576. 2011-12-01 11:56:16.129762 7f9997d32700 osd.0 5 update_heartbeat_peers: hb from: {}
  577. 2011-12-01 11:56:16.129792 7f9997d32700 osd.0 0 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  578. 2011-12-01 11:56:16.129818 7f9997d32700 osd.0 0 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  579. 2011-12-01 11:56:16.129859 7f9997d32700 osd.0 0 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  580. 2011-12-01 11:56:16.129882 7f9997d32700 osd.0 0 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  581. 2011-12-01 11:56:16.129907 7f9997d32700 osd.0 0 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  582. 2011-12-01 11:56:16.129929 7f9997d32700 osd.0 0 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  583. 2011-12-01 11:56:16.129953 7f9997d32700 osd.0 0 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  584. 2011-12-01 11:56:16.129976 7f9997d32700 osd.0 0 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  585. 2011-12-01 11:56:16.130017 7f9997d32700 osd.0 0 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  586. 2011-12-01 11:56:16.130040 7f9997d32700 osd.0 0 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  587. 2011-12-01 11:56:16.130065 7f9997d32700 osd.0 0 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info info 337
  588. 2011-12-01 11:56:16.130090 7f9997d32700 osd.0 0 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] write_info bigbl 60
  589. 2011-12-01 11:56:16.130117 7f9997d32700 osd.0 5 write_superblock sb(99ec31f1-3e67-4f35-9b24-6ca7d38d498b osd.0 e5 [1,5] lci=[5,5])
  590. 2011-12-01 11:56:16.133118 7f9997d32700 osd.0 5 trim_map_bl_cache up to 6
  591. 2011-12-01 11:56:16.133141 7f9997d32700 osd.0 5 trim_map_cache prior to 0
  592. 2011-12-01 11:56:16.133206 7f9997d32700 osd.0 5 unlocking map_in_progress
  593. 2011-12-01 11:56:16.133338 7f9997d32700 osd.0 5 _dispatch 0xd00600 osd_map(4..5 src has 1..5) v2
  594. 2011-12-01 11:56:16.133358 7f9997d32700 osd.0 5 handle_osd_map epochs [4,5], i have 5, src has [1,5]
  595. 2011-12-01 11:56:16.133371 7f9997d32700 osd.0 5 no new maps here, dropping
  596. 2011-12-01 11:56:16.133401 7f9997d32700 osd.0 5 _dispatch 0xd00400 osd_map(5..5 src has 1..5) v2
  597. 2011-12-01 11:56:16.133415 7f9997d32700 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5]
  598. 2011-12-01 11:56:16.133428 7f9997d32700 osd.0 5 no new maps here, dropping
  599. 2011-12-01 11:56:16.439999 7f9992426700 osd.0 5 update_osd_stat osd_stat(167 MB used, 14182 MB avail, 15116 MB total, peers []/[])
  600. 2011-12-01 11:56:16.440051 7f9992426700 osd.0 5 heartbeat: osd_stat(167 MB used, 14182 MB avail, 15116 MB total, peers []/[])
  601. 2011-12-01 11:56:16.440104 7f9992426700 osd.0 5 i have no heartbeat peers; checking mon for new map
  602. 2011-12-01 11:56:17.040040 7f999ed40700 osd.0 5 tick
  603. 2011-12-01 11:56:17.040229 7f999ed40700 osd.0 5 scrub_should_schedule loadavg 0.09 < max 0.5 = yes
  604. 2011-12-01 11:56:17.040263 7f999ed40700 osd.0 5 sched_scrub
  605. 2011-12-01 11:56:17.040340 7f999ed40700 osd.0 5 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:17.040307 (86400 seconds ago)
  606. 2011-12-01 11:56:17.040367 7f999ed40700 osd.0 5 sched_scrub done
  607. 2011-12-01 11:56:17.114457 7f9997d32700 osd.0 5 _dispatch 0xd00000 osd_map(6..6 src has 1..6) v2
  608. 2011-12-01 11:56:17.114513 7f9997d32700 osd.0 5 handle_osd_map epochs [6,6], i have 5, src has [1,6]
  609. 2011-12-01 11:56:17.114589 7f9997d32700 osd.0 5 locking handle_osd_map permissions
  610. 2011-12-01 11:56:17.114624 7f9997d32700 osd.0 5 handle_osd_map got inc map for epoch 6
  611. 2011-12-01 11:56:17.114665 7f9997d32700 osd.0 5 add_map_inc_bl 6 124 bytes
  612. 2011-12-01 11:56:17.114815 7f9997d32700 osd.0 5 add_map 6 0xcf2c00
  613. 2011-12-01 11:56:17.114888 7f9997d32700 osd.0 5 add_map_bl 6 1250 bytes
  614. 2011-12-01 11:56:17.114926 7f9997d32700 osd.0 5 advance to epoch 6 (<= newest 6)
  615. 2011-12-01 11:56:17.114955 7f9997d32700 osd.0 6 advance_map epoch 6 6 pgs
  616. 2011-12-01 11:56:17.114979 7f9997d32700 osd.0 6 pool 0 removed snaps [], unchanged (snap_epoch = 0)
  617. 2011-12-01 11:56:17.115001 7f9997d32700 osd.0 6 pool 1 removed snaps [], unchanged (snap_epoch = 0)
  618. 2011-12-01 11:56:17.115025 7f9997d32700 osd.0 6 pool 2 removed snaps [], unchanged (snap_epoch = 0)
  619. 2011-12-01 11:56:17.115102 7f9997d32700 osd.0 6 Scanning pg pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  620. 2011-12-01 11:56:17.115142 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  621. 2011-12-01 11:56:17.115180 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  622. 2011-12-01 11:56:17.115216 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  623. 2011-12-01 11:56:17.115268 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  624. 2011-12-01 11:56:17.115335 7f9997d32700 osd.0 6 Scanning pg pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  625. 2011-12-01 11:56:17.115374 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  626. 2011-12-01 11:56:17.115409 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  627. 2011-12-01 11:56:17.115446 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  628. 2011-12-01 11:56:17.115480 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  629. 2011-12-01 11:56:17.115521 7f9997d32700 osd.0 6 Scanning pg pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  630. 2011-12-01 11:56:17.115580 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  631. 2011-12-01 11:56:17.115619 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  632. 2011-12-01 11:56:17.115653 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  633. 2011-12-01 11:56:17.115690 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  634. 2011-12-01 11:56:17.115746 7f9997d32700 osd.0 6 Scanning pg pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  635. 2011-12-01 11:56:17.115781 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  636. 2011-12-01 11:56:17.115839 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  637. 2011-12-01 11:56:17.115876 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  638. 2011-12-01 11:56:17.115913 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  639. 2011-12-01 11:56:17.115953 7f9997d32700 osd.0 6 Scanning pg pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  640. 2011-12-01 11:56:17.115990 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  641. 2011-12-01 11:56:17.116026 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  642. 2011-12-01 11:56:17.116082 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  643. 2011-12-01 11:56:17.116118 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  644. 2011-12-01 11:56:17.116159 7f9997d32700 osd.0 6 Scanning pg pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering]
  645. 2011-12-01 11:56:17.116193 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_advance_map [0]/[0]
  646. 2011-12-01 11:56:17.116231 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Peering advmap
  647. 2011-12-01 11:56:17.116266 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] adjust_need_up_thru now 5, need_up_thru now false
  648. 2011-12-01 11:56:17.116327 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started>: Started advmap
  649. 2011-12-01 11:56:17.116365 7f9997d32700 osd.0 6 activate_map version 6
  650. 2011-12-01 11:56:17.116400 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  651. 2011-12-01 11:56:17.116463 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.993869 2 0.001551
  652. 2011-12-01 11:56:17.116510 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  653. 2011-12-01 11:56:17.116579 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995106 0 0.000000
  654. 2011-12-01 11:56:17.116643 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  655. 2011-12-01 11:56:17.116679 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  656. 2011-12-01 11:56:17.116719 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] cancel_generate_backlog
  657. 2011-12-01 11:56:17.116762 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info info 337
  658. 2011-12-01 11:56:17.116839 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info bigbl 60
  659. 2011-12-01 11:56:17.117265 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log
  660. 2011-12-01 11:56:17.117366 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log to 0~0
  661. 2011-12-01 11:56:17.117474 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] clean_up_local
  662. 2011-12-01 11:56:17.117544 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - snap_trimq []
  663. 2011-12-01 11:56:17.117624 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - complete
  664. 2011-12-01 11:56:17.117683 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] is_all_uptodate: everyone is uptodate
  665. 2011-12-01 11:56:17.117741 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] finish_recovery
  666. 2011-12-01 11:56:17.117826 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  667. 2011-12-01 11:56:17.117880 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  668. 2011-12-01 11:56:17.117937 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  669. 2011-12-01 11:56:17.117992 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  670. 2011-12-01 11:56:17.118097 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  671. 2011-12-01 11:56:17.118153 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  672. 2011-12-01 11:56:17.118219 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'5
  673. 2011-12-01 11:56:17.118266 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  674. 2011-12-01 11:56:17.118341 7f9997d32700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Activate Finished
  675. 2011-12-01 11:56:17.118391 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  676. 2011-12-01 11:56:17.118475 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.994531 2 0.001319
  677. 2011-12-01 11:56:17.118573 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  678. 2011-12-01 11:56:17.118639 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995640 0 0.000000
  679. 2011-12-01 11:56:17.118720 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  680. 2011-12-01 11:56:17.118776 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  681. 2011-12-01 11:56:17.118866 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay interval(4-4 []/[])
  682. 2011-12-01 11:56:17.118917 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay interval(2-3 [0]/[0] maybe_went_rw)
  683. 2011-12-01 11:56:17.118977 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay no known survivors of interval 2-3, may need replay
  684. 2011-12-01 11:56:17.119027 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] activate starting replay interval for 45 until 2011-12-01 11:57:02.119004
  685. 2011-12-01 11:56:17.119085 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] cancel_generate_backlog
  686. 2011-12-01 11:56:17.119128 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_info info 337
  687. 2011-12-01 11:56:17.119166 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_info bigbl 60
  688. 2011-12-01 11:56:17.119206 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_log
  689. 2011-12-01 11:56:17.119246 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_log to 0~0
  690. 2011-12-01 11:56:17.119281 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] clean_up_local
  691. 2011-12-01 11:56:17.119318 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] activate - snap_trimq []
  692. 2011-12-01 11:56:17.119352 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] activate - complete
  693. 2011-12-01 11:56:17.119386 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] is_all_uptodate: everyone is uptodate
  694. 2011-12-01 11:56:17.119422 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] finish_recovery
  695. 2011-12-01 11:56:17.119456 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  696. 2011-12-01 11:56:17.119491 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] clear_recovery_state
  697. 2011-12-01 11:56:17.119529 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  698. 2011-12-01 11:56:17.119565 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  699. 2011-12-01 11:56:17.119608 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  700. 2011-12-01 11:56:17.119645 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  701. 2011-12-01 11:56:17.119687 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'5
  702. 2011-12-01 11:56:17.119725 7f9997d32700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Activate Finished
  703. 2011-12-01 11:56:17.119769 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  704. 2011-12-01 11:56:17.119849 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.994295 2 0.001552
  705. 2011-12-01 11:56:17.119894 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  706. 2011-12-01 11:56:17.119948 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995572 0 0.000000
  707. 2011-12-01 11:56:17.120002 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  708. 2011-12-01 11:56:17.120063 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  709. 2011-12-01 11:56:17.120102 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay interval(4-4 []/[])
  710. 2011-12-01 11:56:17.120137 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay interval(2-3 [0]/[0] maybe_went_rw)
  711. 2011-12-01 11:56:17.120170 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] may_need_replay no known survivors of interval 2-3, may need replay
  712. 2011-12-01 11:56:17.120214 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] activate starting replay interval for 45 until 2011-12-01 11:57:02.120190
  713. 2011-12-01 11:56:17.120246 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] cancel_generate_backlog
  714. 2011-12-01 11:56:17.120311 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_info info 337
  715. 2011-12-01 11:56:17.120351 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_info bigbl 60
  716. 2011-12-01 11:56:17.120389 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_log
  717. 2011-12-01 11:56:17.120425 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] write_log to 0~0
  718. 2011-12-01 11:56:17.120459 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] clean_up_local
  719. 2011-12-01 11:56:17.120495 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] activate - snap_trimq []
  720. 2011-12-01 11:56:17.120550 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] activate - complete
  721. 2011-12-01 11:56:17.120585 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] is_all_uptodate: everyone is uptodate
  722. 2011-12-01 11:56:17.120621 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+replay+degraded] finish_recovery
  723. 2011-12-01 11:56:17.120654 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  724. 2011-12-01 11:56:17.120689 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] clear_recovery_state
  725. 2011-12-01 11:56:17.120723 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  726. 2011-12-01 11:56:17.120762 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  727. 2011-12-01 11:56:17.120807 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  728. 2011-12-01 11:56:17.120890 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  729. 2011-12-01 11:56:17.121017 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'5
  730. 2011-12-01 11:56:17.121097 7f9997d32700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Activate Finished
  731. 2011-12-01 11:56:17.121157 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  732. 2011-12-01 11:56:17.121215 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.994326 2 0.001296
  733. 2011-12-01 11:56:17.121261 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  734. 2011-12-01 11:56:17.121328 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995371 0 0.000000
  735. 2011-12-01 11:56:17.121412 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  736. 2011-12-01 11:56:17.121450 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  737. 2011-12-01 11:56:17.121485 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] cancel_generate_backlog
  738. 2011-12-01 11:56:17.121530 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info info 337
  739. 2011-12-01 11:56:17.121573 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info bigbl 60
  740. 2011-12-01 11:56:17.121613 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log
  741. 2011-12-01 11:56:17.121650 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log to 0~0
  742. 2011-12-01 11:56:17.121684 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] clean_up_local
  743. 2011-12-01 11:56:17.121720 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - snap_trimq []
  744. 2011-12-01 11:56:17.121752 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - complete
  745. 2011-12-01 11:56:17.121812 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] is_all_uptodate: everyone is uptodate
  746. 2011-12-01 11:56:17.121850 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] finish_recovery
  747. 2011-12-01 11:56:17.121883 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  748. 2011-12-01 11:56:17.121919 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  749. 2011-12-01 11:56:17.121925 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  750. 2011-12-01 11:56:17.122031 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  751. 2011-12-01 11:56:17.122097 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  752. 2011-12-01 11:56:17.122153 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  753. 2011-12-01 11:56:17.122254 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'5
  754. 2011-12-01 11:56:17.122315 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  755. 2011-12-01 11:56:17.122369 7f9997d32700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Activate Finished
  756. 2011-12-01 11:56:17.122436 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  757. 2011-12-01 11:56:17.122544 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.994365 2 0.001237
  758. 2011-12-01 11:56:17.122612 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  759. 2011-12-01 11:56:17.122710 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995439 0 0.000000
  760. 2011-12-01 11:56:17.122808 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  761. 2011-12-01 11:56:17.122850 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  762. 2011-12-01 11:56:17.122892 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] cancel_generate_backlog
  763. 2011-12-01 11:56:17.122972 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info info 337
  764. 2011-12-01 11:56:17.123083 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info bigbl 60
  765. 2011-12-01 11:56:17.123139 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log
  766. 2011-12-01 11:56:17.123191 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log to 0~0
  767. 2011-12-01 11:56:17.123255 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] clean_up_local
  768. 2011-12-01 11:56:17.123308 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - snap_trimq []
  769. 2011-12-01 11:56:17.123358 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - complete
  770. 2011-12-01 11:56:17.123410 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] is_all_uptodate: everyone is uptodate
  771. 2011-12-01 11:56:17.123524 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] finish_recovery
  772. 2011-12-01 11:56:17.123575 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  773. 2011-12-01 11:56:17.123615 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  774. 2011-12-01 11:56:17.123652 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  775. 2011-12-01 11:56:17.123704 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  776. 2011-12-01 11:56:17.123807 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  777. 2011-12-01 11:56:17.123921 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  778. 2011-12-01 11:56:17.124014 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'5
  779. 2011-12-01 11:56:17.124065 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  780. 2011-12-01 11:56:17.124116 7f9997d32700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Activate Finished
  781. 2011-12-01 11:56:17.124183 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] handle_activate_map
  782. 2011-12-01 11:56:17.124293 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering/WaitUpThru 0.994569 2 0.001481
  783. 2011-12-01 11:56:17.124362 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Peering>: Leaving Peering
  784. 2011-12-01 11:56:17.124427 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] exit Started/Primary/Peering 0.995809 0 0.000000
  785. 2011-12-01 11:56:17.124512 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] enter Started/Primary/Active
  786. 2011-12-01 11:56:17.124564 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml degraded+peering] state<Started/Primary/Active>: In Active, about to call activate
  787. 2011-12-01 11:56:17.124616 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] cancel_generate_backlog
  788. 2011-12-01 11:56:17.124688 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info info 337
  789. 2011-12-01 11:56:17.124786 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_info bigbl 60
  790. 2011-12-01 11:56:17.124842 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log
  791. 2011-12-01 11:56:17.125057 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] write_log to 0~0
  792. 2011-12-01 11:56:17.125112 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] clean_up_local
  793. 2011-12-01 11:56:17.125166 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - snap_trimq []
  794. 2011-12-01 11:56:17.125464 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] activate - complete
  795. 2011-12-01 11:56:17.125550 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] is_all_uptodate: everyone is uptodate
  796. 2011-12-01 11:56:17.125596 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/3 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+degraded] finish_recovery
  797. 2011-12-01 11:56:17.125624 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  798. 2011-12-01 11:56:17.125645 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  799. 2011-12-01 11:56:17.125689 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(2-3 [0]/[0] maybe_went_rw)
  800. 2011-12-01 11:56:17.125724 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] trim_past_intervals: trimming interval(4-4 []/[])
  801. 2011-12-01 11:56:17.125776 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  802. 2011-12-01 11:56:17.125827 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  803. 2011-12-01 11:56:17.125854 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'5
  804. 2011-12-01 11:56:17.125882 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  805. 2011-12-01 11:56:17.125916 7f9997d32700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Activate Finished
  806. 2011-12-01 11:56:17.125951 7f9997d32700 osd.0 6 trim_map_cache prior to 3
  807. 2011-12-01 11:56:17.125976 7f9997d32700 osd.0 6 update_heartbeat_peers: hb to: {}
  808. 2011-12-01 11:56:17.126029 7f9997d32700 osd.0 6 update_heartbeat_peers: hb from: {}
  809. 2011-12-01 11:56:17.126062 7f9997d32700 osd.0 6 write_superblock sb(99ec31f1-3e67-4f35-9b24-6ca7d38d498b osd.0 e6 [1,6] lci=[5,6])
  810. 2011-12-01 11:56:17.136735 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  811. 2011-12-01 11:56:17.136793 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] all_activated_and_committed
  812. 2011-12-01 11:56:17.136827 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  813. 2011-12-01 11:56:17.137069 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  814. 2011-12-01 11:56:17.137133 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  815. 2011-12-01 11:56:17.137216 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  816. 2011-12-01 11:56:17.137264 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  817. 2011-12-01 11:56:17.137298 7f9998d34700 osd.0 6 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'6
  818. 2011-12-01 11:56:17.137327 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  819. 2011-12-01 11:56:17.137538 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] all_activated_and_committed
  820. 2011-12-01 11:56:17.137569 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  821. 2011-12-01 11:56:17.137625 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  822. 2011-12-01 11:56:17.137738 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  823. 2011-12-01 11:56:17.137945 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _finish_recovery
  824. 2011-12-01 11:56:17.137996 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] purge_strays
  825. 2011-12-01 11:56:17.138541 7f9998d34700 osd.0 6 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'6
  826. 2011-12-01 11:56:17.138597 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  827. 2011-12-01 11:56:17.138661 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] all_activated_and_committed
  828. 2011-12-01 11:56:17.138714 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  829. 2011-12-01 11:56:17.138884 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  830. 2011-12-01 11:56:17.138960 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  831. 2011-12-01 11:56:17.139428 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _finish_recovery
  832. 2011-12-01 11:56:17.139469 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] purge_strays
  833. 2011-12-01 11:56:17.139509 7f9998d34700 osd.0 6 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'6
  834. 2011-12-01 11:56:17.139747 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  835. 2011-12-01 11:56:17.139786 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] all_activated_and_committed
  836. 2011-12-01 11:56:17.139924 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  837. 2011-12-01 11:56:17.140106 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  838. 2011-12-01 11:56:17.140148 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  839. 2011-12-01 11:56:17.140232 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  840. 2011-12-01 11:56:17.140281 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  841. 2011-12-01 11:56:17.140429 7f9998d34700 osd.0 6 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'6
  842. 2011-12-01 11:56:17.140622 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  843. 2011-12-01 11:56:17.140654 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] all_activated_and_committed
  844. 2011-12-01 11:56:17.140819 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  845. 2011-12-01 11:56:17.140874 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  846. 2011-12-01 11:56:17.140993 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  847. 2011-12-01 11:56:17.141071 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  848. 2011-12-01 11:56:17.141125 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  849. 2011-12-01 11:56:17.141176 7f9998d34700 osd.0 6 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'6
  850. 2011-12-01 11:56:17.141401 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _activate_committed 5 peer_activated now 0 last_epoch_started 3 same_interval_since 5
  851. 2011-12-01 11:56:17.141454 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 3/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] all_activated_and_committed
  852. 2011-12-01 11:56:17.141485 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  853. 2011-12-01 11:56:17.141531 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  854. 2011-12-01 11:56:17.141666 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  855. 2011-12-01 11:56:17.141857 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  856. 2011-12-01 11:56:17.142113 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  857. 2011-12-01 11:56:17.142158 7f9998d34700 osd.0 6 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'6
  858. 2011-12-01 11:56:17.142243 7f9997d32700 osd.0 6 trim_map_bl_cache up to 7
  859. 2011-12-01 11:56:17.142277 7f9997d32700 osd.0 6 trim_map_cache prior to 0
  860. 2011-12-01 11:56:17.142321 7f9997d32700 osd.0 6 unlocking map_in_progress
  861. 2011-12-01 11:56:17.142350 7f9997d32700 osd.0 6 _dispatch 0xd00800 osd_map(5..6 src has 1..6) v2
  862. 2011-12-01 11:56:17.142365 7f9997d32700 osd.0 6 handle_osd_map epochs [5,6], i have 6, src has [1,6]
  863. 2011-12-01 11:56:17.142392 7f9997d32700 osd.0 6 no new maps here, dropping
  864. 2011-12-01 11:56:17.142411 7f9997d32700 osd.0 6 _dispatch 0xd00400 osd_map(5..6 src has 1..6) v2
  865. 2011-12-01 11:56:17.142424 7f9997d32700 osd.0 6 handle_osd_map epochs [5,6], i have 6, src has [1,6]
  866. 2011-12-01 11:56:17.142448 7f9997d32700 osd.0 6 no new maps here, dropping
  867. 2011-12-01 11:56:17.840497 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  868. 2011-12-01 11:56:17.840558 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  869. 2011-12-01 11:56:18.040687 7f999ed40700 osd.0 6 tick
  870. 2011-12-01 11:56:18.041155 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = no, randomly backing off
  871. 2011-12-01 11:56:19.041537 7f999ed40700 osd.0 6 tick
  872. 2011-12-01 11:56:19.041748 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = no, randomly backing off
  873. 2011-12-01 11:56:19.240968 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  874. 2011-12-01 11:56:19.241029 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  875. 2011-12-01 11:56:20.042111 7f999ed40700 osd.0 6 tick
  876. 2011-12-01 11:56:20.042300 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = no, randomly backing off
  877. 2011-12-01 11:56:20.241423 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  878. 2011-12-01 11:56:20.241503 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  879. 2011-12-01 11:56:21.042714 7f999ed40700 osd.0 6 tick
  880. 2011-12-01 11:56:21.042976 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = yes
  881. 2011-12-01 11:56:21.043016 7f999ed40700 osd.0 6 sched_scrub
  882. 2011-12-01 11:56:21.043078 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:21.043045 (86400 seconds ago)
  883. 2011-12-01 11:56:21.043113 7f999ed40700 osd.0 6 sched_scrub done
  884. 2011-12-01 11:56:21.641882 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  885. 2011-12-01 11:56:21.641987 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  886. 2011-12-01 11:56:22.043368 7f999ed40700 osd.0 6 tick
  887. 2011-12-01 11:56:22.043584 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = yes
  888. 2011-12-01 11:56:22.043617 7f999ed40700 osd.0 6 sched_scrub
  889. 2011-12-01 11:56:22.043657 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:22.043635 (86400 seconds ago)
  890. 2011-12-01 11:56:22.043696 7f999ed40700 osd.0 6 sched_scrub done
  891. 2011-12-01 11:56:22.043737 7f999ed40700 osd.0 6 do_mon_report
  892. 2011-12-01 11:56:22.043762 7f999ed40700 osd.0 6 send_alive up_thru currently 5 want 5
  893. 2011-12-01 11:56:22.043780 7f999ed40700 osd.0 6 send_pg_stats
  894. 2011-12-01 11:56:22.043799 7f999ed40700 osd.0 6 send_pg_stats - 6 pgs updated
  895. 2011-12-01 11:56:22.103223 7f9997d32700 osd.0 6 _dispatch 0xd10380 pg_stats_ack(6 pgs tid 2) v1
  896. 2011-12-01 11:56:22.103254 7f9997d32700 osd.0 6 handle_pg_stats_ack
  897. 2011-12-01 11:56:22.442267 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  898. 2011-12-01 11:56:22.442319 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  899. 2011-12-01 11:56:23.044078 7f999ed40700 osd.0 6 tick
  900. 2011-12-01 11:56:23.044296 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = no, randomly backing off
  901. 2011-12-01 11:56:23.842643 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  902. 2011-12-01 11:56:23.842701 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  903. 2011-12-01 11:56:24.044530 7f999ed40700 osd.0 6 tick
  904. 2011-12-01 11:56:24.044761 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = no, randomly backing off
  905. 2011-12-01 11:56:25.045066 7f999ed40700 osd.0 6 tick
  906. 2011-12-01 11:56:25.045237 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = yes
  907. 2011-12-01 11:56:25.045263 7f999ed40700 osd.0 6 sched_scrub
  908. 2011-12-01 11:56:25.045305 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:25.045281 (86400 seconds ago)
  909. 2011-12-01 11:56:25.045327 7f999ed40700 osd.0 6 sched_scrub done
  910. 2011-12-01 11:56:25.243052 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  911. 2011-12-01 11:56:25.243132 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  912. 2011-12-01 11:56:26.045777 7f999ed40700 osd.0 6 tick
  913. 2011-12-01 11:56:26.045987 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.09 < max 0.5 = yes
  914. 2011-12-01 11:56:26.046012 7f999ed40700 osd.0 6 sched_scrub
  915. 2011-12-01 11:56:26.046050 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:26.046029 (86400 seconds ago)
  916. 2011-12-01 11:56:26.046070 7f999ed40700 osd.0 6 sched_scrub done
  917. 2011-12-01 11:56:26.146357 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  918. 2011-12-01 11:56:26.146487 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  919. 2011-12-01 11:56:27.046305 7f999ed40700 osd.0 6 tick
  920. 2011-12-01 11:56:27.046539 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.08 < max 0.5 = yes
  921. 2011-12-01 11:56:27.046610 7f999ed40700 osd.0 6 sched_scrub
  922. 2011-12-01 11:56:27.046659 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:27.046635 (86400 seconds ago)
  923. 2011-12-01 11:56:27.046688 7f999ed40700 osd.0 6 sched_scrub done
  924. 2011-12-01 11:56:27.046737 7f999ed40700 osd.0 6 do_mon_report
  925. 2011-12-01 11:56:27.046804 7f999ed40700 osd.0 6 send_alive up_thru currently 5 want 5
  926. 2011-12-01 11:56:27.046831 7f999ed40700 osd.0 6 send_pg_stats
  927. 2011-12-01 11:56:27.246906 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  928. 2011-12-01 11:56:27.246994 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  929. 2011-12-01 11:56:28.047098 7f999ed40700 osd.0 6 tick
  930. 2011-12-01 11:56:28.047293 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.08 < max 0.5 = no, randomly backing off
  931. 2011-12-01 11:56:28.249802 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  932. 2011-12-01 11:56:28.249942 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  933. 2011-12-01 11:56:29.047567 7f999ed40700 osd.0 6 tick
  934. 2011-12-01 11:56:29.047813 7f999ed40700 osd.0 6 scrub_should_schedule loadavg 0.08 < max 0.5 = yes
  935. 2011-12-01 11:56:29.047851 7f999ed40700 osd.0 6 sched_scrub
  936. 2011-12-01 11:56:29.047916 7f999ed40700 osd.0 6 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:29.047884 (86400 seconds ago)
  937. 2011-12-01 11:56:29.047953 7f999ed40700 osd.0 6 sched_scrub done
  938. 2011-12-01 11:56:29.151600 7f9992426700 osd.0 6 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  939. 2011-12-01 11:56:29.151763 7f9992426700 osd.0 6 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  940. 2011-12-01 11:56:29.682294 7f9997d32700 osd.0 6 _dispatch 0xd00600 osd_map(6..7 src has 1..7) v2
  941. 2011-12-01 11:56:29.682388 7f9997d32700 osd.0 6 handle_osd_map epochs [6,7], i have 6, src has [1,7]
  942. 2011-12-01 11:56:29.682426 7f9997d32700 osd.0 6 locking handle_osd_map permissions
  943. 2011-12-01 11:56:29.682461 7f9997d32700 osd.0 6 handle_osd_map got inc map for epoch 7
  944. 2011-12-01 11:56:29.682528 7f9997d32700 osd.0 6 add_map_inc_bl 7 260 bytes
  945. 2011-12-01 11:56:29.682657 7f9997d32700 osd.0 6 add_map 7 0xd11c00
  946. 2011-12-01 11:56:29.682725 7f9997d32700 osd.0 6 add_map_bl 7 1394 bytes
  947. 2011-12-01 11:56:29.682853 7f9997d32700 osd.0 6 advance to epoch 7 (<= newest 7)
  948. 2011-12-01 11:56:29.682893 7f9997d32700 osd.0 7 advance_map epoch 7 6 pgs
  949. 2011-12-01 11:56:29.682959 7f9997d32700 osd.0 7 pool 0 removed snaps [], unchanged (snap_epoch = 0)
  950. 2011-12-01 11:56:29.682994 7f9997d32700 osd.0 7 pool 1 removed snaps [], unchanged (snap_epoch = 0)
  951. 2011-12-01 11:56:29.683027 7f9997d32700 osd.0 7 pool 2 removed snaps [], unchanged (snap_epoch = 0)
  952. 2011-12-01 11:56:29.683101 7f9997d32700 osd.0 7 Scanning pg pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded]
  953. 2011-12-01 11:56:29.683157 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_advance_map [0]/[0]
  954. 2011-12-01 11:56:29.683208 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active advmap
  955. 2011-12-01 11:56:29.683246 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started>: Started advmap
  956. 2011-12-01 11:56:29.683289 7f9997d32700 osd.0 7 Scanning pg pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded]
  957. 2011-12-01 11:56:29.683323 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] handle_advance_map [0]/[0]
  958. 2011-12-01 11:56:29.683357 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active advmap
  959. 2011-12-01 11:56:29.683394 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started>: Started advmap
  960. 2011-12-01 11:56:29.683444 7f9997d32700 osd.0 7 Scanning pg pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded]
  961. 2011-12-01 11:56:29.683496 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] handle_advance_map [0]/[0]
  962. 2011-12-01 11:56:29.683548 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active advmap
  963. 2011-12-01 11:56:29.683600 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started>: Started advmap
  964. 2011-12-01 11:56:29.683657 7f9997d32700 osd.0 7 Scanning pg pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded]
  965. 2011-12-01 11:56:29.683709 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_advance_map [0]/[0]
  966. 2011-12-01 11:56:29.683810 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active advmap
  967. 2011-12-01 11:56:29.683862 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started>: Started advmap
  968. 2011-12-01 11:56:29.683971 7f9997d32700 osd.0 7 Scanning pg pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded]
  969. 2011-12-01 11:56:29.684026 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_advance_map [0]/[0]
  970. 2011-12-01 11:56:29.684065 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active advmap
  971. 2011-12-01 11:56:29.684099 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started>: Started advmap
  972. 2011-12-01 11:56:29.684148 7f9997d32700 osd.0 7 Scanning pg pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded]
  973. 2011-12-01 11:56:29.684190 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_advance_map [0]/[0]
  974. 2011-12-01 11:56:29.684246 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active advmap
  975. 2011-12-01 11:56:29.684299 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started>: Started advmap
  976. 2011-12-01 11:56:29.684397 7f9997d32700 osd.0 7 activate_map version 7
  977. 2011-12-01 11:56:29.684451 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_activate_map
  978. 2011-12-01 11:56:29.684582 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: handling ActMap
  979. 2011-12-01 11:56:29.684619 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] is_all_uptodate: everyone is uptodate
  980. 2011-12-01 11:56:29.684689 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  981. 2011-12-01 11:56:29.684723 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] finish_recovery
  982. 2011-12-01 11:56:29.684793 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  983. 2011-12-01 11:56:29.684827 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  984. 2011-12-01 11:56:29.685133 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  985. 2011-12-01 11:56:29.685182 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  986. 2011-12-01 11:56:29.685226 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting best_info is 1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  987. 2011-12-01 11:56:29.685294 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] osd.0 (up) accepted 1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  988. 2011-12-01 11:56:29.685330 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting want [0] (== acting)
  989. 2011-12-01 11:56:29.685363 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary>: handle ActMap primary
  990. 2011-12-01 11:56:29.685438 7f9997d32700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'7
  991. 2011-12-01 11:56:29.685482 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] handle_activate_map
  992. 2011-12-01 11:56:29.685523 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active: handling ActMap
  993. 2011-12-01 11:56:29.685555 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] is_all_uptodate: everyone is uptodate
  994. 2011-12-01 11:56:29.685588 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  995. 2011-12-01 11:56:29.685622 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] finish_recovery
  996. 2011-12-01 11:56:29.685653 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  997. 2011-12-01 11:56:29.685685 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] clear_recovery_state
  998. 2011-12-01 11:56:29.685767 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  999. 2011-12-01 11:56:29.685825 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  1000. 2011-12-01 11:56:29.685954 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] choose_acting best_info is 0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  1001. 2011-12-01 11:56:29.686036 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] osd.0 (up) accepted 0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  1002. 2011-12-01 11:56:29.686076 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] choose_acting want [0] (== acting)
  1003. 2011-12-01 11:56:29.686111 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary>: handle ActMap primary
  1004. 2011-12-01 11:56:29.686152 7f9997d32700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'7
  1005. 2011-12-01 11:56:29.686214 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] handle_activate_map
  1006. 2011-12-01 11:56:29.686328 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active: handling ActMap
  1007. 2011-12-01 11:56:29.686371 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] is_all_uptodate: everyone is uptodate
  1008. 2011-12-01 11:56:29.686406 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  1009. 2011-12-01 11:56:29.686438 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] finish_recovery
  1010. 2011-12-01 11:56:29.686470 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] share_pg_info
  1011. 2011-12-01 11:56:29.686502 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] clear_recovery_state
  1012. 2011-12-01 11:56:29.686552 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info info 337
  1013. 2011-12-01 11:56:29.686590 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] write_info bigbl 8
  1014. 2011-12-01 11:56:29.686630 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] choose_acting best_info is 0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  1015. 2011-12-01 11:56:29.686665 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] osd.0 (up) accepted 0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  1016. 2011-12-01 11:56:29.686699 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] choose_acting want [0] (== acting)
  1017. 2011-12-01 11:56:29.686732 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] state<Started/Primary>: handle ActMap primary
  1018. 2011-12-01 11:56:29.686767 7f9997d32700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'7
  1019. 2011-12-01 11:56:29.686823 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_activate_map
  1020. 2011-12-01 11:56:29.686957 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: handling ActMap
  1021. 2011-12-01 11:56:29.687004 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] is_all_uptodate: everyone is uptodate
  1022. 2011-12-01 11:56:29.687039 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  1023. 2011-12-01 11:56:29.687106 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] finish_recovery
  1024. 2011-12-01 11:56:29.687140 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  1025. 2011-12-01 11:56:29.687172 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  1026. 2011-12-01 11:56:29.687249 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  1027. 2011-12-01 11:56:29.687287 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  1028. 2011-12-01 11:56:29.687327 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting best_info is 1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  1029. 2011-12-01 11:56:29.687362 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] osd.0 (up) accepted 1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  1030. 2011-12-01 11:56:29.687395 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting want [0] (== acting)
  1031. 2011-12-01 11:56:29.687428 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary>: handle ActMap primary
  1032. 2011-12-01 11:56:29.687469 7f9997d32700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'7
  1033. 2011-12-01 11:56:29.687522 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_activate_map
  1034. 2011-12-01 11:56:29.687584 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: handling ActMap
  1035. 2011-12-01 11:56:29.687649 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] is_all_uptodate: everyone is uptodate
  1036. 2011-12-01 11:56:29.687711 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  1037. 2011-12-01 11:56:29.687757 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] finish_recovery
  1038. 2011-12-01 11:56:29.687837 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  1039. 2011-12-01 11:56:29.687901 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  1040. 2011-12-01 11:56:29.687946 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  1041. 2011-12-01 11:56:29.687982 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  1042. 2011-12-01 11:56:29.688022 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting best_info is 2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  1043. 2011-12-01 11:56:29.688074 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] osd.0 (up) accepted 2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  1044. 2011-12-01 11:56:29.688107 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting want [0] (== acting)
  1045. 2011-12-01 11:56:29.688141 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary>: handle ActMap primary
  1046. 2011-12-01 11:56:29.688174 7f9997d32700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'7
  1047. 2011-12-01 11:56:29.688216 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] handle_activate_map
  1048. 2011-12-01 11:56:29.688256 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: handling ActMap
  1049. 2011-12-01 11:56:29.688288 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] is_all_uptodate: everyone is uptodate
  1050. 2011-12-01 11:56:29.688320 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary/Active>: Active: all up to date, going clean
  1051. 2011-12-01 11:56:29.688352 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/6 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] finish_recovery
  1052. 2011-12-01 11:56:29.688384 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] share_pg_info
  1053. 2011-12-01 11:56:29.688415 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] clear_recovery_state
  1054. 2011-12-01 11:56:29.688453 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info info 337
  1055. 2011-12-01 11:56:29.688489 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] write_info bigbl 8
  1056. 2011-12-01 11:56:29.688528 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting best_info is 2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) from osd.0
  1057. 2011-12-01 11:56:29.688562 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] osd.0 (up) accepted 2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5)
  1058. 2011-12-01 11:56:29.688605 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] choose_acting want [0] (== acting)
  1059. 2011-12-01 11:56:29.688639 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] state<Started/Primary>: handle ActMap primary
  1060. 2011-12-01 11:56:29.688673 7f9997d32700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'7
  1061. 2011-12-01 11:56:29.688708 7f9997d32700 osd.0 7 trim_map_cache prior to 6
  1062. 2011-12-01 11:56:29.688729 7f9997d32700 osd.0 7 trim_map_cache 3 0xcf2000
  1063. 2011-12-01 11:56:29.688765 7f9997d32700 osd.0 7 trim_map_cache 4 0xcf2300
  1064. 2011-12-01 11:56:29.688795 7f9997d32700 osd.0 7 trim_map_cache 5 0xd11300
  1065. 2011-12-01 11:56:29.688830 7f9997d32700 osd.0 7 update_heartbeat_peers: hb to: {}
  1066. 2011-12-01 11:56:29.688852 7f9997d32700 osd.0 7 update_heartbeat_peers: hb from: {}
  1067. 2011-12-01 11:56:29.689296 7f9997d32700 osd.0 7 write_superblock sb(99ec31f1-3e67-4f35-9b24-6ca7d38d498b osd.0 e7 [1,7] lci=[5,7])
  1068. 2011-12-01 11:56:29.698071 7f9998d34700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  1069. 2011-12-01 11:56:29.698116 7f9998d34700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  1070. 2011-12-01 11:56:29.698151 7f9998d34700 osd.0 7 pg[1.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'8
  1071. 2011-12-01 11:56:29.698326 7f9998d34700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _finish_recovery
  1072. 2011-12-01 11:56:29.698360 7f9998d34700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] purge_strays
  1073. 2011-12-01 11:56:29.698393 7f9998d34700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'8
  1074. 2011-12-01 11:56:29.698427 7f9998d34700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] _finish_recovery
  1075. 2011-12-01 11:56:29.698459 7f9998d34700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] purge_strays
  1076. 2011-12-01 11:56:29.698535 7f9998d34700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] update_stats 5'8
  1077. 2011-12-01 11:56:29.698609 7f9998d34700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  1078. 2011-12-01 11:56:29.698640 7f9998d34700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  1079. 2011-12-01 11:56:29.698674 7f9998d34700 osd.0 7 pg[1.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'8
  1080. 2011-12-01 11:56:29.698706 7f9998d34700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  1081. 2011-12-01 11:56:29.698760 7f9998d34700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  1082. 2011-12-01 11:56:29.698793 7f9998d34700 osd.0 7 pg[2.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'8
  1083. 2011-12-01 11:56:29.698825 7f9998d34700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] _finish_recovery
  1084. 2011-12-01 11:56:29.698857 7f9998d34700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] purge_strays
  1085. 2011-12-01 11:56:29.698890 7f9998d34700 osd.0 7 pg[2.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'8
  1086. 2011-12-01 11:56:29.699004 7f9997d32700 osd.0 7 trim_map_bl_cache up to 8
  1087. 2011-12-01 11:56:29.699033 7f9997d32700 osd.0 7 trim_map_cache prior to 0
  1088. 2011-12-01 11:56:29.699066 7f9997d32700 osd.0 7 unlocking map_in_progress
  1089. 2011-12-01 11:56:29.954469 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1090. 2011-12-01 11:56:29.954801 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1091. 2011-12-01 11:56:30.048429 7f999ed40700 osd.0 7 tick
  1092. 2011-12-01 11:56:30.050826 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.08 < max 0.5 = no, randomly backing off
  1093. 2011-12-01 11:56:31.052985 7f999ed40700 osd.0 7 tick
  1094. 2011-12-01 11:56:31.053202 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.08 < max 0.5 = yes
  1095. 2011-12-01 11:56:31.053237 7f999ed40700 osd.0 7 sched_scrub
  1096. 2011-12-01 11:56:31.053285 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:31.053261 (86400 seconds ago)
  1097. 2011-12-01 11:56:31.053311 7f999ed40700 osd.0 7 sched_scrub done
  1098. 2011-12-01 11:56:31.355220 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1099. 2011-12-01 11:56:31.355309 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1100. 2011-12-01 11:56:32.053699 7f999ed40700 osd.0 7 tick
  1101. 2011-12-01 11:56:32.054016 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = yes
  1102. 2011-12-01 11:56:32.054170 7f999ed40700 osd.0 7 sched_scrub
  1103. 2011-12-01 11:56:32.054238 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:32.054206 (86400 seconds ago)
  1104. 2011-12-01 11:56:32.054276 7f999ed40700 osd.0 7 sched_scrub done
  1105. 2011-12-01 11:56:32.054327 7f999ed40700 osd.0 7 do_mon_report
  1106. 2011-12-01 11:56:32.054376 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1107. 2011-12-01 11:56:32.054447 7f999ed40700 osd.0 7 send_pg_stats
  1108. 2011-12-01 11:56:32.054472 7f999ed40700 osd.0 7 send_pg_stats - 6 pgs updated
  1109. 2011-12-01 11:56:32.116209 7f9997d32700 osd.0 7 _dispatch 0xd10540 pg_stats_ack(6 pgs tid 3) v1
  1110. 2011-12-01 11:56:32.116269 7f9997d32700 osd.0 7 handle_pg_stats_ack
  1111. 2011-12-01 11:56:32.755785 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1112. 2011-12-01 11:56:32.755982 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1113. 2011-12-01 11:56:33.054860 7f999ed40700 osd.0 7 tick
  1114. 2011-12-01 11:56:33.055016 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = yes
  1115. 2011-12-01 11:56:33.055046 7f999ed40700 osd.0 7 sched_scrub
  1116. 2011-12-01 11:56:33.055113 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:33.055080 (86400 seconds ago)
  1117. 2011-12-01 11:56:33.055193 7f999ed40700 osd.0 7 sched_scrub done
  1118. 2011-12-01 11:56:33.256402 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1119. 2011-12-01 11:56:33.256536 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1120. 2011-12-01 11:56:34.055536 7f999ed40700 osd.0 7 tick
  1121. 2011-12-01 11:56:34.055801 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1122. 2011-12-01 11:56:34.657068 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1123. 2011-12-01 11:56:34.657434 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1124. 2011-12-01 11:56:35.056104 7f999ed40700 osd.0 7 tick
  1125. 2011-12-01 11:56:35.056348 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1126. 2011-12-01 11:56:35.858671 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1127. 2011-12-01 11:56:35.858912 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1128. 2011-12-01 11:56:36.056774 7f999ed40700 osd.0 7 tick
  1129. 2011-12-01 11:56:36.057131 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1130. 2011-12-01 11:56:37.060493 7f999ed40700 osd.0 7 tick
  1131. 2011-12-01 11:56:37.061062 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1132. 2011-12-01 11:56:37.061364 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1133. 2011-12-01 11:56:37.061474 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1134. 2011-12-01 11:56:37.061642 7f999ed40700 osd.0 7 do_mon_report
  1135. 2011-12-01 11:56:37.061717 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1136. 2011-12-01 11:56:37.061743 7f999ed40700 osd.0 7 send_pg_stats
  1137. 2011-12-01 11:56:37.661809 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1138. 2011-12-01 11:56:37.661892 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1139. 2011-12-01 11:56:38.061992 7f999ed40700 osd.0 7 tick
  1140. 2011-12-01 11:56:38.062226 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1141. 2011-12-01 11:56:38.362235 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1142. 2011-12-01 11:56:38.362319 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1143. 2011-12-01 11:56:38.862628 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1144. 2011-12-01 11:56:38.862699 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1145. 2011-12-01 11:56:39.062510 7f999ed40700 osd.0 7 tick
  1146. 2011-12-01 11:56:39.062690 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1147. 2011-12-01 11:56:39.663110 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1148. 2011-12-01 11:56:39.663296 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1149. 2011-12-01 11:56:39.838342 7f9991c25700 osd.0 7 OSD::ms_verify_authorizer name=mds.a auid=18446744073709551615
  1150. 2011-12-01 11:56:39.838440 7f9991c25700 osd.0 7 new session 0xd03120 con=0xd0b640 addr=127.0.0.1:6800/15268
  1151. 2011-12-01 11:56:39.839393 7f9997d32700 osd.0 7 _dispatch 0xd09900 osd_op(mds.0.2:1 200.00000000 [writefull 0~84] 1.844f3494 RETRY) v3
  1152. 2011-12-01 11:56:39.839496 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd09900
  1153. 2011-12-01 11:56:39.839536 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1154. 2011-12-01 11:56:39.839564 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1155. 2011-12-01 11:56:39.839649 7f9997d32700 osd.0 7 _dispatch 0xd0a240 osd_op(mds.0.2:2 200.00000001 [delete] 1.6e5f474 RETRY) v3
  1156. 2011-12-01 11:56:39.839680 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a240
  1157. 2011-12-01 11:56:39.839716 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1158. 2011-12-01 11:56:39.839744 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1159. 2011-12-01 11:56:39.839966 7f9997d32700 osd.0 7 _dispatch 0xd08d80 osd_op(mds.0.2:3 200.00000002 [delete] 1.eb272dbb RETRY) v3
  1160. 2011-12-01 11:56:39.840069 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd08d80
  1161. 2011-12-01 11:56:39.840121 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1162. 2011-12-01 11:56:39.840144 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1163. 2011-12-01 11:56:39.840179 7f9997d32700 osd.0 7 _dispatch 0xd0a900 osd_op(mds.0.2:4 200.00000003 [delete] 1.95e3ab RETRY) v3
  1164. 2011-12-01 11:56:39.840204 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a900
  1165. 2011-12-01 11:56:39.840230 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1166. 2011-12-01 11:56:39.840273 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1167. 2011-12-01 11:56:39.840301 7f9997d32700 osd.0 7 _dispatch 0xd0a6c0 osd_op(mds.0.2:5 200.00000004 [delete] 1.1ba6bc1f RETRY) v3
  1168. 2011-12-01 11:56:39.840325 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a6c0
  1169. 2011-12-01 11:56:39.840361 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1170. 2011-12-01 11:56:39.840385 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1171. 2011-12-01 11:56:39.840434 7f9997d32700 osd.0 7 _dispatch 0xd0a480 osd_op(mds.0.2:6 200.00000005 [delete] 1.96d5827a RETRY) v3
  1172. 2011-12-01 11:56:39.840458 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a480
  1173. 2011-12-01 11:56:39.840484 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1174. 2011-12-01 11:56:39.840506 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1175. 2011-12-01 11:56:39.840658 7f9997d32700 osd.0 7 _dispatch 0xd0ad80 osd_op(mds.0.2:7 200.00000006 [delete] 1.ade7efde RETRY) v3
  1176. 2011-12-01 11:56:39.840720 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0ad80
  1177. 2011-12-01 11:56:39.840753 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1178. 2011-12-01 11:56:39.840813 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1179. 2011-12-01 11:56:39.840870 7f9997d32700 osd.0 7 _dispatch 0xd0ab40 osd_op(mds.0.2:10 1.00000000.inode [writefull 0~428] 1.232c0e14 RETRY) v3
  1180. 2011-12-01 11:56:39.841295 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0ab40
  1181. 2011-12-01 11:56:39.841356 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1182. 2011-12-01 11:56:39.841383 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1183. 2011-12-01 11:56:39.841416 7f9997d32700 osd.0 7 _dispatch 0xd2d900 osd_op(mds.0.2:22 100.00000000.inode [writefull 0~399] 1.85dde07f RETRY) v3
  1184. 2011-12-01 11:56:39.841477 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d900
  1185. 2011-12-01 11:56:39.841507 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1186. 2011-12-01 11:56:39.841538 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1187. 2011-12-01 11:56:39.841574 7f9997d32700 osd.0 7 _dispatch 0xd2d6c0 osd_op(mds.0.2:23 mds0_inotable [writefull 0~29] 1.b852b893 RETRY) v3
  1188. 2011-12-01 11:56:39.841601 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d6c0
  1189. 2011-12-01 11:56:39.841655 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1190. 2011-12-01 11:56:39.841681 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1191. 2011-12-01 11:56:39.841713 7f9997d32700 osd.0 7 _dispatch 0xd2d480 osd_op(mds.0.2:24 mds0_sessionmap [writefull 0~17] 1.3270c60b RETRY) v3
  1192. 2011-12-01 11:56:39.841740 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d480
  1193. 2011-12-01 11:56:39.841771 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1194. 2011-12-01 11:56:39.841797 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1195. 2011-12-01 11:56:39.841828 7f9997d32700 osd.0 7 _dispatch 0xd2d240 osd_op(mds.0.2:25 mds_anchortable [writefull 0~29] 1.a977f6a7 RETRY) v3
  1196. 2011-12-01 11:56:39.841855 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d240
  1197. 2011-12-01 11:56:39.841883 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1198. 2011-12-01 11:56:39.841908 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1199. 2011-12-01 11:56:39.842126 7f9997d32700 osd.0 7 _dispatch 0xd2d000 osd_op(mds.0.2:26 mds_snaptable [writefull 0~41] 1.d90270ad RETRY) v3
  1200. 2011-12-01 11:56:39.842229 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d000
  1201. 2011-12-01 11:56:39.842260 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1202. 2011-12-01 11:56:39.842284 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1203. 2011-12-01 11:56:39.842505 7f9997d32700 osd.0 7 _dispatch 0xd2dd80 osd_op(mds.0.2:8 2.00000000 [setxattr path (6),setxattr parent (38),tmapup 0~0] 1.96f33707 RETRY) v3
  1204. 2011-12-01 11:56:39.842600 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2dd80
  1205. 2011-12-01 11:56:39.842632 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1206. 2011-12-01 11:56:39.842677 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1207. 2011-12-01 11:56:39.842739 7f9997d32700 osd.0 7 _dispatch 0xd2db40 osd_op(mds.0.2:9 1.00000000 [setxattr path,setxattr parent (13),tmapput 0~557] 1.6b2cdaff RETRY) v3
  1208. 2011-12-01 11:56:39.842776 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2db40
  1209. 2011-12-01 11:56:39.842824 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1210. 2011-12-01 11:56:39.842998 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1211. 2011-12-01 11:56:39.843048 7f9997d32700 osd.0 7 _dispatch 0xd30d80 osd_op(mds.0.2:11 600.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.c2e541b0 RETRY) v3
  1212. 2011-12-01 11:56:39.843197 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30d80
  1213. 2011-12-01 11:56:39.843317 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1214. 2011-12-01 11:56:39.843343 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1215. 2011-12-01 11:56:39.843503 7f9997d32700 osd.0 7 _dispatch 0xd30b40 osd_op(mds.0.2:12 601.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.81580290 RETRY) v3
  1216. 2011-12-01 11:56:39.843601 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30b40
  1217. 2011-12-01 11:56:39.843651 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1218. 2011-12-01 11:56:39.843674 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1219. 2011-12-01 11:56:39.843705 7f9997d32700 osd.0 7 _dispatch 0xd30900 osd_op(mds.0.2:13 602.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.2a8b6bd0 RETRY) v3
  1220. 2011-12-01 11:56:39.843759 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30900
  1221. 2011-12-01 11:56:39.843810 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1222. 2011-12-01 11:56:39.843833 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1223. 2011-12-01 11:56:39.843896 7f9997d32700 osd.0 7 _dispatch 0xd306c0 osd_op(mds.0.2:14 603.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.7a4d91b0 RETRY) v3
  1224. 2011-12-01 11:56:39.843923 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd306c0
  1225. 2011-12-01 11:56:39.843998 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1226. 2011-12-01 11:56:39.844023 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1227. 2011-12-01 11:56:39.844054 7f9997d32700 osd.0 7 _dispatch 0xd30480 osd_op(mds.0.2:15 604.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.43e85c95 RETRY) v3
  1228. 2011-12-01 11:56:39.844078 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30480
  1229. 2011-12-01 11:56:39.844153 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1230. 2011-12-01 11:56:39.844178 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1231. 2011-12-01 11:56:39.844208 7f9997d32700 osd.0 7 _dispatch 0xd30240 osd_op(mds.0.2:16 605.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.d0c18e1d RETRY) v3
  1232. 2011-12-01 11:56:39.844282 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30240
  1233. 2011-12-01 11:56:39.844312 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1234. 2011-12-01 11:56:39.844363 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1235. 2011-12-01 11:56:39.844395 7f9997d32700 osd.0 7 _dispatch 0xd30000 osd_op(mds.0.2:17 606.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.f89eaaf4 RETRY) v3
  1236. 2011-12-01 11:56:39.844439 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30000
  1237. 2011-12-01 11:56:39.844495 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1238. 2011-12-01 11:56:39.844519 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1239. 2011-12-01 11:56:39.844549 7f9997d32700 osd.0 7 _dispatch 0xd22d80 osd_op(mds.0.2:18 607.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.bb590b7c RETRY) v3
  1240. 2011-12-01 11:56:39.844604 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22d80
  1241. 2011-12-01 11:56:39.844632 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1242. 2011-12-01 11:56:39.844654 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1243. 2011-12-01 11:56:39.844713 7f9997d32700 osd.0 7 _dispatch 0xd22b40 osd_op(mds.0.2:19 608.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.c4405e8e RETRY) v3
  1244. 2011-12-01 11:56:39.844741 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22b40
  1245. 2011-12-01 11:56:39.844779 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1246. 2011-12-01 11:56:39.844803 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1247. 2011-12-01 11:56:39.844861 7f9997d32700 osd.0 7 _dispatch 0xd22900 osd_op(mds.0.2:20 609.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.60b82d07 RETRY) v3
  1248. 2011-12-01 11:56:39.844886 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22900
  1249. 2011-12-01 11:56:39.844960 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1250. 2011-12-01 11:56:39.844985 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1251. 2011-12-01 11:56:39.845027 7f9997d32700 osd.0 7 _dispatch 0xd226c0 osd_op(mds.0.2:21 100.00000000 [setxattr path (5),setxattr parent (13),tmapput 0~4089] 1.c5265ab3 RETRY) v3
  1252. 2011-12-01 11:56:39.845202 7f9997d32700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd226c0
  1253. 2011-12-01 11:56:39.845256 7f9997d32700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1254. 2011-12-01 11:56:39.845308 7f9997d32700 osd.0 7 hit non-existent pg 1.0, waiting
  1255. 2011-12-01 11:56:40.062954 7f999ed40700 osd.0 7 tick
  1256. 2011-12-01 11:56:40.063213 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = no, randomly backing off
  1257. 2011-12-01 11:56:40.963618 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1258. 2011-12-01 11:56:40.963716 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1259. 2011-12-01 11:56:41.063535 7f999ed40700 osd.0 7 tick
  1260. 2011-12-01 11:56:41.063813 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.07 < max 0.5 = yes
  1261. 2011-12-01 11:56:41.063841 7f999ed40700 osd.0 7 sched_scrub
  1262. 2011-12-01 11:56:41.063890 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:41.063860 (86400 seconds ago)
  1263. 2011-12-01 11:56:41.063914 7f999ed40700 osd.0 7 sched_scrub done
  1264. 2011-12-01 11:56:41.963992 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1265. 2011-12-01 11:56:41.964041 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1266. 2011-12-01 11:56:42.064331 7f999ed40700 osd.0 7 tick
  1267. 2011-12-01 11:56:42.064492 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1268. 2011-12-01 11:56:42.064568 7f999ed40700 osd.0 7 do_mon_report
  1269. 2011-12-01 11:56:42.064599 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1270. 2011-12-01 11:56:42.064620 7f999ed40700 osd.0 7 send_pg_stats
  1271. 2011-12-01 11:56:42.664384 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1272. 2011-12-01 11:56:42.664432 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1273. 2011-12-01 11:56:43.064886 7f999ed40700 osd.0 7 tick
  1274. 2011-12-01 11:56:43.065232 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1275. 2011-12-01 11:56:43.664793 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1276. 2011-12-01 11:56:43.664847 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1277. 2011-12-01 11:56:44.066414 7f999ed40700 osd.0 7 tick
  1278. 2011-12-01 11:56:44.066679 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1279. 2011-12-01 11:56:44.865313 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1280. 2011-12-01 11:56:44.865380 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1281. 2011-12-01 11:56:45.067014 7f999ed40700 osd.0 7 tick
  1282. 2011-12-01 11:56:45.067207 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = yes
  1283. 2011-12-01 11:56:45.067235 7f999ed40700 osd.0 7 sched_scrub
  1284. 2011-12-01 11:56:45.067279 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:45.067254 (86400 seconds ago)
  1285. 2011-12-01 11:56:45.067302 7f999ed40700 osd.0 7 sched_scrub done
  1286. 2011-12-01 11:56:45.565709 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1287. 2011-12-01 11:56:45.565780 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1288. 2011-12-01 11:56:46.070227 7f999ed40700 osd.0 7 tick
  1289. 2011-12-01 11:56:46.070534 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1290. 2011-12-01 11:56:46.866098 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1291. 2011-12-01 11:56:46.866162 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1292. 2011-12-01 11:56:46.866190 7f9992426700 osd.0 7 i have no heartbeat peers; checking mon for new map
  1293. 2011-12-01 11:56:47.070817 7f999ed40700 osd.0 7 tick
  1294. 2011-12-01 11:56:47.071006 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = yes
  1295. 2011-12-01 11:56:47.071033 7f999ed40700 osd.0 7 sched_scrub
  1296. 2011-12-01 11:56:47.071099 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:47.071052 (86400 seconds ago)
  1297. 2011-12-01 11:56:47.071123 7f999ed40700 osd.0 7 sched_scrub done
  1298. 2011-12-01 11:56:47.071167 7f999ed40700 osd.0 7 do_mon_report
  1299. 2011-12-01 11:56:47.071196 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1300. 2011-12-01 11:56:47.071241 7f999ed40700 osd.0 7 send_pg_stats
  1301. 2011-12-01 11:56:48.066596 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1302. 2011-12-01 11:56:48.066652 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1303. 2011-12-01 11:56:48.071445 7f999ed40700 osd.0 7 tick
  1304. 2011-12-01 11:56:48.071572 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1305. 2011-12-01 11:56:49.067003 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1306. 2011-12-01 11:56:49.067051 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1307. 2011-12-01 11:56:49.071842 7f999ed40700 osd.0 7 tick
  1308. 2011-12-01 11:56:49.072014 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1309. 2011-12-01 11:56:49.767400 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1310. 2011-12-01 11:56:49.767448 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1311. 2011-12-01 11:56:50.072280 7f999ed40700 osd.0 7 tick
  1312. 2011-12-01 11:56:50.072434 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = no, randomly backing off
  1313. 2011-12-01 11:56:50.867867 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1314. 2011-12-01 11:56:50.867953 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1315. 2011-12-01 11:56:51.072728 7f999ed40700 osd.0 7 tick
  1316. 2011-12-01 11:56:51.073247 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.06 < max 0.5 = yes
  1317. 2011-12-01 11:56:51.073861 7f999ed40700 osd.0 7 sched_scrub
  1318. 2011-12-01 11:56:51.073930 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:51.073901 (86400 seconds ago)
  1319. 2011-12-01 11:56:51.073951 7f999ed40700 osd.0 7 sched_scrub done
  1320. 2011-12-01 11:56:51.768301 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1321. 2011-12-01 11:56:51.768349 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1322. 2011-12-01 11:56:52.074254 7f999ed40700 osd.0 7 tick
  1323. 2011-12-01 11:56:52.074423 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1324. 2011-12-01 11:56:52.074512 7f999ed40700 osd.0 7 do_mon_report
  1325. 2011-12-01 11:56:52.074541 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1326. 2011-12-01 11:56:52.074560 7f999ed40700 osd.0 7 send_pg_stats
  1327. 2011-12-01 11:56:52.468655 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1328. 2011-12-01 11:56:52.468725 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1329. 2011-12-01 11:56:53.074833 7f999ed40700 osd.0 7 tick
  1330. 2011-12-01 11:56:53.075003 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = yes
  1331. 2011-12-01 11:56:53.075030 7f999ed40700 osd.0 7 sched_scrub
  1332. 2011-12-01 11:56:53.075071 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:53.075050 (86400 seconds ago)
  1333. 2011-12-01 11:56:53.075091 7f999ed40700 osd.0 7 sched_scrub done
  1334. 2011-12-01 11:56:53.769183 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1335. 2011-12-01 11:56:53.769237 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1336. 2011-12-01 11:56:54.075487 7f999ed40700 osd.0 7 tick
  1337. 2011-12-01 11:56:54.075715 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1338. 2011-12-01 11:56:54.269578 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1339. 2011-12-01 11:56:54.269644 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1340. 2011-12-01 11:56:54.837288 7f9997d32700 osd.0 7 _dispatch 0xd01d80 ping v1
  1341. 2011-12-01 11:56:54.837337 7f9997d32700 osd.0 7 ping from mds.0
  1342. 2011-12-01 11:56:55.076018 7f999ed40700 osd.0 7 tick
  1343. 2011-12-01 11:56:55.076199 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = yes
  1344. 2011-12-01 11:56:55.076262 7f999ed40700 osd.0 7 sched_scrub
  1345. 2011-12-01 11:56:55.076420 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:55.076365 (86400 seconds ago)
  1346. 2011-12-01 11:56:55.076449 7f999ed40700 osd.0 7 sched_scrub done
  1347. 2011-12-01 11:56:55.169951 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1348. 2011-12-01 11:56:55.169983 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1349. 2011-12-01 11:56:56.076771 7f999ed40700 osd.0 7 tick
  1350. 2011-12-01 11:56:56.077114 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = yes
  1351. 2011-12-01 11:56:56.077678 7f999ed40700 osd.0 7 sched_scrub
  1352. 2011-12-01 11:56:56.077723 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:56:56.077700 (86400 seconds ago)
  1353. 2011-12-01 11:56:56.077744 7f999ed40700 osd.0 7 sched_scrub done
  1354. 2011-12-01 11:56:56.370387 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1355. 2011-12-01 11:56:56.370467 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1356. 2011-12-01 11:56:57.078033 7f999ed40700 osd.0 7 tick
  1357. 2011-12-01 11:56:57.078200 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1358. 2011-12-01 11:56:57.078247 7f999ed40700 osd.0 7 do_mon_report
  1359. 2011-12-01 11:56:57.078273 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1360. 2011-12-01 11:56:57.078292 7f999ed40700 osd.0 7 send_pg_stats
  1361. 2011-12-01 11:56:57.670814 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1362. 2011-12-01 11:56:57.670861 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1363. 2011-12-01 11:56:58.078505 7f999ed40700 osd.0 7 tick
  1364. 2011-12-01 11:56:58.078665 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1365. 2011-12-01 11:56:58.773806 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1366. 2011-12-01 11:56:58.773875 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1367. 2011-12-01 11:56:59.078995 7f999ed40700 osd.0 7 tick
  1368. 2011-12-01 11:56:59.079168 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1369. 2011-12-01 11:56:59.837642 7f9997d32700 osd.0 7 _dispatch 0xcd7900 ping v1
  1370. 2011-12-01 11:56:59.837680 7f9997d32700 osd.0 7 ping from mds.0
  1371. 2011-12-01 11:57:00.081069 7f999ed40700 osd.0 7 tick
  1372. 2011-12-01 11:57:00.081310 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = no, randomly backing off
  1373. 2011-12-01 11:57:00.174194 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1374. 2011-12-01 11:57:00.174241 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1375. 2011-12-01 11:57:00.774582 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1376. 2011-12-01 11:57:00.774633 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1377. 2011-12-01 11:57:01.083976 7f999ed40700 osd.0 7 tick
  1378. 2011-12-01 11:57:01.084590 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.05 < max 0.5 = yes
  1379. 2011-12-01 11:57:01.085339 7f999ed40700 osd.0 7 sched_scrub
  1380. 2011-12-01 11:57:01.085386 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:01.085363 (86400 seconds ago)
  1381. 2011-12-01 11:57:01.085406 7f999ed40700 osd.0 7 sched_scrub done
  1382. 2011-12-01 11:57:01.574980 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1383. 2011-12-01 11:57:01.575030 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1384. 2011-12-01 11:57:02.085700 7f999ed40700 osd.0 7 tick
  1385. 2011-12-01 11:57:02.085878 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = yes
  1386. 2011-12-01 11:57:02.085904 7f999ed40700 osd.0 7 sched_scrub
  1387. 2011-12-01 11:57:02.085963 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:02.085942 (86400 seconds ago)
  1388. 2011-12-01 11:57:02.085983 7f999ed40700 osd.0 7 sched_scrub done
  1389. 2011-12-01 11:57:02.086105 7f999ed40700 osd.0 7 do_mon_report
  1390. 2011-12-01 11:57:02.086133 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1391. 2011-12-01 11:57:02.086152 7f999ed40700 osd.0 7 send_pg_stats
  1392. 2011-12-01 11:57:02.675342 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1393. 2011-12-01 11:57:02.675390 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1394. 2011-12-01 11:57:03.087071 7f999ed40700 osd.0 7 tick
  1395. 2011-12-01 11:57:03.087394 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = yes
  1396. 2011-12-01 11:57:03.087427 7f999ed40700 osd.0 7 sched_scrub
  1397. 2011-12-01 11:57:03.087475 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:03.087450 (86400 seconds ago)
  1398. 2011-12-01 11:57:03.087498 7f999ed40700 osd.0 7 sched_scrub done
  1399. 2011-12-01 11:57:03.087562 7f999ed40700 osd.0 7 activate_pg pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded]
  1400. 2011-12-01 11:57:03.087595 7f999ed40700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] replay_queued_ops
  1401. 2011-12-01 11:57:03.087623 7f999ed40700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  1402. 2011-12-01 11:57:03.087650 7f999ed40700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  1403. 2011-12-01 11:57:03.087681 7f999ed40700 osd.0 7 pg[0.0p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'9
  1404. 2011-12-01 11:57:03.087712 7f999ed40700 osd.0 7 activate_pg pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded]
  1405. 2011-12-01 11:57:03.087740 7f999ed40700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+replay+degraded] replay_queued_ops
  1406. 2011-12-01 11:57:03.087781 7f999ed40700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  1407. 2011-12-01 11:57:03.087816 7f999ed40700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] requeue_ops
  1408. 2011-12-01 11:57:03.087864 7f999ed40700 osd.0 7 pg[0.1p0( empty n=0 ec=1 les/c 6/7 5/5/5) [0] r=0 lpr=5 mlcod 0'0 !hml active+clean+degraded] update_stats 5'9
  1409. 2011-12-01 11:57:03.087905 7f999ed40700 osd.0 7 do_waiters -- start
  1410. 2011-12-01 11:57:03.087931 7f999ed40700 osd.0 7 _dispatch 0xd09900 osd_op(mds.0.2:1 200.00000000 [writefull 0~84] 1.844f3494 RETRY) v3
  1411. 2011-12-01 11:57:03.087953 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd09900
  1412. 2011-12-01 11:57:03.087979 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1413. 2011-12-01 11:57:03.088001 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1414. 2011-12-01 11:57:03.088049 7f999ed40700 osd.0 7 _dispatch 0xd0a240 osd_op(mds.0.2:2 200.00000001 [delete] 1.6e5f474 RETRY) v3
  1415. 2011-12-01 11:57:03.088070 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a240
  1416. 2011-12-01 11:57:03.088093 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1417. 2011-12-01 11:57:03.088112 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1418. 2011-12-01 11:57:03.088135 7f999ed40700 osd.0 7 _dispatch 0xd08d80 osd_op(mds.0.2:3 200.00000002 [delete] 1.eb272dbb RETRY) v3
  1419. 2011-12-01 11:57:03.088155 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd08d80
  1420. 2011-12-01 11:57:03.088177 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1421. 2011-12-01 11:57:03.088196 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1422. 2011-12-01 11:57:03.088218 7f999ed40700 osd.0 7 _dispatch 0xd0a900 osd_op(mds.0.2:4 200.00000003 [delete] 1.95e3ab RETRY) v3
  1423. 2011-12-01 11:57:03.088238 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a900
  1424. 2011-12-01 11:57:03.088279 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1425. 2011-12-01 11:57:03.088300 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1426. 2011-12-01 11:57:03.088323 7f999ed40700 osd.0 7 _dispatch 0xd0a6c0 osd_op(mds.0.2:5 200.00000004 [delete] 1.1ba6bc1f RETRY) v3
  1427. 2011-12-01 11:57:03.088343 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a6c0
  1428. 2011-12-01 11:57:03.088366 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1429. 2011-12-01 11:57:03.088385 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1430. 2011-12-01 11:57:03.088407 7f999ed40700 osd.0 7 _dispatch 0xd0a480 osd_op(mds.0.2:6 200.00000005 [delete] 1.96d5827a RETRY) v3
  1431. 2011-12-01 11:57:03.088427 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0a480
  1432. 2011-12-01 11:57:03.088449 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1433. 2011-12-01 11:57:03.088468 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1434. 2011-12-01 11:57:03.088490 7f999ed40700 osd.0 7 _dispatch 0xd0ad80 osd_op(mds.0.2:7 200.00000006 [delete] 1.ade7efde RETRY) v3
  1435. 2011-12-01 11:57:03.088529 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0ad80
  1436. 2011-12-01 11:57:03.088552 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1437. 2011-12-01 11:57:03.088572 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1438. 2011-12-01 11:57:03.088595 7f999ed40700 osd.0 7 _dispatch 0xd0ab40 osd_op(mds.0.2:10 1.00000000.inode [writefull 0~428] 1.232c0e14 RETRY) v3
  1439. 2011-12-01 11:57:03.088615 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd0ab40
  1440. 2011-12-01 11:57:03.088638 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1441. 2011-12-01 11:57:03.088657 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1442. 2011-12-01 11:57:03.088681 7f999ed40700 osd.0 7 _dispatch 0xd2d900 osd_op(mds.0.2:22 100.00000000.inode [writefull 0~399] 1.85dde07f RETRY) v3
  1443. 2011-12-01 11:57:03.088701 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d900
  1444. 2011-12-01 11:57:03.088723 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1445. 2011-12-01 11:57:03.088743 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1446. 2011-12-01 11:57:03.088787 7f999ed40700 osd.0 7 _dispatch 0xd2d6c0 osd_op(mds.0.2:23 mds0_inotable [writefull 0~29] 1.b852b893 RETRY) v3
  1447. 2011-12-01 11:57:03.088807 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d6c0
  1448. 2011-12-01 11:57:03.088830 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1449. 2011-12-01 11:57:03.088861 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1450. 2011-12-01 11:57:03.088885 7f999ed40700 osd.0 7 _dispatch 0xd2d480 osd_op(mds.0.2:24 mds0_sessionmap [writefull 0~17] 1.3270c60b RETRY) v3
  1451. 2011-12-01 11:57:03.088973 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d480
  1452. 2011-12-01 11:57:03.089057 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1453. 2011-12-01 11:57:03.089082 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1454. 2011-12-01 11:57:03.089106 7f999ed40700 osd.0 7 _dispatch 0xd2d240 osd_op(mds.0.2:25 mds_anchortable [writefull 0~29] 1.a977f6a7 RETRY) v3
  1455. 2011-12-01 11:57:03.089126 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d240
  1456. 2011-12-01 11:57:03.089149 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1457. 2011-12-01 11:57:03.089168 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1458. 2011-12-01 11:57:03.089191 7f999ed40700 osd.0 7 _dispatch 0xd2d000 osd_op(mds.0.2:26 mds_snaptable [writefull 0~41] 1.d90270ad RETRY) v3
  1459. 2011-12-01 11:57:03.089211 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2d000
  1460. 2011-12-01 11:57:03.089236 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1461. 2011-12-01 11:57:03.089256 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1462. 2011-12-01 11:57:03.089283 7f999ed40700 osd.0 7 _dispatch 0xd2dd80 osd_op(mds.0.2:8 2.00000000 [setxattr path (6),setxattr parent (38),tmapup 0~0] 1.96f33707 RETRY) v3
  1463. 2011-12-01 11:57:03.089304 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2dd80
  1464. 2011-12-01 11:57:03.089326 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1465. 2011-12-01 11:57:03.089346 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1466. 2011-12-01 11:57:03.089371 7f999ed40700 osd.0 7 _dispatch 0xd2db40 osd_op(mds.0.2:9 1.00000000 [setxattr path,setxattr parent (13),tmapput 0~557] 1.6b2cdaff RETRY) v3
  1467. 2011-12-01 11:57:03.089391 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd2db40
  1468. 2011-12-01 11:57:03.089413 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1469. 2011-12-01 11:57:03.089433 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1470. 2011-12-01 11:57:03.089507 7f999ed40700 osd.0 7 _dispatch 0xd30d80 osd_op(mds.0.2:11 600.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.c2e541b0 RETRY) v3
  1471. 2011-12-01 11:57:03.089530 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30d80
  1472. 2011-12-01 11:57:03.089553 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1473. 2011-12-01 11:57:03.089573 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1474. 2011-12-01 11:57:03.089598 7f999ed40700 osd.0 7 _dispatch 0xd30b40 osd_op(mds.0.2:12 601.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.81580290 RETRY) v3
  1475. 2011-12-01 11:57:03.089618 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30b40
  1476. 2011-12-01 11:57:03.089640 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1477. 2011-12-01 11:57:03.089660 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1478. 2011-12-01 11:57:03.089685 7f999ed40700 osd.0 7 _dispatch 0xd30900 osd_op(mds.0.2:13 602.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.2a8b6bd0 RETRY) v3
  1479. 2011-12-01 11:57:03.089705 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30900
  1480. 2011-12-01 11:57:03.089740 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1481. 2011-12-01 11:57:03.089767 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1482. 2011-12-01 11:57:03.089793 7f999ed40700 osd.0 7 _dispatch 0xd306c0 osd_op(mds.0.2:14 603.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.7a4d91b0 RETRY) v3
  1483. 2011-12-01 11:57:03.089814 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd306c0
  1484. 2011-12-01 11:57:03.089836 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1485. 2011-12-01 11:57:03.089856 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1486. 2011-12-01 11:57:03.089880 7f999ed40700 osd.0 7 _dispatch 0xd30480 osd_op(mds.0.2:15 604.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.43e85c95 RETRY) v3
  1487. 2011-12-01 11:57:03.089911 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30480
  1488. 2011-12-01 11:57:03.089934 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1489. 2011-12-01 11:57:03.089954 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1490. 2011-12-01 11:57:03.089999 7f999ed40700 osd.0 7 _dispatch 0xd30240 osd_op(mds.0.2:16 605.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.d0c18e1d RETRY) v3
  1491. 2011-12-01 11:57:03.090020 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30240
  1492. 2011-12-01 11:57:03.090043 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1493. 2011-12-01 11:57:03.090063 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1494. 2011-12-01 11:57:03.090088 7f999ed40700 osd.0 7 _dispatch 0xd30000 osd_op(mds.0.2:17 606.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.f89eaaf4 RETRY) v3
  1495. 2011-12-01 11:57:03.090108 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd30000
  1496. 2011-12-01 11:57:03.090130 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1497. 2011-12-01 11:57:03.090150 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1498. 2011-12-01 11:57:03.090174 7f999ed40700 osd.0 7 _dispatch 0xd22d80 osd_op(mds.0.2:18 607.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.bb590b7c RETRY) v3
  1499. 2011-12-01 11:57:03.090195 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22d80
  1500. 2011-12-01 11:57:03.090217 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1501. 2011-12-01 11:57:03.090239 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1502. 2011-12-01 11:57:03.090264 7f999ed40700 osd.0 7 _dispatch 0xd22b40 osd_op(mds.0.2:19 608.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.c4405e8e RETRY) v3
  1503. 2011-12-01 11:57:03.090284 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22b40
  1504. 2011-12-01 11:57:03.090307 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1505. 2011-12-01 11:57:03.090326 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1506. 2011-12-01 11:57:03.090351 7f999ed40700 osd.0 7 _dispatch 0xd22900 osd_op(mds.0.2:20 609.00000000 [setxattr path (12),setxattr parent (39),tmapup 0~0] 1.60b82d07 RETRY) v3
  1507. 2011-12-01 11:57:03.090371 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd22900
  1508. 2011-12-01 11:57:03.090394 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1509. 2011-12-01 11:57:03.090413 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1510. 2011-12-01 11:57:03.090439 7f999ed40700 osd.0 7 _dispatch 0xd226c0 osd_op(mds.0.2:21 100.00000000 [setxattr path (5),setxattr parent (13),tmapput 0~4089] 1.c5265ab3 RETRY) v3
  1511. 2011-12-01 11:57:03.090459 7f999ed40700 osd.0 7 require_same_or_newer_map 7 (i am 7) 0xd226c0
  1512. 2011-12-01 11:57:03.090500 7f999ed40700 osd.0 7 _share_map_incoming mds.0 127.0.0.1:6800/15268 7
  1513. 2011-12-01 11:57:03.090520 7f999ed40700 osd.0 7 hit non-existent pg 1.0, waiting
  1514. 2011-12-01 11:57:03.090540 7f999ed40700 osd.0 7 do_waiters -- finish
  1515. 2011-12-01 11:57:04.075698 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1516. 2011-12-01 11:57:04.075738 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1517. 2011-12-01 11:57:04.088115 7f999ed40700 osd.0 7 tick
  1518. 2011-12-01 11:57:04.088242 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = yes
  1519. 2011-12-01 11:57:04.088267 7f999ed40700 osd.0 7 sched_scrub
  1520. 2011-12-01 11:57:04.088328 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:04.088284 (86400 seconds ago)
  1521. 2011-12-01 11:57:04.088350 7f999ed40700 osd.0 7 sched_scrub done
  1522. 2011-12-01 11:57:04.838733 7f9997d32700 osd.0 7 _dispatch 0xcd7a80 ping v1
  1523. 2011-12-01 11:57:04.838770 7f9997d32700 osd.0 7 ping from mds.0
  1524. 2011-12-01 11:57:05.088630 7f999ed40700 osd.0 7 tick
  1525. 2011-12-01 11:57:05.088829 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = no, randomly backing off
  1526. 2011-12-01 11:57:05.476070 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1527. 2011-12-01 11:57:05.476118 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1528. 2011-12-01 11:57:06.093964 7f999ed40700 osd.0 7 tick
  1529. 2011-12-01 11:57:06.094310 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = no, randomly backing off
  1530. 2011-12-01 11:57:06.676472 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1531. 2011-12-01 11:57:06.676522 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1532. 2011-12-01 11:57:07.094623 7f999ed40700 osd.0 7 tick
  1533. 2011-12-01 11:57:07.094873 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = yes
  1534. 2011-12-01 11:57:07.094899 7f999ed40700 osd.0 7 sched_scrub
  1535. 2011-12-01 11:57:07.094938 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:07.094917 (86400 seconds ago)
  1536. 2011-12-01 11:57:07.094957 7f999ed40700 osd.0 7 sched_scrub done
  1537. 2011-12-01 11:57:07.094996 7f999ed40700 osd.0 7 do_mon_report
  1538. 2011-12-01 11:57:07.095065 7f999ed40700 osd.0 7 send_alive up_thru currently 5 want 5
  1539. 2011-12-01 11:57:07.095083 7f999ed40700 osd.0 7 send_pg_stats
  1540. 2011-12-01 11:57:07.095102 7f999ed40700 osd.0 7 send_pg_stats - 2 pgs updated
  1541. 2011-12-01 11:57:07.152742 7f9997d32700 osd.0 7 _dispatch 0xd101c0 pg_stats_ack(2 pgs tid 4) v1
  1542. 2011-12-01 11:57:07.152774 7f9997d32700 osd.0 7 handle_pg_stats_ack
  1543. 2011-12-01 11:57:07.376972 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1544. 2011-12-01 11:57:07.377032 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1545. 2011-12-01 11:57:08.095423 7f999ed40700 osd.0 7 tick
  1546. 2011-12-01 11:57:08.095599 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = no, randomly backing off
  1547. 2011-12-01 11:57:08.281278 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1548. 2011-12-01 11:57:08.281325 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1549. 2011-12-01 11:57:08.881686 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1550. 2011-12-01 11:57:08.881736 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1551. 2011-12-01 11:57:09.095866 7f999ed40700 osd.0 7 tick
  1552. 2011-12-01 11:57:09.096186 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = yes
  1553. 2011-12-01 11:57:09.096245 7f999ed40700 osd.0 7 sched_scrub
  1554. 2011-12-01 11:57:09.096348 7f999ed40700 osd.0 7 0.0p0 at 2011-12-01 11:36:37.845127 > 2011-11-30 11:57:09.096291 (86400 seconds ago)
  1555. 2011-12-01 11:57:09.096402 7f999ed40700 osd.0 7 sched_scrub done
  1556. 2011-12-01 11:57:09.382035 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1557. 2011-12-01 11:57:09.382087 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1558. 2011-12-01 11:57:09.839557 7f9997d32700 osd.0 7 _dispatch 0xd01c00 ping v1
  1559. 2011-12-01 11:57:09.839585 7f9997d32700 osd.0 7 ping from mds.0
  1560. 2011-12-01 11:57:10.082396 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1561. 2011-12-01 11:57:10.082448 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1562. 2011-12-01 11:57:10.096712 7f999ed40700 osd.0 7 tick
  1563. 2011-12-01 11:57:10.097058 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = no, randomly backing off
  1564. 2011-12-01 11:57:11.082796 7f9992426700 osd.0 7 update_osd_stat osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1565. 2011-12-01 11:57:11.082849 7f9992426700 osd.0 7 heartbeat: osd_stat(167 MB used, 14181 MB avail, 15116 MB total, peers []/[])
  1566. 2011-12-01 11:57:11.097338 7f999ed40700 osd.0 7 tick
  1567. 2011-12-01 11:57:11.097606 7f999ed40700 osd.0 7 scrub_should_schedule loadavg 0.04 < max 0.5 = no, randomly backing off
  1568.  
  1569.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!