API tools faq
paste
Advertisement
Guest User

mon.log

a guest
Dec 18th, 2012
272
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 442.82 KB | None | 0 0
raw download clone embed print report
  1. -2415> 2012-12-18 19:54:56.853681 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 5463)
  2. -2414> 2012-12-18 19:54:56.853687 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 5463)
  3. -2413> 2012-12-18 19:54:56.853738 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7502801 10.1.0.111:0/1000629 2 ==== auth(proto 2 144 bytes epoch 0) v1 ==== 174+0+0 (2762348381 0 0) 0x9cb8400 con 0x9cd1680
  4. -2412> 2012-12-18 19:54:56.853750 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.853751 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  5. -2411> 2012-12-18 19:54:56.853929 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (5463 -> 5621)
  6. -2410> 2012-12-18 19:54:56.853942 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (5463 -> 5621)
  7. -2409> 2012-12-18 19:54:56.853979 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.111:0/1000629 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb9800 con 0x9cd1680
  8. -2408> 2012-12-18 19:54:56.854021 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 174 (0x8e4928 -> 5447)
  9. -2407> 2012-12-18 19:54:56.854037 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 174 (0x8e4928 -> 5447)
  10. -2406> 2012-12-18 19:54:56.854042 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2849011740 0 0) 0x9cb8e00 con 0x9cd1b80
  11. -2405> 2012-12-18 19:54:56.854053 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.854054 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  12. -2404> 2012-12-18 19:54:56.854075 7f977487d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 174 (5447 -> 5621)
  13. -2403> 2012-12-18 19:54:56.854086 7f977487d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 174 (5447 -> 5621)
  14. -2402> 2012-12-18 19:54:56.854205 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6803/29184 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8400 con 0x9cd1b80
  15. -2401> 2012-12-18 19:54:56.854223 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5463)
  16. -2400> 2012-12-18 19:54:56.854228 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5463)
  17. -2399> 2012-12-18 19:54:56.854233 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 18 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cba540 con 0x722c8c0
  18. -2398> 2012-12-18 19:54:56.854249 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c464e0
  19. -2397> 2012-12-18 19:54:56.854260 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 5402)
  20. -2396> 2012-12-18 19:54:56.854267 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 5402)
  21. -2395> 2012-12-18 19:54:56.854300 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7502687 10.1.0.111:0/1020428 2 ==== auth(proto 2 144 bytes epoch 0) v1 ==== 174+0+0 (2395213653 0 0) 0x9cb8000 con 0x722cb40
  22. -2394> 2012-12-18 19:54:56.854314 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.854314 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  23. -2393> 2012-12-18 19:54:56.854461 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.111:0/1020428 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8e00 con 0x722cb40
  24. -2392> 2012-12-18 19:54:56.854480 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 174 (0x8e4928 -> 5228)
  25. -2391> 2012-12-18 19:54:56.854485 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 174 (0x8e4928 -> 5228)
  26. -2390> 2012-12-18 19:54:56.854490 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (33408521 0 0) 0x700ae00 con 0x9a93400
  27. -2389> 2012-12-18 19:54:56.854500 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.854501 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  28. -2388> 2012-12-18 19:54:56.854641 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6818/10436 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8000 con 0x9a93400
  29. -2387> 2012-12-18 19:54:56.854660 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5070)
  30. -2386> 2012-12-18 19:54:56.854714 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5070)
  31. -2385> 2012-12-18 19:54:56.854718 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503630 10.1.0.246:0/1001758 2 ==== auth(proto 2 144 bytes epoch 0) v1 ==== 174+0+0 (285044815 0 0) 0x9cb9800 con 0x722de00
  32. -2384> 2012-12-18 19:54:56.854728 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.854729 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  33. -2383> 2012-12-18 19:54:56.854804 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (5070 -> 5112)
  34. -2382> 2012-12-18 19:54:56.854814 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (5070 -> 5112)
  35. -2381> 2012-12-18 19:54:56.854815 7f977295e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (5112 -> 5154)
  36. -2380> 2012-12-18 19:54:56.854828 7f977295e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (5112 -> 5154)
  37. -2379> 2012-12-18 19:54:56.854919 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.246:0/1001758 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x700ae00 con 0x722de00
  38. -2378> 2012-12-18 19:54:56.854929 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (5154 -> 5176)
  39. -2377> 2012-12-18 19:54:56.854935 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (5154 -> 5176)
  40. -2376> 2012-12-18 19:54:56.854936 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (5176 -> 5334)
  41. -2375> 2012-12-18 19:54:56.854952 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (5176 -> 5334)
  42. -2374> 2012-12-18 19:54:56.854969 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 789 (5334 -> 6123)
  43. -2373> 2012-12-18 19:54:56.855132 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 789 (5334 -> 6123)
  44. -2372> 2012-12-18 19:54:56.855143 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 174 (0x8e4928 -> 5949)
  45. -2371> 2012-12-18 19:54:56.855162 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 174 (0x8e4928 -> 5949)
  46. -2370> 2012-12-18 19:54:56.855169 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 19 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (994304932 0 0) 0x9cb9600 con 0x722c8c0
  47. -2369> 2012-12-18 19:54:56.855186 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.855188 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  48. -2368> 2012-12-18 19:54:56.855203 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cb9800 con 0x722c8c0
  49. -2367> 2012-12-18 19:54:56.855230 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 5754)
  50. -2366> 2012-12-18 19:54:56.855296 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 5754)
  51. -2365> 2012-12-18 19:54:56.855313 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7502801 10.1.0.111:0/1000629 3 ==== mon_subscribe({monmap=2+,osdmap=10022}) v2 ==== 42+0+0 (1430664777 0 0) 0x9cba000 con 0x9cd1680
  52. -2364> 2012-12-18 19:54:56.855328 7f9774d82700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (5754 -> 5796)
  53. -2363> 2012-12-18 19:54:56.855331 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to client.7502801 10.1.0.111:0/1000629
  54. -2362> 2012-12-18 19:54:56.855350 7f9774d82700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (5754 -> 5796)
  55. -2361> 2012-12-18 19:54:56.855361 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (5796 -> 5954)
  56. -2360> 2012-12-18 19:54:56.855380 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7502801 10.1.0.111:0/1000629 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9600
  57. -2359> 2012-12-18 19:54:56.855388 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (5796 -> 5954)
  58. -2358> 2012-12-18 19:54:56.855399 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7502801 10.1.0.111:0/1000629 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c46000
  59. -2357> 2012-12-18 19:54:56.855409 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 5912)
  60. -2356> 2012-12-18 19:54:56.855412 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 5912)
  61. -2355> 2012-12-18 19:54:56.855416 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 3 ==== mon_subscribe({monmap=2+,osdmap=9920}) v2 ==== 42+0+0 (2996390116 0 0) 0x9cba380 con 0x9cd1b80
  62. -2354> 2012-12-18 19:54:56.855430 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9920..10022] to osd.61 10.1.0.17:6803/29184
  63. -2353> 2012-12-18 19:54:56.855501 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (5912 -> 6070)
  64. -2352> 2012-12-18 19:54:56.855515 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (5912 -> 6070)
  65. -2351> 2012-12-18 19:54:56.855570 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (6070 -> 6228)
  66. -2350> 2012-12-18 19:54:56.855583 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (6070 -> 6228)
  67. -2349> 2012-12-18 19:54:56.855675 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (6228 -> 6386)
  68. -2348> 2012-12-18 19:54:56.855685 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (6228 -> 6386)
  69. -2347> 2012-12-18 19:54:56.855744 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (6386 -> 6544)
  70. -2346> 2012-12-18 19:54:56.855754 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (6386 -> 6544)
  71. -2345> 2012-12-18 19:54:56.855884 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (6544 -> 6702)
  72. -2344> 2012-12-18 19:54:56.855893 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (6544 -> 6702)
  73. -2343> 2012-12-18 19:54:56.856304 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.61 10.1.0.17:6803/29184 -- osd_map(9920..10020 src has 7326..10022) v3 -- ?+0 0x7004400
  74. -2342> 2012-12-18 19:54:56.856314 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (6702 -> 6860)
  75. -2341> 2012-12-18 19:54:56.856323 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (6702 -> 6860)
  76. -2340> 2012-12-18 19:54:56.856323 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.61 10.1.0.17:6803/29184 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c46680
  77. -2339> 2012-12-18 19:54:56.856375 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 6818)
  78. -2338> 2012-12-18 19:54:56.856382 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 6818)
  79. -2337> 2012-12-18 19:54:56.856386 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (23557490 0 0) 0x700ae00 con 0x9a93a40
  80. -2336> 2012-12-18 19:54:56.856396 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.856397 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  81. -2335> 2012-12-18 19:54:56.856419 7f977487d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 23 (6818 -> 6841)
  82. -2334> 2012-12-18 19:54:56.856430 7f977487d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 23 (6818 -> 6841)
  83. -2333> 2012-12-18 19:54:56.856565 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193ce00 con 0x9a93a40
  84. -2332> 2012-12-18 19:54:56.856584 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 6683)
  85. -2331> 2012-12-18 19:54:56.856589 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 6683)
  86. -2330> 2012-12-18 19:54:56.856632 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 20 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cb9400 con 0x722c8c0
  87. -2329> 2012-12-18 19:54:56.856642 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.856643 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  88. -2328> 2012-12-18 19:54:56.856656 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x700ae00 con 0x722c8c0
  89. -2327> 2012-12-18 19:54:56.856669 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 6651)
  90. -2326> 2012-12-18 19:54:56.856673 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 6651)
  91. -2325> 2012-12-18 19:54:56.856723 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7502687 10.1.0.111:0/1020428 3 ==== mon_subscribe({monmap=2+,osdmap=10022}) v2 ==== 42+0+0 (1430664777 0 0) 0x9cba1c0 con 0x722cb40
  92. -2324> 2012-12-18 19:54:56.856741 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to client.7502687 10.1.0.111:0/1020428
  93. -2323> 2012-12-18 19:54:56.856763 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7502687 10.1.0.111:0/1020428 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9400
  94. -2322> 2012-12-18 19:54:56.856779 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7502687 10.1.0.111:0/1020428 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c46820
  95. -2321> 2012-12-18 19:54:56.856788 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 6609)
  96. -2320> 2012-12-18 19:54:56.856792 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 6609)
  97. -2319> 2012-12-18 19:54:56.856796 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0xbb57860 con 0x9cd1b80
  98. -2318> 2012-12-18 19:54:56.856810 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.856811 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  99. -2317> 2012-12-18 19:54:56.856834 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 6587)
  100. -2316> 2012-12-18 19:54:56.856840 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3033156916 0 0) 0x9cb8000 con 0x72123c0
  101. -2315> 2012-12-18 19:54:56.856850 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.856851 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  102. -2314> 2012-12-18 19:54:56.857000 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6809/18192 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193dc00 con 0x72123c0
  103. -2313> 2012-12-18 19:54:56.857018 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 6451)
  104. -2312> 2012-12-18 19:54:56.857023 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 6429)
  105. -2314> 2012-12-18 19:54:56.857068 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3102344374 0 0) 0x9cb8400 con 0x722d7c0
  106. -2313> 2012-12-18 19:54:56.857079 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.857079 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  107. -2312> 2012-12-18 19:54:56.857229 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6818/19812 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8000 con 0x722d7c0
  108. -2313> 2012-12-18 19:54:56.857246 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 6293)
  109. -2312> 2012-12-18 19:54:56.857252 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 6271)
  110. -2311> 2012-12-18 19:54:56.857296 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (84246623 0 0) 0x7223e00 con 0x7212780
  111. -2310> 2012-12-18 19:54:56.857307 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.857308 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  112. -2309> 2012-12-18 19:54:56.857454 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6821/19143 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8400 con 0x7212780
  113. -2308> 2012-12-18 19:54:56.857471 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 6135)
  114. -2307> 2012-12-18 19:54:56.857477 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 6113)
  115. -2306> 2012-12-18 19:54:56.857521 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (4075785262 0 0) 0x7222000 con 0x722db80
  116. -2305> 2012-12-18 19:54:56.857532 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.857533 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  117. -2304> 2012-12-18 19:54:56.857680 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6803/7344 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7223e00 con 0x722db80
  118. -2303> 2012-12-18 19:54:56.857698 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5977)
  119. -2302> 2012-12-18 19:54:56.857702 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5955)
  120. -2301> 2012-12-18 19:54:56.857707 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (4223674221 0 0) 0x7210000 con 0x7212b40
  121. -2300> 2012-12-18 19:54:56.857718 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.857719 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  122. -2299> 2012-12-18 19:54:56.857860 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6809/17050 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7222000 con 0x7212b40
  123. -2298> 2012-12-18 19:54:56.857879 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5819)
  124. -2297> 2012-12-18 19:54:56.857929 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5797)
  125. -2296> 2012-12-18 19:54:56.857933 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1627417936 0 0) 0x721dc00 con 0x9cd1180
  126. -2295> 2012-12-18 19:54:56.857943 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.857944 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  127. -2294> 2012-12-18 19:54:56.858096 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6812/8435 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7210000 con 0x9cd1180
  128. -2293> 2012-12-18 19:54:56.858114 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5661)
  129. -2292> 2012-12-18 19:54:56.858148 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5639)
  130. -2291> 2012-12-18 19:54:56.858154 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2875116879 0 0) 0x6092200 con 0x722c3c0
  131. -2290> 2012-12-18 19:54:56.858166 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858167 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  132. -2289> 2012-12-18 19:54:56.858320 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6806/16967 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x721dc00 con 0x722c3c0
  133. -2288> 2012-12-18 19:54:56.858338 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 5503)
  134. -2287> 2012-12-18 19:54:56.858346 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 5481)
  135. -2286> 2012-12-18 19:54:56.858349 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503630 10.1.0.246:0/1001758 3 ==== mon_subscribe({monmap=2+}) v2 ==== 23+0+0 (2366214315 0 0) 0x9ccba40 con 0x722de00
  136. -2285> 2012-12-18 19:54:56.858365 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7503630 10.1.0.246:0/1001758 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968ea0
  137. -2284> 2012-12-18 19:54:56.858377 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 23 (0x8e4928 -> 5480)
  138. -2283> 2012-12-18 19:54:56.858383 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 23 (0x8e4928 -> 5458)
  139. -2282> 2012-12-18 19:54:56.858385 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 21 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbafc0 con 0x722c8c0
  140. -2281> 2012-12-18 19:54:56.858399 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19684e0
  141. -2280> 2012-12-18 19:54:56.858410 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 5419)
  142. -2279> 2012-12-18 19:54:56.858415 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 5397)
  143. -2278> 2012-12-18 19:54:56.858452 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 5 ==== osd_pgtemp(e10021 {18.4f9=[61,33,20],18.6807=[61,37,16],18.8c27=[61,53,69],18.8e9b=[61,50,24],18.ac2c=[61,40,53],18.115f0=[61,57,71],18.1374b=[61,46,21],18.17813=[61,59,69],18.1875f=[61,48,18],18.199a9=[61,54,74],18.1b1bd=[61,59,69],18.1fcef=[61,53,28],18.20f09=[61,50,74],18.21fd0=[61,44,70],18.2398f=[61,55,13],18.25d80=[61,42,36,28],18.25e76=[61,40,54],18.25f21=[61,34,22],18.268a5=[61,42,71],18.28368=[61,55,71],18.2a61b=[61,34,69],18.2dbcc=[61,47,16],18.2ff33=[61,52,56]} v10021) v1 ==== 789+0+0 (767566006 0 0) 0x9cdc1e0 con 0x9cd1b80
  144. -2277> 2012-12-18 19:54:56.858490 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.858491 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  145. -2276> 2012-12-18 19:54:56.858518 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 789 (0x8e4928 -> 4608)
  146. -2275> 2012-12-18 19:54:56.858523 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 22 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (1596598913 0 0) 0x9cb9200 con 0x722c8c0
  147. -2274> 2012-12-18 19:54:56.858533 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858534 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  148. -2273> 2012-12-18 19:54:56.858545 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x6092200 con 0x722c8c0
  149. -2272> 2012-12-18 19:54:56.858558 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 5224)
  150. -2271> 2012-12-18 19:54:56.858561 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 4413)
  151. -2270> 2012-12-18 19:54:56.858596 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 23 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cb9e00 con 0x722c8c0
  152. -2269> 2012-12-18 19:54:56.858604 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858604 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  153. -2268> 2012-12-18 19:54:56.858615 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cb9200 con 0x722c8c0
  154. -2267> 2012-12-18 19:54:56.858624 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 5192)
  155. -2266> 2012-12-18 19:54:56.858630 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 4381)
  156. -2265> 2012-12-18 19:54:56.858634 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 24 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbae00 con 0x722c8c0
  157. -2264> 2012-12-18 19:54:56.858648 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19681a0
  158. -2263> 2012-12-18 19:54:56.858664 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 5131)
  159. -2262> 2012-12-18 19:54:56.858671 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 4320)
  160. -2261> 2012-12-18 19:54:56.858676 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 25 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (447631998 0 0) 0x9cb9c00 con 0x722c8c0
  161. -2260> 2012-12-18 19:54:56.858685 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858686 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  162. -2259> 2012-12-18 19:54:56.858697 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cb9e00 con 0x722c8c0
  163. -2258> 2012-12-18 19:54:56.858717 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 4936)
  164. -2257> 2012-12-18 19:54:56.858722 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 4125)
  165. -2256> 2012-12-18 19:54:56.858725 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 26 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc0c00 con 0x722c8c0
  166. -2255> 2012-12-18 19:54:56.858734 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858735 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  167. -2254> 2012-12-18 19:54:56.858746 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cb9c00 con 0x722c8c0
  168. -2253> 2012-12-18 19:54:56.858757 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 4904)
  169. -2252> 2012-12-18 19:54:56.858762 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 4093)
  170. -2251> 2012-12-18 19:54:56.858780 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 27 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbac40 con 0x722c8c0
  171. -2250> 2012-12-18 19:54:56.858794 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968340
  172. -2249> 2012-12-18 19:54:56.858805 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 4843)
  173. -2248> 2012-12-18 19:54:56.858809 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 4032)
  174. -2247> 2012-12-18 19:54:56.858812 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 28 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (58812840 0 0) 0x9cc0a00 con 0x722c8c0
  175. -2246> 2012-12-18 19:54:56.858821 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858822 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  176. -2245> 2012-12-18 19:54:56.858833 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0c00 con 0x722c8c0
  177. -2244> 2012-12-18 19:54:56.858845 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 4648)
  178. -2243> 2012-12-18 19:54:56.858849 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 3837)
  179. -2242> 2012-12-18 19:54:56.858853 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 29 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc0800 con 0x722c8c0
  180. -2241> 2012-12-18 19:54:56.858863 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858864 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  181. -2242> 2012-12-18 19:54:56.858874 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0a00 con 0x722c8c0
  182. -2241> 2012-12-18 19:54:56.858885 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 4616)
  183. -2240> 2012-12-18 19:54:56.858890 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 3805)
  184. -2239> 2012-12-18 19:54:56.858893 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 30 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbaa80 con 0x722c8c0
  185. -2241> 2012-12-18 19:54:56.858906 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968680
  186. -2240> 2012-12-18 19:54:56.858938 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 4555)
  187. -2239> 2012-12-18 19:54:56.858946 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 3744)
  188. -2238> 2012-12-18 19:54:56.858949 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 31 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2166446330 0 0) 0x9cc0600 con 0x722c8c0
  189. -2237> 2012-12-18 19:54:56.858960 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.858961 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  190. -2236> 2012-12-18 19:54:56.858973 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0800 con 0x722c8c0
  191. -2235> 2012-12-18 19:54:56.858987 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 4360)
  192. -2234> 2012-12-18 19:54:56.858992 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 3549)
  193. -2233> 2012-12-18 19:54:56.858996 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 32 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc0400 con 0x722c8c0
  194. -2232> 2012-12-18 19:54:56.859007 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859007 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  195. -2231> 2012-12-18 19:54:56.859018 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0600 con 0x722c8c0
  196. -2230> 2012-12-18 19:54:56.859029 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 4328)
  197. -2229> 2012-12-18 19:54:56.859035 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 3517)
  198. -2228> 2012-12-18 19:54:56.859038 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 33 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbb880 con 0x722c8c0
  199. -2227> 2012-12-18 19:54:56.859053 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968820
  200. -2226> 2012-12-18 19:54:56.859064 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 4267)
  201. -2225> 2012-12-18 19:54:56.859069 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 3456)
  202. -2224> 2012-12-18 19:54:56.859071 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 34 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (3378936191 0 0) 0x9cc0200 con 0x722c8c0
  203. -2223> 2012-12-18 19:54:56.859081 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859082 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  204. -2222> 2012-12-18 19:54:56.859092 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0400 con 0x722c8c0
  205. -2221> 2012-12-18 19:54:56.859107 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 4072)
  206. -2220> 2012-12-18 19:54:56.859111 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 3261)
  207. -2219> 2012-12-18 19:54:56.859115 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 35 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc0000 con 0x722c8c0
  208. -2218> 2012-12-18 19:54:56.859125 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859125 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  209. -2217> 2012-12-18 19:54:56.859136 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0200 con 0x722c8c0
  210. -2216> 2012-12-18 19:54:56.859147 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 4040)
  211. -2215> 2012-12-18 19:54:56.859150 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 3229)
  212. -2214> 2012-12-18 19:54:56.859153 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 36 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbb6c0 con 0x722c8c0
  213. -2213> 2012-12-18 19:54:56.859166 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c461a0
  214. -2212> 2012-12-18 19:54:56.859174 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 3979)
  215. -2211> 2012-12-18 19:54:56.859180 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 3168)
  216. -2210> 2012-12-18 19:54:56.859185 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 37 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (3767825454 0 0) 0x9cc1c00 con 0x722c8c0
  217. -2209> 2012-12-18 19:54:56.859194 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859195 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  218. -2208> 2012-12-18 19:54:56.859205 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0000 con 0x722c8c0
  219. -2207> 2012-12-18 19:54:56.859217 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 3784)
  220. -2206> 2012-12-18 19:54:56.859221 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 2973)
  221. -2205> 2012-12-18 19:54:56.859240 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 38 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc1a00 con 0x722c8c0
  222. -2204> 2012-12-18 19:54:56.859250 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859251 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  223. -2203> 2012-12-18 19:54:56.859260 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1c00 con 0x722c8c0
  224. -2202> 2012-12-18 19:54:56.859270 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 3752)
  225. -2201> 2012-12-18 19:54:56.859273 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 2941)
  226. -2200> 2012-12-18 19:54:56.859276 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 39 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbb500 con 0x722c8c0
  227. -2199> 2012-12-18 19:54:56.859291 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19696c0
  228. -2198> 2012-12-18 19:54:56.859316 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 3691)
  229. -2197> 2012-12-18 19:54:56.859322 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 2880)
  230. -2196> 2012-12-18 19:54:56.859325 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 40 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (3075075782 0 0) 0x9cc1800 con 0x722c8c0
  231. -2195> 2012-12-18 19:54:56.859336 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859337 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  232. -2194> 2012-12-18 19:54:56.859348 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1a00 con 0x722c8c0
  233. -2193> 2012-12-18 19:54:56.859360 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 3496)
  234. -2192> 2012-12-18 19:54:56.859364 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 2685)
  235. -2191> 2012-12-18 19:54:56.859368 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 41 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc1600 con 0x722c8c0
  236. -2190> 2012-12-18 19:54:56.859378 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859379 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  237. -2189> 2012-12-18 19:54:56.859389 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1800 con 0x722c8c0
  238. -2188> 2012-12-18 19:54:56.859400 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 3464)
  239. -2187> 2012-12-18 19:54:56.859407 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 2653)
  240. -2186> 2012-12-18 19:54:56.859411 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 42 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbb340 con 0x722c8c0
  241. -2185> 2012-12-18 19:54:56.859425 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19689c0
  242. -2184> 2012-12-18 19:54:56.859436 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 3403)
  243. -2183> 2012-12-18 19:54:56.859440 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 2592)
  244. -2182> 2012-12-18 19:54:56.859444 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 43 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (1628160383 0 0) 0x9cc1400 con 0x722c8c0
  245. -2181> 2012-12-18 19:54:56.859453 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859453 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  246. -2180> 2012-12-18 19:54:56.859464 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1600 con 0x722c8c0
  247. -2179> 2012-12-18 19:54:56.859476 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 3208)
  248. -2178> 2012-12-18 19:54:56.859481 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 2397)
  249. -2177> 2012-12-18 19:54:56.859484 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 44 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc1200 con 0x722c8c0
  250. -2176> 2012-12-18 19:54:56.859493 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859494 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  251. -2175> 2012-12-18 19:54:56.859504 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1400 con 0x722c8c0
  252. -2174> 2012-12-18 19:54:56.859517 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 3176)
  253. -2173> 2012-12-18 19:54:56.859521 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 2365)
  254. -2172> 2012-12-18 19:54:56.859524 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 45 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbb180 con 0x722c8c0
  255. -2171> 2012-12-18 19:54:56.859537 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968b60
  256. -2170> 2012-12-18 19:54:56.859548 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 3115)
  257. -2169> 2012-12-18 19:54:56.859551 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 2304)
  258. -2168> 2012-12-18 19:54:56.859555 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 46 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (1014975639 0 0) 0x9cc1000 con 0x722c8c0
  259. -2167> 2012-12-18 19:54:56.859565 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859565 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  260. -2166> 2012-12-18 19:54:56.859576 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1200 con 0x722c8c0
  261. -2165> 2012-12-18 19:54:56.859587 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 2920)
  262. -2164> 2012-12-18 19:54:56.859590 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 2109)
  263. -2163> 2012-12-18 19:54:56.859595 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 47 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc0e00 con 0x722c8c0
  264. -2162> 2012-12-18 19:54:56.859603 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859604 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  265. -2161> 2012-12-18 19:54:56.859614 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1000 con 0x722c8c0
  266. -2160> 2012-12-18 19:54:56.859624 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 2888)
  267. -2159> 2012-12-18 19:54:56.859629 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 2077)
  268. -2158> 2012-12-18 19:54:56.859631 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 48 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbbdc0 con 0x722c8c0
  269. -2157> 2012-12-18 19:54:56.859645 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968d00
  270. -2156> 2012-12-18 19:54:56.859655 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 2827)
  271. -2155> 2012-12-18 19:54:56.859660 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 2016)
  272. -2154> 2012-12-18 19:54:56.859664 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 49 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2236280532 0 0) 0x9cc1e00 con 0x722c8c0
  273. -2153> 2012-12-18 19:54:56.859673 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859674 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  274. -2152> 2012-12-18 19:54:56.859685 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc0e00 con 0x722c8c0
  275. -2151> 2012-12-18 19:54:56.859698 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 2632)
  276. -2150> 2012-12-18 19:54:56.859703 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 1821)
  277. -2149> 2012-12-18 19:54:56.859705 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 50 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc7200 con 0x722c8c0
  278. -2148> 2012-12-18 19:54:56.859715 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859715 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  279. -2147> 2012-12-18 19:54:56.859726 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc1e00 con 0x722c8c0
  280. -2146> 2012-12-18 19:54:56.859739 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 2600)
  281. -2145> 2012-12-18 19:54:56.859748 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 1789)
  282. -2144> 2012-12-18 19:54:56.859750 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 51 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbbc00 con 0x722c8c0
  283. -2143> 2012-12-18 19:54:56.859764 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968340
  284. -2142> 2012-12-18 19:54:56.859776 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 2539)
  285. -2141> 2012-12-18 19:54:56.859779 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 1728)
  286. -2140> 2012-12-18 19:54:56.859783 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 52 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (314313745 0 0) 0x9cc7000 con 0x722c8c0
  287. -2139> 2012-12-18 19:54:56.859792 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859793 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  288. -2138> 2012-12-18 19:54:56.859804 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc7200 con 0x722c8c0
  289. -2137> 2012-12-18 19:54:56.859814 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 2344)
  290. -2136> 2012-12-18 19:54:56.859819 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 1533)
  291. -2135> 2012-12-18 19:54:56.859821 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 53 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc6e00 con 0x722c8c0
  292. -2134> 2012-12-18 19:54:56.859831 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859832 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  293. -2133> 2012-12-18 19:54:56.859842 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc7000 con 0x722c8c0
  294. -2132> 2012-12-18 19:54:56.859853 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 2312)
  295. -2131> 2012-12-18 19:54:56.859857 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 1501)
  296. -2130> 2012-12-18 19:54:56.859861 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 54 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cbba40 con 0x722c8c0
  297. -2129> 2012-12-18 19:54:56.859874 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19681a0
  298. -2128> 2012-12-18 19:54:56.859884 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 2251)
  299. -2127> 2012-12-18 19:54:56.859889 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 1440)
  300. -2126> 2012-12-18 19:54:56.859893 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 55 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (3598513576 0 0) 0x9cc6c00 con 0x722c8c0
  301. -2125> 2012-12-18 19:54:56.859902 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859903 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  302. -2124> 2012-12-18 19:54:56.859912 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6e00 con 0x722c8c0
  303. -2123> 2012-12-18 19:54:56.859924 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 2056)
  304. -2122> 2012-12-18 19:54:56.859929 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 1245)
  305. -2121> 2012-12-18 19:54:56.859933 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 56 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc6a00 con 0x722c8c0
  306. -2120> 2012-12-18 19:54:56.859943 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.859944 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  307. -2119> 2012-12-18 19:54:56.859954 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6c00 con 0x722c8c0
  308. -2118> 2012-12-18 19:54:56.859966 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 2024)
  309. -2117> 2012-12-18 19:54:56.859969 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 1213)
  310. -2116> 2012-12-18 19:54:56.859972 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 57 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9ccaa80 con 0x722c8c0
  311. -2115> 2012-12-18 19:54:56.859987 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19684e0
  312. -2114> 2012-12-18 19:54:56.859998 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 1963)
  313. -2113> 2012-12-18 19:54:56.860002 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 1152)
  314. -2112> 2012-12-18 19:54:56.860006 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 58 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (467599072 0 0) 0x9cc6800 con 0x722c8c0
  315. -2111> 2012-12-18 19:54:56.860015 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860016 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  316. -2110> 2012-12-18 19:54:56.860027 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6a00 con 0x722c8c0
  317. -2109> 2012-12-18 19:54:56.860038 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 1768)
  318. -2108> 2012-12-18 19:54:56.860041 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 957)
  319. -2107> 2012-12-18 19:54:56.860045 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 59 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc6600 con 0x722c8c0
  320. -2106> 2012-12-18 19:54:56.860054 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860054 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  321. -2105> 2012-12-18 19:54:56.860065 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6800 con 0x722c8c0
  322. -2104> 2012-12-18 19:54:56.860077 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 1736)
  323. -2103> 2012-12-18 19:54:56.860083 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 925)
  324. -2102> 2012-12-18 19:54:56.860086 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 60 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cca8c0 con 0x722c8c0
  325. -2101> 2012-12-18 19:54:56.860099 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c464e0
  326. -2100> 2012-12-18 19:54:56.860110 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 1675)
  327. -2099> 2012-12-18 19:54:56.860113 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 864)
  328. -2098> 2012-12-18 19:54:56.860117 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 61 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2110997196 0 0) 0x9cc6400 con 0x722c8c0
  329. -2097> 2012-12-18 19:54:56.860125 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860126 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  330. -2096> 2012-12-18 19:54:56.860136 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6600 con 0x722c8c0
  331. -2095> 2012-12-18 19:54:56.860148 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 1480)
  332. -2094> 2012-12-18 19:54:56.860151 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 669)
  333. -2093> 2012-12-18 19:54:56.860155 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 62 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc6200 con 0x722c8c0
  334. -2092> 2012-12-18 19:54:56.860165 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860166 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  335. -2091> 2012-12-18 19:54:56.860176 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6400 con 0x722c8c0
  336. -2090> 2012-12-18 19:54:56.860186 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 1448)
  337. -2089> 2012-12-18 19:54:56.860191 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 637)
  338. -2088> 2012-12-18 19:54:56.860193 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 63 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cca700 con 0x722c8c0
  339. -2087> 2012-12-18 19:54:56.860208 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968820
  340. -2086> 2012-12-18 19:54:56.860219 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 1387)
  341. -2085> 2012-12-18 19:54:56.860222 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 576)
  342. -2084> 2012-12-18 19:54:56.860225 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 64 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (1198107870 0 0) 0x9cc6000 con 0x722c8c0
  343. -2083> 2012-12-18 19:54:56.860235 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860236 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  344. -2082> 2012-12-18 19:54:56.860246 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6200 con 0x722c8c0
  345. -2081> 2012-12-18 19:54:56.860258 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 1192)
  346. -2080> 2012-12-18 19:54:56.860261 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 381)
  347. -2079> 2012-12-18 19:54:56.860265 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 65 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc7e00 con 0x722c8c0
  348. -2078> 2012-12-18 19:54:56.860274 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860275 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  349. -2077> 2012-12-18 19:54:56.860284 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc6000 con 0x722c8c0
  350. -2076> 2012-12-18 19:54:56.860295 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 1160)
  351. -2075> 2012-12-18 19:54:56.860298 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 349)
  352. -2074> 2012-12-18 19:54:56.860302 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 66 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cca540 con 0x722c8c0
  353. -2073> 2012-12-18 19:54:56.860315 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968680
  354. -2072> 2012-12-18 19:54:56.860326 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 1099)
  355. -2071> 2012-12-18 19:54:56.860329 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 288)
  356. -2070> 2012-12-18 19:54:56.860331 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 67 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2870579466 0 0) 0x9cc7c00 con 0x722c8c0
  357. -2069> 2012-12-18 19:54:56.860341 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860341 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  358. -2068> 2012-12-18 19:54:56.860352 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc7e00 con 0x722c8c0
  359. -2067> 2012-12-18 19:54:56.860364 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 904)
  360. -2066> 2012-12-18 19:54:56.860369 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 93)
  361. -2065> 2012-12-18 19:54:56.860372 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 68 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc7a00 con 0x722c8c0
  362. -2064> 2012-12-18 19:54:56.860381 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.860381 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  363. -2063> 2012-12-18 19:54:56.860392 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6800/1864 -- auth_reply(proto 2 -22 Invalid argument) v1 -- ?+0 0x9cc7c00 con 0x722c8c0
  364. -2062> 2012-12-18 19:54:56.860402 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 872)
  365. -2061> 2012-12-18 19:54:56.860409 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 61)
  366. -2060> 2012-12-18 19:54:56.860412 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.12:6800/1864 69 ==== mon_subscribe({mdsmap=2486+,monmap=2+,osdmap=9917}) v2 ==== 61+0+0 (2693076204 0 0) 0x9cca380 con 0x722c8c0
  367. -2059> 2012-12-18 19:54:56.860426 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.12:6800/1864 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19684e0
  368. -2058> 2012-12-18 19:54:56.860437 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 811)
  369. -2057> 2012-12-18 19:54:56.860440 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 0)
  370. -2056> 2012-12-18 19:54:56.860706 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (811 -> 853)
  371. -2055> 2012-12-18 19:54:56.860722 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  372. -2054> 2012-12-18 19:54:56.860838 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (853 -> 875)
  373. -2053> 2012-12-18 19:54:56.860844 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  374. -2052> 2012-12-18 19:54:56.860840 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 3 ==== mon_subscribe({monmap=2+,osdmap=9302}) v2 ==== 42+0+0 (3235658956 0 0) 0x9cbac40 con 0x9a93400
  375. -2051> 2012-12-18 19:54:56.860866 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9302..10022] to osd.16 10.1.0.13:6818/10436
  376. -2050> 2012-12-18 19:54:56.860883 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1029 (875 -> 1904)
  377. -2049> 2012-12-18 19:54:56.860888 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1029 (64 -> 1093)
  378. -2048> 2012-12-18 19:54:56.861927 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.16 10.1.0.13:6818/10436 -- osd_map(9302..9402 src has 7326..10022) v3 -- ?+0 0x9cc7a00
  379. -2047> 2012-12-18 19:54:56.861950 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.16 10.1.0.13:6818/10436 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1968340
  380. -2046> 2012-12-18 19:54:56.862003 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1862)
  381. -2045> 2012-12-18 19:54:56.862007 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1051)
  382. -2044> 2012-12-18 19:54:56.862017 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x19696c0 con 0x9a93400
  383. -2043> 2012-12-18 19:54:56.862036 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.862037 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  384. -2042> 2012-12-18 19:54:56.862054 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1029)
  385. -2041> 2012-12-18 19:54:56.862059 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 5 ==== osd_pgtemp(e10021 {0.52d=[],0.69e=[],1.29c=[],1.52c=[],1.69d=[],18.e59=[16,41,10],18.ecb=[16,51,55],18.2fda=[],18.51fe=[16,53,10],18.5a48=[16,34,29],18.6151=[],18.7999=[],18.8906=[16,56,65],18.d4ef=[16,51,68],18.df4e=[16,51,10],18.11e33=[],18.12507=[],18.1268c=[16,54,68],18.1314c=[],18.1453b=[16,41,52,68],18.1463c=[16,59,10],18.14741=[16,32,61],18.1c8ab=[55,46,64,16],18.1eb24=[],18.21050=[16,32,68],18.23397=[16,24,31,0],18.2589d=[16,73,38,3],18.25942=[],18.2916d=[16,52,64],18.2925a=[],18.2b4dd=[16,59,67],18.2cc8d=[16,73,11],18.2d3a7=[16,55,60],18.2ec3b=[16,44,10],18.2f650=[16,56,10]} v10021) v1 ==== 1029+0+0 (3585993771 0 0) 0x9cdc780 con 0x9a93400
  386. -2040> 2012-12-18 19:54:56.862098 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.862099 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  387. -2039> 2012-12-18 19:54:56.862131 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1029 (0x8e4928 -> 0)
  388. -2038> 2012-12-18 19:54:56.863036 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (1862 -> 1904)
  389. -2037> 2012-12-18 19:54:56.863052 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  390. -2036> 2012-12-18 19:54:56.863073 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (1904 -> 1946)
  391. -2035> 2012-12-18 19:54:56.863083 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (42 -> 84)
  392. -2034> 2012-12-18 19:54:56.863128 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (1946 -> 1968)
  393. -2033> 2012-12-18 19:54:56.863131 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (84 -> 106)
  394. -2032> 2012-12-18 19:54:56.863162 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 3 ==== mon_subscribe({monmap=2+,osdmap=9281}) v2 ==== 42+0+0 (1924620553 0 0) 0x9cbaa80 con 0x722d7c0
  395. -2031> 2012-12-18 19:54:56.863186 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9281..10022] to osd.27 10.1.0.15:6818/19812
  396. -2030> 2012-12-18 19:54:56.863132 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (1968 -> 1990)
  397. -2029> 2012-12-18 19:54:56.863298 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (106 -> 128)
  398. -2028> 2012-12-18 19:54:56.863304 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 369 (1990 -> 2359)
  399. -2027> 2012-12-18 19:54:56.863310 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 369 (128 -> 497)
  400. -2026> 2012-12-18 19:54:56.863380 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 473 (2359 -> 2832)
  401. -2025> 2012-12-18 19:54:56.863385 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 473 (497 -> 970)
  402. -2024> 2012-12-18 19:54:56.863767 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (2832 -> 2874)
  403. -2023> 2012-12-18 19:54:56.863778 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (970 -> 1012)
  404. -2022> 2012-12-18 19:54:56.863808 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (2874 -> 2896)
  405. -2021> 2012-12-18 19:54:56.863856 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (1012 -> 1034)
  406. -2020> 2012-12-18 19:54:56.863897 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 704 (2896 -> 3600)
  407. -2019> 2012-12-18 19:54:56.863903 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 704 (1034 -> 1738)
  408. -2018> 2012-12-18 19:54:56.864154 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (3600 -> 3642)
  409. -2017> 2012-12-18 19:54:56.864165 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (1738 -> 1780)
  410. -2016> 2012-12-18 19:54:56.864160 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.27 10.1.0.15:6818/19812 -- osd_map(9281..9381 src has 7326..10022) v3 -- ?+0 0xc395c00
  411. -2015> 2012-12-18 19:54:56.864166 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (3642 -> 3684)
  412. -2014> 2012-12-18 19:54:56.864176 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (1780 -> 1822)
  413. -2013> 2012-12-18 19:54:56.864181 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.27 10.1.0.15:6818/19812 -- mon_subscribe_ack(300s) v1 -- ?+0 0x11c461a0
  414. -2012> 2012-12-18 19:54:56.864189 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 3642)
  415. -2011> 2012-12-18 19:54:56.864194 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1780)
  416. -2010> 2012-12-18 19:54:56.864280 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 3 ==== mon_subscribe({monmap=2+,osdmap=9326}) v2 ==== 42+0+0 (3887299130 0 0) 0x9cbb880 con 0x72123c0
  417. -2009> 2012-12-18 19:54:56.864206 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (3642 -> 3664)
  418. -2008> 2012-12-18 19:54:56.864302 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9326..10022] to osd.24 10.1.0.15:6809/18192
  419. -2007> 2012-12-18 19:54:56.864321 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (1780 -> 1802)
  420. -2006> 2012-12-18 19:54:56.864398 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (3664 -> 3686)
  421. -2005> 2012-12-18 19:54:56.864456 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (1802 -> 1824)
  422. -2004> 2012-12-18 19:54:56.864463 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (3686 -> 4166)
  423. -2003> 2012-12-18 19:54:56.864472 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (1824 -> 2304)
  424. -2002> 2012-12-18 19:54:56.864481 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 661 (4166 -> 4827)
  425. -2001> 2012-12-18 19:54:56.864486 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 661 (2304 -> 2965)
  426. -2000> 2012-12-18 19:54:56.864653 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (4827 -> 4869)
  427. -1999> 2012-12-18 19:54:56.864666 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (2965 -> 3007)
  428. -1998> 2012-12-18 19:54:56.864681 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (4869 -> 4911)
  429. -1997> 2012-12-18 19:54:56.864692 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (3007 -> 3049)
  430. -1996> 2012-12-18 19:54:56.864698 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (4911 -> 4933)
  431. -1995> 2012-12-18 19:54:56.864703 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (3049 -> 3071)
  432. -1994> 2012-12-18 19:54:56.864717 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 645 (4933 -> 5578)
  433. -1993> 2012-12-18 19:54:56.864721 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 645 (3071 -> 3716)
  434. -1992> 2012-12-18 19:54:56.864737 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (5578 -> 5600)
  435. -1991> 2012-12-18 19:54:56.864744 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (3716 -> 3738)
  436. -1990> 2012-12-18 19:54:56.864765 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 931 (5600 -> 6531)
  437. -1989> 2012-12-18 19:54:56.864770 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 931 (3738 -> 4669)
  438. -1988> 2012-12-18 19:54:56.865121 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (6531 -> 6573)
  439. -1987> 2012-12-18 19:54:56.865130 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (4669 -> 4711)
  440. -1986> 2012-12-18 19:54:56.865158 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (6573 -> 6595)
  441. -1985> 2012-12-18 19:54:56.865221 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (4711 -> 4733)
  442. -1984> 2012-12-18 19:54:56.865178 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.24 10.1.0.15:6809/18192 -- osd_map(9326..9426 src has 7326..10022) v3 -- ?+0 0xb7b6000
  443. -1983> 2012-12-18 19:54:56.865243 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.24 10.1.0.15:6809/18192 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b6bd40
  444. -1982> 2012-12-18 19:54:56.865297 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 6553)
  445. -1981> 2012-12-18 19:54:56.865302 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 4691)
  446. -1980> 2012-12-18 19:54:56.865306 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x19689c0 con 0x722d7c0
  447. -1979> 2012-12-18 19:54:56.865318 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.865319 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  448. -1978> 2012-12-18 19:54:56.865339 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1106 (6553 -> 7659)
  449. -1977> 2012-12-18 19:54:56.865347 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1106 (4691 -> 5797)
  450. -1976> 2012-12-18 19:54:56.865355 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.865357 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  451. -1975> 2012-12-18 19:54:56.865367 7f977255a700 5 mon.a@0(leader).osd e10022 send_latest to osd.27 10.1.0.15:6818/19812 start 10021
  452. -1974> 2012-12-18 19:54:56.865373 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10021..10022] to osd.27 10.1.0.15:6818/19812
  453. -1973> 2012-12-18 19:54:56.865401 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6818/19812 -- osd_map(10021..10022 src has 7326..10022) v3 -- ?+0 0x632b200 con 0x722d7c0
  454. -1972> 2012-12-18 19:54:56.865411 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 22 (0x8e4928 -> 7637)
  455. -1971> 2012-12-18 19:54:56.865415 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 5775)
  456. -1970> 2012-12-18 19:54:56.865418 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 3 ==== mon_subscribe({monmap=2+,osdmap=9708}) v2 ==== 42+0+0 (3849771159 0 0) 0x9cbb6c0 con 0x7212780
  457. -1969> 2012-12-18 19:54:56.865431 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9708..10022] to osd.68 10.1.0.18:6821/19143
  458. -1968> 2012-12-18 19:54:56.866094 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.68 10.1.0.18:6821/19143 -- osd_map(9708..9808 src has 7326..10022) v3 -- ?+0 0x632a800
  459. -1967> 2012-12-18 19:54:56.866113 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.68 10.1.0.18:6821/19143 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5ade340
  460. -1966> 2012-12-18 19:54:56.866154 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7595)
  461. -1965> 2012-12-18 19:54:56.866158 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 5733)
  462. -1964> 2012-12-18 19:54:56.866161 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 3 ==== mon_subscribe({monmap=2+,osdmap=9314}) v2 ==== 42+0+0 (2165263006 0 0) 0x9cbac40 con 0x9a93a40
  463. -1963> 2012-12-18 19:54:56.866175 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9314..10022] to osd.29 10.1.0.15:6824/20884
  464. -1962> 2012-12-18 19:54:56.866777 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1102004 (7595 -> 1109599)
  465. -1961> 2012-12-18 19:54:56.866791 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1102004 (5733 -> 1107737)
  466. -1960> 2012-12-18 19:54:56.866995 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- osd_map(9314..9414 src has 7326..10022) v3 -- ?+0 0x632bc00
  467. -1959> 2012-12-18 19:54:56.867013 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5ade680
  468. -1958> 2012-12-18 19:54:56.867062 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1109557)
  469. -1957> 2012-12-18 19:54:56.867066 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1107695)
  470. -1956> 2012-12-18 19:54:56.867070 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 3 ==== mon_subscribe({monmap=2+,osdmap=9993}) v2 ==== 42+0+0 (3225562490 0 0) 0x9cbb500 con 0x722db80
  471. -1955> 2012-12-18 19:54:56.867084 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9993..10022] to osd.11 10.1.0.13:6803/7344
  472. -1954> 2012-12-18 19:54:56.867279 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.11 10.1.0.13:6803/7344 -- osd_map(9993..10022 src has 7326..10022) v3 -- ?+0 0x632ba00
  473. -1953> 2012-12-18 19:54:56.867297 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.11 10.1.0.13:6803/7344 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bef520
  474. -1952> 2012-12-18 19:54:56.867306 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1109515)
  475. -1951> 2012-12-18 19:54:56.867310 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1107653)
  476. -1950> 2012-12-18 19:54:56.867313 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x1969520 con 0x72123c0
  477. -1949> 2012-12-18 19:54:56.867323 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.867323 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  478. -1948> 2012-12-18 19:54:56.867339 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1107631)
  479. -1947> 2012-12-18 19:54:56.867344 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 3 ==== mon_subscribe({monmap=2+,osdmap=9992}) v2 ==== 42+0+0 (880606002 0 0) 0x9cbb340 con 0x9cd1180
  480. -1946> 2012-12-18 19:54:56.867358 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9992..10022] to osd.14 10.1.0.13:6812/8435
  481. -1945> 2012-12-18 19:54:56.867567 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.14 10.1.0.13:6812/8435 -- osd_map(9992..10022 src has 7326..10022) v3 -- ?+0 0x632a000
  482. -1944> 2012-12-18 19:54:56.867584 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.14 10.1.0.13:6812/8435 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b9c340
  483. -1943> 2012-12-18 19:54:56.867595 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1109473)
  484. -1942> 2012-12-18 19:54:56.867598 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1107589)
  485. -1941> 2012-12-18 19:54:56.867602 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 3 ==== mon_subscribe({monmap=2+,osdmap=9327}) v2 ==== 42+0+0 (328051826 0 0) 0x9cbb180 con 0x722c3c0
  486. -1940> 2012-12-18 19:54:56.867616 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9327..10022] to osd.72 10.1.0.18:6806/16967
  487. -1939> 2012-12-18 19:54:56.868326 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 573524 (1109473 -> 1682997)
  488. -1938> 2012-12-18 19:54:56.868340 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 573524 (1107589 -> 1681113)
  489. -1937> 2012-12-18 19:54:56.868388 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.72 10.1.0.18:6806/16967 -- osd_map(9327..9427 src has 7326..10022) v3 -- ?+0 0xc395800
  490. -1936> 2012-12-18 19:54:56.868406 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.72 10.1.0.18:6806/16967 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b6b860
  491. -1935> 2012-12-18 19:54:56.868448 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1682955)
  492. -1934> 2012-12-18 19:54:56.868451 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1681071)
  493. -1933> 2012-12-18 19:54:56.868455 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 3 ==== mon_subscribe({monmap=2+,osdmap=9418}) v2 ==== 42+0+0 (1963170396 0 0) 0x9cbbdc0 con 0x7212b40
  494. -1932> 2012-12-18 19:54:56.868468 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9418..10022] to osd.73 10.1.0.18:6809/17050
  495. -1931> 2012-12-18 19:54:56.869140 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.73 10.1.0.18:6809/17050 -- osd_map(9418..9518 src has 7326..10022) v3 -- ?+0 0xc395400
  496. -1930> 2012-12-18 19:54:56.869157 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.73 10.1.0.18:6809/17050 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5beed00
  497. -1929> 2012-12-18 19:54:56.869206 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 1682913)
  498. -1928> 2012-12-18 19:54:56.869210 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1681029)
  499. -1927> 2012-12-18 19:54:56.869213 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 5 ==== osd_pgtemp(e10021 {18.4eee=[27,34,72],18.5562=[27,59,60],18.b4f6=[27,49,10],18.ec3a=[],18.11894=[27,43,13],18.1c686=[27,54,62],18.1e5cd=[41,63,32,27],18.27494=[27,49,62],18.28763=[27,53,64],18.2fed8=[],18.3020a=[27,50,63]} v10021) v1 ==== 369+0+0 (3048990923 0 0) 0x9cdc960 con 0x722d7c0
  500. -1926> 2012-12-18 19:54:56.869234 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869235 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  501. -1925> 2012-12-18 19:54:56.869272 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 369 (0x8e4928 -> 1680660)
  502. -1924> 2012-12-18 19:54:56.869279 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x1969380 con 0x7212780
  503. -1923> 2012-12-18 19:54:56.869290 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869290 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  504. -1922> 2012-12-18 19:54:56.869303 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680638)
  505. -1921> 2012-12-18 19:54:56.869306 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x1968b60 con 0x9a93a40
  506. -1920> 2012-12-18 19:54:56.869315 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869316 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  507. -1919> 2012-12-18 19:54:56.869326 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680616)
  508. -1918> 2012-12-18 19:54:56.869331 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x1968d00 con 0x722db80
  509. -1917> 2012-12-18 19:54:56.869338 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869339 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  510. -1916> 2012-12-18 19:54:56.869349 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869350 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  511. -1915> 2012-12-18 19:54:56.869359 7f977255a700 5 mon.a@0(leader).osd e10022 send_latest to osd.11 10.1.0.13:6803/7344 start 10021
  512. -1914> 2012-12-18 19:54:56.869364 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10021..10022] to osd.11 10.1.0.13:6803/7344
  513. -1913> 2012-12-18 19:54:56.869392 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6803/7344 -- osd_map(10021..10022 src has 7326..10022) v3 -- ?+0 0x7228400 con 0x722db80
  514. -1912> 2012-12-18 19:54:56.869405 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 22 (0x8e4928 -> 1682891)
  515. -1911> 2012-12-18 19:54:56.869408 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680594)
  516. -1910> 2012-12-18 19:54:56.869413 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 5 ==== osd_pgtemp(e10021 {18.1a50=[24,54,65],18.54ed=[],18.6805=[24,45,13],18.10c9c=[24,57,62],18.119a0=[],18.1a102=[24,47,73],18.20916=[24,55,75],18.25bbb=[24,49,69],18.29271=[24,57,66],18.29421=[24,53,64],18.2a378=[24,52,53],18.2a661=[],18.2a8a4=[24,40,74],18.2b172=[],18.2e53d=[24,33,17]} v10021) v1 ==== 473+0+0 (3083706999 0 0) 0x9cdcf00 con 0x72123c0
  517. -1909> 2012-12-18 19:54:56.869437 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869438 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  518. -1908> 2012-12-18 19:54:56.869458 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 473 (0x8e4928 -> 1680121)
  519. -1907> 2012-12-18 19:54:56.869463 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19691e0 con 0x9cd1180
  520. -1906> 2012-12-18 19:54:56.869473 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869474 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  521. -1905> 2012-12-18 19:54:56.869486 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680099)
  522. -1904> 2012-12-18 19:54:56.869491 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x1969040 con 0x722c3c0
  523. -1903> 2012-12-18 19:54:56.869501 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869501 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  524. -1902> 2012-12-18 19:54:56.869513 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680077)
  525. -1901> 2012-12-18 19:54:56.869516 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x1969d40 con 0x7212b40
  526. -1900> 2012-12-18 19:54:56.869526 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869527 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  527. -1899> 2012-12-18 19:54:56.869538 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1680055)
  528. -1898> 2012-12-18 19:54:56.869541 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 5 ==== osd_pgtemp(e10021 {18.4072=[68,36,43,67],18.5e69=[],18.60c9=[68,40,19],18.9c9a=[68,50,44],18.a986=[],18.ac53=[68,52,67],18.bb43=[68,53,45],18.d106=[68,49,16],18.de16=[68,30,19],18.e405=[68,53,15],18.ef2a=[],18.f51c=[68,59,20],18.14f09=[68,44,52,47],18.16aaa=[],18.1d43d=[],18.1e3a6=[68,31,58,62],18.211d2=[68,50,25],18.22253=[68,57,43],18.22f2b=[68,32,60],18.251db=[68,54,67],18.2a0ae=[68,55,63],18.2e0d9=[68,37,49]} v10021) v1 ==== 704+0+0 (3987670576 0 0) 0x9cdd0e0 con 0x7212780
  529. -1897> 2012-12-18 19:54:56.869568 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869569 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  530. -1896> 2012-12-18 19:54:56.869593 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 704 (0x8e4928 -> 1679351)
  531. -1895> 2012-12-18 19:54:56.869598 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 5 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x9cddc20 con 0x9a93a40
  532. -1894> 2012-12-18 19:54:56.869620 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869621 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  533. -1893> 2012-12-18 19:54:56.869649 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 1678871)
  534. -1892> 2012-12-18 19:54:56.869654 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 5 ==== osd_pgtemp(e10021 {0.cde=[11,56,38,73],1.cdd=[11,56,38,73],18.2f97=[11,57,50],18.4b8c=[11,51,74],18.8aaa=[11,50,67],18.a5b2=[11,44,10],18.cbe4=[],18.10b47=[11,45,74],18.12607=[11,30,64,70],18.13c49=[11,42,10],18.1761a=[11,54,10],18.1d1df=[11,35,32,58],18.1e99d=[11,58,57,69],18.2242d=[11,41,10],18.26128=[11,44,68],18.26dd7=[11,35,32],18.2e68c=[11,38,24],18.2e9f0=[11,47,74],18.2ef07=[11,54,57]} v10021) v1 ==== 661+0+0 (2782794572 0 0) 0x9cdde00 con 0x722db80
  535. -1891> 2012-12-18 19:54:56.869682 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869682 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  536. -1890> 2012-12-18 19:54:56.869728 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 661 (0x8e4928 -> 1678210)
  537. -1889> 2012-12-18 19:54:56.869733 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 5 ==== osd_pgtemp(e10021 {0.b8a=[14,46,10],18.3f5=[14,50,10],18.b78=[14,46,10],18.6896=[14,43,10],18.7764=[14,32,10],18.c572=[14,75,10],18.d40c=[14,58,10],18.e540=[14,43,10],18.14ce5=[14,42,10],18.17240=[14,59,53],18.2102a=[14,54,10],18.22089=[14,41,70,47],18.24118=[14,48,10],18.253b2=[14,27,10],18.28201=[],18.28767=[14,41,75],18.29b69=[14,59,29],18.2ed5f=[14,54,10],18.2f973=[14,37,61]} v10021) v1 ==== 645+0+0 (620895844 0 0) 0x7b6c000 con 0x9cd1180
  538. -1888> 2012-12-18 19:54:56.869759 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869760 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  539. -1887> 2012-12-18 19:54:56.869780 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 645 (0x8e4928 -> 1677565)
  540. -1886> 2012-12-18 19:54:56.869784 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 5 ==== osd_pgtemp(e10021 {18.13e3=[72,37,22],18.181c=[72,57,22],18.3af3=[72,37,29],18.4325=[72,49,60],18.4aba=[72,57,67],18.6ed2=[72,46,25],18.7042=[],18.7683=[72,59,24],18.8737=[72,50,28],18.9599=[72,50,66],18.9c4f=[72,58,44],18.e7b1=[72,51,63],18.ed3e=[72,54,65],18.1058c=[72,52,12],18.1226e=[],18.143b1=[72,56,61],18.1bada=[72,31,53,3],18.1e19a=[],18.1e623=[72,57,62],18.1e8f3=[72,46,11],18.23643=[72,53,44],18.249d9=[72,47,56],18.259b8=[72,33,66],18.2a29a=[],18.2b05a=[72,51,35,29],18.2d1e9=[],18.2d83f=[72,44,65],18.2e322=[72,40,24],18.2f47f=[72,32,33]} v10021) v1 ==== 931+0+0 (705663956 0 0) 0x7b6c1e0 con 0x722c3c0
  541. -1885> 2012-12-18 19:54:56.869819 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869820 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  542. -1884> 2012-12-18 19:54:56.869843 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 931 (0x8e4928 -> 1676634)
  543. -1883> 2012-12-18 19:54:56.869848 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 5 ==== osd_pgtemp(e10021 {18.14d0=[73,41,14],18.2e89=[73,46,26],18.4b39=[73,46,27],18.4e83=[73,54,26],18.5e8c=[73,54,16],18.680b=[73,57,12],18.6e4a=[],18.84eb=[73,54,14],18.92c3=[73,59,15],18.bf3c=[73,38,69],18.c770=[],18.c97a=[],18.d918=[73,54,55],18.dd17=[],18.ddc1=[73,53,67],18.e9e2=[],18.f144=[73,42,64],18.194d4=[],18.19ee2=[73,45,62],18.1c2b0=[73,46,60],18.1e691=[],18.20164=[73,55,66],18.20e8a=[],18.21ca1=[73,44,62],18.23180=[73,41,20],18.25797=[73,56,63],18.25b96=[],18.2630b=[73,32,62],18.27d6c=[73,30,29],18.28f3d=[73,47,60],18.29d58=[73,56,20],18.2b811=[73,30,12],18.2ce65=[73,41,50],18.2cf23=[73,34,65],18.2d697=[73,30,45],18.2f6b4=[73,33,61]} v10021) v1 ==== 1106+0+0 (1505313817 0 0) 0x7b6c3c0 con 0x7212b40
  544. -1882> 2012-12-18 19:54:56.869887 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.869888 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  545. -1881> 2012-12-18 19:54:56.869913 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1106 (0x8e4928 -> 1675528)
  546. -1880> 2012-12-18 19:54:56.870148 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (1682891 -> 1683049)
  547. -1879> 2012-12-18 19:54:56.870163 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (1675528 -> 1675686)
  548. -1878> 2012-12-18 19:54:56.870492 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 558232 (1683049 -> 2241281)
  549. -1877> 2012-12-18 19:54:56.870502 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 558232 (1675686 -> 2233918)
  550. -1876> 2012-12-18 19:54:56.870511 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2294793615 0 0) 0x632ac00 con 0x7003e00
  551. -1875> 2012-12-18 19:54:56.870530 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.870531 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  552. -1874> 2012-12-18 19:54:56.870694 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6816/13746 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xb7b6800 con 0x7003e00
  553. -1873> 2012-12-18 19:54:56.870714 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2241123)
  554. -1872> 2012-12-18 19:54:56.870719 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2233760)
  555. -1871> 2012-12-18 19:54:56.872595 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (2241123 -> 2241281)
  556. -1870> 2012-12-18 19:54:56.872608 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (2233760 -> 2233918)
  557. -1869> 2012-12-18 19:54:56.872805 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (2241281 -> 2241439)
  558. -1868> 2012-12-18 19:54:56.872817 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (2233918 -> 2234076)
  559. -1867> 2012-12-18 19:54:56.872810 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3128454968 0 0) 0x7210000 con 0x722d540
  560. -1866> 2012-12-18 19:54:56.872831 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.872832 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  561. -1865> 2012-12-18 19:54:56.872999 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (2241439 -> 2241597)
  562. -1864> 2012-12-18 19:54:56.873014 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (2234076 -> 2234234)
  563. -1863> 2012-12-18 19:54:56.873006 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6801/16285 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x632ac00 con 0x722d540
  564. -1862> 2012-12-18 19:54:56.873021 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (2241597 -> 2241755)
  565. -1861> 2012-12-18 19:54:56.873032 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (2234234 -> 2234392)
  566. -1860> 2012-12-18 19:54:56.873033 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2241597)
  567. -1859> 2012-12-18 19:54:56.873039 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2234234)
  568. -1858> 2012-12-18 19:54:56.873083 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2360977822 0 0) 0x7004e00 con 0x7213e00
  569. -1857> 2012-12-18 19:54:56.873097 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (2241597 -> 2241755)
  570. -1856> 2012-12-18 19:54:56.873107 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (2234234 -> 2234392)
  571. -1855> 2012-12-18 19:54:56.873100 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.873101 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  572. -1854> 2012-12-18 19:54:56.873270 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6810/16733 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7210000 con 0x7213e00
  573. -1853> 2012-12-18 19:54:56.873289 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2241597)
  574. -1852> 2012-12-18 19:54:56.873294 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2234234)
  575. -1851> 2012-12-18 19:54:56.873298 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2530643325 0 0) 0x7004200 con 0x722cdc0
  576. -1850> 2012-12-18 19:54:56.873337 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.873338 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  577. -1849> 2012-12-18 19:54:56.873390 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (2241597 -> 2241639)
  578. -1848> 2012-12-18 19:54:56.873403 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (2234234 -> 2234276)
  579. -1847> 2012-12-18 19:54:56.873437 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (2241639 -> 2241661)
  580. -1846> 2012-12-18 19:54:56.873445 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (2234276 -> 2234298)
  581. -1845> 2012-12-18 19:54:56.873483 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6828/20636 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7004e00 con 0x722cdc0
  582. -1844> 2012-12-18 19:54:56.873527 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2241503)
  583. -1843> 2012-12-18 19:54:56.873533 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2234140)
  584. -1842> 2012-12-18 19:54:56.873537 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (205636995 0 0) 0x7004800 con 0x722d040
  585. -1841> 2012-12-18 19:54:56.873548 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.873549 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  586. -1840> 2012-12-18 19:54:56.873665 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3892 (2241503 -> 2245395)
  587. -1839> 2012-12-18 19:54:56.873677 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3892 (2234140 -> 2238032)
  588. -1838> 2012-12-18 19:54:56.873690 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6813/16922 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7004200 con 0x722d040
  589. -1837> 2012-12-18 19:54:56.873708 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2245237)
  590. -1836> 2012-12-18 19:54:56.873715 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2237874)
  591. -1835> 2012-12-18 19:54:56.873720 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1339345092 0 0) 0x632ac00 con 0x7212dc0
  592. -1834> 2012-12-18 19:54:56.873731 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.873734 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  593. -1833> 2012-12-18 19:54:56.873883 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6819/18681 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7004800 con 0x7212dc0
  594. -1832> 2012-12-18 19:54:56.873900 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 2245079)
  595. -1831> 2012-12-18 19:54:56.873906 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 2237716)
  596. -1830> 2012-12-18 19:54:56.873926 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 3 ==== mon_subscribe({monmap=2+,osdmap=9340}) v2 ==== 42+0+0 (2133406596 0 0) 0x9cba380 con 0x7003e00
  597. -1829> 2012-12-18 19:54:56.873941 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9340..10022] to osd.55 10.1.0.16:6816/13746
  598. -1828> 2012-12-18 19:54:56.874527 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 553236 (2245079 -> 2798315)
  599. -1827> 2012-12-18 19:54:56.874546 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 553236 (2237716 -> 2790952)
  600. -1826> 2012-12-18 19:54:56.874728 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.55 10.1.0.16:6816/13746 -- osd_map(9340..9440 src has 7326..10022) v3 -- ?+0 0x632ac00
  601. -1825> 2012-12-18 19:54:56.874749 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.55 10.1.0.16:6816/13746 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bef1e0
  602. -1824> 2012-12-18 19:54:56.874771 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 2798273)
  603. -1823> 2012-12-18 19:54:56.874775 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 2790910)
  604. -1822> 2012-12-18 19:54:56.874780 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x5ade1a0 con 0x7003e00
  605. -1821> 2012-12-18 19:54:56.874790 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.874791 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  606. -1820> 2012-12-18 19:54:56.874807 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 2790888)
  607. -1819> 2012-12-18 19:54:56.874813 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 5 ==== osd_pgtemp(e10021 {0.777=[27,55,61],1.174=[22,55,60],1.197=[11,55,75],1.776=[27,55,61],18.a7c=[55,48,74],18.1325=[30,55,58],18.180a=[55,48,29],18.1839=[55,46,11],18.18de=[17,55,71],18.20d4=[10,55,27],18.2a2d=[],18.2f59=[55,49,65],18.38e8=[55,41,61],18.3b8e=[50,55,61],18.5206=[55,33,64],18.54be=[25,55,69],18.6354=[38,55,75],18.68f0=[55,36,31],18.6de1=[55,48,58],18.6fe7=[55,71,63],18.73c8=[53,55,73],18.793e=[66,55,75],18.7d57=[58,55,71],18.8231=[55,46,65],18.847c=[],18.9e5c=[55,45,20],18.9f7f=[62,55,72],18.aa7b=[42,55,57],18.ac39=[32,55,68],18.ad58=[55,45,13],18.af76=[22,55,33],18.b7cf=[],18.c0de=[18,55,34],18.c641=[55,67,17],18.c657=[55,47,14],18.c8cb=[55,73,28],18.c979=[55,43,60],18.d5b0=[55,20,14],18.d7bb=[55,49,46],18.e325=[12,55,42],18.feec=[55,37,15],18.107d6=[55,67,22],18.10d5f=[55,71,74],18.114ff=[55,28,66],18.1266c=[59,55,68],18.1282c=[55,46,51],18.12967=[55,72,16],18.129d0=[55,43,40],18.12f92=[28,55,61],18.13173=[43,55,59],18.13d15=[10,55,13],18.13e17=[55,74,62],18.142d7=[28,55,66],18.14308=[55,43,27],18.1431f=[43,55,60],18.14b73=[55,43,10],18.15192=[55,71,72],18.1636d=[55,42,66,24],18.16e69=[19,55,73],18.17e10=[],18.17e63=[48,55,66],18.18006=[32,55,69],18.1861a=[13,55,60],18.18633=[],18.195c6=[55,69,32],18.1977a=[29,55,52],18.19b2e=[55,27,64],18.1aa1a=[44,55,58],18.1ab80=[55,69,18],18.1aecb=[29,55,72],18.1af59=[26,55,53],18.1b920=[55,60,21],18.1bb5d=[55,46,69],18.1c59f=[49,55,56],18.1c699=[],18.1cab5=[55,64,13],18.1d226=[10,55,29],18.1da8f=[],18.1ef5a=[],18.1f970=[47,55,67],18.1f9ec=[66,55,74],18.2011a=[55,37,71],18.20ea7=[14,55,59],18.20f5a=[55,21,70],18.21a22=[42,65,55,63],18.21e6a=[],18.21ee6=[55,75,24],18.22a10=[55,42,60],18.22af0=[55,47,61],18.23c5a=[55,38,70],18.24160=[],18.24bcd=[55,60,10],18.2500d=[55,20,15],18.25dec=[55,45,19],18.26152=[55,72,69],18.269c9=[55,32,74],18.26e87=[55,63,16],18.28754=[55,32,65],18.28beb=[52,55,72],18.29042=[20,55,48],18.294d1=[55,60,54],18.299fa=[55,49,25],18.29ab2=[55,44,17],18.2a11a=[55,60,27],18.2a19d=[],18.2a456=[25,55,62],18.2a6be=[41,55,67],18.2c3eb=[55,37,69],18.2cc5a=[55,61,20],18.2cd13=[55,32,26],18.2d809=[55,75,56],18.2d920=[55,75,67],18.2dc18=[],18.2ddac=[55,40,68],18.2e2b1=[55,68,11],18.2eaa7=[53,55,74],18.2f4a8=[55,47,18],18.2f4f3=[55,28,67],18.2ff43=[55,49,25],18.3045c=[],18.30aa5=[],18.30beb=[55,70,62]} v10021) v1 ==== 3892+0+0 (2148732139 0 0) 0x9cdc5a0 con 0x7003e00
  608. -1818> 2012-12-18 19:54:56.874921 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.874922 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  609. -1817> 2012-12-18 19:54:56.874986 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3892 (0x8e4928 -> 2786996)
  610. -1816> 2012-12-18 19:54:56.875608 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (2798273 -> 3456033)
  611. -1815> 2012-12-18 19:54:56.875646 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (2786996 -> 3444756)
  612. -1814> 2012-12-18 19:54:56.875929 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 614792 (3456033 -> 4070825)
  613. -1814> 2012-12-18 19:54:56.875941 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 614792 (3444756 -> 4059548)
  614. -1815> 2012-12-18 19:54:56.876272 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 625704 (4070825 -> 4696529)
  615. -1814> 2012-12-18 19:54:56.876282 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 625704 (4059548 -> 4685252)
  616. -1813> 2012-12-18 19:54:56.876674 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1109416 (4696529 -> 5805945)
  617. -1812> 2012-12-18 19:54:56.876684 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1109416 (4685252 -> 5794668)
  618. -1811> 2012-12-18 19:54:56.877798 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1082484 (5805945 -> 6888429)
  619. -1810> 2012-12-18 19:54:56.877809 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1082484 (5794668 -> 6877152)
  620. -1809> 2012-12-18 19:54:56.878008 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1089896 (6888429 -> 7978325)
  621. -1808> 2012-12-18 19:54:56.878018 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1089896 (6877152 -> 7967048)
  622. -1807> 2012-12-18 19:54:56.878976 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (7978325 -> 7978367)
  623. -1806> 2012-12-18 19:54:56.878993 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (7967048 -> 7967090)
  624. -1805> 2012-12-18 19:54:56.879142 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (7978367 -> 7978409)
  625. -1804> 2012-12-18 19:54:56.879153 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (7967090 -> 7967132)
  626. -1803> 2012-12-18 19:54:56.879323 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (7978409 -> 7978431)
  627. -1802> 2012-12-18 19:54:56.879463 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (7967132 -> 7967154)
  628. -1801> 2012-12-18 19:54:56.879516 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3257 (7978431 -> 7981688)
  629. -1800> 2012-12-18 19:54:56.879587 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3257 (7967154 -> 7970411)
  630. -1799> 2012-12-18 19:54:56.879329 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 3 ==== mon_subscribe({monmap=2+,osdmap=9356}) v2 ==== 42+0+0 (2543827843 0 0) 0x9ccb6c0 con 0x722d540
  631. -1798> 2012-12-18 19:54:56.879676 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9356..10022] to osd.50 10.1.0.16:6801/16285
  632. -1797> 2012-12-18 19:54:56.879748 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (7981688 -> 7981730)
  633. -1796> 2012-12-18 19:54:56.879762 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (7970411 -> 7970453)
  634. -1795> 2012-12-18 19:54:56.879764 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (7981730 -> 7981752)
  635. -1794> 2012-12-18 19:54:56.879788 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (7970453 -> 7970475)
  636. -1793> 2012-12-18 19:54:56.879807 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (7981752 -> 7981774)
  637. -1792> 2012-12-18 19:54:56.879819 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (7970475 -> 7970497)
  638. -1791> 2012-12-18 19:54:56.879983 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3567 (7981774 -> 7985341)
  639. -1790> 2012-12-18 19:54:56.879992 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3567 (7970497 -> 7974064)
  640. -1789> 2012-12-18 19:54:56.880197 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3687 (7985341 -> 7989028)
  641. -1788> 2012-12-18 19:54:56.880210 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3687 (7974064 -> 7977751)
  642. -1787> 2012-12-18 19:54:56.880384 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.50 10.1.0.16:6801/16285 -- osd_map(9356..9456 src has 7326..10022) v3 -- ?+0 0x721dc00
  643. -1786> 2012-12-18 19:54:56.880406 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.50 10.1.0.16:6801/16285 -- mon_subscribe_ack(300s) v1 -- ?+0 0x1969ba0
  644. -1785> 2012-12-18 19:54:56.880429 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7988986)
  645. -1784> 2012-12-18 19:54:56.880434 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 7977709)
  646. -1783> 2012-12-18 19:54:56.880439 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 3 ==== mon_subscribe({monmap=2+,osdmap=9365}) v2 ==== 42+0+0 (2933460099 0 0) 0x9cca380 con 0x7213e00
  647. -1782> 2012-12-18 19:54:56.880454 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9365..10022] to osd.53 10.1.0.16:6810/16733
  648. -1781> 2012-12-18 19:54:56.881083 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.53 10.1.0.16:6810/16733 -- osd_map(9365..9465 src has 7326..10022) v3 -- ?+0 0x9cc6600
  649. -1780> 2012-12-18 19:54:56.881101 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.53 10.1.0.16:6810/16733 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bb1a00
  650. -1779> 2012-12-18 19:54:56.881150 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7988944)
  651. -1778> 2012-12-18 19:54:56.881154 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 7977667)
  652. -1777> 2012-12-18 19:54:56.881158 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 3 ==== mon_subscribe({monmap=2+,osdmap=9358}) v2 ==== 42+0+0 (2047762914 0 0) 0x9ccba40 con 0x722cdc0
  653. -1776> 2012-12-18 19:54:56.881171 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9358..10022] to osd.59 10.1.0.16:6828/20636
  654. -1775> 2012-12-18 19:54:56.881835 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.59 10.1.0.16:6828/20636 -- osd_map(9358..9458 src has 7326..10022) v3 -- ?+0 0x9cc7000
  655. -1774> 2012-12-18 19:54:56.881854 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.59 10.1.0.16:6828/20636 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bee1a0
  656. -1774> 2012-12-18 19:54:56.881863 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7988902)
  657. -1773> 2012-12-18 19:54:56.881867 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 7977625)
  658. -1774> 2012-12-18 19:54:56.881870 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x1969860 con 0x722d540
  659. -1773> 2012-12-18 19:54:56.881881 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.881881 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  660. -1772> 2012-12-18 19:54:56.881899 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 7977603)
  661. -1771> 2012-12-18 19:54:56.881902 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5ade000 con 0x7213e00
  662. -1770> 2012-12-18 19:54:56.881912 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.881913 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  663. -1769> 2012-12-18 19:54:56.881925 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 7977581)
  664. -1768> 2012-12-18 19:54:56.881928 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5bee820 con 0x722cdc0
  665. -1767> 2012-12-18 19:54:56.881938 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.881938 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  666. -1766> 2012-12-18 19:54:56.881950 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 7977559)
  667. -1765> 2012-12-18 19:54:56.881953 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 5 ==== osd_pgtemp(e10021 {0.12b=[],0.1ed=[42,50,66,64],0.768=[50,19,71],0.b5d=[10,50,38],1.12a=[],1.6db=[29,50,57],1.767=[50,19,71],1.b5c=[50,38,10],1.c09=[28,50,45],18.6ca=[29,50,57],18.b4b=[10,50,38],18.bf8=[50,45,28],18.1b19=[50,45,13],18.2514=[50,41,72],18.3f12=[50,29,64],18.403a=[17,50,61],18.452e=[50,75,69],18.4f96=[50,38,71],18.52b8=[50,25,11],18.5481=[50,46,69],18.5e10=[50,46,42],18.61e9=[50,75,16],18.661c=[50,42,40],18.78de=[50,27,63],18.7b0f=[50,34,12],18.855a=[50,43,24],18.8cd1=[50,74,18],18.9818=[50,49,27],18.9e9e=[50,69,60],18.9f89=[50,44,10],18.a86c=[50,44,60],18.b33a=[50,32,65],18.b6a9=[50,66,19],18.c25d=[50,45,73],18.cb52=[50,38,67],18.ccff=[50,73,68],18.d1d8=[50,47,56],18.d252=[50,32,24],18.efe4=[50,67,68],18.fb49=[50,63,69],18.fed4=[50,38,44],18.fedd=[50,70,71],18.115d9=[30,50,72],18.12a90=[26,50,57],18.13671=[48,50,73],18.13e68=[50,43,73],18.145ee=[19,50,53],18.14858=[50,44,66],18.14953=[20,50,55],18.14d8f=[10,50,26],18.173e5=[43,50,51],18.17b75=[29,50,73],18.17ca5=[28,50,72],18.17cc6=[46,50,60],18.1894e=[],18.19472=[50,35,39],18.19571=[63,50,74],18.195a7=[33,50,75],18.199b1=[50,46,68],18.19dcf=[13,50,71],18.1a189=[50,45,62],18.1a99e=[40,50,51],18.1bed1=[50,43,44],18.1c85d=[65,50,71],18.1d170=[],18.1d357=[59,50,69],18.1de70=[50,45,30],18.1dea4=[22,50,68],18.1dfde=[19,50,45],18.1ea8b=[],18.1f166=[21,50,72],18.1f1aa=[50,40,68],18.1f1b5=[19,50,72],18.1fe73=[50,71,75],18.2011d=[50,40,70],18.2017c=[66,50,75],18.205b5=[50,75,65],18.213aa=[50,44,62],18.21bb3=[50,68,16],18.221cf=[50,27,18],18.22206=[50,61,64],18.228ca=[50,45,63],18.22d9d=[50,46,74],18.23e66=[17,50,62],18.24382=[50,22,64],18.247e5=[50,49,26],18.2507f=[50,22,66],18.25475=[50,48,21],18.256de=[18,50,27],18.25a44=[50,47,63],18.27b01=[50,28,66],18.2894e=[50,48,13],18.29edc=[50,38,28],18.2a48d=[28,50,30],18.2a4a6=[50,44,66],18.2a64b=[50,65,25],18.2bb82=[50,47,17],18.2bcf5=[50,62,29],18.2ce5c=[50,65,22],18.2cf0b=[50,42,20],18.2d10b=[50,30,61],18.2de2d=[54,50,64],18.2f01a=[29,50,64],18.30017=[50,74,55],18.302f7=[62,50,71],18.3062b=[27,50,44],18.3082a=[50,45,60],18.30832=[15,50,58],18.30c57=[17,50,38]} v10021) v1 ==== 3567+0+0 (3808363155 0 0) 0x7b6cf00 con 0x722d540
  668. -1764> 2012-12-18 19:54:56.882122 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.882124 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  669. -1763> 2012-12-18 19:54:56.882186 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3567 (0x8e4928 -> 7973992)
  670. -1762> 2012-12-18 19:54:56.882191 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 5 ==== osd_pgtemp(e10021 {0.457=[40,53,69],0.7bc=[10,53,20],1.456=[40,53,69],1.7bb=[10,53,20],18.445=[53,40,69],18.7aa=[53,10,20],18.1967=[53,41,28],18.1dbb=[53,34,60],18.1efd=[53,42,73],18.22f8=[53,73,61],18.2949=[53,75,16],18.4581=[],18.6418=[53,11,68],18.6c25=[41,53,56],18.70b6=[53,24,27],18.7722=[11,53,21],18.784d=[53,46,16],18.871d=[58,53,65],18.8bab=[53,42,73],18.a5aa=[53,34,24],18.aebe=[53,21,17],18.b6da=[14,53,51],18.b7f2=[53,46,63],18.b974=[44,53,66],18.c5be=[53,46,10],18.c9c2=[53,49,32],18.dc63=[48,53,62],18.fe59=[18,53,66],18.100ff=[53,30,64],18.10f3c=[53,43,11,61],18.11149=[],18.124b8=[59,53,70],18.12ea7=[27,53,40],18.1306e=[33,53,50],18.135c8=[44,53,70],18.138f9=[38,53,60],18.13a33=[44,53,60],18.13be2=[34,53,63],18.13d3b=[51,53,69],18.1476b=[53,34,61],18.14a6b=[53,20,54],18.14c59=[53,31,33,70],18.14c74=[32,53,75],18.15adf=[12,53,72],18.16171=[33,53,42],18.169a9=[29,53,65],18.16d03=[53,60,64],18.16d23=[],18.170ac=[32,53,42],18.17501=[53,48,75],18.17658=[24,53,72],18.1779c=[14,53,48],18.191ae=[10,53,33],18.1ab69=[55,53,66],18.1ad37=[53,45,19],18.1b0bd=[65,53,69],18.1b3dd=[67,53,74],18.1b465=[16,53,40],18.1ba80=[60,53,74],18.1c5cc=[18,53,75],18.1d82a=[53,44,61],18.1db52=[13,53,73],18.1dc90=[29,53,68],18.1e0c6=[12,53,25],18.1e84d=[34,53,52],18.1fc96=[40,53,66],18.1feee=[53,74,65],18.21457=[16,53,72],18.215bb=[54,53,66],18.22176=[53,69,67],18.2232c=[53,44,42],18.24a02=[53,43,50],18.253fa=[26,53,71],18.255ad=[53,68,69],18.257c4=[16,53,63],18.25ee0=[53,44,60],18.25f28=[53,41,66],18.25fc7=[28,53,55,69],18.2605d=[],18.283a8=[30,53,69],18.28dda=[53,64,15],18.29198=[14,53,64],18.29200=[15,53,48],18.292e9=[53,38,45],18.299dc=[53,32,58],18.29cfe=[53,75,14],18.2a282=[53,44,19],18.2a2f6=[53,32,64],18.2a99d=[53,38,44],18.2ab20=[53,32,69],18.2ab58=[53,32,73],18.2c9e2=[53,71,64],18.2cbb0=[53,64,21],18.2d0bb=[53,43,42],18.2e553=[10,53,55],18.2f341=[53,62,61],18.2f9b1=[53,60,10],18.2fcce=[53,67,18],18.307a3=[53,38,44]} v10021) v1 ==== 3257+0+0 (2659842686 0 0) 0x7b6cd20 con 0x7213e00
  671. -1761> 2012-12-18 19:54:56.882283 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.882286 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  672. -1760> 2012-12-18 19:54:56.882338 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3257 (0x8e4928 -> 7970735)
  673. -1759> 2012-12-18 19:54:56.882343 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 5 ==== osd_pgtemp(e10021 {0.b8=[34,59,66],1.b7=[34,59,66],1.81d=[],1.90d=[43,59,75],18.400=[59,69,57],18.667=[59,37,55],18.8fc=[43,59,75],18.12b0=[41,59,69],18.2040=[34,59,75],18.2e0d=[46,59,73],18.2f77=[37,59,68],18.335d=[59,42,17],18.3d9b=[59,47,29],18.41ca=[59,40,43],18.4368=[59,32,53],18.44f7=[59,49,13],18.47d5=[59,66,57],18.4bc1=[59,48,53],18.4ea0=[59,41,74],18.505e=[59,46,21],18.6076=[27,59,62],18.6e1c=[59,30,73],18.701e=[],18.7363=[59,46,73],18.7b73=[59,32,54],18.84c9=[59,45,61],18.908f=[16,59,30],18.90e8=[59,44,25],18.91d9=[59,46,12],18.9f31=[59,48,14],18.b251=[59,38,55],18.b78b=[59,70,24],18.cc0c=[59,48,47],18.d09e=[59,49,70],18.d65b=[59,48,21],18.ed00=[33,59,68],18.f558=[59,15,20],18.f810=[],18.fc60=[59,37,69],18.1005f=[59,47,53],18.10204=[59,63,72],18.10764=[59,69,61],18.1165c=[62,59,72],18.1227e=[13,59,60],18.1233d=[49,59,66],18.12eb9=[59,47,69,27],18.1314e=[15,59,64],18.137aa=[12,59,72],18.13809=[42,59,60],18.13bde=[59,33,57],18.14663=[59,45,60],18.14f19=[59,73,56],18.1592e=[59,30,40],18.15c8d=[41,59,70],18.15dcc=[59,26,14],18.16720=[24,59,74],18.16af6=[],18.1733d=[41,59,72],18.17742=[19,59,75],18.17b95=[],18.17d46=[50,59,61],18.181f5=[59,66,27],18.1878e=[59,47,41,67],18.18913=[33,59,72],18.189f0=[],18.19117=[56,59,69],18.198a6=[25,59,67],18.1996b=[21,59,46],18.1a74a=[59,36,62,25],18.1a759=[],18.1b40b=[12,59,26],18.1b72b=[62,59,73],18.1bc4e=[40,59,60],18.1bd92=[59,41,61],18.1d46a=[34,59,67],18.1d605=[59,69,13],18.1d8f9=[59,67,60],18.1e1f2=[59,46,68],18.1f0b3=[21,59,66],18.20db7=[59,34,52],18.2125d=[59,38,41],18.215f8=[59,42,26],18.21f00=[59,33,44],18.22261=[33,59,69],18.23589=[59,49,12],18.23af7=[59,32,10],18.23fd9=[59,13,64],18.2472b=[59,46,63,11],18.25000=[64,59,74],18.253f8=[59,71,66],18.2646a=[59,67,69],18.26add=[59,32,19],18.26c3d=[59,28,68],18.26cc3=[59,45,26],18.27ab2=[59,17,67],18.27bee=[20,59,44],18.27ca2=[59,40,20],18.281ac=[11,59,75],18.282e0=[59,43,32],18.293a3=[59,48,67],18.29684=[26,59,70],18.2a61e=[59,47,60],18.2aa10=[59,32,15],18.2b939=[59,49,15],18.2bb18=[40,59,63],18.2c5b9=[59,37,64],18.2c82d=[59,41,50],18.2c8ea=[59,68,16],18.2e097=[59,48,63],18.2e563=[59,48,12],18.2e7b3=[59,67,26],18.2ff41=[53,59,72],18.30449=[14,59,41]} v10021) v1 ==== 3687+0+0 (2475911660 0 0) 0x7b6d0e0 con 0x722cdc0
  674. -1758> 2012-12-18 19:54:56.882443 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.882444 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  675. -1757> 2012-12-18 19:54:56.882499 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3687 (0x8e4928 -> 7967048)
  676. -1756> 2012-12-18 19:54:56.885335 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (7988902 -> 7988944)
  677. -1755> 2012-12-18 19:54:56.885353 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (7967048 -> 7967090)
  678. -1754> 2012-12-18 19:54:56.885418 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (7988944 -> 7988966)
  679. -1753> 2012-12-18 19:54:56.885450 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (7967090 -> 7967112)
  680. -1752> 2012-12-18 19:54:56.885462 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 3 ==== mon_subscribe({monmap=2+,osdmap=10019}) v2 ==== 42+0+0 (2142708274 0 0) 0x9cca8c0 con 0x7212dc0
  681. -1751> 2012-12-18 19:54:56.885488 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10019..10022] to osd.56 10.1.0.16:6819/18681
  682. -1750> 2012-12-18 19:54:56.885550 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.56 10.1.0.16:6819/18681 -- osd_map(10019..10022 src has 7326..10022) v3 -- ?+0 0x700aa00
  683. -1749> 2012-12-18 19:54:56.885570 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.56 10.1.0.16:6819/18681 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bee4e0
  684. -1748> 2012-12-18 19:54:56.885612 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7988924)
  685. -1747> 2012-12-18 19:54:56.885616 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 7967070)
  686. -1746> 2012-12-18 19:54:56.885624 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x11c464e0 con 0x7212dc0
  687. -1745> 2012-12-18 19:54:56.885636 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.885636 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  688. -1744> 2012-12-18 19:54:56.885652 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 7967048)
  689. -1743> 2012-12-18 19:54:56.885707 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3935 (7988924 -> 7992859)
  690. -1742> 2012-12-18 19:54:56.885714 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3935 (7967048 -> 7970983)
  691. -1741> 2012-12-18 19:54:56.885862 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 5 ==== osd_pgtemp(e10021 {1.209=[47,56,74],1.47d=[56,38,17],18.116a=[56,38,53],18.2721=[56,69,15],18.278a=[],18.293d=[22,56,70],18.2b31=[56,66,70],18.2ff9=[21,56,49],18.30a4=[56,34,24],18.3319=[],18.33d6=[56,40,66],18.47f2=[12,56,49],18.504a=[56,45,17],18.50a0=[56,72,15],18.56df=[56,16,18],18.6383=[56,40,22],18.6775=[56,40,75],18.6912=[56,69,10],18.6aac=[16,56,65],18.6cb9=[22,56,67],18.72fa=[56,65,19],18.79e2=[56,38,51],18.7f22=[56,75,17],18.84df=[45,56,63],18.8794=[56,48,17],18.879c=[56,65,22],18.8902=[56,29,21],18.8be9=[56,34,71],18.8dd1=[51,56,72],18.900c=[19,56,47],18.9338=[56,74,60],18.9e6b=[51,56,62],18.a297=[56,34,53],18.a622=[],18.b654=[56,41,22],18.b93f=[12,56,10],18.bae9=[41,56,68],18.baff=[56,43,53],18.bc66=[56,24,60],18.bc76=[56,20,27],18.bd8c=[],18.cac9=[44,56,59],18.d1bb=[38,56,67],18.eac9=[56,64,60],18.efde=[56,40,75],18.f01a=[56,48,15],18.f20e=[56,33,29],18.f543=[56,49,71],18.f704=[56,43,69],18.f715=[30,56,67],18.1263d=[26,56,71],18.1270c=[56,40,52],18.127ab=[56,60,11],18.12a89=[56,29,13],18.132c7=[],18.140c3=[56,65,26],18.144c8=[34,56,46],18.14b3f=[20,56,71],18.15110=[21,56,69],18.15204=[37,56,75],18.1571e=[48,56,67],18.15dd1=[41,56,74],18.16364=[56,41,28],18.16bc2=[56,70,26],18.17d8f=[25,56,34],18.189d6=[20,56,60],18.19318=[56,67,60],18.19ed2=[49,56,69],18.1a0cd=[56,21,63],18.1a435=[30,56,49],18.1aa4a=[56,62,64],18.1b189=[],18.1bb21=[38,56,67],18.1bf81=[56,41,64],18.1c57c=[29,56,38],18.1c5d6=[56,47,67],18.1c929=[66,56,73],18.1cccc=[56,35,39,68],18.1cd63=[37,56,60],18.1cdd6=[56,70,64],18.1cea8=[56,27,66],18.1db01=[56,70,52],18.1e1f4=[14,56,54],18.1e8e8=[40,56,68],18.1ec13=[12,56,60],18.1f194=[14,56,67],18.2162e=[56,44,72],18.22069=[56,74,62],18.238d7=[56,30,28],18.23d1c=[56,44,29],18.23e0e=[56,38,14],18.2440c=[16,56,48],18.245a3=[56,44,63],18.24db8=[56,62,54],18.25202=[56,69,10],18.25820=[56,66,12],18.25988=[56,33,21],18.259e6=[56,34,68],18.25c44=[33,56,66],18.26243=[56,74,29],18.26284=[56,38,55],18.2658f=[27,56,38],18.26781=[56,46,17],18.2761d=[],18.286ce=[11,56,68],18.29f5b=[42,56,61],18.2ab47=[56,38,44,72],18.2acdb=[],18.2b4cc=[56,37,42],18.2c422=[56,33,55],18.2c48b=[56,30,74],18.2c888=[56,37,49],18.2c97e=[56,27,15],18.2cdfa=[56,37,20],18.2e09e=[56,43,18],18.2e778=[56,28,67],18.2efe5=[56,45,65],18.2f1d7=[28,56,75],18.2fee0=[56,49,70],18.308a5=[56,40,52],18.30cdd=[56,35,39,70]} v10021) v1 ==== 3935+0+0 (4248973102 0 0) 0x7b6de00 con 0x7212dc0
  692. -1740> 2012-12-18 19:54:56.885979 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.885981 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  693. -1739> 2012-12-18 19:54:56.886057 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3935 (0x8e4928 -> 7967048)
  694. -1738> 2012-12-18 19:54:56.886291 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (7992859 -> 7992901)
  695. -1737> 2012-12-18 19:54:56.886302 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (7967048 -> 7967090)
  696. -1736> 2012-12-18 19:54:56.886335 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (7992901 -> 7992923)
  697. -1735> 2012-12-18 19:54:56.886358 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (7967090 -> 7967112)
  698. -1734> 2012-12-18 19:54:56.886370 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 3 ==== mon_subscribe({monmap=2+,osdmap=9740}) v2 ==== 42+0+0 (2744075302 0 0) 0x9cca700 con 0x722d040
  699. -1733> 2012-12-18 19:54:56.886396 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9740..10022] to osd.54 10.1.0.16:6813/16922
  700. -1732> 2012-12-18 19:54:56.886534 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4258 (7992923 -> 7997181)
  701. -1731> 2012-12-18 19:54:56.886544 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4258 (7967112 -> 7971370)
  702. -1730> 2012-12-18 19:54:56.887058 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.54 10.1.0.16:6813/16922 -- osd_map(9740..9840 src has 7326..10022) v3 -- ?+0 0x9cb9800
  703. -1729> 2012-12-18 19:54:56.887080 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.54 10.1.0.16:6813/16922 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5beeea0
  704. -1728> 2012-12-18 19:54:56.887112 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 7997139)
  705. -1727> 2012-12-18 19:54:56.887120 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 7971328)
  706. -1726> 2012-12-18 19:54:56.887124 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x5bee340 con 0x722d040
  707. -1725> 2012-12-18 19:54:56.887135 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.887136 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  708. -1724> 2012-12-18 19:54:56.887152 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 7971306)
  709. -1723> 2012-12-18 19:54:56.887157 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 5 ==== osd_pgtemp(e10021 {0.96=[52,54,61],0.693=[62,54,68],1.95=[52,54,61],1.4b1=[54,75,27],1.692=[62,54,68],1.acb=[41,54,55],18.84=[52,54,61],18.2e5=[15,54,75],18.4a0=[54,75,27],18.5a8=[54,47,10],18.aba=[41,54,55],18.1de0=[15,54,27],18.2558=[54,71,19],18.3667=[54,43,73],18.391c=[59,54,67],18.472b=[54,75,28],18.4dc1=[54,64,60],18.4eb8=[54,68,58],18.535a=[54,61,18],18.59aa=[54,34,10],18.5d2a=[54,46,28],18.6439=[54,49,60],18.685e=[54,68,73],18.74fa=[54,70,75],18.788b=[54,61,72],18.7e80=[54,44,22],18.827d=[54,61,71],18.8fbf=[54,74,64],18.951d=[19,54,37],18.99a9=[54,37,72],18.a119=[54,16,19],18.a1ae=[14,54,60],18.a1db=[54,74,67],18.a714=[42,71,54,66],18.a726=[54,30,16],18.b4d1=[54,44,72],18.c60d=[54,37,74],18.c757=[54,44,61],18.cca2=[16,54,45],18.d14a=[54,47,59],18.d155=[54,69,65],18.de00=[54,33,56],18.e546=[17,54,53],18.e590=[54,49,40],18.e5af=[54,35,47,61],18.e66b=[54,30,29],18.efd3=[54,48,29],18.fd73=[54,34,75],18.fe91=[54,46,20],18.ffef=[54,37,44,75],18.12526=[54,73,21],18.12756=[55,54,75],18.1277d=[32,54,60],18.1287b=[37,54,72],18.13148=[54,41,64],18.13942=[50,54,62],18.1398e=[54,42,44,69],18.13b29=[64,54,73],18.1422c=[54,47,18],18.14a98=[54,41,70],18.14d75=[30,54,73],18.14fb0=[33,54,58],18.15377=[54,70,60],18.1575b=[58,54,67],18.17023=[65,54,72],18.17509=[30,54,73],18.1760f=[33,54,64],18.176c9=[16,54,20],18.17744=[47,54,67],18.17c26=[],18.182e0=[12,54,66],18.1841c=[54,65,74],18.189f9=[54,34,72],18.18a2b=[44,54,55],18.18a3f=[54,21,25,74],18.19c95=[44,54,75],18.19d58=[53,54,70],18.1a8b1=[17,54,43],18.1abf2=[54,44,66],18.1b30b=[22,54,51],18.1b64e=[54,26,14],18.1c106=[40,54,75],18.1c34c=[34,54,42],18.1c533=[54,61,24],18.1d57c=[54,73,64],18.1d672=[29,54,67],18.1e9ab=[43,54,66],18.1fc03=[19,54,58],18.1fc4f=[41,54,73],18.214de=[54,70,75],18.21dd4=[54,68,71],18.21de8=[54,33,11],18.22b5f=[54,38,74],18.2362b=[54,33,72],18.23f3d=[54,46,72],18.2406c=[54,41,34],18.24f1c=[50,54,62],18.25d11=[54,32,74],18.263e5=[],18.26723=[54,75,71],18.26f2e=[29,54,72],18.27105=[54,42,69],18.283a9=[54,75,67],18.28a7d=[54,24,66],18.28c29=[32,54,41],18.28daa=[54,37,50],18.2952e=[54,46,21],18.2960b=[54,43,61],18.29842=[54,67,17],18.299d9=[54,46,11],18.29de9=[11,54,73],18.2a091=[54,45,60,22],18.2a09e=[32,54,63],18.2ae8c=[54,43,60],18.2ba8a=[54,26,21],18.2bcb6=[34,54,46],18.2bcf0=[54,47,62],18.2c10a=[21,54,58,66],18.2c3ce=[54,42,66],18.2c6e5=[54,43,29],18.2c80c=[54,69,13],18.2ca54=[32,54,53,73],18.2ccbf=[54,49,69],18.2d930=[54,13,74],18.2da0e=[54,34,21],18.2f506=[54,46,62],18.2f867=[54,29,63],18.30cc9=[54,70,60]} v10021) v1 ==== 4258+0+0 (1460722192 0 0) 0x7b72000 con 0x722d040
  710. -1722> 2012-12-18 19:54:56.887275 7f977255a700 1 mon.a@0(leader).paxos(osdmap active c 7326..10022) is_readable now=2012-12-18 19:54:56.887275 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  711. -1721> 2012-12-18 19:54:56.887347 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4258 (0x8e4928 -> 7967048)
  712. -1720> 2012-12-18 19:54:56.891347 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1282068 (7997139 -> 9279207)
  713. -1719> 2012-12-18 19:54:56.891380 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1282068 (7967048 -> 9249116)
  714. -1718> 2012-12-18 19:54:56.892473 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1283800 (9279207 -> 10563007)
  715. -1717> 2012-12-18 19:54:56.892484 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1283800 (9249116 -> 10532916)
  716. -1716> 2012-12-18 19:54:56.898157 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1274388 (10563007 -> 11837395)
  717. -1715> 2012-12-18 19:54:56.898172 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1274388 (10532916 -> 11807304)
  718. -1714> 2012-12-18 19:54:56.898678 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1275484 (11837395 -> 13112879)
  719. -1713> 2012-12-18 19:54:56.898688 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1275484 (11807304 -> 13082788)
  720. -1712> 2012-12-18 19:54:56.899233 7f9771d59700 5 mon.a@0(leader).paxos(logm active c 5962045..5962551) propose_new_value 5962552 2370 bytes
  721. -1711> 2012-12-18 19:54:56.904212 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1273024 (13112879 -> 14385903)
  722. -1710> 2012-12-18 19:54:56.904228 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1273024 (13082788 -> 14355812)
  723. -1709> 2012-12-18 19:54:56.905819 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1295320 (14385903 -> 15681223)
  724. -1708> 2012-12-18 19:54:56.905829 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1295320 (14355812 -> 15651132)
  725. -1707> 2012-12-18 19:54:56.906744 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (15681223 -> 15681381)
  726. -1706> 2012-12-18 19:54:56.906763 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (15651132 -> 15651290)
  727. -1705> 2012-12-18 19:54:56.906948 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (887467545 0 0) 0x7223e00 con 0x9cd1900
  728. -1704> 2012-12-18 19:54:56.910488 7f9774f84700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (15681381 -> 15681539)
  729. -1703> 2012-12-18 19:54:56.910502 7f9774f84700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (15651290 -> 15651448)
  730. -1702> 2012-12-18 19:54:56.919117 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (15681539 -> 15681697)
  731. -1701> 2012-12-18 19:54:56.919158 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (15651448 -> 15651606)
  732. -1700> 2012-12-18 19:54:56.922000 7f9771d59700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(logm begin lc 5962551 fc 0 pn 129200 opn 0) v1 -- ?+0 0x721b400
  733. -1699> 2012-12-18 19:54:56.922056 7f9771d59700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(logm begin lc 5962551 fc 0 pn 129200 opn 0) v1 -- ?+0 0x772a780
  734. -1698> 2012-12-18 19:54:56.922835 7f9771d59700 5 mon.a@0(leader).paxos(osdmap active c 7326..10022) propose_new_value 10023 29615 bytes
  735. -1697> 2012-12-18 19:54:56.928506 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (15681697 -> 15681855)
  736. -1696> 2012-12-18 19:54:56.928524 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (15651606 -> 15651764)
  737. -1695> 2012-12-18 19:54:56.937973 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (15681855 -> 15681877)
  738. -1694> 2012-12-18 19:54:56.937996 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (15651764 -> 15651786)
  739. -1693> 2012-12-18 19:54:56.938018 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 473 (15681877 -> 15682350)
  740. -1692> 2012-12-18 19:54:56.938023 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 473 (15651786 -> 15652259)
  741. -1691> 2012-12-18 19:54:56.938062 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 614792 (15682350 -> 16297142)
  742. -1690> 2012-12-18 19:54:56.938067 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 614792 (15652259 -> 16267051)
  743. -1689> 2012-12-18 19:54:56.942821 7f9771d59700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(osdmap begin lc 10022 fc 0 pn 129200 opn 0) v1 -- ?+0 0x772ac80
  744. -1688> 2012-12-18 19:54:56.942874 7f9771d59700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(osdmap begin lc 10022 fc 0 pn 129200 opn 0) v1 -- ?+0 0x84af680
  745. -1687> 2012-12-18 19:54:56.942975 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.942979 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  746. -1686> 2012-12-18 19:54:56.943174 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6825/3740 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb9a00 con 0x9cd1900
  747. -1685> 2012-12-18 19:54:56.943194 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 16296984)
  748. -1684> 2012-12-18 19:54:56.943201 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 16266893)
  749. -1683> 2012-12-18 19:54:56.943207 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.14:6826/14574 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3685423826 0 0) 0xc394a00 con 0x722d2c0
  750. -1682> 2012-12-18 19:54:56.943218 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.943218 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  751. -1681> 2012-12-18 19:54:56.943368 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6826/14574 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7223e00 con 0x722d2c0
  752. -1680> 2012-12-18 19:54:56.943386 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 16296826)
  753. -1679> 2012-12-18 19:54:56.943391 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 16266735)
  754. -1678> 2012-12-18 19:54:56.943398 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3520798379 0 0) 0x700ae00 con 0x7212140
  755. -1677> 2012-12-18 19:54:56.943409 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.943410 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  756. -1676> 2012-12-18 19:54:56.943553 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6822/19203 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xc394a00 con 0x7212140
  757. -1675> 2012-12-18 19:54:56.943570 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 16296668)
  758. -1674> 2012-12-18 19:54:56.943577 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 16266577)
  759. -1673> 2012-12-18 19:54:56.943619 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (651719958 0 0) 0x9cc6200 con 0x9cd1400
  760. -1672> 2012-12-18 19:54:56.943630 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:56.943631 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  761. -1671> 2012-12-18 19:54:56.943774 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6813/17486 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x700ae00 con 0x9cd1400
  762. -1670> 2012-12-18 19:54:56.943793 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 16296510)
  763. -1669> 2012-12-18 19:54:56.943797 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 16266419)
  764. -1668> 2012-12-18 19:54:56.943800 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 6 ==== pg_stats(1929 pgs tid 580712 v 10021) v1 ==== 614792+0+0 (1586582540 0 0) 0x772dd40 con 0x72123c0
  765. -1667> 2012-12-18 19:54:56.943812 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.943813 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  766. -1666> 2012-12-18 19:54:56.943827 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.943827 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  767. -1665> 2012-12-18 19:54:56.943836 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.24 10.1.0.15:6809/18192
  768. -1664> 2012-12-18 19:54:56.943858 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6809/18192 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cc6200 con 0x72123c0
  769. -1663> 2012-12-18 19:54:56.943872 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 614792 (0x8e4928 -> 15651627)
  770. -1662> 2012-12-18 19:54:56.943875 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 7 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x5ade680 con 0x72123c0
  771. -1661> 2012-12-18 19:54:56.943883 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.943887 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  772. -1660> 2012-12-18 19:54:56.943898 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 15651605)
  773. -1659> 2012-12-18 19:54:56.943901 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 8 ==== osd_pgtemp(e10021 {18.1a50=[24,54,65],18.54ed=[],18.6805=[24,45,13],18.10c9c=[24,57,62],18.119a0=[],18.1a102=[24,47,73],18.20916=[24,55,75],18.25bbb=[24,49,69],18.29271=[24,57,66],18.29421=[24,53,64],18.2a378=[24,52,53],18.2a661=[],18.2a8a4=[24,40,74],18.2b172=[],18.2e53d=[24,33,17]} v10021) v1 ==== 473+0+0 (3083706999 0 0) 0x7b6d680 con 0x72123c0
  774. -1658> 2012-12-18 19:54:56.943924 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.943924 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  775. -1657> 2012-12-18 19:54:56.943936 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 473 (0x8e4928 -> 15651132)
  776. -1656> 2012-12-18 19:54:56.946639 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 6 ==== pg_stats(1752 pgs tid 578067 v 10021) v1 ==== 558232+0+0 (520328891 0 0) 0x772d680 con 0x722db80
  777. -1655> 2012-12-18 19:54:56.946643 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 7069 (16296510 -> 16303579)
  778. -1654> 2012-12-18 19:54:56.946671 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 7069 (15651132 -> 15658201)
  779. -1653> 2012-12-18 19:54:56.946665 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.946667 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  780. -1652> 2012-12-18 19:54:56.946687 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.946689 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  781. -1651> 2012-12-18 19:54:56.946701 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.11 10.1.0.13:6803/7344
  782. -1650> 2012-12-18 19:54:56.946736 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6803/7344 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb8a00 con 0x722db80
  783. -1649> 2012-12-18 19:54:56.946752 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 558232 (0x8e4928 -> 15099969)
  784. -1648> 2012-12-18 19:54:56.946780 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (16303579 -> 16303601)
  785. -1647> 2012-12-18 19:54:56.946791 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (15099969 -> 15099991)
  786. -1646> 2012-12-18 19:54:56.946786 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 7 ==== log(29 entries) v1 ==== 7069+0+0 (2679247488 0 0) 0x7222400 con 0x722db80
  787. -1645> 2012-12-18 19:54:56.946801 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:56.946802 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  788. -1644> 2012-12-18 19:54:56.946935 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 7069 (0x8e4928 -> 15092922)
  789. -1643> 2012-12-18 19:54:56.946946 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 8 ==== osd_alive(want up_thru 10022 have 10022) v1 ==== 22+0+0 (1276594390 0 0) 0x5b26000 con 0x722db80
  790. -1642> 2012-12-18 19:54:56.946958 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.946959 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  791. -1641> 2012-12-18 19:54:56.946973 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 15092900)
  792. -1640> 2012-12-18 19:54:56.953543 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (16303601 -> 16303643)
  793. -1639> 2012-12-18 19:54:56.953564 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (15092900 -> 15092942)
  794. -1638> 2012-12-18 19:54:56.953753 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (16303643 -> 16303665)
  795. -1637> 2012-12-18 19:54:56.953798 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (15092942 -> 15092964)
  796. -1636> 2012-12-18 19:54:56.953761 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 3 ==== mon_subscribe({monmap=2+,osdmap=9970}) v2 ==== 42+0+0 (3222546148 0 0) 0xb662000 con 0x9cd1900
  797. -1635> 2012-12-18 19:54:56.953838 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9970..10022] to osd.48 10.1.0.14:6825/3740
  798. -1634> 2012-12-18 19:54:56.953852 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3929 (16303665 -> 16307594)
  799. -1633> 2012-12-18 19:54:56.953859 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3929 (15092964 -> 15096893)
  800. -1632> 2012-12-18 19:54:56.954312 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.48 10.1.0.14:6825/3740 -- osd_map(9970..10022 src has 7326..10022) v3 -- ?+0 0x9cb8c00
  801. -1631> 2012-12-18 19:54:56.954332 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.48 10.1.0.14:6825/3740 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b6bd40
  802. -1630> 2012-12-18 19:54:56.954342 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 16307552)
  803. -1629> 2012-12-18 19:54:56.954347 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 15096851)
  804. -1628> 2012-12-18 19:54:56.954352 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x19681a0 con 0x9cd1900
  805. -1627> 2012-12-18 19:54:56.954364 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.954365 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  806. -1626> 2012-12-18 19:54:56.954378 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 15096829)
  807. -1625> 2012-12-18 19:54:56.954383 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 6 ==== pg_stats(3463 pgs tid 17773 v 10021) v1 ==== 1102004+0+0 (274093867 0 0) 0x772db00 con 0x9cd1b80
  808. -1624> 2012-12-18 19:54:56.954393 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.954394 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  809. -1623> 2012-12-18 19:54:56.954400 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 289038 (16307552 -> 16596590)
  810. -1622> 2012-12-18 19:54:56.954405 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.954406 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  811. -1621> 2012-12-18 19:54:56.954412 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 289038 (15096829 -> 15385867)
  812. -1620> 2012-12-18 19:54:56.954415 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.61 10.1.0.17:6803/29184
  813. -1619> 2012-12-18 19:54:56.954438 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6803/29184 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9200 con 0x9cd1b80
  814. -1618> 2012-12-18 19:54:56.954452 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1102004 (0x8e4928 -> 14283863)
  815. -1617> 2012-12-18 19:54:56.956116 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (16596590 -> 16596612)
  816. -1616> 2012-12-18 19:54:56.956131 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (14283863 -> 14283885)
  817. -1615> 2012-12-18 19:54:56.956153 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 369 (16596612 -> 16596981)
  818. -1614> 2012-12-18 19:54:56.956162 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 369 (14283885 -> 14284254)
  819. -1613> 2012-12-18 19:54:56.956193 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 625704 (16596981 -> 17222685)
  820. -1612> 2012-12-18 19:54:56.956200 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 625704 (14284254 -> 14909958)
  821. -1611> 2012-12-18 19:54:56.956108 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 5 ==== osd_pgtemp(e10021 {1.4f4=[48,74,26],1.8c3=[63,48,69],18.4e3=[48,74,26],18.8b2=[48,69,63],18.fbb=[48,57,71],18.1032=[48,51,27],18.115d=[],18.14ab=[48,34,65],18.1732=[18,48,57],18.1f1a=[48,75,44],18.208b=[48,56,73,11],18.2560=[59,48,63],18.25a6=[48,55,26],18.25df=[48,18,75],18.3fb1=[],18.46d2=[48,55,67],18.5154=[15,48,75],18.6033=[26,48,45],18.7a8b=[26,48,71],18.7ab7=[48,68,19],18.89f5=[48,58,13],18.8ed0=[48,57,41],18.a3fd=[],18.a770=[],18.aeac=[48,11,72],18.af22=[12,48,69],18.b4d3=[48,50,41],18.b72f=[30,48,68],18.b865=[48,65,62],18.bf52=[48,38,67],18.c399=[48,59,18],18.cbc7=[48,51,10],18.cdb8=[48,32,11],18.d10e=[48,54,22],18.e042=[48,11,73],18.e496=[48,54,18],18.ebb0=[48,57,62],18.f43b=[48,58,16],18.f84e=[48,70,26],18.1039b=[48,51,73],18.1066b=[48,34,69],18.10bf4=[48,57,75],18.113b5=[56,48,71],18.1191c=[16,48,34],18.11ac6=[25,48,62],18.11e96=[20,48,46],18.12af5=[28,48,33],18.13030=[48,70,10],18.13215=[48,27,65],18.133ef=[46,48,67],18.13542=[10,48,17],18.13cb0=[33,48,47],18.13e50=[48,55,62],18.14825=[48,73,19],18.1487c=[24,48,68],18.160dd=[30,48,73],18.16265=[48,50,17],18.164a2=[48,56,43],18.17411=[48,14,71],18.18042=[17,48,50],18.18ef2=[25,48,74],18.18f16=[21,48,34],18.192e9=[25,48,61],18.192ea=[48,59,60],18.19b6f=[67,48,72],18.19d75=[48,52,24],18.1a2a5=[12,48,72],18.1a442=[42,48,74],18.1a94b=[12,48,65],18.1b1e8=[48,56,43],18.1b436=[48,75,68],18.1b969=[18,48,72],18.1ba59=[44,48,60],18.1bf0f=[11,48,73],18.1cdc8=[11,48,34],18.1d7e4=[],18.1e342=[48,57,64],18.1e359=[27,48,62],18.1e6eb=[28,48,71],18.1ea1a=[30,48,69],18.1f8ca=[66,48,71],18.1fa07=[59,48,74],18.1ff01=[48,30,26],18.20c73=[48,68,12],18.210d5=[34,48,75],18.217b9=[],18.21aad=[],18.229fa=[48,68,64],18.22a17=[],18.22ab2=[47,48,64],18.231d8=[48,27,64],18.23684=[],18.23a65=[48,75,25],18.24bf9=[22,48,60],18.24f25=[48,74,44],18.250d0=[48,54,62],18.25f58=[],18.2610f=[48,33,40],18.264c1=[15,48,75],18.269fe=[48,51,16],18.2706c=[48,18,20],18.27938=[48,34,69],18.27b8f=[34,48,70],18.27f34=[18,48,32],18.28249=[53,48,71],18.28ffa=[48,55,14],18.29bec=[48,56,74],18.2a636=[],18.2a811=[48,38,52,68],18.2aaa2=[48,58,71],18.2b2e5=[48,34,22],18.2cd1f=[48,50,65],18.2cd7d=[48,74,24],18.2cf6a=[48,72,55],18.2d805=[22,48,51],18.2e4f3=[],18.2eb01=[45,48,63],18.2efb5=[48,32,14],18.2f023=[48,55,18,61],18.2f445=[],18.2fce2=[48,53,60],18.306e7=[48,32,22],18.30701=[]} v10021) v1 ==== 3929+0+0 (817064615 0 0) 0x7b6d860 con 0x9cd1900
  822. -1610> 2012-12-18 19:54:56.956246 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.956248 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  823. -1609> 2012-12-18 19:54:56.956266 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3929 (0x8e4928 -> 14906029)
  824. -1608> 2012-12-18 19:54:56.956271 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 6 ==== pg_stats(1965 pgs tid 601159 v 10021) v1 ==== 625704+0+0 (1005345982 0 0) 0x772c240 con 0x722d7c0
  825. -1607> 2012-12-18 19:54:56.956283 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.956284 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  826. -1606> 2012-12-18 19:54:56.956294 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.956295 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  827. -1605> 2012-12-18 19:54:56.956303 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.27 10.1.0.15:6818/19812
  828. -1604> 2012-12-18 19:54:56.956328 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6818/19812 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394e00 con 0x722d7c0
  829. -1603> 2012-12-18 19:54:56.956343 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 625704 (0x8e4928 -> 14280325)
  830. -1602> 2012-12-18 19:54:56.956347 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5bee9c0 con 0x722d7c0
  831. -1601> 2012-12-18 19:54:56.956356 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.956357 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  832. -1600> 2012-12-18 19:54:56.956369 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.956370 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  833. -1599> 2012-12-18 19:54:56.956378 7f977255a700 5 mon.a@0(leader).osd e10022 send_latest to osd.27 10.1.0.15:6818/19812 start 10021
  834. -1598> 2012-12-18 19:54:56.956383 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10021..10022] to osd.27 10.1.0.15:6818/19812
  835. -1597> 2012-12-18 19:54:56.956407 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6818/19812 -- osd_map(10021..10022 src has 7326..10022) v3 -- ?+0 0xc394800 con 0x722d7c0
  836. -1596> 2012-12-18 19:54:56.956419 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 22 (0x8e4928 -> 17222663)
  837. -1595> 2012-12-18 19:54:56.956425 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 14280303)
  838. -1594> 2012-12-18 19:54:56.956429 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 8 ==== osd_pgtemp(e10021 {18.4eee=[27,34,72],18.5562=[27,59,60],18.b4f6=[27,49,10],18.ec3a=[],18.11894=[27,43,13],18.1c686=[27,54,62],18.1e5cd=[41,63,32,27],18.27494=[27,49,62],18.28763=[27,53,64],18.2fed8=[],18.3020a=[27,50,63]} v10021) v1 ==== 369+0+0 (3048990923 0 0) 0x7b6dc20 con 0x722d7c0
  839. -1593> 2012-12-18 19:54:56.956453 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.956454 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  840. -1592> 2012-12-18 19:54:56.956471 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 369 (0x8e4928 -> 14279934)
  841. -1591> 2012-12-18 19:54:56.957227 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 6 ==== pg_stats(1800 pgs tid 567086 v 10021) v1 ==== 573524+0+0 (969070575 0 0) 0x772cd80 con 0x9a93400
  842. -1590> 2012-12-18 19:54:56.957247 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.957249 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  843. -1589> 2012-12-18 19:54:56.957260 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (17222663 -> 17222705)
  844. -1588> 2012-12-18 19:54:56.957272 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (14279934 -> 14279976)
  845. -1587> 2012-12-18 19:54:56.957269 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.957270 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  846. -1586> 2012-12-18 19:54:56.957280 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.16 10.1.0.13:6818/10436
  847. -1585> 2012-12-18 19:54:56.957306 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (17222705 -> 17222727)
  848. -1584> 2012-12-18 19:54:56.957305 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6818/10436 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394400 con 0x9a93400
  849. -1583> 2012-12-18 19:54:56.957314 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (14279976 -> 14279998)
  850. -1582> 2012-12-18 19:54:56.957323 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 573524 (0x8e4928 -> 13706474)
  851. -1581> 2012-12-18 19:54:56.957330 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 7 ==== mon_subscribe({monmap=2+,osdmap=9302}) v2 ==== 42+0+0 (3235658956 0 0) 0x6b73dc0 con 0x9a93400
  852. -1580> 2012-12-18 19:54:56.957345 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9302..10022] to osd.16 10.1.0.13:6818/10436
  853. -1579> 2012-12-18 19:54:56.957356 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1029 (17222727 -> 17223756)
  854. -1578> 2012-12-18 19:54:56.957364 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1029 (13706474 -> 13707503)
  855. -1577> 2012-12-18 19:54:56.957422 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 573524 (17223756 -> 17797280)
  856. -1576> 2012-12-18 19:54:56.957429 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 573524 (13707503 -> 14281027)
  857. -1575> 2012-12-18 19:54:56.957501 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (17797280 -> 17797322)
  858. -1574> 2012-12-18 19:54:56.957511 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (14281027 -> 14281069)
  859. -1573> 2012-12-18 19:54:56.957545 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (17797322 -> 17797344)
  860. -1572> 2012-12-18 19:54:56.957555 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (14281069 -> 14281091)
  861. -1571> 2012-12-18 19:54:56.958117 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4461 (17797344 -> 17801805)
  862. -1570> 2012-12-18 19:54:56.958128 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4461 (14281091 -> 14285552)
  863. -1569> 2012-12-18 19:54:56.958172 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (17801805 -> 17801847)
  864. -1568> 2012-12-18 19:54:56.958185 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (14285552 -> 14285594)
  865. -1567> 2012-12-18 19:54:56.958255 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.16 10.1.0.13:6818/10436 -- osd_map(9302..9402 src has 7326..10022) v3 -- ?+0 0xc395000
  866. -1566> 2012-12-18 19:54:56.958275 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (17801847 -> 17801869)
  867. -1565> 2012-12-18 19:54:56.958278 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.16 10.1.0.13:6818/10436 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bef040
  868. -1564> 2012-12-18 19:54:56.958286 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (14285594 -> 14285616)
  869. -1563> 2012-12-18 19:54:56.958288 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 17801827)
  870. -1562> 2012-12-18 19:54:56.958294 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 14285574)
  871. -1561> 2012-12-18 19:54:56.958297 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 8 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5beeb60 con 0x9a93400
  872. -1560> 2012-12-18 19:54:56.958307 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.958308 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  873. -1559> 2012-12-18 19:54:56.958322 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 14285552)
  874. -1558> 2012-12-18 19:54:56.958326 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 3 ==== mon_subscribe({monmap=2+,osdmap=9410}) v2 ==== 42+0+0 (3446696139 0 0) 0x6b8e000 con 0x9cd1400
  875. -1557> 2012-12-18 19:54:56.958340 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9410..10022] to osd.44 10.1.0.14:6813/17486
  876. -1556> 2012-12-18 19:54:56.958406 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3468 (17801827 -> 17805295)
  877. -1555> 2012-12-18 19:54:56.958458 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3468 (14285552 -> 14289020)
  878. -1554> 2012-12-18 19:54:56.958962 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.44 10.1.0.14:6813/17486 -- osd_map(9410..9510 src has 7326..10022) v3 -- ?+0 0x632a400
  879. -1553> 2012-12-18 19:54:56.958984 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.44 10.1.0.14:6813/17486 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19fc4e0
  880. -1552> 2012-12-18 19:54:56.958994 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 17805253)
  881. -1551> 2012-12-18 19:54:56.958998 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 14288978)
  882. -1550> 2012-12-18 19:54:56.959002 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 3 ==== mon_subscribe({monmap=2+,osdmap=9370}) v2 ==== 42+0+0 (3519688718 0 0) 0x8312000 con 0x7212140
  883. -1549> 2012-12-18 19:54:56.959016 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9370..10022] to osd.47 10.1.0.14:6822/19203
  884. -1548> 2012-12-18 19:54:56.959667 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.47 10.1.0.14:6822/19203 -- osd_map(9370..9470 src has 7326..10022) v3 -- ?+0 0x193d400
  885. -1547> 2012-12-18 19:54:56.959685 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.47 10.1.0.14:6822/19203 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5af5d40
  886. -1546> 2012-12-18 19:54:56.959724 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 17805211)
  887. -1545> 2012-12-18 19:54:56.959728 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 14288936)
  888. -1544> 2012-12-18 19:54:56.959732 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 9 ==== osd_pgtemp(e10021 {0.52d=[],0.69e=[],1.29c=[],1.52c=[],1.69d=[],18.e59=[16,41,10],18.ecb=[16,51,55],18.2fda=[],18.51fe=[16,53,10],18.5a48=[16,34,29],18.6151=[],18.7999=[],18.8906=[16,56,65],18.d4ef=[16,51,68],18.df4e=[16,51,10],18.11e33=[],18.12507=[],18.1268c=[16,54,68],18.1314c=[],18.1453b=[16,41,52,68],18.1463c=[16,59,10],18.14741=[16,32,61],18.1c8ab=[55,46,64,16],18.1eb24=[],18.21050=[16,32,68],18.23397=[16,24,31,0],18.2589d=[16,73,38,3],18.25942=[],18.2916d=[16,52,64],18.2925a=[],18.2b4dd=[16,59,67],18.2cc8d=[16,73,11],18.2d3a7=[16,55,60],18.2ec3b=[16,44,10],18.2f650=[16,56,10]} v10021) v1 ==== 1029+0+0 (3585993771 0 0) 0x7b723c0 con 0x9a93400
  889. -1543> 2012-12-18 19:54:56.959772 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.959772 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  890. -1542> 2012-12-18 19:54:56.959799 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1029 (0x8e4928 -> 14287907)
  891. -1541> 2012-12-18 19:54:56.959804 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x5bb1d40 con 0x9cd1400
  892. -1540> 2012-12-18 19:54:56.959815 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.959815 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  893. -1539> 2012-12-18 19:54:56.959827 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 14287885)
  894. -1538> 2012-12-18 19:54:56.959829 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 4 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5bee000 con 0x7212140
  895. -1537> 2012-12-18 19:54:56.959839 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.959839 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  896. -1536> 2012-12-18 19:54:56.959850 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 14287863)
  897. -1535> 2012-12-18 19:54:56.959853 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 5 ==== osd_pgtemp(e10021 {0.9ad=[44,52,29],1.2bd=[27,44,43],1.c53=[44,36,58,71],1.cd8=[48,44,71],18.116=[44,34,63],18.2ac=[44,55,27],18.cc7=[48,44,71],18.1903=[44,52,11],18.1cd3=[44,60,69],18.1da8=[44,57,13],18.26c1=[12,44,10],18.2e67=[42,44,66],18.2f9e=[45,44,58],18.30be=[44,56,24],18.31ab=[44,65,11],18.3279=[44,54,64],18.340b=[44,69,22],18.391d=[27,44,61],18.53df=[44,25,66],18.56a0=[10,44,74],18.5b4c=[44,68,66],18.5d8a=[44,51,70],18.5e08=[44,58,36,21],18.60fd=[44,54,18],18.6945=[44,58,65],18.6c72=[44,56,68],18.6f70=[42,44,70],18.737c=[44,53,41],18.779f=[44,26,12],18.7aab=[24,44,75],18.7ae1=[44,58,68],18.7cc0=[44,56,74],18.84c5=[44,11,19],18.87f0=[25,44,34],18.8912=[44,53,13],18.8dbd=[44,62,43],18.8e00=[44,51,18],18.9c26=[25,44,40],18.9d02=[44,53,55],18.a1de=[34,44,64],18.a723=[44,28,25],18.b085=[44,51,61],18.bd36=[44,62,11],18.c7e3=[21,44,52],18.d07b=[44,71,19],18.daab=[44,60,20],18.dbaf=[25,44,62],18.dbb5=[46,44,63],18.e154=[],18.e93a=[44,58,65],18.ef2c=[33,44,70],18.f62e=[44,70,61],18.fd31=[51,44,65],18.1005c=[44,53,61],18.10303=[44,52,47],18.1034b=[65,44,68],18.11520=[52,44,65],18.1156b=[44,12,69],18.121a6=[44,64,69],18.1231a=[28,44,40],18.1234f=[44,53,27],18.12560=[19,44,21],18.12f1d=[17,44,60],18.1330a=[29,44,32],18.13755=[44,61,75],18.13b00=[47,44,66],18.13be0=[44,56,73],18.152fe=[14,44,58],18.15b75=[],18.1664b=[10,44,74],18.16899=[61,44,74],18.16ee5=[26,44,74],18.17413=[44,65,49],18.1766f=[44,33,66],18.17997=[44,64,37,60],18.17d68=[44,51,24],18.18344=[28,44,59],18.18628=[44,56,40],18.19131=[44,54,62],18.192e6=[20,44,30],18.19be1=[30,44,57],18.19f04=[22,44,34],18.1a722=[63,44,68],18.1b2d7=[44,38,66],18.1be8d=[62,44,74],18.1c265=[19,44,65],18.1c568=[40,44,63],18.1d503=[44,75,60],18.1d5aa=[13,44,24],18.1d84e=[47,44,56],18.1d9d6=[32,44,53],18.1e780=[30,44,73],18.1eab7=[44,60,71],18.1f635=[21,44,75],18.1f8d1=[42,44,57],18.20f90=[29,44,72],18.21762=[44,62,73],18.21915=[44,38,75],18.222f4=[44,50,45],18.232dc=[40,44,66],18.238ac=[44,58,19],18.23a24=[44,30,60],18.23be9=[44,59,66],18.23c8a=[44,55,24],18.242c6=[55,44,63],18.24781=[44,68,19],18.24a78=[18,44,21],18.24b04=[59,44,71],18.25387=[13,44,75],18.2547a=[44,52,75],18.263d0=[44,58,22],18.2732c=[28,44,66],18.274ba=[44,69,24],18.27c09=[44,53,72],18.28917=[44,74,20],18.29509=[44,50,10],18.29da2=[44,63,11],18.2a618=[44,32,69],18.2a6b5=[46,44,60],18.2ae05=[44,69,61],18.2b2c4=[44,51,15],18.2bbf5=[62,44,74],18.2bc8e=[44,57,60],18.2c1cd=[44,51,65],18.2c8ec=[44,55,57,68],18.2d915=[44,57,66],18.2e1f6=[44,38,53],18.2eecb=[44,67,75],18.2f1ad=[],18.2f1e6=[14,44,74],18.2f303=[44,55,15],18.2fe58=[44,32,22],18.309f2=[44,34,25],18.30a3e=[44,33,25],18.30bbd=[12,44,65]} v10021) v1 ==== 4461+0+0 (659776974 0 0) 0x7b6d2c0 con 0x9cd1400
  898. -1534> 2012-12-18 19:54:56.959976 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.959977 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  899. -1533> 2012-12-18 19:54:56.959989 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4461 (0x8e4928 -> 14283402)
  900. -1532> 2012-12-18 19:54:56.959994 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 5 ==== osd_pgtemp(e10021 {0.b59=[13,47,75],1.b58=[13,47,75],1.bbc=[40,47,57],18.b47=[47,58,13],18.1081=[37,47,65],18.1e47=[49,47,58],18.3190=[47,53,73],18.5915=[47,51,25],18.5a4c=[47,32,29],18.5a98=[47,57,13],18.5e3c=[47,74,24],18.617e=[47,57,66],18.61f9=[47,55,71],18.6a23=[47,53,10],18.6c18=[38,47,71],18.6e79=[47,55,73],18.8cfd=[51,47,64],18.9a8c=[47,57,71],18.9b7b=[47,58,19],18.b567=[],18.b56a=[47,56,72],18.b60c=[47,54,28],18.c753=[47,56,50],18.cfff=[47,50,64],18.daec=[47,34,69],18.dc97=[47,58,70],18.e1e6=[44,47,64],18.ed46=[47,57,26],18.f031=[],18.f28e=[],18.f6dd=[47,57,13],18.fdd4=[47,64,15],18.109ab=[21,47,72],18.11700=[47,30,72],18.12356=[26,47,44],18.1246b=[58,47,66],18.127c8=[17,47,65],18.12dc0=[41,47,64],18.13658=[61,47,69],18.13740=[33,47,74],18.139e3=[54,47,63],18.13b1f=[47,74,24],18.13de1=[47,20,68],18.1433a=[47,22,11],18.14c05=[47,57,63],18.14c81=[51,47,70],18.15140=[47,38,26,73],18.1561e=[16,47,27],18.1594a=[17,47,63],18.15a6b=[47,74,63],18.16243=[19,47,71],18.172ef=[38,47,54],18.17964=[27,47,41],18.17ce1=[],18.1991c=[40,47,53],18.1a068=[47,10,66],18.1a4d8=[47,41,11,61],18.1a81e=[54,47,67],18.1ac31=[45,47,68],18.1b00b=[22,47,71],18.1b7e5=[62,47,69],18.1bece=[47,50,75],18.1bf55=[47,38,68],18.1c0bf=[22,47,59],18.1c669=[47,33,50],18.1cd56=[47,34,29],18.1dbdd=[47,33,22],18.1dc40=[44,47,68],18.1e02f=[40,47,60],18.1ee79=[12,47,75],18.1f7de=[17,47,45],18.1fbee=[47,17,42,74],18.20d4f=[47,53,71],18.213b6=[47,57,67],18.21b41=[47,75,22],18.21f19=[47,62,69],18.22800=[47,28,22],18.2366b=[],18.237b8=[15,47,30],18.23847=[47,13,12],18.243f5=[38,47,60],18.25569=[47,38,61],18.257a7=[47,70,74],18.26463=[47,37,70],18.26939=[47,38,63],18.26b52=[47,56,63],18.27985=[47,56,17],18.28c9d=[47,52,46],18.28e23=[47,38,69],18.28f2e=[47,34,15],18.295a7=[47,37,10],18.295c9=[47,52,14],18.29b21=[47,29,14],18.29ec3=[26,47,59],18.2a4ac=[47,70,60],18.2b424=[47,70,46,72],18.2b61d=[47,51,10],18.2bbcf=[47,63,15],18.2c27e=[47,55,26],18.2cf74=[47,58,11],18.2d116=[],18.2d593=[47,69,64],18.2dd74=[47,50,22],18.2e0e0=[47,17,60],18.2eca4=[47,75,61],18.301f7=[47,57,51]} v10021) v1 ==== 3468+0+0 (4193348384 0 0) 0x7b725a0 con 0x7212140
  901. -1531> 2012-12-18 19:54:56.960089 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.960090 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  902. -1530> 2012-12-18 19:54:56.960100 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3468 (0x8e4928 -> 14279934)
  903. -1529> 2012-12-18 19:54:56.960145 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 6 ==== pg_stats(2066 pgs tid 601507 v 10021) v1 ==== 657760+0+0 (3499592165 0 0) 0x69a7440 con 0x9a93a40
  904. -1528> 2012-12-18 19:54:56.960165 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.960166 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  905. -1527> 2012-12-18 19:54:56.960186 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.960187 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  906. -1526> 2012-12-18 19:54:56.960199 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  907. -1525> 2012-12-18 19:54:56.960213 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (17805211 -> 17805253)
  908. -1524> 2012-12-18 19:54:56.960225 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (14279934 -> 14279976)
  909. -1523> 2012-12-18 19:54:56.960225 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9a00 con 0x9a93a40
  910. -1522> 2012-12-18 19:54:56.960241 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 13622216)
  911. -1521> 2012-12-18 19:54:56.960249 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (17805253 -> 17805275)
  912. -1520> 2012-12-18 19:54:56.960269 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (13622216 -> 13622238)
  913. -1519> 2012-12-18 19:54:56.960247 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 7 ==== mon_subscribe({monmap=2+,osdmap=9314}) v2 ==== 42+0+0 (2165263006 0 0) 0x5f65880 con 0x9a93a40
  914. -1518> 2012-12-18 19:54:56.960346 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9314..10022] to osd.29 10.1.0.15:6824/20884
  915. -1517> 2012-12-18 19:54:56.960363 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (17805275 -> 17805755)
  916. -1516> 2012-12-18 19:54:56.960374 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (13622238 -> 13622718)
  917. -1515> 2012-12-18 19:54:56.960401 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (17805755 -> 18463515)
  918. -1514> 2012-12-18 19:54:56.960409 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (13622718 -> 14280478)
  919. -1513> 2012-12-18 19:54:56.961197 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- osd_map(9314..9414 src has 7326..10022) v3 -- ?+0 0x9cb8400
  920. -1512> 2012-12-18 19:54:56.961218 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b2fd40
  921. -1511> 2012-12-18 19:54:56.961231 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 18463473)
  922. -1510> 2012-12-18 19:54:56.961235 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 14280436)
  923. -1509> 2012-12-18 19:54:56.961239 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 8 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19fc000 con 0x9a93a40
  924. -1508> 2012-12-18 19:54:56.961249 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.961250 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  925. -1507> 2012-12-18 19:54:56.961265 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 14280414)
  926. -1506> 2012-12-18 19:54:56.961268 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 9 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x7b73860 con 0x9a93a40
  927. -1505> 2012-12-18 19:54:56.961296 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.961296 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  928. -1504> 2012-12-18 19:54:56.961313 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 14279934)
  929. -1503> 2012-12-18 19:54:56.970710 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1271908 (18463473 -> 19735381)
  930. -1502> 2012-12-18 19:54:56.970739 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1271908 (14279934 -> 15551842)
  931. -1501> 2012-12-18 19:54:56.971345 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1313024 (19735381 -> 21048405)
  932. -1500> 2012-12-18 19:54:56.971355 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1313024 (15551842 -> 16864866)
  933. -1499> 2012-12-18 19:54:56.972394 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 6 ==== pg_stats(1736 pgs tid 571359 v 10021) v1 ==== 553236+0+0 (2147213993 0 0) 0x69a6240 con 0x9cd1180
  934. -1498> 2012-12-18 19:54:56.972409 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4397 (21048405 -> 21052802)
  935. -1497> 2012-12-18 19:54:56.972420 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4397 (16864866 -> 16869263)
  936. -1496> 2012-12-18 19:54:56.972415 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.972416 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  937. -1495> 2012-12-18 19:54:56.972429 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.972430 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  938. -1494> 2012-12-18 19:54:56.972440 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.14 10.1.0.13:6812/8435
  939. -1493> 2012-12-18 19:54:56.972464 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6812/8435 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9c00 con 0x9cd1180
  940. -1492> 2012-12-18 19:54:56.972479 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 553236 (0x8e4928 -> 16316027)
  941. -1491> 2012-12-18 19:54:56.972524 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 7 ==== log(18 entries) v1 ==== 4397+0+0 (2666755496 0 0) 0x7210800 con 0x9cd1180
  942. -1490> 2012-12-18 19:54:56.972544 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:56.972545 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  943. -1489> 2012-12-18 19:54:56.972605 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4397 (0x8e4928 -> 16311630)
  944. -1488> 2012-12-18 19:54:56.973848 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1255292 (21052802 -> 22308094)
  945. -1487> 2012-12-18 19:54:56.973859 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1255292 (16311630 -> 17566922)
  946. -1486> 2012-12-18 19:54:56.978753 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 6 ==== pg_stats(3401 pgs tid 740763 v 10021) v1 ==== 1082484+0+0 (3775070116 0 0) 0x6afcb40 con 0x7212b40
  947. -1485> 2012-12-18 19:54:56.978774 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 288449 (22308094 -> 22596543)
  948. -1484> 2012-12-18 19:54:56.978774 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:56.978775 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  949. -1483> 2012-12-18 19:54:56.978805 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 288449 (17566922 -> 17855371)
  950. -1482> 2012-12-18 19:54:56.978821 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:56.978824 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  951. -1481> 2012-12-18 19:54:56.978841 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.73 10.1.0.18:6809/17050
  952. -1480> 2012-12-18 19:54:56.978866 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6809/17050 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9e00 con 0x7212b40
  953. -1479> 2012-12-18 19:54:56.978880 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1082484 (0x8e4928 -> 16772887)
  954. -1478> 2012-12-18 19:54:56.985927 7f9774f84700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 61 (22596543 -> 22596604)
  955. -1477> 2012-12-18 19:54:56.985945 7f9774f84700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 61 (16772887 -> 16772948)
  956. -1476> 2012-12-18 19:54:56.986018 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.14:6826/14574 3 ==== mon_subscribe({mdsmap=2488+,monmap=2+,osdmap=10022}) v2 ==== 61+0+0 (1329771386 0 0) 0x6016000 con 0x722d2c0
  957. -1475> 2012-12-18 19:54:56.986056 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to mds.0 10.1.0.14:6826/14574
  958. -1474> 2012-12-18 19:54:56.986085 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.14:6826/14574 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x700a000
  959. -1473> 2012-12-18 19:54:56.986108 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.14:6826/14574 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bb1520
  960. -1472> 2012-12-18 19:54:56.986119 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 61 (0x8e4928 -> 22596543)
  961. -1471> 2012-12-18 19:54:56.986123 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 61 (0x8e4928 -> 16772887)
  962. -1470> 2012-12-18 19:54:56.986864 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 7 ==== log(1000 entries) v1 ==== 289038+0+0 (4170076503 0 0) 0x9cb9000 con 0x9cd1b80
  963. -1469> 2012-12-18 19:54:56.986887 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:56.986889 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  964. -1468> 2012-12-18 19:54:56.988940 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 289038 (0x8e4928 -> 16483849)
  965. -1467> 2012-12-18 19:54:57.000065 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 7 ==== log(1000 entries) v1 ==== 288449+0+0 (1761453039 0 0) 0x7223c00 con 0x7212b40
  966. -1466> 2012-12-18 19:54:57.000074 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (22596543 -> 22596585)
  967. -1465> 2012-12-18 19:54:57.000086 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.000087 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  968. -1464> 2012-12-18 19:54:57.000097 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (16483849 -> 16483891)
  969. -1463> 2012-12-18 19:54:57.000155 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 288448 (22596585 -> 22885033)
  970. -1462> 2012-12-18 19:54:57.000168 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 288448 (16483891 -> 16772339)
  971. -1461> 2012-12-18 19:54:57.001840 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 297038 (22885033 -> 23182071)
  972. -1460> 2012-12-18 19:54:57.001850 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 297038 (16772339 -> 17069377)
  973. -1459> 2012-12-18 19:54:57.002207 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 288449 (0x8e4928 -> 16780928)
  974. -1458> 2012-12-18 19:54:57.002218 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 8 ==== mon_subscribe({monmap=2+,osdmap=9418}) v2 ==== 42+0+0 (1963170396 0 0) 0x5fcdc00 con 0x7212b40
  975. -1457> 2012-12-18 19:54:57.002235 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9418..10022] to osd.73 10.1.0.18:6809/17050
  976. -1456> 2012-12-18 19:54:57.002846 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.73 10.1.0.18:6809/17050 -- osd_map(9418..9518 src has 7326..10022) v3 -- ?+0 0x700ae00
  977. -1455> 2012-12-18 19:54:57.002864 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.73 10.1.0.18:6809/17050 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19ff1e0
  978. -1454> 2012-12-18 19:54:57.002893 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 23182029)
  979. -1453> 2012-12-18 19:54:57.002897 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 16780886)
  980. -1454> 2012-12-18 19:54:57.002902 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 9 ==== pg_stats(1929 pgs tid 580713 v 10021) v1 ==== 614792+0+0 (3453896148 0 0) 0x772c6c0 con 0x72123c0
  981. -1453> 2012-12-18 19:54:57.002912 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.002913 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  982. -1452> 2012-12-18 19:54:57.002927 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.002927 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  983. -1451> 2012-12-18 19:54:57.002936 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.24 10.1.0.15:6809/18192
  984. -1450> 2012-12-18 19:54:57.002956 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6809/18192 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x7228000 con 0x72123c0
  985. -1449> 2012-12-18 19:54:57.002971 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 614792 (0x8e4928 -> 16166094)
  986. -1448> 2012-12-18 19:54:57.017774 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (23182029 -> 23182051)
  987. -1447> 2012-12-18 19:54:57.017809 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (16166094 -> 16166116)
  988. -1446> 2012-12-18 19:54:57.017780 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 6 ==== pg_stats(3487 pgs tid 731925 v 10021) v1 ==== 1109416+0+0 (733067148 0 0) 0x6afc480 con 0x722c3c0
  989. -1445> 2012-12-18 19:54:57.017856 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 931 (23182051 -> 23182982)
  990. -1444> 2012-12-18 19:54:57.017849 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.017852 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  991. -1443> 2012-12-18 19:54:57.017867 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 931 (16166116 -> 16167047)
  992. -1443> 2012-12-18 19:54:57.017870 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.017871 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  993. -1443> 2012-12-18 19:54:57.017884 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.72 10.1.0.18:6806/16967
  994. -1442> 2012-12-18 19:54:57.017916 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6806/16967 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x7004a00 con 0x722c3c0
  995. -1441> 2012-12-18 19:54:57.017924 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1109416 (23182982 -> 24292398)
  996. -1440> 2012-12-18 19:54:57.017930 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1109416 (16167047 -> 17276463)
  997. -1440> 2012-12-18 19:54:57.017949 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1109416 (0x8e4928 -> 16167047)
  998. -1439> 2012-12-18 19:54:57.017958 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5b6b6c0 con 0x722c3c0
  999. -1439> 2012-12-18 19:54:57.017969 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.017970 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1000. -1439> 2012-12-18 19:54:57.017984 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 16167025)
  1001. -1438> 2012-12-18 19:54:57.017990 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 8 ==== osd_pgtemp(e10021 {18.13e3=[72,37,22],18.181c=[72,57,22],18.3af3=[72,37,29],18.4325=[72,49,60],18.4aba=[72,57,67],18.6ed2=[72,46,25],18.7042=[],18.7683=[72,59,24],18.8737=[72,50,28],18.9599=[72,50,66],18.9c4f=[72,58,44],18.e7b1=[72,51,63],18.ed3e=[72,54,65],18.1058c=[72,52,12],18.1226e=[],18.143b1=[72,56,61],18.1bada=[72,31,53,3],18.1e19a=[],18.1e623=[72,57,62],18.1e8f3=[72,46,11],18.23643=[72,53,44],18.249d9=[72,47,56],18.259b8=[72,33,66],18.2a29a=[],18.2b05a=[72,51,35,29],18.2d1e9=[],18.2d83f=[72,44,65],18.2e322=[72,40,24],18.2f47f=[72,32,33]} v10021) v1 ==== 931+0+0 (705663956 0 0) 0x7b73c20 con 0x722c3c0
  1002. -1437> 2012-12-18 19:54:57.018025 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.018083 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1003. -1436> 2012-12-18 19:54:57.018097 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 931 (0x8e4928 -> 16166094)
  1004. -1435> 2012-12-18 19:54:57.026004 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 288805 (24292398 -> 24581203)
  1005. -1434> 2012-12-18 19:54:57.026010 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 6 ==== pg_stats(4072 pgs tid 745164 v 10021) v1 ==== 1295320+0+0 (3555513503 0 0) 0x6afc900 con 0x722d040
  1006. -1433> 2012-12-18 19:54:57.026022 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 288805 (16166094 -> 16454899)
  1007. -1432> 2012-12-18 19:54:57.026093 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.026098 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1008. -1431> 2012-12-18 19:54:57.026114 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.026114 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1009. -1430> 2012-12-18 19:54:57.026128 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.54 10.1.0.16:6813/16922
  1010. -1429> 2012-12-18 19:54:57.026152 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6813/16922 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xb7b6000 con 0x722d040
  1011. -1428> 2012-12-18 19:54:57.026166 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1295320 (0x8e4928 -> 15159579)
  1012. -1427> 2012-12-18 19:54:57.030772 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 10 ==== log(1000 entries) v1 ==== 297038+0+0 (1575413925 0 0) 0x700a200 con 0x72123c0
  1013. -1426> 2012-12-18 19:54:57.030788 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (24581203 -> 24581235)
  1014. -1425> 2012-12-18 19:54:57.030791 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.030793 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1015. -1424> 2012-12-18 19:54:57.030805 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (15159579 -> 15159611)
  1016. -1423> 2012-12-18 19:54:57.032855 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 297038 (0x8e4928 -> 14862573)
  1017. -1422> 2012-12-18 19:54:57.032864 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 11 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0xc395800 con 0x72123c0
  1018. -1421> 2012-12-18 19:54:57.032875 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.032875 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1019. -1420> 2012-12-18 19:54:57.032928 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6809/18192 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x632b400 con 0x72123c0
  1020. -1419> 2012-12-18 19:54:57.032945 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 24581203)
  1021. -1418> 2012-12-18 19:54:57.032950 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 14862541)
  1022. -1417> 2012-12-18 19:54:57.038866 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 9 ==== pg_stats(1965 pgs tid 601160 v 10021) v1 ==== 625704+0+0 (251715021 0 0) 0x5f32900 con 0x722d7c0
  1023. -1416> 2012-12-18 19:54:57.038870 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1980 (24581203 -> 24583183)
  1024. -1415> 2012-12-18 19:54:57.038891 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1980 (14862541 -> 14864521)
  1025. -1414> 2012-12-18 19:54:57.038887 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.038888 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1026. -1413> 2012-12-18 19:54:57.038918 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.038920 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1027. -1412> 2012-12-18 19:54:57.038936 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.27 10.1.0.15:6818/19812
  1028. -1411> 2012-12-18 19:54:57.038963 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6818/19812 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc395800 con 0x722d7c0
  1029. -1410> 2012-12-18 19:54:57.038979 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 625704 (0x8e4928 -> 14238817)
  1030. -1409> 2012-12-18 19:54:57.038984 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 10 ==== log(8 entries) v1 ==== 1980+0+0 (3457941815 0 0) 0x9cb8e00 con 0x722d7c0
  1031. -1408> 2012-12-18 19:54:57.038994 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.038995 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1032. -1407> 2012-12-18 19:54:57.039027 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1980 (0x8e4928 -> 14236837)
  1033. -1406> 2012-12-18 19:54:57.049986 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 6 ==== pg_stats(4035 pgs tid 21629 v 10021) v1 ==== 1283800+0+0 (47606247 0 0) 0x5f32b40 con 0x7003e00
  1034. -1405> 2012-12-18 19:54:57.050007 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.050009 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1035. -1404> 2012-12-18 19:54:57.050004 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (24583183 -> 24583225)
  1036. -1403> 2012-12-18 19:54:57.050136 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (14236837 -> 14236879)
  1037. -1402> 2012-12-18 19:54:57.050134 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.050136 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1038. -1401> 2012-12-18 19:54:57.050148 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.55 10.1.0.16:6816/13746
  1039. -1400> 2012-12-18 19:54:57.050179 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6816/13746 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x632b200 con 0x7003e00
  1040. -1399> 2012-12-18 19:54:57.050188 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (24583225 -> 24583247)
  1041. -1398> 2012-12-18 19:54:57.050208 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1283800 (0x8e4928 -> 12953079)
  1042. -1397> 2012-12-18 19:54:57.050230 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (12953079 -> 12953101)
  1043. -1396> 2012-12-18 19:54:57.050219 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 7 ==== mon_subscribe({monmap=2+,osdmap=9340}) v2 ==== 42+0+0 (2133406596 0 0) 0x74aec40 con 0x7003e00
  1044. -1395> 2012-12-18 19:54:57.050268 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9340..10022] to osd.55 10.1.0.16:6816/13746
  1045. -1394> 2012-12-18 19:54:57.050293 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3892 (24583247 -> 24587139)
  1046. -1393> 2012-12-18 19:54:57.050304 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3892 (12953101 -> 12956993)
  1047. -1392> 2012-12-18 19:54:57.051217 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.55 10.1.0.16:6816/13746 -- osd_map(9340..9440 src has 7326..10022) v3 -- ?+0 0x632ba00
  1048. -1391> 2012-12-18 19:54:57.051273 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.55 10.1.0.16:6816/13746 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19ff6c0
  1049. -1390> 2012-12-18 19:54:57.051312 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 24587097)
  1050. -1389> 2012-12-18 19:54:57.051326 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 12956951)
  1051. -1388> 2012-12-18 19:54:57.051332 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 8 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x5b6ba00 con 0x7003e00
  1052. -1387> 2012-12-18 19:54:57.051345 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.051346 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1053. -1386> 2012-12-18 19:54:57.051373 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 12956929)
  1054. -1385> 2012-12-18 19:54:57.051380 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 9 ==== osd_pgtemp(e10021 {0.777=[27,55,61],1.174=[22,55,60],1.197=[11,55,75],1.776=[27,55,61],18.a7c=[55,48,74],18.1325=[30,55,58],18.180a=[55,48,29],18.1839=[55,46,11],18.18de=[17,55,71],18.20d4=[10,55,27],18.2a2d=[],18.2f59=[55,49,65],18.38e8=[55,41,61],18.3b8e=[50,55,61],18.5206=[55,33,64],18.54be=[25,55,69],18.6354=[38,55,75],18.68f0=[55,36,31],18.6de1=[55,48,58],18.6fe7=[55,71,63],18.73c8=[53,55,73],18.793e=[66,55,75],18.7d57=[58,55,71],18.8231=[55,46,65],18.847c=[],18.9e5c=[55,45,20],18.9f7f=[62,55,72],18.aa7b=[42,55,57],18.ac39=[32,55,68],18.ad58=[55,45,13],18.af76=[22,55,33],18.b7cf=[],18.c0de=[18,55,34],18.c641=[55,67,17],18.c657=[55,47,14],18.c8cb=[55,73,28],18.c979=[55,43,60],18.d5b0=[55,20,14],18.d7bb=[55,49,46],18.e325=[12,55,42],18.feec=[55,37,15],18.107d6=[55,67,22],18.10d5f=[55,71,74],18.114ff=[55,28,66],18.1266c=[59,55,68],18.1282c=[55,46,51],18.12967=[55,72,16],18.129d0=[55,43,40],18.12f92=[28,55,61],18.13173=[43,55,59],18.13d15=[10,55,13],18.13e17=[55,74,62],18.142d7=[28,55,66],18.14308=[55,43,27],18.1431f=[43,55,60],18.14b73=[55,43,10],18.15192=[55,71,72],18.1636d=[55,42,66,24],18.16e69=[19,55,73],18.17e10=[],18.17e63=[48,55,66],18.18006=[32,55,69],18.1861a=[13,55,60],18.18633=[],18.195c6=[55,69,32],18.1977a=[29,55,52],18.19b2e=[55,27,64],18.1aa1a=[44,55,58],18.1ab80=[55,69,18],18.1aecb=[29,55,72],18.1af59=[26,55,53],18.1b920=[55,60,21],18.1bb5d=[55,46,69],18.1c59f=[49,55,56],18.1c699=[],18.1cab5=[55,64,13],18.1d226=[10,55,29],18.1da8f=[],18.1ef5a=[],18.1f970=[47,55,67],18.1f9ec=[66,55,74],18.2011a=[55,37,71],18.20ea7=[14,55,59],18.20f5a=[55,21,70],18.21a22=[42,65,55,63],18.21e6a=[],18.21ee6=[55,75,24],18.22a10=[55,42,60],18.22af0=[55,47,61],18.23c5a=[55,38,70],18.24160=[],18.24bcd=[55,60,10],18.2500d=[55,20,15],18.25dec=[55,45,19],18.26152=[55,72,69],18.269c9=[55,32,74],18.26e87=[55,63,16],18.28754=[55,32,65],18.28beb=[52,55,72],18.29042=[20,55,48],18.294d1=[55,60,54],18.299fa=[55,49,25],18.29ab2=[55,44,17],18.2a11a=[55,60,27],18.2a19d=[],18.2a456=[25,55,62],18.2a6be=[41,55,67],18.2c3eb=[55,37,69],18.2cc5a=[55,61,20],18.2cd13=[55,32,26],18.2d809=[55,75,56],18.2d920=[55,75,67],18.2dc18=[],18.2ddac=[55,40,68],18.2e2b1=[55,68,11],18.2eaa7=[53,55,74],18.2f4a8=[55,47,18],18.2f4f3=[55,28,67],18.2ff43=[55,49,25],18.3045c=[],18.30aa5=[],18.30beb=[55,70,62]} v10021) v1 ==== 3892+0+0 (2148732139 0 0) 0x7b73e00 con 0x7003e00
  1055. -1384> 2012-12-18 19:54:57.051507 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.051509 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1056. -1383> 2012-12-18 19:54:57.051524 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3892 (0x8e4928 -> 12953037)
  1057. -1382> 2012-12-18 19:54:57.052193 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1283800 (24587097 -> 25870897)
  1058. -1381> 2012-12-18 19:54:57.052206 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1283800 (12953037 -> 14236837)
  1059. -1380> 2012-12-18 19:54:57.052463 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3682 (25870897 -> 25874579)
  1060. -1379> 2012-12-18 19:54:57.052452 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 10 ==== pg_stats(1800 pgs tid 567087 v 10021) v1 ==== 573524+0+0 (3853531656 0 0) 0x5f32fc0 con 0x9a93400
  1061. -1378> 2012-12-18 19:54:57.052486 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3682 (14236837 -> 14240519)
  1062. -1377> 2012-12-18 19:54:57.052484 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.052486 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1063. -1376> 2012-12-18 19:54:57.052507 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.052508 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1064. -1375> 2012-12-18 19:54:57.052520 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.16 10.1.0.13:6818/10436
  1065. -1374> 2012-12-18 19:54:57.052549 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6818/10436 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x6092400 con 0x9a93400
  1066. -1373> 2012-12-18 19:54:57.052569 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 573524 (0x8e4928 -> 13666995)
  1067. -1372> 2012-12-18 19:54:57.052605 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 11 ==== log(15 entries) v1 ==== 3682+0+0 (2569575521 0 0) 0x9cb8800 con 0x9a93400
  1068. -1371> 2012-12-18 19:54:57.052665 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.052666 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1069. -1370> 2012-12-18 19:54:57.052724 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3682 (0x8e4928 -> 13663313)
  1070. -1369> 2012-12-18 19:54:57.054497 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (25874579 -> 25874601)
  1071. -1368> 2012-12-18 19:54:57.054543 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (13663313 -> 13663335)
  1072. -1367> 2012-12-18 19:54:57.054571 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (25874601 -> 25875081)
  1073. -1366> 2012-12-18 19:54:57.054560 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 10 ==== pg_stats(2066 pgs tid 601508 v 10021) v1 ==== 657760+0+0 (1203920538 0 0) 0x5f32000 con 0x9a93a40
  1074. -1365> 2012-12-18 19:54:57.054581 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (13663335 -> 13663815)
  1075. -1364> 2012-12-18 19:54:57.054588 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.054591 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1076. -1363> 2012-12-18 19:54:57.054620 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (25875081 -> 26532841)
  1077. -1362> 2012-12-18 19:54:57.054645 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (13663815 -> 14321575)
  1078. -1361> 2012-12-18 19:54:57.054619 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.054622 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1079. -1360> 2012-12-18 19:54:57.054664 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  1080. -1359> 2012-12-18 19:54:57.054695 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x6092200 con 0x9a93a40
  1081. -1358> 2012-12-18 19:54:57.054710 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 13663815)
  1082. -1357> 2012-12-18 19:54:57.054715 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 11 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19fc1a0 con 0x9a93a40
  1083. -1356> 2012-12-18 19:54:57.054726 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.054726 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1084. -1355> 2012-12-18 19:54:57.054741 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 13663793)
  1085. -1354> 2012-12-18 19:54:57.054745 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 12 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x7b734a0 con 0x9a93a40
  1086. -1353> 2012-12-18 19:54:57.054770 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.054770 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1087. -1352> 2012-12-18 19:54:57.054788 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 13663313)
  1088. -1351> 2012-12-18 19:54:57.055851 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (26532841 -> 26532863)
  1089. -1350> 2012-12-18 19:54:57.055863 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (13663313 -> 13663335)
  1090. -1349> 2012-12-18 19:54:57.055851 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 6 ==== pg_stats(4010 pgs tid 713588 v 10021) v1 ==== 1275484+0+0 (3567883784 0 0) 0x5f32480 con 0x722cdc0
  1091. -1348> 2012-12-18 19:54:57.055881 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3687 (26532863 -> 26536550)
  1092. -1347> 2012-12-18 19:54:57.055888 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3687 (13663335 -> 13667022)
  1093. -1346> 2012-12-18 19:54:57.055876 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.055878 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1094. -1345> 2012-12-18 19:54:57.055898 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.055900 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1095. -1344> 2012-12-18 19:54:57.055911 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.59 10.1.0.16:6828/20636
  1096. -1343> 2012-12-18 19:54:57.055941 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6828/20636 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193ce00 con 0x722cdc0
  1097. -1342> 2012-12-18 19:54:57.055962 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1275484 (0x8e4928 -> 12391538)
  1098. -1341> 2012-12-18 19:54:57.055969 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1275484 (26536550 -> 27812034)
  1099. -1340> 2012-12-18 19:54:57.055969 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5beeea0 con 0x722cdc0
  1100. -1339> 2012-12-18 19:54:57.055974 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1275484 (12391538 -> 13667022)
  1101. -1338> 2012-12-18 19:54:57.055989 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.055990 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1102. -1337> 2012-12-18 19:54:57.056001 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 13667000)
  1103. -1336> 2012-12-18 19:54:57.056005 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 8 ==== osd_pgtemp(e10021 {0.b8=[34,59,66],1.b7=[34,59,66],1.81d=[],1.90d=[43,59,75],18.400=[59,69,57],18.667=[59,37,55],18.8fc=[43,59,75],18.12b0=[41,59,69],18.2040=[34,59,75],18.2e0d=[46,59,73],18.2f77=[37,59,68],18.335d=[59,42,17],18.3d9b=[59,47,29],18.41ca=[59,40,43],18.4368=[59,32,53],18.44f7=[59,49,13],18.47d5=[59,66,57],18.4bc1=[59,48,53],18.4ea0=[59,41,74],18.505e=[59,46,21],18.6076=[27,59,62],18.6e1c=[59,30,73],18.701e=[],18.7363=[59,46,73],18.7b73=[59,32,54],18.84c9=[59,45,61],18.908f=[16,59,30],18.90e8=[59,44,25],18.91d9=[59,46,12],18.9f31=[59,48,14],18.b251=[59,38,55],18.b78b=[59,70,24],18.cc0c=[59,48,47],18.d09e=[59,49,70],18.d65b=[59,48,21],18.ed00=[33,59,68],18.f558=[59,15,20],18.f810=[],18.fc60=[59,37,69],18.1005f=[59,47,53],18.10204=[59,63,72],18.10764=[59,69,61],18.1165c=[62,59,72],18.1227e=[13,59,60],18.1233d=[49,59,66],18.12eb9=[59,47,69,27],18.1314e=[15,59,64],18.137aa=[12,59,72],18.13809=[42,59,60],18.13bde=[59,33,57],18.14663=[59,45,60],18.14f19=[59,73,56],18.1592e=[59,30,40],18.15c8d=[41,59,70],18.15dcc=[59,26,14],18.16720=[24,59,74],18.16af6=[],18.1733d=[41,59,72],18.17742=[19,59,75],18.17b95=[],18.17d46=[50,59,61],18.181f5=[59,66,27],18.1878e=[59,47,41,67],18.18913=[33,59,72],18.189f0=[],18.19117=[56,59,69],18.198a6=[25,59,67],18.1996b=[21,59,46],18.1a74a=[59,36,62,25],18.1a759=[],18.1b40b=[12,59,26],18.1b72b=[62,59,73],18.1bc4e=[40,59,60],18.1bd92=[59,41,61],18.1d46a=[34,59,67],18.1d605=[59,69,13],18.1d8f9=[59,67,60],18.1e1f2=[59,46,68],18.1f0b3=[21,59,66],18.20db7=[59,34,52],18.2125d=[59,38,41],18.215f8=[59,42,26],18.21f00=[59,33,44],18.22261=[33,59,69],18.23589=[59,49,12],18.23af7=[59,32,10],18.23fd9=[59,13,64],18.2472b=[59,46,63,11],18.25000=[64,59,74],18.253f8=[59,71,66],18.2646a=[59,67,69],18.26add=[59,32,19],18.26c3d=[59,28,68],18.26cc3=[59,45,26],18.27ab2=[59,17,67],18.27bee=[20,59,44],18.27ca2=[59,40,20],18.281ac=[11,59,75],18.282e0=[59,43,32],18.293a3=[59,48,67],18.29684=[26,59,70],18.2a61e=[59,47,60],18.2aa10=[59,32,15],18.2b939=[59,49,15],18.2bb18=[40,59,63],18.2c5b9=[59,37,64],18.2c82d=[59,41,50],18.2c8ea=[59,68,16],18.2e097=[59,48,63],18.2e563=[59,48,12],18.2e7b3=[59,67,26],18.2ff41=[53,59,72],18.30449=[14,59,41]} v10021) v1 ==== 3687+0+0 (2475911660 0 0) 0x89921e0 con 0x722cdc0
  1104. -1335> 2012-12-18 19:54:57.056109 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.056110 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1105. -1334> 2012-12-18 19:54:57.056122 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3687 (0x8e4928 -> 13663313)
  1106. -1333> 2012-12-18 19:54:57.057456 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 7 ==== log(1000 entries) v1 ==== 288805+0+0 (1080566667 0 0) 0x7004400 con 0x722d040
  1107. -1332> 2012-12-18 19:54:57.057476 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.057478 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1108. -1331> 2012-12-18 19:54:57.059535 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 288805 (0x8e4928 -> 13374508)
  1109. -1330> 2012-12-18 19:54:57.063289 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 9 ==== log(1000 entries) v1 ==== 288448+0+0 (1159026588 0 0) 0x632ba00 con 0x7212b40
  1110. -1329> 2012-12-18 19:54:57.063314 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.063315 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1111. -1328> 2012-12-18 19:54:57.065359 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 288448 (0x8e4928 -> 13086060)
  1112. -1327> 2012-12-18 19:54:57.069594 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 6 ==== pg_stats(4031 pgs tid 792327 v 10021) v1 ==== 1282068+0+0 (655841184 0 0) 0x5f32240 con 0x7213e00
  1113. -1326> 2012-12-18 19:54:57.069619 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.069620 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1114. -1325> 2012-12-18 19:54:57.069617 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (27812034 -> 27812056)
  1115. -1324> 2012-12-18 19:54:57.069642 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.069643 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1116. -1323> 2012-12-18 19:54:57.069653 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.53 10.1.0.16:6810/16733
  1117. -1322> 2012-12-18 19:54:57.069654 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (13086060 -> 13086082)
  1118. -1321> 2012-12-18 19:54:57.069687 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6810/16733 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193cc00 con 0x7213e00
  1119. -1320> 2012-12-18 19:54:57.069704 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1282068 (0x8e4928 -> 11804014)
  1120. -1319> 2012-12-18 19:54:57.069702 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3257 (27812056 -> 27815313)
  1121. -1318> 2012-12-18 19:54:57.069716 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5bee4e0 con 0x7213e00
  1122. -1317> 2012-12-18 19:54:57.069740 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3257 (11804014 -> 11807271)
  1123. -1316> 2012-12-18 19:54:57.069752 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.069754 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1124. -1315> 2012-12-18 19:54:57.069771 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 11807249)
  1125. -1314> 2012-12-18 19:54:57.069926 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1282068 (27815313 -> 29097381)
  1126. -1313> 2012-12-18 19:54:57.069937 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1282068 (11807249 -> 13089317)
  1127. -1312> 2012-12-18 19:54:57.069989 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 290870 (29097381 -> 29388251)
  1128. -1311> 2012-12-18 19:54:57.070024 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 290870 (13089317 -> 13380187)
  1129. -1310> 2012-12-18 19:54:57.069925 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 8 ==== osd_pgtemp(e10021 {0.457=[40,53,69],0.7bc=[10,53,20],1.456=[40,53,69],1.7bb=[10,53,20],18.445=[53,40,69],18.7aa=[53,10,20],18.1967=[53,41,28],18.1dbb=[53,34,60],18.1efd=[53,42,73],18.22f8=[53,73,61],18.2949=[53,75,16],18.4581=[],18.6418=[53,11,68],18.6c25=[41,53,56],18.70b6=[53,24,27],18.7722=[11,53,21],18.784d=[53,46,16],18.871d=[58,53,65],18.8bab=[53,42,73],18.a5aa=[53,34,24],18.aebe=[53,21,17],18.b6da=[14,53,51],18.b7f2=[53,46,63],18.b974=[44,53,66],18.c5be=[53,46,10],18.c9c2=[53,49,32],18.dc63=[48,53,62],18.fe59=[18,53,66],18.100ff=[53,30,64],18.10f3c=[53,43,11,61],18.11149=[],18.124b8=[59,53,70],18.12ea7=[27,53,40],18.1306e=[33,53,50],18.135c8=[44,53,70],18.138f9=[38,53,60],18.13a33=[44,53,60],18.13be2=[34,53,63],18.13d3b=[51,53,69],18.1476b=[53,34,61],18.14a6b=[53,20,54],18.14c59=[53,31,33,70],18.14c74=[32,53,75],18.15adf=[12,53,72],18.16171=[33,53,42],18.169a9=[29,53,65],18.16d03=[53,60,64],18.16d23=[],18.170ac=[32,53,42],18.17501=[53,48,75],18.17658=[24,53,72],18.1779c=[14,53,48],18.191ae=[10,53,33],18.1ab69=[55,53,66],18.1ad37=[53,45,19],18.1b0bd=[65,53,69],18.1b3dd=[67,53,74],18.1b465=[16,53,40],18.1ba80=[60,53,74],18.1c5cc=[18,53,75],18.1d82a=[53,44,61],18.1db52=[13,53,73],18.1dc90=[29,53,68],18.1e0c6=[12,53,25],18.1e84d=[34,53,52],18.1fc96=[40,53,66],18.1feee=[53,74,65],18.21457=[16,53,72],18.215bb=[54,53,66],18.22176=[53,69,67],18.2232c=[53,44,42],18.24a02=[53,43,50],18.253fa=[26,53,71],18.255ad=[53,68,69],18.257c4=[16,53,63],18.25ee0=[53,44,60],18.25f28=[53,41,66],18.25fc7=[28,53,55,69],18.2605d=[],18.283a8=[30,53,69],18.28dda=[53,64,15],18.29198=[14,53,64],18.29200=[15,53,48],18.292e9=[53,38,45],18.299dc=[53,32,58],18.29cfe=[53,75,14],18.2a282=[53,44,19],18.2a2f6=[53,32,64],18.2a99d=[53,38,44],18.2ab20=[53,32,69],18.2ab58=[53,32,73],18.2c9e2=[53,71,64],18.2cbb0=[53,64,21],18.2d0bb=[53,43,42],18.2e553=[10,53,55],18.2f341=[53,62,61],18.2f9b1=[53,60,10],18.2fcce=[53,67,18],18.307a3=[53,38,44]} v10021) v1 ==== 3257+0+0 (2659842686 0 0) 0x7b73a40 con 0x7213e00
  1130. -1309> 2012-12-18 19:54:57.070103 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.070106 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1131. -1308> 2012-12-18 19:54:57.070129 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3257 (0x8e4928 -> 13376930)
  1132. -1307> 2012-12-18 19:54:57.070141 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 6 ==== pg_stats(3998 pgs tid 717012 v 10021) v1 ==== 1271908+0+0 (3515263070 0 0) 0x6afd680 con 0x7212140
  1133. -1306> 2012-12-18 19:54:57.070155 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.070156 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1134. -1305> 2012-12-18 19:54:57.070169 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.070170 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1135. -1304> 2012-12-18 19:54:57.070184 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.47 10.1.0.14:6822/19203
  1136. -1303> 2012-12-18 19:54:57.070207 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6822/19203 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193d800 con 0x7212140
  1137. -1302> 2012-12-18 19:54:57.070220 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1271908 (0x8e4928 -> 12105022)
  1138. -1301> 2012-12-18 19:54:57.079636 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 300192 (29388251 -> 29688443)
  1139. -1300> 2012-12-18 19:54:57.079656 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 300192 (12105022 -> 12405214)
  1140. -1299> 2012-12-18 19:54:57.079648 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 6 ==== pg_stats(4127 pgs tid 260782 v 10021) v1 ==== 1313024+0+0 (1299898749 0 0) 0x6afd8c0 con 0x9cd1900
  1141. -1298> 2012-12-18 19:54:57.079680 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.079682 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1142. -1297> 2012-12-18 19:54:57.079702 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.079703 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1143. -1296> 2012-12-18 19:54:57.079717 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.48 10.1.0.14:6825/3740
  1144. -1295> 2012-12-18 19:54:57.079748 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6825/3740 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193d600 con 0x9cd1900
  1145. -1294> 2012-12-18 19:54:57.079764 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1313024 (0x8e4928 -> 11092190)
  1146. -1293> 2012-12-18 19:54:57.081154 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 6 ==== pg_stats(3947 pgs tid 702266 v 10021) v1 ==== 1255292+0+0 (3478548762 0 0) 0x69a6480 con 0x9cd1400
  1147. -1292> 2012-12-18 19:54:57.081142 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 295959 (29688443 -> 29984402)
  1148. -1291> 2012-12-18 19:54:57.081184 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 295959 (11092190 -> 11388149)
  1149. -1290> 2012-12-18 19:54:57.081180 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.081182 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1150. -1289> 2012-12-18 19:54:57.081202 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.081203 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1151. -1288> 2012-12-18 19:54:57.081215 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.44 10.1.0.14:6813/17486
  1152. -1287> 2012-12-18 19:54:57.081247 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6813/17486 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193d000 con 0x9cd1400
  1153. -1286> 2012-12-18 19:54:57.081263 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1255292 (0x8e4928 -> 10132857)
  1154. -1285> 2012-12-18 19:54:57.082174 7f977457a700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccf800 sd=18 pgs=0 cs=0 l=0).accept sd=18
  1155. -1284> 2012-12-18 19:54:57.084297 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (29984402 -> 29984459)
  1156. -1283> 2012-12-18 19:54:57.084311 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (10132857 -> 10132914)
  1157. -1282> 2012-12-18 19:54:57.084428 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3703736316 0 0) 0x193d400 con 0x9cd0c80
  1158. -1281> 2012-12-18 19:54:57.084455 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.084457 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1159. -1280> 2012-12-18 19:54:57.084503 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6827/13431 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7228200 con 0x9cd0c80
  1160. -1279> 2012-12-18 19:54:57.084526 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 29984402)
  1161. -1278> 2012-12-18 19:54:57.084587 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 10132857)
  1162. -1277> 2012-12-18 19:54:57.085048 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 7 ==== log(1000 entries) v1 ==== 290870+0+0 (3193489689 0 0) 0x700a400 con 0x7212140
  1163. -1276> 2012-12-18 19:54:57.085067 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.085069 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1164. -1275> 2012-12-18 19:54:57.085076 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (29984402 -> 29984424)
  1165. -1274> 2012-12-18 19:54:57.085114 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (10132857 -> 10132879)
  1166. -1273> 2012-12-18 19:54:57.085159 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3468 (29984424 -> 29987892)
  1167. -1272> 2012-12-18 19:54:57.085164 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3468 (10132879 -> 10136347)
  1168. -1271> 2012-12-18 19:54:57.087218 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 290870 (0x8e4928 -> 9845477)
  1169. -1270> 2012-12-18 19:54:57.087229 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 8 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x19ff520 con 0x7212140
  1170. -1269> 2012-12-18 19:54:57.087243 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.087244 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1171. -1268> 2012-12-18 19:54:57.087258 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 9845455)
  1172. -1267> 2012-12-18 19:54:57.087262 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 9 ==== osd_pgtemp(e10021 {0.b59=[13,47,75],1.b58=[13,47,75],1.bbc=[40,47,57],18.b47=[47,58,13],18.1081=[37,47,65],18.1e47=[49,47,58],18.3190=[47,53,73],18.5915=[47,51,25],18.5a4c=[47,32,29],18.5a98=[47,57,13],18.5e3c=[47,74,24],18.617e=[47,57,66],18.61f9=[47,55,71],18.6a23=[47,53,10],18.6c18=[38,47,71],18.6e79=[47,55,73],18.8cfd=[51,47,64],18.9a8c=[47,57,71],18.9b7b=[47,58,19],18.b567=[],18.b56a=[47,56,72],18.b60c=[47,54,28],18.c753=[47,56,50],18.cfff=[47,50,64],18.daec=[47,34,69],18.dc97=[47,58,70],18.e1e6=[44,47,64],18.ed46=[47,57,26],18.f031=[],18.f28e=[],18.f6dd=[47,57,13],18.fdd4=[47,64,15],18.109ab=[21,47,72],18.11700=[47,30,72],18.12356=[26,47,44],18.1246b=[58,47,66],18.127c8=[17,47,65],18.12dc0=[41,47,64],18.13658=[61,47,69],18.13740=[33,47,74],18.139e3=[54,47,63],18.13b1f=[47,74,24],18.13de1=[47,20,68],18.1433a=[47,22,11],18.14c05=[47,57,63],18.14c81=[51,47,70],18.15140=[47,38,26,73],18.1561e=[16,47,27],18.1594a=[17,47,63],18.15a6b=[47,74,63],18.16243=[19,47,71],18.172ef=[38,47,54],18.17964=[27,47,41],18.17ce1=[],18.1991c=[40,47,53],18.1a068=[47,10,66],18.1a4d8=[47,41,11,61],18.1a81e=[54,47,67],18.1ac31=[45,47,68],18.1b00b=[22,47,71],18.1b7e5=[62,47,69],18.1bece=[47,50,75],18.1bf55=[47,38,68],18.1c0bf=[22,47,59],18.1c669=[47,33,50],18.1cd56=[47,34,29],18.1dbdd=[47,33,22],18.1dc40=[44,47,68],18.1e02f=[40,47,60],18.1ee79=[12,47,75],18.1f7de=[17,47,45],18.1fbee=[47,17,42,74],18.20d4f=[47,53,71],18.213b6=[47,57,67],18.21b41=[47,75,22],18.21f19=[47,62,69],18.22800=[47,28,22],18.2366b=[],18.237b8=[15,47,30],18.23847=[47,13,12],18.243f5=[38,47,60],18.25569=[47,38,61],18.257a7=[47,70,74],18.26463=[47,37,70],18.26939=[47,38,63],18.26b52=[47,56,63],18.27985=[47,56,17],18.28c9d=[47,52,46],18.28e23=[47,38,69],18.28f2e=[47,34,15],18.295a7=[47,37,10],18.295c9=[47,52,14],18.29b21=[47,29,14],18.29ec3=[26,47,59],18.2a4ac=[47,70,60],18.2b424=[47,70,46,72],18.2b61d=[47,51,10],18.2bbcf=[47,63,15],18.2c27e=[47,55,26],18.2cf74=[47,58,11],18.2d116=[],18.2d593=[47,69,64],18.2dd74=[47,50,22],18.2e0e0=[47,17,60],18.2eca4=[47,75,61],18.301f7=[47,57,51]} v10021) v1 ==== 3468+0+0 (4193348384 0 0) 0x89925a0 con 0x7212140
  1173. -1266> 2012-12-18 19:54:57.087360 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.087361 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1174. -1265> 2012-12-18 19:54:57.087376 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3468 (0x8e4928 -> 9841987)
  1175. -1264> 2012-12-18 19:54:57.091530 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (29987892 -> 29988050)
  1176. -1263> 2012-12-18 19:54:57.091553 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (9841987 -> 9842145)
  1177. -1262> 2012-12-18 19:54:57.091680 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1296785430 0 0) 0x9cc6c00 con 0x9cd0c80
  1178. -1261> 2012-12-18 19:54:57.091700 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.091701 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1179. -1260> 2012-12-18 19:54:57.091907 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6827/13431 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193d400 con 0x9cd0c80
  1180. -1259> 2012-12-18 19:54:57.091926 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 29987892)
  1181. -1258> 2012-12-18 19:54:57.091932 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 9841987)
  1182. -1257> 2012-12-18 19:54:57.092393 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1271908 (29987892 -> 31259800)
  1183. -1256> 2012-12-18 19:54:57.092407 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1271908 (9841987 -> 11113895)
  1184. -1255> 2012-12-18 19:54:57.093094 7f9771154700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (31259800 -> 31259832)
  1185. -1254> 2012-12-18 19:54:57.093114 7f9771154700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (11113895 -> 11113927)
  1186. -1253> 2012-12-18 19:54:57.093127 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 7 ==== log(1000 entries) v1 ==== 300192+0+0 (2428173855 0 0) 0x7222000 con 0x9cd1900
  1187. -1252> 2012-12-18 19:54:57.093147 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.093148 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1188. -1251> 2012-12-18 19:54:57.095245 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 300192 (0x8e4928 -> 10813735)
  1189. -1250> 2012-12-18 19:54:57.095255 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.48 10.1.0.14:6825/3740 8 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x9cc6a00 con 0x9cd1900
  1190. -1249> 2012-12-18 19:54:57.095267 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.095268 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1191. -1248> 2012-12-18 19:54:57.095317 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6825/3740 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6c00 con 0x9cd1900
  1192. -1247> 2012-12-18 19:54:57.095334 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 31259800)
  1193. -1246> 2012-12-18 19:54:57.095339 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 10813703)
  1194. -1245> 2012-12-18 19:54:57.095699 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (31259800 -> 31259832)
  1195. -1244> 2012-12-18 19:54:57.095711 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (10813703 -> 10813735)
  1196. -1243> 2012-12-18 19:54:57.095700 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 7 ==== log(1000 entries) v1 ==== 295959+0+0 (1005101586 0 0) 0x9cc6400 con 0x9cd1400
  1197. -1242> 2012-12-18 19:54:57.095718 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.095720 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1198. -1241> 2012-12-18 19:54:57.097162 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (31259832 -> 31259874)
  1199. -1240> 2012-12-18 19:54:57.097172 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (10813735 -> 10813777)
  1200. -1239> 2012-12-18 19:54:57.097194 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (31259874 -> 31259896)
  1201. -1238> 2012-12-18 19:54:57.097199 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (10813777 -> 10813799)
  1202. -1237> 2012-12-18 19:54:57.097217 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (31259896 -> 31260376)
  1203. -1236> 2012-12-18 19:54:57.097222 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (10813799 -> 10814279)
  1204. -1235> 2012-12-18 19:54:57.097250 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (31260376 -> 31918136)
  1205. -1234> 2012-12-18 19:54:57.097254 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (10814279 -> 11472039)
  1206. -1233> 2012-12-18 19:54:57.097775 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 295959 (0x8e4928 -> 11176080)
  1207. -1232> 2012-12-18 19:54:57.097785 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 8 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x193d600 con 0x9cd1400
  1208. -1231> 2012-12-18 19:54:57.097797 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.097798 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1209. -1230> 2012-12-18 19:54:57.097843 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6813/17486 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6a00 con 0x9cd1400
  1210. -1229> 2012-12-18 19:54:57.097860 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 31918104)
  1211. -1228> 2012-12-18 19:54:57.097864 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 11176048)
  1212. -1227> 2012-12-18 19:54:57.097869 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 13 ==== pg_stats(2066 pgs tid 601509 v 10021) v1 ==== 657760+0+0 (1111895650 0 0) 0x5f33200 con 0x9a93a40
  1213. -1226> 2012-12-18 19:54:57.097878 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.097879 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1214. -1225> 2012-12-18 19:54:57.097892 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.097893 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1215. -1224> 2012-12-18 19:54:57.097902 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  1216. -1223> 2012-12-18 19:54:57.097931 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193d600 con 0x9a93a40
  1217. -1222> 2012-12-18 19:54:57.097945 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 10518288)
  1218. -1221> 2012-12-18 19:54:57.097950 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 14 ==== mon_subscribe({monmap=2+,osdmap=9314}) v2 ==== 42+0+0 (2165263006 0 0) 0xada1a40 con 0x9a93a40
  1219. -1220> 2012-12-18 19:54:57.097965 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9314..10022] to osd.29 10.1.0.15:6824/20884
  1220. -1219> 2012-12-18 19:54:57.098627 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (31918104 -> 31918146)
  1221. -1218> 2012-12-18 19:54:57.098664 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (10518288 -> 10518330)
  1222. -1217> 2012-12-18 19:54:57.098703 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (31918146 -> 31918168)
  1223. -1216> 2012-12-18 19:54:57.098784 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (10518330 -> 10518352)
  1224. -1215> 2012-12-18 19:54:57.098844 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- osd_map(9314..9414 src has 7326..10022) v3 -- ?+0 0x7004e00
  1225. -1214> 2012-12-18 19:54:57.098865 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.29 10.1.0.15:6824/20884 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19ffa00
  1226. -1213> 2012-12-18 19:54:57.098912 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 31918126)
  1227. -1212> 2012-12-18 19:54:57.098919 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 10518310)
  1228. -1211> 2012-12-18 19:54:57.098924 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 15 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19feea0 con 0x9a93a40
  1229. -1210> 2012-12-18 19:54:57.098935 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.098936 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1230. -1209> 2012-12-18 19:54:57.098951 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 10518288)
  1231. -1208> 2012-12-18 19:54:57.098978 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 558 (31918126 -> 31918684)
  1232. -1207> 2012-12-18 19:54:57.099077 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 558 (10518288 -> 10518846)
  1233. -1206> 2012-12-18 19:54:57.098988 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 3 ==== mon_subscribe({monmap=2+,osdmap=9764}) v2 ==== 42+0+0 (4058747151 0 0) 0xae8a000 con 0x9cd0c80
  1234. -1205> 2012-12-18 19:54:57.099126 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9764..10022] to osd.19 10.1.0.13:6827/13431
  1235. -1204> 2012-12-18 19:54:57.099764 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.19 10.1.0.13:6827/13431 -- osd_map(9764..9864 src has 7326..10022) v3 -- ?+0 0x7004200
  1236. -1203> 2012-12-18 19:54:57.099784 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.19 10.1.0.13:6827/13431 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19ffd40
  1237. -1202> 2012-12-18 19:54:57.099792 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 31918642)
  1238. -1201> 2012-12-18 19:54:57.099835 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 10518804)
  1239. -1200> 2012-12-18 19:54:57.099838 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 16 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x8992b40 con 0x9a93a40
  1240. -1199> 2012-12-18 19:54:57.099863 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.099864 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1241. -1198> 2012-12-18 19:54:57.099886 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 10518324)
  1242. -1197> 2012-12-18 19:54:57.099889 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 4 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19fed00 con 0x9cd0c80
  1243. -1196> 2012-12-18 19:54:57.099899 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.099900 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1244. -1195> 2012-12-18 19:54:57.099910 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 10518302)
  1245. -1194> 2012-12-18 19:54:57.099915 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 5 ==== osd_pgtemp(e10021 {18.c03=[19,56,10],18.804c=[19,51,72],18.9007=[19,57,10],18.95be=[19,51,68],18.e0da=[19,59,56],18.113ea=[19,59,22],18.1528a=[19,37,60],18.19f3c=[19,39,21,8],18.1a6dd=[19,59,64],18.21ecf=[19,48,72],18.22168=[19,57,68],18.26bfd=[19,37,64],18.29293=[19,52,24],18.29fd8=[19,27,10],18.2ccf2=[19,49,45],18.2d1bb=[19,70,10]} v10021) v1 ==== 558+0+0 (4132626247 0 0) 0x8992f00 con 0x9cd0c80
  1246. -1193> 2012-12-18 19:54:57.099944 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.099945 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1247. -1192> 2012-12-18 19:54:57.099979 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.099980 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1248. -1191> 2012-12-18 19:54:57.099989 7f977255a700 5 mon.a@0(leader).osd e10022 send_latest to osd.19 10.1.0.13:6827/13431 start 10021
  1249. -1190> 2012-12-18 19:54:57.099994 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10021..10022] to osd.19 10.1.0.13:6827/13431
  1250. -1189> 2012-12-18 19:54:57.100023 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6827/13431 -- osd_map(10021..10022 src has 7326..10022) v3 -- ?+0 0x7004800 con 0x9cd0c80
  1251. -1188> 2012-12-18 19:54:57.100035 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 558 (0x8e4928 -> 31918084)
  1252. -1187> 2012-12-18 19:54:57.100039 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 558 (0x8e4928 -> 10517744)
  1253. -1186> 2012-12-18 19:54:57.101342 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 9 ==== pg_stats(3487 pgs tid 731926 v 10021) v1 ==== 1109416+0+0 (1691202241 0 0) 0x6afc000 con 0x722c3c0
  1254. -1185> 2012-12-18 19:54:57.101354 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1253 (31918084 -> 31919337)
  1255. -1184> 2012-12-18 19:54:57.101365 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1253 (10517744 -> 10518997)
  1256. -1183> 2012-12-18 19:54:57.101362 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.101363 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1257. -1182> 2012-12-18 19:54:57.101379 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.101380 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1258. -1181> 2012-12-18 19:54:57.101391 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.72 10.1.0.18:6806/16967
  1259. -1180> 2012-12-18 19:54:57.101424 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6806/16967 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x7004a00 con 0x722c3c0
  1260. -1179> 2012-12-18 19:54:57.101442 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1109416 (0x8e4928 -> 9409581)
  1261. -1178> 2012-12-18 19:54:57.101452 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 10 ==== log(5 entries) v1 ==== 1253+0+0 (4119175919 0 0) 0x6092000 con 0x722c3c0
  1262. -1177> 2012-12-18 19:54:57.101462 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.101463 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1263. -1176> 2012-12-18 19:54:57.101491 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1253 (0x8e4928 -> 9408328)
  1264. -1175> 2012-12-18 19:54:57.106666 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 567388 (31919337 -> 32486725)
  1265. -1174> 2012-12-18 19:54:57.106684 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 567388 (9408328 -> 9975716)
  1266. -1173> 2012-12-18 19:54:57.124675 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 10 ==== pg_stats(4035 pgs tid 21630 v 10021) v1 ==== 1283800+0+0 (1982489362 0 0) 0x6afdd40 con 0x7003e00
  1267. -1172> 2012-12-18 19:54:57.124683 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 10264 (32486725 -> 32496989)
  1268. -1171> 2012-12-18 19:54:57.124708 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 10264 (9975716 -> 9985980)
  1269. -1170> 2012-12-18 19:54:57.124697 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.124698 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1270. -1169> 2012-12-18 19:54:57.124716 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.124718 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1271. -1168> 2012-12-18 19:54:57.124729 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.55 10.1.0.16:6816/13746
  1272. -1167> 2012-12-18 19:54:57.124755 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6816/13746 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cc6e00 con 0x7003e00
  1273. -1166> 2012-12-18 19:54:57.124775 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1283800 (0x8e4928 -> 8702180)
  1274. -1165> 2012-12-18 19:54:57.124823 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 11 ==== log(32 entries) v1 ==== 10264+0+0 (2209837264 0 0) 0x632a000 con 0x7003e00
  1275. -1164> 2012-12-18 19:54:57.124843 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.124844 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1276. -1163> 2012-12-18 19:54:57.124937 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 10264 (0x8e4928 -> 8691916)
  1277. -1162> 2012-12-18 19:54:57.137241 7f9773166700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccf600 sd=43 pgs=0 cs=0 l=0).accept sd=43
  1278. -1161> 2012-12-18 19:54:57.139401 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 17 ==== pg_stats(2066 pgs tid 601510 v 10021) v1 ==== 657760+0+0 (1786248916 0 0) 0x69a7b00 con 0x9a93a40
  1279. -1160> 2012-12-18 19:54:57.139421 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.139423 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1280. -1159> 2012-12-18 19:54:57.139446 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.139448 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1281. -1158> 2012-12-18 19:54:57.139463 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  1282. -1157> 2012-12-18 19:54:57.139486 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cc6000 con 0x9a93a40
  1283. -1156> 2012-12-18 19:54:57.139509 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 8034156)
  1284. -1155> 2012-12-18 19:54:57.139479 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (32496989 -> 32497011)
  1285. -1154> 2012-12-18 19:54:57.139616 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (8034156 -> 8034178)
  1286. -1153> 2012-12-18 19:54:57.139775 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 18 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19ff040 con 0x9a93a40
  1287. -1152> 2012-12-18 19:54:57.139791 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (32497011 -> 32497491)
  1288. -1151> 2012-12-18 19:54:57.139801 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (8034178 -> 8034658)
  1289. -1150> 2012-12-18 19:54:57.139796 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.139797 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1290. -1149> 2012-12-18 19:54:57.139814 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 8034636)
  1291. -1148> 2012-12-18 19:54:57.139860 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (32497491 -> 33155251)
  1292. -1147> 2012-12-18 19:54:57.139872 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (8034636 -> 8692396)
  1293. -1146> 2012-12-18 19:54:57.139864 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 19 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x8992780 con 0x9a93a40
  1294. -1145> 2012-12-18 19:54:57.139899 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.140259 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1295. -1144> 2012-12-18 19:54:57.140288 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 8691916)
  1296. -1143> 2012-12-18 19:54:57.140525 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (33155251 -> 33155308)
  1297. -1142> 2012-12-18 19:54:57.140538 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (8691916 -> 8691973)
  1298. -1141> 2012-12-18 19:54:57.140686 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (4087948922 0 0) 0x193d600 con 0x9cd0a00
  1299. -1140> 2012-12-18 19:54:57.140708 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.140710 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1300. -1139> 2012-12-18 19:54:57.140755 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6825/20566 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb8400 con 0x9cd0a00
  1301. -1138> 2012-12-18 19:54:57.140775 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 33155251)
  1302. -1137> 2012-12-18 19:54:57.140780 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 8691916)
  1303. -1136> 2012-12-18 19:54:57.149915 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (33155251 -> 33155409)
  1304. -1135> 2012-12-18 19:54:57.149943 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (8691916 -> 8692074)
  1305. -1134> 2012-12-18 19:54:57.150110 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3218420420 0 0) 0x9cc7a00 con 0x9cd0a00
  1306. -1133> 2012-12-18 19:54:57.150131 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.150132 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1307. -1132> 2012-12-18 19:54:57.150146 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4164 (33155409 -> 33159573)
  1308. -1131> 2012-12-18 19:54:57.150158 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4164 (8692074 -> 8696238)
  1309. -1130> 2012-12-18 19:54:57.150306 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6825/20566 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193d600 con 0x9cd0a00
  1310. -1129> 2012-12-18 19:54:57.150324 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 33159415)
  1311. -1128> 2012-12-18 19:54:57.150329 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 8696080)
  1312. -1127> 2012-12-18 19:54:57.150376 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 6 ==== pg_stats(1780 pgs tid 641974 v 10021) v1 ==== 567388+0+0 (502198562 0 0) 0x69a6000 con 0x9cd0c80
  1313. -1126> 2012-12-18 19:54:57.150388 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.150389 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1314. -1125> 2012-12-18 19:54:57.150402 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.150402 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1315. -1124> 2012-12-18 19:54:57.150411 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.19 10.1.0.13:6827/13431
  1316. -1123> 2012-12-18 19:54:57.150431 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6827/13431 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cc7a00 con 0x9cd0c80
  1317. -1122> 2012-12-18 19:54:57.150445 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 567388 (0x8e4928 -> 8128692)
  1318. -1121> 2012-12-18 19:54:57.150450 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 7 ==== log(17 entries) v1 ==== 4164+0+0 (2163271952 0 0) 0x193d800 con 0x9cd0c80
  1319. -1120> 2012-12-18 19:54:57.150460 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.150461 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1320. -1119> 2012-12-18 19:54:57.150509 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4164 (0x8e4928 -> 8124528)
  1321. -1118> 2012-12-18 19:54:57.150846 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 473 (33159415 -> 33159888)
  1322. -1117> 2012-12-18 19:54:57.150858 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 473 (8124528 -> 8125001)
  1323. -1116> 2012-12-18 19:54:57.150940 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 12 ==== osd_pgtemp(e10022 {18.1a50=[24,54,65],18.54ed=[],18.6805=[24,45,13],18.10c9c=[24,57,62],18.119a0=[],18.1a102=[24,47,73],18.20916=[24,55,75],18.25bbb=[24,49,69],18.29271=[24,57,66],18.29421=[24,53,64],18.2a378=[24,52,53],18.2a661=[],18.2a8a4=[24,40,74],18.2b172=[],18.2e53d=[24,33,17]} v10022) v1 ==== 473+0+0 (2708262280 0 0) 0x7b6c780 con 0x72123c0
  1324. -1115> 2012-12-18 19:54:57.150975 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.150977 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1325. -1114> 2012-12-18 19:54:57.150997 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 473 (0x8e4928 -> 8124528)
  1326. -1113> 2012-12-18 19:54:57.152304 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 9 ==== pg_stats(4010 pgs tid 713589 v 10021) v1 ==== 1275484+0+0 (4060009348 0 0) 0x69a6fc0 con 0x722cdc0
  1327. -1112> 2012-12-18 19:54:57.152324 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.152325 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1328. -1111> 2012-12-18 19:54:57.152331 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 291262 (33159888 -> 33451150)
  1329. -1110> 2012-12-18 19:54:57.152348 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.152349 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1330. -1109> 2012-12-18 19:54:57.152354 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 291262 (8124528 -> 8415790)
  1331. -1108> 2012-12-18 19:54:57.152363 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.59 10.1.0.16:6828/20636
  1332. -1107> 2012-12-18 19:54:57.152387 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6828/20636 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9a00 con 0x722cdc0
  1333. -1106> 2012-12-18 19:54:57.152406 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1275484 (0x8e4928 -> 7140306)
  1334. -1105> 2012-12-18 19:54:57.160303 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 10 ==== log(1000 entries) v1 ==== 291262+0+0 (1664185668 0 0) 0x7004000 con 0x722cdc0
  1335. -1104> 2012-12-18 19:54:57.160327 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.160329 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1336. -1103> 2012-12-18 19:54:57.161488 7f9770f52700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 645 (33451150 -> 33451795)
  1337. -1102> 2012-12-18 19:54:57.161526 7f9770f52700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 645 (7140306 -> 7140951)
  1338. -1101> 2012-12-18 19:54:57.162399 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 291262 (0x8e4928 -> 6849689)
  1339. -1100> 2012-12-18 19:54:57.162413 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.14 10.1.0.13:6812/8435 8 ==== osd_pgtemp(e10022 {0.b8a=[14,46,10],18.3f5=[14,50,10],18.b78=[14,46,10],18.6896=[14,43,10],18.7764=[14,32,10],18.c572=[14,75,10],18.d40c=[14,58,10],18.e540=[14,43,10],18.14ce5=[14,42,10],18.17240=[14,59,53],18.2102a=[14,54,10],18.22089=[14,41,70,47],18.24118=[14,48,10],18.253b2=[14,27,10],18.28201=[],18.28767=[14,41,75],18.29b69=[14,59,29],18.2ed5f=[14,54,10],18.2f973=[14,37,61]} v10022) v1 ==== 645+0+0 (3317655964 0 0) 0x89932c0 con 0x9cd1180
  1340. -1099> 2012-12-18 19:54:57.162444 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.162445 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1341. -1098> 2012-12-18 19:54:57.162462 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 645 (0x8e4928 -> 6849044)
  1342. -1097> 2012-12-18 19:54:57.164237 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 10 ==== pg_stats(3998 pgs tid 717013 v 10021) v1 ==== 1271908+0+0 (109223641 0 0) 0xca566900 con 0x7212140
  1343. -1096> 2012-12-18 19:54:57.164257 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.164258 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1344. -1095> 2012-12-18 19:54:57.164277 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.164279 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1345. -1094> 2012-12-18 19:54:57.164288 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.47 10.1.0.14:6822/19203
  1346. -1093> 2012-12-18 19:54:57.164312 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6822/19203 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x9cb9e00 con 0x7212140
  1347. -1092> 2012-12-18 19:54:57.164331 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1271908 (0x8e4928 -> 5577136)
  1348. -1091> 2012-12-18 19:54:57.164897 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1029 (33451795 -> 33452824)
  1349. -1090> 2012-12-18 19:54:57.164912 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1029 (5577136 -> 5578165)
  1350. -1089> 2012-12-18 19:54:57.165104 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 12 ==== osd_pgtemp(e10022 {0.52d=[],0.69e=[],1.29c=[],1.52c=[],1.69d=[],18.e59=[16,41,10],18.ecb=[16,51,55],18.2fda=[],18.51fe=[16,53,10],18.5a48=[16,34,29],18.6151=[],18.7999=[],18.8906=[16,56,65],18.d4ef=[16,51,68],18.df4e=[16,51,10],18.11e33=[],18.12507=[],18.1268c=[16,54,68],18.1314c=[],18.1453b=[16,41,52,68],18.1463c=[16,59,10],18.14741=[16,32,61],18.1c8ab=[55,46,64,16],18.1eb24=[],18.21050=[16,32,68],18.23397=[16,24,31,0],18.2589d=[16,73,38,3],18.25942=[],18.2916d=[16,52,64],18.2925a=[],18.2b4dd=[16,59,67],18.2cc8d=[16,73,11],18.2d3a7=[16,55,60],18.2ec3b=[16,44,10],18.2f650=[16,56,10]} v10022) v1 ==== 1029+0+0 (980354403 0 0) 0x7b721e0 con 0x9a93400
  1351. -1088> 2012-12-18 19:54:57.165154 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.165156 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1352. -1087> 2012-12-18 19:54:57.165171 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1029 (0x8e4928 -> 5577136)
  1353. -1086> 2012-12-18 19:54:57.166650 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 6 ==== pg_stats(3424 pgs tid 717643 v 10021) v1 ==== 1089896+0+0 (1487534658 0 0) 0xca566480 con 0x7212780
  1354. -1085> 2012-12-18 19:54:57.166671 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.166672 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1355. -1084> 2012-12-18 19:54:57.166678 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (33452824 -> 33452846)
  1356. -1083> 2012-12-18 19:54:57.166694 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.166696 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1357. -1082> 2012-12-18 19:54:57.166702 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (5577136 -> 5577158)
  1358. -1081> 2012-12-18 19:54:57.166710 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.68 10.1.0.18:6821/19143
  1359. -1080> 2012-12-18 19:54:57.166735 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6821/19143 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x632bc00 con 0x7212780
  1360. -1079> 2012-12-18 19:54:57.166743 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 704 (33452846 -> 33453550)
  1361. -1078> 2012-12-18 19:54:57.166752 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 704 (5577158 -> 5577862)
  1362. -1077> 2012-12-18 19:54:57.166792 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1089896 (0x8e4928 -> 4487966)
  1363. -1076> 2012-12-18 19:54:57.166807 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x5b2fd40 con 0x7212780
  1364. -1075> 2012-12-18 19:54:57.166823 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.166824 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1365. -1074> 2012-12-18 19:54:57.166843 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 4487944)
  1366. -1073> 2012-12-18 19:54:57.166846 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1089896 (33453550 -> 34543446)
  1367. -1072> 2012-12-18 19:54:57.166857 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1089896 (4487944 -> 5577840)
  1368. -1071> 2012-12-18 19:54:57.166850 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 8 ==== osd_pgtemp(e10021 {18.4072=[68,36,43,67],18.5e69=[],18.60c9=[68,40,19],18.9c9a=[68,50,44],18.a986=[],18.ac53=[68,52,67],18.bb43=[68,53,45],18.d106=[68,49,16],18.de16=[68,30,19],18.e405=[68,53,15],18.ef2a=[],18.f51c=[68,59,20],18.14f09=[68,44,52,47],18.16aaa=[],18.1d43d=[],18.1e3a6=[68,31,58,62],18.211d2=[68,50,25],18.22253=[68,57,43],18.22f2b=[68,32,60],18.251db=[68,54,67],18.2a0ae=[68,55,63],18.2e0d9=[68,37,49]} v10021) v1 ==== 704+0+0 (3987670576 0 0) 0x8993680 con 0x7212780
  1369. -1070> 2012-12-18 19:54:57.166887 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.166888 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1370. -1069> 2012-12-18 19:54:57.166902 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 704 (0x8e4928 -> 5577136)
  1371. -1068> 2012-12-18 19:54:57.171321 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 298253 (34543446 -> 34841699)
  1372. -1067> 2012-12-18 19:54:57.171321 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 6 ==== pg_stats(4005 pgs tid 723946 v 10021) v1 ==== 1274388+0+0 (2489288226 0 0) 0xca566b40 con 0x722d540
  1373. -1066> 2012-12-18 19:54:57.171337 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 298253 (5577136 -> 5875389)
  1374. -1065> 2012-12-18 19:54:57.171339 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.171341 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1375. -1064> 2012-12-18 19:54:57.171359 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.171360 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1376. -1063> 2012-12-18 19:54:57.171372 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.50 10.1.0.16:6801/16285
  1377. -1062> 2012-12-18 19:54:57.171397 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6801/16285 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394400 con 0x722d540
  1378. -1061> 2012-12-18 19:54:57.171416 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1274388 (0x8e4928 -> 4601001)
  1379. -1060> 2012-12-18 19:54:57.173152 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 20 ==== pg_stats(2066 pgs tid 601511 v 10021) v1 ==== 657760+0+0 (1705400655 0 0) 0x69a7200 con 0x9a93a40
  1380. -1059> 2012-12-18 19:54:57.173160 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (34841699 -> 34841721)
  1381. -1058> 2012-12-18 19:54:57.173174 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (4601001 -> 4601023)
  1382. -1057> 2012-12-18 19:54:57.173173 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.173178 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1383. -1056> 2012-12-18 19:54:57.173193 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 480 (34841721 -> 34842201)
  1384. -1055> 2012-12-18 19:54:57.173197 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 480 (4601023 -> 4601503)
  1385. -1054> 2012-12-18 19:54:57.173197 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.173198 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1386. -1053> 2012-12-18 19:54:57.173211 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  1387. -1052> 2012-12-18 19:54:57.173237 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394200 con 0x9a93a40
  1388. -1051> 2012-12-18 19:54:57.173258 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 3943743)
  1389. -1050> 2012-12-18 19:54:57.173266 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 21 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19ff380 con 0x9a93a40
  1390. -1049> 2012-12-18 19:54:57.173276 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.173277 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1391. -1048> 2012-12-18 19:54:57.173290 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 3943721)
  1392. -1047> 2012-12-18 19:54:57.173293 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 22 ==== osd_pgtemp(e10021 {18.9fd=[],18.3812=[29,58,11],18.40e2=[29,37,12],18.668d=[29,52,14],18.8e95=[29,51,43],18.c0f0=[29,33,72],18.105ce=[29,56,13],18.16b0f=[47,29,75,49],18.20ae6=[29,51,75],18.21bb9=[29,55,61],18.27e20=[29,58,72],18.2a381=[29,43,18],18.2d336=[29,41,21],18.2f2b1=[29,30,74]} v10021) v1 ==== 480+0+0 (238162170 0 0) 0x8993c20 con 0x9a93a40
  1393. -1046> 2012-12-18 19:54:57.173318 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.173318 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1394. -1045> 2012-12-18 19:54:57.173335 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 480 (0x8e4928 -> 3943241)
  1395. -1044> 2012-12-18 19:54:57.175373 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 657760 (34842201 -> 35499961)
  1396. -1043> 2012-12-18 19:54:57.175388 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 657760 (3943241 -> 4601001)
  1397. -1042> 2012-12-18 19:54:57.176237 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 7 ==== log(1000 entries) v1 ==== 298253+0+0 (1271207217 0 0) 0xb7b6800 con 0x722d540
  1398. -1041> 2012-12-18 19:54:57.176257 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.176259 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1399. -1040> 2012-12-18 19:54:57.176715 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 13572 (35499961 -> 35513533)
  1400. -1039> 2012-12-18 19:54:57.176726 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 13572 (4601001 -> 4614573)
  1401. -1038> 2012-12-18 19:54:57.178361 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 298253 (0x8e4928 -> 4316320)
  1402. -1037> 2012-12-18 19:54:57.178370 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 9 ==== pg_stats(4031 pgs tid 792328 v 10021) v1 ==== 1282068+0+0 (2240413266 0 0) 0xca566240 con 0x7213e00
  1403. -1036> 2012-12-18 19:54:57.178384 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.178385 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1404. -1035> 2012-12-18 19:54:57.178400 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.178400 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1405. -1034> 2012-12-18 19:54:57.178409 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.53 10.1.0.16:6810/16733
  1406. -1033> 2012-12-18 19:54:57.178431 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6810/16733 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394600 con 0x7213e00
  1407. -1032> 2012-12-18 19:54:57.178444 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1282068 (0x8e4928 -> 3034252)
  1408. -1031> 2012-12-18 19:54:57.178449 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 10 ==== log(44 entries) v1 ==== 13572+0+0 (2841232144 0 0) 0x7210200 con 0x7213e00
  1409. -1030> 2012-12-18 19:54:57.178459 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.179145 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1410. -1029> 2012-12-18 19:54:57.179257 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 13572 (0x8e4928 -> 3020680)
  1411. -1028> 2012-12-18 19:54:57.190640 7f9772f64700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 321665 (35513533 -> 35835198)
  1412. -1027> 2012-12-18 19:54:57.190642 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 23 ==== pg_stats(2066 pgs tid 601512 v 10021) v1 ==== 657760+0+0 (2637484981 0 0) 0x69a78c0 con 0x9a93a40
  1413. -1026> 2012-12-18 19:54:57.190660 7f9772f64700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 321665 (3020680 -> 3342345)
  1414. -1025> 2012-12-18 19:54:57.190661 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.190663 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1415. -1024> 2012-12-18 19:54:57.190681 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.190682 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1416. -1023> 2012-12-18 19:54:57.190695 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.29 10.1.0.15:6824/20884
  1417. -1022> 2012-12-18 19:54:57.190720 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6824/20884 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc395400 con 0x9a93a40
  1418. -1021> 2012-12-18 19:54:57.190739 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 657760 (0x8e4928 -> 2684585)
  1419. -1020> 2012-12-18 19:54:57.192163 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 766 (35835198 -> 35835964)
  1420. -1019> 2012-12-18 19:54:57.192180 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 766 (2684585 -> 2685351)
  1421. -1018> 2012-12-18 19:54:57.192192 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 9 ==== pg_stats(3424 pgs tid 717644 v 10021) v1 ==== 1089896+0+0 (150951627 0 0) 0xca566fc0 con 0x7212780
  1422. -1017> 2012-12-18 19:54:57.192218 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (35835964 -> 35836006)
  1423. -1016> 2012-12-18 19:54:57.192251 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (2685351 -> 2685393)
  1424. -1015> 2012-12-18 19:54:57.192212 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.192213 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1425. -1014> 2012-12-18 19:54:57.192265 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.192268 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1426. -1013> 2012-12-18 19:54:57.192285 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.68 10.1.0.18:6821/19143
  1427. -1012> 2012-12-18 19:54:57.192318 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6821/19143 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc395200 con 0x7212780
  1428. -1011> 2012-12-18 19:54:57.192336 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1089896 (0x8e4928 -> 1595497)
  1429. -1010> 2012-12-18 19:54:57.192361 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 10 ==== log(3 entries) v1 ==== 766+0+0 (3062516747 0 0) 0x7222e00 con 0x7212780
  1430. -1009> 2012-12-18 19:54:57.192372 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.192373 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1431. -1008> 2012-12-18 19:54:57.192393 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 766 (0x8e4928 -> 1594731)
  1432. -1007> 2012-12-18 19:54:57.192398 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 11 ==== mon_subscribe({monmap=2+,osdmap=9708}) v2 ==== 42+0+0 (3849771159 0 0) 0xcb4e9180 con 0x7212780
  1433. -1006> 2012-12-18 19:54:57.192413 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9708..10022] to osd.68 10.1.0.18:6821/19143
  1434. -1005> 2012-12-18 19:54:57.193039 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.68 10.1.0.18:6821/19143 -- osd_map(9708..9808 src has 7326..10022) v3 -- ?+0 0xc395e00
  1435. -1004> 2012-12-18 19:54:57.193059 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.68 10.1.0.18:6821/19143 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5b9c1a0
  1436. -1003> 2012-12-18 19:54:57.193078 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 35835964)
  1437. -1002> 2012-12-18 19:54:57.193082 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 1594689)
  1438. -1001> 2012-12-18 19:54:57.195268 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.29 10.1.0.15:6824/20884 24 ==== log(1000 entries) v1 ==== 321665+0+0 (156870754 0 0) 0x700a800 con 0x9a93a40
  1439. -1000> 2012-12-18 19:54:57.195288 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.195289 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1440. -999> 2012-12-18 19:54:57.197342 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 321665 (0x8e4928 -> 1273024)
  1441. -998> 2012-12-18 19:54:57.228925 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (35835964 -> 35835986)
  1442. -997> 2012-12-18 19:54:57.228984 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (1273024 -> 1273046)
  1443. -996> 2012-12-18 19:54:57.229160 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 8 ==== osd_alive(want up_thru 10022 have 10022) v1 ==== 22+0+0 (1276594390 0 0) 0x5ade340 con 0x9cd0c80
  1444. -995> 2012-12-18 19:54:57.229189 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.229190 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1445. -994> 2012-12-18 19:54:57.229209 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1273024)
  1446. -993> 2012-12-18 19:54:57.274744 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 789 (35835986 -> 35836775)
  1447. -992> 2012-12-18 19:54:57.274785 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 789 (1273024 -> 1273813)
  1448. -991> 2012-12-18 19:54:57.274951 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 8 ==== osd_pgtemp(e10022 {18.4f9=[61,33,20],18.6807=[61,37,16],18.8c27=[61,53,69],18.8e9b=[61,50,24],18.ac2c=[61,40,53],18.115f0=[61,57,71],18.1374b=[61,46,21],18.17813=[61,59,69],18.1875f=[61,48,18],18.199a9=[61,54,74],18.1b1bd=[61,59,69],18.1fcef=[61,53,28],18.20f09=[61,50,74],18.21fd0=[61,44,70],18.2398f=[61,55,13],18.25d80=[61,42,36,28],18.25e76=[61,40,54],18.25f21=[61,34,22],18.268a5=[61,42,71],18.28368=[61,55,71],18.2a61b=[61,34,69],18.2dbcc=[61,47,16],18.2ff33=[61,52,56]} v10022) v1 ==== 789+0+0 (3805876721 0 0) 0xcb61a1e0 con 0x9cd1b80
  1449. -990> 2012-12-18 19:54:57.275000 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.275002 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1450. -989> 2012-12-18 19:54:57.275022 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 789 (0x8e4928 -> 1273024)
  1451. -988> 2012-12-18 19:54:57.279658 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (35836775 -> 35836797)
  1452. -987> 2012-12-18 19:54:57.279692 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (1273024 -> 1273046)
  1453. -986> 2012-12-18 19:54:57.279706 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3935 (35836797 -> 35840732)
  1454. -985> 2012-12-18 19:54:57.279744 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3935 (1273046 -> 1276981)
  1455. -984> 2012-12-18 19:54:57.279711 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 6 ==== pg_stats(4002 pgs tid 731867 v 10021) v1 ==== 1273024+0+0 (2668894290 0 0) 0x19318c0 con 0x7212dc0
  1456. -983> 2012-12-18 19:54:57.279762 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.279766 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1457. -982> 2012-12-18 19:54:57.279784 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.279785 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1458. -981> 2012-12-18 19:54:57.279795 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.56 10.1.0.16:6819/18681
  1459. -980> 2012-12-18 19:54:57.279825 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6819/18681 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394a00 con 0x7212dc0
  1460. -979> 2012-12-18 19:54:57.279842 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1273024 (0x8e4928 -> 3957)
  1461. -978> 2012-12-18 19:54:57.279848 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 7 ==== osd_alive(want up_thru 10019 have 10021) v1 ==== 22+0+0 (240050241 0 0) 0x19ff860 con 0x7212dc0
  1462. -977> 2012-12-18 19:54:57.279859 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.279860 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1463. -976> 2012-12-18 19:54:57.279877 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 3935)
  1464. -975> 2012-12-18 19:54:57.279882 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 8 ==== osd_pgtemp(e10021 {1.209=[47,56,74],1.47d=[56,38,17],18.116a=[56,38,53],18.2721=[56,69,15],18.278a=[],18.293d=[22,56,70],18.2b31=[56,66,70],18.2ff9=[21,56,49],18.30a4=[56,34,24],18.3319=[],18.33d6=[56,40,66],18.47f2=[12,56,49],18.504a=[56,45,17],18.50a0=[56,72,15],18.56df=[56,16,18],18.6383=[56,40,22],18.6775=[56,40,75],18.6912=[56,69,10],18.6aac=[16,56,65],18.6cb9=[22,56,67],18.72fa=[56,65,19],18.79e2=[56,38,51],18.7f22=[56,75,17],18.84df=[45,56,63],18.8794=[56,48,17],18.879c=[56,65,22],18.8902=[56,29,21],18.8be9=[56,34,71],18.8dd1=[51,56,72],18.900c=[19,56,47],18.9338=[56,74,60],18.9e6b=[51,56,62],18.a297=[56,34,53],18.a622=[],18.b654=[56,41,22],18.b93f=[12,56,10],18.bae9=[41,56,68],18.baff=[56,43,53],18.bc66=[56,24,60],18.bc76=[56,20,27],18.bd8c=[],18.cac9=[44,56,59],18.d1bb=[38,56,67],18.eac9=[56,64,60],18.efde=[56,40,75],18.f01a=[56,48,15],18.f20e=[56,33,29],18.f543=[56,49,71],18.f704=[56,43,69],18.f715=[30,56,67],18.1263d=[26,56,71],18.1270c=[56,40,52],18.127ab=[56,60,11],18.12a89=[56,29,13],18.132c7=[],18.140c3=[56,65,26],18.144c8=[34,56,46],18.14b3f=[20,56,71],18.15110=[21,56,69],18.15204=[37,56,75],18.1571e=[48,56,67],18.15dd1=[41,56,74],18.16364=[56,41,28],18.16bc2=[56,70,26],18.17d8f=[25,56,34],18.189d6=[20,56,60],18.19318=[56,67,60],18.19ed2=[49,56,69],18.1a0cd=[56,21,63],18.1a435=[30,56,49],18.1aa4a=[56,62,64],18.1b189=[],18.1bb21=[38,56,67],18.1bf81=[56,41,64],18.1c57c=[29,56,38],18.1c5d6=[56,47,67],18.1c929=[66,56,73],18.1cccc=[56,35,39,68],18.1cd63=[37,56,60],18.1cdd6=[56,70,64],18.1cea8=[56,27,66],18.1db01=[56,70,52],18.1e1f4=[14,56,54],18.1e8e8=[40,56,68],18.1ec13=[12,56,60],18.1f194=[14,56,67],18.2162e=[56,44,72],18.22069=[56,74,62],18.238d7=[56,30,28],18.23d1c=[56,44,29],18.23e0e=[56,38,14],18.2440c=[16,56,48],18.245a3=[56,44,63],18.24db8=[56,62,54],18.25202=[56,69,10],18.25820=[56,66,12],18.25988=[56,33,21],18.259e6=[56,34,68],18.25c44=[33,56,66],18.26243=[56,74,29],18.26284=[56,38,55],18.2658f=[27,56,38],18.26781=[56,46,17],18.2761d=[],18.286ce=[11,56,68],18.29f5b=[42,56,61],18.2ab47=[56,38,44,72],18.2acdb=[],18.2b4cc=[56,37,42],18.2c422=[56,33,55],18.2c48b=[56,30,74],18.2c888=[56,37,49],18.2c97e=[56,27,15],18.2cdfa=[56,37,20],18.2e09e=[56,43,18],18.2e778=[56,28,67],18.2efe5=[56,45,65],18.2f1d7=[28,56,75],18.2fee0=[56,49,70],18.308a5=[56,40,52],18.30cdd=[56,35,39,70]} v10021) v1 ==== 3935+0+0 (4248973102 0 0) 0xcb61a5a0 con 0x7212dc0
  1465. -974> 2012-12-18 19:54:57.280000 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.280001 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1466. -973> 2012-12-18 19:54:57.280014 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3935 (0x8e4928 -> 0)
  1467. -972> 2012-12-18 19:54:57.284984 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1273024 (35840732 -> 37113756)
  1468. -971> 2012-12-18 19:54:57.285001 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1273024 (0 -> 1273024)
  1469. -970> 2012-12-18 19:54:57.306508 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3717 (37113756 -> 37117473)
  1470. -969> 2012-12-18 19:54:57.306545 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3717 (1273024 -> 1276741)
  1471. -968> 2012-12-18 19:54:57.306561 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 9 ==== pg_stats(4002 pgs tid 731868 v 10021) v1 ==== 1273024+0+0 (3322092921 0 0) 0x6afcfc0 con 0x7212dc0
  1472. -967> 2012-12-18 19:54:57.306597 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (37117473 -> 37117515)
  1473. -966> 2012-12-18 19:54:57.306693 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (1276741 -> 1276783)
  1474. -965> 2012-12-18 19:54:57.306586 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.306588 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1475. -964> 2012-12-18 19:54:57.306754 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.306758 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1476. -963> 2012-12-18 19:54:57.306776 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.56 10.1.0.16:6819/18681
  1477. -962> 2012-12-18 19:54:57.306809 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6819/18681 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0xc394e00 con 0x7212dc0
  1478. -961> 2012-12-18 19:54:57.306826 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1273024 (0x8e4928 -> 3759)
  1479. -960> 2012-12-18 19:54:57.306832 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 10 ==== log(14 entries) v1 ==== 3717+0+0 (932696627 0 0) 0xb7b6a00 con 0x7212dc0
  1480. -959> 2012-12-18 19:54:57.306841 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.306842 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1481. -958> 2012-12-18 19:54:57.306889 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3717 (0x8e4928 -> 42)
  1482. -957> 2012-12-18 19:54:57.306894 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 11 ==== mon_subscribe({monmap=2+,osdmap=10019}) v2 ==== 42+0+0 (2142708274 0 0) 0xcba3ce00 con 0x7212dc0
  1483. -956> 2012-12-18 19:54:57.306909 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10019..10022] to osd.56 10.1.0.16:6819/18681
  1484. -955> 2012-12-18 19:54:57.306953 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.56 10.1.0.16:6819/18681 -- osd_map(10019..10022 src has 7326..10022) v3 -- ?+0 0xc395c00
  1485. -954> 2012-12-18 19:54:57.307007 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.56 10.1.0.16:6819/18681 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5af5ba0
  1486. -953> 2012-12-18 19:54:57.307015 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 37117473)
  1487. -952> 2012-12-18 19:54:57.307022 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1488. -951> 2012-12-18 19:54:57.313322 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 195 (37117473 -> 37117668)
  1489. -950> 2012-12-18 19:54:57.313337 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 195 (0 -> 195)
  1490. -949> 2012-12-18 19:54:57.313498 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 3 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2305896960 0 0) 0x9cc6c00 con 0x9cd0a00
  1491. -948> 2012-12-18 19:54:57.313522 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.313524 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1492. -947> 2012-12-18 19:54:57.315085 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6825/20566 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xc394000 con 0x9cd0a00
  1493. -946> 2012-12-18 19:54:57.315152 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 37117473)
  1494. -945> 2012-12-18 19:54:57.315158 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 0)
  1495. -944> 2012-12-18 19:54:57.319749 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (37117473 -> 37117515)
  1496. -943> 2012-12-18 19:54:57.319763 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1497. -942> 2012-12-18 19:54:57.319877 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (37117515 -> 37117537)
  1498. -941> 2012-12-18 19:54:57.319935 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  1499. -940> 2012-12-18 19:54:57.319953 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 4 ==== mon_subscribe({monmap=2+,osdmap=9355}) v2 ==== 42+0+0 (1358919065 0 0) 0xcba4e000 con 0x9cd0a00
  1500. -939> 2012-12-18 19:54:57.319983 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9355..10022] to osd.58 10.1.0.16:6825/20566
  1501. -938> 2012-12-18 19:54:57.319995 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3677 (37117537 -> 37121214)
  1502. -937> 2012-12-18 19:54:57.319998 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3677 (64 -> 3741)
  1503. -936> 2012-12-18 19:54:57.320750 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.58 10.1.0.16:6825/20566 -- osd_map(9355..9455 src has 7326..10022) v3 -- ?+0 0x9cc6c00
  1504. -935> 2012-12-18 19:54:57.320813 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.58 10.1.0.16:6825/20566 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19fe820
  1505. -934> 2012-12-18 19:54:57.320821 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 37121172)
  1506. -933> 2012-12-18 19:54:57.320861 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 3699)
  1507. -932> 2012-12-18 19:54:57.320865 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 5 ==== osd_alive(want up_thru 10020 have 10021) v1 ==== 22+0+0 (391010443 0 0) 0x19ffa00 con 0x9cd0a00
  1508. -931> 2012-12-18 19:54:57.320874 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.320875 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1509. -930> 2012-12-18 19:54:57.320890 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 3677)
  1510. -929> 2012-12-18 19:54:57.320896 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 6 ==== osd_pgtemp(e10021 {0.ec=[42,58,74],0.43f=[58,38,51],0.873=[25,58,69],1.72a=[24,58,50],1.872=[25,58,69],18.da=[58,44,42],18.719=[24,58,50],18.e92=[58,26,75],18.1cee=[58,38,54],18.2045=[58,41,67],18.2a51=[58,74,65],18.2c40=[58,34,51],18.2e58=[58,43,73],18.2ff2=[58,73,67],18.330c=[58,49,69],18.371c=[58,40,25],18.4109=[58,27,21],18.4702=[52,58,69],18.489f=[58,40,48],18.4cb6=[20,58,40],18.54de=[58,32,17],18.5851=[58,71,17],18.5fbd=[58,15,56],18.6701=[58,67,66],18.687f=[58,32,48],18.6f00=[58,43,42],18.8160=[58,48,52],18.8349=[58,40,66],18.84d0=[58,71,24],18.8a29=[58,47,18],18.9088=[58,32,68],18.983b=[58,40,75],18.9c17=[58,32,10],18.a06d=[34,58,61],18.a2dd=[58,28,20],18.a309=[40,58,51],18.a70d=[58,40,68],18.afa3=[11,58,32],18.b61d=[27,58,65],18.c14e=[58,73,67],18.c7ee=[59,58,62],18.d0bd=[58,41,47,74],18.dc12=[58,33,61],18.e04c=[30,58,46],18.e2eb=[58,46,22],18.e5bb=[58,44,54],18.ef20=[58,48,53],18.f2de=[58,42,72],18.f5bd=[24,58,61],18.f9c6=[58,47,11],18.1179e=[22,58,72],18.1186b=[17,58,63],18.11ae3=[58,33,15],18.120f9=[59,58,60],18.12a94=[58,37,57],18.13a20=[58,45,48,68],18.13ad8=[58,40,49],18.13ef1=[10,58,69],18.14459=[58,29,10],18.14cd5=[58,30,53],18.14f91=[59,58,60],18.1544b=[53,58,63],18.158df=[54,58,64],18.15c7c=[58,47,18],18.15cc3=[25,58,62],18.16662=[58,44,71],18.17ead=[51,58,61],18.17fc0=[29,58,38],18.180b8=[25,58,65],18.18c96=[58,49,64,31],18.18dba=[58,74,65],18.199c5=[58,42,66],18.19dc6=[15,58,43],18.1bb18=[58,20,34],18.1c26d=[58,37,74],18.1c622=[18,58,47],18.1c7b3=[24,58,32],18.1cff4=[22,58,74],18.1d579=[12,58,38],18.1eb6a=[20,58,59],18.1eb79=[46,58,73],18.1f14f=[58,41,71],18.1f946=[58,45,43],18.1fd69=[46,58,61],18.218e0=[19,58,65],18.2330d=[58,20,59],18.23574=[58,41,10],18.237d9=[58,47,12],18.25369=[58,66,10],18.25632=[58,64,12],18.263be=[],18.26560=[58,47,22],18.2664d=[58,48,40],18.27318=[58,49,69],18.274e6=[54,58,74],18.27a67=[45,58,73],18.27e61=[58,45,66],18.29679=[58,29,24],18.29a35=[58,62,15],18.2a014=[58,38,62],18.2a93b=[58,71,67],18.2d490=[21,58,47],18.2d819=[58,61,12],18.2da26=[58,41,21],18.2dc95=[58,68,63],18.2e8a0=[30,58,62],18.2e9f8=[58,48,75],18.2ee0d=[],18.2fed7=[58,25,62],18.30be3=[58,43,22],18.30ccc=[33,58,74]} v10021) v1 ==== 3677+0+0 (2015188324 0 0) 0xcb61a780 con 0x9cd0a00
  1511. -928> 2012-12-18 19:54:57.321005 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.321006 lease_expire=2012-12-18 19:55:01.703181 has v10021 lc 10022
  1512. -927> 2012-12-18 19:54:57.321018 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3677 (0x8e4928 -> 0)
  1513. -926> 2012-12-18 19:54:57.326495 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (37121172 -> 37121194)
  1514. -925> 2012-12-18 19:54:57.326509 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (0 -> 22)
  1515. -924> 2012-12-18 19:54:57.326642 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 11 ==== osd_alive(want up_thru 10022 have 10022) v1 ==== 22+0+0 (1276594390 0 0) 0x5b6bd40 con 0x722d7c0
  1516. -923> 2012-12-18 19:54:57.326667 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.326668 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1517. -922> 2012-12-18 19:54:57.326687 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 0)
  1518. -921> 2012-12-18 19:54:57.332919 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1254136 (37121194 -> 38375330)
  1519. -920> 2012-12-18 19:54:57.332931 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1254136 (0 -> 1254136)
  1520. -919> 2012-12-18 19:54:57.339610 7f977074a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1106 (38375330 -> 38376436)
  1521. -918> 2012-12-18 19:54:57.339625 7f977074a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1106 (1254136 -> 1255242)
  1522. -917> 2012-12-18 19:54:57.339806 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.73 10.1.0.18:6809/17050 10 ==== osd_pgtemp(e10022 {18.14d0=[73,41,14],18.2e89=[73,46,26],18.4b39=[73,46,27],18.4e83=[73,54,26],18.5e8c=[73,54,16],18.680b=[73,57,12],18.6e4a=[],18.84eb=[73,54,14],18.92c3=[73,59,15],18.bf3c=[73,38,69],18.c770=[],18.c97a=[],18.d918=[73,54,55],18.dd17=[],18.ddc1=[73,53,67],18.e9e2=[],18.f144=[73,42,64],18.194d4=[],18.19ee2=[73,45,62],18.1c2b0=[73,46,60],18.1e691=[],18.20164=[73,55,66],18.20e8a=[],18.21ca1=[73,44,62],18.23180=[73,41,20],18.25797=[73,56,63],18.25b96=[],18.2630b=[73,32,62],18.27d6c=[73,30,29],18.28f3d=[73,47,60],18.29d58=[73,56,20],18.2b811=[73,30,12],18.2ce65=[73,41,50],18.2cf23=[73,34,65],18.2d697=[73,30,45],18.2f6b4=[73,33,61]} v10022) v1 ==== 1106+0+0 (2850131974 0 0) 0x8992f00 con 0x7212b40
  1523. -916> 2012-12-18 19:54:57.339866 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.339868 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1524. -915> 2012-12-18 19:54:57.339886 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1106 (0x8e4928 -> 1254136)
  1525. -914> 2012-12-18 19:54:57.355102 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 7 ==== pg_stats(3942 pgs tid 748150 v 10021) v1 ==== 1254136+0+0 (3982768682 0 0) 0x5f33680 con 0x9cd0a00
  1526. -913> 2012-12-18 19:54:57.355127 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.355128 lease_expire=2012-12-18 19:55:01.683842 has v10021 lc 6066306
  1527. -912> 2012-12-18 19:54:57.355145 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.355147 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1528. -911> 2012-12-18 19:54:57.355156 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.58 10.1.0.16:6825/20566
  1529. -910> 2012-12-18 19:54:57.355183 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6825/20566 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x6092a00 con 0x9cd0a00
  1530. -909> 2012-12-18 19:54:57.355200 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1254136 (0x8e4928 -> 0)
  1531. -908> 2012-12-18 19:54:57.360782 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (38376436 -> 38376478)
  1532. -907> 2012-12-18 19:54:57.360807 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1533. -906> 2012-12-18 19:54:57.360965 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 13 ==== mon_subscribe({monmap=2+,osdmap=9326}) v2 ==== 42+0+0 (3887299130 0 0) 0x5c2e380 con 0x72123c0
  1534. -905> 2012-12-18 19:54:57.360995 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9326..10022] to osd.24 10.1.0.15:6809/18192
  1535. -904> 2012-12-18 19:54:57.361893 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.24 10.1.0.15:6809/18192 -- osd_map(9326..9426 src has 7326..10022) v3 -- ?+0 0x6092800
  1536. -903> 2012-12-18 19:54:57.361942 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.24 10.1.0.15:6809/18192 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19feb60
  1537. -902> 2012-12-18 19:54:57.361987 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 38376436)
  1538. -901> 2012-12-18 19:54:57.361993 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1539. -900> 2012-12-18 19:54:57.363633 7f9774176700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 296930 (38376436 -> 38673366)
  1540. -899> 2012-12-18 19:54:57.363643 7f9774176700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 296930 (0 -> 296930)
  1541. -898> 2012-12-18 19:54:57.363790 7f977ad79700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3935 (38673366 -> 38677301)
  1542. -897> 2012-12-18 19:54:57.363802 7f977ad79700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3935 (296930 -> 300865)
  1543. -896> 2012-12-18 19:54:57.364028 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.56 10.1.0.16:6819/18681 12 ==== osd_pgtemp(e10022 {1.209=[47,56,74],1.47d=[56,38,17],18.116a=[56,38,53],18.2721=[56,69,15],18.278a=[],18.293d=[22,56,70],18.2b31=[56,66,70],18.2ff9=[21,56,49],18.30a4=[56,34,24],18.3319=[],18.33d6=[56,40,66],18.47f2=[12,56,49],18.504a=[56,45,17],18.50a0=[56,72,15],18.56df=[56,16,18],18.6383=[56,40,22],18.6775=[56,40,75],18.6912=[56,69,10],18.6aac=[16,56,65],18.6cb9=[22,56,67],18.72fa=[56,65,19],18.79e2=[56,38,51],18.7f22=[56,75,17],18.84df=[45,56,63],18.8794=[56,48,17],18.879c=[56,65,22],18.8902=[56,29,21],18.8be9=[56,34,71],18.8dd1=[51,56,72],18.900c=[19,56,47],18.9338=[56,74,60],18.9e6b=[51,56,62],18.a297=[56,34,53],18.a622=[],18.b654=[56,41,22],18.b93f=[12,56,10],18.bae9=[41,56,68],18.baff=[56,43,53],18.bc66=[56,24,60],18.bc76=[56,20,27],18.bd8c=[],18.cac9=[44,56,59],18.d1bb=[38,56,67],18.eac9=[56,64,60],18.efde=[56,40,75],18.f01a=[56,48,15],18.f20e=[56,33,29],18.f543=[56,49,71],18.f704=[56,43,69],18.f715=[30,56,67],18.1263d=[26,56,71],18.1270c=[56,40,52],18.127ab=[56,60,11],18.12a89=[56,29,13],18.132c7=[],18.140c3=[56,65,26],18.144c8=[34,56,46],18.14b3f=[20,56,71],18.15110=[21,56,69],18.15204=[37,56,75],18.1571e=[48,56,67],18.15dd1=[41,56,74],18.16364=[56,41,28],18.16bc2=[56,70,26],18.17d8f=[25,56,34],18.189d6=[20,56,60],18.19318=[56,67,60],18.19ed2=[49,56,69],18.1a0cd=[56,21,63],18.1a435=[30,56,49],18.1aa4a=[56,62,64],18.1b189=[],18.1bb21=[38,56,67],18.1bf81=[56,41,64],18.1c57c=[29,56,38],18.1c5d6=[56,47,67],18.1c929=[66,56,73],18.1cccc=[56,35,39,68],18.1cd63=[37,56,60],18.1cdd6=[56,70,64],18.1cea8=[56,27,66],18.1db01=[56,70,52],18.1e1f4=[14,56,54],18.1e8e8=[40,56,68],18.1ec13=[12,56,60],18.1f194=[14,56,67],18.2162e=[56,44,72],18.22069=[56,74,62],18.238d7=[56,30,28],18.23d1c=[56,44,29],18.23e0e=[56,38,14],18.2440c=[16,56,48],18.245a3=[56,44,63],18.24db8=[56,62,54],18.25202=[56,69,10],18.25820=[56,66,12],18.25988=[56,33,21],18.259e6=[56,34,68],18.25c44=[33,56,66],18.26243=[56,74,29],18.26284=[56,38,55],18.2658f=[27,56,38],18.26781=[56,46,17],18.2761d=[],18.286ce=[11,56,68],18.29f5b=[42,56,61],18.2ab47=[56,38,44,72],18.2acdb=[],18.2b4cc=[56,37,42],18.2c422=[56,33,55],18.2c48b=[56,30,74],18.2c888=[56,37,49],18.2c97e=[56,27,15],18.2cdfa=[56,37,20],18.2e09e=[56,43,18],18.2e778=[56,28,67],18.2efe5=[56,45,65],18.2f1d7=[28,56,75],18.2fee0=[56,49,70],18.308a5=[56,40,52],18.30cdd=[56,35,39,70]} v10022) v1 ==== 3935+0+0 (2996072597 0 0) 0xcb61a3c0 con 0x7212dc0
  1544. -895> 2012-12-18 19:54:57.364163 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.364167 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1545. -894> 2012-12-18 19:54:57.364185 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3935 (0x8e4928 -> 296930)
  1546. -893> 2012-12-18 19:54:57.371732 7f9770346700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccf400 sd=47 pgs=0 cs=0 l=0).accept sd=47
  1547. -892> 2012-12-18 19:54:57.386961 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (38677301 -> 38677358)
  1548. -891> 2012-12-18 19:54:57.386993 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (296930 -> 296987)
  1549. -890> 2012-12-18 19:54:57.387155 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (864874821 0 0) 0x9cc6c00 con 0x9cd0780
  1550. -889> 2012-12-18 19:54:57.387183 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.387185 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1551. -888> 2012-12-18 19:54:57.387225 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6809/8019 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x6092200 con 0x9cd0780
  1552. -887> 2012-12-18 19:54:57.387276 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 38677301)
  1553. -886> 2012-12-18 19:54:57.387281 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 296930)
  1554. -885> 2012-12-18 19:54:57.403078 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.24 10.1.0.15:6809/18192 14 ==== log(1000 entries) v1 ==== 296930+0+0 (4188725370 0 0) 0xc395600 con 0x72123c0
  1555. -884> 2012-12-18 19:54:57.403105 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.403107 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1556. -883> 2012-12-18 19:54:57.405315 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 296930 (0x8e4928 -> 0)
  1557. -882> 2012-12-18 19:54:57.442990 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (38677301 -> 38677459)
  1558. -881> 2012-12-18 19:54:57.443043 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  1559. -880> 2012-12-18 19:54:57.443192 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3305459279 0 0) 0xc394000 con 0x9cd0780
  1560. -879> 2012-12-18 19:54:57.443217 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.443219 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1561. -878> 2012-12-18 19:54:57.443408 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6809/8019 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6c00 con 0x9cd0780
  1562. -877> 2012-12-18 19:54:57.443453 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 38677301)
  1563. -876> 2012-12-18 19:54:57.443459 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  1564. -875> 2012-12-18 19:54:57.449446 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 316631 (38677301 -> 38993932)
  1565. -874> 2012-12-18 19:54:57.449462 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 316631 (0 -> 316631)
  1566. -873> 2012-12-18 19:54:57.452716 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (38993932 -> 38993974)
  1567. -872> 2012-12-18 19:54:57.452730 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (316631 -> 316673)
  1568. -871> 2012-12-18 19:54:57.452893 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 3 ==== mon_subscribe({monmap=2+,osdmap=10009}) v2 ==== 42+0+0 (3041509029 0 0) 0x74ecc40 con 0x9cd0780
  1569. -870> 2012-12-18 19:54:57.452924 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10009..10022] to osd.13 10.1.0.13:6809/8019
  1570. -869> 2012-12-18 19:54:57.453068 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.13 10.1.0.13:6809/8019 -- osd_map(10009..10022 src has 7326..10022) v3 -- ?+0 0xc394000
  1571. -868> 2012-12-18 19:54:57.453178 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.13 10.1.0.13:6809/8019 -- mon_subscribe_ack(300s) v1 -- ?+0 0x19fe4e0
  1572. -867> 2012-12-18 19:54:57.453191 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 38993932)
  1573. -866> 2012-12-18 19:54:57.453196 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 316631)
  1574. -865> 2012-12-18 19:54:57.453506 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 903 (38993932 -> 38994835)
  1575. -864> 2012-12-18 19:54:57.453516 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 903 (316631 -> 317534)
  1576. -863> 2012-12-18 19:54:57.453674 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 8 ==== log(1000 entries) v1 ==== 316631+0+0 (645509550 0 0) 0x6093000 con 0x9cd0a00
  1577. -862> 2012-12-18 19:54:57.453676 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (38994835 -> 38994867)
  1578. -861> 2012-12-18 19:54:57.453694 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.453695 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1579. -860> 2012-12-18 19:54:57.453723 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (317534 -> 317566)
  1580. -859> 2012-12-18 19:54:57.453728 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (38994867 -> 38994889)
  1581. -858> 2012-12-18 19:54:57.453736 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (317566 -> 317588)
  1582. -857> 2012-12-18 19:54:57.453816 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 903 (38994889 -> 38995792)
  1583. -856> 2012-12-18 19:54:57.453857 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 903 (317588 -> 318491)
  1584. -855> 2012-12-18 19:54:57.455924 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 316631 (0x8e4928 -> 1860)
  1585. -854> 2012-12-18 19:54:57.455933 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 4 ==== osd_pgtemp(e10022 {18.583=[13,57,54],18.37b9=[],18.599d=[13,40,10],18.909e=[13,33,21],18.a305=[13,32,24],18.c608=[13,45,25],18.d712=[13,30,75],18.107db=[13,40,10],18.10cfc=[13,59,75],18.11d29=[13,72,10],18.13a45=[],18.15340=[],18.16751=[13,58,56],18.17ab3=[],18.18967=[13,38,69],18.1d9bf=[13,29,10],18.1e457=[],18.1e8cf=[],18.1ea76=[13,40,61],18.22351=[],18.23ff0=[13,37,38],18.2400c=[13,43,30],18.26c3b=[13,30,71],18.27538=[13,38,10],18.2790b=[13,34,32],18.2896d=[13,29,10],18.2994c=[13,47,60,40],18.2b4eb=[13,69,68],18.2f6bb=[13,56,62]} v10022) v1 ==== 903+0+0 (2521837416 0 0) 0x9cdd680 con 0x9cd0780
  1586. -853> 2012-12-18 19:54:57.455970 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.455971 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1587. -852> 2012-12-18 19:54:57.455984 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 903 (0x8e4928 -> 957)
  1588. -851> 2012-12-18 19:54:57.455988 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 9 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0x6092200 con 0x9cd0a00
  1589. -850> 2012-12-18 19:54:57.455999 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.456000 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1590. -849> 2012-12-18 19:54:57.456053 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.16:6825/20566 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xc394200 con 0x9cd0a00
  1591. -848> 2012-12-18 19:54:57.456115 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 38995760)
  1592. -847> 2012-12-18 19:54:57.456120 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 925)
  1593. -846> 2012-12-18 19:54:57.456123 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 5 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x19ffba0 con 0x9cd0780
  1594. -845> 2012-12-18 19:54:57.456157 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.456158 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1595. -844> 2012-12-18 19:54:57.456174 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 903)
  1596. -843> 2012-12-18 19:54:57.456178 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 6 ==== osd_pgtemp(e10022 {18.583=[13,57,54],18.37b9=[],18.599d=[13,40,10],18.909e=[13,33,21],18.a305=[13,32,24],18.c608=[13,45,25],18.d712=[13,30,75],18.107db=[13,40,10],18.10cfc=[13,59,75],18.11d29=[13,72,10],18.13a45=[],18.15340=[],18.16751=[13,58,56],18.17ab3=[],18.18967=[13,38,69],18.1d9bf=[13,29,10],18.1e457=[],18.1e8cf=[],18.1ea76=[13,40,61],18.22351=[],18.23ff0=[13,37,38],18.2400c=[13,43,30],18.26c3b=[13,30,71],18.27538=[13,38,10],18.2790b=[13,34,32],18.2896d=[13,29,10],18.2994c=[13,47,60,40],18.2b4eb=[13,69,68],18.2f6bb=[13,56,62]} v10022) v1 ==== 903+0+0 (2521837416 0 0) 0xcb61ab40 con 0x9cd0780
  1597. -842> 2012-12-18 19:54:57.456213 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.456214 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1598. -841> 2012-12-18 19:54:57.456224 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 903 (0x8e4928 -> 0)
  1599. -840> 2012-12-18 19:54:57.456964 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 562860 (38995760 -> 39558620)
  1600. -839> 2012-12-18 19:54:57.456975 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 562860 (0 -> 562860)
  1601. -838> 2012-12-18 19:54:57.459899 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (39558620 -> 39558662)
  1602. -837> 2012-12-18 19:54:57.459911 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (562860 -> 562902)
  1603. -836> 2012-12-18 19:54:57.460076 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 11 ==== mon_subscribe({monmap=2+,osdmap=9370}) v2 ==== 42+0+0 (3519688718 0 0) 0xcac7f6c0 con 0x7212140
  1604. -835> 2012-12-18 19:54:57.460106 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9370..10022] to osd.47 10.1.0.14:6822/19203
  1605. -834> 2012-12-18 19:54:57.460969 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.47 10.1.0.14:6822/19203 -- osd_map(9370..9470 src has 7326..10022) v3 -- ?+0 0x6092200
  1606. -833> 2012-12-18 19:54:57.461014 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.47 10.1.0.14:6822/19203 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31ba0
  1607. -832> 2012-12-18 19:54:57.461065 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 39558620)
  1608. -831> 2012-12-18 19:54:57.461070 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 562860)
  1609. -830> 2012-12-18 19:54:57.461575 7f9770b4e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 321287 (39558620 -> 39879907)
  1610. -829> 2012-12-18 19:54:57.461595 7f9770b4e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 321287 (562860 -> 884147)
  1611. -828> 2012-12-18 19:54:57.468808 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.47 10.1.0.14:6822/19203 12 ==== log(1000 entries) v1 ==== 321287+0+0 (3879539262 0 0) 0xc394a00 con 0x7212140
  1612. -827> 2012-12-18 19:54:57.468832 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.468834 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1613. -826> 2012-12-18 19:54:57.471066 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 321287 (0x8e4928 -> 562860)
  1614. -825> 2012-12-18 19:54:57.471636 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 9432 (39879907 -> 39889339)
  1615. -824> 2012-12-18 19:54:57.471694 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 9432 (562860 -> 572292)
  1616. -823> 2012-12-18 19:54:57.471694 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 7 ==== pg_stats(1766 pgs tid 584544 v 10022) v1 ==== 562860+0+0 (778506444 0 0) 0x5f33d40 con 0x9cd0780
  1617. -822> 2012-12-18 19:54:57.471717 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.471719 lease_expire=2012-12-18 19:55:01.683842 has v10022 lc 6066306
  1618. -821> 2012-12-18 19:54:57.471741 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 562860 (0x8e4928 -> 9432)
  1619. -820> 2012-12-18 19:54:57.471873 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 8 ==== log(37 entries) v1 ==== 9432+0+0 (2173054603 0 0) 0x632b400 con 0x9cd0780
  1620. -819> 2012-12-18 19:54:57.471892 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.471894 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1621. -818> 2012-12-18 19:54:57.471993 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 9432 (0x8e4928 -> 0)
  1622. -817> 2012-12-18 19:54:57.517103 7f977356a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3257 (39889339 -> 39892596)
  1623. -816> 2012-12-18 19:54:57.517136 7f977356a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3257 (0 -> 3257)
  1624. -815> 2012-12-18 19:54:57.517541 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.53 10.1.0.16:6810/16733 11 ==== osd_pgtemp(e10022 {0.457=[40,53,69],0.7bc=[10,53,20],1.456=[40,53,69],1.7bb=[10,53,20],18.445=[53,40,69],18.7aa=[53,10,20],18.1967=[53,41,28],18.1dbb=[53,34,60],18.1efd=[53,42,73],18.22f8=[53,73,61],18.2949=[53,75,16],18.4581=[],18.6418=[53,11,68],18.6c25=[41,53,56],18.70b6=[53,24,27],18.7722=[11,53,21],18.784d=[53,46,16],18.871d=[58,53,65],18.8bab=[53,42,73],18.a5aa=[53,34,24],18.aebe=[53,21,17],18.b6da=[14,53,51],18.b7f2=[53,46,63],18.b974=[44,53,66],18.c5be=[53,46,10],18.c9c2=[53,49,32],18.dc63=[48,53,62],18.fe59=[18,53,66],18.100ff=[53,30,64],18.10f3c=[53,43,11,61],18.11149=[],18.124b8=[59,53,70],18.12ea7=[27,53,40],18.1306e=[33,53,50],18.135c8=[44,53,70],18.138f9=[38,53,60],18.13a33=[44,53,60],18.13be2=[34,53,63],18.13d3b=[51,53,69],18.1476b=[53,34,61],18.14a6b=[53,20,54],18.14c59=[53,31,33,70],18.14c74=[32,53,75],18.15adf=[12,53,72],18.16171=[33,53,42],18.169a9=[29,53,65],18.16d03=[53,60,64],18.16d23=[],18.170ac=[32,53,42],18.17501=[53,48,75],18.17658=[24,53,72],18.1779c=[14,53,48],18.191ae=[10,53,33],18.1ab69=[55,53,66],18.1ad37=[53,45,19],18.1b0bd=[65,53,69],18.1b3dd=[67,53,74],18.1b465=[16,53,40],18.1ba80=[60,53,74],18.1c5cc=[18,53,75],18.1d82a=[53,44,61],18.1db52=[13,53,73],18.1dc90=[29,53,68],18.1e0c6=[12,53,25],18.1e84d=[34,53,52],18.1fc96=[40,53,66],18.1feee=[53,74,65],18.21457=[16,53,72],18.215bb=[54,53,66],18.22176=[53,69,67],18.2232c=[53,44,42],18.24a02=[53,43,50],18.253fa=[26,53,71],18.255ad=[53,68,69],18.257c4=[16,53,63],18.25ee0=[53,44,60],18.25f28=[53,41,66],18.25fc7=[28,53,55,69],18.2605d=[],18.283a8=[30,53,69],18.28dda=[53,64,15],18.29198=[14,53,64],18.29200=[15,53,48],18.292e9=[53,38,45],18.299dc=[53,32,58],18.29cfe=[53,75,14],18.2a282=[53,44,19],18.2a2f6=[53,32,64],18.2a99d=[53,38,44],18.2ab20=[53,32,69],18.2ab58=[53,32,73],18.2c9e2=[53,71,64],18.2cbb0=[53,64,21],18.2d0bb=[53,43,42],18.2e553=[10,53,55],18.2f341=[53,62,61],18.2f9b1=[53,60,10],18.2fcce=[53,67,18],18.307a3=[53,38,44]} v10022) v1 ==== 3257+0+0 (3917270056 0 0) 0x7b6cb40 con 0x7213e00
  1625. -814> 2012-12-18 19:54:57.517661 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.517663 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1626. -813> 2012-12-18 19:54:57.517681 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3257 (0x8e4928 -> 0)
  1627. -812> 2012-12-18 19:54:57.591145 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 931 (39892596 -> 39893527)
  1628. -811> 2012-12-18 19:54:57.591180 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 931 (0 -> 931)
  1629. -810> 2012-12-18 19:54:57.591316 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 11 ==== osd_pgtemp(e10022 {18.13e3=[72,37,22],18.181c=[72,57,22],18.3af3=[72,37,29],18.4325=[72,49,60],18.4aba=[72,57,67],18.6ed2=[72,46,25],18.7042=[],18.7683=[72,59,24],18.8737=[72,50,28],18.9599=[72,50,66],18.9c4f=[72,58,44],18.e7b1=[72,51,63],18.ed3e=[72,54,65],18.1058c=[72,52,12],18.1226e=[],18.143b1=[72,56,61],18.1bada=[72,31,53,3],18.1e19a=[],18.1e623=[72,57,62],18.1e8f3=[72,46,11],18.23643=[72,53,44],18.249d9=[72,47,56],18.259b8=[72,33,66],18.2a29a=[],18.2b05a=[72,51,35,29],18.2d1e9=[],18.2d83f=[72,44,65],18.2e322=[72,40,24],18.2f47f=[72,32,33]} v10022) v1 ==== 931+0+0 (563904079 0 0) 0x9cdd2c0 con 0x722c3c0
  1630. -809> 2012-12-18 19:54:57.591371 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.591373 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1631. -808> 2012-12-18 19:54:57.591390 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 931 (0x8e4928 -> 0)
  1632. -807> 2012-12-18 19:54:57.592084 7f9770144700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccf200 sd=49 pgs=0 cs=0 l=0).accept sd=49
  1633. -806> 2012-12-18 19:54:57.594960 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (39893527 -> 39893584)
  1634. -805> 2012-12-18 19:54:57.594976 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1635. -804> 2012-12-18 19:54:57.595141 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (606640603 0 0) 0xc394e00 con 0x9cd03c0
  1636. -803> 2012-12-18 19:54:57.595168 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.595170 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1637. -802> 2012-12-18 19:54:57.595209 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6821/10533 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193de00 con 0x9cd03c0
  1638. -801> 2012-12-18 19:54:57.595226 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 39893527)
  1639. -800> 2012-12-18 19:54:57.595277 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1640. -799> 2012-12-18 19:54:57.603157 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (39893527 -> 39893685)
  1641. -798> 2012-12-18 19:54:57.603180 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  1642. -797> 2012-12-18 19:54:57.603343 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1164047740 0 0) 0xc395000 con 0x9cd03c0
  1643. -796> 2012-12-18 19:54:57.603368 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.603369 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1644. -795> 2012-12-18 19:54:57.603567 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6821/10533 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xc394e00 con 0x9cd03c0
  1645. -794> 2012-12-18 19:54:57.603632 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 39893527)
  1646. -793> 2012-12-18 19:54:57.603638 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  1647. -792> 2012-12-18 19:54:57.610809 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (39893527 -> 39893569)
  1648. -791> 2012-12-18 19:54:57.610828 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1649. -790> 2012-12-18 19:54:57.611029 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (39893569 -> 39893591)
  1650. -789> 2012-12-18 19:54:57.611089 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  1651. -788> 2012-12-18 19:54:57.611039 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 3 ==== mon_subscribe({monmap=2+,osdmap=9281}) v2 ==== 42+0+0 (1924620553 0 0) 0xcc0d4000 con 0x9cd03c0
  1652. -787> 2012-12-18 19:54:57.611132 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 825 (39893591 -> 39894416)
  1653. -786> 2012-12-18 19:54:57.611135 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 825 (64 -> 889)
  1654. -785> 2012-12-18 19:54:57.611141 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9281..10022] to osd.17 10.1.0.13:6821/10533
  1655. -784> 2012-12-18 19:54:57.612375 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.17 10.1.0.13:6821/10533 -- osd_map(9281..9381 src has 7326..10022) v3 -- ?+0 0xc395000
  1656. -783> 2012-12-18 19:54:57.612442 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.17 10.1.0.13:6821/10533 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31a00
  1657. -782> 2012-12-18 19:54:57.612450 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 39894374)
  1658. -781> 2012-12-18 19:54:57.612456 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 847)
  1659. -780> 2012-12-18 19:54:57.612494 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x19f7ba0 con 0x9cd03c0
  1660. -779> 2012-12-18 19:54:57.612504 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.612505 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1661. -778> 2012-12-18 19:54:57.612520 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 825)
  1662. -777> 2012-12-18 19:54:57.612525 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 5 ==== osd_pgtemp(e10022 {0.de4=[17,29,10],18.dd2=[17,29,10],18.13b4=[17,56,57],18.2bdb=[17,58,25],18.abbc=[17,52,10],18.b163=[17,57,10],18.d4f3=[17,44,66],18.e6cb=[17,31,48,2],18.e7fa=[],18.f509=[17,38,43],18.fc0b=[17,57,70],18.12395=[],18.1695c=[17,37,32],18.16fb2=[17,40,10],18.17ac6=[],18.17d3a=[],18.1924c=[],18.1a62d=[],18.202ed=[17,30,73],18.21512=[],18.23f1b=[17,54,72],18.25d2b=[17,40,10],18.26a76=[17,29,10],18.278af=[17,52,55],18.29906=[17,46,10],18.2fb57=[17,43,68],18.30361=[]} v10022) v1 ==== 825+0+0 (3665041218 0 0) 0x7b72d20 con 0x9cd03c0
  1663. -776> 2012-12-18 19:54:57.612564 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.612565 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1664. -775> 2012-12-18 19:54:57.612578 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 825 (0x8e4928 -> 0)
  1665. -774> 2012-12-18 19:54:57.619171 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 584100 (39894374 -> 40478474)
  1666. -773> 2012-12-18 19:54:57.619191 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 584100 (0 -> 584100)
  1667. -772> 2012-12-18 19:54:57.623179 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4258 (40478474 -> 40482732)
  1668. -771> 2012-12-18 19:54:57.623206 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4258 (584100 -> 588358)
  1669. -770> 2012-12-18 19:54:57.623444 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 8 ==== osd_pgtemp(e10022 {0.96=[52,54,61],0.693=[62,54,68],1.95=[52,54,61],1.4b1=[54,75,27],1.692=[62,54,68],1.acb=[41,54,55],18.84=[52,54,61],18.2e5=[15,54,75],18.4a0=[54,75,27],18.5a8=[54,47,10],18.aba=[41,54,55],18.1de0=[15,54,27],18.2558=[54,71,19],18.3667=[54,43,73],18.391c=[59,54,67],18.472b=[54,75,28],18.4dc1=[54,64,60],18.4eb8=[54,68,58],18.535a=[54,61,18],18.59aa=[54,34,10],18.5d2a=[54,46,28],18.6439=[54,49,60],18.685e=[54,68,73],18.74fa=[54,70,75],18.788b=[54,61,72],18.7e80=[54,44,22],18.827d=[54,61,71],18.8fbf=[54,74,64],18.951d=[19,54,37],18.99a9=[54,37,72],18.a119=[54,16,19],18.a1ae=[14,54,60],18.a1db=[54,74,67],18.a714=[42,71,54,66],18.a726=[54,30,16],18.b4d1=[54,44,72],18.c60d=[54,37,74],18.c757=[54,44,61],18.cca2=[16,54,45],18.d14a=[54,47,59],18.d155=[54,69,65],18.de00=[54,33,56],18.e546=[17,54,53],18.e590=[54,49,40],18.e5af=[54,35,47,61],18.e66b=[54,30,29],18.efd3=[54,48,29],18.fd73=[54,34,75],18.fe91=[54,46,20],18.ffef=[54,37,44,75],18.12526=[54,73,21],18.12756=[55,54,75],18.1277d=[32,54,60],18.1287b=[37,54,72],18.13148=[54,41,64],18.13942=[50,54,62],18.1398e=[54,42,44,69],18.13b29=[64,54,73],18.1422c=[54,47,18],18.14a98=[54,41,70],18.14d75=[30,54,73],18.14fb0=[33,54,58],18.15377=[54,70,60],18.1575b=[58,54,67],18.17023=[65,54,72],18.17509=[30,54,73],18.1760f=[33,54,64],18.176c9=[16,54,20],18.17744=[47,54,67],18.17c26=[],18.182e0=[12,54,66],18.1841c=[54,65,74],18.189f9=[54,34,72],18.18a2b=[44,54,55],18.18a3f=[54,21,25,74],18.19c95=[44,54,75],18.19d58=[53,54,70],18.1a8b1=[17,54,43],18.1abf2=[54,44,66],18.1b30b=[22,54,51],18.1b64e=[54,26,14],18.1c106=[40,54,75],18.1c34c=[34,54,42],18.1c533=[54,61,24],18.1d57c=[54,73,64],18.1d672=[29,54,67],18.1e9ab=[43,54,66],18.1fc03=[19,54,58],18.1fc4f=[41,54,73],18.214de=[54,70,75],18.21dd4=[54,68,71],18.21de8=[54,33,11],18.22b5f=[54,38,74],18.2362b=[54,33,72],18.23f3d=[54,46,72],18.2406c=[54,41,34],18.24f1c=[50,54,62],18.25d11=[54,32,74],18.263e5=[],18.26723=[54,75,71],18.26f2e=[29,54,72],18.27105=[54,42,69],18.283a9=[54,75,67],18.28a7d=[54,24,66],18.28c29=[32,54,41],18.28daa=[54,37,50],18.2952e=[54,46,21],18.2960b=[54,43,61],18.29842=[54,67,17],18.299d9=[54,46,11],18.29de9=[11,54,73],18.2a091=[54,45,60,22],18.2a09e=[32,54,63],18.2ae8c=[54,43,60],18.2ba8a=[54,26,21],18.2bcb6=[34,54,46],18.2bcf0=[54,47,62],18.2c10a=[21,54,58,66],18.2c3ce=[54,42,66],18.2c6e5=[54,43,29],18.2c80c=[54,69,13],18.2ca54=[32,54,53,73],18.2ccbf=[54,49,69],18.2d930=[54,13,74],18.2da0e=[54,34,21],18.2f506=[54,46,62],18.2f867=[54,29,63],18.30cc9=[54,70,60]} v10022) v1 ==== 4258+0+0 (4063220183 0 0) 0xcb61bc20 con 0x722d040
  1670. -769> 2012-12-18 19:54:57.623595 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.623597 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1671. -768> 2012-12-18 19:54:57.623615 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4258 (0x8e4928 -> 584100)
  1672. -767> 2012-12-18 19:54:57.637420 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 6 ==== pg_stats(1833 pgs tid 588430 v 10022) v1 ==== 584100+0+0 (1473292587 0 0) 0xca567200 con 0x9cd03c0
  1673. -766> 2012-12-18 19:54:57.637445 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.637447 lease_expire=2012-12-18 19:55:01.683842 has v10022 lc 6066306
  1674. -765> 2012-12-18 19:54:57.637465 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 584100 (0x8e4928 -> 0)
  1675. -764> 2012-12-18 19:54:57.637512 7f9770144700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 8038 (40482732 -> 40490770)
  1676. -763> 2012-12-18 19:54:57.637537 7f9770144700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 8038 (0 -> 8038)
  1677. -762> 2012-12-18 19:54:57.637662 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.17 10.1.0.13:6821/10533 7 ==== log(33 entries) v1 ==== 8038+0+0 (3225590023 0 0) 0xc395200 con 0x9cd03c0
  1678. -761> 2012-12-18 19:54:57.637682 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.637687 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1679. -760> 2012-12-18 19:54:57.637777 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 8038 (0x8e4928 -> 0)
  1680. -759> 2012-12-18 19:54:57.653080 7f9770548700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 704 (40490770 -> 40491474)
  1681. -758> 2012-12-18 19:54:57.653113 7f9770548700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 704 (0 -> 704)
  1682. -757> 2012-12-18 19:54:57.653298 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.68 10.1.0.18:6821/19143 12 ==== osd_pgtemp(e10022 {18.4072=[68,36,43,67],18.5e69=[],18.60c9=[68,40,19],18.9c9a=[68,50,44],18.a986=[],18.ac53=[68,52,67],18.bb43=[68,53,45],18.d106=[68,49,16],18.de16=[68,30,19],18.e405=[68,53,15],18.ef2a=[],18.f51c=[68,59,20],18.14f09=[68,44,52,47],18.16aaa=[],18.1d43d=[],18.1e3a6=[68,31,58,62],18.211d2=[68,50,25],18.22253=[68,57,43],18.22f2b=[68,32,60],18.251db=[68,54,67],18.2a0ae=[68,55,63],18.2e0d9=[68,37,49]} v10022) v1 ==== 704+0+0 (3941973155 0 0) 0x89934a0 con 0x7212780
  1683. -756> 2012-12-18 19:54:57.653345 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.653347 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1684. -755> 2012-12-18 19:54:57.653365 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 704 (0x8e4928 -> 0)
  1685. -754> 2012-12-18 19:54:57.662725 7f976ff42700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccf000 sd=50 pgs=0 cs=0 l=0).accept sd=50
  1686. -753> 2012-12-18 19:54:57.667374 7f976ff42700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (40491474 -> 40491531)
  1687. -752> 2012-12-18 19:54:57.667402 7f976ff42700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1688. -751> 2012-12-18 19:54:57.667565 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.64 10.1.0.17:6812/30162 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3681154154 0 0) 0x193cc00 con 0x9cd0000
  1689. -750> 2012-12-18 19:54:57.667592 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.667594 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1690. -749> 2012-12-18 19:54:57.667634 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6812/30162 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cb9a00 con 0x9cd0000
  1691. -748> 2012-12-18 19:54:57.667651 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 40491474)
  1692. -747> 2012-12-18 19:54:57.667703 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1693. -746> 2012-12-18 19:54:57.698141 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (40491474 -> 40491496)
  1694. -745> 2012-12-18 19:54:57.698171 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (0 -> 22)
  1695. -744> 2012-12-18 19:54:57.698338 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4258 (40491496 -> 40495754)
  1696. -743> 2012-12-18 19:54:57.698467 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4258 (22 -> 4280)
  1697. -742> 2012-12-18 19:54:57.698334 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 9 ==== osd_alive(want up_thru 10020 have 10022) v1 ==== 22+0+0 (4084230999 0 0) 0xcbf31d40 con 0x722d040
  1698. -741> 2012-12-18 19:54:57.698486 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.698489 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1699. -740> 2012-12-18 19:54:57.698514 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 4258)
  1700. -739> 2012-12-18 19:54:57.698674 7f9773166700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3677 (40495754 -> 40499431)
  1701. -738> 2012-12-18 19:54:57.698680 7f9773166700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3677 (4258 -> 7935)
  1702. -737> 2012-12-18 19:54:57.698667 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 10 ==== osd_pgtemp(e10022 {0.96=[52,54,61],0.693=[62,54,68],1.95=[52,54,61],1.4b1=[54,75,27],1.692=[62,54,68],1.acb=[41,54,55],18.84=[52,54,61],18.2e5=[15,54,75],18.4a0=[54,75,27],18.5a8=[54,47,10],18.aba=[41,54,55],18.1de0=[15,54,27],18.2558=[54,71,19],18.3667=[54,43,73],18.391c=[59,54,67],18.472b=[54,75,28],18.4dc1=[54,64,60],18.4eb8=[54,68,58],18.535a=[54,61,18],18.59aa=[54,34,10],18.5d2a=[54,46,28],18.6439=[54,49,60],18.685e=[54,68,73],18.74fa=[54,70,75],18.788b=[54,61,72],18.7e80=[54,44,22],18.827d=[54,61,71],18.8fbf=[54,74,64],18.951d=[19,54,37],18.99a9=[54,37,72],18.a119=[54,16,19],18.a1ae=[14,54,60],18.a1db=[54,74,67],18.a714=[42,71,54,66],18.a726=[54,30,16],18.b4d1=[54,44,72],18.c60d=[54,37,74],18.c757=[54,44,61],18.cca2=[16,54,45],18.d14a=[54,47,59],18.d155=[54,69,65],18.de00=[54,33,56],18.e546=[17,54,53],18.e590=[54,49,40],18.e5af=[54,35,47,61],18.e66b=[54,30,29],18.efd3=[54,48,29],18.fd73=[54,34,75],18.fe91=[54,46,20],18.ffef=[54,37,44,75],18.12526=[54,73,21],18.12756=[55,54,75],18.1277d=[32,54,60],18.1287b=[37,54,72],18.13148=[54,41,64],18.13942=[50,54,62],18.1398e=[54,42,44,69],18.13b29=[64,54,73],18.1422c=[54,47,18],18.14a98=[54,41,70],18.14d75=[30,54,73],18.14fb0=[33,54,58],18.15377=[54,70,60],18.1575b=[58,54,67],18.17023=[65,54,72],18.17509=[30,54,73],18.1760f=[33,54,64],18.176c9=[16,54,20],18.17744=[47,54,67],18.17c26=[],18.182e0=[12,54,66],18.1841c=[54,65,74],18.189f9=[54,34,72],18.18a2b=[44,54,55],18.18a3f=[54,21,25,74],18.19c95=[44,54,75],18.19d58=[53,54,70],18.1a8b1=[17,54,43],18.1abf2=[54,44,66],18.1b30b=[22,54,51],18.1b64e=[54,26,14],18.1c106=[40,54,75],18.1c34c=[34,54,42],18.1c533=[54,61,24],18.1d57c=[54,73,64],18.1d672=[29,54,67],18.1e9ab=[43,54,66],18.1fc03=[19,54,58],18.1fc4f=[41,54,73],18.214de=[54,70,75],18.21dd4=[54,68,71],18.21de8=[54,33,11],18.22b5f=[54,38,74],18.2362b=[54,33,72],18.23f3d=[54,46,72],18.2406c=[54,41,34],18.24f1c=[50,54,62],18.25d11=[54,32,74],18.263e5=[],18.26723=[54,75,71],18.26f2e=[29,54,72],18.27105=[54,42,69],18.283a9=[54,75,67],18.28a7d=[54,24,66],18.28c29=[32,54,41],18.28daa=[54,37,50],18.2952e=[54,46,21],18.2960b=[54,43,61],18.29842=[54,67,17],18.299d9=[54,46,11],18.29de9=[11,54,73],18.2a091=[54,45,60,22],18.2a09e=[32,54,63],18.2ae8c=[54,43,60],18.2ba8a=[54,26,21],18.2bcb6=[34,54,46],18.2bcf0=[54,47,62],18.2c10a=[21,54,58,66],18.2c3ce=[54,42,66],18.2c6e5=[54,43,29],18.2c80c=[54,69,13],18.2ca54=[32,54,53,73],18.2ccbf=[54,49,69],18.2d930=[54,13,74],18.2da0e=[54,34,21],18.2f506=[54,46,62],18.2f867=[54,29,63],18.30cc9=[54,70,60]} v10022) v1 ==== 4258+0+0 (4063220183 0 0) 0xcb61ba40 con 0x722d040
  1703. -736> 2012-12-18 19:54:57.698813 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.698814 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1704. -735> 2012-12-18 19:54:57.698835 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4258 (0x8e4928 -> 3677)
  1705. -734> 2012-12-18 19:54:57.698841 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.58 10.1.0.16:6825/20566 10 ==== osd_pgtemp(e10022 {0.ec=[42,58,74],0.43f=[58,38,51],0.873=[25,58,69],1.72a=[24,58,50],1.872=[25,58,69],18.da=[58,44,42],18.719=[24,58,50],18.e92=[58,26,75],18.1cee=[58,38,54],18.2045=[58,41,67],18.2a51=[58,74,65],18.2c40=[58,34,51],18.2e58=[58,43,73],18.2ff2=[58,73,67],18.330c=[58,49,69],18.371c=[58,40,25],18.4109=[58,27,21],18.4702=[52,58,69],18.489f=[58,40,48],18.4cb6=[20,58,40],18.54de=[58,32,17],18.5851=[58,71,17],18.5fbd=[58,15,56],18.6701=[58,67,66],18.687f=[58,32,48],18.6f00=[58,43,42],18.8160=[58,48,52],18.8349=[58,40,66],18.84d0=[58,71,24],18.8a29=[58,47,18],18.9088=[58,32,68],18.983b=[58,40,75],18.9c17=[58,32,10],18.a06d=[34,58,61],18.a2dd=[58,28,20],18.a309=[40,58,51],18.a70d=[58,40,68],18.afa3=[11,58,32],18.b61d=[27,58,65],18.c14e=[58,73,67],18.c7ee=[59,58,62],18.d0bd=[58,41,47,74],18.dc12=[58,33,61],18.e04c=[30,58,46],18.e2eb=[58,46,22],18.e5bb=[58,44,54],18.ef20=[58,48,53],18.f2de=[58,42,72],18.f5bd=[24,58,61],18.f9c6=[58,47,11],18.1179e=[22,58,72],18.1186b=[17,58,63],18.11ae3=[58,33,15],18.120f9=[59,58,60],18.12a94=[58,37,57],18.13a20=[58,45,48,68],18.13ad8=[58,40,49],18.13ef1=[10,58,69],18.14459=[58,29,10],18.14cd5=[58,30,53],18.14f91=[59,58,60],18.1544b=[53,58,63],18.158df=[54,58,64],18.15c7c=[58,47,18],18.15cc3=[25,58,62],18.16662=[58,44,71],18.17ead=[51,58,61],18.17fc0=[29,58,38],18.180b8=[25,58,65],18.18c96=[58,49,64,31],18.18dba=[58,74,65],18.199c5=[58,42,66],18.19dc6=[15,58,43],18.1bb18=[58,20,34],18.1c26d=[58,37,74],18.1c622=[18,58,47],18.1c7b3=[24,58,32],18.1cff4=[22,58,74],18.1d579=[12,58,38],18.1eb6a=[20,58,59],18.1eb79=[46,58,73],18.1f14f=[58,41,71],18.1f946=[58,45,43],18.1fd69=[46,58,61],18.218e0=[19,58,65],18.2330d=[58,20,59],18.23574=[58,41,10],18.237d9=[58,47,12],18.25369=[58,66,10],18.25632=[58,64,12],18.263be=[],18.26560=[58,47,22],18.2664d=[58,48,40],18.27318=[58,49,69],18.274e6=[54,58,74],18.27a67=[45,58,73],18.27e61=[58,45,66],18.29679=[58,29,24],18.29a35=[58,62,15],18.2a014=[58,38,62],18.2a93b=[58,71,67],18.2d490=[21,58,47],18.2d819=[58,61,12],18.2da26=[58,41,21],18.2dc95=[58,68,63],18.2e8a0=[30,58,62],18.2e9f8=[58,48,75],18.2ee0d=[],18.2fed7=[58,25,62],18.30be3=[58,43,22],18.30ccc=[33,58,74]} v10022) v1 ==== 3677+0+0 (471621799 0 0) 0xcb61be00 con 0x9cd0a00
  1706. -733> 2012-12-18 19:54:57.698950 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.698951 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1707. -732> 2012-12-18 19:54:57.698962 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3677 (0x8e4928 -> 0)
  1708. -731> 2012-12-18 19:54:57.722576 7f977457a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 63 (40499431 -> 40499494)
  1709. -730> 2012-12-18 19:54:57.722611 7f977457a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 63 (0 -> 63)
  1710. -729> 2012-12-18 19:54:57.722792 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.19 10.1.0.13:6827/13431 9 ==== osd_pgtemp(e10022 {18.19f3c=[19,39,21,8]} v10022) v1 ==== 63+0+0 (2926300024 0 0) 0xcc24a1e0 con 0x9cd0c80
  1711. -728> 2012-12-18 19:54:57.722820 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.722821 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1712. -727> 2012-12-18 19:54:57.722840 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.722842 lease_expire=2012-12-18 19:55:01.703181 has v0 lc 10022
  1713. -726> 2012-12-18 19:54:57.722851 7f977255a700 5 mon.a@0(leader).osd e10022 send_latest to osd.19 10.1.0.13:6827/13431 start 10022
  1714. -725> 2012-12-18 19:54:57.722857 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10022..10022] to osd.19 10.1.0.13:6827/13431
  1715. -724> 2012-12-18 19:54:57.722883 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.13:6827/13431 -- osd_map(10022..10022 src has 7326..10022) v3 -- ?+0 0x193cc00 con 0x9cd0c80
  1716. -723> 2012-12-18 19:54:57.722916 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 63 (0x8e4928 -> 40499431)
  1717. -722> 2012-12-18 19:54:57.722922 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 63 (0x8e4928 -> 0)
  1718. -721> 2012-12-18 19:54:57.729165 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1295320 (40499431 -> 41794751)
  1719. -720> 2012-12-18 19:54:57.729191 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1295320 (0 -> 1295320)
  1720. -719> 2012-12-18 19:54:57.748746 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 59 (41794751 -> 41794810)
  1721. -718> 2012-12-18 19:54:57.748778 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 59 (1295320 -> 1295379)
  1722. -717> 2012-12-18 19:54:57.748917 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 9 ==== osd_pgtemp(e10022 {18.2e68c=[11,38,24]} v10022) v1 ==== 59+0+0 (2724005182 0 0) 0xcc24a5a0 con 0x722db80
  1723. -716> 2012-12-18 19:54:57.748945 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.748947 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1724. -715> 2012-12-18 19:54:57.748966 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 59 (0x8e4928 -> 1295320)
  1725. -714> 2012-12-18 19:54:57.762326 7f976fd40700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccee00 sd=51 pgs=0 cs=0 l=0).accept sd=51
  1726. -713> 2012-12-18 19:54:57.763023 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (41794810 -> 41794867)
  1727. -712> 2012-12-18 19:54:57.763050 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (1295320 -> 1295377)
  1728. -711> 2012-12-18 19:54:57.763217 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (639752457 0 0) 0x193de00 con 0x7723b80
  1729. -710> 2012-12-18 19:54:57.763243 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.763245 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1730. -709> 2012-12-18 19:54:57.763285 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6821/30448 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc7000 con 0x7723b80
  1731. -708> 2012-12-18 19:54:57.763321 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 41794810)
  1732. -707> 2012-12-18 19:54:57.763326 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 1295320)
  1733. -706> 2012-12-18 19:54:57.767946 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 11 ==== pg_stats(4072 pgs tid 745165 v 10022) v1 ==== 1295320+0+0 (1224922545 0 0) 0xca567d40 con 0x722d040
  1734. -705> 2012-12-18 19:54:57.767971 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.767972 lease_expire=2012-12-18 19:55:01.683842 has v10022 lc 6066306
  1735. -704> 2012-12-18 19:54:57.767990 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1295320 (0x8e4928 -> 0)
  1736. -703> 2012-12-18 19:54:57.815420 7f976fb3e700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccec00 sd=52 pgs=0 cs=0 l=0).accept sd=52
  1737. -702> 2012-12-18 19:54:57.817514 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (41794810 -> 41794968)
  1738. -701> 2012-12-18 19:54:57.817543 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  1739. -700> 2012-12-18 19:54:57.817709 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2826371846 0 0) 0xc394000 con 0x7723b80
  1740. -699> 2012-12-18 19:54:57.817733 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.817735 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1741. -698> 2012-12-18 19:54:57.817939 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6821/30448 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x193de00 con 0x7723b80
  1742. -697> 2012-12-18 19:54:57.817999 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 41794810)
  1743. -696> 2012-12-18 19:54:57.818005 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  1744. -695> 2012-12-18 19:54:57.818384 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (41794810 -> 41794867)
  1745. -694> 2012-12-18 19:54:57.818394 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1746. -693> 2012-12-18 19:54:57.818502 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (41794867 -> 41794909)
  1747. -692> 2012-12-18 19:54:57.818511 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (57 -> 99)
  1748. -691> 2012-12-18 19:54:57.818598 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (41794909 -> 41794931)
  1749. -690> 2012-12-18 19:54:57.818727 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (99 -> 121)
  1750. -689> 2012-12-18 19:54:57.818638 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3041323802 0 0) 0x9cc7000 con 0xcc245cc0
  1751. -688> 2012-12-18 19:54:57.818796 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.818800 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1752. -687> 2012-12-18 19:54:57.818846 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6806/29971 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6000 con 0xcc245cc0
  1753. -686> 2012-12-18 19:54:57.818866 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 41794874)
  1754. -685> 2012-12-18 19:54:57.818871 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 64)
  1755. -684> 2012-12-18 19:54:57.818875 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 3 ==== mon_subscribe({monmap=2+,osdmap=9322}) v2 ==== 42+0+0 (964474889 0 0) 0xcc57a000 con 0x7723b80
  1756. -683> 2012-12-18 19:54:57.818893 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9322..10022] to osd.67 10.1.0.17:6821/30448
  1757. -682> 2012-12-18 19:54:57.819827 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.67 10.1.0.17:6821/30448 -- osd_map(9322..9422 src has 7326..10022) v3 -- ?+0 0x9cc7000
  1758. -681> 2012-12-18 19:54:57.819878 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 637 (41794874 -> 41795511)
  1759. -680> 2012-12-18 19:54:57.819885 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.67 10.1.0.17:6821/30448 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf316c0
  1760. -679> 2012-12-18 19:54:57.819893 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 41795469)
  1761. -678> 2012-12-18 19:54:57.819885 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 637 (64 -> 701)
  1762. -677> 2012-12-18 19:54:57.819898 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 659)
  1763. -676> 2012-12-18 19:54:57.819935 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xcc57c000 con 0x7723b80
  1764. -675> 2012-12-18 19:54:57.819944 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.819945 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1765. -674> 2012-12-18 19:54:57.819959 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 637)
  1766. -673> 2012-12-18 19:54:57.819962 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 5 ==== osd_pgtemp(e10022 {18.2d3f=[],18.3b80=[67,54,28],18.74e3=[67,43,46],18.78cc=[67,59,52],18.b38d=[67,40,71],18.bc0d=[],18.d32b=[67,47,19],18.f35d=[67,51,71],18.17cd7=[67,48,18],18.22d77=[67,38,53,26],18.260cd=[67,49,68],18.26d06=[67,37,17],18.275bd=[67,49,25],18.279ae=[67,49,75],18.2b2d0=[67,36,58,27],18.2bd82=[67,52,74],18.2c407=[67,50,13],18.2e6ab=[67,32,22],18.30165=[67,49,19]} v10022) v1 ==== 637+0+0 (1205839544 0 0) 0xcc24a780 con 0x7723b80
  1767. -672> 2012-12-18 19:54:57.820002 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.820003 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1768. -671> 2012-12-18 19:54:57.820015 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 637 (0x8e4928 -> 0)
  1769. -670> 2012-12-18 19:54:57.825713 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (41795469 -> 41795627)
  1770. -669> 2012-12-18 19:54:57.825732 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  1771. -668> 2012-12-18 19:54:57.825893 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (141847868 0 0) 0x6093200 con 0xcc245cc0
  1772. -667> 2012-12-18 19:54:57.825918 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:57.825919 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1773. -666> 2012-12-18 19:54:57.826129 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6806/29971 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6600 con 0xcc245cc0
  1774. -665> 2012-12-18 19:54:57.826192 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 41795469)
  1775. -664> 2012-12-18 19:54:57.826198 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  1776. -663> 2012-12-18 19:54:57.827161 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (41795469 -> 41795511)
  1777. -662> 2012-12-18 19:54:57.827172 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1778. -661> 2012-12-18 19:54:57.827291 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (41795511 -> 41795533)
  1779. -660> 2012-12-18 19:54:57.827361 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 3 ==== mon_subscribe({monmap=2+,osdmap=9287}) v2 ==== 42+0+0 (1105819995 0 0) 0xcc57a1c0 con 0xcc245cc0
  1780. -659> 2012-12-18 19:54:57.827383 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  1781. -658> 2012-12-18 19:54:57.827390 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9287..10022] to osd.62 10.1.0.17:6806/29971
  1782. -657> 2012-12-18 19:54:57.827398 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 744 (41795533 -> 41796277)
  1783. -656> 2012-12-18 19:54:57.827524 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 744 (64 -> 808)
  1784. -655> 2012-12-18 19:54:57.829516 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.62 10.1.0.17:6806/29971 -- osd_map(9287..9387 src has 7326..10022) v3 -- ?+0 0x6093200
  1785. -654> 2012-12-18 19:54:57.829563 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.62 10.1.0.17:6806/29971 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31520
  1786. -653> 2012-12-18 19:54:57.829609 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 41796235)
  1787. -652> 2012-12-18 19:54:57.829614 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 766)
  1788. -651> 2012-12-18 19:54:57.829618 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xcc57c1a0 con 0xcc245cc0
  1789. -650> 2012-12-18 19:54:57.829627 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.829628 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1790. -649> 2012-12-18 19:54:57.829643 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 744)
  1791. -648> 2012-12-18 19:54:57.829646 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 5 ==== osd_pgtemp(e10022 {18.9e0=[62,55,70],18.1f2e=[62,55,19],18.eb2a=[62,46,68],18.12e64=[62,38,40],18.14876=[],18.1496d=[62,47,73],18.159d1=[],18.19956=[62,36,59,71],18.1ab4f=[62,59,71],18.1bfc8=[62,36,22,74],18.1cc8e=[62,45,36,54],18.1d5e2=[62,40,13],18.1efd6=[62,36,56,42],18.212ff=[62,49,42],18.22c92=[62,53,19],18.2395f=[62,42,74],18.25280=[62,32,57],18.26127=[62,59,19],18.2912c=[62,54,20],18.2b703=[62,55,70],18.2c332=[62,53,70],18.2c696=[62,40,49]} v10022) v1 ==== 744+0+0 (1160722756 0 0) 0xcc24a960 con 0xcc245cc0
  1792. -647> 2012-12-18 19:54:57.829679 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.829679 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1793. -646> 2012-12-18 19:54:57.829694 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 744 (0x8e4928 -> 0)
  1794. -645> 2012-12-18 19:54:57.837158 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1053884 (41796235 -> 42850119)
  1795. -644> 2012-12-18 19:54:57.837184 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1053884 (0 -> 1053884)
  1796. -643> 2012-12-18 19:54:57.843320 7f9774f84700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 240 (42850119 -> 42850359)
  1797. -642> 2012-12-18 19:54:57.843344 7f9774f84700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 240 (1053884 -> 1054124)
  1798. -641> 2012-12-18 19:54:57.843468 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.14:6826/14574 4 ==== mdsbeacon(7503750/b up:replay seq 871 v2487) v2 ==== 240+0+0 (1474866746 0 0) 0x6086500 con 0x722d2c0
  1799. -640> 2012-12-18 19:54:57.843500 7f977255a700 1 mon.a@0(leader).paxos(mdsmap active c 1..2487) is_readable now=2012-12-18 19:54:57.843505 lease_expire=2012-12-18 19:55:01.702253 has v2487 lc 2487
  1800. -639> 2012-12-18 19:54:57.843555 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6826/14574 -- mdsbeacon(7503750/b up:replay seq 871 v2487) v2 -- ?+0 0x84ae500 con 0x722d2c0
  1801. -638> 2012-12-18 19:54:57.843572 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 240 (0x8e4928 -> 42850119)
  1802. -637> 2012-12-18 19:54:57.843579 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 240 (0x8e4928 -> 1053884)
  1803. -636> 2012-12-18 19:54:57.848672 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1070096 (42850119 -> 43920215)
  1804. -635> 2012-12-18 19:54:57.848688 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1070096 (1053884 -> 2123980)
  1805. -634> 2012-12-18 19:54:57.864301 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 6 ==== pg_stats(3312 pgs tid 17650 v 10022) v1 ==== 1053884+0+0 (3000544185 0 0) 0xca5678c0 con 0x7723b80
  1806. -633> 2012-12-18 19:54:57.864325 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.864327 lease_expire=2012-12-18 19:55:01.683842 has v10022 lc 6066306
  1807. -632> 2012-12-18 19:54:57.864345 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1053884 (0x8e4928 -> 1070096)
  1808. -631> 2012-12-18 19:54:57.866826 7f976fd40700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 300239 (43920215 -> 44220454)
  1809. -630> 2012-12-18 19:54:57.866853 7f976fd40700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 300239 (1070096 -> 1370335)
  1810. -629> 2012-12-18 19:54:57.870939 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.67 10.1.0.17:6821/30448 7 ==== log(1000 entries) v1 ==== 300239+0+0 (2992251622 0 0) 0xc394e00 con 0x7723b80
  1811. -628> 2012-12-18 19:54:57.870964 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.870965 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1812. -627> 2012-12-18 19:54:57.873176 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 300239 (0x8e4928 -> 1070096)
  1813. -626> 2012-12-18 19:54:57.876419 7f976fb3e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 764 (44220454 -> 44221218)
  1814. -625> 2012-12-18 19:54:57.876444 7f976fb3e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 764 (1070096 -> 1070860)
  1815. -624> 2012-12-18 19:54:57.876432 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 6 ==== pg_stats(3363 pgs tid 17771 v 10022) v1 ==== 1070096+0+0 (516645426 0 0) 0xca567b00 con 0xcc245cc0
  1816. -623> 2012-12-18 19:54:57.876453 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066306) is_readable now=2012-12-18 19:54:57.876455 lease_expire=2012-12-18 19:55:01.683842 has v10022 lc 6066306
  1817. -622> 2012-12-18 19:54:57.876473 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1070096 (0x8e4928 -> 764)
  1818. -621> 2012-12-18 19:54:57.876576 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.62 10.1.0.17:6806/29971 7 ==== log(3 entries) v1 ==== 764+0+0 (2098778714 0 0) 0x193de00 con 0xcc245cc0
  1819. -620> 2012-12-18 19:54:57.876601 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.876603 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1820. -619> 2012-12-18 19:54:57.876628 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 764 (0x8e4928 -> 0)
  1821. -618> 2012-12-18 19:54:57.888586 7f977275c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 4461 (44221218 -> 44225679)
  1822. -617> 2012-12-18 19:54:57.888615 7f977275c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 4461 (0 -> 4461)
  1823. -616> 2012-12-18 19:54:57.888818 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.44 10.1.0.14:6813/17486 9 ==== osd_pgtemp(e10022 {0.9ad=[44,52,29],1.2bd=[27,44,43],1.c53=[44,36,58,71],1.cd8=[48,44,71],18.116=[44,34,63],18.2ac=[44,55,27],18.cc7=[48,44,71],18.1903=[44,52,11],18.1cd3=[44,60,69],18.1da8=[44,57,13],18.26c1=[12,44,10],18.2e67=[42,44,66],18.2f9e=[45,44,58],18.30be=[44,56,24],18.31ab=[44,65,11],18.3279=[44,54,64],18.340b=[44,69,22],18.391d=[27,44,61],18.53df=[44,25,66],18.56a0=[10,44,74],18.5b4c=[44,68,66],18.5d8a=[44,51,70],18.5e08=[44,58,36,21],18.60fd=[44,54,18],18.6945=[44,58,65],18.6c72=[44,56,68],18.6f70=[42,44,70],18.737c=[44,53,41],18.779f=[44,26,12],18.7aab=[24,44,75],18.7ae1=[44,58,68],18.7cc0=[44,56,74],18.84c5=[44,11,19],18.87f0=[25,44,34],18.8912=[44,53,13],18.8dbd=[44,62,43],18.8e00=[44,51,18],18.9c26=[25,44,40],18.9d02=[44,53,55],18.a1de=[34,44,64],18.a723=[44,28,25],18.b085=[44,51,61],18.bd36=[44,62,11],18.c7e3=[21,44,52],18.d07b=[44,71,19],18.daab=[44,60,20],18.dbaf=[25,44,62],18.dbb5=[46,44,63],18.e154=[],18.e93a=[44,58,65],18.ef2c=[33,44,70],18.f62e=[44,70,61],18.fd31=[51,44,65],18.1005c=[44,53,61],18.10303=[44,52,47],18.1034b=[65,44,68],18.11520=[52,44,65],18.1156b=[44,12,69],18.121a6=[44,64,69],18.1231a=[28,44,40],18.1234f=[44,53,27],18.12560=[19,44,21],18.12f1d=[17,44,60],18.1330a=[29,44,32],18.13755=[44,61,75],18.13b00=[47,44,66],18.13be0=[44,56,73],18.152fe=[14,44,58],18.15b75=[],18.1664b=[10,44,74],18.16899=[61,44,74],18.16ee5=[26,44,74],18.17413=[44,65,49],18.1766f=[44,33,66],18.17997=[44,64,37,60],18.17d68=[44,51,24],18.18344=[28,44,59],18.18628=[44,56,40],18.19131=[44,54,62],18.192e6=[20,44,30],18.19be1=[30,44,57],18.19f04=[22,44,34],18.1a722=[63,44,68],18.1b2d7=[44,38,66],18.1be8d=[62,44,74],18.1c265=[19,44,65],18.1c568=[40,44,63],18.1d503=[44,75,60],18.1d5aa=[13,44,24],18.1d84e=[47,44,56],18.1d9d6=[32,44,53],18.1e780=[30,44,73],18.1eab7=[44,60,71],18.1f635=[21,44,75],18.1f8d1=[42,44,57],18.20f90=[29,44,72],18.21762=[44,62,73],18.21915=[44,38,75],18.222f4=[44,50,45],18.232dc=[40,44,66],18.238ac=[44,58,19],18.23a24=[44,30,60],18.23be9=[44,59,66],18.23c8a=[44,55,24],18.242c6=[55,44,63],18.24781=[44,68,19],18.24a78=[18,44,21],18.24b04=[59,44,71],18.25387=[13,44,75],18.2547a=[44,52,75],18.263d0=[44,58,22],18.2732c=[28,44,66],18.274ba=[44,69,24],18.27c09=[44,53,72],18.28917=[44,74,20],18.29509=[44,50,10],18.29da2=[44,63,11],18.2a618=[44,32,69],18.2a6b5=[46,44,60],18.2ae05=[44,69,61],18.2b2c4=[44,51,15],18.2bbf5=[62,44,74],18.2bc8e=[44,57,60],18.2c1cd=[44,51,65],18.2c8ec=[44,55,57,68],18.2d915=[44,57,66],18.2e1f6=[44,38,53],18.2eecb=[44,67,75],18.2f1ad=[],18.2f1e6=[14,44,74],18.2f303=[44,55,15],18.2fe58=[44,32,22],18.309f2=[44,34,25],18.30a3e=[44,33,25],18.30bbd=[12,44,65]} v10022) v1 ==== 4461+0+0 (1073760261 0 0) 0xcc24ad20 con 0x9cd1400
  1824. -615> 2012-12-18 19:54:57.888973 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.889020 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1825. -614> 2012-12-18 19:54:57.889041 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 4461 (0x8e4928 -> 0)
  1826. -613> 2012-12-18 19:54:57.899950 7f977376c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (44225679 -> 44225721)
  1827. -612> 2012-12-18 19:54:57.899982 7f977376c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1828. -611> 2012-12-18 19:54:57.900163 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.27 10.1.0.15:6818/19812 12 ==== mon_subscribe({monmap=2+,osdmap=9281}) v2 ==== 42+0+0 (1924620553 0 0) 0xccbb5dc0 con 0x722d7c0
  1829. -610> 2012-12-18 19:54:57.900197 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9281..10022] to osd.27 10.1.0.15:6818/19812
  1830. -609> 2012-12-18 19:54:57.901385 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.27 10.1.0.15:6818/19812 -- osd_map(9281..9381 src has 7326..10022) v3 -- ?+0 0x7004800
  1831. -608> 2012-12-18 19:54:57.901449 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.27 10.1.0.15:6818/19812 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31380
  1832. -607> 2012-12-18 19:54:57.901458 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 44225679)
  1833. -606> 2012-12-18 19:54:57.901487 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1834. -605> 2012-12-18 19:54:57.951746 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3567 (44225679 -> 44229246)
  1835. -604> 2012-12-18 19:54:57.951780 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3567 (0 -> 3567)
  1836. -603> 2012-12-18 19:54:57.952004 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 8 ==== osd_pgtemp(e10022 {0.12b=[],0.1ed=[42,50,66,64],0.768=[50,19,71],0.b5d=[10,50,38],1.12a=[],1.6db=[29,50,57],1.767=[50,19,71],1.b5c=[50,38,10],1.c09=[28,50,45],18.6ca=[29,50,57],18.b4b=[10,50,38],18.bf8=[50,45,28],18.1b19=[50,45,13],18.2514=[50,41,72],18.3f12=[50,29,64],18.403a=[17,50,61],18.452e=[50,75,69],18.4f96=[50,38,71],18.52b8=[50,25,11],18.5481=[50,46,69],18.5e10=[50,46,42],18.61e9=[50,75,16],18.661c=[50,42,40],18.78de=[50,27,63],18.7b0f=[50,34,12],18.855a=[50,43,24],18.8cd1=[50,74,18],18.9818=[50,49,27],18.9e9e=[50,69,60],18.9f89=[50,44,10],18.a86c=[50,44,60],18.b33a=[50,32,65],18.b6a9=[50,66,19],18.c25d=[50,45,73],18.cb52=[50,38,67],18.ccff=[50,73,68],18.d1d8=[50,47,56],18.d252=[50,32,24],18.efe4=[50,67,68],18.fb49=[50,63,69],18.fed4=[50,38,44],18.fedd=[50,70,71],18.115d9=[30,50,72],18.12a90=[26,50,57],18.13671=[48,50,73],18.13e68=[50,43,73],18.145ee=[19,50,53],18.14858=[50,44,66],18.14953=[20,50,55],18.14d8f=[10,50,26],18.173e5=[43,50,51],18.17b75=[29,50,73],18.17ca5=[28,50,72],18.17cc6=[46,50,60],18.1894e=[],18.19472=[50,35,39],18.19571=[63,50,74],18.195a7=[33,50,75],18.199b1=[50,46,68],18.19dcf=[13,50,71],18.1a189=[50,45,62],18.1a99e=[40,50,51],18.1bed1=[50,43,44],18.1c85d=[65,50,71],18.1d170=[],18.1d357=[59,50,69],18.1de70=[50,45,30],18.1dea4=[22,50,68],18.1dfde=[19,50,45],18.1ea8b=[],18.1f166=[21,50,72],18.1f1aa=[50,40,68],18.1f1b5=[19,50,72],18.1fe73=[50,71,75],18.2011d=[50,40,70],18.2017c=[66,50,75],18.205b5=[50,75,65],18.213aa=[50,44,62],18.21bb3=[50,68,16],18.221cf=[50,27,18],18.22206=[50,61,64],18.228ca=[50,45,63],18.22d9d=[50,46,74],18.23e66=[17,50,62],18.24382=[50,22,64],18.247e5=[50,49,26],18.2507f=[50,22,66],18.25475=[50,48,21],18.256de=[18,50,27],18.25a44=[50,47,63],18.27b01=[50,28,66],18.2894e=[50,48,13],18.29edc=[50,38,28],18.2a48d=[28,50,30],18.2a4a6=[50,44,66],18.2a64b=[50,65,25],18.2bb82=[50,47,17],18.2bcf5=[50,62,29],18.2ce5c=[50,65,22],18.2cf0b=[50,42,20],18.2d10b=[50,30,61],18.2de2d=[54,50,64],18.2f01a=[29,50,64],18.30017=[50,74,55],18.302f7=[62,50,71],18.3062b=[27,50,44],18.3082a=[50,45,60],18.30832=[15,50,58],18.30c57=[17,50,38]} v10022) v1 ==== 3567+0+0 (3021488171 0 0) 0xcc24b0e0 con 0x722d540
  1837. -602> 2012-12-18 19:54:57.952126 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:57.952127 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1838. -601> 2012-12-18 19:54:57.952145 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3567 (0x8e4928 -> 0)
  1839. -600> 2012-12-18 19:54:57.953921 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (44229246 -> 44229288)
  1840. -599> 2012-12-18 19:54:57.953930 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1841. -598> 2012-12-18 19:54:57.954097 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 9 ==== mon_subscribe({monmap=2+,osdmap=9356}) v2 ==== 42+0+0 (2543827843 0 0) 0xcb009880 con 0x722d540
  1842. -597> 2012-12-18 19:54:57.954128 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9356..10022] to osd.50 10.1.0.16:6801/16285
  1843. -596> 2012-12-18 19:54:57.954866 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.50 10.1.0.16:6801/16285 -- osd_map(9356..9456 src has 7326..10022) v3 -- ?+0 0x7004e00
  1844. -595> 2012-12-18 19:54:57.954929 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.50 10.1.0.16:6801/16285 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf30ea0
  1845. -594> 2012-12-18 19:54:57.954938 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 44229246)
  1846. -593> 2012-12-18 19:54:57.954967 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1847. -592> 2012-12-18 19:54:57.956934 7f977477c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 296433 (44229246 -> 44525679)
  1848. -591> 2012-12-18 19:54:57.956945 7f977477c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 296433 (0 -> 296433)
  1849. -590> 2012-12-18 19:54:57.961890 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.50 10.1.0.16:6801/16285 10 ==== log(1000 entries) v1 ==== 296433+0+0 (322455987 0 0) 0xc394400 con 0x722d540
  1850. -589> 2012-12-18 19:54:57.961914 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:57.961916 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1851. -588> 2012-12-18 19:54:57.964139 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 296433 (0x8e4928 -> 0)
  1852. -587> 2012-12-18 19:54:58.026558 7f976f93c700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9ccea00 sd=53 pgs=0 cs=0 l=0).accept sd=53
  1853. -586> 2012-12-18 19:54:58.028600 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (44525679 -> 44525736)
  1854. -585> 2012-12-18 19:54:58.028628 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1855. -584> 2012-12-18 19:54:58.028793 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (287088113 0 0) 0x6093e00 con 0xcc245900
  1856. -583> 2012-12-18 19:54:58.028819 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.028821 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1857. -582> 2012-12-18 19:54:58.028863 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6828/20647 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x6093a00 con 0xcc245900
  1858. -581> 2012-12-18 19:54:58.028919 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 44525679)
  1859. -580> 2012-12-18 19:54:58.028924 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1860. -579> 2012-12-18 19:54:58.074180 7f976f73a700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9cce800 sd=54 pgs=0 cs=0 l=0).accept sd=54
  1861. -578> 2012-12-18 19:54:58.074520 7f976f73a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (44525679 -> 44525736)
  1862. -577> 2012-12-18 19:54:58.074548 7f976f73a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1863. -576> 2012-12-18 19:54:58.074733 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.39 10.1.0.12:6828/22667 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (612119862 0 0) 0xccc8a000 con 0xcc245540
  1864. -575> 2012-12-18 19:54:58.074760 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.074762 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1865. -574> 2012-12-18 19:54:58.074802 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6828/22667 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x6093800 con 0xcc245540
  1866. -573> 2012-12-18 19:54:58.074822 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 44525679)
  1867. -572> 2012-12-18 19:54:58.074874 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1868. -571> 2012-12-18 19:54:58.112304 7f9774f84700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (44525679 -> 44525721)
  1869. -570> 2012-12-18 19:54:58.112338 7f9774f84700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1870. -569> 2012-12-18 19:54:58.112470 7f977255a700 1 -- 10.1.0.11:6789/0 <== mds.0 10.1.0.14:6826/14574 5 ==== mon_subscribe({mdsmap=2488+,monmap=2+}) v2 ==== 42+0+0 (3769148227 0 0) 0xcc57a000 con 0x722d2c0
  1871. -568> 2012-12-18 19:54:58.112501 7f977255a700 1 -- 10.1.0.11:6789/0 --> mds.0 10.1.0.14:6826/14574 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf30d00
  1872. -567> 2012-12-18 19:54:58.112531 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 44525679)
  1873. -566> 2012-12-18 19:54:58.112538 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1874. -565> 2012-12-18 19:54:58.137735 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (44525679 -> 44525721)
  1875. -564> 2012-12-18 19:54:58.137769 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1876. -563> 2012-12-18 19:54:58.137960 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 12 ==== mon_subscribe({monmap=2+,osdmap=9740}) v2 ==== 42+0+0 (2744075302 0 0) 0xcc56d6c0 con 0x722d040
  1877. -562> 2012-12-18 19:54:58.137991 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9740..10022] to osd.54 10.1.0.16:6813/16922
  1878. -561> 2012-12-18 19:54:58.138686 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.54 10.1.0.16:6813/16922 -- osd_map(9740..9840 src has 7326..10022) v3 -- ?+0 0xccc8a000
  1879. -560> 2012-12-18 19:54:58.138748 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.54 10.1.0.16:6813/16922 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf309c0
  1880. -559> 2012-12-18 19:54:58.138757 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 44525679)
  1881. -558> 2012-12-18 19:54:58.138788 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  1882. -557> 2012-12-18 19:54:58.140602 7f9774378700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 267315 (44525679 -> 44792994)
  1883. -556> 2012-12-18 19:54:58.140661 7f9774378700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 267315 (0 -> 267315)
  1884. -555> 2012-12-18 19:54:58.140711 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (44792994 -> 44793036)
  1885. -554> 2012-12-18 19:54:58.140722 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (267315 -> 267357)
  1886. -553> 2012-12-18 19:54:58.140835 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 11 ==== mon_subscribe({monmap=2+,osdmap=9358}) v2 ==== 42+0+0 (2047762914 0 0) 0xca7b4000 con 0x722cdc0
  1887. -552> 2012-12-18 19:54:58.140866 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9358..10022] to osd.59 10.1.0.16:6828/20636
  1888. -551> 2012-12-18 19:54:58.141643 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.59 10.1.0.16:6828/20636 -- osd_map(9358..9458 src has 7326..10022) v3 -- ?+0 0x6093600
  1889. -550> 2012-12-18 19:54:58.141693 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.59 10.1.0.16:6828/20636 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf30340
  1890. -549> 2012-12-18 19:54:58.141701 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 44792994)
  1891. -548> 2012-12-18 19:54:58.141714 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 267315)
  1892. -547> 2012-12-18 19:54:58.143362 7f9773368700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 293762 (44792994 -> 45086756)
  1893. -546> 2012-12-18 19:54:58.143373 7f9773368700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 293762 (267315 -> 561077)
  1894. -545> 2012-12-18 19:54:58.146615 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.54 10.1.0.16:6813/16922 13 ==== log(925 entries) v1 ==== 267315+0+0 (255801339 0 0) 0x6092e00 con 0x722d040
  1895. -544> 2012-12-18 19:54:58.146640 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:58.146642 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1896. -543> 2012-12-18 19:54:58.148684 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 267315 (0x8e4928 -> 293762)
  1897. -542> 2012-12-18 19:54:58.148693 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.59 10.1.0.16:6828/20636 12 ==== log(1000 entries) v1 ==== 293762+0+0 (4227245652 0 0) 0x9cb9800 con 0x722cdc0
  1898. -541> 2012-12-18 19:54:58.148704 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:58.148705 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1899. -540> 2012-12-18 19:54:58.150965 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 293762 (0x8e4928 -> 0)
  1900. -539> 2012-12-18 19:54:58.247108 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (0 -> 84)
  1901. -538> 2012-12-18 19:54:58.247234 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (84 -> 168)
  1902. -537> 2012-12-18 19:54:58.247297 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 13 ==== paxos(mdsmap lease_ack lc 2487 fc 1 pn 0 opn 0) v1 ==== 84+0+0 (2662026556 0 0) 0x84af680 con 0xb7b4dc0
  1903. -536> 2012-12-18 19:54:58.247323 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 84)
  1904. -535> 2012-12-18 19:54:58.247330 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 14 ==== paxos(osdmap lease_ack lc 10022 fc 7326 pn 0 opn 0) v1 ==== 84+0+0 (3214099009 0 0) 0x772a780 con 0xb7b4dc0
  1905. -534> 2012-12-18 19:54:58.247362 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 0)
  1906. -533> 2012-12-18 19:54:58.254906 7f977396e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3892 (45086756 -> 45090648)
  1907. -532> 2012-12-18 19:54:58.254930 7f977396e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3892 (0 -> 3892)
  1908. -531> 2012-12-18 19:54:58.255155 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.55 10.1.0.16:6816/13746 12 ==== osd_pgtemp(e10022 {0.777=[27,55,61],1.174=[22,55,60],1.197=[11,55,75],1.776=[27,55,61],18.a7c=[55,48,74],18.1325=[30,55,58],18.180a=[55,48,29],18.1839=[55,46,11],18.18de=[17,55,71],18.20d4=[10,55,27],18.2a2d=[],18.2f59=[55,49,65],18.38e8=[55,41,61],18.3b8e=[50,55,61],18.5206=[55,33,64],18.54be=[25,55,69],18.6354=[38,55,75],18.68f0=[55,36,31],18.6de1=[55,48,58],18.6fe7=[55,71,63],18.73c8=[53,55,73],18.793e=[66,55,75],18.7d57=[58,55,71],18.8231=[55,46,65],18.847c=[],18.9e5c=[55,45,20],18.9f7f=[62,55,72],18.aa7b=[42,55,57],18.ac39=[32,55,68],18.ad58=[55,45,13],18.af76=[22,55,33],18.b7cf=[],18.c0de=[18,55,34],18.c641=[55,67,17],18.c657=[55,47,14],18.c8cb=[55,73,28],18.c979=[55,43,60],18.d5b0=[55,20,14],18.d7bb=[55,49,46],18.e325=[12,55,42],18.feec=[55,37,15],18.107d6=[55,67,22],18.10d5f=[55,71,74],18.114ff=[55,28,66],18.1266c=[59,55,68],18.1282c=[55,46,51],18.12967=[55,72,16],18.129d0=[55,43,40],18.12f92=[28,55,61],18.13173=[43,55,59],18.13d15=[10,55,13],18.13e17=[55,74,62],18.142d7=[28,55,66],18.14308=[55,43,27],18.1431f=[43,55,60],18.14b73=[55,43,10],18.15192=[55,71,72],18.1636d=[55,42,66,24],18.16e69=[19,55,73],18.17e10=[],18.17e63=[48,55,66],18.18006=[32,55,69],18.1861a=[13,55,60],18.18633=[],18.195c6=[55,69,32],18.1977a=[29,55,52],18.19b2e=[55,27,64],18.1aa1a=[44,55,58],18.1ab80=[55,69,18],18.1aecb=[29,55,72],18.1af59=[26,55,53],18.1b920=[55,60,21],18.1bb5d=[55,46,69],18.1c59f=[49,55,56],18.1c699=[],18.1cab5=[55,64,13],18.1d226=[10,55,29],18.1da8f=[],18.1ef5a=[],18.1f970=[47,55,67],18.1f9ec=[66,55,74],18.2011a=[55,37,71],18.20ea7=[14,55,59],18.20f5a=[55,21,70],18.21a22=[42,65,55,63],18.21e6a=[],18.21ee6=[55,75,24],18.22a10=[55,42,60],18.22af0=[55,47,61],18.23c5a=[55,38,70],18.24160=[],18.24bcd=[55,60,10],18.2500d=[55,20,15],18.25dec=[55,45,19],18.26152=[55,72,69],18.269c9=[55,32,74],18.26e87=[55,63,16],18.28754=[55,32,65],18.28beb=[52,55,72],18.29042=[20,55,48],18.294d1=[55,60,54],18.299fa=[55,49,25],18.29ab2=[55,44,17],18.2a11a=[55,60,27],18.2a19d=[],18.2a456=[25,55,62],18.2a6be=[41,55,67],18.2c3eb=[55,37,69],18.2cc5a=[55,61,20],18.2cd13=[55,32,26],18.2d809=[55,75,56],18.2d920=[55,75,67],18.2dc18=[],18.2ddac=[55,40,68],18.2e2b1=[55,68,11],18.2eaa7=[53,55,74],18.2f4a8=[55,47,18],18.2f4f3=[55,28,67],18.2ff43=[55,49,25],18.3045c=[],18.30aa5=[],18.30beb=[55,70,62]} v10022) v1 ==== 3892+0+0 (2791988561 0 0) 0xcc24b2c0 con 0x7003e00
  1909. -530> 2012-12-18 19:54:58.255285 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.255289 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1910. -529> 2012-12-18 19:54:58.255307 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3892 (0x8e4928 -> 0)
  1911. -528> 2012-12-18 19:54:58.280558 7f976f538700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9cce600 sd=55 pgs=0 cs=0 l=0).accept sd=55
  1912. -527> 2012-12-18 19:54:58.280900 7f976f538700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (45090648 -> 45090705)
  1913. -526> 2012-12-18 19:54:58.280929 7f976f538700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1914. -525> 2012-12-18 19:54:58.281096 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.37 10.1.0.12:6822/21280 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (1301933151 0 0) 0xb7b6000 con 0xcc245180
  1915. -524> 2012-12-18 19:54:58.281122 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.281124 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1916. -523> 2012-12-18 19:54:58.281164 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6822/21280 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6000 con 0xcc245180
  1917. -522> 2012-12-18 19:54:58.281194 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 45090648)
  1918. -521> 2012-12-18 19:54:58.281199 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1919. -520> 2012-12-18 19:54:58.305185 7f9774d82700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 23 (45090648 -> 45090671)
  1920. -519> 2012-12-18 19:54:58.305218 7f9774d82700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 23 (0 -> 23)
  1921. -518> 2012-12-18 19:54:58.305350 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7502687 10.1.0.111:0/1020428 4 ==== mon_subscribe({monmap=2+}) v2 ==== 23+0+0 (2366214315 0 0) 0x9cca8c0 con 0x722cb40
  1922. -517> 2012-12-18 19:54:58.305380 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7502687 10.1.0.111:0/1020428 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf301a0
  1923. -516> 2012-12-18 19:54:58.305424 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 23 (0x8e4928 -> 45090648)
  1924. -515> 2012-12-18 19:54:58.305432 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 23 (0x8e4928 -> 0)
  1925. -514> 2012-12-18 19:54:58.456300 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (0 -> 84)
  1926. -513> 2012-12-18 19:54:58.456423 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (84 -> 168)
  1927. -512> 2012-12-18 19:54:58.456550 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 15 ==== paxos(pgmap accept lc 6066306 fc 0 pn 129200 opn 0) v1 ==== 84+0+0 (2777155071 0 0) 0x772b900 con 0xb7b4dc0
  1928. -511> 2012-12-18 19:54:58.474231 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(pgmap commit lc 6066307 fc 0 pn 129200 opn 0) v1 -- ?+0 0x772a780
  1929. -510> 2012-12-18 19:54:58.474256 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(pgmap commit lc 6066307 fc 0 pn 129200 opn 0) v1 -- ?+0 0x84af680
  1930. -509> 2012-12-18 19:54:58.474296 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 84)
  1931. -508> 2012-12-18 19:54:58.474301 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 16 ==== paxos(logm lease_ack lc 5962551 fc 5962045 pn 0 opn 0) v1 ==== 84+0+0 (338408313 0 0) 0x772a000 con 0xb7b4dc0
  1932. -507> 2012-12-18 19:54:58.474313 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 0)
  1933. -506> 2012-12-18 19:54:58.515049 7f9772d62700 2 -- 10.1.0.11:6789/0 >> 10.1.0.14:6804/16775 pipe(0x7228a00 sd=44 pgs=28297 cs=1 l=1).reader couldn't read tag, Success
  1934. -505> 2012-12-18 19:54:58.515150 7f9772d62700 2 -- 10.1.0.11:6789/0 >> 10.1.0.14:6804/16775 pipe(0x7228a00 sd=44 pgs=28297 cs=1 l=1).fault 0: Success
  1935. -504> 2012-12-18 19:54:58.550955 7f9772d62700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x9cce400 sd=44 pgs=0 cs=0 l=0).accept sd=44
  1936. -503> 2012-12-18 19:54:58.552719 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (45090648 -> 45090705)
  1937. -502> 2012-12-18 19:54:58.552748 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  1938. -501> 2012-12-18 19:54:58.552863 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3366702108 0 0) 0x700ac00 con 0xcc244dc0
  1939. -500> 2012-12-18 19:54:58.552896 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.552899 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1940. -499> 2012-12-18 19:54:58.552937 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6806/30125 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xb7b6000 con 0xcc244dc0
  1941. -498> 2012-12-18 19:54:58.552959 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 45090648)
  1942. -497> 2012-12-18 19:54:58.552969 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  1943. -496> 2012-12-18 19:54:58.560211 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (45090648 -> 45090806)
  1944. -495> 2012-12-18 19:54:58.560234 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  1945. -494> 2012-12-18 19:54:58.560360 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3160045977 0 0) 0xccdc4200 con 0xcc244dc0
  1946. -493> 2012-12-18 19:54:58.560391 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.560393 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1947. -492> 2012-12-18 19:54:58.561217 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6806/30125 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x700ac00 con 0xcc244dc0
  1948. -491> 2012-12-18 19:54:58.561285 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 45090648)
  1949. -490> 2012-12-18 19:54:58.561293 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  1950. -489> 2012-12-18 19:54:58.564160 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 722 (0 -> 722)
  1951. -488> 2012-12-18 19:54:58.564360 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 17 ==== forward(log(2 entries) v1) to leader v1 ==== 722+0+0 (2051010203 0 0) 0x7725b80 con 0xb7b4dc0
  1952. -487> 2012-12-18 19:54:58.564388 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:58.564390 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  1953. -486> 2012-12-18 19:54:58.564414 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 722 (0x8e4928 -> 0)
  1954. -485> 2012-12-18 19:54:58.566017 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (0 -> 84)
  1955. -484> 2012-12-18 19:54:58.566152 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (84 -> 168)
  1956. -483> 2012-12-18 19:54:58.566217 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 18 ==== paxos(monmap lease_ack lc 1 fc 1 pn 0 opn 0) v1 ==== 84+0+0 (3611950277 0 0) 0x84af680 con 0xb7b4dc0
  1957. -482> 2012-12-18 19:54:58.566246 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 84)
  1958. -481> 2012-12-18 19:54:58.566252 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 19 ==== paxos(auth lease_ack lc 79006 fc 78981 pn 0 opn 0) v1 ==== 84+0+0 (3304152681 0 0) 0x9c79680 con 0xb7b4dc0
  1959. -480> 2012-12-18 19:54:58.566267 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 0)
  1960. -479> 2012-12-18 19:54:58.572772 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 195 (45090648 -> 45090843)
  1961. -478> 2012-12-18 19:54:58.572786 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 195 (0 -> 195)
  1962. -477> 2012-12-18 19:54:58.572894 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 3 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (376721265 0 0) 0xccdc4000 con 0xcc244dc0
  1963. -476> 2012-12-18 19:54:58.572927 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.572929 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1964. -475> 2012-12-18 19:54:58.573189 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6806/30125 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x9cc6800 con 0xcc244dc0
  1965. -474> 2012-12-18 19:54:58.573256 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 45090648)
  1966. -473> 2012-12-18 19:54:58.573264 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 0)
  1967. -472> 2012-12-18 19:54:58.581128 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (45090648 -> 45090690)
  1968. -471> 2012-12-18 19:54:58.581149 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1969. -470> 2012-12-18 19:54:58.581258 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (45090690 -> 45090712)
  1970. -469> 2012-12-18 19:54:58.581262 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  1971. -468> 2012-12-18 19:54:58.581268 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 4 ==== mon_subscribe({monmap=2+,osdmap=9309}) v2 ==== 42+0+0 (1627439730 0 0) 0xcc57a000 con 0xcc244dc0
  1972. -467> 2012-12-18 19:54:58.581301 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9309..10022] to osd.22 10.1.0.15:6806/30125
  1973. -466> 2012-12-18 19:54:58.581325 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 386 (45090712 -> 45091098)
  1974. -465> 2012-12-18 19:54:58.581333 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 386 (64 -> 450)
  1975. -464> 2012-12-18 19:54:58.582199 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (45091098 -> 45091256)
  1976. -463> 2012-12-18 19:54:58.582209 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (450 -> 608)
  1977. -462> 2012-12-18 19:54:58.582452 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.22 10.1.0.15:6806/30125 -- osd_map(9309..9409 src has 7326..10022) v3 -- ?+0 0xccdc4000
  1978. -461> 2012-12-18 19:54:58.582522 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.22 10.1.0.15:6806/30125 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57d380
  1979. -460> 2012-12-18 19:54:58.582540 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 45091214)
  1980. -459> 2012-12-18 19:54:58.582548 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 566)
  1981. -458> 2012-12-18 19:54:58.582557 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 5 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x11c46820 con 0xcc244dc0
  1982. -457> 2012-12-18 19:54:58.582573 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.582575 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1983. -456> 2012-12-18 19:54:58.582599 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 544)
  1984. -455> 2012-12-18 19:54:58.582605 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1692907642 0 0) 0x700ac00 con 0xcc245900
  1985. -454> 2012-12-18 19:54:58.582620 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.582621 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  1986. -453> 2012-12-18 19:54:58.582874 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6828/20647 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x7004200 con 0xcc245900
  1987. -452> 2012-12-18 19:54:58.582941 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 45091056)
  1988. -451> 2012-12-18 19:54:58.582949 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 386)
  1989. -450> 2012-12-18 19:54:58.582994 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 6 ==== osd_pgtemp(e10022 {18.f1f=[22,54,63],18.76b7=[22,51,65],18.b914=[22,55,56],18.c5a7=[22,32,72],18.14af3=[],18.1bdd1=[],18.1e019=[],18.20762=[22,49,66],18.24f9c=[22,32,12],18.2ca17=[22,50,11],18.2e30e=[22,58,66],18.2f2c8=[22,57,67]} v10022) v1 ==== 386+0+0 (4204728770 0 0) 0xcc24b4a0 con 0xcc244dc0
  1990. -449> 2012-12-18 19:54:58.583030 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.583032 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  1991. -448> 2012-12-18 19:54:58.583053 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 386 (0x8e4928 -> 0)
  1992. -447> 2012-12-18 19:54:58.588626 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (45091056 -> 45091098)
  1993. -446> 2012-12-18 19:54:58.588643 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  1994. -445> 2012-12-18 19:54:58.588832 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (45091098 -> 45091120)
  1995. -444> 2012-12-18 19:54:58.588917 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  1996. -443> 2012-12-18 19:54:58.588842 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 3 ==== mon_subscribe({monmap=2+,osdmap=10021}) v2 ==== 42+0+0 (1290353248 0 0) 0xcc57a1c0 con 0xcc245900
  1997. -442> 2012-12-18 19:54:58.588970 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 3938 (45091120 -> 45095058)
  1998. -441> 2012-12-18 19:54:58.588973 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 3938 (64 -> 4002)
  1999. -440> 2012-12-18 19:54:58.588966 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10021..10022] to osd.49 10.1.0.14:6828/20647
  2000. -439> 2012-12-18 19:54:58.589014 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.49 10.1.0.14:6828/20647 -- osd_map(10021..10022 src has 7326..10022) v3 -- ?+0 0x700ac00
  2001. -438> 2012-12-18 19:54:58.589034 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.49 10.1.0.14:6828/20647 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57d040
  2002. -437> 2012-12-18 19:54:58.589042 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 45095016)
  2003. -436> 2012-12-18 19:54:58.589047 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 3960)
  2004. -435> 2012-12-18 19:54:58.589051 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 4 ==== osd_alive(want up_thru 10020 have 10022) v1 ==== 22+0+0 (4084230999 0 0) 0xcc57d520 con 0xcc245900
  2005. -434> 2012-12-18 19:54:58.589060 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.589064 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2006. -433> 2012-12-18 19:54:58.589079 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 3938)
  2007. -432> 2012-12-18 19:54:58.589134 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 5 ==== osd_pgtemp(e10022 {0.43a=[53,49,61],1.439=[53,49,61],18.428=[49,53,61],18.13ca=[49,66,46],18.1b1c=[49,58,63],18.311f=[49,37,27],18.37e0=[43,49,69],18.3817=[49,18,15],18.3853=[49,10,15],18.42cd=[49,50,67],18.42fa=[14,49,42],18.4418=[12,49,10],18.5595=[61,49,69],18.56e3=[55,49,69],18.5872=[49,53,75],18.5a74=[21,49,33],18.5b3f=[13,49,33],18.5cdf=[49,32,68],18.61c9=[49,55,72],18.6226=[49,57,51],18.68fa=[49,50,28],18.6f3f=[49,55,11],18.7153=[19,49,68],18.7417=[49,75,73],18.81bd=[49,58,60],18.82c3=[49,71,12],18.8805=[10,49,53],18.88a5=[33,49,74],18.8d4d=[42,49,69],18.a1a1=[49,10,16],18.a8a4=[49,68,29],18.a98b=[49,63,10],18.ae04=[49,38,32],18.c325=[49,37,66],18.c545=[49,38,70],18.d0d3=[49,75,17],18.d52d=[59,49,75],18.d589=[49,37,71],18.dc66=[49,37,61],18.f5cb=[49,51,29],18.f6d1=[60,49,68],18.fc7e=[18,49,62],18.fcfc=[49,57,69],18.1032c=[49,52,46],18.104c9=[49,32,43],18.107d4=[41,49,64],18.1100a=[53,49,63],18.11428=[49,63,70],18.120e6=[21,49,52],18.129fd=[49,63,28,64],18.134cc=[49,34,21],18.15c98=[49,53,66,11],18.15fb4=[50,49,73],18.163a8=[24,49,73],18.16743=[],18.1700d=[43,49,62],18.173df=[33,49,71],18.1760b=[59,49,72],18.17662=[49,58,62],18.17c44=[37,49,74],18.18cb2=[11,49,46],18.1926c=[16,49,42],18.1a82c=[42,49,57],18.1aa42=[49,53,10],18.1b7ed=[49,32,22],18.1b896=[27,49,71],18.1c2d9=[49,52,22],18.1c8f2=[49,39,22,75],18.1dce0=[53,49,68],18.1def3=[24,49,66],18.1e213=[49,18,71],18.1e323=[17,49,41],18.1ea23=[49,32,17],18.1ed5e=[44,49,51],18.1f009=[48,49,67],18.1f2d6=[49,58,65],18.1fac0=[26,49,44],18.20631=[],18.20833=[49,62,61],18.20982=[],18.2106b=[49,50,72],18.2200d=[49,75,61],18.2201b=[49,33,70],18.22068=[56,49,74],18.220a0=[49,50,74],18.22171=[49,70,18],18.22460=[49,69,68],18.22a94=[],18.22dff=[58,49,61],18.2358c=[15,49,26],18.24bd9=[49,64,25],18.24e8a=[49,57,10],18.256ea=[49,57,29],18.26a80=[49,32,14],18.26b95=[43,49,57],18.274e7=[49,25,21],18.276fe=[],18.2867e=[27,49,68],18.29257=[49,51,10],18.2958b=[49,53,69],18.2975a=[49,65,64],18.29e28=[49,59,68],18.2a0d5=[15,49,24],18.2a415=[49,33,10],18.2a6bf=[49,29,16],18.2a6ec=[11,49,64],18.2aaed=[49,51,70],18.2ab95=[49,59,18],18.2ad5f=[49,64,69],18.2b1c3=[11,49,44],18.2c9ca=[49,57,20],18.2caf2=[49,53,25],18.2d0b4=[49,32,65],18.2d661=[49,32,37],18.2d6f5=[49,53,17],18.2d78b=[49,57,66],18.2e3c9=[49,71,25],18.2ef3f=[49,73,14],18.2ef56=[49,51,60],18.30b1d=[22,49,44]} v10022) v1 ==== 3938+0+0 (178643618 0 0) 0xcc24b680 con 0xcc245900
  2008. -431> 2012-12-18 19:54:58.589257 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.589258 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2009. -430> 2012-12-18 19:54:58.589271 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 3938 (0x8e4928 -> 0)
  2010. -429> 2012-12-18 19:54:58.589381 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 610932 (45095016 -> 45705948)
  2011. -428> 2012-12-18 19:54:58.589395 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 610932 (0 -> 610932)
  2012. -427> 2012-12-18 19:54:58.600302 7f9772d62700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 289538 (45705948 -> 45995486)
  2013. -426> 2012-12-18 19:54:58.600338 7f9772d62700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 289538 (610932 -> 900470)
  2014. -425> 2012-12-18 19:54:58.600353 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 7 ==== pg_stats(1917 pgs tid 694224 v 10022) v1 ==== 610932+0+0 (3413177303 0 0) 0x772c900 con 0xcc244dc0
  2015. -424> 2012-12-18 19:54:58.600375 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.600376 lease_expire=0.000000 has v10022 lc 6066307
  2016. -423> 2012-12-18 19:54:58.600390 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 610932 (0x8e4928 -> 289538)
  2017. -422> 2012-12-18 19:54:58.603196 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.22 10.1.0.15:6806/30125 8 ==== log(1000 entries) v1 ==== 289538+0+0 (2144910775 0 0) 0x7004200 con 0xcc244dc0
  2018. -421> 2012-12-18 19:54:58.603221 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:58.603222 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  2019. -420> 2012-12-18 19:54:58.605432 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 289538 (0x8e4928 -> 0)
  2020. -419> 2012-12-18 19:54:58.609200 7f976f336700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x7005600 sd=56 pgs=0 cs=0 l=0).accept sd=56
  2021. -418> 2012-12-18 19:54:58.611843 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (45995486 -> 45995543)
  2022. -417> 2012-12-18 19:54:58.611868 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2023. -416> 2012-12-18 19:54:58.611981 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3551403537 0 0) 0xccdc4400 con 0xcc244b40
  2024. -415> 2012-12-18 19:54:58.612018 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.612021 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2025. -414> 2012-12-18 19:54:58.612060 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6800/17135 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc5c00 con 0xcc244b40
  2026. -413> 2012-12-18 19:54:58.612079 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 45995486)
  2027. -412> 2012-12-18 19:54:58.612090 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2028. -411> 2012-12-18 19:54:58.617361 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 26 (0 -> 26)
  2029. -410> 2012-12-18 19:54:58.617525 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 20 ==== global_id (7504196) v1 ==== 26+0+0 (2658689508 0 0) 0xcbf30000 con 0xb7b4dc0
  2030. -409> 2012-12-18 19:54:58.617549 7f977255a700 1 mon.a@0(leader).paxos(auth active c 78981..79006) is_readable now=2012-12-18 19:54:58.617550 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2031. -408> 2012-12-18 19:54:58.617576 7f977255a700 5 mon.a@0(leader).paxos(auth active c 78981..79006) propose_new_value 79007 19 bytes
  2032. -407> 2012-12-18 19:54:58.619419 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (45995486 -> 45995644)
  2033. -406> 2012-12-18 19:54:58.619429 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (26 -> 184)
  2034. -405> 2012-12-18 19:54:58.620092 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1241208 (45995644 -> 47236852)
  2035. -404> 2012-12-18 19:54:58.620100 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1241208 (184 -> 1241392)
  2036. -403> 2012-12-18 19:54:58.633542 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(auth begin lc 79006 fc 0 pn 129200 opn 0) v1 -- ?+0 0x9c79680
  2037. -402> 2012-12-18 19:54:58.633568 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(auth begin lc 79006 fc 0 pn 129200 opn 0) v1 -- ?+0 0x84af680
  2038. -401> 2012-12-18 19:54:58.633605 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 26 (0x8e4928 -> 1241366)
  2039. -400> 2012-12-18 19:54:58.633611 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1183570595 0 0) 0xccffc200 con 0xcc244b40
  2040. -399> 2012-12-18 19:54:58.633621 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.633622 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2041. -398> 2012-12-18 19:54:58.633825 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6800/17135 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc4400 con 0xcc244b40
  2042. -397> 2012-12-18 19:54:58.633880 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 47236694)
  2043. -396> 2012-12-18 19:54:58.633888 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 1241208)
  2044. -395> 2012-12-18 19:54:58.640410 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 297348 (47236694 -> 47534042)
  2045. -394> 2012-12-18 19:54:58.640423 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 6 ==== pg_stats(3902 pgs tid 725032 v 10022) v1 ==== 1241208+0+0 (2845936806 0 0) 0x69a6900 con 0xcc245900
  2046. -393> 2012-12-18 19:54:58.640550 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 297348 (1241208 -> 1538556)
  2047. -392> 2012-12-18 19:54:58.640550 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.640555 lease_expire=0.000000 has v10022 lc 6066307
  2048. -391> 2012-12-18 19:54:58.640575 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1241208 (0x8e4928 -> 297348)
  2049. -390> 2012-12-18 19:54:58.643140 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 195 (47534042 -> 47534237)
  2050. -389> 2012-12-18 19:54:58.643150 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 195 (297348 -> 297543)
  2051. -388> 2012-12-18 19:54:58.643287 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 3 ==== auth(proto 2 165 bytes epoch 0) v1 ==== 195+0+0 (2676852378 0 0) 0xccffc000 con 0xcc244b40
  2052. -387> 2012-12-18 19:54:58.643320 7f976f93c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (47534237 -> 47534269)
  2053. -386> 2012-12-18 19:54:58.643407 7f976f93c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (297543 -> 297575)
  2054. -385> 2012-12-18 19:54:58.643401 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.643403 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2055. -384> 2012-12-18 19:54:58.643591 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6800/17135 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccffc200 con 0xcc244b40
  2056. -383> 2012-12-18 19:54:58.643647 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 195 (0x8e4928 -> 47534074)
  2057. -382> 2012-12-18 19:54:58.643655 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 195 (0x8e4928 -> 297380)
  2058. -381> 2012-12-18 19:54:58.643669 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 7 ==== log(1000 entries) v1 ==== 297348+0+0 (893282694 0 0) 0xb7b6000 con 0xcc245900
  2059. -380> 2012-12-18 19:54:58.643687 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962551) is_readable now=2012-12-18 19:54:58.643689 lease_expire=2012-12-18 19:55:01.848948 has v0 lc 5962551
  2060. -379> 2012-12-18 19:54:58.644735 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (297380 -> 297464)
  2061. -378> 2012-12-18 19:54:58.645914 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 297348 (0x8e4928 -> 116)
  2062. -377> 2012-12-18 19:54:58.645926 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 21 ==== paxos(logm accept lc 5962551 fc 0 pn 129200 opn 0) v1 ==== 84+0+0 (2474618919 0 0) 0x84af680 con 0xb7b4dc0
  2063. -376> 2012-12-18 19:54:58.646818 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (47534074 -> 47534116)
  2064. -375> 2012-12-18 19:54:58.646827 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (116 -> 158)
  2065. -374> 2012-12-18 19:54:58.646866 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (47534116 -> 47534138)
  2066. -373> 2012-12-18 19:54:58.646884 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (158 -> 180)
  2067. -372> 2012-12-18 19:54:58.646948 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 303 (47534138 -> 47534441)
  2068. -371> 2012-12-18 19:54:58.646980 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 303 (180 -> 483)
  2069. -370> 2012-12-18 19:54:58.655248 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 623148 (47534441 -> 48157589)
  2070. -369> 2012-12-18 19:54:58.655273 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 623148 (483 -> 623631)
  2071. -368> 2012-12-18 19:54:58.666089 7f976f134700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0x7005800 sd=58 pgs=0 cs=0 l=0).accept sd=58
  2072. -367> 2012-12-18 19:54:58.666627 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1007 (48157589 -> 48158596)
  2073. -366> 2012-12-18 19:54:58.666668 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1007 (623631 -> 624638)
  2074. -365> 2012-12-18 19:54:58.666722 7f976f336700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 32 (48158596 -> 48158628)
  2075. -364> 2012-12-18 19:54:58.666762 7f976f336700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 32 (624638 -> 624670)
  2076. -363> 2012-12-18 19:54:58.669303 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (48158628 -> 48158685)
  2077. -362> 2012-12-18 19:54:58.669312 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (624670 -> 624727)
  2078. -361> 2012-12-18 19:54:58.669729 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(logm commit lc 5962552 fc 0 pn 129200 opn 0) v1 -- ?+0 0x772a000
  2079. -360> 2012-12-18 19:54:58.669753 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(logm commit lc 5962552 fc 0 pn 129200 opn 0) v1 -- ?+0 0x772b900
  2080. -359> 2012-12-18 19:54:58.669799 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 624643)
  2081. -358> 2012-12-18 19:54:58.669804 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.49 10.1.0.14:6828/20647 8 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0xccffc400 con 0xcc245900
  2082. -357> 2012-12-18 19:54:58.669837 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.669838 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2083. -356> 2012-12-18 19:54:58.669890 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.14:6828/20647 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccffc000 con 0xcc245900
  2084. -355> 2012-12-18 19:54:58.669964 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 48158653)
  2085. -354> 2012-12-18 19:54:58.669969 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 624611)
  2086. -353> 2012-12-18 19:54:58.669973 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 4 ==== mon_subscribe({monmap=2+,osdmap=9269}) v2 ==== 42+0+0 (3802541782 0 0) 0xcd3b2000 con 0xcc244b40
  2087. -352> 2012-12-18 19:54:58.670019 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9269..10022] to osd.20 10.1.0.15:6800/17135
  2088. -351> 2012-12-18 19:54:58.671299 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.20 10.1.0.15:6800/17135 -- osd_map(9269..9369 src has 7326..10022) v3 -- ?+0 0xccffc400
  2089. -350> 2012-12-18 19:54:58.671362 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.20 10.1.0.15:6800/17135 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf30000
  2090. -349> 2012-12-18 19:54:58.671383 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 48158611)
  2091. -348> 2012-12-18 19:54:58.671393 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 624569)
  2092. -347> 2012-12-18 19:54:58.671398 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (4282417406 0 0) 0xccffcc00 con 0xcc244780
  2093. -346> 2012-12-18 19:54:58.671417 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.671419 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2094. -345> 2012-12-18 19:54:58.671462 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6809/30066 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc5600 con 0xcc244780
  2095. -344> 2012-12-18 19:54:58.671487 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 48158554)
  2096. -343> 2012-12-18 19:54:58.671541 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 624512)
  2097. -342> 2012-12-18 19:54:58.671546 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 5 ==== osd_alive(want up_thru 10020 have 10022) v1 ==== 22+0+0 (4084230999 0 0) 0xcc57d380 con 0xcc244b40
  2098. -341> 2012-12-18 19:54:58.671561 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.671562 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2099. -340> 2012-12-18 19:54:58.671586 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 624490)
  2100. -339> 2012-12-18 19:54:58.671594 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 6 ==== osd_pgtemp(e10022 {18.d7b=[],18.6038=[20,56,70],18.9a3e=[20,59,62],18.9c53=[20,45,62,66],18.bfda=[20,57,67],18.1958e=[20,41,68],18.294f6=[20,42,61],18.29e8d=[20,58,16],18.2b37c=[]} v10022) v1 ==== 303+0+0 (2598819599 0 0) 0xcc24b860 con 0xcc244b40
  2101. -338> 2012-12-18 19:54:58.671629 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.671631 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2102. -337> 2012-12-18 19:54:58.671648 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 303 (0x8e4928 -> 624187)
  2103. -336> 2012-12-18 19:54:58.671669 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 7 ==== pg_stats(1956 pgs tid 656136 v 10022) v1 ==== 623148+0+0 (2843919216 0 0) 0x69a6b40 con 0xcc244b40
  2104. -335> 2012-12-18 19:54:58.671687 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.671688 lease_expire=0.000000 has v10022 lc 6066307
  2105. -334> 2012-12-18 19:54:58.671700 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 623148 (0x8e4928 -> 1039)
  2106. -333> 2012-12-18 19:54:58.671704 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 8 ==== log(4 entries) v1 ==== 1007+0+0 (2412770463 0 0) 0xccffca00 con 0xcc244b40
  2107. -332> 2012-12-18 19:54:58.671718 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:58.671719 lease_expire=0.000000 has v0 lc 5962552
  2108. -331> 2012-12-18 19:54:58.671729 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1007 (0x8e4928 -> 32)
  2109. -330> 2012-12-18 19:54:58.671735 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.20 10.1.0.15:6800/17135 9 ==== auth(proto 2 2 bytes epoch 0) v1 ==== 32+0+0 (2260890001 0 0) 0xccffc800 con 0xcc244b40
  2110. -329> 2012-12-18 19:54:58.671751 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.671752 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2111. -328> 2012-12-18 19:54:58.671817 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6800/17135 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccffcc00 con 0xcc244b40
  2112. -327> 2012-12-18 19:54:58.671838 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 32 (0x8e4928 -> 48158522)
  2113. -326> 2012-12-18 19:54:58.671845 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 32 (0x8e4928 -> 0)
  2114. -325> 2012-12-18 19:54:58.674571 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (48158522 -> 48158680)
  2115. -324> 2012-12-18 19:54:58.674588 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  2116. -323> 2012-12-18 19:54:58.674734 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (2793020268 0 0) 0xccffd000 con 0xcc244780
  2117. -322> 2012-12-18 19:54:58.674758 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.674760 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2118. -321> 2012-12-18 19:54:58.674955 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6809/30066 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccffc800 con 0xcc244780
  2119. -320> 2012-12-18 19:54:58.675022 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 48158522)
  2120. -319> 2012-12-18 19:54:58.675028 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  2121. -318> 2012-12-18 19:54:58.676051 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (48158522 -> 48158564)
  2122. -317> 2012-12-18 19:54:58.676061 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2123. -316> 2012-12-18 19:54:58.676128 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (48158564 -> 48158586)
  2124. -315> 2012-12-18 19:54:58.676188 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  2125. -314> 2012-12-18 19:54:58.676221 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 525 (48158586 -> 48159111)
  2126. -313> 2012-12-18 19:54:58.676282 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 525 (64 -> 589)
  2127. -312> 2012-12-18 19:54:58.676205 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 3 ==== mon_subscribe({monmap=2+,osdmap=9265}) v2 ==== 42+0+0 (1013574885 0 0) 0xcd5c4000 con 0xcc244780
  2128. -311> 2012-12-18 19:54:58.676340 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9265..10022] to osd.63 10.1.0.17:6809/30066
  2129. -310> 2012-12-18 19:54:58.677801 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.63 10.1.0.17:6809/30066 -- osd_map(9265..9365 src has 7326..10022) v3 -- ?+0 0xccffd000
  2130. -309> 2012-12-18 19:54:58.677869 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.63 10.1.0.17:6809/30066 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57cea0
  2131. -308> 2012-12-18 19:54:58.677917 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 48159069)
  2132. -307> 2012-12-18 19:54:58.677926 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 547)
  2133. -306> 2012-12-18 19:54:58.677929 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xcc57d1e0 con 0xcc244780
  2134. -305> 2012-12-18 19:54:58.677938 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.677939 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2135. -304> 2012-12-18 19:54:58.677956 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 525)
  2136. -303> 2012-12-18 19:54:58.677960 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 5 ==== osd_pgtemp(e10022 {18.28b9=[63,53,24],18.3f60=[63,56,48],18.49ff=[63,53,29],18.6811=[63,34,52],18.6f6d=[63,48,68],18.87a2=[63,34,26],18.d9f3=[63,37,21],18.e287=[63,42,17],18.f99b=[63,41,69],18.1085a=[63,41,75],18.1a68c=[63,53,16],18.1cfb2=[63,41,34,29],18.23ce8=[63,54,73],18.28e77=[63,49,71],18.2fe8c=[63,59,75]} v10022) v1 ==== 525+0+0 (643107873 0 0) 0xcc24ba40 con 0xcc244780
  2137. -302> 2012-12-18 19:54:58.677986 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.677987 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2138. -301> 2012-12-18 19:54:58.677998 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 525 (0x8e4928 -> 0)
  2139. -300> 2012-12-18 19:54:58.686057 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1054684 (48159069 -> 49213753)
  2140. -299> 2012-12-18 19:54:58.686081 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1054684 (0 -> 1054684)
  2141. -298> 2012-12-18 19:54:58.706874 7f976f134700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 313303 (49213753 -> 49527056)
  2142. -297> 2012-12-18 19:54:58.706886 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 6 ==== pg_stats(3315 pgs tid 17279 v 10022) v1 ==== 1054684+0+0 (1490577503 0 0) 0xcbf2c900 con 0xcc244780
  2143. -296> 2012-12-18 19:54:58.706904 7f976f134700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 313303 (1054684 -> 1367987)
  2144. -295> 2012-12-18 19:54:58.706908 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.706909 lease_expire=0.000000 has v10022 lc 6066307
  2145. -294> 2012-12-18 19:54:58.706922 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1054684 (0x8e4928 -> 313303)
  2146. -293> 2012-12-18 19:54:58.710115 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.63 10.1.0.17:6809/30066 7 ==== log(1000 entries) v1 ==== 313303+0+0 (2523699661 0 0) 0xccffce00 con 0xcc244780
  2147. -292> 2012-12-18 19:54:58.710142 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:58.710144 lease_expire=0.000000 has v0 lc 5962552
  2148. -291> 2012-12-18 19:54:58.710157 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 313303 (0x8e4928 -> 0)
  2149. -290> 2012-12-18 19:54:58.711380 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (49527056 -> 49527078)
  2150. -289> 2012-12-18 19:54:58.711390 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (0 -> 22)
  2151. -288> 2012-12-18 19:54:58.711493 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1029 (49527078 -> 49528107)
  2152. -287> 2012-12-18 19:54:58.711568 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1029 (22 -> 1051)
  2153. -286> 2012-12-18 19:54:58.711565 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 13 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x5bef520 con 0x9a93400
  2154. -285> 2012-12-18 19:54:58.711590 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.711592 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2155. -284> 2012-12-18 19:54:58.711613 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 1029)
  2156. -283> 2012-12-18 19:54:58.711621 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 14 ==== osd_pgtemp(e10022 {0.52d=[],0.69e=[],1.29c=[],1.52c=[],1.69d=[],18.e59=[16,41,10],18.ecb=[16,51,55],18.2fda=[],18.51fe=[16,53,10],18.5a48=[16,34,29],18.6151=[],18.7999=[],18.8906=[16,56,65],18.d4ef=[16,51,68],18.df4e=[16,51,10],18.11e33=[],18.12507=[],18.1268c=[16,54,68],18.1314c=[],18.1453b=[16,41,52,68],18.1463c=[16,59,10],18.14741=[16,32,61],18.1c8ab=[55,46,64,16],18.1eb24=[],18.21050=[16,32,68],18.23397=[16,24,31,0],18.2589d=[16,73,38,3],18.25942=[],18.2916d=[16,52,64],18.2925a=[],18.2b4dd=[16,59,67],18.2cc8d=[16,73,11],18.2d3a7=[16,55,60],18.2ec3b=[16,44,10],18.2f650=[16,56,10]} v10022) v1 ==== 1029+0+0 (980354403 0 0) 0xcc24be00 con 0x9a93400
  2157. -282> 2012-12-18 19:54:58.711663 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.711668 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2158. -281> 2012-12-18 19:54:58.711680 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1029 (0x8e4928 -> 0)
  2159. -280> 2012-12-18 19:54:58.715348 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (49528107 -> 49528129)
  2160. -279> 2012-12-18 19:54:58.715372 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (0 -> 22)
  2161. -278> 2012-12-18 19:54:58.715420 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 789 (49528129 -> 49528918)
  2162. -277> 2012-12-18 19:54:58.715518 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 789 (22 -> 811)
  2163. -276> 2012-12-18 19:54:58.715487 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 9 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xcc57d6c0 con 0x9cd1b80
  2164. -275> 2012-12-18 19:54:58.715533 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.715535 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2165. -274> 2012-12-18 19:54:58.715555 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 789)
  2166. -273> 2012-12-18 19:54:58.715561 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 10 ==== osd_pgtemp(e10022 {18.4f9=[61,33,20],18.6807=[61,37,16],18.8c27=[61,53,69],18.8e9b=[61,50,24],18.ac2c=[61,40,53],18.115f0=[61,57,71],18.1374b=[61,46,21],18.17813=[61,59,69],18.1875f=[61,48,18],18.199a9=[61,54,74],18.1b1bd=[61,59,69],18.1fcef=[61,53,28],18.20f09=[61,50,74],18.21fd0=[61,44,70],18.2398f=[61,55,13],18.25d80=[61,42,36,28],18.25e76=[61,40,54],18.25f21=[61,34,22],18.268a5=[61,42,71],18.28368=[61,55,71],18.2a61b=[61,34,69],18.2dbcc=[61,47,16],18.2ff33=[61,52,56]} v10022) v1 ==== 789+0+0 (3805876721 0 0) 0xcb61a000 con 0x9cd1b80
  2167. -272> 2012-12-18 19:54:58.715596 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10022) is_readable now=2012-12-18 19:54:58.715597 lease_expire=2012-12-18 19:55:01.703181 has v10022 lc 10022
  2168. -271> 2012-12-18 19:54:58.715612 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 789 (0x8e4928 -> 0)
  2169. -270> 2012-12-18 19:54:58.720164 7f9774b80700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 573524 (49528918 -> 50102442)
  2170. -269> 2012-12-18 19:54:58.720177 7f9774b80700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 573524 (0 -> 573524)
  2171. -268> 2012-12-18 19:54:58.730879 7f9773e73700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (50102442 -> 50102484)
  2172. -267> 2012-12-18 19:54:58.730909 7f9773e73700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (573524 -> 573566)
  2173. -266> 2012-12-18 19:54:58.731014 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.72 10.1.0.18:6806/16967 12 ==== mon_subscribe({monmap=2+,osdmap=9327}) v2 ==== 42+0+0 (328051826 0 0) 0xaefe540 con 0x722c3c0
  2174. -265> 2012-12-18 19:54:58.731045 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9327..10022] to osd.72 10.1.0.18:6806/16967
  2175. -264> 2012-12-18 19:54:58.731956 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.72 10.1.0.18:6806/16967 -- osd_map(9327..9427 src has 7326..10022) v3 -- ?+0 0xccdc5200
  2176. -263> 2012-12-18 19:54:58.731989 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.72 10.1.0.18:6806/16967 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57cb60
  2177. -262> 2012-12-18 19:54:58.732037 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 50102442)
  2178. -261> 2012-12-18 19:54:58.732043 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 573524)
  2179. -260> 2012-12-18 19:54:58.732603 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.16 10.1.0.13:6818/10436 15 ==== pg_stats(1800 pgs tid 567088 v 10022) v1 ==== 573524+0+0 (4138568916 0 0) 0x5f32d80 con 0x9a93400
  2180. -259> 2012-12-18 19:54:58.732628 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.732629 lease_expire=0.000000 has v10022 lc 6066307
  2181. -258> 2012-12-18 19:54:58.732643 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 573524 (0x8e4928 -> 0)
  2182. -257> 2012-12-18 19:54:58.735118 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1102004 (50102442 -> 51204446)
  2183. -256> 2012-12-18 19:54:58.735135 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1102004 (0 -> 1102004)
  2184. -255> 2012-12-18 19:54:58.778733 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 11 ==== pg_stats(3463 pgs tid 17774 v 10022) v1 ==== 1102004+0+0 (1107955512 0 0) 0x69a6d80 con 0x9cd1b80
  2185. -254> 2012-12-18 19:54:58.778759 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.778760 lease_expire=0.000000 has v10022 lc 6066307
  2186. -253> 2012-12-18 19:54:58.778774 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1102004 (0x8e4928 -> 0)
  2187. -252> 2012-12-18 19:54:58.816670 7f976ef32700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffde00 sd=57 pgs=0 cs=0 l=0).accept sd=57
  2188. -251> 2012-12-18 19:54:58.818532 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (51204446 -> 51204503)
  2189. -250> 2012-12-18 19:54:58.818561 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2190. -249> 2012-12-18 19:54:58.818684 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (3628391085 0 0) 0xcdd3a000 con 0xcc2443c0
  2191. -248> 2012-12-18 19:54:58.818719 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.818721 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2192. -247> 2012-12-18 19:54:58.818762 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6812/18696 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc4c00 con 0xcc2443c0
  2193. -246> 2012-12-18 19:54:58.818782 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 51204446)
  2194. -245> 2012-12-18 19:54:58.818792 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2195. -244> 2012-12-18 19:54:58.824415 7f9772a5f700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 84 (0 -> 84)
  2196. -243> 2012-12-18 19:54:58.824578 7f977255a700 1 -- 10.1.0.11:6789/0 <== mon.2 10.1.0.15:6789/0 22 ==== paxos(osdmap accept lc 10022 fc 0 pn 129200 opn 0) v1 ==== 84+0+0 (3055545234 0 0) 0x772b900 con 0xb7b4dc0
  2197. -242> 2012-12-18 19:54:58.825634 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (51204446 -> 51204604)
  2198. -241> 2012-12-18 19:54:58.825645 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (84 -> 242)
  2199. -240> 2012-12-18 19:54:58.826945 7f976ed30700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffdc00 sd=60 pgs=0 cs=0 l=0).accept sd=60
  2200. -239> 2012-12-18 19:54:58.828665 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (51204604 -> 51204661)
  2201. -238> 2012-12-18 19:54:58.828675 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (242 -> 299)
  2202. -237> 2012-12-18 19:54:58.837845 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.1 10.1.0.13:6789/0 -- paxos(osdmap commit lc 10023 fc 0 pn 129200 opn 0) v1 -- ?+0 0x84af180
  2203. -236> 2012-12-18 19:54:58.837870 7f977255a700 1 -- 10.1.0.11:6789/0 --> mon.2 10.1.0.15:6789/0 -- paxos(osdmap commit lc 10023 fc 0 pn 129200 opn 0) v1 -- ?+0 0x6086500
  2204. -235> 2012-12-18 19:54:58.837916 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 84 (0x8e4928 -> 215)
  2205. -234> 2012-12-18 19:54:58.837921 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (1003870309 0 0) 0xcdd3a400 con 0xcc2443c0
  2206. -233> 2012-12-18 19:54:58.837931 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.837932 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2207. -232> 2012-12-18 19:54:58.838143 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.15:6812/18696 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3a000 con 0xcc2443c0
  2208. -231> 2012-12-18 19:54:58.838209 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 51204503)
  2209. -230> 2012-12-18 19:54:58.838217 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 57)
  2210. -229> 2012-12-18 19:54:58.838224 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (2850087469 0 0) 0xcdd3a600 con 0xcc571cc0
  2211. -228> 2012-12-18 19:54:58.838244 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.838246 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2212. -227> 2012-12-18 19:54:58.838292 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6818/30353 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc4a00 con 0xcc571cc0
  2213. -226> 2012-12-18 19:54:58.838341 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 51204446)
  2214. -225> 2012-12-18 19:54:58.838349 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2215. -224> 2012-12-18 19:54:58.839253 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (51204446 -> 51204604)
  2216. -223> 2012-12-18 19:54:58.839292 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (0 -> 158)
  2217. -222> 2012-12-18 19:54:58.839403 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (694681583 0 0) 0xcdd3a000 con 0xcc571cc0
  2218. -221> 2012-12-18 19:54:58.839433 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.839436 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2219. -220> 2012-12-18 19:54:58.839641 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6818/30353 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3a600 con 0xcc571cc0
  2220. -219> 2012-12-18 19:54:58.839707 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 51204446)
  2221. -218> 2012-12-18 19:54:58.839715 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 0)
  2222. -217> 2012-12-18 19:54:58.840579 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (51204446 -> 51204488)
  2223. -216> 2012-12-18 19:54:58.840587 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2224. -215> 2012-12-18 19:54:58.840679 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (51204488 -> 51204510)
  2225. -214> 2012-12-18 19:54:58.840684 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  2226. -213> 2012-12-18 19:54:58.840693 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 3 ==== mon_subscribe({monmap=2+,osdmap=9288}) v2 ==== 42+0+0 (1056589270 0 0) 0x707a000 con 0xcc571cc0
  2227. -212> 2012-12-18 19:54:58.840726 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9288..10022] to osd.66 10.1.0.17:6818/30353
  2228. -211> 2012-12-18 19:54:58.840777 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 872 (51204510 -> 51205382)
  2229. -210> 2012-12-18 19:54:58.840782 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 872 (64 -> 936)
  2230. -209> 2012-12-18 19:54:58.841670 7f977487d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 23 (51205382 -> 51205405)
  2231. -208> 2012-12-18 19:54:58.841678 7f977487d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 23 (936 -> 959)
  2232. -207> 2012-12-18 19:54:58.841907 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.66 10.1.0.17:6818/30353 -- osd_map(9288..9388 src has 7326..10022) v3 -- ?+0 0xcdd3a000
  2233. -206> 2012-12-18 19:54:58.841973 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.66 10.1.0.17:6818/30353 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57c9c0
  2234. -205> 2012-12-18 19:54:58.841990 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 51205363)
  2235. -204> 2012-12-18 19:54:58.842000 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 917)
  2236. -203> 2012-12-18 19:54:58.842008 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 4 ==== osd_alive(want up_thru 10020 have 10022) v1 ==== 22+0+0 (4084230999 0 0) 0xcc57d860 con 0xcc571cc0
  2237. -202> 2012-12-18 19:54:58.842024 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.842026 lease_expire=0.000000 has v10022 lc 10023
  2238. -201> 2012-12-18 19:54:58.842067 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 895)
  2239. -200> 2012-12-18 19:54:58.842073 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503630 10.1.0.246:0/1001758 4 ==== mon_subscribe({monmap=2+}) v2 ==== 23+0+0 (2366214315 0 0) 0x707a380 con 0x722de00
  2240. -199> 2012-12-18 19:54:58.842096 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7503630 10.1.0.246:0/1001758 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57c820
  2241. -198> 2012-12-18 19:54:58.842118 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 23 (0x8e4928 -> 51205340)
  2242. -197> 2012-12-18 19:54:58.842126 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 23 (0x8e4928 -> 872)
  2243. -196> 2012-12-18 19:54:58.842133 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 5 ==== osd_pgtemp(e10022 {0.9f9=[66,41,75,54],18.5746=[66,55,47],18.6e81=[66,46,26],18.7bd2=[66,58,22],18.8218=[66,50,68],18.82b8=[66,48,17],18.8f31=[66,44,50],18.93b4=[66,55,25],18.b9e8=[66,40,18],18.e7be=[66,44,14],18.10e20=[66,40,11],18.12de0=[66,37,72],18.19392=[66,51,37,59],18.1951f=[],18.1b337=[66,40,68,44],18.1e77c=[66,38,73],18.21d8a=[66,46,74],18.21f30=[66,41,16],18.230a4=[66,48,25],18.23358=[],18.254f7=[66,52,73],18.26a73=[66,43,73],18.26b40=[66,59,57],18.27fc7=[66,55,73],18.2e020=[66,43,53],18.2fc5b=[66,55,69]} v10022) v1 ==== 872+0+0 (3094629281 0 0) 0x9cdd4a0 con 0xcc571cc0
  2244. -195> 2012-12-18 19:54:58.842198 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.842200 lease_expire=0.000000 has v10022 lc 10023
  2245. -194> 2012-12-18 19:54:58.842214 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 872 (0x8e4928 -> 0)
  2246. -193> 2012-12-18 19:54:58.844653 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (51205340 -> 51205382)
  2247. -192> 2012-12-18 19:54:58.844673 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2248. -191> 2012-12-18 19:54:58.844719 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (51205382 -> 51205404)
  2249. -190> 2012-12-18 19:54:58.844722 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (42 -> 64)
  2250. -189> 2012-12-18 19:54:58.844758 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 458 (51205404 -> 51205862)
  2251. -188> 2012-12-18 19:54:58.844762 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 458 (64 -> 522)
  2252. -187> 2012-12-18 19:54:58.844745 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 3 ==== mon_subscribe({monmap=2+,osdmap=9287}) v2 ==== 42+0+0 (1105819995 0 0) 0x707a540 con 0xcc2443c0
  2253. -186> 2012-12-18 19:54:58.844779 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9287..10022] to osd.25 10.1.0.15:6812/18696
  2254. -185> 2012-12-18 19:54:58.846134 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.25 10.1.0.15:6812/18696 -- osd_map(9287..9387 src has 7326..10022) v3 -- ?+0 0xccdc4600
  2255. -184> 2012-12-18 19:54:58.846184 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.25 10.1.0.15:6812/18696 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57c680
  2256. -183> 2012-12-18 19:54:58.846200 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 51205820)
  2257. -182> 2012-12-18 19:54:58.846208 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 480)
  2258. -181> 2012-12-18 19:54:58.846216 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x1968ea0 con 0xcc2443c0
  2259. -180> 2012-12-18 19:54:58.846232 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.846234 lease_expire=0.000000 has v10022 lc 10023
  2260. -179> 2012-12-18 19:54:58.846250 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 458)
  2261. -178> 2012-12-18 19:54:58.846256 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 5 ==== osd_pgtemp(e10022 {18.1c83=[25,40,14],18.59a1=[25,33,73],18.878c=[25,59,62],18.955d=[],18.9613=[25,57,63],18.ffde=[],18.15806=[],18.16ac5=[],18.1899d=[25,58,68],18.1b427=[],18.1f387=[],18.1f938=[],18.292dd=[25,46,64],18.292f3=[],18.29374=[25,46,74],18.29c43=[25,32,74]} v10022) v1 ==== 458+0+0 (1896413211 0 0) 0xcddbe1e0 con 0xcc2443c0
  2262. -177> 2012-12-18 19:54:58.846297 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.846298 lease_expire=0.000000 has v10022 lc 10023
  2263. -176> 2012-12-18 19:54:58.846310 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 458 (0x8e4928 -> 0)
  2264. -175> 2012-12-18 19:54:58.850487 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1070844 (51205820 -> 52276664)
  2265. -174> 2012-12-18 19:54:58.850504 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1070844 (0 -> 1070844)
  2266. -174> 2012-12-18 19:54:58.852834 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 630868 (52276664 -> 52907532)
  2267. -173> 2012-12-18 19:54:58.852846 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 630868 (1070844 -> 1701712)
  2268. -172> 2012-12-18 19:54:58.870476 7f976ef32700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1492 (52907532 -> 52909024)
  2269. -171> 2012-12-18 19:54:58.870515 7f976ef32700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1492 (1701712 -> 1703204)
  2270. -170> 2012-12-18 19:54:58.870480 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 6 ==== pg_stats(1980 pgs tid 611902 v 10022) v1 ==== 630868+0+0 (249774854 0 0) 0x7072000 con 0xcc2443c0
  2271. -169> 2012-12-18 19:54:58.870524 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.870528 lease_expire=0.000000 has v10022 lc 6066307
  2272. -168> 2012-12-18 19:54:58.870545 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 630868 (0x8e4928 -> 1072336)
  2273. -167> 2012-12-18 19:54:58.870551 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.25 10.1.0.15:6812/18696 7 ==== log(6 entries) v1 ==== 1492+0+0 (3898086563 0 0) 0xcdd3a200 con 0xcc2443c0
  2274. -166> 2012-12-18 19:54:58.870566 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:58.870567 lease_expire=0.000000 has v0 lc 5962552
  2275. -165> 2012-12-18 19:54:58.870576 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1492 (0x8e4928 -> 1070844)
  2276. -164> 2012-12-18 19:54:58.877438 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 6 ==== pg_stats(3365 pgs tid 17744 v 10022) v1 ==== 1070844+0+0 (3979919988 0 0) 0x7072240 con 0xcc571cc0
  2277. -163> 2012-12-18 19:54:58.877447 7f976ed30700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 297430 (52909024 -> 53206454)
  2278. -162> 2012-12-18 19:54:58.877459 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.877461 lease_expire=0.000000 has v10022 lc 6066307
  2279. -161> 2012-12-18 19:54:58.877470 7f976ed30700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 297430 (1070844 -> 1368274)
  2280. -160> 2012-12-18 19:54:58.877491 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1070844 (0x8e4928 -> 297430)
  2281. -159> 2012-12-18 19:54:58.880488 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.66 10.1.0.17:6818/30353 7 ==== log(1000 entries) v1 ==== 297430+0+0 (2450690300 0 0) 0xccdc4c00 con 0xcc571cc0
  2282. -158> 2012-12-18 19:54:58.880510 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:58.880512 lease_expire=0.000000 has v0 lc 5962552
  2283. -157> 2012-12-18 19:54:58.880523 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 297430 (0x8e4928 -> 0)
  2284. -156> 2012-12-18 19:54:58.882533 7f9773f74700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (53206454 -> 53206496)
  2285. -155> 2012-12-18 19:54:58.882542 7f9773f74700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2286. -154> 2012-12-18 19:54:58.882641 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.11 10.1.0.13:6803/7344 10 ==== mon_subscribe({monmap=2+,osdmap=9993}) v2 ==== 42+0+0 (3225562490 0 0) 0x883f340 con 0x722db80
  2287. -153> 2012-12-18 19:54:58.882670 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9993..10022] to osd.11 10.1.0.13:6803/7344
  2288. -152> 2012-12-18 19:54:58.882945 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.11 10.1.0.13:6803/7344 -- osd_map(9993..10022 src has 7326..10022) v3 -- ?+0 0xcdd3bc00
  2289. -151> 2012-12-18 19:54:58.882991 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.11 10.1.0.13:6803/7344 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31860
  2290. -150> 2012-12-18 19:54:58.882999 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 53206454)
  2291. -149> 2012-12-18 19:54:58.883024 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  2292. -148> 2012-12-18 19:54:58.888579 7f977094c700 2 -- 10.1.0.11:6789/0 >> 10.1.0.12:6801/17968 pipe(0x7228600 sd=35 pgs=27829 cs=1 l=1).reader couldn't read tag, Success
  2293. -147> 2012-12-18 19:54:58.888652 7f977094c700 2 -- 10.1.0.11:6789/0 >> 10.1.0.12:6801/17968 pipe(0x7228600 sd=35 pgs=27829 cs=1 l=1).fault 0: Success
  2294. -146> 2012-12-18 19:54:58.904283 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (53206454 -> 53206476)
  2295. -145> 2012-12-18 19:54:58.904321 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (0 -> 22)
  2296. -144> 2012-12-18 19:54:58.904380 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 903 (53206476 -> 53207379)
  2297. -143> 2012-12-18 19:54:58.904383 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 903 (22 -> 925)
  2298. -142> 2012-12-18 19:54:58.904393 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 9 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xcbf31860 con 0x9cd0780
  2299. -141> 2012-12-18 19:54:58.904419 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.904421 lease_expire=0.000000 has v10022 lc 10023
  2300. -140> 2012-12-18 19:54:58.904442 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 903)
  2301. -139> 2012-12-18 19:54:58.904476 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 10 ==== osd_pgtemp(e10022 {18.583=[13,57,54],18.37b9=[],18.599d=[13,40,10],18.909e=[13,33,21],18.a305=[13,32,24],18.c608=[13,45,25],18.d712=[13,30,75],18.107db=[13,40,10],18.10cfc=[13,59,75],18.11d29=[13,72,10],18.13a45=[],18.15340=[],18.16751=[13,58,56],18.17ab3=[],18.18967=[13,38,69],18.1d9bf=[13,29,10],18.1e457=[],18.1e8cf=[],18.1ea76=[13,40,61],18.22351=[],18.23ff0=[13,37,38],18.2400c=[13,43,30],18.26c3b=[13,30,71],18.27538=[13,38,10],18.2790b=[13,34,32],18.2896d=[13,29,10],18.2994c=[13,47,60,40],18.2b4eb=[13,69,68],18.2f6bb=[13,56,62]} v10022) v1 ==== 903+0+0 (2521837416 0 0) 0xcb61a960 con 0x9cd0780
  2302. -138> 2012-12-18 19:54:58.904527 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:58.904528 lease_expire=0.000000 has v10022 lc 10023
  2303. -137> 2012-12-18 19:54:58.904538 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 903 (0x8e4928 -> 0)
  2304. -136> 2012-12-18 19:54:58.915430 7f9770346700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 562860 (53207379 -> 53770239)
  2305. -135> 2012-12-18 19:54:58.915463 7f9770346700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 562860 (0 -> 562860)
  2306. -134> 2012-12-18 19:54:58.924941 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.13 10.1.0.13:6809/8019 11 ==== pg_stats(1766 pgs tid 584545 v 10022) v1 ==== 562860+0+0 (126852414 0 0) 0xcbf2c240 con 0x9cd0780
  2307. -133> 2012-12-18 19:54:58.924972 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:58.924974 lease_expire=0.000000 has v10022 lc 6066307
  2308. -132> 2012-12-18 19:54:58.924988 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 562860 (0x8e4928 -> 0)
  2309. -131> 2012-12-18 19:54:58.971977 7f977094c700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffda00 sd=35 pgs=0 cs=0 l=0).accept sd=35
  2310. -130> 2012-12-18 19:54:58.972562 7f977094c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 65 (53770239 -> 53770304)
  2311. -129> 2012-12-18 19:54:58.972591 7f977094c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 65 (0 -> 65)
  2312. -128> 2012-12-18 19:54:58.972687 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503690 10.1.0.142:0/2005396 1 ==== auth(proto 0 35 bytes epoch 1) v1 ==== 65+0+0 (2809612803 0 0) 0x700a600 con 0xcc571900
  2313. -127> 2012-12-18 19:54:58.972724 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.972726 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2314. -126> 2012-12-18 19:54:58.972765 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.142:0/2005396 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3b600 con 0xcc571900
  2315. -125> 2012-12-18 19:54:58.972783 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 65 (0x8e4928 -> 53770239)
  2316. -124> 2012-12-18 19:54:58.972792 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 65 (0x8e4928 -> 0)
  2317. -123> 2012-12-18 19:54:58.973719 7f977094c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 174 (53770239 -> 53770413)
  2318. -122> 2012-12-18 19:54:58.973729 7f977094c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 174 (0 -> 174)
  2319. -121> 2012-12-18 19:54:58.973822 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503690 10.1.0.142:0/2005396 2 ==== auth(proto 2 144 bytes epoch 0) v1 ==== 174+0+0 (3561864937 0 0) 0x9cb8a00 con 0xcc571900
  2320. -120> 2012-12-18 19:54:58.973855 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.973858 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2321. -119> 2012-12-18 19:54:58.974056 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.142:0/2005396 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0x700a600 con 0xcc571900
  2322. -118> 2012-12-18 19:54:58.974098 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 174 (0x8e4928 -> 53770239)
  2323. -117> 2012-12-18 19:54:58.974106 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 174 (0x8e4928 -> 0)
  2324. -116> 2012-12-18 19:54:58.974783 7f977094c700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (53770239 -> 53770281)
  2325. -115> 2012-12-18 19:54:58.974793 7f977094c700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2326. -114> 2012-12-18 19:54:58.974906 7f977255a700 1 -- 10.1.0.11:6789/0 <== client.7503690 10.1.0.142:0/2005396 3 ==== mon_subscribe({monmap=2+,osdmap=10020}) v2 ==== 42+0+0 (3101158440 0 0) 0xce3e4000 con 0xcc571900
  2327. -113> 2012-12-18 19:54:58.974945 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [10020..10022] to client.7503690 10.1.0.142:0/2005396
  2328. -112> 2012-12-18 19:54:58.974989 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7503690 10.1.0.142:0/2005396 -- osd_map(10020..10022 src has 7326..10022) v3 -- ?+0 0x9cb8a00
  2329. -111> 2012-12-18 19:54:58.975017 7f977255a700 1 -- 10.1.0.11:6789/0 --> client.7503690 10.1.0.142:0/2005396 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcbf31040
  2330. -110> 2012-12-18 19:54:58.975033 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 53770239)
  2331. -109> 2012-12-18 19:54:58.975048 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  2332. -108> 2012-12-18 19:54:58.991303 7f976eb2e700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffd800 sd=59 pgs=0 cs=0 l=0).accept sd=59
  2333. -107> 2012-12-18 19:54:58.991342 7f976ea2d700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffd600 sd=61 pgs=0 cs=0 l=0).accept sd=61
  2334. -106> 2012-12-18 19:54:58.993002 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (53770239 -> 53770296)
  2335. -105> 2012-12-18 19:54:58.993030 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2336. -104> 2012-12-18 19:54:58.993175 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (2869221074 0 0) 0x700a600 con 0xcdd3db80
  2337. -103> 2012-12-18 19:54:58.993202 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.993204 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2338. -102> 2012-12-18 19:54:58.993244 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6815/30258 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3b200 con 0xcdd3db80
  2339. -101> 2012-12-18 19:54:58.993262 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 53770239)
  2340. -100> 2012-12-18 19:54:58.993268 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2341. -99> 2012-12-18 19:54:58.994399 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (53770239 -> 53770296)
  2342. -98> 2012-12-18 19:54:58.994408 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2343. -97> 2012-12-18 19:54:58.994492 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (2888338387 0 0) 0xcdd3b600 con 0xcc571680
  2344. -96> 2012-12-18 19:54:58.994528 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.994530 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2345. -95> 2012-12-18 19:54:58.994569 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6800/29876 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3b000 con 0xcc571680
  2346. -94> 2012-12-18 19:54:58.994587 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 53770239)
  2347. -93> 2012-12-18 19:54:58.994600 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2348. -92> 2012-12-18 19:54:58.994770 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (53770239 -> 53770281)
  2349. -91> 2012-12-18 19:54:58.994779 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (0 -> 42)
  2350. -90> 2012-12-18 19:54:58.994851 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 12 ==== mon_subscribe({monmap=2+,osdmap=9920}) v2 ==== 42+0+0 (2996390116 0 0) 0xcdd38540 con 0x9cd1b80
  2351. -89> 2012-12-18 19:54:58.994879 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9920..10022] to osd.61 10.1.0.17:6803/29184
  2352. -88> 2012-12-18 19:54:58.995735 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.61 10.1.0.17:6803/29184 -- osd_map(9920..10020 src has 7326..10022) v3 -- ?+0 0xcdd3b600
  2353. -87> 2012-12-18 19:54:58.995777 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.61 10.1.0.17:6803/29184 -- mon_subscribe_ack(300s) v1 -- ?+0 0x5bb1520
  2354. -86> 2012-12-18 19:54:58.995803 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 53770239)
  2355. -85> 2012-12-18 19:54:58.995807 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 0)
  2356. -84> 2012-12-18 19:54:58.996836 7f9771356700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 288937 (53770239 -> 54059176)
  2357. -83> 2012-12-18 19:54:58.996846 7f9771356700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 288937 (0 -> 288937)
  2358. -82> 2012-12-18 19:54:58.999149 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (54059176 -> 54059334)
  2359. -81> 2012-12-18 19:54:58.999162 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (288937 -> 289095)
  2360. -80> 2012-12-18 19:54:58.999287 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (702420452 0 0) 0xcdd3a600 con 0xcdd3db80
  2361. -79> 2012-12-18 19:54:58.999307 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:58.999336 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2362. -78> 2012-12-18 19:54:58.999536 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6815/30258 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xccdc4800 con 0xcdd3db80
  2363. -77> 2012-12-18 19:54:58.999579 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 54059176)
  2364. -76> 2012-12-18 19:54:58.999584 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 288937)
  2365. -75> 2012-12-18 19:54:59.000188 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (54059176 -> 54059218)
  2366. -74> 2012-12-18 19:54:59.000198 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (288937 -> 288979)
  2367. -73> 2012-12-18 19:54:59.000286 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (54059218 -> 54059240)
  2368. -72> 2012-12-18 19:54:59.000315 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (288979 -> 289001)
  2369. -71> 2012-12-18 19:54:59.000335 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 777 (54059240 -> 54060017)
  2370. -70> 2012-12-18 19:54:59.000337 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 777 (289001 -> 289778)
  2371. -69> 2012-12-18 19:54:59.000328 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 3 ==== mon_subscribe({monmap=2+,osdmap=9291}) v2 ==== 42+0+0 (659900927 0 0) 0xcda43880 con 0xcdd3db80
  2372. -68> 2012-12-18 19:54:59.000355 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9291..10022] to osd.65 10.1.0.17:6815/30258
  2373. -67> 2012-12-18 19:54:59.000768 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 158 (54060017 -> 54060175)
  2374. -66> 2012-12-18 19:54:59.000787 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 158 (289778 -> 289936)
  2375. -65> 2012-12-18 19:54:59.001520 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.65 10.1.0.17:6815/30258 -- osd_map(9291..9391 src has 7326..10022) v3 -- ?+0 0xcdd3a600
  2376. -64> 2012-12-18 19:54:59.001571 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.65 10.1.0.17:6815/30258 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57dd40
  2377. -63> 2012-12-18 19:54:59.001619 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 54060133)
  2378. -62> 2012-12-18 19:54:59.001624 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 289894)
  2379. -61> 2012-12-18 19:54:59.001627 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0x11c46680 con 0xcdd3db80
  2380. -60> 2012-12-18 19:54:59.001636 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:59.001637 lease_expire=0.000000 has v10022 lc 10023
  2381. -59> 2012-12-18 19:54:59.001647 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 289872)
  2382. -58> 2012-12-18 19:54:59.001651 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 2 ==== auth(proto 2 128 bytes epoch 0) v1 ==== 158+0+0 (3895485612 0 0) 0xccdc4800 con 0xcc571680
  2383. -57> 2012-12-18 19:54:59.001661 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:59.001662 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2384. -56> 2012-12-18 19:54:59.001839 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.17:6800/29876 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3be00 con 0xcc571680
  2385. -55> 2012-12-18 19:54:59.001884 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 158 (0x8e4928 -> 54059975)
  2386. -54> 2012-12-18 19:54:59.001891 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 158 (0x8e4928 -> 289714)
  2387. -53> 2012-12-18 19:54:59.001899 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 5 ==== osd_pgtemp(e10022 {18.1529=[65,47,12],18.2b83=[65,38,71],18.2e44=[65,56,72],18.43ac=[65,51,72],18.4d7d=[65,38,75],18.5caf=[65,50,21],18.5cc9=[65,50,69],18.705a=[65,47,22],18.b107=[65,54,14],18.d3d7=[65,59,43],18.d552=[65,49,70],18.e59b=[65,48,20],18.154ac=[65,48,70],18.1a517=[65,55,69],18.214d6=[65,54,70],18.22354=[],18.23b96=[65,47,74],18.26801=[65,54,28],18.28547=[65,31,44,55],18.292e6=[65,38,14],18.298da=[65,50,32],18.2b28f=[65,53,70],18.300e6=[65,59,73]} v10022) v1 ==== 777+0+0 (543787006 0 0) 0xcddbf680 con 0xcdd3db80
  2388. -52> 2012-12-18 19:54:59.001956 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:59.001958 lease_expire=0.000000 has v10022 lc 10023
  2389. -51> 2012-12-18 19:54:59.001969 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 777 (0x8e4928 -> 288937)
  2390. -50> 2012-12-18 19:54:59.002557 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 42 (54059975 -> 54060017)
  2391. -49> 2012-12-18 19:54:59.002567 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 42 (288937 -> 288979)
  2392. -48> 2012-12-18 19:54:59.002647 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 22 (54060017 -> 54060039)
  2393. -47> 2012-12-18 19:54:59.002652 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 22 (288979 -> 289001)
  2394. -46> 2012-12-18 19:54:59.002694 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 732 (54060039 -> 54060771)
  2395. -45> 2012-12-18 19:54:59.002697 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 732 (289001 -> 289733)
  2396. -44> 2012-12-18 19:54:59.002660 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 3 ==== mon_subscribe({monmap=2+,osdmap=9264}) v2 ==== 42+0+0 (3361159853 0 0) 0xaefe540 con 0xcc571680
  2397. -43> 2012-12-18 19:54:59.002715 7f977255a700 5 mon.a@0(leader).osd e10022 send_incremental [9264..10022] to osd.60 10.1.0.17:6800/29876
  2398. -42> 2012-12-18 19:54:59.004203 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.60 10.1.0.17:6800/29876 -- osd_map(9264..9364 src has 7326..10022) v3 -- ?+0 0xccdc4800
  2399. -41> 2012-12-18 19:54:59.004238 7f977255a700 1 -- 10.1.0.11:6789/0 --> osd.60 10.1.0.17:6800/29876 -- mon_subscribe_ack(300s) v1 -- ?+0 0xcc57dba0
  2400. -40> 2012-12-18 19:54:59.004254 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 42 (0x8e4928 -> 54060729)
  2401. -39> 2012-12-18 19:54:59.004268 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 42 (0x8e4928 -> 289691)
  2402. -38> 2012-12-18 19:54:59.004276 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 4 ==== osd_alive(want up_thru 10019 have 10022) v1 ==== 22+0+0 (3933262749 0 0) 0xce4aa000 con 0xcc571680
  2403. -37> 2012-12-18 19:54:59.004293 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:59.004295 lease_expire=0.000000 has v10022 lc 10023
  2404. -36> 2012-12-18 19:54:59.004312 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 22 (0x8e4928 -> 289669)
  2405. -35> 2012-12-18 19:54:59.004318 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.61 10.1.0.17:6803/29184 13 ==== log(1000 entries) v1 ==== 288937+0+0 (2062592166 0 0) 0xcdd3be00 con 0x9cd1b80
  2406. -34> 2012-12-18 19:54:59.004335 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:59.004336 lease_expire=0.000000 has v0 lc 5962552
  2407. -33> 2012-12-18 19:54:59.004350 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 288937 (0x8e4928 -> 732)
  2408. -32> 2012-12-18 19:54:59.004361 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 5 ==== osd_pgtemp(e10022 {18.26cb=[60,52,75],18.2efc=[60,49,73],18.3c9b=[60,56,74],18.4856=[60,52,69],18.539f=[60,59,69],18.66bb=[60,52,26],18.9ed1=[60,49,71],18.a5c0=[60,57,73],18.a6bb=[38,51,66,60],18.a92d=[60,48,10],18.13453=[],18.1f32f=[60,56,11],18.1f80e=[],18.209c3=[60,57,15],18.21a8d=[60,53,72],18.2212f=[60,57,75],18.25247=[60,56,21],18.25b3e=[60,54,19],18.2826a=[60,57,24],18.2bfce=[60,34,71],18.2c36b=[60,49,22],18.2d10d=[60,58,71]} v10022) v1 ==== 732+0+0 (71591500 0 0) 0xcddbf860 con 0xcc571680
  2409. -31> 2012-12-18 19:54:59.004417 7f977255a700 1 mon.a@0(leader).paxos(osdmap updating c 7326..10023) is_readable now=2012-12-18 19:54:59.004419 lease_expire=0.000000 has v10022 lc 10023
  2410. -30> 2012-12-18 19:54:59.004431 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 732 (0x8e4928 -> 0)
  2411. -29> 2012-12-18 19:54:59.011135 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1057512 (54060729 -> 55118241)
  2412. -28> 2012-12-18 19:54:59.011161 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1057512 (0 -> 1057512)
  2413. -27> 2012-12-18 19:54:59.013609 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 1068272 (55118241 -> 56186513)
  2414. -26> 2012-12-18 19:54:59.013618 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 1068272 (1057512 -> 2125784)
  2415. -25> 2012-12-18 19:54:59.027835 7f9771558700 -1 msg/SimpleMessenger.cc: In function 'int SimpleMessenger::Pipe::write_message(Message*)' thread 7f9771558700 time 2012-12-18 19:54:59.027255
  2416. msg/SimpleMessenger.cc: 2147: FAILED assert(left == 0)
  2417.  
  2418. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  2419. 1: (SimpleMessenger::Pipe::write_message(Message*)+0xb7f) [0x5ed51f]
  2420. 2: (SimpleMessenger::Pipe::writer()+0x458) [0x600f48]
  2421. 3: (SimpleMessenger::Pipe::Writer::entry()+0xd) [0x5c40dd]
  2422. 4: (()+0x6b50) [0x7f977a959b50]
  2423. 5: (clone()+0x6d) [0x7f977930b6dd]
  2424. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  2425.  
  2426. -24> 2012-12-18 19:54:59.036725 7f976e72a700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffd400 sd=62 pgs=0 cs=0 l=0).accept sd=62
  2427. -23> 2012-12-18 19:54:59.040458 7f976ea2d700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 287450 (56186513 -> 56473963)
  2428. -22> 2012-12-18 19:54:59.040470 7f976ea2d700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 287450 (2125784 -> 2413234)
  2429. -21> 2012-12-18 19:54:59.040455 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 6 ==== pg_stats(3322 pgs tid 17284 v 10022) v1 ==== 1057512+0+0 (25665014 0 0) 0x7073b00 con 0xcdd3db80
  2430. -20> 2012-12-18 19:54:59.040476 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:59.040478 lease_expire=0.000000 has v10022 lc 6066307
  2431. -19> 2012-12-18 19:54:59.040496 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1057512 (0x8e4928 -> 1355722)
  2432. -18> 2012-12-18 19:54:59.041170 7f976eb2e700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 288669 (56473963 -> 56762632)
  2433. -17> 2012-12-18 19:54:59.041178 7f976eb2e700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 288669 (1355722 -> 1644391)
  2434. -16> 2012-12-18 19:54:59.041188 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 6 ==== pg_stats(3357 pgs tid 17889 v 10022) v1 ==== 1068272+0+0 (3724293600 0 0) 0x7073d40 con 0xcc571680
  2435. -15> 2012-12-18 19:54:59.041213 7f977255a700 1 mon.a@0(leader).paxos(pgmap updating c 6065800..6066307) is_readable now=2012-12-18 19:54:59.041215 lease_expire=0.000000 has v10022 lc 6066307
  2436. -14> 2012-12-18 19:54:59.041240 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 1068272 (0x8e4928 -> 576119)
  2437. -13> 2012-12-18 19:54:59.045318 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.65 10.1.0.17:6815/30258 7 ==== log(1000 entries) v1 ==== 287450+0+0 (3958981749 0 0) 0x9cb9200 con 0xcdd3db80
  2438. -12> 2012-12-18 19:54:59.045343 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:59.045345 lease_expire=0.000000 has v0 lc 5962552
  2439. -11> 2012-12-18 19:54:59.045363 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 287450 (0x8e4928 -> 288669)
  2440. -10> 2012-12-18 19:54:59.045732 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.60 10.1.0.17:6800/29876 7 ==== log(1000 entries) v1 ==== 288669+0+0 (3197194104 0 0) 0xcdd3b200 con 0xcc571680
  2441. -9> 2012-12-18 19:54:59.045756 7f977255a700 1 mon.a@0(leader).paxos(logm updating c 5962045..5962552) is_readable now=2012-12-18 19:54:59.045758 lease_expire=0.000000 has v0 lc 5962552
  2442. -8> 2012-12-18 19:54:59.045774 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 288669 (0x8e4928 -> 0)
  2443. -7> 2012-12-18 19:54:59.048794 7f976e72a700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (56762632 -> 56762689)
  2444. -6> 2012-12-18 19:54:59.048803 7f976e72a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2445. -5> 2012-12-18 19:54:59.048891 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.31 10.1.0.12:6804/18035 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (2581307537 0 0) 0xcdd3ba00 con 0xcdd3d7c0
  2446. -4> 2012-12-18 19:54:59.048926 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:59.048929 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2447. -3> 2012-12-18 19:54:59.048978 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.12:6804/18035 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3aa00 con 0xcdd3d7c0
  2448. -2> 2012-12-18 19:54:59.048996 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 56762632)
  2449. -1> 2012-12-18 19:54:59.049010 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2450. 0> 2012-12-18 19:54:59.061870 7f976e528700 1 -- 10.1.0.11:6789/0 >> :/0 pipe(0xccffd200 sd=63 pgs=0 cs=0 l=0).accept sd=63
  2451. --- end dump of recent events ---
  2452. 2012-12-18 19:54:59.065446 7f9771558700 -1 *** Caught signal (Aborted) **
  2453. in thread 7f9771558700
  2454.  
  2455. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  2456. 1: /usr/bin/ceph-mon() [0x54a532]
  2457. 2: (()+0xf030) [0x7f977a962030]
  2458. 3: (gsignal()+0x35) [0x7f9779265475]
  2459. 4: (abort()+0x180) [0x7f97792686f0]
  2460. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f9779ab768d]
  2461. 6: (()+0x63796) [0x7f9779ab5796]
  2462. 7: (()+0x637c3) [0x7f9779ab57c3]
  2463. 8: (()+0x639ee) [0x7f9779ab59ee]
  2464. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3c1) [0x6129b1]
  2465. 10: (SimpleMessenger::Pipe::write_message(Message*)+0xb7f) [0x5ed51f]
  2466. 11: (SimpleMessenger::Pipe::writer()+0x458) [0x600f48]
  2467. 12: (SimpleMessenger::Pipe::Writer::entry()+0xd) [0x5c40dd]
  2468. 13: (()+0x6b50) [0x7f977a959b50]
  2469. 14: (clone()+0x6d) [0x7f977930b6dd]
  2470. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  2471.  
  2472. --- begin dump of recent events ---
  2473. -7> 2012-12-18 19:54:59.065137 7f976e528700 5 throttle(mon_client_bytes 0x7fffff0f3230) get 57 (56762632 -> 56762689)
  2474. -6> 2012-12-18 19:54:59.065146 7f976e528700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) get 57 (0 -> 57)
  2475. -5> 2012-12-18 19:54:59.065238 7f977255a700 1 -- 10.1.0.11:6789/0 <== osd.69 10.1.0.18:6818/19017 1 ==== auth(proto 0 27 bytes epoch 1) v1 ==== 57+0+0 (1216219163 0 0) 0xce438000 con 0xcdd3d400
  2476. -4> 2012-12-18 19:54:59.065274 7f977255a700 1 mon.a@0(leader).paxos(auth updating c 78981..79006) is_readable now=2012-12-18 19:54:59.065276 lease_expire=2012-12-18 19:55:01.849545 has v0 lc 79006
  2477. -3> 2012-12-18 19:54:59.065316 7f977255a700 1 -- 10.1.0.11:6789/0 --> 10.1.0.18:6818/19017 -- auth_reply(proto 2 0 Success) v1 -- ?+0 0xcdd3a800 con 0xcdd3d400
  2478. -2> 2012-12-18 19:54:59.065360 7f977255a700 5 throttle(mon_client_bytes 0x7fffff0f3230) put 57 (0x8e4928 -> 56762632)
  2479. -1> 2012-12-18 19:54:59.065368 7f977255a700 5 throttle(msgr_dispatch_throttler-mon 0x19267e0) put 57 (0x8e4928 -> 0)
  2480. 0> 2012-12-18 19:54:59.065446 7f9771558700 -1 *** Caught signal (Aborted) **
  2481. in thread 7f9771558700
  2482.  
  2483. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  2484. 1: /usr/bin/ceph-mon() [0x54a532]
  2485. 2: (()+0xf030) [0x7f977a962030]
  2486. 3: (gsignal()+0x35) [0x7f9779265475]
  2487. 4: (abort()+0x180) [0x7f97792686f0]
  2488. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f9779ab768d]
  2489. 6: (()+0x63796) [0x7f9779ab5796]
  2490. 7: (()+0x637c3) [0x7f9779ab57c3]
  2491. 8: (()+0x639ee) [0x7f9779ab59ee]
  2492. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3c1) [0x6129b1]
  2493. 10: (SimpleMessenger::Pipe::write_message(Message*)+0xb7f) [0x5ed51f]
  2494. 11: (SimpleMessenger::Pipe::writer()+0x458) [0x600f48]
  2495. 12: (SimpleMessenger::Pipe::Writer::entry()+0xd) [0x5c40dd]
  2496. 13: (()+0x6b50) [0x7f977a959b50]
  2497. 14: (clone()+0x6d) [0x7f977930b6dd]
  2498. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  2499.  
  2500. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!