API tools faq
paste
Advertisement
Guest User

ceph-log

a guest
Jan 9th, 2013
87
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 116.89 KB | None | 0 0
raw download clone embed print report
  1. time: 2013-01-09 01:02:03.985087, event: waiting_for_osdmap, request: osd_sub_op_reply(osd.16.0:2952 2.154 764b0554/rb.0.1063.670df140.000000000001/10f//2 [] ondisk, result = 0) v1
  2. -370> 2013-01-09 01:02:03.985092 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371435, time: 2013-01-09 01:02:03.985092, event: queued_for_pg, request: osd_sub_op_reply(osd.16.0:2952 2.154 764b0554/rb.0.1063.670df140.000000000001/10f//2 [] ondisk, result = 0) v1
  3. -369> 2013-01-09 01:02:03.985096 7fe964397700 5 throttle(msgr_dispatch_throttler-cluster 0x23bf560) put 157 (0xbd3548 -> 0)
  4. -368> 2013-01-09 01:02:03.985146 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217255, seq: 371430, time: 2013-01-09 01:02:03.985146, event: started, request: osd_op(client.93634.1:188217255 rb.0.1038.180f78e2.00000007258a [read 3538944~131072] 2.7de066a2)
  5. -367> 2013-01-09 01:02:03.985599 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (278 -> 417)
  6. -366> 2013-01-09 01:02:03.985602 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (278 -> 417)
  7. -365> 2013-01-09 01:02:04.009527 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217255 rb.0.1038.180f78e2.00000007258a [read 3538944~131072] = 0) v4 -- ?+0 0xa58ab40 con 0x80b2140
  8. -364> 2013-01-09 01:02:04.009545 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217255, seq: 371430, time: 2013-01-09 01:02:04.009545, event: done, request: osd_op(client.93634.1:188217255 rb.0.1038.180f78e2.00000007258a [read 3538944~131072] 2.7de066a2)
  9. -363> 2013-01-09 01:02:04.009552 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 417)
  10. -362> 2013-01-09 01:02:04.009558 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217256, seq: 371431, time: 2013-01-09 01:02:04.009558, event: reached_pg, request: osd_op(client.93634.1:188217256 rb.0.1038.180f78e2.00000007258a [read 524288~131072] 2.7de066a2)
  11. -361> 2013-01-09 01:02:04.009633 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217256, seq: 371431, time: 2013-01-09 01:02:04.009633, event: started, request: osd_op(client.93634.1:188217256 rb.0.1038.180f78e2.00000007258a [read 524288~131072] 2.7de066a2)
  12. -360> 2013-01-09 01:02:04.009683 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:03.832419, event: header_read, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  13. -359> 2013-01-09 01:02:04.009693 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:03.832427, event: throttled, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  14. -358> 2013-01-09 01:02:04.009697 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:03.832433, event: all_read, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  15. -357> 2013-01-09 01:02:04.009700 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:03.985039, event: dispatched, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  16. -356> 2013-01-09 01:02:04.009703 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:04.009703, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  17. -355> 2013-01-09 01:02:04.009707 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 278)
  18. -354> 2013-01-09 01:02:04.009741 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (278 -> 417)
  19. -353> 2013-01-09 01:02:04.009746 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (417 -> 556)
  20. -352> 2013-01-09 01:02:04.009765 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217256 rb.0.1038.180f78e2.00000007258a [read 524288~131072] = 0) v4 -- ?+0 0xa588f00 con 0x80b2140
  21. -351> 2013-01-09 01:02:04.009781 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217256, seq: 371431, time: 2013-01-09 01:02:04.009781, event: done, request: osd_op(client.93634.1:188217256 rb.0.1038.180f78e2.00000007258a [read 524288~131072] 2.7de066a2)
  22. -350> 2013-01-09 01:02:04.009788 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 417)
  23. -349> 2013-01-09 01:02:04.009905 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:04.009905, event: queued_for_pg, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  24. -348> 2013-01-09 01:02:04.009916 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 417)
  25. -347> 2013-01-09 01:02:04.009920 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217257, seq: 371432, time: 2013-01-09 01:02:04.009920, event: reached_pg, request: osd_op(client.93634.1:188217257 rb.0.1038.180f78e2.00000007258a [read 655360~131072] 2.7de066a2)
  26. -346> 2013-01-09 01:02:04.009919 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234421 ==== osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2) ==== 139+0+0 (2374763149 0 0) 0x7b6ab40 con 0x80b2140
  27. -345> 2013-01-09 01:02:04.009927 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:03.871664, event: header_read, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  28. -344> 2013-01-09 01:02:04.009931 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:03.871672, event: throttled, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  29. -343> 2013-01-09 01:02:04.009934 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:03.871678, event: all_read, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  30. -342> 2013-01-09 01:02:04.009937 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.009926, event: dispatched, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  31. -341> 2013-01-09 01:02:04.009940 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.009940, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  32. -340> 2013-01-09 01:02:04.009944 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 278)
  33. -339> 2013-01-09 01:02:04.009994 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217257, seq: 371432, time: 2013-01-09 01:02:04.009994, event: started, request: osd_op(client.93634.1:188217257 rb.0.1038.180f78e2.00000007258a [read 655360~131072] 2.7de066a2)
  34. -338> 2013-01-09 01:02:04.010461 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (278 -> 417)
  35. -337> 2013-01-09 01:02:04.010464 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (417 -> 556)
  36. -336> 2013-01-09 01:02:04.033887 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217257 rb.0.1038.180f78e2.00000007258a [read 655360~131072] = 0) v4 -- ?+0 0x8afe1e0 con 0x80b2140
  37. -335> 2013-01-09 01:02:04.033905 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217257, seq: 371432, time: 2013-01-09 01:02:04.033905, event: done, request: osd_op(client.93634.1:188217257 rb.0.1038.180f78e2.00000007258a [read 655360~131072] 2.7de066a2)
  38. -334> 2013-01-09 01:02:04.033912 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 417)
  39. -333> 2013-01-09 01:02:04.033912 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.033912, event: queued_for_pg, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  40. -332> 2013-01-09 01:02:04.033923 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 417)
  41. -331> 2013-01-09 01:02:04.033925 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234422 ==== osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2) ==== 139+0+0 (433316068 0 0) 0xaae9d40 con 0x80b2140
  42. -330> 2013-01-09 01:02:04.033932 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:03.985598, event: header_read, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  43. -329> 2013-01-09 01:02:04.033936 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:03.985604, event: throttled, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  44. -328> 2013-01-09 01:02:04.033940 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:03.985610, event: all_read, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  45. -327> 2013-01-09 01:02:04.033943 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.033931, event: dispatched, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  46. -326> 2013-01-09 01:02:04.033946 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.033946, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  47. -325> 2013-01-09 01:02:04.033949 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 278)
  48. -324> 2013-01-09 01:02:04.033953 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.033953, event: queued_for_pg, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  49. -323> 2013-01-09 01:02:04.033958 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 278)
  50. -322> 2013-01-09 01:02:04.033959 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234423 ==== osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2) ==== 139+0+0 (952289793 0 0) 0x86bbd40 con 0x80b2140
  51. -321> 2013-01-09 01:02:04.033964 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.009741, event: header_read, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  52. -320> 2013-01-09 01:02:04.033968 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.009747, event: throttled, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  53. -319> 2013-01-09 01:02:04.033971 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.009755, event: all_read, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  54. -318> 2013-01-09 01:02:04.033974 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.033964, event: dispatched, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  55. -317> 2013-01-09 01:02:04.033977 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.033976, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  56. -316> 2013-01-09 01:02:04.033980 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 139)
  57. -315> 2013-01-09 01:02:04.033982 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.033982, event: queued_for_pg, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  58. -314> 2013-01-09 01:02:04.033985 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 139)
  59. -313> 2013-01-09 01:02:04.033987 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234424 ==== osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2) ==== 139+0+0 (2895980648 0 0) 0x86bb680 con 0x80b2140
  60. -312> 2013-01-09 01:02:04.033992 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.010460, event: header_read, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  61. -311> 2013-01-09 01:02:04.033996 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.010465, event: throttled, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  62. -310> 2013-01-09 01:02:04.033999 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.010471, event: all_read, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  63. -309> 2013-01-09 01:02:04.034002 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.033992, event: dispatched, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  64. -308> 2013-01-09 01:02:04.034005 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.034005, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  65. -307> 2013-01-09 01:02:04.034008 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  66. -306> 2013-01-09 01:02:04.034013 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.034013, event: queued_for_pg, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  67. -305> 2013-01-09 01:02:04.034016 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 0)
  68. -304> 2013-01-09 01:02:04.034039 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217258, seq: 371433, time: 2013-01-09 01:02:04.034039, event: reached_pg, request: osd_op(client.93634.1:188217258 rb.0.1038.180f78e2.00000007258a [read 1572864~131072] 2.7de066a2)
  69. -303> 2013-01-09 01:02:04.034054 7fe96c3a7700 5 osd.16 3713 tick
  70. -302> 2013-01-09 01:02:04.034092 7fe961a91700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.034092, event: reached_pg, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  71. -301> 2013-01-09 01:02:04.034100 7fe961a91700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.034100, event: started, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  72. -300> 2013-01-09 01:02:04.034116 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217258, seq: 371433, time: 2013-01-09 01:02:04.034116, event: started, request: osd_op(client.93634.1:188217258 rb.0.1038.180f78e2.00000007258a [read 1572864~131072] 2.7de066a2)
  73. -299> 2013-01-09 01:02:04.034136 7fe961a91700 5 filestore(/home/cb1/osd) queue_transactions existing osr(2.37f 0x35a0170)/0x35a0170
  74. -298> 2013-01-09 01:02:04.034139 7fe961a91700 5 throttle(filestore_ops 0x2402100) wait
  75. -297> 2013-01-09 01:02:04.034140 7fe961a91700 5 throttle(filestore_bytes 0x2402178) wait
  76. -296> 2013-01-09 01:02:04.034141 7fe961a91700 5 filestore(/home/cb1/osd) queue_transactions (writeahead) 3638428 0x8a73c48,0x8a73bc0
  77. -295> 2013-01-09 01:02:04.034145 7fe961a91700 5 journal submit_entry seq 3638428 len 4048 (0x7591230)
  78. -294> 2013-01-09 01:02:04.034146 7fe961a91700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.034146, event: commit_queued_for_journal_write, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  79. -293> 2013-01-09 01:02:04.034151 7fe961a91700 5 throttle(filestore_ops 0x2402100) take 1
  80. -292> 2013-01-09 01:02:04.034153 7fe961a91700 5 throttle(filestore_bytes 0x2402178) take 4048
  81. -291> 2013-01-09 01:02:04.034160 7fe961a91700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371435, time: 2013-01-09 01:02:04.034160, event: reached_pg, request: osd_sub_op_reply(osd.16.0:2952 2.154 764b0554/rb.0.1063.670df140.000000000001/10f//2 [] ondisk, result = 0) v1
  82. -290> 2013-01-09 01:02:04.034164 7fe961a91700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371435, time: 2013-01-09 01:02:04.034164, event: started, request: osd_sub_op_reply(osd.16.0:2952 2.154 764b0554/rb.0.1063.670df140.000000000001/10f//2 [] ondisk, result = 0) v1
  83. -289> 2013-01-09 01:02:04.034173 7fe961a91700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371435, time: 2013-01-09 01:02:04.034173, event: done, request: osd_sub_op_reply(osd.16.0:2952 2.154 764b0554/rb.0.1063.670df140.000000000001/10f//2 [] ondisk, result = 0) v1
  84. -288> 2013-01-09 01:02:04.034193 7fe9693a1700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.034193, event: write_thread_in_journal_buffer, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  85. -287> 2013-01-09 01:02:04.034195 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217258 rb.0.1038.180f78e2.00000007258a [read 1572864~131072] = 0) v4 -- ?+0 0xa588960 con 0x80b2140
  86. -286> 2013-01-09 01:02:04.034211 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217258, seq: 371433, time: 2013-01-09 01:02:04.034211, event: done, request: osd_op(client.93634.1:188217258 rb.0.1038.180f78e2.00000007258a [read 1572864~131072] 2.7de066a2)
  87. -285> 2013-01-09 01:02:04.034216 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  88. -284> 2013-01-09 01:02:04.034231 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (0 -> 139)
  89. -283> 2013-01-09 01:02:04.034233 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (0 -> 139)
  90. -282> 2013-01-09 01:02:04.034236 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:04.034236, event: reached_pg, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  91. -281> 2013-01-09 01:02:04.034256 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234425 ==== osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2) ==== 139+0+0 (3484309319 0 0) 0x885f440 con 0x80b2140
  92. -280> 2013-01-09 01:02:04.034266 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.034231, event: header_read, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  93. -279> 2013-01-09 01:02:04.034270 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.034234, event: throttled, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  94. -278> 2013-01-09 01:02:04.034273 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.034241, event: all_read, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  95. -277> 2013-01-09 01:02:04.034276 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.034265, event: dispatched, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  96. -276> 2013-01-09 01:02:04.034279 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.034279, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  97. -275> 2013-01-09 01:02:04.034283 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  98. -274> 2013-01-09 01:02:04.034316 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:04.034315, event: started, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  99. -273> 2013-01-09 01:02:04.034566 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (0 -> 139)
  100. -272> 2013-01-09 01:02:04.034570 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (139 -> 278)
  101. -271> 2013-01-09 01:02:04.035038 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (139 -> 278)
  102. -270> 2013-01-09 01:02:04.035042 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (278 -> 417)
  103. -269> 2013-01-09 01:02:04.057672 7fe957d1e700 1 -- 10.75.76.153:6804/6052 >> :/0 pipe(0x7f9ac00 sd=48 pgs=0 cs=0 l=0).accept sd=48
  104. -268> 2013-01-09 01:02:04.057714 7fe957d1e700 0 -- 10.75.76.153:6804/6052 >> 10.75.76.143:6806/10599 pipe(0x7f9ac00 sd=48 pgs=0 cs=0 l=0).accept connect_seq 10 vs existing 9 state 3
  105. -267> 2013-01-09 01:02:04.057812 7fe957d1e700 5 throttle(msgr_dispatch_throttler-cluster 0x23bf560) get 2855 (0 -> 2855)
  106. -266> 2013-01-09 01:02:04.057841 7fe964397700 1 -- 10.75.76.153:6804/6052 <== osd.7 10.75.76.143:6806/10599 372 ==== osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7 ==== 1185+0+1670 (2205825988 0 3831585357) 0xb37a800 con 0x25bba40
  107. -265> 2013-01-09 01:02:04.063055 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] = 0) v4 -- ?+0 0xa8ccd20 con 0x80b2140
  108. -264> 2013-01-09 01:02:04.063073 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217259, seq: 371436, time: 2013-01-09 01:02:04.063072, event: done, request: osd_op(client.93634.1:188217259 rb.0.1038.180f78e2.00000007258a [read 1703936~131072] 2.7de066a2)
  109. -263> 2013-01-09 01:02:04.063078 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 278)
  110. -262> 2013-01-09 01:02:04.063225 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.063225, event: queued_for_pg, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  111. -261> 2013-01-09 01:02:04.063234 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 278)
  112. -260> 2013-01-09 01:02:04.063237 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234426 ==== osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2) ==== 139+0+0 (1542567214 0 0) 0xc095680 con 0x80b2140
  113. -259> 2013-01-09 01:02:04.063244 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.034566, event: header_read, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  114. -258> 2013-01-09 01:02:04.063247 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.034571, event: throttled, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  115. -257> 2013-01-09 01:02:04.063250 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.034577, event: all_read, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  116. -256> 2013-01-09 01:02:04.063254 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.063243, event: dispatched, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  117. -255> 2013-01-09 01:02:04.063257 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.063257, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  118. -254> 2013-01-09 01:02:04.063260 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 139)
  119. -253> 2013-01-09 01:02:04.063263 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.063263, event: queued_for_pg, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  120. -252> 2013-01-09 01:02:04.063267 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 139)
  121. -251> 2013-01-09 01:02:04.063269 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234427 ==== osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2) ==== 139+0+0 (2061541323 0 0) 0xa43bb00 con 0x80b2140
  122. -250> 2013-01-09 01:02:04.063274 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.035038, event: header_read, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  123. -249> 2013-01-09 01:02:04.063278 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.035043, event: throttled, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  124. -248> 2013-01-09 01:02:04.063281 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.035048, event: all_read, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  125. -247> 2013-01-09 01:02:04.063284 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.063274, event: dispatched, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  126. -246> 2013-01-09 01:02:04.063287 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.063287, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  127. -245> 2013-01-09 01:02:04.063290 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  128. -244> 2013-01-09 01:02:04.063292 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.063292, event: queued_for_pg, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  129. -243> 2013-01-09 01:02:04.063296 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 0)
  130. -242> 2013-01-09 01:02:04.063301 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.063300, event: reached_pg, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  131. -241> 2013-01-09 01:02:04.063376 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.063376, event: started, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  132. -240> 2013-01-09 01:02:04.063496 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] = 0) v4 -- ?+0 0x8cc3680 con 0x80b2140
  133. -239> 2013-01-09 01:02:04.063504 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.057811, event: header_read, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  134. -238> 2013-01-09 01:02:04.063515 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.057817, event: throttled, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  135. -237> 2013-01-09 01:02:04.063519 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.057835, event: all_read, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  136. -236> 2013-01-09 01:02:04.063523 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.057855, event: dispatched, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  137. -235> 2013-01-09 01:02:04.063527 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.063527, event: waiting_for_osdmap, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  138. -234> 2013-01-09 01:02:04.063535 7fe964397700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.063535, event: queued_for_pg, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  139. -233> 2013-01-09 01:02:04.063541 7fe964397700 5 throttle(msgr_dispatch_throttler-cluster 0x23bf560) put 2855 (0xbd3548 -> 0)
  140. -232> 2013-01-09 01:02:04.063549 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.063549, event: reached_pg, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  141. -231> 2013-01-09 01:02:04.063633 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.063633, event: started, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  142. -230> 2013-01-09 01:02:04.063687 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217260, seq: 371437, time: 2013-01-09 01:02:04.063512, event: done, request: osd_op(client.93634.1:188217260 rb.0.1038.180f78e2.00000007258a [read 2621440~131072] 2.7de066a2)
  143. -229> 2013-01-09 01:02:04.063696 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  144. -228> 2013-01-09 01:02:04.064074 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (0 -> 139)
  145. -227> 2013-01-09 01:02:04.064080 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (0 -> 139)
  146. -226> 2013-01-09 01:02:04.064095 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234428 ==== osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2) ==== 139+0+0 (4005526946 0 0) 0x854ab40 con 0x80b2140
  147. -225> 2013-01-09 01:02:04.077369 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] = 0) v4 -- ?+0 0x7d71860 con 0x80b2140
  148. -224> 2013-01-09 01:02:04.077386 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217261, seq: 371438, time: 2013-01-09 01:02:04.077385, event: done, request: osd_op(client.93634.1:188217261 rb.0.1038.180f78e2.00000007258a [read 2752512~131072] 2.7de066a2)
  149. -223> 2013-01-09 01:02:04.077393 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 139)
  150. -222> 2013-01-09 01:02:04.077518 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.077389, event: reached_pg, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  151. -221> 2013-01-09 01:02:04.077598 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.077598, event: started, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  152. -220> 2013-01-09 01:02:04.077617 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (139 -> 278)
  153. -219> 2013-01-09 01:02:04.077621 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (139 -> 278)
  154. -218> 2013-01-09 01:02:04.077675 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] = 0) v4 -- ?+0 0x7d71e00 con 0x80b2140
  155. -217> 2013-01-09 01:02:04.077690 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217262, seq: 371439, time: 2013-01-09 01:02:04.077690, event: done, request: osd_op(client.93634.1:188217262 rb.0.1038.180f78e2.00000007258a [read 3670016~131072] 2.7de066a2)
  156. -216> 2013-01-09 01:02:04.077697 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 278)
  157. -215> 2013-01-09 01:02:04.077697 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.077692, event: reached_pg, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  158. -214> 2013-01-09 01:02:04.077774 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.077773, event: started, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  159. -213> 2013-01-09 01:02:04.077828 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.064073, event: header_read, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  160. -212> 2013-01-09 01:02:04.077836 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.064082, event: throttled, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  161. -211> 2013-01-09 01:02:04.077839 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.064089, event: all_read, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  162. -210> 2013-01-09 01:02:04.077844 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.064107, event: dispatched, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  163. -209> 2013-01-09 01:02:04.077847 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.077847, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  164. -208> 2013-01-09 01:02:04.077851 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 139)
  165. -207> 2013-01-09 01:02:04.078280 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (139 -> 278)
  166. -206> 2013-01-09 01:02:04.078284 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (278 -> 417)
  167. -205> 2013-01-09 01:02:04.103868 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] = 0) v4 -- ?+0 0x8afe960 con 0x80b2140
  168. -204> 2013-01-09 01:02:04.103886 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217263, seq: 371440, time: 2013-01-09 01:02:04.103886, event: done, request: osd_op(client.93634.1:188217263 rb.0.1038.180f78e2.00000007258a [read 3801088~131072] 2.7de066a2)
  169. -203> 2013-01-09 01:02:04.103893 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 278)
  170. -202> 2013-01-09 01:02:04.103902 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.103902, event: queued_for_pg, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  171. -201> 2013-01-09 01:02:04.103908 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 278)
  172. -200> 2013-01-09 01:02:04.103910 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234429 ==== osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2) ==== 139+0+0 (2699014318 0 0) 0xa192900 con 0x80b2140
  173. -199> 2013-01-09 01:02:04.103916 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.077617, event: header_read, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  174. -198> 2013-01-09 01:02:04.103920 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.077622, event: throttled, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  175. -197> 2013-01-09 01:02:04.103923 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.077628, event: all_read, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  176. -196> 2013-01-09 01:02:04.103926 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.103915, event: dispatched, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  177. -195> 2013-01-09 01:02:04.103929 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.103929, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  178. -194> 2013-01-09 01:02:04.103933 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 139)
  179. -193> 2013-01-09 01:02:04.103936 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.103936, event: queued_for_pg, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  180. -192> 2013-01-09 01:02:04.103940 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 139)
  181. -191> 2013-01-09 01:02:04.103941 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234430 ==== osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2) ==== 139+0+0 (880825031 0 0) 0xc5bb200 con 0x80b2140
  182. -190> 2013-01-09 01:02:04.103947 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.078279, event: header_read, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  183. -189> 2013-01-09 01:02:04.103950 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.078287, event: throttled, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  184. -188> 2013-01-09 01:02:04.103954 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.078293, event: all_read, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  185. -187> 2013-01-09 01:02:04.103957 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.103946, event: dispatched, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  186. -186> 2013-01-09 01:02:04.103960 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.103960, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  187. -185> 2013-01-09 01:02:04.103963 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  188. -184> 2013-01-09 01:02:04.103965 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.103965, event: queued_for_pg, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  189. -183> 2013-01-09 01:02:04.103969 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 0)
  190. -182> 2013-01-09 01:02:04.103978 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.103977, event: reached_pg, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  191. -181> 2013-01-09 01:02:04.104052 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.104051, event: started, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  192. -180> 2013-01-09 01:02:04.104073 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (0 -> 139)
  193. -179> 2013-01-09 01:02:04.104079 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (0 -> 139)
  194. -178> 2013-01-09 01:02:04.104125 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] = 0) v4 -- ?+0 0x7d703c0 con 0x80b2140
  195. -177> 2013-01-09 01:02:04.104141 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217264, seq: 371441, time: 2013-01-09 01:02:04.104141, event: done, request: osd_op(client.93634.1:188217264 rb.0.1038.180f78e2.00000007258a [read 786432~131072] 2.7de066a2)
  196. -176> 2013-01-09 01:02:04.104147 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 139)
  197. -175> 2013-01-09 01:02:04.104151 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.104151, event: reached_pg, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  198. -174> 2013-01-09 01:02:04.104222 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.104222, event: started, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  199. -173> 2013-01-09 01:02:04.104252 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234431 ==== osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2) ==== 139+0+0 (361855010 0 0) 0x9ecc6c0 con 0x80b2140
  200. -172> 2013-01-09 01:02:04.104261 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.104073, event: header_read, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  201. -171> 2013-01-09 01:02:04.104269 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.104081, event: throttled, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  202. -170> 2013-01-09 01:02:04.104274 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.104088, event: all_read, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  203. -169> 2013-01-09 01:02:04.104279 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.104260, event: dispatched, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  204. -168> 2013-01-09 01:02:04.104283 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.104283, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  205. -167> 2013-01-09 01:02:04.104288 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  206. -166> 2013-01-09 01:02:04.104775 7fe95d071700 5 throttle(osd_client_bytes 0x7fff5ff84470) get 139 (0 -> 139)
  207. -165> 2013-01-09 01:02:04.104780 7fe95d071700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) get 139 (139 -> 278)
  208. -164> 2013-01-09 01:02:04.119355 7fe95c96a700 5 throttle(msgr_dispatch_throttler-cluster 0x23bf560) get 157 (0 -> 157)
  209. -163> 2013-01-09 01:02:04.119382 7fe964397700 1 -- 10.75.76.153:6804/6052 <== osd.12 10.75.76.146:6804/32563 199 ==== osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1 ==== 157+0+0 (3291168116 0 0) 0x8078000 con 0x7b77b80
  210. -162> 2013-01-09 01:02:04.119745 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d6c30 seq 3638355 osr(default 0x240c678)/0x240c678 start
  211. -161> 2013-01-09 01:02:04.119750 7fe96639b700 5 --OSD::tracker-- reqid: osd.1.0:39254, seq: 371417, time: 2013-01-09 01:02:04.119750, event: sub_op_applied, request: osd_sub_op(osd.1.0:39254 2.83 b056ec83/rb.0.1063.670df140.000000000621/10f//2 [] v 3713'38440 snapset=0=[]:[] snapc=0=[]) v7
  212. -160> 2013-01-09 01:02:04.119762 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d7a40 seq 3638356 osr(default 0x240c678)/0x240c678 start
  213. -159> 2013-01-09 01:02:04.119763 7fe96639b700 5 --OSD::tracker-- reqid: osd.1.0:39254, seq: 371417, time: 2013-01-09 01:02:04.119763, event: done, request: osd_sub_op(osd.1.0:39254 2.83 b056ec83/rb.0.1063.670df140.000000000621/10f//2 [] v 3713'38440 snapset=0=[]:[] snapc=0=[]) v7
  214. -158> 2013-01-09 01:02:04.119770 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x80840f0 seq 3638357 osr(default 0x240c678)/0x240c678 start
  215. -157> 2013-01-09 01:02:04.119784 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d450 seq 3638358 osr(default 0x240c678)/0x240c678 start
  216. -156> 2013-01-09 01:02:04.119793 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x9fc35e0 seq 3638359 osr(default 0x240c678)/0x240c678 start
  217. -155> 2013-01-09 01:02:04.119801 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x80840a0 seq 3638360 osr(default 0x240c678)/0x240c678 start
  218. -154> 2013-01-09 01:02:04.119809 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df9a0 seq 3638361 osr(default 0x240c678)/0x240c678 start
  219. -153> 2013-01-09 01:02:04.119818 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7975130 seq 3638362 osr(default 0x240c678)/0x240c678 start
  220. -152> 2013-01-09 01:02:04.119825 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d3b0 seq 3638363 osr(default 0x240c678)/0x240c678 start
  221. -151> 2013-01-09 01:02:04.119834 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7d7a0f0 seq 3638364 osr(default 0x240c678)/0x240c678 start
  222. -150> 2013-01-09 01:02:04.119841 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7f772c0 seq 3638365 osr(default 0x240c678)/0x240c678 start
  223. -149> 2013-01-09 01:02:04.119850 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df8b0 seq 3638366 osr(default 0x240c678)/0x240c678 start
  224. -148> 2013-01-09 01:02:04.119857 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d7c0 seq 3638367 osr(default 0x240c678)/0x240c678 start
  225. -147> 2013-01-09 01:02:04.119865 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d2c0 seq 3638368 osr(default 0x240c678)/0x240c678 start
  226. -146> 2013-01-09 01:02:04.119873 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79da9b0 seq 3638369 osr(default 0x240c678)/0x240c678 start
  227. -145> 2013-01-09 01:02:04.119881 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7975770 seq 3638370 osr(default 0x240c678)/0x240c678 start
  228. -144> 2013-01-09 01:02:04.119959 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68dae0 seq 3638371 osr(default 0x240c678)/0x240c678 start
  229. -143> 2013-01-09 01:02:04.119969 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df220 seq 3638372 osr(default 0x240c678)/0x240c678 start
  230. -142> 2013-01-09 01:02:04.119977 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7975bd0 seq 3638373 osr(default 0x240c678)/0x240c678 start
  231. -141> 2013-01-09 01:02:04.119985 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d6d0 seq 3638374 osr(default 0x240c678)/0x240c678 start
  232. -140> 2013-01-09 01:02:04.119993 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x9fc2960 seq 3638375 osr(default 0x240c678)/0x240c678 start
  233. -139> 2013-01-09 01:02:04.120000 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8085400 seq 3638376 osr(default 0x240c678)/0x240c678 start
  234. -138> 2013-01-09 01:02:04.120008 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7975540 seq 3638377 osr(default 0x240c678)/0x240c678 start
  235. -137> 2013-01-09 01:02:04.120016 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d6730 seq 3638378 osr(default 0x240c678)/0x240c678 start
  236. -136> 2013-01-09 01:02:04.120023 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8de550 seq 3638379 osr(default 0x240c678)/0x240c678 start
  237. -135> 2013-01-09 01:02:04.120031 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x9fc3ef0 seq 3638380 osr(default 0x240c678)/0x240c678 start
  238. -134> 2013-01-09 01:02:04.120039 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d7db0 seq 3638381 osr(default 0x240c678)/0x240c678 start
  239. -133> 2013-01-09 01:02:04.120047 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8084460 seq 3638382 osr(default 0x240c678)/0x240c678 start
  240. -132> 2013-01-09 01:02:04.120055 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x9fc3cc0 seq 3638383 osr(default 0x240c678)/0x240c678 start
  241. -131> 2013-01-09 01:02:04.120125 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7d7a280 seq 3638384 osr(default 0x240c678)/0x240c678 start
  242. -130> 2013-01-09 01:02:04.120135 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8dfa90 seq 3638385 osr(default 0x240c678)/0x240c678 start
  243. -129> 2013-01-09 01:02:04.120142 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8dfe50 seq 3638386 osr(default 0x240c678)/0x240c678 start
  244. -128> 2013-01-09 01:02:04.120149 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79dab90 seq 3638387 osr(default 0x240c678)/0x240c678 start
  245. -127> 2013-01-09 01:02:04.120155 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df310 seq 3638388 osr(default 0x240c678)/0x240c678 start
  246. -126> 2013-01-09 01:02:04.120162 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8084d20 seq 3638389 osr(default 0x240c678)/0x240c678 start
  247. -125> 2013-01-09 01:02:04.120169 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7d7b590 seq 3638390 osr(default 0x240c678)/0x240c678 start
  248. -124> 2013-01-09 01:02:04.120175 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d900 seq 3638391 osr(default 0x240c678)/0x240c678 start
  249. -123> 2013-01-09 01:02:04.120182 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8de1e0 seq 3638392 osr(default 0x240c678)/0x240c678 start
  250. -122> 2013-01-09 01:02:04.120190 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68db80 seq 3638393 osr(default 0x240c678)/0x240c678 start
  251. -121> 2013-01-09 01:02:04.120197 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d6780 seq 3638394 osr(default 0x240c678)/0x240c678 start
  252. -120> 2013-01-09 01:02:04.120203 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79db860 seq 3638395 osr(default 0x240c678)/0x240c678 start
  253. -119> 2013-01-09 01:02:04.120209 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79dad70 seq 3638396 osr(default 0x240c678)/0x240c678 start
  254. -118> 2013-01-09 01:02:04.120215 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df400 seq 3638397 osr(default 0x240c678)/0x240c678 start
  255. -117> 2013-01-09 01:02:04.120222 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d9a0 seq 3638398 osr(default 0x240c678)/0x240c678 start
  256. -116> 2013-01-09 01:02:04.120228 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68ceb0 seq 3638399 osr(default 0x240c678)/0x240c678 start
  257. -115> 2013-01-09 01:02:04.120236 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8df630 seq 3638400 osr(default 0x240c678)/0x240c678 start
  258. -114> 2013-01-09 01:02:04.120242 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d70e0 seq 3638401 osr(default 0x240c678)/0x240c678 start
  259. -113> 2013-01-09 01:02:04.120305 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68c5f0 seq 3638402 osr(default 0x240c678)/0x240c678 start
  260. -112> 2013-01-09 01:02:04.120314 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8de780 seq 3638403 osr(default 0x240c678)/0x240c678 start
  261. -111> 2013-01-09 01:02:04.120322 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68c0a0 seq 3638404 osr(default 0x240c678)/0x240c678 start
  262. -110> 2013-01-09 01:02:04.120330 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8084b40 seq 3638405 osr(default 0x240c678)/0x240c678 start
  263. -109> 2013-01-09 01:02:04.120338 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7974b40 seq 3638406 osr(default 0x240c678)/0x240c678 start
  264. -108> 2013-01-09 01:02:04.120409 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8085720 seq 3638407 osr(default 0x240c678)/0x240c678 start
  265. -107> 2013-01-09 01:02:04.120421 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x9fc24b0 seq 3638408 osr(default 0x240c678)/0x240c678 start
  266. -106> 2013-01-09 01:02:04.120429 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79758b0 seq 3638409 osr(default 0x240c678)/0x240c678 start
  267. -105> 2013-01-09 01:02:04.120437 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d630 seq 3638410 osr(default 0x240c678)/0x240c678 start
  268. -104> 2013-01-09 01:02:04.120445 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d860 seq 3638411 osr(default 0x240c678)/0x240c678 start
  269. -103> 2013-01-09 01:02:04.120453 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68dd10 seq 3638412 osr(default 0x240c678)/0x240c678 start
  270. -102> 2013-01-09 01:02:04.120460 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x80855e0 seq 3638413 osr(default 0x240c678)/0x240c678 start
  271. -101> 2013-01-09 01:02:04.120468 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7cd6be0 seq 3638414 osr(default 0x240c678)/0x240c678 start
  272. -100> 2013-01-09 01:02:04.120476 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68d540 seq 3638415 osr(default 0x240c678)/0x240c678 start
  273. -99> 2013-01-09 01:02:04.120483 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7c169b0 seq 3638416 osr(default 0x240c678)/0x240c678 start
  274. -98> 2013-01-09 01:02:04.120491 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7974af0 seq 3638417 osr(default 0x240c678)/0x240c678 start
  275. -97> 2013-01-09 01:02:04.120499 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xa8de190 seq 3638418 osr(default 0x240c678)/0x240c678 start
  276. -96> 2013-01-09 01:02:04.120506 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7975310 seq 3638419 osr(default 0x240c678)/0x240c678 start
  277. -95> 2013-01-09 01:02:04.120514 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x79daaf0 seq 3638420 osr(default 0x240c678)/0x240c678 start
  278. -94> 2013-01-09 01:02:04.120522 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x8084730 seq 3638421 osr(default 0x240c678)/0x240c678 start
  279. -93> 2013-01-09 01:02:04.120530 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d6b90 seq 3638422 osr(default 0x240c678)/0x240c678 start
  280. -92> 2013-01-09 01:02:04.120540 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0x7974690 seq 3638423 osr(default 0x240c678)/0x240c678 start
  281. -91> 2013-01-09 01:02:04.120549 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc4d7b80 seq 3638424 osr(default 0x240c678)/0x240c678 start
  282. -90> 2013-01-09 01:02:04.120557 7fe96739d700 5 filestore(/home/cb1/osd) _do_op 0xc68c7d0 seq 3638425 osr(2.154 0x4666d70)/0x4666d70 start
  283. -89> 2013-01-09 01:02:04.120835 7fe96839f700 5 filestore(/home/cb1/osd) _journaled_ahead 0x79db270 seq 3638428 osr(2.37f 0x35a0170) 0x8a73c48,0x8a73bc0
  284. -88> 2013-01-09 01:02:04.120833 7fe9693a1700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.120833, event: journaled_completion_queued, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  285. -87> 2013-01-09 01:02:04.120843 7fe96839f700 5 filestore(/home/cb1/osd) queue_op 0x79db270 seq 3638428 osr(2.37f 0x35a0170) 4038 bytes (queue has 5 ops and 30717 bytes)
  286. -86> 2013-01-09 01:02:04.120843 7fe9693a1700 5 throttle(filestore_ops 0x2402100) put 1 (0xbd3548 -> 0)
  287. -85> 2013-01-09 01:02:04.120845 7fe9693a1700 5 throttle(filestore_bytes 0x2402178) put 4048 (0xbd3548 -> 0)
  288. -84> 2013-01-09 01:02:04.120846 7fe9693a1700 5 journal put_throttle finished 1 ops and 4048 bytes, now 0 ops and 0 bytes
  289. -83> 2013-01-09 01:02:04.120852 7fe965b9a700 5 --OSD::tracker-- reqid: osd.18.0:8743, seq: 371434, time: 2013-01-09 01:02:04.120852, event: sub_op_commit, request: osd_sub_op(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] v 3713'22059 snapset=0=[]:[] snapc=0=[]) v7
  290. -82> 2013-01-09 01:02:04.120862 7fe965b9a700 1 -- 10.75.76.153:6804/6052 --> osd.18 10.75.76.154:6805/8885 -- osd_sub_op_reply(osd.18.0:8743 2.37f d932477f/rb.0.1063.670df140.000000000024/10f//2 [] ondisk, result = 0) v1 -- ?+0 0xac9e500
  291. -81> 2013-01-09 01:02:04.134895 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] = 0) v4 -- ?+0 0xa5890e0 con 0x80b2140
  292. -80> 2013-01-09 01:02:04.134912 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217265, seq: 371442, time: 2013-01-09 01:02:04.134912, event: done, request: osd_op(client.93634.1:188217265 rb.0.1038.180f78e2.00000007258a [read 917504~131072] 2.7de066a2)
  293. -79> 2013-01-09 01:02:04.134919 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 139)
  294. -78> 2013-01-09 01:02:04.134919 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.134918, event: queued_for_pg, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  295. -77> 2013-01-09 01:02:04.134930 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 139)
  296. -76> 2013-01-09 01:02:04.134933 7fe964b98700 1 -- 172.27.124.153:6802/6052 <== client.93634 172.27.124.103:0/4181287074 234432 ==== osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2) ==== 139+0+0 (2177800779 0 0) 0x86bdd40 con 0x80b2140
  297. -75> 2013-01-09 01:02:04.134941 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.104774, event: header_read, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  298. -74> 2013-01-09 01:02:04.134944 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.104781, event: throttled, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  299. -73> 2013-01-09 01:02:04.134947 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.104787, event: all_read, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  300. -72> 2013-01-09 01:02:04.134950 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.134940, event: dispatched, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  301. -71> 2013-01-09 01:02:04.134953 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.134953, event: waiting_for_osdmap, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  302. -70> 2013-01-09 01:02:04.134956 7fe964b98700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 139 (0xbd3548 -> 0)
  303. -69> 2013-01-09 01:02:04.134960 7fe964b98700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.134960, event: queued_for_pg, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  304. -68> 2013-01-09 01:02:04.134964 7fe964b98700 5 throttle(msgr_dispatch_throttler-client 0x23bf9e0) put 139 (0xbd3548 -> 0)
  305. -67> 2013-01-09 01:02:04.134978 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.134978, event: reached_pg, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  306. -66> 2013-01-09 01:02:04.135055 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.135055, event: started, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  307. -65> 2013-01-09 01:02:04.135067 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.119354, event: header_read, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  308. -64> 2013-01-09 01:02:04.135074 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.119366, event: throttled, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  309. -63> 2013-01-09 01:02:04.135077 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.119374, event: all_read, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  310. -62> 2013-01-09 01:02:04.135127 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] = 0) v4 -- ?+0 0x8afef00 con 0x80b2140
  311. -61> 2013-01-09 01:02:04.135081 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.119394, event: dispatched, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  312. -60> 2013-01-09 01:02:04.135152 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.135152, event: waiting_for_osdmap, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  313. -59> 2013-01-09 01:02:04.135159 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217266, seq: 371443, time: 2013-01-09 01:02:04.135141, event: done, request: osd_op(client.93634.1:188217266 rb.0.1038.180f78e2.00000007258a [read 1835008~131072] 2.7de066a2)
  314. -58> 2013-01-09 01:02:04.135167 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  315. -57> 2013-01-09 01:02:04.135288 7fe964397700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.135162, event: queued_for_pg, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  316. -56> 2013-01-09 01:02:04.135298 7fe964397700 5 throttle(msgr_dispatch_throttler-cluster 0x23bf560) put 157 (0xbd3548 -> 0)
  317. -55> 2013-01-09 01:02:04.135305 7fe961290700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.135304, event: reached_pg, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  318. -54> 2013-01-09 01:02:04.135312 7fe961290700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.135312, event: started, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  319. -53> 2013-01-09 01:02:04.135319 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.135306, event: reached_pg, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  320. -52> 2013-01-09 01:02:04.135352 7fe961290700 5 filestore(/home/cb1/osd) queue_transactions existing osr(2.7b 0x53bad70)/0x53bad70
  321. -51> 2013-01-09 01:02:04.135355 7fe961290700 5 throttle(filestore_ops 0x2402100) wait
  322. -50> 2013-01-09 01:02:04.135356 7fe961290700 5 throttle(filestore_bytes 0x2402178) wait
  323. -49> 2013-01-09 01:02:04.135357 7fe961290700 5 filestore(/home/cb1/osd) queue_transactions (writeahead) 3638429 0x251efe8,0x251ef60
  324. -48> 2013-01-09 01:02:04.135360 7fe961290700 5 journal submit_entry seq 3638429 len 4518 (0x7c128d0)
  325. -47> 2013-01-09 01:02:04.135362 7fe961290700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.135362, event: commit_queued_for_journal_write, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  326. -46> 2013-01-09 01:02:04.135367 7fe961290700 5 throttle(filestore_ops 0x2402100) take 1
  327. -45> 2013-01-09 01:02:04.135368 7fe961290700 5 throttle(filestore_bytes 0x2402178) take 4518
  328. -44> 2013-01-09 01:02:04.135392 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.135392, event: started, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  329. -43> 2013-01-09 01:02:04.135446 7fe9693a1700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.135446, event: write_thread_in_journal_buffer, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  330. -42> 2013-01-09 01:02:04.135881 7fe958a2b700 5 throttle(msgr_dispatch_throttler-hbserver 0x23c0be0) get 47 (0 -> 47)
  331. -41> 2013-01-09 01:02:04.135907 7fe963395700 1 -- 10.75.76.153:6805/6052 <== osd.3 10.75.76.141:0/13364 9093 ==== osd_ping(ping e3713 stamp 2013-01-09 01:02:04.135623) v2 ==== 47+0+0 (1402780558 0 0) 0x90a6a80 con 0x7d3ac80
  332. -40> 2013-01-09 01:02:04.135920 7fe963395700 1 -- 10.75.76.153:6805/6052 --> 10.75.76.141:0/13364 -- osd_ping(ping_reply e3713 stamp 2013-01-09 01:02:04.135623) v2 -- ?+0 0x7ad6c40 con 0x7d3ac80
  333. -39> 2013-01-09 01:02:04.135928 7fe963395700 5 throttle(msgr_dispatch_throttler-hbserver 0x23c0be0) put 47 (0xbd3548 -> 0)
  334. -38> 2013-01-09 01:02:04.167226 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] = 0) v4 -- ?+0 0xa8cd4a0 con 0x80b2140
  335. -37> 2013-01-09 01:02:04.167244 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217267, seq: 371445, time: 2013-01-09 01:02:04.167244, event: done, request: osd_op(client.93634.1:188217267 rb.0.1038.180f78e2.00000007258a [read 1966080~131072] 2.7de066a2)
  336. -36> 2013-01-09 01:02:04.167251 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.167250, event: reached_pg, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  337. -35> 2013-01-09 01:02:04.167334 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.167334, event: started, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  338. -34> 2013-01-09 01:02:04.167381 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  339. -33> 2013-01-09 01:02:04.167408 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] = 0) v4 -- ?+0 0x8cc3e00 con 0x80b2140
  340. -32> 2013-01-09 01:02:04.167424 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217268, seq: 371446, time: 2013-01-09 01:02:04.167424, event: done, request: osd_op(client.93634.1:188217268 rb.0.1038.180f78e2.00000007258a [read 2883584~131072] 2.7de066a2)
  341. -31> 2013-01-09 01:02:04.167430 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  342. -30> 2013-01-09 01:02:04.167550 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.167550, event: reached_pg, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  343. -29> 2013-01-09 01:02:04.167641 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.167641, event: started, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  344. -28> 2013-01-09 01:02:04.188531 7fe959d3e700 5 throttle(msgr_dispatch_throttler-hbserver 0x23c0be0) get 47 (0 -> 47)
  345. -27> 2013-01-09 01:02:04.188560 7fe963395700 1 -- 10.75.76.153:6805/6052 <== osd.7 10.75.76.143:0/10599 9130 ==== osd_ping(ping e3713 stamp 2013-01-09 01:02:04.188052) v2 ==== 47+0+0 (3066753724 0 0) 0x88f8000 con 0x7dbc500
  346. -26> 2013-01-09 01:02:04.188572 7fe963395700 1 -- 10.75.76.153:6805/6052 --> 10.75.76.143:0/10599 -- osd_ping(ping_reply e3713 stamp 2013-01-09 01:02:04.188052) v2 -- ?+0 0x90a6a80 con 0x7dbc500
  347. -25> 2013-01-09 01:02:04.188579 7fe963395700 5 throttle(msgr_dispatch_throttler-hbserver 0x23c0be0) put 47 (0xbd3548 -> 0)
  348. -24> 2013-01-09 01:02:04.189732 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] = 0) v4 -- ?+0 0xa5883c0 con 0x80b2140
  349. -23> 2013-01-09 01:02:04.189752 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217269, seq: 371447, time: 2013-01-09 01:02:04.189752, event: done, request: osd_op(client.93634.1:188217269 rb.0.1038.180f78e2.00000007258a [read 3014656~131072] 2.7de066a2)
  350. -22> 2013-01-09 01:02:04.189759 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  351. -21> 2013-01-09 01:02:04.189866 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.189754, event: reached_pg, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  352. -20> 2013-01-09 01:02:04.189948 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.189948, event: started, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  353. -19> 2013-01-09 01:02:04.190020 7fe961290700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] = 0) v4 -- ?+0 0xa56b2c0 con 0x80b2140
  354. -18> 2013-01-09 01:02:04.190037 7fe961290700 5 --OSD::tracker-- reqid: client.93634.1:188217270, seq: 371448, time: 2013-01-09 01:02:04.190037, event: done, request: osd_op(client.93634.1:188217270 rb.0.1038.180f78e2.00000007258a [read 3932160~131072] 2.7de066a2)
  355. -17> 2013-01-09 01:02:04.190045 7fe961290700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  356. -16> 2013-01-09 01:02:04.190045 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.190039, event: reached_pg, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  357. -15> 2013-01-09 01:02:04.190059 7fe961290700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.190049, event: reached_pg, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  358. -14> 2013-01-09 01:02:04.190065 7fe961290700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.190065, event: started, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  359. -13> 2013-01-09 01:02:04.190075 7fe961290700 5 --OSD::tracker-- reqid: unknown.0.0:0, seq: 371450, time: 2013-01-09 01:02:04.190075, event: done, request: osd_sub_op_reply(osd.16.0:2954 2.d6 bc1fbcd6/rb.0.1063.670df140.000000000622/10f//2 [] ondisk, result = 0) v1
  360. -12> 2013-01-09 01:02:04.190122 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.190122, event: started, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  361. -11> 2013-01-09 01:02:04.207023 7fe961a91700 1 -- 172.27.124.153:6802/6052 --> 172.27.124.103:0/4181287074 -- osd_op_reply(188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] = 0) v4 -- ?+0 0xad6c1e0 con 0x80b2140
  362. -10> 2013-01-09 01:02:04.207040 7fe961a91700 5 --OSD::tracker-- reqid: client.93634.1:188217271, seq: 371449, time: 2013-01-09 01:02:04.207039, event: done, request: osd_op(client.93634.1:188217271 rb.0.1038.180f78e2.00000007258a [read 4063232~131072] 2.7de066a2)
  363. -9> 2013-01-09 01:02:04.207045 7fe961a91700 5 throttle(osd_client_bytes 0x7fff5ff84470) put 0 (0xbd3548 -> 0)
  364. -8> 2013-01-09 01:02:04.228412 7fe9693a1700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.228411, event: journaled_completion_queued, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  365. -7> 2013-01-09 01:02:04.228428 7fe9693a1700 5 throttle(filestore_ops 0x2402100) put 1 (0xbd3548 -> 0)
  366. -6> 2013-01-09 01:02:04.228430 7fe9693a1700 5 throttle(filestore_bytes 0x2402178) put 4518 (0xbd3548 -> 0)
  367. -5> 2013-01-09 01:02:04.228432 7fe9693a1700 5 journal put_throttle finished 1 ops and 4518 bytes, now 0 ops and 0 bytes
  368. -4> 2013-01-09 01:02:04.228440 7fe96839f700 5 filestore(/home/cb1/osd) _journaled_ahead 0x7c175e0 seq 3638429 osr(2.7b 0x53bad70) 0x251efe8,0x251ef60
  369. -3> 2013-01-09 01:02:04.228445 7fe96839f700 5 filestore(/home/cb1/osd) queue_op 0x7c175e0 seq 3638429 osr(2.7b 0x53bad70) 4508 bytes (queue has 6 ops and 35225 bytes)
  370. -2> 2013-01-09 01:02:04.228455 7fe965b9a700 5 --OSD::tracker-- reqid: osd.7.0:10315, seq: 371444, time: 2013-01-09 01:02:04.228454, event: sub_op_commit, request: osd_sub_op(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] v 3713'23497 snapset=0=[]:[] snapc=0=[]) v7
  371. -1> 2013-01-09 01:02:04.228464 7fe965b9a700 1 -- 10.75.76.153:6804/6052 --> osd.7 10.75.76.143:6806/10599 -- osd_sub_op_reply(osd.7.0:10315 2.7b b863f47b/rb.0.1063.670df140.000000000240/10f//2 [] ondisk, result = 0) v1 -- ?+0 0xb256c80
  372. 0> 2013-01-09 01:02:04.257391 7fe967b9e700 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7fe967b9e700 time 2013-01-09 01:02:04.218080
  373. os/DBObjectMap.cc: 1131: FAILED assert(0)
  374.  
  375. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  376. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  377. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  378. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  379. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  380. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  381. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  382. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  383. 8: (FileStore::_do_op(FileStore::OpSequencer*)+0x90) [0x6ef800]
  384. 9: (ThreadPool::worker()+0x8f1) [0x826e71]
  385. 10: (ThreadPool::WorkThread::entry()+0xd) [0x663d3d]
  386. 11: (()+0x7e9a) [0x7fe972796e9a]
  387. 12: (clone()+0x6d) [0x7fe97142fcbd]
  388. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  389.  
  390. --- end dump of recent events ---
  391. 2013-01-09 01:02:04.318811 7fe967b9e700 -1 *** Caught signal (Aborted) **
  392. in thread 7fe967b9e700
  393.  
  394. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  395. 1: /usr/bin/ceph-osd() [0x778de0]
  396. 2: (()+0xfcb0) [0x7fe97279ecb0]
  397. 3: (gsignal()+0x35) [0x7fe971372425]
  398. 4: (abort()+0x17b) [0x7fe971375b8b]
  399. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fe971c6de2d]
  400. 6: (()+0x5ef26) [0x7fe971c6bf26]
  401. 7: (()+0x5ef53) [0x7fe971c6bf53]
  402. 8: (()+0x5f17e) [0x7fe971c6c17e]
  403. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  404. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  405. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  406. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  407. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  408. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  409. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  410. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  411. 17: (FileStore::_do_op(FileStore::OpSequencer*)+0x90) [0x6ef800]
  412. 18: (ThreadPool::worker()+0x8f1) [0x826e71]
  413. 19: (ThreadPool::WorkThread::entry()+0xd) [0x663d3d]
  414. 20: (()+0x7e9a) [0x7fe972796e9a]
  415. 21: (clone()+0x6d) [0x7fe97142fcbd]
  416. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  417.  
  418. --- begin dump of recent events ---
  419. 0> 2013-01-09 01:02:04.318811 7fe967b9e700 -1 *** Caught signal (Aborted) **
  420. in thread 7fe967b9e700
  421.  
  422. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  423. 1: /usr/bin/ceph-osd() [0x778de0]
  424. 2: (()+0xfcb0) [0x7fe97279ecb0]
  425. 3: (gsignal()+0x35) [0x7fe971372425]
  426. 4: (abort()+0x17b) [0x7fe971375b8b]
  427. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fe971c6de2d]
  428. 6: (()+0x5ef26) [0x7fe971c6bf26]
  429. 7: (()+0x5ef53) [0x7fe971c6bf53]
  430. 8: (()+0x5f17e) [0x7fe971c6c17e]
  431. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  432. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  433. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  434. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  435. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  436. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  437. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  438. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  439. 17: (FileStore::_do_op(FileStore::OpSequencer*)+0x90) [0x6ef800]
  440. 18: (ThreadPool::worker()+0x8f1) [0x826e71]
  441. 19: (ThreadPool::WorkThread::entry()+0xd) [0x663d3d]
  442. 20: (()+0x7e9a) [0x7fe972796e9a]
  443. 21: (clone()+0x6d) [0x7fe97142fcbd]
  444. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  445.  
  446. --- end dump of recent events ---
  447. 2013-01-09 05:54:01.281493 7f452fbe4780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  448. 2013-01-09 05:54:01.281502 7f452fbe4780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  449. 2013-01-09 05:54:01.281708 7f452fbe4780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  450. 2013-01-09 05:54:01.323043 7f452fbe4780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  451. 2013-01-09 05:54:01.323130 7f452fbe4780 0 filestore(/home/cb1/osd) mount found snaps <>
  452. 2013-01-09 05:54:01.427338 7f452fbe4780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  453. 2013-01-09 05:54:02.585604 7f452fbe4780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  454. 2013-01-09 05:54:02.767350 7f452fbe4780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7f452fbe4780 time 2013-01-09 05:54:02.765500
  455. os/DBObjectMap.cc: 1131: FAILED assert(0)
  456.  
  457. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  458. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  459. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  460. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  461. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  462. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  463. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  464. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  465. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  466. 9: (FileStore::mount()+0x322a) [0x71084a]
  467. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  468. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  469. 12: (main()+0x1fc3) [0x5992c3]
  470. 13: (__libc_start_main()+0xed) [0x7f452dc3f76d]
  471. 14: /usr/bin/ceph-osd() [0x59b5a9]
  472. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  473.  
  474. --- begin dump of recent events ---
  475. -42> 2013-01-09 05:54:01.171357 7f452fbe4780 5 asok(0x2ef8000) register_command perfcounters_dump hook 0x2ee0010
  476. -41> 2013-01-09 05:54:01.171380 7f452fbe4780 5 asok(0x2ef8000) register_command 1 hook 0x2ee0010
  477. -40> 2013-01-09 05:54:01.171381 7f452fbe4780 5 asok(0x2ef8000) register_command perf dump hook 0x2ee0010
  478. -39> 2013-01-09 05:54:01.171387 7f452fbe4780 5 asok(0x2ef8000) register_command perfcounters_schema hook 0x2ee0010
  479. -38> 2013-01-09 05:54:01.171390 7f452fbe4780 5 asok(0x2ef8000) register_command 2 hook 0x2ee0010
  480. -37> 2013-01-09 05:54:01.171393 7f452fbe4780 5 asok(0x2ef8000) register_command perf schema hook 0x2ee0010
  481. -36> 2013-01-09 05:54:01.171395 7f452fbe4780 5 asok(0x2ef8000) register_command config show hook 0x2ee0010
  482. -35> 2013-01-09 05:54:01.171397 7f452fbe4780 5 asok(0x2ef8000) register_command config set hook 0x2ee0010
  483. -34> 2013-01-09 05:54:01.171399 7f452fbe4780 5 asok(0x2ef8000) register_command log flush hook 0x2ee0010
  484. -33> 2013-01-09 05:54:01.171401 7f452fbe4780 5 asok(0x2ef8000) register_command log dump hook 0x2ee0010
  485. -32> 2013-01-09 05:54:01.171403 7f452fbe4780 5 asok(0x2ef8000) register_command log reopen hook 0x2ee0010
  486. -31> 2013-01-09 05:54:01.217624 7f452fbe4780 0 ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd), process ceph-osd, pid 25816
  487. -30> 2013-01-09 05:54:01.217673 7f452fbe4780 1 -- 172.27.124.153:6800/25816 accepter.bind my_inst.addr is 172.27.124.153:6800/25816 need_addr=0
  488. -29> 2013-01-09 05:54:01.217691 7f452fbe4780 1 -- 10.75.76.153:6800/25816 accepter.bind my_inst.addr is 10.75.76.153:6800/25816 need_addr=0
  489. -28> 2013-01-09 05:54:01.217701 7f452fbe4780 1 -- 10.75.76.153:6801/25816 accepter.bind my_inst.addr is 10.75.76.153:6801/25816 need_addr=0
  490. -27> 2013-01-09 05:54:01.217927 7f452fbe4780 1 finished global_init_daemonize
  491. -26> 2013-01-09 05:54:01.219437 7f452fbe4780 5 asok(0x2ef8000) init /var/run/ceph/ceph-osd.16.asok
  492. -25> 2013-01-09 05:54:01.219451 7f452fbe4780 5 asok(0x2ef8000) bind_and_listen /var/run/ceph/ceph-osd.16.asok
  493. -24> 2013-01-09 05:54:01.219475 7f452fbe4780 5 asok(0x2ef8000) register_command 0 hook 0x2ede0a0
  494. -23> 2013-01-09 05:54:01.219479 7f452fbe4780 5 asok(0x2ef8000) register_command version hook 0x2ede0a0
  495. -22> 2013-01-09 05:54:01.219483 7f452fbe4780 5 asok(0x2ef8000) register_command git_version hook 0x2ede0a0
  496. -21> 2013-01-09 05:54:01.219485 7f452fbe4780 5 asok(0x2ef8000) register_command help hook 0x2ee00c0
  497. -20> 2013-01-09 05:54:01.219503 7f452bc8f700 5 asok(0x2ef8000) entry start
  498. -19> 2013-01-09 05:54:01.219572 7f452fbe4780 5 filestore(/home/cb1/osd) basedir /home/cb1/osd journal /var/lib/ceph/osd/ceph-16/journal
  499. -18> 2013-01-09 05:54:01.281493 7f452fbe4780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  500. -17> 2013-01-09 05:54:01.281502 7f452fbe4780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  501. -16> 2013-01-09 05:54:01.281708 7f452fbe4780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  502. -15> 2013-01-09 05:54:01.323043 7f452fbe4780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  503. -14> 2013-01-09 05:54:01.323130 7f452fbe4780 0 filestore(/home/cb1/osd) mount found snaps <>
  504. -13> 2013-01-09 05:54:01.323149 7f452fbe4780 5 filestore(/home/cb1/osd) mount op_seq is 3638350
  505. -12> 2013-01-09 05:54:01.427338 7f452fbe4780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  506. -11> 2013-01-09 05:54:02.585530 7f452fbe4780 2 journal open /var/lib/ceph/osd/ceph-16/journal fsid c9424f98-e5b6-498b-8d29-4ef0042ac139 fs_op_seq 3638350
  507. -10> 2013-01-09 05:54:02.585604 7f452fbe4780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  508. -9> 2013-01-09 05:54:02.622254 7f452fbe4780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  509. -8> 2013-01-09 05:54:02.622280 7f452fbe4780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  510. -7> 2013-01-09 05:54:02.622283 7f452fbe4780 3 journal journal_replay: applying op seq 3638351
  511. -6> 2013-01-09 05:54:02.670302 7f452fbe4780 3 journal journal_replay: r = 0, op_seq now 3638351
  512. -5> 2013-01-09 05:54:02.671949 7f452fbe4780 2 journal read_entry 3399671808 : seq 3638352 76998 bytes
  513. -4> 2013-01-09 05:54:02.671961 7f452fbe4780 3 journal journal_replay: applying op seq 3638352
  514. -3> 2013-01-09 05:54:02.734229 7f452fbe4780 3 journal journal_replay: r = 0, op_seq now 3638352
  515. -2> 2013-01-09 05:54:02.734256 7f452fbe4780 2 journal read_entry 3399753728 : seq 3638353 4394 bytes
  516. -1> 2013-01-09 05:54:02.734259 7f452fbe4780 3 journal journal_replay: applying op seq 3638353
  517. 0> 2013-01-09 05:54:02.767350 7f452fbe4780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7f452fbe4780 time 2013-01-09 05:54:02.765500
  518. os/DBObjectMap.cc: 1131: FAILED assert(0)
  519.  
  520. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  521. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  522. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  523. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  524. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  525. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  526. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  527. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  528. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  529. 9: (FileStore::mount()+0x322a) [0x71084a]
  530. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  531. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  532. 12: (main()+0x1fc3) [0x5992c3]
  533. 13: (__libc_start_main()+0xed) [0x7f452dc3f76d]
  534. 14: /usr/bin/ceph-osd() [0x59b5a9]
  535. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  536.  
  537. --- end dump of recent events ---
  538. 2013-01-09 05:54:02.769739 7f452fbe4780 -1 *** Caught signal (Aborted) **
  539. in thread 7f452fbe4780
  540.  
  541. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  542. 1: /usr/bin/ceph-osd() [0x778de0]
  543. 2: (()+0xfcb0) [0x7f452f080cb0]
  544. 3: (gsignal()+0x35) [0x7f452dc54425]
  545. 4: (abort()+0x17b) [0x7f452dc57b8b]
  546. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f452e54fe2d]
  547. 6: (()+0x5ef26) [0x7f452e54df26]
  548. 7: (()+0x5ef53) [0x7f452e54df53]
  549. 8: (()+0x5f17e) [0x7f452e54e17e]
  550. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  551. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  552. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  553. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  554. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  555. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  556. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  557. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  558. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  559. 18: (FileStore::mount()+0x322a) [0x71084a]
  560. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  561. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  562. 21: (main()+0x1fc3) [0x5992c3]
  563. 22: (__libc_start_main()+0xed) [0x7f452dc3f76d]
  564. 23: /usr/bin/ceph-osd() [0x59b5a9]
  565. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  566.  
  567. --- begin dump of recent events ---
  568. 0> 2013-01-09 05:54:02.769739 7f452fbe4780 -1 *** Caught signal (Aborted) **
  569. in thread 7f452fbe4780
  570.  
  571. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  572. 1: /usr/bin/ceph-osd() [0x778de0]
  573. 2: (()+0xfcb0) [0x7f452f080cb0]
  574. 3: (gsignal()+0x35) [0x7f452dc54425]
  575. 4: (abort()+0x17b) [0x7f452dc57b8b]
  576. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f452e54fe2d]
  577. 6: (()+0x5ef26) [0x7f452e54df26]
  578. 7: (()+0x5ef53) [0x7f452e54df53]
  579. 8: (()+0x5f17e) [0x7f452e54e17e]
  580. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  581. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  582. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  583. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  584. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  585. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  586. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  587. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  588. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  589. 18: (FileStore::mount()+0x322a) [0x71084a]
  590. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  591. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  592. 21: (main()+0x1fc3) [0x5992c3]
  593. 22: (__libc_start_main()+0xed) [0x7f452dc3f76d]
  594. 23: /usr/bin/ceph-osd() [0x59b5a9]
  595. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  596.  
  597. --- end dump of recent events ---
  598. 2013-01-09 05:55:32.067571 7fa1724b5780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  599. 2013-01-09 05:55:32.067591 7fa1724b5780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  600. 2013-01-09 05:55:32.067974 7fa1724b5780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  601. 2013-01-09 05:55:32.113426 7fa1724b5780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  602. 2013-01-09 05:55:32.113514 7fa1724b5780 0 filestore(/home/cb1/osd) mount found snaps <>
  603. 2013-01-09 05:55:32.163915 7fa1724b5780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  604. 2013-01-09 05:55:32.177693 7fa1724b5780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  605. 2013-01-09 05:55:32.185170 7fa1724b5780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7fa1724b5780 time 2013-01-09 05:55:32.183265
  606. os/DBObjectMap.cc: 1131: FAILED assert(0)
  607.  
  608. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  609. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  610. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  611. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  612. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  613. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  614. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  615. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  616. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  617. 9: (FileStore::mount()+0x322a) [0x71084a]
  618. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  619. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  620. 12: (main()+0x1fc3) [0x5992c3]
  621. 13: (__libc_start_main()+0xed) [0x7fa17051076d]
  622. 14: /usr/bin/ceph-osd() [0x59b5a9]
  623. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  624.  
  625. --- begin dump of recent events ---
  626. -42> 2013-01-09 05:55:31.881765 7fa1724b5780 5 asok(0x28fc000) register_command perfcounters_dump hook 0x28e4010
  627. -41> 2013-01-09 05:55:31.881789 7fa1724b5780 5 asok(0x28fc000) register_command 1 hook 0x28e4010
  628. -40> 2013-01-09 05:55:31.881791 7fa1724b5780 5 asok(0x28fc000) register_command perf dump hook 0x28e4010
  629. -39> 2013-01-09 05:55:31.881795 7fa1724b5780 5 asok(0x28fc000) register_command perfcounters_schema hook 0x28e4010
  630. -38> 2013-01-09 05:55:31.881798 7fa1724b5780 5 asok(0x28fc000) register_command 2 hook 0x28e4010
  631. -37> 2013-01-09 05:55:31.881800 7fa1724b5780 5 asok(0x28fc000) register_command perf schema hook 0x28e4010
  632. -36> 2013-01-09 05:55:31.881803 7fa1724b5780 5 asok(0x28fc000) register_command config show hook 0x28e4010
  633. -35> 2013-01-09 05:55:31.881805 7fa1724b5780 5 asok(0x28fc000) register_command config set hook 0x28e4010
  634. -34> 2013-01-09 05:55:31.881808 7fa1724b5780 5 asok(0x28fc000) register_command log flush hook 0x28e4010
  635. -33> 2013-01-09 05:55:31.881811 7fa1724b5780 5 asok(0x28fc000) register_command log dump hook 0x28e4010
  636. -32> 2013-01-09 05:55:31.881814 7fa1724b5780 5 asok(0x28fc000) register_command log reopen hook 0x28e4010
  637. -31> 2013-01-09 05:55:31.882930 7fa1724b5780 0 ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd), process ceph-osd, pid 26215
  638. -30> 2013-01-09 05:55:31.883011 7fa1724b5780 1 -- 172.27.124.153:6800/26215 accepter.bind my_inst.addr is 172.27.124.153:6800/26215 need_addr=0
  639. -29> 2013-01-09 05:55:31.883033 7fa1724b5780 1 -- 10.75.76.153:6800/26215 accepter.bind my_inst.addr is 10.75.76.153:6800/26215 need_addr=0
  640. -28> 2013-01-09 05:55:31.883044 7fa1724b5780 1 -- 10.75.76.153:6801/26215 accepter.bind my_inst.addr is 10.75.76.153:6801/26215 need_addr=0
  641. -27> 2013-01-09 05:55:31.883330 7fa1724b5780 1 finished global_init_daemonize
  642. -26> 2013-01-09 05:55:31.884861 7fa1724b5780 5 asok(0x28fc000) init /var/run/ceph/ceph-osd.16.asok
  643. -25> 2013-01-09 05:55:31.884872 7fa1724b5780 5 asok(0x28fc000) bind_and_listen /var/run/ceph/ceph-osd.16.asok
  644. -24> 2013-01-09 05:55:31.884896 7fa1724b5780 5 asok(0x28fc000) register_command 0 hook 0x28e20a0
  645. -23> 2013-01-09 05:55:31.884899 7fa1724b5780 5 asok(0x28fc000) register_command version hook 0x28e20a0
  646. -22> 2013-01-09 05:55:31.884902 7fa1724b5780 5 asok(0x28fc000) register_command git_version hook 0x28e20a0
  647. -21> 2013-01-09 05:55:31.884903 7fa1724b5780 5 asok(0x28fc000) register_command help hook 0x28e40c0
  648. -20> 2013-01-09 05:55:31.884940 7fa16e560700 5 asok(0x28fc000) entry start
  649. -19> 2013-01-09 05:55:31.884984 7fa1724b5780 5 filestore(/home/cb1/osd) basedir /home/cb1/osd journal /var/lib/ceph/osd/ceph-16/journal
  650. -18> 2013-01-09 05:55:32.067571 7fa1724b5780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  651. -17> 2013-01-09 05:55:32.067591 7fa1724b5780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  652. -16> 2013-01-09 05:55:32.067974 7fa1724b5780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  653. -15> 2013-01-09 05:55:32.113426 7fa1724b5780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  654. -14> 2013-01-09 05:55:32.113514 7fa1724b5780 0 filestore(/home/cb1/osd) mount found snaps <>
  655. -13> 2013-01-09 05:55:32.113530 7fa1724b5780 5 filestore(/home/cb1/osd) mount op_seq is 3638350
  656. -12> 2013-01-09 05:55:32.163915 7fa1724b5780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  657. -11> 2013-01-09 05:55:32.177660 7fa1724b5780 2 journal open /var/lib/ceph/osd/ceph-16/journal fsid c9424f98-e5b6-498b-8d29-4ef0042ac139 fs_op_seq 3638350
  658. -10> 2013-01-09 05:55:32.177693 7fa1724b5780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  659. -9> 2013-01-09 05:55:32.177737 7fa1724b5780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  660. -8> 2013-01-09 05:55:32.177753 7fa1724b5780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  661. -7> 2013-01-09 05:55:32.177755 7fa1724b5780 3 journal journal_replay: applying op seq 3638351
  662. -6> 2013-01-09 05:55:32.177997 7fa1724b5780 3 journal journal_replay: r = 0, op_seq now 3638351
  663. -5> 2013-01-09 05:55:32.178079 7fa1724b5780 2 journal read_entry 3399671808 : seq 3638352 76998 bytes
  664. -4> 2013-01-09 05:55:32.178084 7fa1724b5780 3 journal journal_replay: applying op seq 3638352
  665. -3> 2013-01-09 05:55:32.182788 7fa1724b5780 3 journal journal_replay: r = 0, op_seq now 3638352
  666. -2> 2013-01-09 05:55:32.182804 7fa1724b5780 2 journal read_entry 3399753728 : seq 3638353 4394 bytes
  667. -1> 2013-01-09 05:55:32.182807 7fa1724b5780 3 journal journal_replay: applying op seq 3638353
  668. 0> 2013-01-09 05:55:32.185170 7fa1724b5780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7fa1724b5780 time 2013-01-09 05:55:32.183265
  669. os/DBObjectMap.cc: 1131: FAILED assert(0)
  670.  
  671. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  672. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  673. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  674. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  675. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  676. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  677. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  678. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  679. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  680. 9: (FileStore::mount()+0x322a) [0x71084a]
  681. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  682. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  683. 12: (main()+0x1fc3) [0x5992c3]
  684. 13: (__libc_start_main()+0xed) [0x7fa17051076d]
  685. 14: /usr/bin/ceph-osd() [0x59b5a9]
  686. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  687.  
  688. --- end dump of recent events ---
  689. 2013-01-09 05:55:32.187528 7fa1724b5780 -1 *** Caught signal (Aborted) **
  690. in thread 7fa1724b5780
  691.  
  692. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  693. 1: /usr/bin/ceph-osd() [0x778de0]
  694. 2: (()+0xfcb0) [0x7fa171951cb0]
  695. 3: (gsignal()+0x35) [0x7fa170525425]
  696. 4: (abort()+0x17b) [0x7fa170528b8b]
  697. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fa170e20e2d]
  698. 6: (()+0x5ef26) [0x7fa170e1ef26]
  699. 7: (()+0x5ef53) [0x7fa170e1ef53]
  700. 8: (()+0x5f17e) [0x7fa170e1f17e]
  701. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  702. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  703. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  704. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  705. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  706. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  707. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  708. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  709. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  710. 18: (FileStore::mount()+0x322a) [0x71084a]
  711. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  712. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  713. 21: (main()+0x1fc3) [0x5992c3]
  714. 22: (__libc_start_main()+0xed) [0x7fa17051076d]
  715. 23: /usr/bin/ceph-osd() [0x59b5a9]
  716. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  717.  
  718. --- begin dump of recent events ---
  719. 0> 2013-01-09 05:55:32.187528 7fa1724b5780 -1 *** Caught signal (Aborted) **
  720. in thread 7fa1724b5780
  721.  
  722. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  723. 1: /usr/bin/ceph-osd() [0x778de0]
  724. 2: (()+0xfcb0) [0x7fa171951cb0]
  725. 3: (gsignal()+0x35) [0x7fa170525425]
  726. 4: (abort()+0x17b) [0x7fa170528b8b]
  727. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fa170e20e2d]
  728. 6: (()+0x5ef26) [0x7fa170e1ef26]
  729. 7: (()+0x5ef53) [0x7fa170e1ef53]
  730. 8: (()+0x5f17e) [0x7fa170e1f17e]
  731. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  732. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  733. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  734. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  735. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  736. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  737. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  738. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  739. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  740. 18: (FileStore::mount()+0x322a) [0x71084a]
  741. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  742. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  743. 21: (main()+0x1fc3) [0x5992c3]
  744. 22: (__libc_start_main()+0xed) [0x7fa17051076d]
  745. 23: /usr/bin/ceph-osd() [0x59b5a9]
  746. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  747.  
  748. --- end dump of recent events ---
  749. 2013-01-09 06:02:33.449238 7fd9bb40f780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  750. 2013-01-09 06:02:33.449257 7fd9bb40f780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  751. 2013-01-09 06:02:33.449551 7fd9bb40f780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  752. 2013-01-09 06:02:33.490601 7fd9bb40f780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  753. 2013-01-09 06:02:33.490701 7fd9bb40f780 0 filestore(/home/cb1/osd) mount found snaps <>
  754. 2013-01-09 06:02:33.532733 7fd9bb40f780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  755. 2013-01-09 06:02:33.546506 7fd9bb40f780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  756. 2013-01-09 06:02:33.555097 7fd9bb40f780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7fd9bb40f780 time 2013-01-09 06:02:33.553271
  757. os/DBObjectMap.cc: 1131: FAILED assert(0)
  758.  
  759. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  760. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  761. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  762. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  763. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  764. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  765. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  766. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  767. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  768. 9: (FileStore::mount()+0x322a) [0x71084a]
  769. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  770. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  771. 12: (main()+0x1fc3) [0x5992c3]
  772. 13: (__libc_start_main()+0xed) [0x7fd9b946a76d]
  773. 14: /usr/bin/ceph-osd() [0x59b5a9]
  774. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  775.  
  776. --- begin dump of recent events ---
  777. -42> 2013-01-09 06:02:33.381504 7fd9bb40f780 5 asok(0x1dbc000) register_command perfcounters_dump hook 0x1da4010
  778. -41> 2013-01-09 06:02:33.381527 7fd9bb40f780 5 asok(0x1dbc000) register_command 1 hook 0x1da4010
  779. -40> 2013-01-09 06:02:33.381530 7fd9bb40f780 5 asok(0x1dbc000) register_command perf dump hook 0x1da4010
  780. -39> 2013-01-09 06:02:33.381535 7fd9bb40f780 5 asok(0x1dbc000) register_command perfcounters_schema hook 0x1da4010
  781. -38> 2013-01-09 06:02:33.381538 7fd9bb40f780 5 asok(0x1dbc000) register_command 2 hook 0x1da4010
  782. -37> 2013-01-09 06:02:33.381540 7fd9bb40f780 5 asok(0x1dbc000) register_command perf schema hook 0x1da4010
  783. -36> 2013-01-09 06:02:33.381543 7fd9bb40f780 5 asok(0x1dbc000) register_command config show hook 0x1da4010
  784. -35> 2013-01-09 06:02:33.381546 7fd9bb40f780 5 asok(0x1dbc000) register_command config set hook 0x1da4010
  785. -34> 2013-01-09 06:02:33.381548 7fd9bb40f780 5 asok(0x1dbc000) register_command log flush hook 0x1da4010
  786. -33> 2013-01-09 06:02:33.381550 7fd9bb40f780 5 asok(0x1dbc000) register_command log dump hook 0x1da4010
  787. -32> 2013-01-09 06:02:33.381552 7fd9bb40f780 5 asok(0x1dbc000) register_command log reopen hook 0x1da4010
  788. -31> 2013-01-09 06:02:33.382673 7fd9bb40f780 0 ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd), process ceph-osd, pid 26708
  789. -30> 2013-01-09 06:02:33.382721 7fd9bb40f780 1 -- 172.27.124.153:6800/26708 accepter.bind my_inst.addr is 172.27.124.153:6800/26708 need_addr=0
  790. -29> 2013-01-09 06:02:33.382740 7fd9bb40f780 1 -- 10.75.76.153:6800/26708 accepter.bind my_inst.addr is 10.75.76.153:6800/26708 need_addr=0
  791. -28> 2013-01-09 06:02:33.382751 7fd9bb40f780 1 -- 10.75.76.153:6801/26708 accepter.bind my_inst.addr is 10.75.76.153:6801/26708 need_addr=0
  792. -27> 2013-01-09 06:02:33.382980 7fd9bb40f780 1 finished global_init_daemonize
  793. -26> 2013-01-09 06:02:33.384478 7fd9bb40f780 5 asok(0x1dbc000) init /var/run/ceph/ceph-osd.16.asok
  794. -25> 2013-01-09 06:02:33.384492 7fd9bb40f780 5 asok(0x1dbc000) bind_and_listen /var/run/ceph/ceph-osd.16.asok
  795. -24> 2013-01-09 06:02:33.384516 7fd9bb40f780 5 asok(0x1dbc000) register_command 0 hook 0x1da20a0
  796. -23> 2013-01-09 06:02:33.384521 7fd9bb40f780 5 asok(0x1dbc000) register_command version hook 0x1da20a0
  797. -22> 2013-01-09 06:02:33.384524 7fd9bb40f780 5 asok(0x1dbc000) register_command git_version hook 0x1da20a0
  798. -21> 2013-01-09 06:02:33.384527 7fd9bb40f780 5 asok(0x1dbc000) register_command help hook 0x1da40c0
  799. -20> 2013-01-09 06:02:33.384544 7fd9b74ba700 5 asok(0x1dbc000) entry start
  800. -19> 2013-01-09 06:02:33.384605 7fd9bb40f780 5 filestore(/home/cb1/osd) basedir /home/cb1/osd journal /var/lib/ceph/osd/ceph-16/journal
  801. -18> 2013-01-09 06:02:33.449238 7fd9bb40f780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is supported and appears to work
  802. -17> 2013-01-09 06:02:33.449257 7fd9bb40f780 0 filestore(/home/cb1/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  803. -16> 2013-01-09 06:02:33.449551 7fd9bb40f780 0 filestore(/home/cb1/osd) mount did NOT detect btrfs
  804. -15> 2013-01-09 06:02:33.490601 7fd9bb40f780 0 filestore(/home/cb1/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  805. -14> 2013-01-09 06:02:33.490701 7fd9bb40f780 0 filestore(/home/cb1/osd) mount found snaps <>
  806. -13> 2013-01-09 06:02:33.490719 7fd9bb40f780 5 filestore(/home/cb1/osd) mount op_seq is 3638350
  807. -12> 2013-01-09 06:02:33.532733 7fd9bb40f780 0 filestore(/home/cb1/osd) mount: enabling WRITEAHEAD journal mode: btrfs not detected
  808. -11> 2013-01-09 06:02:33.546472 7fd9bb40f780 2 journal open /var/lib/ceph/osd/ceph-16/journal fsid c9424f98-e5b6-498b-8d29-4ef0042ac139 fs_op_seq 3638350
  809. -10> 2013-01-09 06:02:33.546506 7fd9bb40f780 1 journal _open /var/lib/ceph/osd/ceph-16/journal fd 21: 5242880000 bytes, block size 4096 bytes, directio = 1, aio = 0
  810. -9> 2013-01-09 06:02:33.546550 7fd9bb40f780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  811. -8> 2013-01-09 06:02:33.546563 7fd9bb40f780 2 journal read_entry 3399651328 : seq 3638351 13436 bytes
  812. -7> 2013-01-09 06:02:33.546565 7fd9bb40f780 3 journal journal_replay: applying op seq 3638351
  813. -6> 2013-01-09 06:02:33.546811 7fd9bb40f780 3 journal journal_replay: r = 0, op_seq now 3638351
  814. -5> 2013-01-09 06:02:33.546894 7fd9bb40f780 2 journal read_entry 3399671808 : seq 3638352 76998 bytes
  815. -4> 2013-01-09 06:02:33.546896 7fd9bb40f780 3 journal journal_replay: applying op seq 3638352
  816. -3> 2013-01-09 06:02:33.552771 7fd9bb40f780 3 journal journal_replay: r = 0, op_seq now 3638352
  817. -2> 2013-01-09 06:02:33.552794 7fd9bb40f780 2 journal read_entry 3399753728 : seq 3638353 4394 bytes
  818. -1> 2013-01-09 06:02:33.552799 7fd9bb40f780 3 journal journal_replay: applying op seq 3638353
  819. 0> 2013-01-09 06:02:33.555097 7fd9bb40f780 -1 os/DBObjectMap.cc: In function 'DBObjectMap::Header DBObjectMap::lookup_parent(DBObjectMap::Header)' thread 7fd9bb40f780 time 2013-01-09 06:02:33.553271
  820. os/DBObjectMap.cc: 1131: FAILED assert(0)
  821.  
  822. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  823. 1: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  824. 2: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  825. 3: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  826. 4: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  827. 5: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  828. 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  829. 7: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  830. 8: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  831. 9: (FileStore::mount()+0x322a) [0x71084a]
  832. 10: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  833. 11: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  834. 12: (main()+0x1fc3) [0x5992c3]
  835. 13: (__libc_start_main()+0xed) [0x7fd9b946a76d]
  836. 14: /usr/bin/ceph-osd() [0x59b5a9]
  837. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  838.  
  839. --- end dump of recent events ---
  840. 2013-01-09 06:02:33.557474 7fd9bb40f780 -1 *** Caught signal (Aborted) **
  841. in thread 7fd9bb40f780
  842.  
  843. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  844. 1: /usr/bin/ceph-osd() [0x778de0]
  845. 2: (()+0xfcb0) [0x7fd9ba8abcb0]
  846. 3: (gsignal()+0x35) [0x7fd9b947f425]
  847. 4: (abort()+0x17b) [0x7fd9b9482b8b]
  848. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fd9b9d7ae2d]
  849. 6: (()+0x5ef26) [0x7fd9b9d78f26]
  850. 7: (()+0x5ef53) [0x7fd9b9d78f53]
  851. 8: (()+0x5f17e) [0x7fd9b9d7917e]
  852. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  853. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  854. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  855. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  856. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  857. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  858. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  859. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  860. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  861. 18: (FileStore::mount()+0x322a) [0x71084a]
  862. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  863. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  864. 21: (main()+0x1fc3) [0x5992c3]
  865. 22: (__libc_start_main()+0xed) [0x7fd9b946a76d]
  866. 23: /usr/bin/ceph-osd() [0x59b5a9]
  867. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  868.  
  869. --- begin dump of recent events ---
  870. 0> 2013-01-09 06:02:33.557474 7fd9bb40f780 -1 *** Caught signal (Aborted) **
  871. in thread 7fd9bb40f780
  872.  
  873. ceph version 0.48.3argonaut (commit:920f82e805efec2cae05b79c155c07df0f3ed5dd)
  874. 1: /usr/bin/ceph-osd() [0x778de0]
  875. 2: (()+0xfcb0) [0x7fd9ba8abcb0]
  876. 3: (gsignal()+0x35) [0x7fd9b947f425]
  877. 4: (abort()+0x17b) [0x7fd9b9482b8b]
  878. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fd9b9d7ae2d]
  879. 6: (()+0x5ef26) [0x7fd9b9d78f26]
  880. 7: (()+0x5ef53) [0x7fd9b9d78f53]
  881. 8: (()+0x5f17e) [0x7fd9b9d7917e]
  882. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x3f4) [0x82ef84]
  883. 10: (DBObjectMap::lookup_parent(std::tr1::shared_ptr<DBObjectMap::_Header>)+0x599) [0x7390f9]
  884. 11: (DBObjectMap::_clear(std::tr1::shared_ptr<DBObjectMap::_Header>, std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0xae) [0x73b8de]
  885. 12: (DBObjectMap::clear(hobject_t const&, SequencerPosition const*)+0x145) [0x740c95]
  886. 13: (FileStore::lfn_unlink(coll_t, hobject_t const&, SequencerPosition const&)+0x23d) [0x703bcd]
  887. 14: (FileStore::_remove(coll_t, hobject_t const&, SequencerPosition const&)+0x7f) [0x703cdf]
  888. 15: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x12fa) [0x7165aa]
  889. 16: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x7e) [0x718f9e]
  890. 17: (JournalingObjectStore::journal_replay(unsigned long)+0x81b) [0x7273fb]
  891. 18: (FileStore::mount()+0x322a) [0x71084a]
  892. 19: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x63035a]
  893. 20: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x6310b7]
  894. 21: (main()+0x1fc3) [0x5992c3]
  895. 22: (__libc_start_main()+0xed) [0x7fd9b946a76d]
  896. 23: /usr/bin/ceph-osd() [0x59b5a9]
  897. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  898.  
  899. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!