API tools faq
paste
Advertisement
rlr219

new_log

rlr219
Oct 16th, 2012
38
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 40.56 KB | None | 0 0
raw download clone embed print report
  1. 2012-10-16 17:11:18.740500 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  2. 2012-10-16 17:11:18.742748 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 218
  3. 2012-10-16 17:11:18.742776 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  4. 2012-10-16 17:11:18.742835 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 663
  5. 2012-10-16 17:11:18.745904 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  6. 2012-10-16 17:11:18.745931 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217343~135
  7. 2012-10-16 17:11:18.745995 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 29 4217343~135 qlen 8
  8. 2012-10-16 17:11:18.746012 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217343~135 = 135
  9. 2012-10-16 17:11:18.746028 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30
  10. 2012-10-16 17:11:18.746059 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30 = 30
  11. 2012-10-16 17:11:18.746076 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508
  12. 2012-10-16 17:11:18.746104 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508 = 508
  13. 2012-10-16 17:11:18.746120 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0
  14. 2012-10-16 17:11:18.746185 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0 = 0
  15. 2012-10-16 17:11:18.746207 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784
  16. 2012-10-16 17:11:18.746254 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 33 0~784 qlen 9
  17. 2012-10-16 17:11:18.746271 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784 = 784
  18. 2012-10-16 17:11:18.746284 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  19. 2012-10-16 17:11:18.746297 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 1080832~3113472
  20. 2012-10-16 17:11:18.747472 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 34 1080832~3113472 qlen 10
  21. 2012-10-16 17:11:18.747497 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 1080832~3113472 = 3113472
  22. 2012-10-16 17:11:18.747515 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  23. 2012-10-16 17:11:18.747576 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 218
  24. 2012-10-16 17:11:18.747599 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  25. 2012-10-16 17:11:18.747652 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 647
  26. 2012-10-16 17:11:18.751144 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  27. 2012-10-16 17:11:18.751175 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237161~302
  28. 2012-10-16 17:11:18.751242 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 35 237161~302 qlen 11
  29. 2012-10-16 17:11:18.751259 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237161~302 = 302
  30. 2012-10-16 17:11:18.751274 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30
  31. 2012-10-16 17:11:18.751289 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30 = -2
  32. 2012-10-16 17:11:18.751304 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508
  33. 2012-10-16 17:11:18.751317 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508 = -2
  34. 2012-10-16 17:11:18.751331 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0
  35. 2012-10-16 17:11:18.751401 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0 = 0
  36. 2012-10-16 17:11:18.751423 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134
  37. 2012-10-16 17:11:18.751470 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 36 0~134 qlen 12
  38. 2012-10-16 17:11:18.751487 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134 = 134
  39. 2012-10-16 17:11:18.751500 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  40. 2012-10-16 17:11:18.751513 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) clone 4.2_head/fe56915a/rb.0.9.000000000dee/head//4 -> 4.2_head/fe56915a/rb.0.9.000000000dee/e4//4
  41. 2012-10-16 17:11:18.751530 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) error getting collection index for 4.2_head: (2) No such file or directory
  42. 2012-10-16 17:11:18.751555 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) clone 4.2_head/fe56915a/rb.0.9.000000000dee/head//4 -> 4.2_head/fe56915a/rb.0.9.000000000dee/e4//4 = -2
  43. 2012-10-16 17:11:18.751569 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) error (2) No such file or directory not handled on operation 17 (50235141.1.0, or op 0, counting from 0)
  44. 2012-10-16 17:11:18.751585 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) ENOENT on clone suggests osd bug
  45. 2012-10-16 17:11:18.751596 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) transaction dump:
  46. { "ops": [
  47. { "op_num": 0,
  48. "op_name": "clone",
  49. "collection": "4.2_head",
  50. "src_oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  51. "dst_oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  52. { "op_num": 1,
  53. "op_name": "setattr",
  54. "collection": "4.2_head",
  55. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4",
  56. "name": "_",
  57. "length": 219},
  58. { "op_num": 2,
  59. "op_name": "rmattr",
  60. "collection": "4.2_head",
  61. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4",
  62. "name": "snapset"},
  63. { "op_num": 3,
  64. "op_name": "collection_add",
  65. "src_collection": "4.2_e4",
  66. "dst_collection": "4.2_head",
  67. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  68. { "op_num": 4,
  69. "op_name": "collection_add",
  70. "src_collection": "4.2_db",
  71. "dst_collection": "4.2_head",
  72. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  73. { "op_num": 5,
  74. "op_name": "write",
  75. "collection": "4.2_head",
  76. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  77. "length": 4194304,
  78. "offset": 0,
  79. "bufferlist length": 4194304},
  80. { "op_num": 6,
  81. "op_name": "setattr",
  82. "collection": "4.2_head",
  83. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  84. "name": "_",
  85. "length": 218},
  86. { "op_num": 7,
  87. "op_name": "setattr",
  88. "collection": "4.2_head",
  89. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  90. "name": "snapset",
  91. "length": 647}]}
  92. 2012-10-16 17:11:18.752702 7f402d40d780 -1 os/FileStore.cc: In function 'unsigned int FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int)' thread 7f402d40d780 time 2012-10-16 17:11:18.751687
  93. os/FileStore.cc: 3007: FAILED assert(0 == "unexpected error")
  94.  
  95. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  96. 1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0xa6b) [0x684d8b]
  97. 2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x84) [0x68aa74]
  98. 3: (JournalingObjectStore::journal_replay(unsigned long)+0x884) [0x6997c4]
  99. 4: (FileStore::mount()+0x35a1) [0x67daf1]
  100. 5: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x59f49a]
  101. 6: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x59ff07]
  102. 7: (main()+0x2256) [0x517f46]
  103. 8: (__libc_start_main()+0xed) [0x7f402b46f76d]
  104. 9: /usr/bin/ceph-osd() [0x51a239]
  105. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  106.  
  107. --- begin dump of recent events ---
  108. -196> 2012-10-16 17:11:17.275632 7f402d40d780 5 asok(0x222f000) register_command perfcounters_dump hook 0x2224010
  109. -195> 2012-10-16 17:11:17.275649 7f402d40d780 5 asok(0x222f000) register_command 1 hook 0x2224010
  110. -194> 2012-10-16 17:11:17.275650 7f402d40d780 5 asok(0x222f000) register_command perf dump hook 0x2224010
  111. -193> 2012-10-16 17:11:17.275655 7f402d40d780 5 asok(0x222f000) register_command perfcounters_schema hook 0x2224010
  112. -192> 2012-10-16 17:11:17.275657 7f402d40d780 5 asok(0x222f000) register_command 2 hook 0x2224010
  113. -191> 2012-10-16 17:11:17.275658 7f402d40d780 5 asok(0x222f000) register_command perf schema hook 0x2224010
  114. -190> 2012-10-16 17:11:17.275660 7f402d40d780 5 asok(0x222f000) register_command config show hook 0x2224010
  115. -189> 2012-10-16 17:11:17.275662 7f402d40d780 5 asok(0x222f000) register_command config set hook 0x2224010
  116. -188> 2012-10-16 17:11:17.275663 7f402d40d780 5 asok(0x222f000) register_command log flush hook 0x2224010
  117. -187> 2012-10-16 17:11:17.275664 7f402d40d780 5 asok(0x222f000) register_command log dump hook 0x2224010
  118. -186> 2012-10-16 17:11:17.275665 7f402d40d780 5 asok(0x222f000) register_command log reopen hook 0x2224010
  119. -185> 2012-10-16 17:11:17.276804 7f402d40d780 0 ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe), process ceph-osd, pid 28467
  120. -184> 2012-10-16 17:11:17.276895 7f402d40d780 1 -- 172.27.124.141:6800/28467 accepter.bind my_inst.addr is 172.27.124.141:6800/28467 need_addr=0
  121. -183> 2012-10-16 17:11:17.276911 7f402d40d780 1 -- 10.75.76.141:6800/28467 accepter.bind my_inst.addr is 10.75.76.141:6800/28467 need_addr=0
  122. -182> 2012-10-16 17:11:17.276920 7f402d40d780 1 -- 10.75.76.141:6801/28467 accepter.bind my_inst.addr is 10.75.76.141:6801/28467 need_addr=0
  123. -181> 2012-10-16 17:11:17.277295 7f402d40d780 1 finished global_init_daemonize
  124. -180> 2012-10-16 17:11:17.278800 7f402d40d780 5 asok(0x222f000) init /var/run/ceph/ceph-osd.10.asok
  125. -179> 2012-10-16 17:11:17.278811 7f402d40d780 5 asok(0x222f000) bind_and_listen /var/run/ceph/ceph-osd.10.asok
  126. -178> 2012-10-16 17:11:17.278827 7f402d40d780 5 asok(0x222f000) register_command 0 hook 0x22230a0
  127. -177> 2012-10-16 17:11:17.278830 7f402d40d780 5 asok(0x222f000) register_command version hook 0x22230a0
  128. -176> 2012-10-16 17:11:17.278833 7f402d40d780 5 asok(0x222f000) register_command git_version hook 0x22230a0
  129. -175> 2012-10-16 17:11:17.278834 7f402d40d780 5 asok(0x222f000) register_command help hook 0x22240c0
  130. -174> 2012-10-16 17:11:17.278906 7f402d40d780 5 filestore(/home/cephbrick/ceph/osd) basedir /home/cephbrick/ceph/osd journal /var/lib/ceph/osd/ceph-10/journal
  131. -173> 2012-10-16 17:11:17.278930 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) mount fsid is 61e0f2aa-0e09-44e9-9775-d61791a44968
  132. -172> 2012-10-16 17:11:17.279608 7f40294c5700 5 asok(0x222f000) entry start
  133. -171> 2012-10-16 17:11:17.362707 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount FIEMAP ioctl is supported and appears to work
  134. -170> 2012-10-16 17:11:17.362713 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
  135. -169> 2012-10-16 17:11:17.362820 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount detected btrfs
  136. -168> 2012-10-16 17:11:17.362829 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) _do_clone_range 0~1 to 0
  137. -167> 2012-10-16 17:11:17.362832 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs CLONE_RANGE ioctl is supported
  138. -166> 2012-10-16 17:11:17.679254 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs SNAP_CREATE is supported
  139. -165> 2012-10-16 17:11:17.687932 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs SNAP_DESTROY is supported
  140. -164> 2012-10-16 17:11:17.688352 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs START_SYNC is supported (transid 410217)
  141. -163> 2012-10-16 17:11:17.812521 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs WAIT_SYNC is supported
  142. -162> 2012-10-16 17:11:17.817282 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount btrfs SNAP_CREATE_V2 is supported
  143. -161> 2012-10-16 17:11:18.045746 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount syncfs(2) syscall fully supported (by glibc and kernel)
  144. -160> 2012-10-16 17:11:18.045901 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount found snaps <50235072,50235136>
  145. -159> 2012-10-16 17:11:18.045948 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) current/ seq was 50235136
  146. -158> 2012-10-16 17:11:18.045966 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) most recent snap from <50235072,50235136> is 50235136
  147. -157> 2012-10-16 17:11:18.045988 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) mount rolling back to consistent snap 50235136
  148. -156> 2012-10-16 17:11:18.195730 7f402d40d780 5 filestore(/home/cephbrick/ceph/osd) mount op_seq is 50235136
  149. -155> 2012-10-16 17:11:18.622974 7f402d40d780 20 filestore (init)dbobjectmap: seq is 620960
  150. -154> 2012-10-16 17:11:18.623026 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) open_journal at /var/lib/ceph/osd/ceph-10/journal
  151. -153> 2012-10-16 17:11:18.623058 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) mount: enabling PARALLEL journal mode: btrfs, SNAP_CREATE_V2 detected and 'filestore btrfs snap' mode is enabled
  152. -152> 2012-10-16 17:11:18.623075 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) list_collections
  153. -151> 2012-10-16 17:11:18.662275 7f402d40d780 2 journal open /var/lib/ceph/osd/ceph-10/journal fsid 61e0f2aa-0e09-44e9-9775-d61791a44968 fs_op_seq 50235136
  154. -150> 2012-10-16 17:11:18.662320 7f402d40d780 1 journal _open /var/lib/ceph/osd/ceph-10/journal fd 21: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
  155. -149> 2012-10-16 17:11:18.662396 7f402d40d780 2 journal read_entry 5215735808 : seq 50235137 6609 bytes
  156. -148> 2012-10-16 17:11:18.662411 7f402d40d780 2 journal read_entry 5215735808 : seq 50235137 6609 bytes
  157. -147> 2012-10-16 17:11:18.662417 7f402d40d780 3 journal journal_replay: applying op seq 50235137
  158. -146> 2012-10-16 17:11:18.662425 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  159. -145> 2012-10-16 17:11:18.662449 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 236891~135
  160. -144> 2012-10-16 17:11:18.662632 7f4028cc4700 20 filestore(/home/cephbrick/ceph/osd) sync_entry waiting for max_interval 5.000000
  161. -143> 2012-10-16 17:11:18.668254 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 22 236891~135 qlen 1
  162. -142> 2012-10-16 17:11:18.668259 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 236891~135 = 135
  163. -141> 2012-10-16 17:11:18.668268 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30
  164. -140> 2012-10-16 17:11:18.668278 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30 = -2
  165. -139> 2012-10-16 17:11:18.668281 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508
  166. -138> 2012-10-16 17:11:18.668284 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508 = -2
  167. -137> 2012-10-16 17:11:18.668288 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0
  168. -136> 2012-10-16 17:11:18.670057 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0 = 0
  169. -135> 2012-10-16 17:11:18.670064 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134
  170. -134> 2012-10-16 17:11:18.670107 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 23 0~134 qlen 2
  171. -133> 2012-10-16 17:11:18.670110 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134 = 134
  172. -132> 2012-10-16 17:11:18.670113 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  173. -131> 2012-10-16 17:11:18.670115 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write 4.2_head/3068d4da/rb.0.c.000000004601/head//4 1613312~4096
  174. -130> 2012-10-16 17:11:18.670122 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) error getting collection index for 4.2_head: (2) No such file or directory
  175. -129> 2012-10-16 17:11:18.670136 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) write couldn't open 4.2_head/3068d4da/rb.0.c.000000004601/head//4 flags 65: (2) No such file or directory
  176. -128> 2012-10-16 17:11:18.670138 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write 4.2_head/3068d4da/rb.0.c.000000004601/head//4 1613312~4096 = -2
  177. -127> 2012-10-16 17:11:18.670148 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/3068d4da/rb.0.c.000000004601/head//4
  178. -126> 2012-10-16 17:11:18.670152 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) FileStore::_setattrs: do_setxattr returned -2
  179. -125> 2012-10-16 17:11:18.670154 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/3068d4da/rb.0.c.000000004601/head//4 = -2
  180. -124> 2012-10-16 17:11:18.670158 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/3068d4da/rb.0.c.000000004601/head//4
  181. -123> 2012-10-16 17:11:18.670162 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) FileStore::_setattrs: do_setxattr returned -2
  182. -122> 2012-10-16 17:11:18.670163 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/3068d4da/rb.0.c.000000004601/head//4 = -2
  183. -121> 2012-10-16 17:11:18.670168 7f402d40d780 3 journal journal_replay: r = 0, op_seq now 50235137
  184. -120> 2012-10-16 17:11:18.670204 7f402d40d780 2 journal read_entry 5215748096 : seq 50235138 39785 bytes
  185. -119> 2012-10-16 17:11:18.670205 7f402d40d780 3 journal journal_replay: applying op seq 50235138
  186. -118> 2012-10-16 17:11:18.670207 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  187. -117> 2012-10-16 17:11:18.670209 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237026~135
  188. -116> 2012-10-16 17:11:18.670245 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 24 237026~135 qlen 3
  189. -115> 2012-10-16 17:11:18.670247 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237026~135 = 135
  190. -114> 2012-10-16 17:11:18.670251 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30
  191. -113> 2012-10-16 17:11:18.670256 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30 = -2
  192. -112> 2012-10-16 17:11:18.670258 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508
  193. -111> 2012-10-16 17:11:18.670260 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508 = -2
  194. -110> 2012-10-16 17:11:18.670262 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0
  195. -109> 2012-10-16 17:11:18.670304 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0 = 0
  196. -108> 2012-10-16 17:11:18.670309 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134
  197. -107> 2012-10-16 17:11:18.670341 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 25 0~134 qlen 4
  198. -106> 2012-10-16 17:11:18.670344 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134 = 134
  199. -105> 2012-10-16 17:11:18.670346 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  200. -104> 2012-10-16 17:11:18.670348 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4 1166848~4096
  201. -103> 2012-10-16 17:11:18.670352 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) error getting collection index for 4.2_head: (2) No such file or directory
  202. -102> 2012-10-16 17:11:18.670356 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) write couldn't open 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4 flags 65: (2) No such file or directory
  203. -101> 2012-10-16 17:11:18.670358 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4 1166848~4096 = -2
  204. -100> 2012-10-16 17:11:18.670361 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4
  205. -99> 2012-10-16 17:11:18.670365 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) FileStore::_setattrs: do_setxattr returned -2
  206. -98> 2012-10-16 17:11:18.670366 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4 = -2
  207. -97> 2012-10-16 17:11:18.670375 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4
  208. -96> 2012-10-16 17:11:18.670379 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) FileStore::_setattrs: do_setxattr returned -2
  209. -95> 2012-10-16 17:11:18.670380 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.2_head/cf3f7d4a/rb.0.c.000000005800/head//4 = -2
  210. -94> 2012-10-16 17:11:18.670383 7f402d40d780 3 journal journal_replay: r = 0, op_seq now 50235138
  211. -93> 2012-10-16 17:11:18.671842 7f402d40d780 2 journal read_entry 5215793152 : seq 50235139 1084672 bytes
  212. -92> 2012-10-16 17:11:18.671848 7f402d40d780 3 journal journal_replay: applying op seq 50235139
  213. -91> 2012-10-16 17:11:18.671850 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  214. -90> 2012-10-16 17:11:18.671852 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217041~302
  215. -89> 2012-10-16 17:11:18.680095 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 26 4217041~302 qlen 5
  216. -88> 2012-10-16 17:11:18.680101 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217041~302 = 302
  217. -87> 2012-10-16 17:11:18.680106 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30
  218. -86> 2012-10-16 17:11:18.680127 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30 = 30
  219. -85> 2012-10-16 17:11:18.680130 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508
  220. -84> 2012-10-16 17:11:18.680189 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508 = 508
  221. -83> 2012-10-16 17:11:18.680194 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0
  222. -82> 2012-10-16 17:11:18.680250 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0 = 0
  223. -81> 2012-10-16 17:11:18.680255 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784
  224. -80> 2012-10-16 17:11:18.680292 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 27 0~784 qlen 6
  225. -79> 2012-10-16 17:11:18.680295 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784 = 784
  226. -78> 2012-10-16 17:11:18.680297 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  227. -77> 2012-10-16 17:11:18.680300 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) clone 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 -> 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4
  228. -76> 2012-10-16 17:11:18.685254 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) _do_clone_range 0~4194304 to 0
  229. -75> 2012-10-16 17:11:18.685260 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) _do_clone_range cloning 0~4194304 to 0 = 0
  230. -74> 2012-10-16 17:11:18.685381 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) _do_clone_range finished 0~4194304 to 0 = 0
  231. -73> 2012-10-16 17:11:18.685385 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) objectmap clone
  232. -72> 2012-10-16 17:11:18.707317 7f402d40d780 20 filestore dbobjectmap: seq is 620961
  233. -71> 2012-10-16 17:11:18.707413 7f402d40d780 20 filestore dbobjectmap: seq is 620962
  234. -70> 2012-10-16 17:11:18.707440 7f402d40d780 20 filestore set_header: setting seq 477721
  235. -69> 2012-10-16 17:11:18.707462 7f402d40d780 20 filestore set_map_header: setting 620960 hoid 40a55bff/rb.0.9.000000000de9/head//4 parent seq 477721
  236. -68> 2012-10-16 17:11:18.707482 7f402d40d780 20 filestore set_map_header: setting 620961 hoid 40a55bff/rb.0.9.000000000de9/e4//4 parent seq 477721
  237. -67> 2012-10-16 17:11:18.732934 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) getattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 getting '_'
  238. -66> 2012-10-16 17:11:18.733046 7f402d40d780 20 filestore(/home/cephbrick/ceph/osd) getattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 getting 'snapset'
  239. -65> 2012-10-16 17:11:18.733151 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4
  240. -64> 2012-10-16 17:11:18.733233 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 = 587
  241. -63> 2012-10-16 17:11:18.733256 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) clone 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 -> 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 = 587
  242. -62> 2012-10-16 17:11:18.733274 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4
  243. -61> 2012-10-16 17:11:18.733326 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 = 219
  244. -60> 2012-10-16 17:11:18.733348 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) rmattr 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 'snapset'
  245. -59> 2012-10-16 17:11:18.733401 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) rmattr 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 'snapset' = 0
  246. -58> 2012-10-16 17:11:18.733423 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) collection_add 4.7_e4/40a55bff/rb.0.9.000000000de9/e4//4 from 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4
  247. -57> 2012-10-16 17:11:18.734239 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_add 4.7_e4/40a55bff/rb.0.9.000000000de9/e4//4 from 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 = 0
  248. -56> 2012-10-16 17:11:18.734266 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) collection_add 4.7_db/40a55bff/rb.0.9.000000000de9/e4//4 from 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4
  249. -55> 2012-10-16 17:11:18.736817 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_add 4.7_db/40a55bff/rb.0.9.000000000de9/e4//4 from 4.7_head/40a55bff/rb.0.9.000000000de9/e4//4 = 0
  250. -54> 2012-10-16 17:11:18.736847 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 0~1080832
  251. -53> 2012-10-16 17:11:18.740458 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 28 0~1080832 qlen 7
  252. -52> 2012-10-16 17:11:18.740483 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 0~1080832 = 1080832
  253. -51> 2012-10-16 17:11:18.740500 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  254. -50> 2012-10-16 17:11:18.742748 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 218
  255. -49> 2012-10-16 17:11:18.742776 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  256. -48> 2012-10-16 17:11:18.742835 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 663
  257. -47> 2012-10-16 17:11:18.742860 7f402d40d780 3 journal journal_replay: r = 0, op_seq now 50235139
  258. -46> 2012-10-16 17:11:18.745880 7f402d40d780 2 journal read_entry 5216882688 : seq 50235140 3116435 bytes
  259. -45> 2012-10-16 17:11:18.745898 7f402d40d780 3 journal journal_replay: applying op seq 50235140
  260. -44> 2012-10-16 17:11:18.745904 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  261. -43> 2012-10-16 17:11:18.745931 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217343~135
  262. -42> 2012-10-16 17:11:18.745995 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 29 4217343~135 qlen 8
  263. -41> 2012-10-16 17:11:18.746012 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d51e/pglog_4.7/0//-1 4217343~135 = 135
  264. -40> 2012-10-16 17:11:18.746028 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30
  265. -39> 2012-10-16 17:11:18.746059 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'ondisklog' len 30 = 30
  266. -38> 2012-10-16 17:11:18.746076 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508
  267. -37> 2012-10-16 17:11:18.746104 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.7_head 'info' len 508 = 508
  268. -36> 2012-10-16 17:11:18.746120 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0
  269. -35> 2012-10-16 17:11:18.746185 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06082/pginfo_4.7/0//-1 size 0 = 0
  270. -34> 2012-10-16 17:11:18.746207 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784
  271. -33> 2012-10-16 17:11:18.746254 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 33 0~784 qlen 9
  272. -32> 2012-10-16 17:11:18.746271 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06082/pginfo_4.7/0//-1 0~784 = 784
  273. -31> 2012-10-16 17:11:18.746284 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  274. -30> 2012-10-16 17:11:18.746297 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 1080832~3113472
  275. -29> 2012-10-16 17:11:18.747472 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 34 1080832~3113472 qlen 10
  276. -28> 2012-10-16 17:11:18.747497 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 1080832~3113472 = 3113472
  277. -27> 2012-10-16 17:11:18.747515 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  278. -26> 2012-10-16 17:11:18.747576 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 218
  279. -25> 2012-10-16 17:11:18.747599 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4
  280. -24> 2012-10-16 17:11:18.747652 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) setattrs 4.7_head/40a55bff/rb.0.9.000000000de9/head//4 = 647
  281. -23> 2012-10-16 17:11:18.747673 7f402d40d780 3 journal journal_replay: r = 0, op_seq now 50235140
  282. -22> 2012-10-16 17:11:18.751119 7f402d40d780 2 journal read_entry 5220003840 : seq 50235141 4197478 bytes
  283. -21> 2012-10-16 17:11:18.751137 7f402d40d780 3 journal journal_replay: applying op seq 50235141
  284. -20> 2012-10-16 17:11:18.751144 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x22571b0
  285. -19> 2012-10-16 17:11:18.751175 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237161~302
  286. -18> 2012-10-16 17:11:18.751242 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 35 237161~302 qlen 11
  287. -17> 2012-10-16 17:11:18.751259 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/1039d68e/pglog_4.2/0//-1 237161~302 = 302
  288. -16> 2012-10-16 17:11:18.751274 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30
  289. -15> 2012-10-16 17:11:18.751289 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'ondisklog' len 30 = -2
  290. -14> 2012-10-16 17:11:18.751304 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508
  291. -13> 2012-10-16 17:11:18.751317 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) collection_setattr /home/cephbrick/ceph/osd/current/4.2_head 'info' len 508 = -2
  292. -12> 2012-10-16 17:11:18.751331 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0
  293. -11> 2012-10-16 17:11:18.751401 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) truncate meta/dae06c32/pginfo_4.2/0//-1 size 0 = 0
  294. -10> 2012-10-16 17:11:18.751423 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134
  295. -9> 2012-10-16 17:11:18.751470 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) queue_flusher ep 0 fd 36 0~134 qlen 12
  296. -8> 2012-10-16 17:11:18.751487 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) write meta/dae06c32/pginfo_4.2/0//-1 0~134 = 134
  297. -7> 2012-10-16 17:11:18.751500 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) _do_transaction on 0x2257d80
  298. -6> 2012-10-16 17:11:18.751513 7f402d40d780 15 filestore(/home/cephbrick/ceph/osd) clone 4.2_head/fe56915a/rb.0.9.000000000dee/head//4 -> 4.2_head/fe56915a/rb.0.9.000000000dee/e4//4
  299. -5> 2012-10-16 17:11:18.751530 7f402d40d780 -1 filestore(/home/cephbrick/ceph/osd) error getting collection index for 4.2_head: (2) No such file or directory
  300. -4> 2012-10-16 17:11:18.751555 7f402d40d780 10 filestore(/home/cephbrick/ceph/osd) clone 4.2_head/fe56915a/rb.0.9.000000000dee/head//4 -> 4.2_head/fe56915a/rb.0.9.000000000dee/e4//4 = -2
  301. -3> 2012-10-16 17:11:18.751569 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) error (2) No such file or directory not handled on operation 17 (50235141.1.0, or op 0, counting from 0)
  302. -2> 2012-10-16 17:11:18.751585 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) ENOENT on clone suggests osd bug
  303. -1> 2012-10-16 17:11:18.751596 7f402d40d780 0 filestore(/home/cephbrick/ceph/osd) transaction dump:
  304. { "ops": [
  305. { "op_num": 0,
  306. "op_name": "clone",
  307. "collection": "4.2_head",
  308. "src_oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  309. "dst_oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  310. { "op_num": 1,
  311. "op_name": "setattr",
  312. "collection": "4.2_head",
  313. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4",
  314. "name": "_",
  315. "length": 219},
  316. { "op_num": 2,
  317. "op_name": "rmattr",
  318. "collection": "4.2_head",
  319. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4",
  320. "name": "snapset"},
  321. { "op_num": 3,
  322. "op_name": "collection_add",
  323. "src_collection": "4.2_e4",
  324. "dst_collection": "4.2_head",
  325. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  326. { "op_num": 4,
  327. "op_name": "collection_add",
  328. "src_collection": "4.2_db",
  329. "dst_collection": "4.2_head",
  330. "oid": "fe56915a\/rb.0.9.000000000dee\/e4\/\/4"},
  331. { "op_num": 5,
  332. "op_name": "write",
  333. "collection": "4.2_head",
  334. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  335. "length": 4194304,
  336. "offset": 0,
  337. "bufferlist length": 4194304},
  338. { "op_num": 6,
  339. "op_name": "setattr",
  340. "collection": "4.2_head",
  341. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  342. "name": "_",
  343. "length": 218},
  344. { "op_num": 7,
  345. "op_name": "setattr",
  346. "collection": "4.2_head",
  347. "oid": "fe56915a\/rb.0.9.000000000dee\/head\/\/4",
  348. "name": "snapset",
  349. "length": 647}]}
  350. 0> 2012-10-16 17:11:18.752702 7f402d40d780 -1 os/FileStore.cc: In function 'unsigned int FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int)' thread 7f402d40d780 time 2012-10-16 17:11:18.751687
  351. os/FileStore.cc: 3007: FAILED assert(0 == "unexpected error")
  352.  
  353. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  354. 1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0xa6b) [0x684d8b]
  355. 2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x84) [0x68aa74]
  356. 3: (JournalingObjectStore::journal_replay(unsigned long)+0x884) [0x6997c4]
  357. 4: (FileStore::mount()+0x35a1) [0x67daf1]
  358. 5: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x59f49a]
  359. 6: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x59ff07]
  360. 7: (main()+0x2256) [0x517f46]
  361. 8: (__libc_start_main()+0xed) [0x7f402b46f76d]
  362. 9: /usr/bin/ceph-osd() [0x51a239]
  363. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  364.  
  365. --- end dump of recent events ---
  366. 2012-10-16 17:11:18.755148 7f402d40d780 -1 *** Caught signal (Aborted) **
  367. in thread 7f402d40d780
  368.  
  369. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  370. 1: /usr/bin/ceph-osd() [0x6edaba]
  371. 2: (()+0xfcb0) [0x7f402c8a8cb0]
  372. 3: (gsignal()+0x35) [0x7f402b484445]
  373. 4: (abort()+0x17b) [0x7f402b487bab]
  374. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f402bdd269d]
  375. 6: (()+0xb5846) [0x7f402bdd0846]
  376. 7: (()+0xb5873) [0x7f402bdd0873]
  377. 8: (()+0xb596e) [0x7f402bdd096e]
  378. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1de) [0x7a82fe]
  379. 10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0xa6b) [0x684d8b]
  380. 11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x84) [0x68aa74]
  381. 12: (JournalingObjectStore::journal_replay(unsigned long)+0x884) [0x6997c4]
  382. 13: (FileStore::mount()+0x35a1) [0x67daf1]
  383. 14: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x59f49a]
  384. 15: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x59ff07]
  385. 16: (main()+0x2256) [0x517f46]
  386. 17: (__libc_start_main()+0xed) [0x7f402b46f76d]
  387. 18: /usr/bin/ceph-osd() [0x51a239]
  388. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  389.  
  390. --- begin dump of recent events ---
  391. 0> 2012-10-16 17:11:18.755148 7f402d40d780 -1 *** Caught signal (Aborted) **
  392. in thread 7f402d40d780
  393.  
  394. ceph version 0.48.2argonaut (commit:3e02b2fad88c2a95d9c0c86878f10d1beb780bfe)
  395. 1: /usr/bin/ceph-osd() [0x6edaba]
  396. 2: (()+0xfcb0) [0x7f402c8a8cb0]
  397. 3: (gsignal()+0x35) [0x7f402b484445]
  398. 4: (abort()+0x17b) [0x7f402b487bab]
  399. 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f402bdd269d]
  400. 6: (()+0xb5846) [0x7f402bdd0846]
  401. 7: (()+0xb5873) [0x7f402bdd0873]
  402. 8: (()+0xb596e) [0x7f402bdd096e]
  403. 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1de) [0x7a82fe]
  404. 10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0xa6b) [0x684d8b]
  405. 11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x84) [0x68aa74]
  406. 12: (JournalingObjectStore::journal_replay(unsigned long)+0x884) [0x6997c4]
  407. 13: (FileStore::mount()+0x35a1) [0x67daf1]
  408. 14: (OSD::do_convertfs(ObjectStore*)+0x1a) [0x59f49a]
  409. 15: (OSD::convertfs(std::string const&, std::string const&)+0x47) [0x59ff07]
  410. 16: (main()+0x2256) [0x517f46]
  411. 17: (__libc_start_main()+0xed) [0x7f402b46f76d]
  412. 18: /usr/bin/ceph-osd() [0x51a239]
  413. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  414.  
  415. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!