API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Aug 6th, 2019
440
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 150.90 KB | None | 0 0
raw download clone embed print report
  1. 2019-08-06 12:50:00.596 7f8a32e4f200 0 set uid:gid to 64045:64045 (ceph:ceph)
  2. 2019-08-06 12:50:00.596 7f8a32e4f200 0 ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable), process ceph-osd, pid 2696
  3. 2019-08-06 12:50:00.860 7f8a32e4f200 0 pidfile_write: ignore empty --pid-file
  4. 2019-08-06 12:50:00.864 7f8a32e4f200 -1 Public network was set, but cluster network was not set
  5. 2019-08-06 12:50:00.864 7f8a32e4f200 -1 Using public network also for cluster network
  6. 2019-08-06 12:50:00.872 7f8a32e4f200 0 load: jerasure load: lrc load: isa
  7. 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  8. 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  9. 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  10. 2019-08-06 12:50:00.872 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  11. 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) close
  12. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _mount path /var/lib/ceph/osd/ceph-37
  13. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  14. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  15. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  16. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  17. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  18. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8380 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  19. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8380 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  20. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-37/block size 3.6 TiB
  21. 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluefs mount
  22. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compaction_readahead_size = 2097152
  23. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compression = kNoCompression
  24. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option max_write_buffer_number = 4
  25. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  26. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option recycle_log_file_num = 4
  27. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option writable_file_max_buffer_size = 0
  28. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option write_buffer_size = 268435456
  29. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compaction_readahead_size = 2097152
  30. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compression = kNoCompression
  31. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option max_write_buffer_number = 4
  32. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  33. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option recycle_log_file_num = 4
  34. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option writable_file_max_buffer_size = 0
  35. 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option write_buffer_size = 268435456
  36. 2019-08-06 12:50:01.324 7f8a32e4f200 1 rocksdb: do_open column families: [default]
  37. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: RocksDB version: 5.13.0
  38.  
  39. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  40. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Compile date Jan 4 2019
  41. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: DB SUMMARY
  42.  
  43. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: CURRENT file: CURRENT
  44.  
  45. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: IDENTITY file: IDENTITY
  46.  
  47. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: MANIFEST file: MANIFEST-012064 size: 46845 Bytes
  48.  
  49. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: SST files in db dir, Total Num: 191, files: 009870.sst 010106.sst 010110.sst 010155.sst 010199.sst 010200.sst 010201.sst 010286.sst 010287.sst
  50.  
  51. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Write Ahead Log file in db: 012065.log size: 0 ;
  52.  
  53. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.error_if_exists: 0
  54. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.create_if_missing: 0
  55. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.paranoid_checks: 1
  56. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.env: 0x33c5a40
  57. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.info_log: 0x36dfd40
  58. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_file_opening_threads: 16
  59. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.statistics: (nil)
  60. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_fsync: 0
  61. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_log_file_size: 0
  62. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
  63. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.log_file_time_to_roll: 0
  64. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.keep_log_file_num: 1000
  65. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.recycle_log_file_num: 4
  66. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_fallocate: 1
  67. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_mmap_reads: 0
  68. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_mmap_writes: 0
  69. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_direct_reads: 0
  70. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  71. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.create_missing_column_families: 0
  72. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.db_log_dir:
  73. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_dir: db
  74. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_cache_numshardbits: 6
  75. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_subcompactions: 1
  76. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_flushes: -1
  77. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.WAL_ttl_seconds: 0
  78. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.WAL_size_limit_MB: 0
  79. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.manifest_preallocation_size: 4194304
  80. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.is_fd_close_on_exec: 1
  81. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.advise_random_on_open: 1
  82. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.db_write_buffer_size: 0
  83. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_buffer_manager: 0x33676c0
  84. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.access_hint_on_compaction_start: 1
  85. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  86. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  87. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_adaptive_mutex: 0
  88. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.rate_limiter: (nil)
  89. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  90. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_recovery_mode: 0
  91. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_thread_tracking: 0
  92. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_pipelined_write: 0
  93. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  94. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  95. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_thread_max_yield_usec: 100
  96. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  97. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.row_cache: None
  98. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_filter: None
  99. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.avoid_flush_during_recovery: 0
  100. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_ingest_behind: 0
  101. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.preserve_deletes: 0
  102. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.two_write_queues: 0
  103. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.manual_wal_flush: 0
  104. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_jobs: 2
  105. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_compactions: -1
  106. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  107. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.writable_file_max_buffer_size: 0
  108. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.delayed_write_rate : 16777216
  109. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_total_wal_size: 0
  110. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  111. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.stats_dump_period_sec: 600
  112. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_open_files: -1
  113. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bytes_per_sync: 0
  114. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_bytes_per_sync: 0
  115. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_readahead_size: 2097152
  116. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Compression algorithms supported:
  117. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZSTDNotFinalCompression supported: 0
  118. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZSTD supported: 0
  119. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kXpressCompression supported: 0
  120. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kLZ4HCCompression supported: 1
  121. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kLZ4Compression supported: 1
  122. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kBZip2Compression supported: 0
  123. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZlibCompression supported: 1
  124. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kSnappyCompression supported: 1
  125. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Fast CRC32 supported: Supported on x86
  126. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3088] Recovering from manifest file: MANIFEST-012064
  127.  
  128. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/column_family.cc:474] --------------- Options for column family [default]:
  129.  
  130. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  131. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  132. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_filter: None
  133. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_filter_factory: None
  134. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_factory: SkipListFactory
  135. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_factory: BlockBasedTable
  136. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2fe54e0)
  137. cache_index_and_filter_blocks: 1
  138. cache_index_and_filter_blocks_with_high_priority: 1
  139. pin_l0_filter_and_index_blocks_in_cache: 1
  140. index_type: 0
  141. hash_index_allow_collision: 1
  142. checksum: 1
  143. no_block_cache: 0
  144. block_cache: 0x34027f0
  145. block_cache_name: BinnedLRUCache
  146. block_cache_options:
  147. capacity : 429496729
  148. num_shard_bits : 4
  149. strict_capacity_limit : 0
  150. high_pri_pool_ratio: 0.000
  151. block_cache_compressed: (nil)
  152. persistent_cache: (nil)
  153. block_size: 4096
  154. block_size_deviation: 10
  155. block_restart_interval: 16
  156. index_block_restart_interval: 1
  157. metadata_block_size: 4096
  158. partition_filters: 0
  159. use_delta_encoding: 1
  160. filter_policy: rocksdb.BuiltinBloomFilter
  161. whole_key_filtering: 1
  162. verify_compression: 0
  163. read_amp_bytes_per_bit: 0
  164. format_version: 2
  165. enable_index_compression: 1
  166. block_align: 0
  167.  
  168. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_buffer_size: 268435456
  169. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_write_buffer_number: 4
  170. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression: NoCompression
  171. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bottommost_compression: Disabled
  172. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.prefix_extractor: nullptr
  173. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  174. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.num_levels: 7
  175. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  176. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  177. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.window_bits: -14
  178. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.level: -1
  179. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.strategy: 0
  180. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  181. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  182. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  183. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  184. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_stop_writes_trigger: 36
  185. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.target_file_size_base: 67108864
  186. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.target_file_size_multiplier: 1
  187. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  188. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  189. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  190. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  191. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  192. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  193. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  194. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  195. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  196. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  197. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  198. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_compaction_bytes: 1677721600
  199. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.arena_block_size: 33554432
  200. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  201. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  202. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  203. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.disable_auto_compactions: 0
  204. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  205. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_pri: kByCompensatedSize
  206. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  207. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  208. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  209. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  210. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  211. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  212. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  213. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  214. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.ttl: 0
  215. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_properties_collectors:
  216. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.inplace_update_support: 0
  217. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.inplace_update_num_locks: 10000
  218. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  219. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_huge_page_size: 0
  220. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bloom_locality: 0
  221. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_successive_merges: 0
  222. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.optimize_filters_for_hits: 0
  223. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.paranoid_file_checks: 0
  224. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.force_consistency_checks: 0
  225. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.report_bg_io_stats: 0
  226. 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.ttl: 0
  227. 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3362] Recovered from manifest file:db/MANIFEST-012064 succeeded,manifest_file_number is 12064, next_file_number is 12067, last_sequence is 173668276, log_number is 0,prev_log_number is 0,max_column_family is 0,deleted_log_number is 12062
  228.  
  229. 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3370] Column family [default] (ID 0), log number is 12063
  230.  
  231. 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095802900978, "job": 1, "event": "recovery_started", "log_files": [12065]}
  232. 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:551] Recovering log #12065 mode 0
  233. 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:2863] Creating manifest 12067
  234.  
  235. 2019-08-06 12:50:02.908 7f8a32e4f200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095802910612, "job": 1, "event": "recovery_finished"}
  236. 2019-08-06 12:50:02.924 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:1218] DB pointer 0x36e8000
  237. 2019-08-06 12:50:02.924 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
  238. 2019-08-06 12:50:03.012 7f8a32e4f200 1 freelist init
  239. 2019-08-06 12:50:03.072 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc opening allocation metadata
  240. 2019-08-06 12:50:04.200 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc loaded 1.3 TiB in 937418 extents
  241. 2019-08-06 12:50:04.300 7f8a32e4f200 0 <cls> /build/ceph-13.2.4/src/cls/hello/cls_hello.cc:296: loading cls_hello
  242. 2019-08-06 12:50:04.304 7f8a32e4f200 0 _get_class not permitted to load sdk
  243. 2019-08-06 12:50:04.304 7f8a32e4f200 0 _get_class not permitted to load kvs
  244. 2019-08-06 12:50:04.312 7f8a32e4f200 0 _get_class not permitted to load lua
  245. 2019-08-06 12:50:04.312 7f8a32e4f200 0 <cls> /build/ceph-13.2.4/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
  246. 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 432629342416404480, adjusting msgr requires for clients
  247. 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 432629342416404480 was 8705, adjusting msgr requires for mons
  248. 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 1009090094719844352, adjusting msgr requires for osds
  249. 2019-08-06 12:50:14.156 7f8a32e4f200 0 osd.37 196588 load_pgs
  250. 2019-08-06 12:50:14.208 7f8a32e4f200 -1 /build/ceph-13.2.4/src/osd/ECUtil.h: In function 'ECUtil::stripe_info_t::stripe_info_t(uint64_t, uint64_t)' thread 7f8a32e4f200 time 2019-08-06 12:50:14.208743
  251. /build/ceph-13.2.4/src/osd/ECUtil.h: 34: FAILED assert(stripe_width % stripe_size == 0)
  252.  
  253. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  254. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f8a2a1843c2]
  255. 2: (()+0x2e5587) [0x7f8a2a184587]
  256. 3: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  257. 4: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  258. 5: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  259. 6: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  260. 7: (OSD::load_pgs()+0x4a9) [0x758339]
  261. 8: (OSD::init()+0xcd3) [0x7619c3]
  262. 9: (main()+0x3678) [0x64d6a8]
  263. 10: (__libc_start_main()+0xf0) [0x7f8a27d34830]
  264. 11: (_start()+0x29) [0x717389]
  265. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  266.  
  267. --- begin dump of recent events ---
  268. -348> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perfcounters_dump hook 0x2fe8250
  269. -347> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command 1 hook 0x2fe8250
  270. -346> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perf dump hook 0x2fe8250
  271. -345> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perfcounters_schema hook 0x2fe8250
  272. -344> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perf histogram dump hook 0x2fe8250
  273. -343> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command 2 hook 0x2fe8250
  274. -342> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perf schema hook 0x2fe8250
  275. -341> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perf histogram schema hook 0x2fe8250
  276. -340> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command perf reset hook 0x2fe8250
  277. -339> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config show hook 0x2fe8250
  278. -338> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config help hook 0x2fe8250
  279. -337> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config set hook 0x2fe8250
  280. -336> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config unset hook 0x2fe8250
  281. -335> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config get hook 0x2fe8250
  282. -334> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config diff hook 0x2fe8250
  283. -333> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command config diff get hook 0x2fe8250
  284. -332> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command log flush hook 0x2fe8250
  285. -331> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command log dump hook 0x2fe8250
  286. -330> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command log reopen hook 0x2fe8250
  287. -329> 2019-08-06 12:50:00.580 7f8a32e4f200 5 asok(0x3048240) register_command dump_mempools hook 0x338e068
  288. -328> 2019-08-06 12:50:00.584 7f8a32e4f200 10 monclient: get_monmap_and_config
  289. -327> 2019-08-06 12:50:00.584 7f8a32e4f200 10 monclient: build_initial_monmap
  290. -326> 2019-08-06 12:50:00.584 7f8a24a2b700 2 Event(0x3048e00 nevent=5000 time_id=1).set_owner idx=0 owner=140231296857856
  291. -325> 2019-08-06 12:50:00.584 7f8a2422a700 2 Event(0x3049040 nevent=5000 time_id=1).set_owner idx=1 owner=140231288465152
  292. -324> 2019-08-06 12:50:00.584 7f8a23a29700 2 Event(0x3049280 nevent=5000 time_id=1).set_owner idx=2 owner=140231280072448
  293. -323> 2019-08-06 12:50:00.584 7f8a32e4f200 1 Processor -- start
  294. -322> 2019-08-06 12:50:00.584 7f8a32e4f200 1 -- - start start
  295. -321> 2019-08-06 12:50:00.584 7f8a32e4f200 10 monclient: init
  296. -320> 2019-08-06 12:50:00.584 7f8a32e4f200 5 adding auth protocol: cephx
  297. -319> 2019-08-06 12:50:00.584 7f8a32e4f200 10 monclient: auth_supported 2 method cephx
  298. -318> 2019-08-06 12:50:00.584 7f8a32e4f200 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-37/keyring
  299. -317> 2019-08-06 12:50:00.588 7f8a32e4f200 10 monclient: _reopen_session rank -1
  300. -316> 2019-08-06 12:50:00.588 7f8a32e4f200 10 monclient(hunting): picked mon.noname-b con 0x3494700 addr 10.100.219.12:6789/0
  301. -315> 2019-08-06 12:50:00.588 7f8a32e4f200 10 monclient(hunting): picked mon.noname-a con 0x3494e00 addr 10.100.219.11:6789/0
  302. -314> 2019-08-06 12:50:00.588 7f8a32e4f200 1 -- - --> 10.100.219.11:6789/0 -- auth(proto 0 27 bytes epoch 0) v1 -- 0x3406780 con 0
  303. -313> 2019-08-06 12:50:00.588 7f8a32e4f200 1 -- - --> 10.100.219.12:6789/0 -- auth(proto 0 27 bytes epoch 0) v1 -- 0x3406a00 con 0
  304. -312> 2019-08-06 12:50:00.588 7f8a32e4f200 10 monclient(hunting): _renew_subs
  305. -311> 2019-08-06 12:50:00.588 7f8a32e4f200 10 monclient(hunting): authenticate will time out at 2019-08-06 12:55:00.590119
  306. -310> 2019-08-06 12:50:00.588 7f8a2422a700 1 -- 10.100.219.111:0/2463307753 learned_addr learned my addr 10.100.219.111:0/2463307753
  307. -309> 2019-08-06 12:50:00.588 7f8a2422a700 2 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0
  308. -308> 2019-08-06 12:50:00.588 7f8a23a29700 2 -- 10.100.219.111:0/2463307753 >> 10.100.219.11:6789/0 conn(0x3494e00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0
  309. -307> 2019-08-06 12:50:00.588 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 1 0x3049d40 mon_map magic: 0 v1
  310. -306> 2019-08-06 12:50:00.588 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 2 0x3406a00 auth_reply(proto 2 0 (0) Success) v1
  311. -305> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 1 ==== mon_map magic: 0 v1 ==== 397+0+0 (1649140285 0 0) 0x3049d40 con 0x3494700
  312. -304> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1
  313. -303> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): got monmap 9, mon.noname-b is now rank -1
  314. -302> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): dump:
  315. epoch 9
  316. fsid f5e1c608-870b-425e-ab8a-bdc86789e37c
  317. last_changed 2018-10-22 11:56:22.979297
  318. created 2016-10-26 12:14:35.362412
  319. 0: 10.100.219.11:6789/0 mon.mon01
  320. 1: 10.100.219.12:6789/0 mon.mon02
  321. 2: 10.100.219.13:6789/0 mon.mon03
  322.  
  323. -301> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (708966280 0 0) 0x3406a00 con 0x3494700
  324. -300> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): my global_id is 50117542
  325. -299> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 --> 10.100.219.12:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x3406f00 con 0
  326. -298> 2019-08-06 12:50:00.588 7f8a23a29700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.11:6789/0 conn(0x3494e00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10899 cs=1 l=1). rx mon.0 seq 1 0x3049f80 mon_map magic: 0 v1
  327. -297> 2019-08-06 12:50:00.588 7f8a23a29700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.11:6789/0 conn(0x3494e00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10899 cs=1 l=1). rx mon.0 seq 2 0x3406780 auth_reply(proto 2 0 (0) Success) v1
  328. -296> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.0 10.100.219.11:6789/0 1 ==== mon_map magic: 0 v1 ==== 397+0+0 (1649140285 0 0) 0x3049f80 con 0x3494e00
  329. -295> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1
  330. -294> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): got monmap 9, mon.mon01 is now rank 0
  331. -293> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): dump:
  332. epoch 9
  333. fsid f5e1c608-870b-425e-ab8a-bdc86789e37c
  334. last_changed 2018-10-22 11:56:22.979297
  335. created 2016-10-26 12:14:35.362412
  336. 0: 10.100.219.11:6789/0 mon.mon01
  337. 1: 10.100.219.12:6789/0 mon.mon02
  338. 2: 10.100.219.13:6789/0 mon.mon03
  339.  
  340. -292> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.0 10.100.219.11:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1885851043 0 0) 0x3406780 con 0x3494e00
  341. -291> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient(hunting): my global_id is 50113887
  342. -290> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 --> 10.100.219.11:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x3406a00 con 0
  343. -289> 2019-08-06 12:50:00.588 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 3 0x3406f00 auth_reply(proto 2 0 (0) Success) v1
  344. -288> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (1761201516 0 0) 0x3406f00 con 0x3494700
  345. -287> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 >> 10.100.219.11:6789/0 conn(0x3494e00 :-1 s=STATE_OPEN pgs=10899 cs=1 l=1).mark_down
  346. -286> 2019-08-06 12:50:00.588 7f8a23228700 2 -- 10.100.219.111:0/2463307753 >> 10.100.219.11:6789/0 conn(0x3494e00 :-1 s=STATE_OPEN pgs=10899 cs=1 l=1)._stop
  347. -285> 2019-08-06 12:50:00.588 7f8a23228700 1 monclient: found mon.mon02
  348. -284> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient: _send_mon_message to mon.mon02 at 10.100.219.12:6789/0
  349. -283> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 --> 10.100.219.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x3049440 con 0
  350. -282> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-08-06 12:49:30.593255)
  351. -281> 2019-08-06 12:50:00.588 7f8a23228700 10 monclient: _send_mon_message to mon.mon02 at 10.100.219.12:6789/0
  352. -280> 2019-08-06 12:50:00.588 7f8a23228700 1 -- 10.100.219.111:0/2463307753 --> 10.100.219.12:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x3406780 con 0
  353. -279> 2019-08-06 12:50:00.588 7f8a32e4f200 5 monclient: authenticate success, global_id 50117542
  354. -278> 2019-08-06 12:50:00.592 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 4 0x3049440 config(0 keys) v1
  355. -277> 2019-08-06 12:50:00.592 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x3049440 con 0x3494700
  356. -276> 2019-08-06 12:50:00.592 7f8a23228700 10 monclient: handle_config config(0 keys) v1
  357. -275> 2019-08-06 12:50:00.592 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 5 0x3049d40 mon_map magic: 0 v1
  358. -274> 2019-08-06 12:50:00.592 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 5 ==== mon_map magic: 0 v1 ==== 397+0+0 (1649140285 0 0) 0x3049d40 con 0x3494700
  359. -273> 2019-08-06 12:50:00.592 7f8a23228700 10 monclient: handle_monmap mon_map magic: 0 v1
  360. -272> 2019-08-06 12:50:00.592 7f8a2422a700 5 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=6768 cs=1 l=1). rx mon.1 seq 6 0x3406780 auth_reply(proto 2 0 (0) Success) v1
  361. -271> 2019-08-06 12:50:00.592 7f8a21a25700 4 set_mon_vals no callback set
  362. -270> 2019-08-06 12:50:00.592 7f8a23228700 10 monclient: got monmap 9, mon.mon02 is now rank 1
  363. -269> 2019-08-06 12:50:00.592 7f8a23228700 10 monclient: dump:
  364. epoch 9
  365. fsid f5e1c608-870b-425e-ab8a-bdc86789e37c
  366. last_changed 2018-10-22 11:56:22.979297
  367. created 2016-10-26 12:14:35.362412
  368. 0: 10.100.219.11:6789/0 mon.mon01
  369. 1: 10.100.219.12:6789/0 mon.mon02
  370. 2: 10.100.219.13:6789/0 mon.mon03
  371.  
  372. -268> 2019-08-06 12:50:00.592 7f8a32e4f200 10 monclient: get_monmap_and_config success
  373. -267> 2019-08-06 12:50:00.592 7f8a23228700 1 -- 10.100.219.111:0/2463307753 <== mon.1 10.100.219.12:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (3762986180 0 0) 0x3406780 con 0x3494700
  374. -266> 2019-08-06 12:50:00.592 7f8a32e4f200 10 monclient: shutdown
  375. -265> 2019-08-06 12:50:00.592 7f8a23228700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-08-06 12:49:30.594199)
  376. -264> 2019-08-06 12:50:00.592 7f8a32e4f200 1 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN pgs=6768 cs=1 l=1).mark_down
  377. -263> 2019-08-06 12:50:00.592 7f8a32e4f200 2 -- 10.100.219.111:0/2463307753 >> 10.100.219.12:6789/0 conn(0x3494700 :-1 s=STATE_OPEN pgs=6768 cs=1 l=1)._stop
  378. -262> 2019-08-06 12:50:00.592 7f8a32e4f200 1 -- 10.100.219.111:0/2463307753 shutdown_connections
  379. -261> 2019-08-06 12:50:00.592 7f8a32e4f200 5 -- 10.100.219.111:0/2463307753 shutdown_connections mark down 10.100.219.11:6789/0 0x3494e00
  380. -260> 2019-08-06 12:50:00.592 7f8a32e4f200 5 -- 10.100.219.111:0/2463307753 shutdown_connections mark down 10.100.219.12:6789/0 0x3494700
  381. -259> 2019-08-06 12:50:00.592 7f8a32e4f200 5 -- 10.100.219.111:0/2463307753 shutdown_connections delete 0x3494700
  382. -258> 2019-08-06 12:50:00.592 7f8a32e4f200 5 -- 10.100.219.111:0/2463307753 shutdown_connections delete 0x3494e00
  383. -257> 2019-08-06 12:50:00.592 7f8a32e4f200 1 -- 10.100.219.111:0/2463307753 shutdown_connections
  384. -256> 2019-08-06 12:50:00.592 7f8a32e4f200 1 -- 10.100.219.111:0/2463307753 wait complete.
  385. -255> 2019-08-06 12:50:00.592 7f8a32e4f200 1 -- 10.100.219.111:0/2463307753 >> 10.100.219.111:0/2463307753 conn(0x3494000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down
  386. -254> 2019-08-06 12:50:00.592 7f8a32e4f200 2 -- 10.100.219.111:0/2463307753 >> 10.100.219.111:0/2463307753 conn(0x3494000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop
  387. -253> 2019-08-06 12:50:00.596 7f8a32e4f200 0 set uid:gid to 64045:64045 (ceph:ceph)
  388. -252> 2019-08-06 12:50:00.596 7f8a32e4f200 0 ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable), process ceph-osd, pid 2696
  389. -251> 2019-08-06 12:50:00.860 7f8a32e4f200 0 pidfile_write: ignore empty --pid-file
  390. -250> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) init /var/run/ceph/ceph-osd.37.asok
  391. -249> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) bind_and_listen /var/run/ceph/ceph-osd.37.asok
  392. -248> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) register_command 0 hook 0x2fe5490
  393. -247> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) register_command version hook 0x2fe5490
  394. -246> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) register_command git_version hook 0x2fe5490
  395. -245> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) register_command help hook 0x2fe81f0
  396. -244> 2019-08-06 12:50:00.860 7f8a32e4f200 5 asok(0x3048240) register_command get_command_descriptions hook 0x2fe81b0
  397. -243> 2019-08-06 12:50:00.860 7f8a22a27700 5 asok(0x3048240) entry start
  398. -242> 2019-08-06 12:50:00.860 7f8a32e4f200 5 object store type is bluestore
  399. -241> 2019-08-06 12:50:00.864 7f8a32e4f200 -1 Public network was set, but cluster network was not set
  400. -240> 2019-08-06 12:50:00.864 7f8a32e4f200 -1 Using public network also for cluster network
  401. -239> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  402. -238> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:6804/2696 _finish_bind bind my_inst.addr is 10.100.219.111:6804/2696
  403. -237> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  404. -236> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:6805/2696 _finish_bind bind my_inst.addr is 10.100.219.111:6805/2696
  405. -235> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  406. -234> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:6806/2696 _finish_bind bind my_inst.addr is 10.100.219.111:6806/2696
  407. -233> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  408. -232> 2019-08-06 12:50:00.864 7f8a32e4f200 1 -- 10.100.219.111:6807/2696 _finish_bind bind my_inst.addr is 10.100.219.111:6807/2696
  409. -231> 2019-08-06 12:50:00.864 7f8a32e4f200 10 monclient: build_initial_monmap
  410. -230> 2019-08-06 12:50:00.872 7f8a32e4f200 0 load: jerasure load: lrc load: isa
  411. -229> 2019-08-06 12:50:00.872 7f8a32e4f200 5 adding auth protocol: cephx
  412. -228> 2019-08-06 12:50:00.872 7f8a32e4f200 5 adding auth protocol: cephx
  413. -227> 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  414. -226> 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  415. -225> 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  416. -224> 2019-08-06 12:50:00.872 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  417. -223> 2019-08-06 12:50:00.872 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) close
  418. -222> 2019-08-06 12:50:01.164 7f8a32e4f200 5 asok(0x3048240) register_command objecter_requests hook 0x2fe8710
  419. -221> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- 10.100.219.111:6804/2696 start start
  420. -220> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- - start start
  421. -219> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- - start start
  422. -218> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- 10.100.219.111:6807/2696 start start
  423. -217> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- 10.100.219.111:6806/2696 start start
  424. -216> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- 10.100.219.111:6805/2696 start start
  425. -215> 2019-08-06 12:50:01.164 7f8a32e4f200 1 -- - start start
  426. -214> 2019-08-06 12:50:01.164 7f8a32e4f200 2 osd.37 0 init /var/lib/ceph/osd/ceph-37 (looks like hdd)
  427. -213> 2019-08-06 12:50:01.164 7f8a32e4f200 2 osd.37 0 journal /var/lib/ceph/osd/ceph-37/journal
  428. -212> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _mount path /var/lib/ceph/osd/ceph-37
  429. -211> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  430. -210> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  431. -209> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  432. -208> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  433. -207> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  434. -206> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8380 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  435. -205> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bdev(0x35c8380 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  436. -204> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-37/block size 3.6 TiB
  437. -203> 2019-08-06 12:50:01.164 7f8a32e4f200 1 bluefs mount
  438. -202> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compaction_readahead_size = 2097152
  439. -201> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compression = kNoCompression
  440. -200> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option max_write_buffer_number = 4
  441. -199> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  442. -198> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option recycle_log_file_num = 4
  443. -197> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option writable_file_max_buffer_size = 0
  444. -196> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option write_buffer_size = 268435456
  445. -195> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compaction_readahead_size = 2097152
  446. -194> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option compression = kNoCompression
  447. -193> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option max_write_buffer_number = 4
  448. -192> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  449. -191> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option recycle_log_file_num = 4
  450. -190> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option writable_file_max_buffer_size = 0
  451. -189> 2019-08-06 12:50:01.304 7f8a32e4f200 0 set rocksdb option write_buffer_size = 268435456
  452. -188> 2019-08-06 12:50:01.324 7f8a32e4f200 1 rocksdb: do_open column families: [default]
  453. -187> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: RocksDB version: 5.13.0
  454.  
  455. -186> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  456. -185> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Compile date Jan 4 2019
  457. -184> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: DB SUMMARY
  458.  
  459. -183> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: CURRENT file: CURRENT
  460.  
  461. -182> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: IDENTITY file: IDENTITY
  462.  
  463. -181> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: MANIFEST file: MANIFEST-012064 size: 46845 Bytes
  464.  
  465. -180> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: SST files in db dir, Total Num: 191, files: 009870.sst 010106.sst 010110.sst 010155.sst 010199.sst 010200.sst 010201.sst 010286.sst 010287.sst
  466.  
  467. -179> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Write Ahead Log file in db: 012065.log size: 0 ;
  468.  
  469. -178> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.error_if_exists: 0
  470. -177> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.create_if_missing: 0
  471. -176> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.paranoid_checks: 1
  472. -175> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.env: 0x33c5a40
  473. -174> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.info_log: 0x36dfd40
  474. -173> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_file_opening_threads: 16
  475. -172> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.statistics: (nil)
  476. -171> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_fsync: 0
  477. -170> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_log_file_size: 0
  478. -169> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
  479. -168> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.log_file_time_to_roll: 0
  480. -167> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.keep_log_file_num: 1000
  481. -166> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.recycle_log_file_num: 4
  482. -165> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_fallocate: 1
  483. -164> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_mmap_reads: 0
  484. -163> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_mmap_writes: 0
  485. -162> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_direct_reads: 0
  486. -161> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  487. -160> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.create_missing_column_families: 0
  488. -159> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.db_log_dir:
  489. -158> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_dir: db
  490. -157> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_cache_numshardbits: 6
  491. -156> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_subcompactions: 1
  492. -155> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_flushes: -1
  493. -154> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.WAL_ttl_seconds: 0
  494. -153> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.WAL_size_limit_MB: 0
  495. -152> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.manifest_preallocation_size: 4194304
  496. -151> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.is_fd_close_on_exec: 1
  497. -150> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.advise_random_on_open: 1
  498. -149> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.db_write_buffer_size: 0
  499. -148> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_buffer_manager: 0x33676c0
  500. -147> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.access_hint_on_compaction_start: 1
  501. -146> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  502. -145> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  503. -144> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.use_adaptive_mutex: 0
  504. -143> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.rate_limiter: (nil)
  505. -142> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  506. -141> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_recovery_mode: 0
  507. -140> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_thread_tracking: 0
  508. -139> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_pipelined_write: 0
  509. -138> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  510. -137> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  511. -136> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_thread_max_yield_usec: 100
  512. -135> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  513. -134> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.row_cache: None
  514. -133> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_filter: None
  515. -132> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.avoid_flush_during_recovery: 0
  516. -131> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.allow_ingest_behind: 0
  517. -130> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.preserve_deletes: 0
  518. -129> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.two_write_queues: 0
  519. -128> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.manual_wal_flush: 0
  520. -127> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_jobs: 2
  521. -126> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_background_compactions: -1
  522. -125> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  523. -124> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.writable_file_max_buffer_size: 0
  524. -123> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.delayed_write_rate : 16777216
  525. -122> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_total_wal_size: 0
  526. -121> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  527. -120> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.stats_dump_period_sec: 600
  528. -119> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_open_files: -1
  529. -118> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bytes_per_sync: 0
  530. -117> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.wal_bytes_per_sync: 0
  531. -116> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_readahead_size: 2097152
  532. -115> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Compression algorithms supported:
  533. -114> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZSTDNotFinalCompression supported: 0
  534. -113> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZSTD supported: 0
  535. -112> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kXpressCompression supported: 0
  536. -111> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kLZ4HCCompression supported: 1
  537. -110> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kLZ4Compression supported: 1
  538. -109> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kBZip2Compression supported: 0
  539. -108> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kZlibCompression supported: 1
  540. -107> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: kSnappyCompression supported: 1
  541. -106> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Fast CRC32 supported: Supported on x86
  542. -105> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3088] Recovering from manifest file: MANIFEST-012064
  543.  
  544. -104> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/column_family.cc:474] --------------- Options for column family [default]:
  545.  
  546. -103> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  547. -102> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  548. -101> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_filter: None
  549. -100> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_filter_factory: None
  550. -99> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_factory: SkipListFactory
  551. -98> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_factory: BlockBasedTable
  552. -97> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2fe54e0)
  553. cache_index_and_filter_blocks: 1
  554. cache_index_and_filter_blocks_with_high_priority: 1
  555. pin_l0_filter_and_index_blocks_in_cache: 1
  556. index_type: 0
  557. hash_index_allow_collision: 1
  558. checksum: 1
  559. no_block_cache: 0
  560. block_cache: 0x34027f0
  561. block_cache_name: BinnedLRUCache
  562. block_cache_options:
  563. capacity : 429496729
  564. num_shard_bits : 4
  565. strict_capacity_limit : 0
  566. high_pri_pool_ratio: 0.000
  567. block_cache_compressed: (nil)
  568. persistent_cache: (nil)
  569. block_size: 4096
  570. block_size_deviation: 10
  571. block_restart_interval: 16
  572. index_block_restart_interval: 1
  573. metadata_block_size: 4096
  574. partition_filters: 0
  575. use_delta_encoding: 1
  576. filter_policy: rocksdb.BuiltinBloomFilter
  577. whole_key_filtering: 1
  578. verify_compression: 0
  579. read_amp_bytes_per_bit: 0
  580. format_version: 2
  581. enable_index_compression: 1
  582. block_align: 0
  583.  
  584. -96> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.write_buffer_size: 268435456
  585. -95> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_write_buffer_number: 4
  586. -94> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression: NoCompression
  587. -93> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bottommost_compression: Disabled
  588. -92> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.prefix_extractor: nullptr
  589. -91> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  590. -90> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.num_levels: 7
  591. -89> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  592. -88> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  593. -87> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.window_bits: -14
  594. -86> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.level: -1
  595. -85> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.strategy: 0
  596. -84> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  597. -83> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  598. -82> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  599. -81> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  600. -80> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level0_stop_writes_trigger: 36
  601. -79> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.target_file_size_base: 67108864
  602. -78> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.target_file_size_multiplier: 1
  603. -77> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  604. -76> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  605. -75> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  606. -74> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  607. -73> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  608. -72> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  609. -71> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  610. -70> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  611. -69> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  612. -68> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  613. -67> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  614. -66> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_compaction_bytes: 1677721600
  615. -65> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.arena_block_size: 33554432
  616. -64> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  617. -63> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  618. -62> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  619. -61> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.disable_auto_compactions: 0
  620. -60> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  621. -59> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_pri: kByCompensatedSize
  622. -58> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  623. -57> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  624. -56> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  625. -55> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  626. -54> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  627. -53> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  628. -52> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  629. -51> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  630. -50> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.compaction_options_fifo.ttl: 0
  631. -49> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.table_properties_collectors:
  632. -48> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.inplace_update_support: 0
  633. -47> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.inplace_update_num_locks: 10000
  634. -46> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  635. -45> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.memtable_huge_page_size: 0
  636. -44> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.bloom_locality: 0
  637. -43> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.max_successive_merges: 0
  638. -42> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.optimize_filters_for_hits: 0
  639. -41> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.paranoid_file_checks: 0
  640. -40> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.force_consistency_checks: 0
  641. -39> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.report_bg_io_stats: 0
  642. -38> 2019-08-06 12:50:01.324 7f8a32e4f200 4 rocksdb: Options.ttl: 0
  643. -37> 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3362] Recovered from manifest file:db/MANIFEST-012064 succeeded,manifest_file_number is 12064, next_file_number is 12067, last_sequence is 173668276, log_number is 0,prev_log_number is 0,max_column_family is 0,deleted_log_number is 12062
  644.  
  645. -36> 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3370] Column family [default] (ID 0), log number is 12063
  646.  
  647. -35> 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095802900978, "job": 1, "event": "recovery_started", "log_files": [12065]}
  648. -34> 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:551] Recovering log #12065 mode 0
  649. -33> 2019-08-06 12:50:02.896 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:2863] Creating manifest 12067
  650.  
  651. -32> 2019-08-06 12:50:02.908 7f8a32e4f200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095802910612, "job": 1, "event": "recovery_finished"}
  652. -31> 2019-08-06 12:50:02.908 7f8a32e4f200 5 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_files.cc:380] [JOB 2] Delete db//MANIFEST-012064 type=3 #12064 -- OK
  653.  
  654. -30> 2019-08-06 12:50:02.916 7f8a32e4f200 5 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_files.cc:380] [JOB 2] Delete db//012065.log type=0 #12065 -- OK
  655.  
  656. -29> 2019-08-06 12:50:02.924 7f8a32e4f200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:1218] DB pointer 0x36e8000
  657. -28> 2019-08-06 12:50:02.924 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
  658. -27> 2019-08-06 12:50:03.012 7f8a32e4f200 1 freelist init
  659. -26> 2019-08-06 12:50:03.072 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc opening allocation metadata
  660. -25> 2019-08-06 12:50:04.200 7f8a32e4f200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc loaded 1.3 TiB in 937418 extents
  661. -24> 2019-08-06 12:50:04.220 7f8a32e4f200 2 osd.37 0 journal looks like hdd
  662. -23> 2019-08-06 12:50:04.220 7f8a32e4f200 2 osd.37 0 boot
  663. -22> 2019-08-06 12:50:04.220 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _tune_cache_size target: 4294967296 heap: 66617344 unmapped: 1024000 mapped: 65593344 old cache_size: 134217728 new cache size: 2804010034
  664. -21> 2019-08-06 12:50:04.220 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _trim_shards cache_size: 2804010034 kv_alloc: 134217728 kv_used: 38626570 meta_alloc: 0 meta_used: 45397 data_alloc: 0 data_used: 0
  665. -20> 2019-08-06 12:50:04.300 7f8a32e4f200 0 <cls> /build/ceph-13.2.4/src/cls/hello/cls_hello.cc:296: loading cls_hello
  666. -19> 2019-08-06 12:50:04.300 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/rgw/cls_rgw.cc:3845: Loaded rgw class!
  667. -18> 2019-08-06 12:50:04.300 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/log/cls_log.cc:299: Loaded log class!
  668. -17> 2019-08-06 12:50:04.304 7f8a32e4f200 0 _get_class not permitted to load sdk
  669. -16> 2019-08-06 12:50:04.304 7f8a32e4f200 0 _get_class not permitted to load kvs
  670. -15> 2019-08-06 12:50:04.308 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/refcount/cls_refcount.cc:399: Loaded refcount class!
  671. -14> 2019-08-06 12:50:04.308 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/statelog/cls_statelog.cc:279: Loaded log class!
  672. -13> 2019-08-06 12:50:04.308 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/replica_log/cls_replica_log.cc:135: Loaded replica log class!
  673. -12> 2019-08-06 12:50:04.312 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/version/cls_version.cc:214: Loaded version class!
  674. -11> 2019-08-06 12:50:04.312 7f8a32e4f200 0 _get_class not permitted to load lua
  675. -10> 2019-08-06 12:50:04.312 7f8a32e4f200 0 <cls> /build/ceph-13.2.4/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
  676. -9> 2019-08-06 12:50:04.312 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/timeindex/cls_timeindex.cc:242: Loaded timeindex class!
  677. -8> 2019-08-06 12:50:04.312 7f8a32e4f200 1 <cls> /build/ceph-13.2.4/src/cls/user/cls_user.cc:419: Loaded user class!
  678. -7> 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 432629342416404480, adjusting msgr requires for clients
  679. -6> 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 432629342416404480 was 8705, adjusting msgr requires for mons
  680. -5> 2019-08-06 12:50:04.312 7f8a32e4f200 0 osd.37 196588 crush map has features 1009090094719844352, adjusting msgr requires for osds
  681. -4> 2019-08-06 12:50:09.228 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _tune_cache_size target: 4294967296 heap: 180920320 unmapped: 6742016 mapped: 174178304 old cache_size: 2845415789 new cache size: 2845415831
  682. -3> 2019-08-06 12:50:09.228 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _trim_shards cache_size: 2845415831 kv_alloc: 1279094930 kv_used: 128513958 meta_alloc: 1044213906 meta_used: 45344 data_alloc: 522106952 data_used: 0
  683. -2> 2019-08-06 12:50:14.156 7f8a32e4f200 0 osd.37 196588 load_pgs
  684. -1> 2019-08-06 12:50:14.204 7f8a32e4f200 5 osd.37 pg_epoch: 196586 pg[23.fcs6(unlocked)] enter Initial
  685. 0> 2019-08-06 12:50:14.208 7f8a32e4f200 -1 /build/ceph-13.2.4/src/osd/ECUtil.h: In function 'ECUtil::stripe_info_t::stripe_info_t(uint64_t, uint64_t)' thread 7f8a32e4f200 time 2019-08-06 12:50:14.208743
  686. /build/ceph-13.2.4/src/osd/ECUtil.h: 34: FAILED assert(stripe_width % stripe_size == 0)
  687.  
  688. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  689. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f8a2a1843c2]
  690. 2: (()+0x2e5587) [0x7f8a2a184587]
  691. 3: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  692. 4: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  693. 5: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  694. 6: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  695. 7: (OSD::load_pgs()+0x4a9) [0x758339]
  696. 8: (OSD::init()+0xcd3) [0x7619c3]
  697. 9: (main()+0x3678) [0x64d6a8]
  698. 10: (__libc_start_main()+0xf0) [0x7f8a27d34830]
  699. 11: (_start()+0x29) [0x717389]
  700. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  701.  
  702. --- logging levels ---
  703. 0/ 5 none
  704. 0/ 1 lockdep
  705. 0/ 1 context
  706. 1/ 1 crush
  707. 1/ 5 mds
  708. 1/ 5 mds_balancer
  709. 1/ 5 mds_locker
  710. 1/ 5 mds_log
  711. 1/ 5 mds_log_expire
  712. 1/ 5 mds_migrator
  713. 0/ 1 buffer
  714. 0/ 1 timer
  715. 0/ 1 filer
  716. 0/ 1 striper
  717. 0/ 1 objecter
  718. 0/ 5 rados
  719. 0/ 5 rbd
  720. 0/ 5 rbd_mirror
  721. 0/ 5 rbd_replay
  722. 0/ 5 journaler
  723. 0/ 5 objectcacher
  724. 0/ 5 client
  725. 1/ 5 osd
  726. 0/ 5 optracker
  727. 0/ 5 objclass
  728. 1/ 3 filestore
  729. 1/ 3 journal
  730. 0/ 5 ms
  731. 1/ 5 mon
  732. 0/10 monc
  733. 1/ 5 paxos
  734. 0/ 5 tp
  735. 1/ 5 auth
  736. 1/ 5 crypto
  737. 1/ 1 finisher
  738. 1/ 1 reserver
  739. 1/ 5 heartbeatmap
  740. 1/ 5 perfcounter
  741. 1/ 5 rgw
  742. 1/ 5 rgw_sync
  743. 1/10 civetweb
  744. 1/ 5 javaclient
  745. 1/ 5 asok
  746. 1/ 1 throttle
  747. 0/ 0 refs
  748. 1/ 5 xio
  749. 1/ 5 compressor
  750. 1/ 5 bluestore
  751. 1/ 5 bluefs
  752. 1/ 3 bdev
  753. 1/ 5 kstore
  754. 4/ 5 rocksdb
  755. 4/ 5 leveldb
  756. 4/ 5 memdb
  757. 1/ 5 kinetic
  758. 1/ 5 fuse
  759. 1/ 5 mgr
  760. 1/ 5 mgrc
  761. 1/ 5 dpdk
  762. 1/ 5 eventtrace
  763. 99/99 (syslog threshold)
  764. -1/-1 (stderr threshold)
  765. max_recent 10000
  766. max_new 1000
  767. log_file /var/log/ceph/ceph-osd.37.log
  768. --- end dump of recent events ---
  769. 2019-08-06 12:50:14.240 7f8a32e4f200 -1 *** Caught signal (Aborted) **
  770. in thread 7f8a32e4f200 thread_name:ceph-osd
  771.  
  772. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  773. 1: /usr/bin/ceph-osd() [0xcac700]
  774. 2: (()+0x11390) [0x7f8a28816390]
  775. 3: (gsignal()+0x38) [0x7f8a27d49428]
  776. 4: (abort()+0x16a) [0x7f8a27d4b02a]
  777. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7f8a2a184510]
  778. 6: (()+0x2e5587) [0x7f8a2a184587]
  779. 7: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  780. 8: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  781. 9: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  782. 10: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  783. 11: (OSD::load_pgs()+0x4a9) [0x758339]
  784. 12: (OSD::init()+0xcd3) [0x7619c3]
  785. 13: (main()+0x3678) [0x64d6a8]
  786. 14: (__libc_start_main()+0xf0) [0x7f8a27d34830]
  787. 15: (_start()+0x29) [0x717389]
  788. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  789.  
  790. --- begin dump of recent events ---
  791. -2> 2019-08-06 12:50:14.240 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _tune_cache_size target: 4294967296 heap: 252764160 unmapped: 983040 mapped: 251781120 old cache_size: 2845415832 new cache size: 2845415832
  792. -1> 2019-08-06 12:50:14.240 7f8a1a3ee700 5 bluestore.MempoolThread(0x33d6b08) _trim_shards cache_size: 2845415832 kv_alloc: 1305938492 kv_used: 216611074 meta_alloc: 1003948604 meta_used: 48534 data_alloc: 535528734 data_used: 98304
  793. 0> 2019-08-06 12:50:14.240 7f8a32e4f200 -1 *** Caught signal (Aborted) **
  794. in thread 7f8a32e4f200 thread_name:ceph-osd
  795.  
  796. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  797. 1: /usr/bin/ceph-osd() [0xcac700]
  798. 2: (()+0x11390) [0x7f8a28816390]
  799. 3: (gsignal()+0x38) [0x7f8a27d49428]
  800. 4: (abort()+0x16a) [0x7f8a27d4b02a]
  801. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7f8a2a184510]
  802. 6: (()+0x2e5587) [0x7f8a2a184587]
  803. 7: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  804. 8: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  805. 9: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  806. 10: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  807. 11: (OSD::load_pgs()+0x4a9) [0x758339]
  808. 12: (OSD::init()+0xcd3) [0x7619c3]
  809. 13: (main()+0x3678) [0x64d6a8]
  810. 14: (__libc_start_main()+0xf0) [0x7f8a27d34830]
  811. 15: (_start()+0x29) [0x717389]
  812. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  813.  
  814. --- logging levels ---
  815. 0/ 5 none
  816. 0/ 1 lockdep
  817. 0/ 1 context
  818. 1/ 1 crush
  819. 1/ 5 mds
  820. 1/ 5 mds_balancer
  821. 1/ 5 mds_locker
  822. 1/ 5 mds_log
  823. 1/ 5 mds_log_expire
  824. 1/ 5 mds_migrator
  825. 0/ 1 buffer
  826. 0/ 1 timer
  827. 0/ 1 filer
  828. 0/ 1 striper
  829. 0/ 1 objecter
  830. 0/ 5 rados
  831. 0/ 5 rbd
  832. 0/ 5 rbd_mirror
  833. 0/ 5 rbd_replay
  834. 0/ 5 journaler
  835. 0/ 5 objectcacher
  836. 0/ 5 client
  837. 1/ 5 osd
  838. 0/ 5 optracker
  839. 0/ 5 objclass
  840. 1/ 3 filestore
  841. 1/ 3 journal
  842. 0/ 5 ms
  843. 1/ 5 mon
  844. 0/10 monc
  845. 1/ 5 paxos
  846. 0/ 5 tp
  847. 1/ 5 auth
  848. 1/ 5 crypto
  849. 1/ 1 finisher
  850. 1/ 1 reserver
  851. 1/ 5 heartbeatmap
  852. 1/ 5 perfcounter
  853. 1/ 5 rgw
  854. 1/ 5 rgw_sync
  855. 1/10 civetweb
  856. 1/ 5 javaclient
  857. 1/ 5 asok
  858. 1/ 1 throttle
  859. 0/ 0 refs
  860. 1/ 5 xio
  861. 1/ 5 compressor
  862. 1/ 5 bluestore
  863. 1/ 5 bluefs
  864. 1/ 3 bdev
  865. 1/ 5 kstore
  866. 4/ 5 rocksdb
  867. 4/ 5 leveldb
  868. 4/ 5 memdb
  869. 1/ 5 kinetic
  870. 1/ 5 fuse
  871. 1/ 5 mgr
  872. 1/ 5 mgrc
  873. 1/ 5 dpdk
  874. 1/ 5 eventtrace
  875. 99/99 (syslog threshold)
  876. -1/-1 (stderr threshold)
  877. max_recent 10000
  878. max_new 1000
  879. log_file /var/log/ceph/ceph-osd.37.log
  880. --- end dump of recent events ---
  881. 2019-08-06 12:50:34.756 7f70a4b76200 0 set uid:gid to 64045:64045 (ceph:ceph)
  882. 2019-08-06 12:50:34.760 7f70a4b76200 0 ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable), process ceph-osd, pid 2987
  883. 2019-08-06 12:50:35.032 7f70a4b76200 0 pidfile_write: ignore empty --pid-file
  884. 2019-08-06 12:50:35.036 7f70a4b76200 -1 Public network was set, but cluster network was not set
  885. 2019-08-06 12:50:35.036 7f70a4b76200 -1 Using public network also for cluster network
  886. 2019-08-06 12:50:35.044 7f70a4b76200 0 load: jerasure load: lrc load: isa
  887. 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  888. 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  889. 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  890. 2019-08-06 12:50:35.044 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  891. 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) close
  892. 2019-08-06 12:50:35.328 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _mount path /var/lib/ceph/osd/ceph-37
  893. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  894. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  895. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  896. 2019-08-06 12:50:35.328 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  897. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  898. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de380 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  899. 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de380 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  900. 2019-08-06 12:50:35.328 7f70a4b76200 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-37/block size 3.6 TiB
  901. 2019-08-06 12:50:35.328 7f70a4b76200 1 bluefs mount
  902. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compaction_readahead_size = 2097152
  903. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compression = kNoCompression
  904. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option max_write_buffer_number = 4
  905. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  906. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option recycle_log_file_num = 4
  907. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option writable_file_max_buffer_size = 0
  908. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option write_buffer_size = 268435456
  909. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compaction_readahead_size = 2097152
  910. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compression = kNoCompression
  911. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option max_write_buffer_number = 4
  912. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  913. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option recycle_log_file_num = 4
  914. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option writable_file_max_buffer_size = 0
  915. 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option write_buffer_size = 268435456
  916. 2019-08-06 12:50:35.516 7f70a4b76200 1 rocksdb: do_open column families: [default]
  917. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: RocksDB version: 5.13.0
  918.  
  919. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  920. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Compile date Jan 4 2019
  921. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: DB SUMMARY
  922.  
  923. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: CURRENT file: CURRENT
  924.  
  925. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: IDENTITY file: IDENTITY
  926.  
  927. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: MANIFEST file: MANIFEST-012067 size: 46845 Bytes
  928.  
  929. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: SST files in db dir, Total Num: 191, files: 009870.sst 010106.sst 010110.sst 010155.sst 010199.sst 010200.sst 010201.sst 010286.sst 010287.sst
  930.  
  931. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Write Ahead Log file in db: 012068.log size: 0 ;
  932.  
  933. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.error_if_exists: 0
  934. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.create_if_missing: 0
  935. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.paranoid_checks: 1
  936. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.env: 0x26dfa60
  937. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.info_log: 0x29f5d60
  938. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_file_opening_threads: 16
  939. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.statistics: (nil)
  940. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_fsync: 0
  941. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_log_file_size: 0
  942. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
  943. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.log_file_time_to_roll: 0
  944. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.keep_log_file_num: 1000
  945. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.recycle_log_file_num: 4
  946. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_fallocate: 1
  947. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_mmap_reads: 0
  948. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_mmap_writes: 0
  949. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_direct_reads: 0
  950. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  951. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.create_missing_column_families: 0
  952. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.db_log_dir:
  953. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_dir: db
  954. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_cache_numshardbits: 6
  955. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_subcompactions: 1
  956. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_flushes: -1
  957. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.WAL_ttl_seconds: 0
  958. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.WAL_size_limit_MB: 0
  959. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.manifest_preallocation_size: 4194304
  960. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.is_fd_close_on_exec: 1
  961. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.advise_random_on_open: 1
  962. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.db_write_buffer_size: 0
  963. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_buffer_manager: 0x2681690
  964. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.access_hint_on_compaction_start: 1
  965. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  966. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  967. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_adaptive_mutex: 0
  968. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.rate_limiter: (nil)
  969. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  970. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_recovery_mode: 0
  971. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_thread_tracking: 0
  972. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_pipelined_write: 0
  973. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  974. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  975. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_thread_max_yield_usec: 100
  976. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  977. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.row_cache: None
  978. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_filter: None
  979. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.avoid_flush_during_recovery: 0
  980. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_ingest_behind: 0
  981. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.preserve_deletes: 0
  982. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.two_write_queues: 0
  983. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.manual_wal_flush: 0
  984. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_jobs: 2
  985. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_compactions: -1
  986. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  987. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.writable_file_max_buffer_size: 0
  988. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.delayed_write_rate : 16777216
  989. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_total_wal_size: 0
  990. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  991. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.stats_dump_period_sec: 600
  992. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_open_files: -1
  993. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bytes_per_sync: 0
  994. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_bytes_per_sync: 0
  995. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_readahead_size: 2097152
  996. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Compression algorithms supported:
  997. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZSTDNotFinalCompression supported: 0
  998. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZSTD supported: 0
  999. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kXpressCompression supported: 0
  1000. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kLZ4HCCompression supported: 1
  1001. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kLZ4Compression supported: 1
  1002. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kBZip2Compression supported: 0
  1003. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZlibCompression supported: 1
  1004. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kSnappyCompression supported: 1
  1005. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Fast CRC32 supported: Supported on x86
  1006. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3088] Recovering from manifest file: MANIFEST-012067
  1007.  
  1008. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/column_family.cc:474] --------------- Options for column family [default]:
  1009.  
  1010. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  1011. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  1012. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_filter: None
  1013. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_filter_factory: None
  1014. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_factory: SkipListFactory
  1015. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_factory: BlockBasedTable
  1016. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x22ff4e0)
  1017. cache_index_and_filter_blocks: 1
  1018. cache_index_and_filter_blocks_with_high_priority: 1
  1019. pin_l0_filter_and_index_blocks_in_cache: 1
  1020. index_type: 0
  1021. hash_index_allow_collision: 1
  1022. checksum: 1
  1023. no_block_cache: 0
  1024. block_cache: 0x271c7f0
  1025. block_cache_name: BinnedLRUCache
  1026. block_cache_options:
  1027. capacity : 429496729
  1028. num_shard_bits : 4
  1029. strict_capacity_limit : 0
  1030. high_pri_pool_ratio: 0.000
  1031. block_cache_compressed: (nil)
  1032. persistent_cache: (nil)
  1033. block_size: 4096
  1034. block_size_deviation: 10
  1035. block_restart_interval: 16
  1036. index_block_restart_interval: 1
  1037. metadata_block_size: 4096
  1038. partition_filters: 0
  1039. use_delta_encoding: 1
  1040. filter_policy: rocksdb.BuiltinBloomFilter
  1041. whole_key_filtering: 1
  1042. verify_compression: 0
  1043. read_amp_bytes_per_bit: 0
  1044. format_version: 2
  1045. enable_index_compression: 1
  1046. block_align: 0
  1047.  
  1048. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_buffer_size: 268435456
  1049. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_write_buffer_number: 4
  1050. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression: NoCompression
  1051. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bottommost_compression: Disabled
  1052. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.prefix_extractor: nullptr
  1053. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  1054. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.num_levels: 7
  1055. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  1056. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  1057. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.window_bits: -14
  1058. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.level: -1
  1059. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.strategy: 0
  1060. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  1061. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  1062. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  1063. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  1064. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_stop_writes_trigger: 36
  1065. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.target_file_size_base: 67108864
  1066. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.target_file_size_multiplier: 1
  1067. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  1068. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  1069. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  1070. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  1071. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  1072. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  1073. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  1074. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  1075. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  1076. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  1077. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  1078. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_compaction_bytes: 1677721600
  1079. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.arena_block_size: 33554432
  1080. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  1081. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  1082. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  1083. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.disable_auto_compactions: 0
  1084. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  1085. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_pri: kByCompensatedSize
  1086. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  1087. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  1088. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  1089. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  1090. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  1091. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  1092. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  1093. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  1094. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.ttl: 0
  1095. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_properties_collectors:
  1096. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.inplace_update_support: 0
  1097. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.inplace_update_num_locks: 10000
  1098. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  1099. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_huge_page_size: 0
  1100. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bloom_locality: 0
  1101. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_successive_merges: 0
  1102. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.optimize_filters_for_hits: 0
  1103. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.paranoid_file_checks: 0
  1104. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.force_consistency_checks: 0
  1105. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.report_bg_io_stats: 0
  1106. 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.ttl: 0
  1107. 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3362] Recovered from manifest file:db/MANIFEST-012067 succeeded,manifest_file_number is 12067, next_file_number is 12070, last_sequence is 173668276, log_number is 0,prev_log_number is 0,max_column_family is 0,deleted_log_number is 12065
  1108.  
  1109. 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3370] Column family [default] (ID 0), log number is 12066
  1110.  
  1111. 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095837076627, "job": 1, "event": "recovery_started", "log_files": [12068]}
  1112. 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:551] Recovering log #12068 mode 0
  1113. 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:2863] Creating manifest 12070
  1114.  
  1115. 2019-08-06 12:50:37.084 7f70a4b76200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095837086334, "job": 1, "event": "recovery_finished"}
  1116. 2019-08-06 12:50:37.100 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:1218] DB pointer 0x27f5000
  1117. 2019-08-06 12:50:37.100 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
  1118. 2019-08-06 12:50:37.180 7f70a4b76200 1 freelist init
  1119. 2019-08-06 12:50:37.240 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc opening allocation metadata
  1120. 2019-08-06 12:50:38.320 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc loaded 1.3 TiB in 937418 extents
  1121. 2019-08-06 12:50:38.408 7f70a4b76200 0 <cls> /build/ceph-13.2.4/src/cls/hello/cls_hello.cc:296: loading cls_hello
  1122. 2019-08-06 12:50:38.412 7f70a4b76200 0 _get_class not permitted to load sdk
  1123. 2019-08-06 12:50:38.412 7f70a4b76200 0 _get_class not permitted to load kvs
  1124. 2019-08-06 12:50:38.420 7f70a4b76200 0 _get_class not permitted to load lua
  1125. 2019-08-06 12:50:38.420 7f70a4b76200 0 <cls> /build/ceph-13.2.4/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
  1126. 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 432629342416404480, adjusting msgr requires for clients
  1127. 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 432629342416404480 was 8705, adjusting msgr requires for mons
  1128. 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 1009090094719844352, adjusting msgr requires for osds
  1129. 2019-08-06 12:50:46.340 7f70a4b76200 0 osd.37 196588 load_pgs
  1130. 2019-08-06 12:50:46.384 7f70a4b76200 -1 /build/ceph-13.2.4/src/osd/ECUtil.h: In function 'ECUtil::stripe_info_t::stripe_info_t(uint64_t, uint64_t)' thread 7f70a4b76200 time 2019-08-06 12:50:46.383353
  1131. /build/ceph-13.2.4/src/osd/ECUtil.h: 34: FAILED assert(stripe_width % stripe_size == 0)
  1132.  
  1133. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  1134. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f709beab3c2]
  1135. 2: (()+0x2e5587) [0x7f709beab587]
  1136. 3: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  1137. 4: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  1138. 5: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  1139. 6: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  1140. 7: (OSD::load_pgs()+0x4a9) [0x758339]
  1141. 8: (OSD::init()+0xcd3) [0x7619c3]
  1142. 9: (main()+0x3678) [0x64d6a8]
  1143. 10: (__libc_start_main()+0xf0) [0x7f7099a5b830]
  1144. 11: (_start()+0x29) [0x717389]
  1145. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1146.  
  1147. --- begin dump of recent events ---
  1148. -339> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perfcounters_dump hook 0x2302250
  1149. -338> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command 1 hook 0x2302250
  1150. -337> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perf dump hook 0x2302250
  1151. -336> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perfcounters_schema hook 0x2302250
  1152. -335> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perf histogram dump hook 0x2302250
  1153. -334> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command 2 hook 0x2302250
  1154. -333> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perf schema hook 0x2302250
  1155. -332> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perf histogram schema hook 0x2302250
  1156. -331> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command perf reset hook 0x2302250
  1157. -330> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config show hook 0x2302250
  1158. -329> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config help hook 0x2302250
  1159. -328> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config set hook 0x2302250
  1160. -327> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config unset hook 0x2302250
  1161. -326> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config get hook 0x2302250
  1162. -325> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config diff hook 0x2302250
  1163. -324> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command config diff get hook 0x2302250
  1164. -323> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command log flush hook 0x2302250
  1165. -322> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command log dump hook 0x2302250
  1166. -321> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command log reopen hook 0x2302250
  1167. -320> 2019-08-06 12:50:34.744 7f70a4b76200 5 asok(0x2362240) register_command dump_mempools hook 0x26a8068
  1168. -319> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient: get_monmap_and_config
  1169. -318> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient: build_initial_monmap
  1170. -317> 2019-08-06 12:50:34.748 7f7096752700 2 Event(0x2362e00 nevent=5000 time_id=1).set_owner idx=0 owner=140121537324800
  1171. -316> 2019-08-06 12:50:34.748 7f7095f51700 2 Event(0x2363040 nevent=5000 time_id=1).set_owner idx=1 owner=140121528932096
  1172. -315> 2019-08-06 12:50:34.748 7f7095750700 2 Event(0x2363280 nevent=5000 time_id=1).set_owner idx=2 owner=140121520539392
  1173. -314> 2019-08-06 12:50:34.748 7f70a4b76200 1 Processor -- start
  1174. -313> 2019-08-06 12:50:34.748 7f70a4b76200 1 -- - start start
  1175. -312> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient: init
  1176. -311> 2019-08-06 12:50:34.748 7f70a4b76200 5 adding auth protocol: cephx
  1177. -310> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient: auth_supported 2 method cephx
  1178. -309> 2019-08-06 12:50:34.748 7f70a4b76200 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-37/keyring
  1179. -308> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient: _reopen_session rank -1
  1180. -307> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient(hunting): picked mon.noname-b con 0x27ae700 addr 10.100.219.12:6789/0
  1181. -306> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient(hunting): picked mon.noname-a con 0x27aee00 addr 10.100.219.11:6789/0
  1182. -305> 2019-08-06 12:50:34.748 7f70a4b76200 1 -- - --> 10.100.219.11:6789/0 -- auth(proto 0 27 bytes epoch 0) v1 -- 0x2720780 con 0
  1183. -304> 2019-08-06 12:50:34.748 7f70a4b76200 1 -- - --> 10.100.219.12:6789/0 -- auth(proto 0 27 bytes epoch 0) v1 -- 0x2720a00 con 0
  1184. -303> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient(hunting): _renew_subs
  1185. -302> 2019-08-06 12:50:34.748 7f70a4b76200 10 monclient(hunting): authenticate will time out at 2019-08-06 12:55:34.753823
  1186. -301> 2019-08-06 12:50:34.752 7f7095750700 1 -- 10.100.219.111:0/4092249503 learned_addr learned my addr 10.100.219.111:0/4092249503
  1187. -300> 2019-08-06 12:50:34.752 7f7095750700 2 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0
  1188. -299> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 1 0x2363d40 mon_map magic: 0 v1
  1189. -298> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 2 0x2720780 auth_reply(proto 2 0 (0) Success) v1
  1190. -297> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 1 ==== mon_map magic: 0 v1 ==== 397+0+0 (1649140285 0 0) 0x2363d40 con 0x27aee00
  1191. -296> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1
  1192. -295> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient(hunting): got monmap 9, mon.noname-a is now rank -1
  1193. -294> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient(hunting): dump:
  1194. epoch 9
  1195. fsid f5e1c608-870b-425e-ab8a-bdc86789e37c
  1196. last_changed 2018-10-22 11:56:22.979297
  1197. created 2016-10-26 12:14:35.362412
  1198. 0: 10.100.219.11:6789/0 mon.mon01
  1199. 1: 10.100.219.12:6789/0 mon.mon02
  1200. 2: 10.100.219.13:6789/0 mon.mon03
  1201.  
  1202. -293> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (3711223035 0 0) 0x2720780 con 0x27aee00
  1203. -292> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient(hunting): my global_id is 50114094
  1204. -291> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 --> 10.100.219.11:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- 0x2720f00 con 0
  1205. -290> 2019-08-06 12:50:34.752 7f7095f51700 2 -- 10.100.219.111:0/4092249503 >> 10.100.219.12:6789/0 conn(0x27ae700 :-1 s=STATE_CONNECTING_WAIT_ACK_SEQ pgs=0 cs=0 l=0)._process_connection got newly_acked_seq 0 vs out_seq 0
  1206. -289> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 3 0x2720f00 auth_reply(proto 2 0 (0) Success) v1
  1207. -288> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (1498746715 0 0) 0x2720f00 con 0x27aee00
  1208. -287> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 >> 10.100.219.12:6789/0 conn(0x27ae700 :-1 s=STATE_OPEN pgs=6851 cs=1 l=1).mark_down
  1209. -286> 2019-08-06 12:50:34.752 7f7094f4f700 2 -- 10.100.219.111:0/4092249503 >> 10.100.219.12:6789/0 conn(0x27ae700 :-1 s=STATE_OPEN pgs=6851 cs=1 l=1)._stop
  1210. -285> 2019-08-06 12:50:34.752 7f7094f4f700 1 monclient: found mon.mon01
  1211. -284> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: _send_mon_message to mon.mon01 at 10.100.219.11:6789/0
  1212. -283> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 --> 10.100.219.11:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2363440 con 0
  1213. -282> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-08-06 12:50:04.756654)
  1214. -281> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: _send_mon_message to mon.mon01 at 10.100.219.11:6789/0
  1215. -280> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 --> 10.100.219.11:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x2720780 con 0
  1216. -279> 2019-08-06 12:50:34.752 7f70a4b76200 5 monclient: authenticate success, global_id 50114094
  1217. -278> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 4 0x2363440 config(0 keys) v1
  1218. -277> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 5 0x2363f80 mon_map magic: 0 v1
  1219. -276> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (0 0 0) 0x2363440 con 0x27aee00
  1220. -275> 2019-08-06 12:50:34.752 7f7095750700 5 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=10968 cs=1 l=1). rx mon.0 seq 6 0x2720780 auth_reply(proto 2 0 (0) Success) v1
  1221. -274> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: handle_config config(0 keys) v1
  1222. -273> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 5 ==== mon_map magic: 0 v1 ==== 397+0+0 (1649140285 0 0) 0x2363f80 con 0x27aee00
  1223. -272> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: handle_monmap mon_map magic: 0 v1
  1224. -271> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: got monmap 9, mon.mon01 is now rank 0
  1225. -270> 2019-08-06 12:50:34.752 7f709374c700 4 set_mon_vals no callback set
  1226. -269> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: dump:
  1227. epoch 9
  1228. fsid f5e1c608-870b-425e-ab8a-bdc86789e37c
  1229. last_changed 2018-10-22 11:56:22.979297
  1230. created 2016-10-26 12:14:35.362412
  1231. 0: 10.100.219.11:6789/0 mon.mon01
  1232. 1: 10.100.219.12:6789/0 mon.mon02
  1233. 2: 10.100.219.13:6789/0 mon.mon03
  1234.  
  1235. -268> 2019-08-06 12:50:34.752 7f70a4b76200 10 monclient: get_monmap_and_config success
  1236. -267> 2019-08-06 12:50:34.752 7f7094f4f700 1 -- 10.100.219.111:0/4092249503 <== mon.0 10.100.219.11:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (3313160579 0 0) 0x2720780 con 0x27aee00
  1237. -266> 2019-08-06 12:50:34.752 7f70a4b76200 10 monclient: shutdown
  1238. -265> 2019-08-06 12:50:34.752 7f7094f4f700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-08-06 12:50:04.757478)
  1239. -264> 2019-08-06 12:50:34.752 7f70a4b76200 1 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN pgs=10968 cs=1 l=1).mark_down
  1240. -263> 2019-08-06 12:50:34.752 7f70a4b76200 2 -- 10.100.219.111:0/4092249503 >> 10.100.219.11:6789/0 conn(0x27aee00 :-1 s=STATE_OPEN pgs=10968 cs=1 l=1)._stop
  1241. -262> 2019-08-06 12:50:34.756 7f70a4b76200 1 -- 10.100.219.111:0/4092249503 shutdown_connections
  1242. -261> 2019-08-06 12:50:34.756 7f70a4b76200 5 -- 10.100.219.111:0/4092249503 shutdown_connections mark down 10.100.219.11:6789/0 0x27aee00
  1243. -260> 2019-08-06 12:50:34.756 7f70a4b76200 5 -- 10.100.219.111:0/4092249503 shutdown_connections mark down 10.100.219.12:6789/0 0x27ae700
  1244. -259> 2019-08-06 12:50:34.756 7f70a4b76200 5 -- 10.100.219.111:0/4092249503 shutdown_connections delete 0x27ae700
  1245. -258> 2019-08-06 12:50:34.756 7f70a4b76200 5 -- 10.100.219.111:0/4092249503 shutdown_connections delete 0x27aee00
  1246. -257> 2019-08-06 12:50:34.756 7f70a4b76200 1 -- 10.100.219.111:0/4092249503 shutdown_connections
  1247. -256> 2019-08-06 12:50:34.756 7f70a4b76200 1 -- 10.100.219.111:0/4092249503 wait complete.
  1248. -255> 2019-08-06 12:50:34.756 7f70a4b76200 1 -- 10.100.219.111:0/4092249503 >> 10.100.219.111:0/4092249503 conn(0x27ae000 :-1 s=STATE_NONE pgs=0 cs=0 l=0).mark_down
  1249. -254> 2019-08-06 12:50:34.756 7f70a4b76200 2 -- 10.100.219.111:0/4092249503 >> 10.100.219.111:0/4092249503 conn(0x27ae000 :-1 s=STATE_NONE pgs=0 cs=0 l=0)._stop
  1250. -253> 2019-08-06 12:50:34.756 7f70a4b76200 0 set uid:gid to 64045:64045 (ceph:ceph)
  1251. -252> 2019-08-06 12:50:34.760 7f70a4b76200 0 ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable), process ceph-osd, pid 2987
  1252. -251> 2019-08-06 12:50:35.032 7f70a4b76200 0 pidfile_write: ignore empty --pid-file
  1253. -250> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) init /var/run/ceph/ceph-osd.37.asok
  1254. -249> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) bind_and_listen /var/run/ceph/ceph-osd.37.asok
  1255. -248> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) register_command 0 hook 0x22ff490
  1256. -247> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) register_command version hook 0x22ff490
  1257. -246> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) register_command git_version hook 0x22ff490
  1258. -245> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) register_command help hook 0x2302200
  1259. -244> 2019-08-06 12:50:35.032 7f70a4b76200 5 asok(0x2362240) register_command get_command_descriptions hook 0x23023f0
  1260. -243> 2019-08-06 12:50:35.032 7f709474e700 5 asok(0x2362240) entry start
  1261. -242> 2019-08-06 12:50:35.032 7f70a4b76200 5 object store type is bluestore
  1262. -241> 2019-08-06 12:50:35.036 7f70a4b76200 -1 Public network was set, but cluster network was not set
  1263. -240> 2019-08-06 12:50:35.036 7f70a4b76200 -1 Using public network also for cluster network
  1264. -239> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  1265. -238> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:6804/2987 _finish_bind bind my_inst.addr is 10.100.219.111:6804/2987
  1266. -237> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  1267. -236> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:6805/2987 _finish_bind bind my_inst.addr is 10.100.219.111:6805/2987
  1268. -235> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  1269. -234> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:6806/2987 _finish_bind bind my_inst.addr is 10.100.219.111:6806/2987
  1270. -233> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:0/0 learned_addr learned my addr 10.100.219.111:0/0
  1271. -232> 2019-08-06 12:50:35.036 7f70a4b76200 1 -- 10.100.219.111:6807/2987 _finish_bind bind my_inst.addr is 10.100.219.111:6807/2987
  1272. -231> 2019-08-06 12:50:35.036 7f70a4b76200 10 monclient: build_initial_monmap
  1273. -230> 2019-08-06 12:50:35.044 7f70a4b76200 0 load: jerasure load: lrc load: isa
  1274. -229> 2019-08-06 12:50:35.044 7f70a4b76200 5 adding auth protocol: cephx
  1275. -228> 2019-08-06 12:50:35.044 7f70a4b76200 5 adding auth protocol: cephx
  1276. -227> 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  1277. -226> 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  1278. -225> 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  1279. -224> 2019-08-06 12:50:35.044 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  1280. -223> 2019-08-06 12:50:35.044 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) close
  1281. -222> 2019-08-06 12:50:35.328 7f70a4b76200 5 asok(0x2362240) register_command objecter_requests hook 0x2302710
  1282. -221> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- 10.100.219.111:6804/2987 start start
  1283. -220> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- - start start
  1284. -219> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- - start start
  1285. -218> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- 10.100.219.111:6807/2987 start start
  1286. -217> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- 10.100.219.111:6806/2987 start start
  1287. -216> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- 10.100.219.111:6805/2987 start start
  1288. -215> 2019-08-06 12:50:35.328 7f70a4b76200 1 -- - start start
  1289. -214> 2019-08-06 12:50:35.328 7f70a4b76200 2 osd.37 0 init /var/lib/ceph/osd/ceph-37 (looks like hdd)
  1290. -213> 2019-08-06 12:50:35.328 7f70a4b76200 2 osd.37 0 journal /var/lib/ceph/osd/ceph-37/journal
  1291. -212> 2019-08-06 12:50:35.328 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _mount path /var/lib/ceph/osd/ceph-37
  1292. -211> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  1293. -210> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  1294. -209> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de000 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  1295. -208> 2019-08-06 12:50:35.328 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2
  1296. -207> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev create path /var/lib/ceph/osd/ceph-37/block type kernel
  1297. -206> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de380 /var/lib/ceph/osd/ceph-37/block) open path /var/lib/ceph/osd/ceph-37/block
  1298. -205> 2019-08-06 12:50:35.328 7f70a4b76200 1 bdev(0x28de380 /var/lib/ceph/osd/ceph-37/block) open size 4000220971008 (0x3a35fc00000, 3.6 TiB) block_size 4096 (4 KiB) rotational
  1299. -204> 2019-08-06 12:50:35.328 7f70a4b76200 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-37/block size 3.6 TiB
  1300. -203> 2019-08-06 12:50:35.328 7f70a4b76200 1 bluefs mount
  1301. -202> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compaction_readahead_size = 2097152
  1302. -201> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compression = kNoCompression
  1303. -200> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option max_write_buffer_number = 4
  1304. -199> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1305. -198> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option recycle_log_file_num = 4
  1306. -197> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option writable_file_max_buffer_size = 0
  1307. -196> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option write_buffer_size = 268435456
  1308. -195> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compaction_readahead_size = 2097152
  1309. -194> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option compression = kNoCompression
  1310. -193> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option max_write_buffer_number = 4
  1311. -192> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1312. -191> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option recycle_log_file_num = 4
  1313. -190> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option writable_file_max_buffer_size = 0
  1314. -189> 2019-08-06 12:50:35.496 7f70a4b76200 0 set rocksdb option write_buffer_size = 268435456
  1315. -188> 2019-08-06 12:50:35.516 7f70a4b76200 1 rocksdb: do_open column families: [default]
  1316. -187> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: RocksDB version: 5.13.0
  1317.  
  1318. -186> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  1319. -185> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Compile date Jan 4 2019
  1320. -184> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: DB SUMMARY
  1321.  
  1322. -183> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: CURRENT file: CURRENT
  1323.  
  1324. -182> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: IDENTITY file: IDENTITY
  1325.  
  1326. -181> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: MANIFEST file: MANIFEST-012067 size: 46845 Bytes
  1327.  
  1328. -180> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: SST files in db dir, Total Num: 191, files: 009870.sst 010106.sst 010110.sst 010155.sst 010199.sst 010200.sst 010201.sst 010286.sst 010287.sst
  1329.  
  1330. -179> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Write Ahead Log file in db: 012068.log size: 0 ;
  1331.  
  1332. -178> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.error_if_exists: 0
  1333. -177> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.create_if_missing: 0
  1334. -176> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.paranoid_checks: 1
  1335. -175> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.env: 0x26dfa60
  1336. -174> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.info_log: 0x29f5d60
  1337. -173> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_file_opening_threads: 16
  1338. -172> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.statistics: (nil)
  1339. -171> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_fsync: 0
  1340. -170> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_log_file_size: 0
  1341. -169> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
  1342. -168> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.log_file_time_to_roll: 0
  1343. -167> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.keep_log_file_num: 1000
  1344. -166> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.recycle_log_file_num: 4
  1345. -165> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_fallocate: 1
  1346. -164> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_mmap_reads: 0
  1347. -163> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_mmap_writes: 0
  1348. -162> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_direct_reads: 0
  1349. -161> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  1350. -160> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.create_missing_column_families: 0
  1351. -159> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.db_log_dir:
  1352. -158> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_dir: db
  1353. -157> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_cache_numshardbits: 6
  1354. -156> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_subcompactions: 1
  1355. -155> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_flushes: -1
  1356. -154> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.WAL_ttl_seconds: 0
  1357. -153> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.WAL_size_limit_MB: 0
  1358. -152> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.manifest_preallocation_size: 4194304
  1359. -151> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.is_fd_close_on_exec: 1
  1360. -150> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.advise_random_on_open: 1
  1361. -149> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.db_write_buffer_size: 0
  1362. -148> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_buffer_manager: 0x2681690
  1363. -147> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.access_hint_on_compaction_start: 1
  1364. -146> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  1365. -145> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  1366. -144> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.use_adaptive_mutex: 0
  1367. -143> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.rate_limiter: (nil)
  1368. -142> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  1369. -141> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_recovery_mode: 0
  1370. -140> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_thread_tracking: 0
  1371. -139> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_pipelined_write: 0
  1372. -138> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  1373. -137> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  1374. -136> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_thread_max_yield_usec: 100
  1375. -135> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  1376. -134> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.row_cache: None
  1377. -133> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_filter: None
  1378. -132> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.avoid_flush_during_recovery: 0
  1379. -131> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.allow_ingest_behind: 0
  1380. -130> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.preserve_deletes: 0
  1381. -129> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.two_write_queues: 0
  1382. -128> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.manual_wal_flush: 0
  1383. -127> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_jobs: 2
  1384. -126> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_background_compactions: -1
  1385. -125> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  1386. -124> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.writable_file_max_buffer_size: 0
  1387. -123> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.delayed_write_rate : 16777216
  1388. -122> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_total_wal_size: 0
  1389. -121> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  1390. -120> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.stats_dump_period_sec: 600
  1391. -119> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_open_files: -1
  1392. -118> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bytes_per_sync: 0
  1393. -117> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.wal_bytes_per_sync: 0
  1394. -116> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_readahead_size: 2097152
  1395. -115> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Compression algorithms supported:
  1396. -114> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZSTDNotFinalCompression supported: 0
  1397. -113> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZSTD supported: 0
  1398. -112> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kXpressCompression supported: 0
  1399. -111> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kLZ4HCCompression supported: 1
  1400. -110> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kLZ4Compression supported: 1
  1401. -109> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kBZip2Compression supported: 0
  1402. -108> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kZlibCompression supported: 1
  1403. -107> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: kSnappyCompression supported: 1
  1404. -106> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Fast CRC32 supported: Supported on x86
  1405. -105> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3088] Recovering from manifest file: MANIFEST-012067
  1406.  
  1407. -104> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/column_family.cc:474] --------------- Options for column family [default]:
  1408.  
  1409. -103> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  1410. -102> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  1411. -101> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_filter: None
  1412. -100> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_filter_factory: None
  1413. -99> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_factory: SkipListFactory
  1414. -98> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_factory: BlockBasedTable
  1415. -97> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x22ff4e0)
  1416. cache_index_and_filter_blocks: 1
  1417. cache_index_and_filter_blocks_with_high_priority: 1
  1418. pin_l0_filter_and_index_blocks_in_cache: 1
  1419. index_type: 0
  1420. hash_index_allow_collision: 1
  1421. checksum: 1
  1422. no_block_cache: 0
  1423. block_cache: 0x271c7f0
  1424. block_cache_name: BinnedLRUCache
  1425. block_cache_options:
  1426. capacity : 429496729
  1427. num_shard_bits : 4
  1428. strict_capacity_limit : 0
  1429. high_pri_pool_ratio: 0.000
  1430. block_cache_compressed: (nil)
  1431. persistent_cache: (nil)
  1432. block_size: 4096
  1433. block_size_deviation: 10
  1434. block_restart_interval: 16
  1435. index_block_restart_interval: 1
  1436. metadata_block_size: 4096
  1437. partition_filters: 0
  1438. use_delta_encoding: 1
  1439. filter_policy: rocksdb.BuiltinBloomFilter
  1440. whole_key_filtering: 1
  1441. verify_compression: 0
  1442. read_amp_bytes_per_bit: 0
  1443. format_version: 2
  1444. enable_index_compression: 1
  1445. block_align: 0
  1446.  
  1447. -96> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.write_buffer_size: 268435456
  1448. -95> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_write_buffer_number: 4
  1449. -94> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression: NoCompression
  1450. -93> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bottommost_compression: Disabled
  1451. -92> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.prefix_extractor: nullptr
  1452. -91> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  1453. -90> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.num_levels: 7
  1454. -89> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  1455. -88> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  1456. -87> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.window_bits: -14
  1457. -86> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.level: -1
  1458. -85> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.strategy: 0
  1459. -84> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  1460. -83> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  1461. -82> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  1462. -81> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  1463. -80> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level0_stop_writes_trigger: 36
  1464. -79> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.target_file_size_base: 67108864
  1465. -78> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.target_file_size_multiplier: 1
  1466. -77> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  1467. -76> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  1468. -75> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  1469. -74> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  1470. -73> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  1471. -72> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  1472. -71> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  1473. -70> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  1474. -69> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  1475. -68> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  1476. -67> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  1477. -66> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_compaction_bytes: 1677721600
  1478. -65> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.arena_block_size: 33554432
  1479. -64> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  1480. -63> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  1481. -62> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  1482. -61> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.disable_auto_compactions: 0
  1483. -60> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  1484. -59> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_pri: kByCompensatedSize
  1485. -58> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  1486. -57> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  1487. -56> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  1488. -55> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  1489. -54> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  1490. -53> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  1491. -52> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  1492. -51> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  1493. -50> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.compaction_options_fifo.ttl: 0
  1494. -49> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.table_properties_collectors:
  1495. -48> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.inplace_update_support: 0
  1496. -47> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.inplace_update_num_locks: 10000
  1497. -46> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  1498. -45> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.memtable_huge_page_size: 0
  1499. -44> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.bloom_locality: 0
  1500. -43> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.max_successive_merges: 0
  1501. -42> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.optimize_filters_for_hits: 0
  1502. -41> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.paranoid_file_checks: 0
  1503. -40> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.force_consistency_checks: 0
  1504. -39> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.report_bg_io_stats: 0
  1505. -38> 2019-08-06 12:50:35.516 7f70a4b76200 4 rocksdb: Options.ttl: 0
  1506. -37> 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3362] Recovered from manifest file:db/MANIFEST-012067 succeeded,manifest_file_number is 12067, next_file_number is 12070, last_sequence is 173668276, log_number is 0,prev_log_number is 0,max_column_family is 0,deleted_log_number is 12065
  1507.  
  1508. -36> 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:3370] Column family [default] (ID 0), log number is 12066
  1509.  
  1510. -35> 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095837076627, "job": 1, "event": "recovery_started", "log_files": [12068]}
  1511. -34> 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:551] Recovering log #12068 mode 0
  1512. -33> 2019-08-06 12:50:37.072 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/version_set.cc:2863] Creating manifest 12070
  1513.  
  1514. -32> 2019-08-06 12:50:37.084 7f70a4b76200 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1565095837086334, "job": 1, "event": "recovery_finished"}
  1515. -31> 2019-08-06 12:50:37.084 7f70a4b76200 5 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_files.cc:380] [JOB 2] Delete db//MANIFEST-012067 type=3 #12067 -- OK
  1516.  
  1517. -30> 2019-08-06 12:50:37.092 7f70a4b76200 5 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_files.cc:380] [JOB 2] Delete db//012068.log type=0 #12068 -- OK
  1518.  
  1519. -29> 2019-08-06 12:50:37.100 7f70a4b76200 4 rocksdb: [/build/ceph-13.2.4/src/rocksdb/db/db_impl_open.cc:1218] DB pointer 0x27f5000
  1520. -28> 2019-08-06 12:50:37.100 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
  1521. -27> 2019-08-06 12:50:37.180 7f70a4b76200 1 freelist init
  1522. -26> 2019-08-06 12:50:37.240 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc opening allocation metadata
  1523. -25> 2019-08-06 12:50:38.320 7f70a4b76200 1 bluestore(/var/lib/ceph/osd/ceph-37) _open_alloc loaded 1.3 TiB in 937418 extents
  1524. -24> 2019-08-06 12:50:38.328 7f70a4b76200 2 osd.37 0 journal looks like hdd
  1525. -23> 2019-08-06 12:50:38.328 7f70a4b76200 2 osd.37 0 boot
  1526. -22> 2019-08-06 12:50:38.328 7f708c115700 5 bluestore.MempoolThread(0x26e4b08) _tune_cache_size target: 4294967296 heap: 66617344 unmapped: 1040384 mapped: 65576960 old cache_size: 134217728 new cache size: 2804020377
  1527. -21> 2019-08-06 12:50:38.328 7f708c115700 5 bluestore.MempoolThread(0x26e4b08) _trim_shards cache_size: 2804020377 kv_alloc: 134217728 kv_used: 38626570 meta_alloc: 0 meta_used: 45397 data_alloc: 0 data_used: 0
  1528. -20> 2019-08-06 12:50:38.408 7f70a4b76200 0 <cls> /build/ceph-13.2.4/src/cls/hello/cls_hello.cc:296: loading cls_hello
  1529. -19> 2019-08-06 12:50:38.408 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/rgw/cls_rgw.cc:3845: Loaded rgw class!
  1530. -18> 2019-08-06 12:50:38.412 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/log/cls_log.cc:299: Loaded log class!
  1531. -17> 2019-08-06 12:50:38.412 7f70a4b76200 0 _get_class not permitted to load sdk
  1532. -16> 2019-08-06 12:50:38.412 7f70a4b76200 0 _get_class not permitted to load kvs
  1533. -15> 2019-08-06 12:50:38.416 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/refcount/cls_refcount.cc:399: Loaded refcount class!
  1534. -14> 2019-08-06 12:50:38.416 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/statelog/cls_statelog.cc:279: Loaded log class!
  1535. -13> 2019-08-06 12:50:38.420 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/replica_log/cls_replica_log.cc:135: Loaded replica log class!
  1536. -12> 2019-08-06 12:50:38.420 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/version/cls_version.cc:214: Loaded version class!
  1537. -11> 2019-08-06 12:50:38.420 7f70a4b76200 0 _get_class not permitted to load lua
  1538. -10> 2019-08-06 12:50:38.420 7f70a4b76200 0 <cls> /build/ceph-13.2.4/src/cls/cephfs/cls_cephfs.cc:197: loading cephfs
  1539. -9> 2019-08-06 12:50:38.424 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/timeindex/cls_timeindex.cc:242: Loaded timeindex class!
  1540. -8> 2019-08-06 12:50:38.424 7f70a4b76200 1 <cls> /build/ceph-13.2.4/src/cls/user/cls_user.cc:419: Loaded user class!
  1541. -7> 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 432629342416404480, adjusting msgr requires for clients
  1542. -6> 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 432629342416404480 was 8705, adjusting msgr requires for mons
  1543. -5> 2019-08-06 12:50:38.424 7f70a4b76200 0 osd.37 196588 crush map has features 1009090094719844352, adjusting msgr requires for osds
  1544. -4> 2019-08-06 12:50:43.336 7f708c115700 5 bluestore.MempoolThread(0x26e4b08) _tune_cache_size target: 4294967296 heap: 189308928 unmapped: 14221312 mapped: 175087616 old cache_size: 2845415780 new cache size: 2845415830
  1545. -3> 2019-08-06 12:50:43.336 7f708c115700 5 bluestore.MempoolThread(0x26e4b08) _trim_shards cache_size: 2845415830 kv_alloc: 1279094926 kv_used: 129164198 meta_alloc: 1044213902 meta_used: 45344 data_alloc: 522106951 data_used: 0
  1546. -2> 2019-08-06 12:50:46.340 7f70a4b76200 0 osd.37 196588 load_pgs
  1547. -1> 2019-08-06 12:50:46.380 7f70a4b76200 5 osd.37 pg_epoch: 196586 pg[23.fcs6(unlocked)] enter Initial
  1548. 0> 2019-08-06 12:50:46.384 7f70a4b76200 -1 /build/ceph-13.2.4/src/osd/ECUtil.h: In function 'ECUtil::stripe_info_t::stripe_info_t(uint64_t, uint64_t)' thread 7f70a4b76200 time 2019-08-06 12:50:46.383353
  1549. /build/ceph-13.2.4/src/osd/ECUtil.h: 34: FAILED assert(stripe_width % stripe_size == 0)
  1550.  
  1551. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  1552. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7f709beab3c2]
  1553. 2: (()+0x2e5587) [0x7f709beab587]
  1554. 3: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  1555. 4: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  1556. 5: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  1557. 6: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  1558. 7: (OSD::load_pgs()+0x4a9) [0x758339]
  1559. 8: (OSD::init()+0xcd3) [0x7619c3]
  1560. 9: (main()+0x3678) [0x64d6a8]
  1561. 10: (__libc_start_main()+0xf0) [0x7f7099a5b830]
  1562. 11: (_start()+0x29) [0x717389]
  1563. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1564.  
  1565. --- logging levels ---
  1566. 0/ 5 none
  1567. 0/ 1 lockdep
  1568. 0/ 1 context
  1569. 1/ 1 crush
  1570. 1/ 5 mds
  1571. 1/ 5 mds_balancer
  1572. 1/ 5 mds_locker
  1573. 1/ 5 mds_log
  1574. 1/ 5 mds_log_expire
  1575. 1/ 5 mds_migrator
  1576. 0/ 1 buffer
  1577. 0/ 1 timer
  1578. 0/ 1 filer
  1579. 0/ 1 striper
  1580. 0/ 1 objecter
  1581. 0/ 5 rados
  1582. 0/ 5 rbd
  1583. 0/ 5 rbd_mirror
  1584. 0/ 5 rbd_replay
  1585. 0/ 5 journaler
  1586. 0/ 5 objectcacher
  1587. 0/ 5 client
  1588. 1/ 5 osd
  1589. 0/ 5 optracker
  1590. 0/ 5 objclass
  1591. 1/ 3 filestore
  1592. 1/ 3 journal
  1593. 0/ 5 ms
  1594. 1/ 5 mon
  1595. 0/10 monc
  1596. 1/ 5 paxos
  1597. 0/ 5 tp
  1598. 1/ 5 auth
  1599. 1/ 5 crypto
  1600. 1/ 1 finisher
  1601. 1/ 1 reserver
  1602. 1/ 5 heartbeatmap
  1603. 1/ 5 perfcounter
  1604. 1/ 5 rgw
  1605. 1/ 5 rgw_sync
  1606. 1/10 civetweb
  1607. 1/ 5 javaclient
  1608. 1/ 5 asok
  1609. 1/ 1 throttle
  1610. 0/ 0 refs
  1611. 1/ 5 xio
  1612. 1/ 5 compressor
  1613. 1/ 5 bluestore
  1614. 1/ 5 bluefs
  1615. 1/ 3 bdev
  1616. 1/ 5 kstore
  1617. 4/ 5 rocksdb
  1618. 4/ 5 leveldb
  1619. 4/ 5 memdb
  1620. 1/ 5 kinetic
  1621. 1/ 5 fuse
  1622. 1/ 5 mgr
  1623. 1/ 5 mgrc
  1624. 1/ 5 dpdk
  1625. 1/ 5 eventtrace
  1626. 99/99 (syslog threshold)
  1627. -1/-1 (stderr threshold)
  1628. max_recent 10000
  1629. max_new 1000
  1630. log_file /var/log/ceph/ceph-osd.37.log
  1631. --- end dump of recent events ---
  1632. 2019-08-06 12:50:46.420 7f70a4b76200 -1 *** Caught signal (Aborted) **
  1633. in thread 7f70a4b76200 thread_name:ceph-osd
  1634.  
  1635. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  1636. 1: /usr/bin/ceph-osd() [0xcac700]
  1637. 2: (()+0x11390) [0x7f709a53d390]
  1638. 3: (gsignal()+0x38) [0x7f7099a70428]
  1639. 4: (abort()+0x16a) [0x7f7099a7202a]
  1640. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7f709beab510]
  1641. 6: (()+0x2e5587) [0x7f709beab587]
  1642. 7: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  1643. 8: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  1644. 9: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  1645. 10: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  1646. 11: (OSD::load_pgs()+0x4a9) [0x758339]
  1647. 12: (OSD::init()+0xcd3) [0x7619c3]
  1648. 13: (main()+0x3678) [0x64d6a8]
  1649. 14: (__libc_start_main()+0xf0) [0x7f7099a5b830]
  1650. 15: (_start()+0x29) [0x717389]
  1651. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1652.  
  1653. --- begin dump of recent events ---
  1654. 0> 2019-08-06 12:50:46.420 7f70a4b76200 -1 *** Caught signal (Aborted) **
  1655. in thread 7f70a4b76200 thread_name:ceph-osd
  1656.  
  1657. ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)
  1658. 1: /usr/bin/ceph-osd() [0xcac700]
  1659. 2: (()+0x11390) [0x7f709a53d390]
  1660. 3: (gsignal()+0x38) [0x7f7099a70428]
  1661. 4: (abort()+0x16a) [0x7f7099a7202a]
  1662. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7f709beab510]
  1663. 6: (()+0x2e5587) [0x7f709beab587]
  1664. 7: (ECBackend::ECBackend(PGBackend::Listener*, coll_t const&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*, std::shared_ptr<ceph::ErasureCodeInterface>, unsigned long)+0x4de) [0xa4cbbe]
  1665. 8: (PGBackend::build_pg_backend(pg_pool_t const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, PGBackend::Listener*, coll_t, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ObjectStore*, CephContext*)+0x2f9) [0x9474e9]
  1666. 9: (PrimaryLogPG::PrimaryLogPG(OSDService*, std::shared_ptr<OSDMap const>, PGPool const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, spg_t)+0x138) [0x8f96e8]
  1667. 10: (OSD::_make_pg(std::shared_ptr<OSDMap const>, spg_t)+0x11d3) [0x753553]
  1668. 11: (OSD::load_pgs()+0x4a9) [0x758339]
  1669. 12: (OSD::init()+0xcd3) [0x7619c3]
  1670. 13: (main()+0x3678) [0x64d6a8]
  1671. 14: (__libc_start_main()+0xf0) [0x7f7099a5b830]
  1672. 15: (_start()+0x29) [0x717389]
  1673. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1674.  
  1675. --- logging levels ---
  1676. 0/ 5 none
  1677. 0/ 1 lockdep
  1678. 0/ 1 context
  1679. 1/ 1 crush
  1680. 1/ 5 mds
  1681. 1/ 5 mds_balancer
  1682. 1/ 5 mds_locker
  1683. 1/ 5 mds_log
  1684. 1/ 5 mds_log_expire
  1685. 1/ 5 mds_migrator
  1686. 0/ 1 buffer
  1687. 0/ 1 timer
  1688. 0/ 1 filer
  1689. 0/ 1 striper
  1690. 0/ 1 objecter
  1691. 0/ 5 rados
  1692. 0/ 5 rbd
  1693. 0/ 5 rbd_mirror
  1694. 0/ 5 rbd_replay
  1695. 0/ 5 journaler
  1696. 0/ 5 objectcacher
  1697. 0/ 5 client
  1698. 1/ 5 osd
  1699. 0/ 5 optracker
  1700. 0/ 5 objclass
  1701. 1/ 3 filestore
  1702. 1/ 3 journal
  1703. 0/ 5 ms
  1704. 1/ 5 mon
  1705. 0/10 monc
  1706. 1/ 5 paxos
  1707. 0/ 5 tp
  1708. 1/ 5 auth
  1709. 1/ 5 crypto
  1710. 1/ 1 finisher
  1711. 1/ 1 reserver
  1712. 1/ 5 heartbeatmap
  1713. 1/ 5 perfcounter
  1714. 1/ 5 rgw
  1715. 1/ 5 rgw_sync
  1716. 1/10 civetweb
  1717. 1/ 5 javaclient
  1718. 1/ 5 asok
  1719. 1/ 1 throttle
  1720. 0/ 0 refs
  1721. 1/ 5 xio
  1722. 1/ 5 compressor
  1723. 1/ 5 bluestore
  1724. 1/ 5 bluefs
  1725. 1/ 3 bdev
  1726. 1/ 5 kstore
  1727. 4/ 5 rocksdb
  1728. 4/ 5 leveldb
  1729. 4/ 5 memdb
  1730. 1/ 5 kinetic
  1731. 1/ 5 fuse
  1732. 1/ 5 mgr
  1733. 1/ 5 mgrc
  1734. 1/ 5 dpdk
  1735. 1/ 5 eventtrace
  1736. 99/99 (syslog threshold)
  1737. -1/-1 (stderr threshold)
  1738. max_recent 10000
  1739. max_new 1000
  1740. log_file /var/log/ceph/ceph-osd.37.log
  1741. --- end dump of recent events ---
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!