API tools faq
paste
Guest User

osd crash log

a guest
Oct 6th, 2022
248
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 166.80 KB | Software | 0 0
raw download clone embed print report
  1. ****** Manual restart ******
  2. 2022-10-06T23:22:47.240+0200 7f66abc55f00 0 set uid:gid to 167:167 (ceph:ceph)
  3. 2022-10-06T23:22:47.240+0200 7f66abc55f00 0 ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable), process ceph-osd, pid 53883
  4. 2022-10-06T23:22:47.886+0200 7f66abc55f00 0 pidfile_write: ignore empty --pid-file
  5. 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  6. 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  7. 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  8. 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  9. 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  10. 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  11. 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  12. 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  13. 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) close
  14. 2022-10-06T23:22:48.174+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) close
  15. 2022-10-06T23:22:48.427+0200 7f66abc55f00 0 starting osd.16 osd_data /var/lib/ceph/osd/ceph-16 /var/lib/ceph/osd/ceph-16/journal
  16. 2022-10-06T23:22:48.460+0200 7f66a61ab700 -1 PosixStack listen unable to listen on v2:192.168.16.72:7068/0: (98) Address already in use
  17. 2022-10-06T23:22:48.523+0200 7f66abc55f00 0 load: jerasure load: lrc load: isa
  18. 2022-10-06T23:22:48.523+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  19. 2022-10-06T23:22:48.523+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  20. 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  21. 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  22. 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) close
  23. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  24. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  25. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  26. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  27. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  28. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:5.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  29. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:6.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  30. 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  31. 2022-10-06T23:22:48.803+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _mount path /var/lib/ceph/osd/ceph-16
  32. 2022-10-06T23:22:48.804+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  33. 2022-10-06T23:22:48.804+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  34. 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  35. 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  36. 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  37. 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  38. 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  39. 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  40. 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs mount
  41. 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0x1749c00000
  42. 2022-10-06T23:22:48.965+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db set db_paths to db,95020489113 db.slow,95020489113
  43. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  44. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  45. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  46. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  47. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  48. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  49. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  50. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  51. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  52. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  53. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  54. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  55. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  56. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  57. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  58. 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  59. 2022-10-06T23:22:48.965+0200 7f66abc55f00 1 rocksdb: do_open column families: [default]
  60. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: RocksDB version: 6.1.2
  61.  
  62. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  63. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Compile date Aug 9 2022
  64. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: DB SUMMARY
  65.  
  66. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: CURRENT file: CURRENT
  67.  
  68. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: IDENTITY file: IDENTITY
  69.  
  70. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: MANIFEST file: MANIFEST-133203 size: 94952 Bytes
  71.  
  72. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: SST files in db dir, Total Num: 445, files: 010487.sst 010492.sst 010973.sst 011042.sst 011054.sst 011055.sst 011120.sst 029094.sst 030205.sst
  73.  
  74. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: SST files in db.slow dir, Total Num: 345, files: 129892.sst 129917.sst 129918.sst 129919.sst 129920.sst 129926.sst 129927.sst 129928.sst 129929.sst
  75.  
  76. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Write Ahead Log file in db: 133204.log size: 0 ;
  77.  
  78. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.error_if_exists: 0
  79. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.create_if_missing: 0
  80. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.paranoid_checks: 1
  81. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.env: 0x55a5d3418780
  82. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.info_log: 0x55a5d342edc0
  83. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_file_opening_threads: 16
  84. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.statistics: (nil)
  85. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_fsync: 0
  86. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_log_file_size: 0
  87. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_manifest_file_size: 1073741824
  88. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.log_file_time_to_roll: 0
  89. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.keep_log_file_num: 1000
  90. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.recycle_log_file_num: 4
  91. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_fallocate: 1
  92. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_reads: 0
  93. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_writes: 0
  94. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_direct_reads: 0
  95. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  96. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.create_missing_column_families: 0
  97. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.db_log_dir:
  98. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_dir: db
  99. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_cache_numshardbits: 6
  100. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_subcompactions: 1
  101. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_flushes: -1
  102. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.WAL_ttl_seconds: 0
  103. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.WAL_size_limit_MB: 0
  104. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.manifest_preallocation_size: 4194304
  105. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.is_fd_close_on_exec: 1
  106. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.advise_random_on_open: 1
  107. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.db_write_buffer_size: 0
  108. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_manager: 0x55a5d4436fc0
  109. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.access_hint_on_compaction_start: 1
  110. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  111. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  112. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_adaptive_mutex: 0
  113. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.rate_limiter: (nil)
  114. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  115. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_recovery_mode: 0
  116. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_thread_tracking: 0
  117. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_pipelined_write: 0
  118. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  119. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  120. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_thread_max_yield_usec: 100
  121. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  122. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.row_cache: None
  123. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_filter: None
  124. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_recovery: 0
  125. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_ingest_behind: 0
  126. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.preserve_deletes: 0
  127. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.two_write_queues: 0
  128. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.manual_wal_flush: 0
  129. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.atomic_flush: 0
  130. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
  131. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_jobs: 2
  132. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_compactions: 2
  133. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  134. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.writable_file_max_buffer_size: 0
  135. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.delayed_write_rate : 16777216
  136. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_total_wal_size: 0
  137. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  138. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_dump_period_sec: 600
  139. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_persist_period_sec: 600
  140. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_history_buffer_size: 1048576
  141. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_open_files: -1
  142. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bytes_per_sync: 0
  143. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_bytes_per_sync: 0
  144. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_readahead_size: 2097152
  145. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Compression algorithms supported:
  146. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZSTDNotFinalCompression supported: 0
  147. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZSTD supported: 0
  148. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kXpressCompression supported: 0
  149. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kLZ4HCCompression supported: 1
  150. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kLZ4Compression supported: 1
  151. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kBZip2Compression supported: 0
  152. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZlibCompression supported: 1
  153. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kSnappyCompression supported: 1
  154. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Fast CRC32 supported: Supported on x86
  155. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode
  156. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-133203
  157.  
  158. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]:
  159.  
  160. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  161. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  162. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter: None
  163. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter_factory: None
  164. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_factory: SkipListFactory
  165. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_factory: BlockBasedTable
  166. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5d34b04e0)
  167. cache_index_and_filter_blocks: 1
  168. cache_index_and_filter_blocks_with_high_priority: 1
  169. pin_l0_filter_and_index_blocks_in_cache: 0
  170. pin_top_level_index_and_filter: 1
  171. index_type: 0
  172. data_block_index_type: 0
  173. data_block_hash_table_util_ratio: 0.750000
  174. hash_index_allow_collision: 1
  175. checksum: 1
  176. no_block_cache: 0
  177. block_cache: 0x55a5d4416810
  178. block_cache_name: BinnedLRUCache
  179. block_cache_options:
  180. capacity : 1288490188
  181. num_shard_bits : 4
  182. strict_capacity_limit : 0
  183. high_pri_pool_ratio: 0.000
  184. block_cache_compressed: (nil)
  185. persistent_cache: (nil)
  186. block_size: 4096
  187. block_size_deviation: 10
  188. block_restart_interval: 16
  189. index_block_restart_interval: 1
  190. metadata_block_size: 4096
  191. partition_filters: 0
  192. use_delta_encoding: 1
  193. filter_policy: rocksdb.BuiltinBloomFilter
  194. whole_key_filtering: 1
  195. verify_compression: 0
  196. read_amp_bytes_per_bit: 0
  197. format_version: 2
  198. enable_index_compression: 1
  199. block_align: 0
  200.  
  201. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_size: 268435456
  202. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number: 4
  203. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression: NoCompression
  204. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression: Disabled
  205. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.prefix_extractor: nullptr
  206. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  207. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.num_levels: 7
  208. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  209. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  210. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
  211. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.level: 32767
  212. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
  213. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
  214. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
  215. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.enabled: false
  216. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.window_bits: -14
  217. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.level: 32767
  218. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.strategy: 0
  219. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  220. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  221. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.enabled: false
  222. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  223. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  224. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_stop_writes_trigger: 36
  225. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_base: 67108864
  226. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_multiplier: 1
  227. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  228. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  229. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  230. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  231. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  232. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  233. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  234. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  235. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  236. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  237. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  238. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_compaction_bytes: 1677721600
  239. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.arena_block_size: 33554432
  240. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  241. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  242. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  243. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.disable_auto_compactions: 0
  244. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  245. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
  246. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  247. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  248. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  249. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  250. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  251. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  252. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  253. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  254. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_properties_collectors:
  255. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_support: 0
  256. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_num_locks: 10000
  257. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  258. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_whole_key_filtering: 0
  259. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_huge_page_size: 0
  260. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bloom_locality: 0
  261. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_successive_merges: 0
  262. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.optimize_filters_for_hits: 0
  263. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.paranoid_file_checks: 0
  264. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.force_consistency_checks: 0
  265. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.report_bg_io_stats: 0
  266. 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.ttl: 0
  267. 2022-10-06T23:22:48.989+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-133203 succeeded,manifest_file_number is 133203, next_file_number is 133205, last_sequence is 3305434714, log_number is 133202,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0
  268.  
  269. 2022-10-06T23:22:48.989+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 133202
  270.  
  271. 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091368991402, "job": 1, "event": "recovery_started", "log_files": [133204]}
  272. 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #133204 mode 0
  273. 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091368991420, "job": 1, "event": "recovery_finished"}
  274. 2022-10-06T23:22:48.991+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2
  275. 2022-10-06T23:22:48.996+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta old nid_max 63595586
  276. 2022-10-06T23:22:48.996+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta old blobid_max 122880
  277. 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta freelist_type bitmap
  278. 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta ondisk_format 4 compat_ondisk_format 3
  279. 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta min_alloc_size 0x4000
  280. 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 freelist init
  281. 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 freelist _init_from_label
  282. 2022-10-06T23:22:49.010+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_alloc opening allocation metadata
  283. 2022-10-06T23:22:49.044+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_alloc loaded 81 GiB in 21981 extents available 593 MiB
  284. 2022-10-06T23:22:49.044+0200 7f66abc55f00 4 rocksdb: [db/db_impl.cc:390] Shutdown: canceling all background work
  285. 2022-10-06T23:22:49.049+0200 7f66abc55f00 4 rocksdb: [db/db_impl.cc:563] Shutdown complete
  286. 2022-10-06T23:22:49.049+0200 7f66abc55f00 1 bluefs umount
  287. 2022-10-06T23:22:49.050+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) close
  288. 2022-10-06T23:22:49.096+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  289. 2022-10-06T23:22:49.096+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  290. 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  291. 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  292. 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs mount
  293. 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0x1749c00000
  294. 2022-10-06T23:22:49.210+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db set db_paths to db,95020489113 db.slow,95020489113
  295. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  296. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  297. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  298. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  299. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  300. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  301. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  302. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  303. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  304. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  305. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  306. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  307. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  308. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  309. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  310. 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  311. 2022-10-06T23:22:49.211+0200 7f66abc55f00 1 rocksdb: do_open column families: [default]
  312. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: RocksDB version: 6.1.2
  313.  
  314. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  315. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Compile date Aug 9 2022
  316. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: DB SUMMARY
  317.  
  318. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: CURRENT file: CURRENT
  319.  
  320. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: IDENTITY file: IDENTITY
  321.  
  322. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: MANIFEST file: MANIFEST-133203 size: 94952 Bytes
  323.  
  324. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: SST files in db dir, Total Num: 445, files: 010487.sst 010492.sst 010973.sst 011042.sst 011054.sst 011055.sst 011120.sst 029094.sst 030205.sst
  325.  
  326. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: SST files in db.slow dir, Total Num: 345, files: 129892.sst 129917.sst 129918.sst 129919.sst 129920.sst 129926.sst 129927.sst 129928.sst 129929.sst
  327.  
  328. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Write Ahead Log file in db: 133204.log size: 0 ;
  329.  
  330. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.error_if_exists: 0
  331. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.create_if_missing: 0
  332. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.paranoid_checks: 1
  333. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.env: 0x55a5d43f6000
  334. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.info_log: 0x55a5d342f960
  335. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_file_opening_threads: 16
  336. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.statistics: (nil)
  337. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_fsync: 0
  338. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_log_file_size: 0
  339. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_manifest_file_size: 1073741824
  340. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.log_file_time_to_roll: 0
  341. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.keep_log_file_num: 1000
  342. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.recycle_log_file_num: 4
  343. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_fallocate: 1
  344. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_reads: 0
  345. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_writes: 0
  346. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_direct_reads: 0
  347. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  348. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.create_missing_column_families: 0
  349. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.db_log_dir:
  350. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_dir: db
  351. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_cache_numshardbits: 6
  352. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_subcompactions: 1
  353. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_flushes: -1
  354. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.WAL_ttl_seconds: 0
  355. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.WAL_size_limit_MB: 0
  356. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.manifest_preallocation_size: 4194304
  357. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.is_fd_close_on_exec: 1
  358. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.advise_random_on_open: 1
  359. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.db_write_buffer_size: 0
  360. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_manager: 0x55a5d43b45d0
  361. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.access_hint_on_compaction_start: 1
  362. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  363. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  364. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_adaptive_mutex: 0
  365. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.rate_limiter: (nil)
  366. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  367. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_recovery_mode: 0
  368. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_thread_tracking: 0
  369. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_pipelined_write: 0
  370. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  371. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  372. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_thread_max_yield_usec: 100
  373. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  374. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.row_cache: None
  375. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_filter: None
  376. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_recovery: 0
  377. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_ingest_behind: 0
  378. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.preserve_deletes: 0
  379. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.two_write_queues: 0
  380. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.manual_wal_flush: 0
  381. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.atomic_flush: 0
  382. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
  383. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_jobs: 2
  384. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_compactions: 2
  385. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  386. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.writable_file_max_buffer_size: 0
  387. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.delayed_write_rate : 16777216
  388. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_total_wal_size: 0
  389. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  390. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_dump_period_sec: 600
  391. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_persist_period_sec: 600
  392. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_history_buffer_size: 1048576
  393. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_open_files: -1
  394. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bytes_per_sync: 0
  395. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_bytes_per_sync: 0
  396. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_readahead_size: 2097152
  397. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Compression algorithms supported:
  398. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZSTDNotFinalCompression supported: 0
  399. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZSTD supported: 0
  400. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kXpressCompression supported: 0
  401. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kLZ4HCCompression supported: 1
  402. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kLZ4Compression supported: 1
  403. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kBZip2Compression supported: 0
  404. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZlibCompression supported: 1
  405. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kSnappyCompression supported: 1
  406. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Fast CRC32 supported: Supported on x86
  407. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-133203
  408.  
  409. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]:
  410.  
  411. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  412. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  413. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter: None
  414. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter_factory: None
  415. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_factory: SkipListFactory
  416. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_factory: BlockBasedTable
  417. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5d33c1818)
  418. cache_index_and_filter_blocks: 1
  419. cache_index_and_filter_blocks_with_high_priority: 1
  420. pin_l0_filter_and_index_blocks_in_cache: 0
  421. pin_top_level_index_and_filter: 1
  422. index_type: 0
  423. data_block_index_type: 0
  424. data_block_hash_table_util_ratio: 0.750000
  425. hash_index_allow_collision: 1
  426. checksum: 1
  427. no_block_cache: 0
  428. block_cache: 0x55a5d3409310
  429. block_cache_name: BinnedLRUCache
  430. block_cache_options:
  431. capacity : 1288490188
  432. num_shard_bits : 4
  433. strict_capacity_limit : 0
  434. high_pri_pool_ratio: 0.000
  435. block_cache_compressed: (nil)
  436. persistent_cache: (nil)
  437. block_size: 4096
  438. block_size_deviation: 10
  439. block_restart_interval: 16
  440. index_block_restart_interval: 1
  441. metadata_block_size: 4096
  442. partition_filters: 0
  443. use_delta_encoding: 1
  444. filter_policy: rocksdb.BuiltinBloomFilter
  445. whole_key_filtering: 1
  446. verify_compression: 0
  447. read_amp_bytes_per_bit: 0
  448. format_version: 2
  449. enable_index_compression: 1
  450. block_align: 0
  451.  
  452. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_size: 268435456
  453. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number: 4
  454. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression: NoCompression
  455. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression: Disabled
  456. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.prefix_extractor: nullptr
  457. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  458. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.num_levels: 7
  459. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  460. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  461. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
  462. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.level: 32767
  463. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
  464. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
  465. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
  466. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.enabled: false
  467. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.window_bits: -14
  468. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.level: 32767
  469. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.strategy: 0
  470. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  471. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  472. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.enabled: false
  473. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  474. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  475. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_stop_writes_trigger: 36
  476. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_base: 67108864
  477. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_multiplier: 1
  478. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  479. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  480. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  481. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  482. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  483. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  484. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  485. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  486. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  487. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  488. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  489. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_compaction_bytes: 1677721600
  490. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.arena_block_size: 33554432
  491. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  492. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  493. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  494. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.disable_auto_compactions: 0
  495. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  496. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
  497. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  498. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  499. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  500. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  501. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  502. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  503. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  504. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  505. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_properties_collectors:
  506. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_support: 0
  507. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_num_locks: 10000
  508. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  509. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_whole_key_filtering: 0
  510. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_huge_page_size: 0
  511. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bloom_locality: 0
  512. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_successive_merges: 0
  513. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.optimize_filters_for_hits: 0
  514. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.paranoid_file_checks: 0
  515. 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.force_consistency_checks: 0
  516. 2022-10-06T23:22:49.212+0200 7f66abc55f00 4 rocksdb: Options.report_bg_io_stats: 0
  517. 2022-10-06T23:22:49.212+0200 7f66abc55f00 4 rocksdb: Options.ttl: 0
  518. 2022-10-06T23:22:49.216+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-133203 succeeded,manifest_file_number is 133203, next_file_number is 133205, last_sequence is 3305434714, log_number is 133202,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0
  519.  
  520. 2022-10-06T23:22:49.216+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 133202
  521.  
  522. 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369218098, "job": 1, "event": "recovery_started", "log_files": [133204]}
  523. 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #133204 mode 0
  524. 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3036] Creating manifest 133206
  525.  
  526. 2022-10-06T23:22:49.219+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369220260, "job": 1, "event": "recovery_finished"}
  527. 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: [db/compaction_job.cc:1645] [default] [JOB 3] Compacting 1@3 files to L4, score 1.09
  528. 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: [db/compaction_job.cc:1649] [default] Compaction start summary: Base version 2 Base level 3, inputs: [133096(64MB)]
  529.  
  530. 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369222004, "job": 3, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L3": [133096], "score": 1.09115, "input_data_size": 68055858}
  531. 2022-10-06T23:22:49.221+0200 7f66abc55f00 4 rocksdb: DB pointer 0x55a5d410c400
  532. 2022-10-06T23:22:49.221+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2
  533. 2022-10-06T23:22:49.221+0200 7f6693916700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS -------
  534. 2022-10-06T23:22:49.222+0200 7f6693916700 4 rocksdb: [db/db_impl.cc:778]
  535. ** DB Stats **
  536. Uptime(secs): 0.0 total, 0.0 interval
  537. Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s
  538. Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
  539. Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
  540. Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s
  541. Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
  542. Interval stall: 00:00:0.000 H:M:S, 0.0 percent
  543.  
  544. ** Compaction Stats [default] **
  545. Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  546. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  547. L0 1/0 61.55 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  548. L1 5/0 256.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  549. L2 29/0 1.67 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  550. L3 180/1 9.85 GB 1.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  551. L4 575/0 34.66 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  552. Sum 790/1 46.49 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  553. Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  554.  
  555. ** Compaction Stats [default] **
  556. Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  557. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  558. Uptime(secs): 0.0 total, 0.0 interval
  559. Flush(GB): cumulative 0.000, interval 0.000
  560. AddFile(GB): cumulative 0.000, interval 0.000
  561. AddFile(Total Files): cumulative 0, interval 0
  562. AddFile(L0 Files): cumulative 0, interval 0
  563. AddFile(Keys): cumulative 0, interval 0
  564. Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  565. Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  566. Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
  567.  
  568. ** File Read Latency Histogram By Level [default] **
  569.  
  570. ** Compaction Stats [default] **
  571. Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  572. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  573. L0 1/0 61.55 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  574. L1 5/0 256.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  575. L2 29/0 1.67 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  576. L3 180/1 9.85 GB 1.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  577. L4 575/0 34.66 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  578. Sum 790/1 46.49 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  579. Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  580.  
  581. ** Compaction Stats [default] **
  582. Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  583. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  584. Uptime(secs): 0.0 total, 0.0 interval
  585. Flush(GB): cumulative 0.000, interval 0.000
  586. AddFile(GB): cumulative 0.000, interval 0.000
  587. AddFile(Total Files): cumulative 0, interval 0
  588. AddFile(L0 Files): cumulative 0, interval 0
  589. AddFile(Keys): cumulative 0, interval 0
  590. Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  591. Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  592. Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
  593.  
  594. ** File Read Latency Histogram By Level [default] **
  595.  
  596. 2022-10-06T23:22:49.223+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _upgrade_super from 4, latest 4
  597. 2022-10-06T23:22:49.223+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _upgrade_super done
  598. 2022-10-06T23:22:49.267+0200 7f669d129700 1 bluefs _allocate unable to allocate 0x1001a9 on bdev 2, free 0xffffffffffffffff; fallback to slow device expander
  599. 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluestore(/var/lib/ceph/osd/ceph-16) allocate_bluefs_freespace failed to allocate on 0x110000 min_size 0x110000 > allocated total 0x30000 bluefs_shared_alloc_size 0x10000 allocated 0x30000 available 0x 25134000
  600. 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluefs _allocate failed to expand slow device to fit +0x1001a9
  601. 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x1001a9
  602. 2022-10-06T23:22:49.275+0200 7f669d129700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.17/rpm/el8/BUILD/ceph-15.2.17/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f669d129700 time 2022-10-06T23:22:49.268150+0200
  603. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.17/rpm/el8/BUILD/ceph-15.2.17/src/os/bluestore/BlueFS.cc: 2931: ceph_abort_msg("bluefs enospc")
  604.  
  605. ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)
  606. 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x55a5c790cb32]
  607. 2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1545) [0x55a5c7f95215]
  608. 3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x110) [0x55a5c7f95600]
  609. 4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x55a5c7fa6952]
  610. 5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x1d9) [0x55a5c7fc1d39]
  611. 6: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x560) [0x55a5c859c700]
  612. 7: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2b0) [0x55a5c859d530]
  613. 8: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x55a5c860d486]
  614. 9: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x55a5c860ddac]
  615. 10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x55a5c860e48c]
  616. 11: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x55a5c860e51d]
  617. 12: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2a8) [0x55a5c8611768]
  618. 13: (rocksdb::CompactionJob::ProcessKeyValueCompaction(rocksdb::CompactionJob::SubcompactionState*)+0x6df) [0x55a5c85de28f]
  619. 14: (rocksdb::CompactionJob::Run()+0x264) [0x55a5c85df424]
  620. 15: (rocksdb::DBImpl::BackgroundCompaction(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xd0e) [0x55a5c845aaee]
  621. 16: (rocksdb::DBImpl::BackgroundCallCompaction(rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xc6) [0x55a5c8461086]
  622. 17: (rocksdb::DBImpl::BGWorkCompaction(void*)+0x3a) [0x55a5c846155a]
  623. 18: (rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24a) [0x55a5c8619f0a]
  624. 19: (rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x5d) [0x55a5c861a08d]
  625. 20: (()+0xc2ba3) [0x7f66a8ff9ba3]
  626. 21: (()+0x81ca) [0x7f66a99a61ca]
  627. 22: (clone()+0x43) [0x7f66a8610dd3]
  628.  
  629. 2022-10-06T23:22:49.283+0200 7f669d129700 -1 *** Caught signal (Aborted) **
  630. in thread 7f669d129700 thread_name:rocksdb:low0
  631.  
  632. ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)
  633. 1: (()+0x12ce0) [0x7f66a99b0ce0]
  634. 2: (gsignal()+0x10f) [0x7f66a8625a9f]
  635. 3: (abort()+0x127) [0x7f66a85f8e05]
  636. 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x55a5c790cc03]
  637. 5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1545) [0x55a5c7f95215]
  638. 6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x110) [0x55a5c7f95600]
  639. 7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x55a5c7fa6952]
  640. 8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x1d9) [0x55a5c7fc1d39]
  641. 9: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x560) [0x55a5c859c700]
  642. 10: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2b0) [0x55a5c859d530]
  643. 11: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x55a5c860d486]
  644. 12: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x55a5c860ddac]
  645. 13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x55a5c860e48c]
  646. 14: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x55a5c860e51d]
  647. 15: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2a8) [0x55a5c8611768]
  648. 16: (rocksdb::CompactionJob::ProcessKeyValueCompaction(rocksdb::CompactionJob::SubcompactionState*)+0x6df) [0x55a5c85de28f]
  649. 17: (rocksdb::CompactionJob::Run()+0x264) [0x55a5c85df424]
  650. 18: (rocksdb::DBImpl::BackgroundCompaction(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xd0e) [0x55a5c845aaee]
  651. 19: (rocksdb::DBImpl::BackgroundCallCompaction(rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xc6) [0x55a5c8461086]
  652. 20: (rocksdb::DBImpl::BGWorkCompaction(void*)+0x3a) [0x55a5c846155a]
  653. 21: (rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24a) [0x55a5c8619f0a]
  654. 22: (rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x5d) [0x55a5c861a08d]
  655. 23: (()+0xc2ba3) [0x7f66a8ff9ba3]
  656. 24: (()+0x81ca) [0x7f66a99a61ca]
  657. 25: (clone()+0x43) [0x7f66a8610dd3]
  658. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  659.  
  660. --- begin dump of recent events ---
  661. -764> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command assert hook 0x55a5d33ba5d0
  662. -763> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command abort hook 0x55a5d33ba5d0
  663. -762> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command leak_some_memory hook 0x55a5d33ba5d0
  664. -761> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perfcounters_dump hook 0x55a5d33ba5d0
  665. -760> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command 1 hook 0x55a5d33ba5d0
  666. -759> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perf dump hook 0x55a5d33ba5d0
  667. -758> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perfcounters_schema hook 0x55a5d33ba5d0
  668. -757> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perf histogram dump hook 0x55a5d33ba5d0
  669. -756> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command 2 hook 0x55a5d33ba5d0
  670. -755> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perf schema hook 0x55a5d33ba5d0
  671. -754> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perf histogram schema hook 0x55a5d33ba5d0
  672. -753> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command perf reset hook 0x55a5d33ba5d0
  673. -752> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config show hook 0x55a5d33ba5d0
  674. -751> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config help hook 0x55a5d33ba5d0
  675. -750> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config set hook 0x55a5d33ba5d0
  676. -749> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config unset hook 0x55a5d33ba5d0
  677. -748> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config get hook 0x55a5d33ba5d0
  678. -747> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config diff hook 0x55a5d33ba5d0
  679. -746> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command config diff get hook 0x55a5d33ba5d0
  680. -745> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command injectargs hook 0x55a5d33ba5d0
  681. -744> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command log flush hook 0x55a5d33ba5d0
  682. -743> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command log dump hook 0x55a5d33ba5d0
  683. -742> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command log reopen hook 0x55a5d33ba5d0
  684. -741> 2022-10-06T23:22:47.210+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command dump_mempools hook 0x55a5d4046068
  685. -740> 2022-10-06T23:22:47.216+0200 7f66abc55f00 10 monclient: get_monmap_and_config
  686. -739> 2022-10-06T23:22:47.216+0200 7f66abc55f00 10 monclient: build_initial_monmap
  687. -738> 2022-10-06T23:22:47.216+0200 7f66abc55f00 10 monclient: monmap:
  688. epoch 0
  689. fsid e4ece518-f2cb-4708-b00f-b6bf511e91d9
  690. last_changed 2022-10-06T23:22:47.217408+0200
  691. created 2022-10-06T23:22:47.217408+0200
  692. min_mon_release 0 (unknown)
  693. 0: v1:192.168.32.65:6789/0 mon.ceph-01
  694. 1: v1:192.168.32.66:6789/0 mon.ceph-02
  695. 2: v1:192.168.32.67:6789/0 mon.ceph-03
  696. 3: v1:192.168.32.68:6789/0 mon.ceph-25
  697. 4: v1:192.168.32.69:6789/0 mon.ceph-26
  698.  
  699. -737> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding auth protocol: cephx
  700. -736> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding auth protocol: cephx
  701. -735> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding auth protocol: cephx
  702. -734> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding auth protocol: none
  703. -733> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  704. -732> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  705. -731> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  706. -730> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  707. -729> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  708. -728> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  709. -727> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  710. -726> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  711. -725> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  712. -724> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  713. -723> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: crc
  714. -722> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x55a5d345e940) adding con mode: secure
  715. -721> 2022-10-06T23:22:47.217+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  716. -720> 2022-10-06T23:22:47.217+0200 7f66abc55f00 10 monclient: init
  717. -719> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding auth protocol: cephx
  718. -718> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding auth protocol: cephx
  719. -717> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding auth protocol: cephx
  720. -716> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding auth protocol: none
  721. -715> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  722. -714> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  723. -713> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  724. -712> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  725. -711> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  726. -710> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  727. -709> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  728. -708> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  729. -707> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  730. -706> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  731. -705> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: crc
  732. -704> 2022-10-06T23:22:47.217+0200 7f66abc55f00 5 AuthRegistry(0x7ffe31b736e0) adding con mode: secure
  733. -703> 2022-10-06T23:22:47.217+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  734. -702> 2022-10-06T23:22:47.217+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  735. -701> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient: _reopen_session rank -1
  736. -700> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient: _add_conns ranks=[1,3,2,0,4]
  737. -699> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient(hunting): picked mon.ceph-02 con 0x55a5d33bf000 addr v1:192.168.32.66:6789/0
  738. -698> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient(hunting): picked mon.ceph-25 con 0x55a5d33bf400 addr v1:192.168.32.68:6789/0
  739. -697> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient(hunting): picked mon.ceph-03 con 0x55a5d33bf800 addr v1:192.168.32.67:6789/0
  740. -696> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient(hunting): _renew_subs
  741. -695> 2022-10-06T23:22:47.218+0200 7f66abc55f00 10 monclient(hunting): authenticate will time out at 2022-10-06T23:27:47.218596+0200
  742. -694> 2022-10-06T23:22:47.237+0200 7f66a49a8700 10 monclient(hunting): _init_auth method 2
  743. -693> 2022-10-06T23:22:47.237+0200 7f66a49a8700 10 monclient(hunting): _init_auth creating new auth
  744. -692> 2022-10-06T23:22:47.237+0200 7f66a49a8700 10 monclient(hunting): my global_id is 144851021
  745. -691> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): _init_auth method 2
  746. -690> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): _init_auth creating new auth
  747. -689> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): my global_id is 144821483
  748. -688> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _finish_hunting 0
  749. -687> 2022-10-06T23:22:47.238+0200 7f66a49a8700 1 monclient: found mon.ceph-02
  750. -686> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _send_mon_message to mon.ceph-02 at v1:192.168.32.66:6789/0
  751. -685> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _finish_auth 0
  752. -684> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2022-10-06T23:22:17.238677+0200)
  753. -683> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _send_mon_message to mon.ceph-02 at v1:192.168.32.66:6789/0
  754. -682> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: handle_monmap mon_map magic: 0 v1
  755. -681> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: got monmap 7 from mon.ceph-02 (according to old e7)
  756. -680> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: dump:
  757. epoch 7
  758. fsid e4ece518-f2cb-4708-b00f-b6bf511e91d9
  759. last_changed 2022-10-05T18:01:16.230052+0200
  760. created 2019-03-14T22:18:15.088212+0100
  761. min_mon_release 15 (octopus)
  762. 0: [v2:192.168.32.65:3300/0,v1:192.168.32.65:6789/0] mon.ceph-01
  763. 1: [v2:192.168.32.66:3300/0,v1:192.168.32.66:6789/0] mon.ceph-02
  764. 2: [v2:192.168.32.67:3300/0,v1:192.168.32.67:6789/0] mon.ceph-03
  765. 3: [v2:192.168.32.68:3300/0,v1:192.168.32.68:6789/0] mon.ceph-25
  766. 4: [v2:192.168.32.69:3300/0,v1:192.168.32.69:6789/0] mon.ceph-26
  767.  
  768. -679> 2022-10-06T23:22:47.238+0200 7f66a49a8700 1 monclient: mon.ceph-02 has (v2) addrs [v2:192.168.32.66:3300/0,v1:192.168.32.66:6789/0] but i'm connected to v1:192.168.32.66:6789/0, reconnecting
  769. -678> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _reopen_session rank -1
  770. -677> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient: _add_conns ranks=[4,0,3,1,2]
  771. -676> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): picked mon.ceph-26 con 0x55a5d33bfc00 addr [v2:192.168.32.69:3300/0,v1:192.168.32.69:6789/0]
  772. -675> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): picked mon.ceph-01 con 0x55a5d40f8000 addr [v2:192.168.32.65:3300/0,v1:192.168.32.65:6789/0]
  773. -674> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): picked mon.ceph-25 con 0x55a5d40f8400 addr [v2:192.168.32.68:3300/0,v1:192.168.32.68:6789/0]
  774. -673> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): start opening mon connection
  775. -672> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): start opening mon connection
  776. -671> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): start opening mon connection
  777. -670> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): _renew_subs
  778. -669> 2022-10-06T23:22:47.238+0200 7f66a49a8700 10 monclient(hunting): _finish_auth 0
  779. -668> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): get_auth_request con 0x55a5d40f8400 auth_method 0
  780. -667> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
  781. -666> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): _init_auth method 2
  782. -665> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): _init_auth already have auth, reseting
  783. -664> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): get_auth_request con 0x55a5d33bfc00 auth_method 0
  784. -663> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
  785. -662> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): _init_auth method 2
  786. -661> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): _init_auth already have auth, reseting
  787. -660> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): get_auth_request con 0x55a5d40f8000 auth_method 0
  788. -659> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
  789. -658> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): _init_auth method 2
  790. -657> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): _init_auth already have auth, reseting
  791. -656> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): handle_auth_reply_more payload 9
  792. -655> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): handle_auth_reply_more payload_len 9
  793. -654> 2022-10-06T23:22:47.238+0200 7f66a59aa700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes
  794. -653> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): handle_auth_reply_more payload 9
  795. -652> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): handle_auth_reply_more payload_len 9
  796. -651> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes
  797. -650> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): handle_auth_reply_more payload 9
  798. -649> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): handle_auth_reply_more payload_len 9
  799. -648> 2022-10-06T23:22:47.238+0200 7f66a51a9700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes
  800. -647> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient(hunting): handle_auth_done global_id 144851021 payload 293
  801. -646> 2022-10-06T23:22:47.238+0200 7f66a61ab700 10 monclient: _finish_hunting 0
  802. -645> 2022-10-06T23:22:47.238+0200 7f66a61ab700 1 monclient: found mon.ceph-25
  803. -644> 2022-10-06T23:22:47.239+0200 7f66a61ab700 10 monclient: _send_mon_message to mon.ceph-25 at v2:192.168.32.68:3300/0
  804. -643> 2022-10-06T23:22:47.239+0200 7f66a61ab700 10 monclient: _finish_auth 0
  805. -642> 2022-10-06T23:22:47.239+0200 7f66a61ab700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2022-10-06T23:22:17.239449+0200)
  806. -641> 2022-10-06T23:22:47.239+0200 7f66a61ab700 10 monclient: _send_mon_message to mon.ceph-25 at v2:192.168.32.68:3300/0
  807. -640> 2022-10-06T23:22:47.239+0200 7f66abc55f00 5 monclient: authenticate success, global_id 144851021
  808. -639> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: handle_monmap mon_map magic: 0 v1
  809. -638> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: got monmap 7 from mon.ceph-25 (according to old e7)
  810. -637> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: dump:
  811. epoch 7
  812. fsid e4ece518-f2cb-4708-b00f-b6bf511e91d9
  813. last_changed 2022-10-05T18:01:16.230052+0200
  814. created 2019-03-14T22:18:15.088212+0100
  815. min_mon_release 15 (octopus)
  816. 0: [v2:192.168.32.65:3300/0,v1:192.168.32.65:6789/0] mon.ceph-01
  817. 1: [v2:192.168.32.66:3300/0,v1:192.168.32.66:6789/0] mon.ceph-02
  818. 2: [v2:192.168.32.67:3300/0,v1:192.168.32.67:6789/0] mon.ceph-03
  819. 3: [v2:192.168.32.68:3300/0,v1:192.168.32.68:6789/0] mon.ceph-25
  820. 4: [v2:192.168.32.69:3300/0,v1:192.168.32.69:6789/0] mon.ceph-26
  821.  
  822. -636> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: handle_config config(21 keys) v1
  823. -635> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: handle_monmap mon_map magic: 0 v1
  824. -634> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: got monmap 7 from mon.ceph-25 (according to old e7)
  825. -633> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: dump:
  826. epoch 7
  827. fsid e4ece518-f2cb-4708-b00f-b6bf511e91d9
  828. last_changed 2022-10-05T18:01:16.230052+0200
  829. created 2019-03-14T22:18:15.088212+0100
  830. min_mon_release 15 (octopus)
  831. 0: [v2:192.168.32.65:3300/0,v1:192.168.32.65:6789/0] mon.ceph-01
  832. 1: [v2:192.168.32.66:3300/0,v1:192.168.32.66:6789/0] mon.ceph-02
  833. 2: [v2:192.168.32.67:3300/0,v1:192.168.32.67:6789/0] mon.ceph-03
  834. 3: [v2:192.168.32.68:3300/0,v1:192.168.32.68:6789/0] mon.ceph-25
  835. 4: [v2:192.168.32.69:3300/0,v1:192.168.32.69:6789/0] mon.ceph-26
  836.  
  837. -632> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 monclient: get_monmap_and_config success
  838. -631> 2022-10-06T23:22:47.239+0200 7f66abc55f00 4 set_mon_vals no callback set
  839. -630> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals bluefs_buffered_io = true
  840. -629> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: _finish_auth 0
  841. -628> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals bluestore_compression_min_blob_size_ssd = 65536
  842. -627> 2022-10-06T23:22:47.239+0200 7f66a49a8700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2022-10-06T23:22:17.240082+0200)
  843. -626> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals mds_beacon_grace = 600.000000
  844. -625> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals mon_mgr_beacon_grace = 7200
  845. -624> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals mon_warn_on_pool_pg_num_not_power_of_two = false
  846. -623> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_deep_scrub_interval = 1209600.000000
  847. -622> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_fast_shutdown = false
  848. -621> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_map_message_max_bytes = 16384
  849. -620> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_max_backfills = 3
  850. -619> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_memory_cache_min = 1073741824
  851. -618> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_memory_target = 2147483648
  852. -617> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_pool_default_pg_autoscale_mode = off
  853. -616> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_recovery_max_active = 8
  854. -615> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_recovery_sleep = 0.050000
  855. -614> 2022-10-06T23:22:47.239+0200 7f66abc55f00 10 set_mon_vals osd_scrub_max_interval = 1209600.000000
  856. -613> 2022-10-06T23:22:47.240+0200 7f66abc55f00 10 monclient: shutdown
  857. -612> 2022-10-06T23:22:47.240+0200 7f66abc55f00 0 set uid:gid to 167:167 (ceph:ceph)
  858. -611> 2022-10-06T23:22:47.240+0200 7f66abc55f00 0 ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable), process ceph-osd, pid 53883
  859. -610> 2022-10-06T23:22:47.886+0200 7f66abc55f00 0 pidfile_write: ignore empty --pid-file
  860. -609> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) init /var/run/ceph/ceph-osd.16.asok
  861. -608> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) bind_and_listen /var/run/ceph/ceph-osd.16.asok
  862. -607> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command 0 hook 0x55a5d33c12e8
  863. -606> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command version hook 0x55a5d33c12e8
  864. -605> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command git_version hook 0x55a5d33c12e8
  865. -604> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command help hook 0x55a5d33ba590
  866. -603> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command get_command_descriptions hook 0x55a5d33ba5a0
  867. -602> 2022-10-06T23:22:47.886+0200 7f66a41a7700 5 asok(0x55a5d3474000) entry start
  868. -601> 2022-10-06T23:22:47.886+0200 7f66abc55f00 5 object store type is bluestore
  869. -600> 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  870. -599> 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  871. -598> 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  872. -597> 2022-10-06T23:22:47.889+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  873. -596> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore bluefs available hook 0x55a5d33ba110
  874. -595> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs stats hook 0x55a5d33ba110
  875. -594> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs debug_inject_read_zeros hook 0x55a5d33ba110
  876. -593> 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  877. -592> 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  878. -591> 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  879. -590> 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  880. -589> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluefs debug_inject_read_zeros
  881. -588> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluefs stats
  882. -587> 2022-10-06T23:22:47.890+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluestore bluefs available
  883. -586> 2022-10-06T23:22:47.890+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) close
  884. -585> 2022-10-06T23:22:48.174+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) close
  885. -584> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding auth protocol: cephx
  886. -583> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding auth protocol: cephx
  887. -582> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding auth protocol: cephx
  888. -581> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding auth protocol: none
  889. -580> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  890. -579> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  891. -578> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  892. -577> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  893. -576> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  894. -575> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  895. -574> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  896. -573> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  897. -572> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  898. -571> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  899. -570> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: crc
  900. -569> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3460940) adding con mode: secure
  901. -568> 2022-10-06T23:22:48.426+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  902. -567> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding auth protocol: cephx
  903. -566> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding auth protocol: cephx
  904. -565> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding auth protocol: cephx
  905. -564> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding auth protocol: none
  906. -563> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  907. -562> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  908. -561> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  909. -560> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  910. -559> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  911. -558> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  912. -557> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  913. -556> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  914. -555> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  915. -554> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  916. -553> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: crc
  917. -552> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d3461140) adding con mode: secure
  918. -551> 2022-10-06T23:22:48.426+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  919. -550> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding auth protocol: cephx
  920. -549> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding auth protocol: cephx
  921. -548> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding auth protocol: cephx
  922. -547> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding auth protocol: none
  923. -546> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  924. -545> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  925. -544> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  926. -543> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  927. -542> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  928. -541> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  929. -540> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  930. -539> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  931. -538> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  932. -537> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  933. -536> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: crc
  934. -535> 2022-10-06T23:22:48.426+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4134140) adding con mode: secure
  935. -534> 2022-10-06T23:22:48.427+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  936. -533> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding auth protocol: cephx
  937. -532> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding auth protocol: cephx
  938. -531> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding auth protocol: cephx
  939. -530> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding auth protocol: none
  940. -529> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  941. -528> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  942. -527> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  943. -526> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  944. -525> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  945. -524> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  946. -523> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  947. -522> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  948. -521> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  949. -520> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  950. -519> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: crc
  951. -518> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4135140) adding con mode: secure
  952. -517> 2022-10-06T23:22:48.427+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  953. -516> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding auth protocol: cephx
  954. -515> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding auth protocol: cephx
  955. -514> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding auth protocol: cephx
  956. -513> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding auth protocol: none
  957. -512> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  958. -511> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  959. -510> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  960. -509> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  961. -508> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  962. -507> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  963. -506> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  964. -505> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  965. -504> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  966. -503> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  967. -502> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: crc
  968. -501> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4136140) adding con mode: secure
  969. -500> 2022-10-06T23:22:48.427+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  970. -499> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding auth protocol: cephx
  971. -498> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding auth protocol: cephx
  972. -497> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding auth protocol: cephx
  973. -496> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding auth protocol: none
  974. -495> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  975. -494> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  976. -493> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  977. -492> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  978. -491> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  979. -490> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  980. -489> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  981. -488> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  982. -487> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  983. -486> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  984. -485> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: crc
  985. -484> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4137140) adding con mode: secure
  986. -483> 2022-10-06T23:22:48.427+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  987. -482> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding auth protocol: cephx
  988. -481> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding auth protocol: cephx
  989. -480> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding auth protocol: cephx
  990. -479> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding auth protocol: none
  991. -478> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  992. -477> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  993. -476> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  994. -475> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  995. -474> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  996. -473> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  997. -472> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  998. -471> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  999. -470> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  1000. -469> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  1001. -468> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: crc
  1002. -467> 2022-10-06T23:22:48.427+0200 7f66abc55f00 5 AuthRegistry(0x55a5d4148140) adding con mode: secure
  1003. -466> 2022-10-06T23:22:48.427+0200 7f66abc55f00 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-16/keyring
  1004. -465> 2022-10-06T23:22:48.427+0200 7f66abc55f00 0 starting osd.16 osd_data /var/lib/ceph/osd/ceph-16 /var/lib/ceph/osd/ceph-16/journal
  1005. -464> 2022-10-06T23:22:48.460+0200 7f66a61ab700 -1 PosixStack listen unable to listen on v2:192.168.16.72:7068/0: (98) Address already in use
  1006. -463> 2022-10-06T23:22:48.516+0200 7f66abc55f00 10 monclient: build_initial_monmap
  1007. -462> 2022-10-06T23:22:48.516+0200 7f66abc55f00 10 monclient: monmap:
  1008. epoch 0
  1009. fsid 00000000-0000-0000-0000-000000000000
  1010. last_changed 0.000000
  1011. created 0.000000
  1012. min_mon_release 0 (unknown)
  1013. 0: [v2:192.168.32.65:3300/0,v1:192.168.32.65:6789/0] mon.noname-a
  1014. 1: [v2:192.168.32.66:3300/0,v1:192.168.32.66:6789/0] mon.noname-b
  1015. 2: [v2:192.168.32.67:3300/0,v1:192.168.32.67:6789/0] mon.noname-c
  1016. 3: [v2:192.168.32.68:3300/0,v1:192.168.32.68:6789/0] mon.noname-d
  1017. 4: [v2:192.168.32.69:3300/0,v1:192.168.32.69:6789/0] mon.noname-e
  1018.  
  1019. -461> 2022-10-06T23:22:48.523+0200 7f66abc55f00 0 load: jerasure load: lrc load: isa
  1020. -460> 2022-10-06T23:22:48.523+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  1021. -459> 2022-10-06T23:22:48.523+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  1022. -458> 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  1023. -457> 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  1024. -456> 2022-10-06T23:22:48.524+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) close
  1025. -455> 2022-10-06T23:22:48.802+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command objecter_requests hook 0x55a5d33ba2f0
  1026. -454> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1027. -453> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1028. -452> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1029. -451> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1030. -450> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1031. -449> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:5.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1032. -448> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:6.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1033. -447> 2022-10-06T23:22:48.802+0200 7f66abc55f00 0 osd.16:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
  1034. -446> 2022-10-06T23:22:48.803+0200 7f66abc55f00 2 osd.16 0 init /var/lib/ceph/osd/ceph-16 (looks like ssd)
  1035. -445> 2022-10-06T23:22:48.803+0200 7f66abc55f00 2 osd.16 0 journal /var/lib/ceph/osd/ceph-16/journal
  1036. -444> 2022-10-06T23:22:48.803+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _mount path /var/lib/ceph/osd/ceph-16
  1037. -443> 2022-10-06T23:22:48.804+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  1038. -442> 2022-10-06T23:22:48.804+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  1039. -441> 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev(0x55a5d40dc000 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  1040. -440> 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _set_cache_sizes cache_size 3221225472 meta 0.4 kv 0.4 data 0.2
  1041. -439> 2022-10-06T23:22:48.805+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore bluefs available hook 0x55a5d33ba330
  1042. -438> 2022-10-06T23:22:48.805+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs stats hook 0x55a5d33ba330
  1043. -437> 2022-10-06T23:22:48.805+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs debug_inject_read_zeros hook 0x55a5d33ba330
  1044. -436> 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  1045. -435> 2022-10-06T23:22:48.805+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  1046. -434> 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  1047. -433> 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  1048. -432> 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs mount
  1049. -431> 2022-10-06T23:22:48.806+0200 7f66abc55f00 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0x1749c00000
  1050. -430> 2022-10-06T23:22:48.806+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator dump bluefs-db hook 0x55a5d34b9a40
  1051. -429> 2022-10-06T23:22:48.806+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator score bluefs-db hook 0x55a5d34b9a40
  1052. -428> 2022-10-06T23:22:48.806+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator fragmentation bluefs-db hook 0x55a5d34b9a40
  1053. -427> 2022-10-06T23:22:48.965+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db set db_paths to db,95020489113 db.slow,95020489113
  1054. -426> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  1055. -425> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  1056. -424> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  1057. -423> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  1058. -422> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1059. -421> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  1060. -420> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  1061. -419> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  1062. -418> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  1063. -417> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  1064. -416> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  1065. -415> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  1066. -414> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1067. -413> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  1068. -412> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  1069. -411> 2022-10-06T23:22:48.965+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  1070. -410> 2022-10-06T23:22:48.965+0200 7f66abc55f00 1 rocksdb: do_open column families: [default]
  1071. -409> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: RocksDB version: 6.1.2
  1072.  
  1073. -408> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  1074. -407> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Compile date Aug 9 2022
  1075. -406> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: DB SUMMARY
  1076.  
  1077. -405> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: CURRENT file: CURRENT
  1078.  
  1079. -404> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: IDENTITY file: IDENTITY
  1080.  
  1081. -403> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: MANIFEST file: MANIFEST-133203 size: 94952 Bytes
  1082.  
  1083. -402> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: SST files in db dir, Total Num: 445, files: 010487.sst 010492.sst 010973.sst 011042.sst 011054.sst 011055.sst 011120.sst 029094.sst 030205.sst
  1084.  
  1085. -401> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: SST files in db.slow dir, Total Num: 345, files: 129892.sst 129917.sst 129918.sst 129919.sst 129920.sst 129926.sst 129927.sst 129928.sst 129929.sst
  1086.  
  1087. -400> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Write Ahead Log file in db: 133204.log size: 0 ;
  1088.  
  1089. -399> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.error_if_exists: 0
  1090. -398> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.create_if_missing: 0
  1091. -397> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.paranoid_checks: 1
  1092. -396> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.env: 0x55a5d3418780
  1093. -395> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.info_log: 0x55a5d342edc0
  1094. -394> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_file_opening_threads: 16
  1095. -393> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.statistics: (nil)
  1096. -392> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_fsync: 0
  1097. -391> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_log_file_size: 0
  1098. -390> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_manifest_file_size: 1073741824
  1099. -389> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.log_file_time_to_roll: 0
  1100. -388> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.keep_log_file_num: 1000
  1101. -387> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.recycle_log_file_num: 4
  1102. -386> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_fallocate: 1
  1103. -385> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_reads: 0
  1104. -384> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_writes: 0
  1105. -383> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_direct_reads: 0
  1106. -382> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  1107. -381> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.create_missing_column_families: 0
  1108. -380> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.db_log_dir:
  1109. -379> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_dir: db
  1110. -378> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_cache_numshardbits: 6
  1111. -377> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_subcompactions: 1
  1112. -376> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_flushes: -1
  1113. -375> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.WAL_ttl_seconds: 0
  1114. -374> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.WAL_size_limit_MB: 0
  1115. -373> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.manifest_preallocation_size: 4194304
  1116. -372> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.is_fd_close_on_exec: 1
  1117. -371> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.advise_random_on_open: 1
  1118. -370> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.db_write_buffer_size: 0
  1119. -369> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_manager: 0x55a5d4436fc0
  1120. -368> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.access_hint_on_compaction_start: 1
  1121. -367> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  1122. -366> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  1123. -365> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.use_adaptive_mutex: 0
  1124. -364> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.rate_limiter: (nil)
  1125. -363> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  1126. -362> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_recovery_mode: 0
  1127. -361> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_thread_tracking: 0
  1128. -360> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_pipelined_write: 0
  1129. -359> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  1130. -358> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  1131. -357> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_thread_max_yield_usec: 100
  1132. -356> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  1133. -355> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.row_cache: None
  1134. -354> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_filter: None
  1135. -353> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_recovery: 0
  1136. -352> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.allow_ingest_behind: 0
  1137. -351> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.preserve_deletes: 0
  1138. -350> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.two_write_queues: 0
  1139. -349> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.manual_wal_flush: 0
  1140. -348> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.atomic_flush: 0
  1141. -347> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
  1142. -346> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_jobs: 2
  1143. -345> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_background_compactions: 2
  1144. -344> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  1145. -343> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.writable_file_max_buffer_size: 0
  1146. -342> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.delayed_write_rate : 16777216
  1147. -341> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_total_wal_size: 0
  1148. -340> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  1149. -339> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_dump_period_sec: 600
  1150. -338> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_persist_period_sec: 600
  1151. -337> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.stats_history_buffer_size: 1048576
  1152. -336> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_open_files: -1
  1153. -335> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bytes_per_sync: 0
  1154. -334> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.wal_bytes_per_sync: 0
  1155. -333> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_readahead_size: 2097152
  1156. -332> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Compression algorithms supported:
  1157. -331> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZSTDNotFinalCompression supported: 0
  1158. -330> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZSTD supported: 0
  1159. -329> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kXpressCompression supported: 0
  1160. -328> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kLZ4HCCompression supported: 1
  1161. -327> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kLZ4Compression supported: 1
  1162. -326> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kBZip2Compression supported: 0
  1163. -325> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kZlibCompression supported: 1
  1164. -324> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: kSnappyCompression supported: 1
  1165. -323> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Fast CRC32 supported: Supported on x86
  1166. -322> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/db_impl_readonly.cc:22] Opening the db in read only mode
  1167. -321> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-133203
  1168.  
  1169. -320> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]:
  1170.  
  1171. -319> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  1172. -318> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  1173. -317> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter: None
  1174. -316> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter_factory: None
  1175. -315> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_factory: SkipListFactory
  1176. -314> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_factory: BlockBasedTable
  1177. -313> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5d34b04e0)
  1178. cache_index_and_filter_blocks: 1
  1179. cache_index_and_filter_blocks_with_high_priority: 1
  1180. pin_l0_filter_and_index_blocks_in_cache: 0
  1181. pin_top_level_index_and_filter: 1
  1182. index_type: 0
  1183. data_block_index_type: 0
  1184. data_block_hash_table_util_ratio: 0.750000
  1185. hash_index_allow_collision: 1
  1186. checksum: 1
  1187. no_block_cache: 0
  1188. block_cache: 0x55a5d4416810
  1189. block_cache_name: BinnedLRUCache
  1190. block_cache_options:
  1191. capacity : 1288490188
  1192. num_shard_bits : 4
  1193. strict_capacity_limit : 0
  1194. high_pri_pool_ratio: 0.000
  1195. block_cache_compressed: (nil)
  1196. persistent_cache: (nil)
  1197. block_size: 4096
  1198. block_size_deviation: 10
  1199. block_restart_interval: 16
  1200. index_block_restart_interval: 1
  1201. metadata_block_size: 4096
  1202. partition_filters: 0
  1203. use_delta_encoding: 1
  1204. filter_policy: rocksdb.BuiltinBloomFilter
  1205. whole_key_filtering: 1
  1206. verify_compression: 0
  1207. read_amp_bytes_per_bit: 0
  1208. format_version: 2
  1209. enable_index_compression: 1
  1210. block_align: 0
  1211.  
  1212. -312> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_size: 268435456
  1213. -311> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number: 4
  1214. -310> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression: NoCompression
  1215. -309> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression: Disabled
  1216. -308> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.prefix_extractor: nullptr
  1217. -307> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  1218. -306> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.num_levels: 7
  1219. -305> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  1220. -304> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  1221. -303> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
  1222. -302> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.level: 32767
  1223. -301> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
  1224. -300> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
  1225. -299> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
  1226. -298> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.enabled: false
  1227. -297> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.window_bits: -14
  1228. -296> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.level: 32767
  1229. -295> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.strategy: 0
  1230. -294> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  1231. -293> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  1232. -292> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.enabled: false
  1233. -291> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  1234. -290> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  1235. -289> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level0_stop_writes_trigger: 36
  1236. -288> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_base: 67108864
  1237. -287> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_multiplier: 1
  1238. -286> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  1239. -285> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  1240. -284> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  1241. -283> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  1242. -282> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  1243. -281> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  1244. -280> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  1245. -279> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  1246. -278> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  1247. -277> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  1248. -276> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  1249. -275> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_compaction_bytes: 1677721600
  1250. -274> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.arena_block_size: 33554432
  1251. -273> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  1252. -272> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  1253. -271> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  1254. -270> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.disable_auto_compactions: 0
  1255. -269> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  1256. -268> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
  1257. -267> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  1258. -266> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  1259. -265> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  1260. -264> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  1261. -263> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  1262. -262> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  1263. -261> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  1264. -260> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  1265. -259> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.table_properties_collectors:
  1266. -258> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_support: 0
  1267. -257> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_num_locks: 10000
  1268. -256> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  1269. -255> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_whole_key_filtering: 0
  1270. -254> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.memtable_huge_page_size: 0
  1271. -253> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.bloom_locality: 0
  1272. -252> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.max_successive_merges: 0
  1273. -251> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.optimize_filters_for_hits: 0
  1274. -250> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.paranoid_file_checks: 0
  1275. -249> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.force_consistency_checks: 0
  1276. -248> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.report_bg_io_stats: 0
  1277. -247> 2022-10-06T23:22:48.966+0200 7f66abc55f00 4 rocksdb: Options.ttl: 0
  1278. -246> 2022-10-06T23:22:48.989+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-133203 succeeded,manifest_file_number is 133203, next_file_number is 133205, last_sequence is 3305434714, log_number is 133202,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0
  1279.  
  1280. -245> 2022-10-06T23:22:48.989+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 133202
  1281.  
  1282. -244> 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091368991402, "job": 1, "event": "recovery_started", "log_files": [133204]}
  1283. -243> 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #133204 mode 0
  1284. -242> 2022-10-06T23:22:48.990+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091368991420, "job": 1, "event": "recovery_finished"}
  1285. -241> 2022-10-06T23:22:48.991+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2
  1286. -240> 2022-10-06T23:22:48.996+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta old nid_max 63595586
  1287. -239> 2022-10-06T23:22:48.996+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta old blobid_max 122880
  1288. -238> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta freelist_type bitmap
  1289. -237> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta ondisk_format 4 compat_ondisk_format 3
  1290. -236> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_super_meta min_alloc_size 0x4000
  1291. -235> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 get compressor snappy = 0
  1292. -234> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 load compressor snappy
  1293. -233> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 add compressor snappy 0x55a5d45a33e0
  1294. -232> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 get compressor snappy = 0x55a5d45a33e0
  1295. -231> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 load: compressor snappy loaded and registered
  1296. -230> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 get compressor snappy = 0x55a5d45a33e0
  1297. -229> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 freelist init
  1298. -228> 2022-10-06T23:22:49.005+0200 7f66abc55f00 1 freelist _init_from_label
  1299. -227> 2022-10-06T23:22:49.010+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator dump block hook 0x55a5d53da450
  1300. -226> 2022-10-06T23:22:49.010+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator score block hook 0x55a5d53da450
  1301. -225> 2022-10-06T23:22:49.010+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator fragmentation block hook 0x55a5d53da450
  1302. -224> 2022-10-06T23:22:49.010+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_alloc opening allocation metadata
  1303. -223> 2022-10-06T23:22:49.044+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_alloc loaded 81 GiB in 21981 extents available 593 MiB
  1304. -222> 2022-10-06T23:22:49.044+0200 7f66abc55f00 4 rocksdb: [db/db_impl.cc:390] Shutdown: canceling all background work
  1305. -221> 2022-10-06T23:22:49.049+0200 7f66abc55f00 4 rocksdb: [db/db_impl.cc:563] Shutdown complete
  1306. -220> 2022-10-06T23:22:49.049+0200 7f66abc55f00 1 bluefs umount
  1307. -219> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluestore allocator dump bluefs-db
  1308. -218> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluestore allocator fragmentation bluefs-db
  1309. -217> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluestore allocator score bluefs-db
  1310. -216> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluefs debug_inject_read_zeros
  1311. -215> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluefs stats
  1312. -214> 2022-10-06T23:22:49.050+0200 7f66abc55f00 5 asok(0x55a5d3474000) unregister_commands bluestore bluefs available
  1313. -213> 2022-10-06T23:22:49.050+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) close
  1314. -212> 2022-10-06T23:22:49.096+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore bluefs available hook 0x55a5d33ba330
  1315. -211> 2022-10-06T23:22:49.096+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs stats hook 0x55a5d33ba330
  1316. -210> 2022-10-06T23:22:49.096+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluefs debug_inject_read_zeros hook 0x55a5d33ba330
  1317. -209> 2022-10-06T23:22:49.096+0200 7f66abc55f00 1 bdev create path /var/lib/ceph/osd/ceph-16/block type kernel
  1318. -208> 2022-10-06T23:22:49.096+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open path /var/lib/ceph/osd/ceph-16/block
  1319. -207> 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bdev(0x55a5d40dc700 /var/lib/ceph/osd/ceph-16/block) open size 100021567488 (0x1749c00000, 93 GiB) block_size 4096 (4 KiB) non-rotational discard supported
  1320. -206> 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-16/block size 93 GiB
  1321. -205> 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs mount
  1322. -204> 2022-10-06T23:22:49.097+0200 7f66abc55f00 1 bluefs _init_alloc id 1 alloc_size 0x10000 size 0x1749c00000
  1323. -203> 2022-10-06T23:22:49.097+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator dump bluefs-db hook 0x55a5d34b9c50
  1324. -202> 2022-10-06T23:22:49.097+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator score bluefs-db hook 0x55a5d34b9c50
  1325. -201> 2022-10-06T23:22:49.097+0200 7f66abc55f00 5 asok(0x55a5d3474000) register_command bluestore allocator fragmentation bluefs-db hook 0x55a5d34b9c50
  1326. -200> 2022-10-06T23:22:49.210+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db set db_paths to db,95020489113 db.slow,95020489113
  1327. -199> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  1328. -198> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  1329. -197> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  1330. -196> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  1331. -195> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1332. -194> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  1333. -193> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  1334. -192> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  1335. -191> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compaction_readahead_size = 2097152
  1336. -190> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option compression = kNoCompression
  1337. -189> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_background_compactions = 2
  1338. -188> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option max_write_buffer_number = 4
  1339. -187> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option min_write_buffer_number_to_merge = 1
  1340. -186> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option recycle_log_file_num = 4
  1341. -185> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option writable_file_max_buffer_size = 0
  1342. -184> 2022-10-06T23:22:49.210+0200 7f66abc55f00 0 set rocksdb option write_buffer_size = 268435456
  1343. -183> 2022-10-06T23:22:49.211+0200 7f66abc55f00 1 rocksdb: do_open column families: [default]
  1344. -182> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: RocksDB version: 6.1.2
  1345.  
  1346. -181> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Git sha rocksdb_build_git_sha:@0@
  1347. -180> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Compile date Aug 9 2022
  1348. -179> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: DB SUMMARY
  1349.  
  1350. -178> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: CURRENT file: CURRENT
  1351.  
  1352. -177> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: IDENTITY file: IDENTITY
  1353.  
  1354. -176> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: MANIFEST file: MANIFEST-133203 size: 94952 Bytes
  1355.  
  1356. -175> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: SST files in db dir, Total Num: 445, files: 010487.sst 010492.sst 010973.sst 011042.sst 011054.sst 011055.sst 011120.sst 029094.sst 030205.sst
  1357.  
  1358. -174> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: SST files in db.slow dir, Total Num: 345, files: 129892.sst 129917.sst 129918.sst 129919.sst 129920.sst 129926.sst 129927.sst 129928.sst 129929.sst
  1359.  
  1360. -173> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Write Ahead Log file in db: 133204.log size: 0 ;
  1361.  
  1362. -172> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.error_if_exists: 0
  1363. -171> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.create_if_missing: 0
  1364. -170> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.paranoid_checks: 1
  1365. -169> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.env: 0x55a5d43f6000
  1366. -168> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.info_log: 0x55a5d342f960
  1367. -167> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_file_opening_threads: 16
  1368. -166> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.statistics: (nil)
  1369. -165> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_fsync: 0
  1370. -164> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_log_file_size: 0
  1371. -163> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_manifest_file_size: 1073741824
  1372. -162> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.log_file_time_to_roll: 0
  1373. -161> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.keep_log_file_num: 1000
  1374. -160> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.recycle_log_file_num: 4
  1375. -159> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_fallocate: 1
  1376. -158> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_reads: 0
  1377. -157> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_mmap_writes: 0
  1378. -156> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_direct_reads: 0
  1379. -155> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0
  1380. -154> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.create_missing_column_families: 0
  1381. -153> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.db_log_dir:
  1382. -152> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_dir: db
  1383. -151> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_cache_numshardbits: 6
  1384. -150> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_subcompactions: 1
  1385. -149> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_flushes: -1
  1386. -148> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.WAL_ttl_seconds: 0
  1387. -147> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.WAL_size_limit_MB: 0
  1388. -146> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.manifest_preallocation_size: 4194304
  1389. -145> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.is_fd_close_on_exec: 1
  1390. -144> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.advise_random_on_open: 1
  1391. -143> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.db_write_buffer_size: 0
  1392. -142> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_manager: 0x55a5d43b45d0
  1393. -141> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.access_hint_on_compaction_start: 1
  1394. -140> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 1
  1395. -139> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.random_access_max_buffer_size: 1048576
  1396. -138> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.use_adaptive_mutex: 0
  1397. -137> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.rate_limiter: (nil)
  1398. -136> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0
  1399. -135> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_recovery_mode: 0
  1400. -134> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_thread_tracking: 0
  1401. -133> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_pipelined_write: 0
  1402. -132> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_concurrent_memtable_write: 1
  1403. -131> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1
  1404. -130> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_thread_max_yield_usec: 100
  1405. -129> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_thread_slow_yield_usec: 3
  1406. -128> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.row_cache: None
  1407. -127> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_filter: None
  1408. -126> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_recovery: 0
  1409. -125> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.allow_ingest_behind: 0
  1410. -124> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.preserve_deletes: 0
  1411. -123> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.two_write_queues: 0
  1412. -122> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.manual_wal_flush: 0
  1413. -121> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.atomic_flush: 0
  1414. -120> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0
  1415. -119> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_jobs: 2
  1416. -118> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_background_compactions: 2
  1417. -117> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.avoid_flush_during_shutdown: 0
  1418. -116> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.writable_file_max_buffer_size: 0
  1419. -115> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.delayed_write_rate : 16777216
  1420. -114> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_total_wal_size: 0
  1421. -113> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000
  1422. -112> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_dump_period_sec: 600
  1423. -111> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_persist_period_sec: 600
  1424. -110> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.stats_history_buffer_size: 1048576
  1425. -109> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_open_files: -1
  1426. -108> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bytes_per_sync: 0
  1427. -107> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.wal_bytes_per_sync: 0
  1428. -106> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_readahead_size: 2097152
  1429. -105> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Compression algorithms supported:
  1430. -104> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZSTDNotFinalCompression supported: 0
  1431. -103> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZSTD supported: 0
  1432. -102> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kXpressCompression supported: 0
  1433. -101> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kLZ4HCCompression supported: 1
  1434. -100> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kLZ4Compression supported: 1
  1435. -99> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kBZip2Compression supported: 0
  1436. -98> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kZlibCompression supported: 1
  1437. -97> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: kSnappyCompression supported: 1
  1438. -96> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Fast CRC32 supported: Supported on x86
  1439. -95> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: db/MANIFEST-133203
  1440.  
  1441. -94> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]:
  1442.  
  1443. -93> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.comparator: leveldb.BytewiseComparator
  1444. -92> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor
  1445. -91> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter: None
  1446. -90> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_filter_factory: None
  1447. -89> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_factory: SkipListFactory
  1448. -88> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_factory: BlockBasedTable
  1449. -87> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5d33c1818)
  1450. cache_index_and_filter_blocks: 1
  1451. cache_index_and_filter_blocks_with_high_priority: 1
  1452. pin_l0_filter_and_index_blocks_in_cache: 0
  1453. pin_top_level_index_and_filter: 1
  1454. index_type: 0
  1455. data_block_index_type: 0
  1456. data_block_hash_table_util_ratio: 0.750000
  1457. hash_index_allow_collision: 1
  1458. checksum: 1
  1459. no_block_cache: 0
  1460. block_cache: 0x55a5d3409310
  1461. block_cache_name: BinnedLRUCache
  1462. block_cache_options:
  1463. capacity : 1288490188
  1464. num_shard_bits : 4
  1465. strict_capacity_limit : 0
  1466. high_pri_pool_ratio: 0.000
  1467. block_cache_compressed: (nil)
  1468. persistent_cache: (nil)
  1469. block_size: 4096
  1470. block_size_deviation: 10
  1471. block_restart_interval: 16
  1472. index_block_restart_interval: 1
  1473. metadata_block_size: 4096
  1474. partition_filters: 0
  1475. use_delta_encoding: 1
  1476. filter_policy: rocksdb.BuiltinBloomFilter
  1477. whole_key_filtering: 1
  1478. verify_compression: 0
  1479. read_amp_bytes_per_bit: 0
  1480. format_version: 2
  1481. enable_index_compression: 1
  1482. block_align: 0
  1483.  
  1484. -86> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.write_buffer_size: 268435456
  1485. -85> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number: 4
  1486. -84> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression: NoCompression
  1487. -83> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression: Disabled
  1488. -82> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.prefix_extractor: nullptr
  1489. -81> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr
  1490. -80> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.num_levels: 7
  1491. -79> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.min_write_buffer_number_to_merge: 1
  1492. -78> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0
  1493. -77> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14
  1494. -76> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.level: 32767
  1495. -75> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.strategy: 0
  1496. -74> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0
  1497. -73> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0
  1498. -72> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bottommost_compression_opts.enabled: false
  1499. -71> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.window_bits: -14
  1500. -70> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.level: 32767
  1501. -69> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.strategy: 0
  1502. -68> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.max_dict_bytes: 0
  1503. -67> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0
  1504. -66> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compression_opts.enabled: false
  1505. -65> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_file_num_compaction_trigger: 4
  1506. -64> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_slowdown_writes_trigger: 20
  1507. -63> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level0_stop_writes_trigger: 36
  1508. -62> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_base: 67108864
  1509. -61> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.target_file_size_multiplier: 1
  1510. -60> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_base: 268435456
  1511. -59> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
  1512. -58> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000
  1513. -57> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
  1514. -56> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
  1515. -55> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
  1516. -54> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
  1517. -53> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
  1518. -52> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
  1519. -51> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
  1520. -50> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_sequential_skip_in_iterations: 8
  1521. -49> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_compaction_bytes: 1677721600
  1522. -48> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.arena_block_size: 33554432
  1523. -47> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736
  1524. -46> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944
  1525. -45> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100
  1526. -44> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.disable_auto_compactions: 0
  1527. -43> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_style: kCompactionStyleLevel
  1528. -42> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio
  1529. -41> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.size_ratio: 1
  1530. -40> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
  1531. -39> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
  1532. -38> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
  1533. -37> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
  1534. -36> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
  1535. -35> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
  1536. -34> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
  1537. -33> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.table_properties_collectors:
  1538. -32> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_support: 0
  1539. -31> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.inplace_update_num_locks: 10000
  1540. -30> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000
  1541. -29> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_whole_key_filtering: 0
  1542. -28> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.memtable_huge_page_size: 0
  1543. -27> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.bloom_locality: 0
  1544. -26> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.max_successive_merges: 0
  1545. -25> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.optimize_filters_for_hits: 0
  1546. -24> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.paranoid_file_checks: 0
  1547. -23> 2022-10-06T23:22:49.211+0200 7f66abc55f00 4 rocksdb: Options.force_consistency_checks: 0
  1548. -22> 2022-10-06T23:22:49.212+0200 7f66abc55f00 4 rocksdb: Options.report_bg_io_stats: 0
  1549. -21> 2022-10-06T23:22:49.212+0200 7f66abc55f00 4 rocksdb: Options.ttl: 0
  1550. -20> 2022-10-06T23:22:49.216+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:db/MANIFEST-133203 succeeded,manifest_file_number is 133203, next_file_number is 133205, last_sequence is 3305434714, log_number is 133202,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0
  1551.  
  1552. -19> 2022-10-06T23:22:49.216+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 133202
  1553.  
  1554. -18> 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369218098, "job": 1, "event": "recovery_started", "log_files": [133204]}
  1555. -17> 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #133204 mode 0
  1556. -16> 2022-10-06T23:22:49.217+0200 7f66abc55f00 4 rocksdb: [db/version_set.cc:3036] Creating manifest 133206
  1557.  
  1558. -15> 2022-10-06T23:22:49.219+0200 7f66abc55f00 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369220260, "job": 1, "event": "recovery_finished"}
  1559. -14> 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: [db/compaction_job.cc:1645] [default] [JOB 3] Compacting 1@3 files to L4, score 1.09
  1560. -13> 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: [db/compaction_job.cc:1649] [default] Compaction start summary: Base version 2 Base level 3, inputs: [133096(64MB)]
  1561.  
  1562. -12> 2022-10-06T23:22:49.221+0200 7f669d129700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1665091369222004, "job": 3, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L3": [133096], "score": 1.09115, "input_data_size": 68055858}
  1563. -11> 2022-10-06T23:22:49.221+0200 7f66abc55f00 4 rocksdb: DB pointer 0x55a5d410c400
  1564. -10> 2022-10-06T23:22:49.221+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2
  1565. -9> 2022-10-06T23:22:49.221+0200 7f6693916700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS -------
  1566. -8> 2022-10-06T23:22:49.222+0200 7f6693916700 4 rocksdb: [db/db_impl.cc:778]
  1567. ** DB Stats **
  1568. Uptime(secs): 0.0 total, 0.0 interval
  1569. Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s
  1570. Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
  1571. Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
  1572. Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s
  1573. Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
  1574. Interval stall: 00:00:0.000 H:M:S, 0.0 percent
  1575.  
  1576. ** Compaction Stats [default] **
  1577. Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  1578. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  1579. L0 1/0 61.55 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1580. L1 5/0 256.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1581. L2 29/0 1.67 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1582. L3 180/1 9.85 GB 1.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1583. L4 575/0 34.66 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1584. Sum 790/1 46.49 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1585. Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1586.  
  1587. ** Compaction Stats [default] **
  1588. Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  1589. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  1590. Uptime(secs): 0.0 total, 0.0 interval
  1591. Flush(GB): cumulative 0.000, interval 0.000
  1592. AddFile(GB): cumulative 0.000, interval 0.000
  1593. AddFile(Total Files): cumulative 0, interval 0
  1594. AddFile(L0 Files): cumulative 0, interval 0
  1595. AddFile(Keys): cumulative 0, interval 0
  1596. Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  1597. Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  1598. Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
  1599.  
  1600. ** File Read Latency Histogram By Level [default] **
  1601.  
  1602. ** Compaction Stats [default] **
  1603. Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  1604. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  1605. L0 1/0 61.55 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1606. L1 5/0 256.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1607. L2 29/0 1.67 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1608. L3 180/1 9.85 GB 1.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1609. L4 575/0 34.66 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1610. Sum 790/1 46.49 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1611. Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
  1612.  
  1613. ** Compaction Stats [default] **
  1614. Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
  1615. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  1616. Uptime(secs): 0.0 total, 0.0 interval
  1617. Flush(GB): cumulative 0.000, interval 0.000
  1618. AddFile(GB): cumulative 0.000, interval 0.000
  1619. AddFile(Total Files): cumulative 0, interval 0
  1620. AddFile(L0 Files): cumulative 0, interval 0
  1621. AddFile(Keys): cumulative 0, interval 0
  1622. Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  1623. Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
  1624. Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
  1625.  
  1626. ** File Read Latency Histogram By Level [default] **
  1627.  
  1628. -7> 2022-10-06T23:22:49.223+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _upgrade_super from 4, latest 4
  1629. -6> 2022-10-06T23:22:49.223+0200 7f66abc55f00 1 bluestore(/var/lib/ceph/osd/ceph-16) _upgrade_super done
  1630. -5> 2022-10-06T23:22:49.267+0200 7f669d129700 1 bluefs _allocate unable to allocate 0x1001a9 on bdev 2, free 0xffffffffffffffff; fallback to slow device expander
  1631. -4> 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluestore(/var/lib/ceph/osd/ceph-16) allocate_bluefs_freespace failed to allocate on 0x110000 min_size 0x110000 > allocated total 0x30000 bluefs_shared_alloc_size 0x10000 allocated 0x30000 available 0x 25134000
  1632. -3> 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluefs _allocate failed to expand slow device to fit +0x1001a9
  1633. -2> 2022-10-06T23:22:49.267+0200 7f669d129700 -1 bluefs _flush_range allocated: 0x0 offset: 0x0 length: 0x1001a9
  1634. -1> 2022-10-06T23:22:49.275+0200 7f669d129700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.17/rpm/el8/BUILD/ceph-15.2.17/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f669d129700 time 2022-10-06T23:22:49.268150+0200
  1635. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.17/rpm/el8/BUILD/ceph-15.2.17/src/os/bluestore/BlueFS.cc: 2931: ceph_abort_msg("bluefs enospc")
  1636.  
  1637. ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)
  1638. 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x55a5c790cb32]
  1639. 2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1545) [0x55a5c7f95215]
  1640. 3: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x110) [0x55a5c7f95600]
  1641. 4: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x55a5c7fa6952]
  1642. 5: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x1d9) [0x55a5c7fc1d39]
  1643. 6: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x560) [0x55a5c859c700]
  1644. 7: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2b0) [0x55a5c859d530]
  1645. 8: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x55a5c860d486]
  1646. 9: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x55a5c860ddac]
  1647. 10: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x55a5c860e48c]
  1648. 11: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x55a5c860e51d]
  1649. 12: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2a8) [0x55a5c8611768]
  1650. 13: (rocksdb::CompactionJob::ProcessKeyValueCompaction(rocksdb::CompactionJob::SubcompactionState*)+0x6df) [0x55a5c85de28f]
  1651. 14: (rocksdb::CompactionJob::Run()+0x264) [0x55a5c85df424]
  1652. 15: (rocksdb::DBImpl::BackgroundCompaction(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xd0e) [0x55a5c845aaee]
  1653. 16: (rocksdb::DBImpl::BackgroundCallCompaction(rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xc6) [0x55a5c8461086]
  1654. 17: (rocksdb::DBImpl::BGWorkCompaction(void*)+0x3a) [0x55a5c846155a]
  1655. 18: (rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24a) [0x55a5c8619f0a]
  1656. 19: (rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x5d) [0x55a5c861a08d]
  1657. 20: (()+0xc2ba3) [0x7f66a8ff9ba3]
  1658. 21: (()+0x81ca) [0x7f66a99a61ca]
  1659. 22: (clone()+0x43) [0x7f66a8610dd3]
  1660.  
  1661. 0> 2022-10-06T23:22:49.283+0200 7f669d129700 -1 *** Caught signal (Aborted) **
  1662. in thread 7f669d129700 thread_name:rocksdb:low0
  1663.  
  1664. ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)
  1665. 1: (()+0x12ce0) [0x7f66a99b0ce0]
  1666. 2: (gsignal()+0x10f) [0x7f66a8625a9f]
  1667. 3: (abort()+0x127) [0x7f66a85f8e05]
  1668. 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x55a5c790cc03]
  1669. 5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1545) [0x55a5c7f95215]
  1670. 6: (BlueFS::_flush(BlueFS::FileWriter*, bool, bool*)+0x110) [0x55a5c7f95600]
  1671. 7: (BlueFS::_flush(BlueFS::FileWriter*, bool, std::unique_lock<std::mutex>&)+0x32) [0x55a5c7fa6952]
  1672. 8: (BlueRocksWritableFile::Append(rocksdb::Slice const&)+0x1d9) [0x55a5c7fc1d39]
  1673. 9: (rocksdb::WritableFileWriter::WriteBuffered(char const*, unsigned long)+0x560) [0x55a5c859c700]
  1674. 10: (rocksdb::WritableFileWriter::Append(rocksdb::Slice const&)+0x2b0) [0x55a5c859d530]
  1675. 11: (rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, bool)+0xb6) [0x55a5c860d486]
  1676. 12: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, bool)+0x26c) [0x55a5c860ddac]
  1677. 13: (rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::BlockBuilder*, rocksdb::BlockHandle*, bool)+0x3c) [0x55a5c860e48c]
  1678. 14: (rocksdb::BlockBasedTableBuilder::Flush()+0x6d) [0x55a5c860e51d]
  1679. 15: (rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x2a8) [0x55a5c8611768]
  1680. 16: (rocksdb::CompactionJob::ProcessKeyValueCompaction(rocksdb::CompactionJob::SubcompactionState*)+0x6df) [0x55a5c85de28f]
  1681. 17: (rocksdb::CompactionJob::Run()+0x264) [0x55a5c85df424]
  1682. 18: (rocksdb::DBImpl::BackgroundCompaction(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xd0e) [0x55a5c845aaee]
  1683. 19: (rocksdb::DBImpl::BackgroundCallCompaction(rocksdb::DBImpl::PrepickedCompaction*, rocksdb::Env::Priority)+0xc6) [0x55a5c8461086]
  1684. 20: (rocksdb::DBImpl::BGWorkCompaction(void*)+0x3a) [0x55a5c846155a]
  1685. 21: (rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24a) [0x55a5c8619f0a]
  1686. 22: (rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x5d) [0x55a5c861a08d]
  1687. 23: (()+0xc2ba3) [0x7f66a8ff9ba3]
  1688. 24: (()+0x81ca) [0x7f66a99a61ca]
  1689. 25: (clone()+0x43) [0x7f66a8610dd3]
  1690. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  1691.  
  1692. --- logging levels ---
  1693. 0/ 5 none
  1694. 0/ 1 lockdep
  1695. 0/ 1 context
  1696. 1/ 1 crush
  1697. 1/ 5 mds
  1698. 1/ 5 mds_balancer
  1699. 1/ 5 mds_locker
  1700. 1/ 5 mds_log
  1701. 1/ 5 mds_log_expire
  1702. 1/ 5 mds_migrator
  1703. 0/ 1 buffer
  1704. 0/ 1 timer
  1705. 0/ 1 filer
  1706. 0/ 1 striper
  1707. 0/ 1 objecter
  1708. 0/ 5 rados
  1709. 0/ 5 rbd
  1710. 0/ 5 rbd_mirror
  1711. 0/ 5 rbd_replay
  1712. 0/ 5 rbd_rwl
  1713. 0/ 5 journaler
  1714. 0/ 5 objectcacher
  1715. 0/ 5 immutable_obj_cache
  1716. 0/ 5 client
  1717. 1/ 5 osd
  1718. 0/ 5 optracker
  1719. 0/ 5 objclass
  1720. 1/ 3 filestore
  1721. 1/ 3 journal
  1722. 0/ 0 ms
  1723. 1/ 5 mon
  1724. 0/10 monc
  1725. 1/ 5 paxos
  1726. 0/ 5 tp
  1727. 1/ 5 auth
  1728. 1/ 5 crypto
  1729. 1/ 1 finisher
  1730. 1/ 1 reserver
  1731. 1/ 5 heartbeatmap
  1732. 1/ 5 perfcounter
  1733. 1/ 5 rgw
  1734. 1/ 5 rgw_sync
  1735. 1/10 civetweb
  1736. 1/ 5 javaclient
  1737. 1/ 5 asok
  1738. 1/ 1 throttle
  1739. 0/ 0 refs
  1740. 1/ 5 compressor
  1741. 1/ 5 bluestore
  1742. 1/ 5 bluefs
  1743. 1/ 3 bdev
  1744. 1/ 5 kstore
  1745. 4/ 5 rocksdb
  1746. 4/ 5 leveldb
  1747. 4/ 5 memdb
  1748. 1/ 5 fuse
  1749. 2/ 5 mgr
  1750. 1/ 5 mgrc
  1751. 1/ 5 dpdk
  1752. 1/ 5 eventtrace
  1753. 1/ 5 prioritycache
  1754. 0/ 5 test
  1755. -1/-1 (syslog threshold)
  1756. -1/-1 (stderr threshold)
  1757. --- pthread ID / name mapping for recent threads ---
  1758. 7f6693916700 / rocksdb:dump_st
  1759. 7f669d129700 / rocksdb:low0
  1760. 7f66a41a7700 / admin_socket
  1761. 7f66a49a8700 / service
  1762. 7f66a51a9700 / msgr-worker-2
  1763. 7f66a59aa700 / msgr-worker-1
  1764. 7f66a61ab700 / msgr-worker-0
  1765. 7f66abc55f00 / ceph-osd
  1766. max_recent 10000
  1767. max_new 1000
  1768. log_file /var/log/ceph/ceph-osd.16.log
  1769. --- end dump of recent events ---
  1770.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!