API tools faq
paste
Guest User

Untitled

a guest
Apr 15th, 2025
57
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 78.34 KB | None | 0 0
raw download clone embed print report
  1. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  2. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  3. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  4. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  5. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  6. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  7. Apr 09 03:15:58 ceph-osd01 ceph-osd[24534]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  8. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  9.  
  10. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  11. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  12. 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  13. 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  14. 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  15. 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  16. 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  17. 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  18. 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  19. 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  20. 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  21. 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  22. 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  23. 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  24. 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  25. 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  26. 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  27. 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  28. 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  29. 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  30. 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  31. 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  32. 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  33. 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  34. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  35. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  36. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  37. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  38. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  39. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  40. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  41. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  42. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  43. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  44. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  45. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  46. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  47. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  48. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  49. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  50. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  51. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  52. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  53. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  54. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: *** Caught signal (Aborted) **
  55. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: in thread 7f4b483b5640 thread_name:
  56. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2025-04-09T03:15:58.257+0000 7f4b483b5640 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  57. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  58. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  59. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  60. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  61. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  62. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  63. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  64. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  65. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  66. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  67. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  68. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  69. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  70. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  71. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  72. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  73. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  74. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  75. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  76. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  77. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  78. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  79. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  80. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  81. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  82. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  83. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  84. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  85. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  86. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  87. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: raise()
  88. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: abort()
  89. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  90. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  91. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  92. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  93. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  94. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  95. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  96. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  97. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  98. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  99. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  100. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  101. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  102. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  103. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  104. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  105. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  106. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  107. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  108. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  109. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  110. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  111. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  112. Apr 09 03:15:58 ceph-osd01 ceph-osd[24534]: *** Caught signal (Aborted) **
  113. in thread 7f4b483b5640 thread_name:
  114.  
  115. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  116. 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  117. 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  118. 3: raise()
  119. 4: abort()
  120. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  121. 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  122. 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  123. 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  124. 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  125. 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  126. 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  127. 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  128. 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  129. 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  130. 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  131. 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  132. 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  133. 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  134. 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  135. 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  136. 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  137. 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  138. 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  139. 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  140. 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  141. 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  142. 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  143. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  144. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2025-04-09T03:15:58.265+0000 7f4b483b5640 -1 *** Caught signal (Aborted) **
  145. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: in thread 7f4b483b5640 thread_name:
  146. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  147. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  148. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  149. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  150. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: raise()
  151. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: abort()
  152. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  153. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  154. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  155. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  156. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  157. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  158. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  159. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  160. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  161. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  162. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  163. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  164. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  165. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  166. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  167. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  168. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  169. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  170. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  171. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  172. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  173. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  174. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  175. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  176. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  177. Apr 09 03:15:58 ceph-osd01 ceph-osd[24534]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  178. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  179.  
  180. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  181. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  182. 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  183. 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  184. 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  185. 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  186. 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  187. 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  188. 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  189. 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  190. 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  191. 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  192. 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  193. 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  194. 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  195. 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  196. 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  197. 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  198. 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  199. 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  200. 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  201. 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  202. 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  203. 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  204. Apr 09 03:15:58 ceph-osd01 ceph-osd[24534]: *** Caught signal (Aborted) **
  205. in thread 7f4b483b5640 thread_name:
  206.  
  207. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  208. 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  209. 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  210. 3: raise()
  211. 4: abort()
  212. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  213. 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  214. 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  215. 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  216. 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  217. 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  218. 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  219. 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  220. 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  221. 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  222. 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  223. 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  224. 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  225. 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  226. 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  227. 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  228. 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  229. 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  230. 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  231. 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  232. 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  233. 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  234. 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  235. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  236. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: -9999> 2025-04-09T03:15:58.257+0000 7f4b483b5640 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  237. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  238. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  239. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  240. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  241. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  242. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  243. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  244. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  245. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  246. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  247. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  248. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  249. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  250. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  251. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  252. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  253. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  254. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  255. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  256. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  257. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  258. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  259. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  260. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  261. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  262. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  263. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  264. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: -9998> 2025-04-09T03:15:58.265+0000 7f4b483b5640 -1 *** Caught signal (Aborted) **
  265. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: in thread 7f4b483b5640 thread_name:
  266. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  267. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  268. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  269. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  270. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: raise()
  271. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: abort()
  272. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  273. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  274. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  275. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  276. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  277. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  278. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  279. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  280. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  281. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  282. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  283. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  284. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  285. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  286. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  287. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  288. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  289. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  290. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  291. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  292. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  293. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  294. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  295. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  296. Apr 09 03:15:58 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  297. Apr 09 03:15:59 ceph-osd01 ceph-osd[24534]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  298. /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  299.  
  300. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  301. 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  302. 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  303. 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  304. 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  305. 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  306. 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  307. 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  308. 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  309. 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  310. 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  311. 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  312. 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  313. 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  314. 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  315. 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  316. 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  317. 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  318. 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  319. 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  320. 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  321. 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  322. 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  323. 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  324. Apr 09 03:15:59 ceph-osd01 ceph-osd[24534]: *** Caught signal (Aborted) **
  325. in thread 7f4b483b5640 thread_name:
  326.  
  327. ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  328. 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  329. 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  330. 3: raise()
  331. 4: abort()
  332. 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  333. 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  334. 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  335. 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  336. 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  337. 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  338. 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  339. 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  340. 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  341. 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  342. 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  343. 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  344. 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  345. 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  346. 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  347. 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  348. 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  349. 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  350. 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  351. 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  352. 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  353. 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  354. 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  355. NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  356. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: -9999> 2025-04-09T03:15:58.257+0000 7f4b483b5640 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::Blob::copy_extents_over_empty(ceph::common::CephContext*, const BlueStore::Blob&, uint32_t, uint32_t)' thread 7f4b483b5640 time 2025-04-09T03:15:58.248221+0000
  357. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.2.1/rpm/el9/BUILD/ceph-19.2.1/src/os/bluestore/BlueStore.cc: 2625: FAILED ceph_assert(!ito->is_valid())
  358. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  359. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  360. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x116) [0x564c4d74a2ac]
  361. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  362. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  363. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  364. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  365. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  366. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  367. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  368. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  369. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  370. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  371. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  372. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  373. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  374. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  375. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  376. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  377. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  378. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  379. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  380. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  381. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  382. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  383. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  384. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: -9998> 2025-04-09T03:15:58.265+0000 7f4b483b5640 -1 *** Caught signal (Aborted) **
  385. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: in thread 7f4b483b5640 thread_name:
  386. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  387. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: ceph version 19.2.1 (58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)
  388. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 1: /lib64/libc.so.6(+0x3e930) [0x7f4b62aad930]
  389. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 2: /lib64/libc.so.6(+0x8bfdc) [0x7f4b62afafdc]
  390. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 3: raise()
  391. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 4: abort()
  392. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16c) [0x564c4d74a302]
  393. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 6: /usr/bin/ceph-osd(+0x3fa463) [0x564c4d74a463]
  394. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 7: /usr/bin/ceph-osd(+0x3db06e) [0x564c4d72b06e]
  395. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 8: (BlueStore::Blob::copy_from(ceph::common::CephContext*, BlueStore::Blob const&, unsigned int, unsigned int, unsigned int)+0x13d) [0x564c4dcbbaed]
  396. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 9: (BlueStore::ExtentMap::dup_esb(BlueStore*, BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long&, unsigned long&, unsigned long&)+0x93e) [0x564c4dccce6e]
  397. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 10: (BlueStore::_do_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x1b0) [0x564c4dd2bc20]
  398. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 11: (BlueStore::_clone_range(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, unsigned long)+0x288) [0x564c4dd3a868]
  399. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 12: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x159d) [0x564c4dd420dd]
  400. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 13: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x303) [0x564c4dd23703]
  401. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 14: (ECBackend::handle_sub_write(pg_shard_t, boost::intrusive_ptr<OpRequest>, ECSubWrite&, ZTracer::Trace const&, ECListener&)+0xac7) [0x564c4dc4efc7]
  402. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 15: (ECCommon::RMWPipeline::try_reads_to_commit()+0x29ae) [0x564c4da4fb6e]
  403. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 16: (ECCommon::RMWPipeline::start_rmw(std::unique_ptr<ECCommon::RMWPipeline::Op, std::default_delete<ECCommon::RMWPipeline::Op> >)+0x114) [0x564c4da501a4]
  404. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 17: (ECBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x23ac) [0x564c4dc56c3c]
  405. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 18: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0x3a4) [0x564c4d9ed8c4]
  406. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 19: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xe1c) [0x564c4d9b473c]
  407. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 20: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x3343) [0x564c4d9aaea3]
  408. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 21: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x197) [0x564c4d8e7987]
  409. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 22: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x69) [0x564c4db30c49]
  410. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd0) [0x564c4d901c80]
  411. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x564c4de0224a]
  412. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 25: /usr/bin/ceph-osd(+0xab2804) [0x564c4de02804]
  413. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 26: /lib64/libc.so.6(+0x8a292) [0x7f4b62af9292]
  414. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: 27: /lib64/libc.so.6(+0x10f300) [0x7f4b62b7e300]
  415. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
  416. Apr 09 03:15:59 ceph-osd01 ceph-d8251434-995c-11ef-a34a-bc24111695cc-osd-48[24522]:
  417.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!