API tools faq
paste
Advertisement
Minerjirou

Fess-Crawler-log

Minerjirou
Jan 11th, 2024
10
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 31.42 KB | None | 0 0
raw download clone embed print report
  1. 2024-01-12 10:53:19,035 [main] INFO ...Loading specified properties and get by the key: fess_env_crawler.properties, lasta_di.smart.deploy.mode
  2. 2024-01-12 10:53:19,039 [main] INFO ...Setting smart deploy mode: warm
  3. 2024-01-12 10:53:19,051 [main] INFO ...Reading lasta_di.xml
  4. 2024-01-12 10:53:19,074 [main] INFO ...Reading redefiner.xml
  5. 2024-01-12 10:53:19,121 [main] INFO ...Reading smartdeploy.xml
  6. 2024-01-12 10:53:19,129 [main] INFO ...Reading smart/warmdeploy.xml
  7. 2024-01-12 10:53:19,132 [main] INFO ...Reading convention.xml
  8. 2024-01-12 10:53:19,136 [main] INFO ...Reading embedded_convention.xml
  9. 2024-01-12 10:53:19,148 [main] INFO ...Reading creator.xml
  10. 2024-01-12 10:53:19,152 [main] INFO ...Reading convention.xml (recycle)
  11. 2024-01-12 10:53:19,152 [main] INFO ...Reading customizer.xml
  12. 2024-01-12 10:53:19,159 [main] INFO ...Reading lastafw_customizer.xml
  13. 2024-01-12 10:53:19,171 [main] INFO ...Reading embedded_customizer.xml
  14. 2024-01-12 10:53:19,174 [main] INFO ...Reading tx_customizer.xml
  15. 2024-01-12 10:53:19,188 [main] INFO ...Reading my_creator.xml
  16. 2024-01-12 10:53:19,192 [main] INFO ...Reading lastafw_creator.xml
  17. 2024-01-12 10:53:19,195 [main] INFO ...Reading convention.xml (recycle)
  18. 2024-01-12 10:53:19,195 [main] INFO ...Reading customizer.xml (recycle)
  19. 2024-01-12 10:53:19,209 [main] INFO ...Reading embedded_creator.xml
  20. 2024-01-12 10:53:19,213 [main] INFO ...Reading convention.xml (recycle)
  21. 2024-01-12 10:53:19,213 [main] INFO ...Reading customizer.xml (recycle)
  22. 2024-01-12 10:53:19,222 [main] INFO ...Reading customizer.xml (recycle)
  23. 2024-01-12 10:53:19,263 [main] INFO ...Reading app.xml
  24. 2024-01-12 10:53:19,268 [main] INFO ...Reading convention.xml
  25. 2024-01-12 10:53:19,273 [main] INFO ...Reading embedded_convention.xml
  26. 2024-01-12 10:53:19,278 [main] INFO ...Reading lastaflute_core.xml
  27. 2024-01-12 10:53:19,283 [main] INFO ...Reading lastaflute_assist.xml
  28. 2024-01-12 10:53:19,287 [main] INFO ...Reading lastaflute_director.xml
  29. 2024-01-12 10:53:19,389 [main] INFO ...Reading fess.xml
  30. 2024-01-12 10:53:19,393 [main] INFO ...Reading fess_config.xml
  31. 2024-01-12 10:53:19,399 [main] INFO ...Reading fess_ds.xml
  32. 2024-01-12 10:53:19,408 [main] INFO ...Reading fess_se.xml
  33. 2024-01-12 10:53:19,413 [main] INFO ...Reading jar:file:/usr/share/fess/app/WEB-INF/plugin/fess-script-groovy-14.11.0.jar!/fess_se++.xml
  34. 2024-01-12 10:53:19,417 [main] INFO ...Reading esflute_config.xml
  35. 2024-01-12 10:53:19,420 [main] INFO ...Reading esclient.xml
  36. 2024-01-12 10:53:19,603 [main] INFO ...Reading esflute_user.xml
  37. 2024-01-12 10:53:19,606 [main] INFO ...Reading esclient.xml (recycle)
  38. 2024-01-12 10:53:19,622 [main] INFO ...Reading esflute_log.xml
  39. 2024-01-12 10:53:19,625 [main] INFO ...Reading esclient.xml (recycle)
  40. 2024-01-12 10:53:20,429 [main] INFO ...Reading crawler_es.xml
  41. 2024-01-12 10:53:20,432 [main] INFO ...Reading crawler/container.xml
  42. 2024-01-12 10:53:20,443 [main] INFO ...Reading crawler/client.xml
  43. 2024-01-12 10:53:20,446 [main] INFO ...Reading crawler/container.xml (recycle)
  44. 2024-01-12 10:53:20,446 [main] INFO ...Reading crawler/robotstxt.xml
  45. 2024-01-12 10:53:20,448 [main] INFO ...Reading crawler/container.xml (recycle)
  46. 2024-01-12 10:53:20,452 [main] INFO ...Reading crawler/contentlength.xml
  47. 2024-01-12 10:53:20,454 [main] INFO ...Reading crawler/container.xml (recycle)
  48. 2024-01-12 10:53:20,455 [main] INFO ...Reading crawler/mimetype.xml
  49. 2024-01-12 10:53:20,457 [main] INFO ...Reading crawler/container.xml (recycle)
  50. 2024-01-12 10:53:20,502 [main] INFO ...Reading jar:file:/usr/share/fess/app/WEB-INF/lib/fess-crawler-playwright-14.11.0.jar!/crawler/client++.xml
  51. 2024-01-12 10:53:20,504 [main] INFO ...Reading crawler/container.xml
  52. 2024-01-12 10:53:20,516 [main] INFO ...Reading crawler/rule.xml
  53. 2024-01-12 10:53:20,518 [main] INFO ...Reading crawler/container.xml (recycle)
  54. 2024-01-12 10:53:20,519 [main] INFO ...Reading crawler/transformer.xml
  55. 2024-01-12 10:53:20,521 [main] INFO ...Reading crawler/transformer_basic.xml
  56. 2024-01-12 10:53:20,524 [main] INFO ...Reading crawler/container.xml (recycle)
  57. 2024-01-12 10:53:20,555 [main] INFO ...Reading crawler/filter.xml
  58. 2024-01-12 10:53:20,557 [main] INFO ...Reading crawler/container.xml (recycle)
  59. 2024-01-12 10:53:20,560 [main] INFO ...Reading crawler/interval.xml
  60. 2024-01-12 10:53:20,563 [main] INFO ...Reading crawler/container.xml (recycle)
  61. 2024-01-12 10:53:20,565 [main] INFO ...Reading crawler/extractor.xml
  62. 2024-01-12 10:53:20,568 [main] INFO ...Reading crawler/container.xml (recycle)
  63. 2024-01-12 10:53:20,573 [main] INFO ...Reading crawler/extractor+tikaExtractor.xml
  64. 2024-01-12 10:53:20,575 [main] INFO ...Reading crawler/container.xml
  65. 2024-01-12 10:53:20,612 [main] INFO ...Reading crawler/mimetype.xml (recycle)
  66. 2024-01-12 10:53:20,612 [main] INFO ...Reading crawler/encoding.xml
  67. 2024-01-12 10:53:20,614 [main] INFO ...Reading crawler/container.xml (recycle)
  68. 2024-01-12 10:53:20,615 [main] INFO ...Reading crawler/urlconverter.xml
  69. 2024-01-12 10:53:20,618 [main] INFO ...Reading crawler/container.xml (recycle)
  70. 2024-01-12 10:53:20,619 [main] INFO ...Reading crawler/log.xml
  71. 2024-01-12 10:53:20,621 [main] INFO ...Reading crawler/container.xml (recycle)
  72. 2024-01-12 10:53:20,624 [main] INFO ...Reading crawler/sitemaps.xml
  73. 2024-01-12 10:53:20,626 [main] INFO ...Reading crawler/container.xml (recycle)
  74. 2024-01-12 10:53:20,630 [main] INFO ...Reading crawler/es.xml
  75. 2024-01-12 10:53:20,644 [main] INFO ...Reading crawler_es+crawlerThread.xml
  76. 2024-01-12 10:53:20,664 [main] INFO ...Reading crawler_es+crawlerConfig.xml
  77. 2024-01-12 10:53:20,668 [main] INFO ...Reading fess_thumbnail.xml
  78. 2024-01-12 10:53:20,679 [main] INFO ...Reading fess_ingest.xml
  79. 2024-01-12 10:53:20,882 [main] INFO [Objective Config]
  80. 2024-01-12 10:53:20,883 [main] INFO fess_config.properties extends [fess_env_crawler.properties]
  81. 2024-01-12 10:53:20,883 [main] INFO checkImplicitOverride=true, propertyCount=533
  82. 2024-01-12 10:53:20,914 [main] INFO [Exception Translator]
  83. 2024-01-12 10:53:20,914 [main] INFO exceptionTranslationProvider: null
  84. 2024-01-12 10:53:20,914 [main] INFO [Async Manager]
  85. 2024-01-12 10:53:20,914 [main] INFO defaultConcurrentAsyncOption: {no option}
  86. 2024-01-12 10:53:20,915 [main] INFO primaryExecutorService: ThreadPoolExecutor@73fb1d7f
  87. 2024-01-12 10:53:20,915 [main] INFO secondaryExecutorService: ThreadPoolExecutor@73d4066e
  88. 2024-01-12 10:53:20,915 [main] INFO tertiaryExecutorService: ThreadPoolExecutor@25d2f66
  89. 2024-01-12 10:53:20,915 [main] INFO [Primary Cipher]
  90. 2024-01-12 10:53:20,915 [main] INFO invertibleCryptographer: {AES, UTF-8}
  91. 2024-01-12 10:53:20,915 [main] INFO oneWayCryptographer: {SHA-256, UTF-8}
  92. 2024-01-12 10:53:20,916 [main] INFO [Time Manager]
  93. 2024-01-12 10:53:20,916 [main] INFO businessTimeHandler: TypicalBusinessTimeHandler:{TypicalTimeResourceProvider$$Lambda$172/0x00007f944a385e28, TypicalTimeResourceProvider$$Lambda$173/0x00007f944a386250}
  94. 2024-01-12 10:53:20,917 [main] INFO currentIgnoreTransaction: false
  95. 2024-01-12 10:53:20,917 [main] INFO adjustTimeMillis: 0
  96. 2024-01-12 10:53:20,917 [main] INFO adjustAbsoluteMode: false
  97. 2024-01-12 10:53:20,950 [main] INFO [JSON Manager]
  98. 2024-01-12 10:53:20,951 [main] INFO realJsonParser: GsonJsonEngine
  99. 2024-01-12 10:53:20,953 [main] INFO mapping: {CAMEL_TO_LOWER_SNAKE}
  100. 2024-01-12 10:53:20,954 [main] INFO [Postbox]
  101. 2024-01-12 10:53:20,954 [main] INFO postOffice: PostOffice@44536de4
  102. 2024-01-12 10:53:20,954 [main] INFO postalParkingLot: SMailPostalParkingLot:{{category:{main}=motorbike:{session=javax.mail.Session@5fcfde70}}}@4d95a72e
  103. 2024-01-12 10:53:20,954 [main] INFO postalPersonnel: FessMailDeliveryDepartmentCreator$1:{SMailConventionReceptionist:{[]}@28da7d11, batch:{[proofreader:{pmcomment}]}}@77b919a3
  104. 2024-01-12 10:53:22,746 [main] INFO Lasta Di boot successfully.
  105. 2024-01-12 10:53:22,748 [main] INFO SmartDeploy Mode: Warm Deploy
  106. 2024-01-12 10:53:22,748 [main] INFO Smart Package: org.codelibs.fess.app
  107. 2024-01-12 10:53:22,929 [main] INFO Starting Crawler..
  108. 2024-01-12 10:53:23,022 [DataStoreCrawler] INFO No crawling target urls.
  109. 2024-01-12 10:53:23,261 [WebFsCrawler] INFO Target URL: https://park.ajinomoto.co.jp/sitemap.xml
  110. 2024-01-12 10:53:23,311 [WebFsCrawler] INFO Target URL: https://park.ajinomoto.co.jp/sitemap-pt-recipe_card-.*
  111. 2024-01-12 10:53:23,411 [WebFsCrawler] INFO Target URL: https://park.ajinomoto.co.jp/recipe/card/.*
  112. 2024-01-12 10:53:23,413 [WebFsCrawler] INFO Included URL: https://park.ajinomoto.co.jp/recipe/card/.*
  113. 2024-01-12 10:53:23,415 [WebFsCrawler] INFO Excluded URL: (?i).*(css|js|jpeg|jpg|gif|png|bmp|wmv|xml|ico|exe)
  114. 2024-01-12 10:53:29,233 [Crawler-20240112105318-1-1] INFO Crawling URL: https://park.ajinomoto.co.jp/recipe/card/.*
  115. 2024-01-12 10:53:33,458 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 7ms}, Mem:{used 150MB, heap 378MB, max 512MB})
  116. 2024-01-12 10:53:43,446 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 151MB, heap 378MB, max 512MB})
  117. 2024-01-12 10:53:53,446 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 152MB, heap 378MB, max 512MB})
  118. 2024-01-12 10:54:03,444 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 6ms}, Mem:{used 153MB, heap 378MB, max 512MB})
  119. 2024-01-12 10:54:13,450 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 7ms}, Mem:{used 140MB, heap 378MB, max 512MB})
  120. 2024-01-12 10:54:22,901 [pool-5-thread-1] INFO [SYSTEM MONITOR] {"os":{"memory":{"physical":{"free":451698688,"total":3971424256},"swap_space":{"free":3941261312,"total":3970953216}},"cpu":{"percent":1},"load_averages":[0.37, 0.31, 0.32]},"process":{"file_descriptor":{"open":241,"max":65535},"cpu":{"percent":0,"total":16240},"virtual_memory":{"total":3260874752}},"jvm":{"memory":{"heap":{"used":148535296,"committed":396361728,"max":536870912,"percent":27},"non_heap":{"used":77189360,"committed":80871424}},"pools":{"mapped":{"count":0,"used":0,"capacity":0},"direct":{"count":11,"used":90112,"capacity":90112},"mapped - 'non-volatile memory'":{"count":0,"used":0,"capacity":0}},"gc":{"young":{"count":57,"time":192},"old":{"count":0,"time":0}},"threads":{"count":29,"peak":29},"classes":{"loaded":10329,"total_loaded":10331,"unloaded":2},"uptime":64692},"search_engine":{"nodes":{"Z_bWfIeDRsOfU5iK2xZEAQ":{"timestamp":1705024462850,"name":"miko","transport_address":"127.0.0.1:9300","host":"127.0.0.1","ip":"127.0.0.1:9300","roles":[],"os":{"timestamp":1705024462851,"cpu":{"percent":6,"load_average":{"1m":0.37,"5m":0.31,"15m":0.32}},"mem":{"total_in_bytes":3971424256,"free_in_bytes":450809856,"used_in_bytes":3520614400,"free_percent":11,"used_percent":89},"swap":{"total_in_bytes":3970953216,"free_in_bytes":3941261312,"used_in_bytes":29691904}},"process":{"timestamp":1705024462851,"open_file_descriptors":899,"max_file_descriptors":65535,"cpu":{"percent":0,"total_in_millis":124750},"mem":{"total_virtual_in_bytes":4991115264}},"jvm":{"timestamp":1705024462852,"uptime_in_millis":1896287,"mem":{"heap_used_in_bytes":209451008,"heap_used_percent":19,"heap_committed_in_bytes":1073741824,"heap_max_in_bytes":1073741824,"non_heap_used_in_bytes":183715016,"non_heap_committed_in_bytes":187826176,"pools":{"young":{"used_in_bytes":90177536,"max_in_bytes":0,"peak_used_in_bytes":607125504,"peak_max_in_bytes":0,"last_gc_stats":{"used_in_bytes":0,"max_in_bytes":0,"usage_percent":-1}},"old":{"used_in_bytes":81524736,"max_in_bytes":1073741824,"peak_used_in_bytes":81524736,"peak_max_in_bytes":1073741824,"last_gc_stats":{"used_in_bytes":0,"max_in_bytes":1073741824,"usage_percent":0}},"survivor":{"used_in_bytes":37748736,"max_in_bytes":0,"peak_used_in_bytes":66584576,"peak_max_in_bytes":0,"last_gc_stats":{"used_in_bytes":37748736,"max_in_bytes":0,"usage_percent":-1}}}},"threads":{"count":71,"peak_count":77},"gc":{"collectors":{"young":{"collection_count":19,"collection_time_in_millis":501},"old":{"collection_count":0,"collection_time_in_millis":0}}},"buffer_pools":{"mapped":{"count":64,"used_in_bytes":322488,"total_capacity_in_bytes":322488},"direct":{"count":54,"used_in_bytes":12588761,"total_capacity_in_bytes":12588760},"mapped - 'non-volatile memory'":{"count":0,"used_in_bytes":0,"total_capacity_in_bytes":0}},"classes":{"current_loaded_count":24479,"total_loaded_count":0,"total_unloaded_count":0}},"thread_pool":{"_plugin_geospatial_ip2geo_datasource_update":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"ad-batch-task-threadpool":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"ad-threadpool":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"analyze":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"fetch_shard_started":{"threads":1,"queue":0,"active":0,"rejected":0,"largest":8,"completed":90,"total_wait_time_in_nanos":0},"fetch_shard_store":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"flush":{"threads":1,"queue":0,"active":0,"rejected":0,"largest":2,"completed":102,"total_wait_time_in_nanos":0},"force_merge":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"generic":{"threads":11,"queue":0,"active":0,"rejected":0,"largest":11,"completed":9120,"total_wait_time_in_nanos":0},"get":{"threads":4,"queue":0,"active":0,"rejected":0,"largest":4,"completed":17,"total_wait_time_in_nanos":0},"listener":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"management":{"threads":5,"queue":0,"active":1,"rejected":0,"largest":5,"completed":9608,"total_wait_time_in_nanos":0},"open_distro_job_scheduler":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_asynchronous_search_generic":{"threads":2,"queue":0,"active":0,"rejected":0,"largest":2,"completed":10,"total_wait_time_in_nanos":0},"opensearch_ml_deploy":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_execute":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_general":{"threads":3,"queue":0,"active":0,"rejected":0,"largest":3,"completed":187,"total_wait_time_in_nanos":0},"opensearch_ml_predict":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_register":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_train":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"refresh":{"threads":2,"queue":0,"active":0,"rejected":0,"largest":2,"completed":78056,"total_wait_time_in_nanos":0},"remote_purge":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"remote_recovery":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"remote_refresh_retry":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"replication_follower":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"replication_leader":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"search":{"threads":7,"queue":0,"active":0,"rejected":0,"largest":7,"completed":3475,"total_wait_time_in_nanos":513956057},"search_throttled":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"snapshot":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"sql-worker":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"system_read":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"system_write":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"training":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"translog_sync":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"translog_transfer":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"warmer":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"write":{"threads":4,"queue":0,"active":0,"rejected":0,"largest":4,"completed":55,"total_wait_time_in_nanos":0}},"fs":{"timestamp":1705024462852,"total":{"total_in_bytes":105089261568,"free_in_bytes":92946145280,"available_in_bytes":87560658944,"cache_reserved_in_bytes":0},"data":[{"mount":"/ (/dev/mapper/ubuntu--vg-ubuntu--lv)","total_in_bytes":105089261568,"free_in_bytes":92946145280,"available_in_bytes":87560658944,"cache_utilized":0}]},"transport":{"server_open":0,"total_outbound_connections":0,"rx_count":0,"rx_size_in_bytes":0,"tx_count":0,"tx_size_in_bytes":0},"http":{"current_open":1,"total_opened":228},"breakers":{"request":{"limit_size_in_bytes":644245094,"limit_size":"614.3mb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.0,"tripped":0},"fielddata":{"limit_size_in_bytes":429496729,"limit_size":"409.5mb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.03,"tripped":0},"in_flight_requests":{"limit_size_in_bytes":1073741824,"limit_size":"1gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":2.0,"tripped":0},"parent":{"limit_size_in_bytes":1020054732,"limit_size":"972.7mb","estimated_size_in_bytes":209451008,"estimated_size":"199.7mb","overhead":1.0,"tripped":0}},"script":{"compilations":0,"cache_evictions":0,"compilation_limit_triggered":0},"discovery":{"cluster_state_queue":{"total":0,"pending":0,"committed":0},"published_cluster_states":{"full_states":2,"incompatible_diffs":0,"compatible_diffs":71}},"ingest":{"total":{"count":0,"time_in_millis":0,"current":0,"failed":0},"pipelines":{}},"adaptive_selection":{},"script_cache":{"sum":{"compilations":0,"cache_evictions":0,"compilation_limit_triggered":0},"contexts":[]},"indexing_pressure":{"memory":{"current":{"combined_coordinating_and_primary_in_bytes":0,"coordinating_in_bytes":0,"primary_in_bytes":0,"replica_in_bytes":0,"all_in_bytes":0},"total":{"combined_coordinating_and_primary_in_bytes":0,"coordinating_in_bytes":0,"primary_in_bytes":0,"replica_in_bytes":0,"all_in_bytes":0,"coordinating_rejections":0,"primary_rejections":0,"replica_rejections":0},"limit_in_bytes":0}},"shard_indexing_pressure":{"stats":{},"total_rejections_breakup_shadow_mode":{"node_limits":0,"no_successful_request_limits":0,"throughput_degradation_limits":0},"enabled":false,"enforced":false},"search_backpressure":{"search_task":{"resource_tracker_stats":{},"cancellation_stats":{"cancellation_count":0,"cancellation_limit_reached_count":0}},"search_shard_task":{"resource_tracker_stats":{},"cancellation_stats":{"cancellation_count":0,"cancellation_limit_reached_count":0}},"mode":"disabled"}},"cancellation_stats":{"timestamp":0,"name":"","transport_address":"0.0.0.0:0","host":"0.0.0.0","ip":"0.0.0.0:0","roles":[]}}},"timestamp":"2024-01-12T01:54:22.901Z"}
  121. 2024-01-12 10:54:23,452 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 146MB, heap 378MB, max 512MB})
  122. 2024-01-12 10:54:33,454 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 146MB, heap 378MB, max 512MB})
  123. 2024-01-12 10:54:43,455 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 147MB, heap 378MB, max 512MB})
  124. 2024-01-12 10:54:53,457 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 148MB, heap 378MB, max 512MB})
  125. 2024-01-12 10:55:03,457 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 9ms}, Mem:{used 148MB, heap 378MB, max 512MB})
  126. 2024-01-12 10:55:13,456 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 7ms}, Mem:{used 149MB, heap 378MB, max 512MB})
  127. 2024-01-12 10:55:23,458 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 8ms}, Mem:{used 149MB, heap 378MB, max 512MB})
  128. 2024-01-12 10:55:23,810 [pool-5-thread-2] INFO [SYSTEM MONITOR] {"os":{"memory":{"physical":{"free":453734400,"total":3971424256},"swap_space":{"free":3929956352,"total":3970953216}},"cpu":{"percent":2},"load_averages":[0.25, 0.28, 0.31]},"process":{"file_descriptor":{"open":240,"max":65535},"cpu":{"percent":0,"total":16710},"virtual_memory":{"total":3260874752}},"jvm":{"memory":{"heap":{"used":155875328,"committed":396361728,"max":536870912,"percent":29},"non_heap":{"used":78428776,"committed":81854464}},"pools":{"mapped":{"count":0,"used":0,"capacity":0},"direct":{"count":10,"used":81920,"capacity":81920},"mapped - 'non-volatile memory'":{"count":0,"used":0,"capacity":0}},"gc":{"young":{"count":57,"time":192},"old":{"count":0,"time":0}},"threads":{"count":27,"peak":29},"classes":{"loaded":10477,"total_loaded":10479,"unloaded":2},"uptime":125670},"search_engine":{"nodes":{"Z_bWfIeDRsOfU5iK2xZEAQ":{"timestamp":1705024523798,"name":"miko","transport_address":"127.0.0.1:9300","host":"127.0.0.1","ip":"127.0.0.1:9300","roles":[],"os":{"timestamp":1705024523800,"cpu":{"percent":5,"load_average":{"1m":0.25,"5m":0.28,"15m":0.31}},"mem":{"total_in_bytes":3971424256,"free_in_bytes":453734400,"used_in_bytes":3517689856,"free_percent":11,"used_percent":89},"swap":{"total_in_bytes":3970953216,"free_in_bytes":3929956352,"used_in_bytes":40996864}},"process":{"timestamp":1705024523800,"open_file_descriptors":899,"max_file_descriptors":65535,"cpu":{"percent":0,"total_in_millis":126650},"mem":{"total_virtual_in_bytes":4991115264}},"jvm":{"timestamp":1705024523801,"uptime_in_millis":1957237,"mem":{"heap_used_in_bytes":388757504,"heap_used_percent":36,"heap_committed_in_bytes":1073741824,"heap_max_in_bytes":1073741824,"non_heap_used_in_bytes":183462752,"non_heap_committed_in_bytes":187957248,"pools":{"young":{"used_in_bytes":269484032,"max_in_bytes":0,"peak_used_in_bytes":607125504,"peak_max_in_bytes":0,"last_gc_stats":{"used_in_bytes":0,"max_in_bytes":0,"usage_percent":-1}},"old":{"used_in_bytes":81524736,"max_in_bytes":1073741824,"peak_used_in_bytes":81524736,"peak_max_in_bytes":1073741824,"last_gc_stats":{"used_in_bytes":0,"max_in_bytes":1073741824,"usage_percent":0}},"survivor":{"used_in_bytes":37748736,"max_in_bytes":0,"peak_used_in_bytes":66584576,"peak_max_in_bytes":0,"last_gc_stats":{"used_in_bytes":37748736,"max_in_bytes":0,"usage_percent":-1}}}},"threads":{"count":71,"peak_count":77},"gc":{"collectors":{"young":{"collection_count":19,"collection_time_in_millis":501},"old":{"collection_count":0,"collection_time_in_millis":0}}},"buffer_pools":{"mapped":{"count":64,"used_in_bytes":322488,"total_capacity_in_bytes":322488},"direct":{"count":54,"used_in_bytes":12588761,"total_capacity_in_bytes":12588760},"mapped - 'non-volatile memory'":{"count":0,"used_in_bytes":0,"total_capacity_in_bytes":0}},"classes":{"current_loaded_count":24479,"total_loaded_count":0,"total_unloaded_count":0}},"thread_pool":{"_plugin_geospatial_ip2geo_datasource_update":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"ad-batch-task-threadpool":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"ad-threadpool":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"analyze":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"fetch_shard_started":{"threads":1,"queue":0,"active":0,"rejected":0,"largest":8,"completed":90,"total_wait_time_in_nanos":0},"fetch_shard_store":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"flush":{"threads":1,"queue":0,"active":0,"rejected":0,"largest":2,"completed":102,"total_wait_time_in_nanos":0},"force_merge":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"generic":{"threads":11,"queue":0,"active":0,"rejected":0,"largest":11,"completed":9290,"total_wait_time_in_nanos":0},"get":{"threads":4,"queue":0,"active":0,"rejected":0,"largest":4,"completed":17,"total_wait_time_in_nanos":0},"listener":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"management":{"threads":5,"queue":0,"active":1,"rejected":0,"largest":5,"completed":9919,"total_wait_time_in_nanos":0},"open_distro_job_scheduler":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_asynchronous_search_generic":{"threads":2,"queue":0,"active":0,"rejected":0,"largest":2,"completed":10,"total_wait_time_in_nanos":0},"opensearch_ml_deploy":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_execute":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_general":{"threads":3,"queue":0,"active":0,"rejected":0,"largest":3,"completed":193,"total_wait_time_in_nanos":0},"opensearch_ml_predict":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_register":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"opensearch_ml_train":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"refresh":{"threads":2,"queue":0,"active":0,"rejected":0,"largest":2,"completed":80623,"total_wait_time_in_nanos":0},"remote_purge":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"remote_recovery":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"remote_refresh_retry":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"replication_follower":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"replication_leader":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"search":{"threads":7,"queue":0,"active":0,"rejected":0,"largest":7,"completed":3804,"total_wait_time_in_nanos":546372457},"search_throttled":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"snapshot":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"sql-worker":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"system_read":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"system_write":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"training":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"translog_sync":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"translog_transfer":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"warmer":{"threads":0,"queue":0,"active":0,"rejected":0,"largest":0,"completed":0,"total_wait_time_in_nanos":0},"write":{"threads":4,"queue":0,"active":0,"rejected":0,"largest":4,"completed":55,"total_wait_time_in_nanos":0}},"fs":{"timestamp":1705024523801,"total":{"total_in_bytes":105089261568,"free_in_bytes":92946120704,"available_in_bytes":87560634368,"cache_reserved_in_bytes":0},"data":[{"mount":"/ (/dev/mapper/ubuntu--vg-ubuntu--lv)","total_in_bytes":105089261568,"free_in_bytes":92946120704,"available_in_bytes":87560634368,"cache_utilized":0}]},"transport":{"server_open":0,"total_outbound_connections":0,"rx_count":0,"rx_size_in_bytes":0,"tx_count":0,"tx_size_in_bytes":0},"http":{"current_open":1,"total_opened":239},"breakers":{"request":{"limit_size_in_bytes":644245094,"limit_size":"614.3mb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.0,"tripped":0},"fielddata":{"limit_size_in_bytes":429496729,"limit_size":"409.5mb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":1.03,"tripped":0},"in_flight_requests":{"limit_size_in_bytes":1073741824,"limit_size":"1gb","estimated_size_in_bytes":0,"estimated_size":"0b","overhead":2.0,"tripped":0},"parent":{"limit_size_in_bytes":1020054732,"limit_size":"972.7mb","estimated_size_in_bytes":388757504,"estimated_size":"370.7mb","overhead":1.0,"tripped":0}},"script":{"compilations":0,"cache_evictions":0,"compilation_limit_triggered":0},"discovery":{"cluster_state_queue":{"total":0,"pending":0,"committed":0},"published_cluster_states":{"full_states":2,"incompatible_diffs":0,"compatible_diffs":71}},"ingest":{"total":{"count":0,"time_in_millis":0,"current":0,"failed":0},"pipelines":{}},"adaptive_selection":{},"script_cache":{"sum":{"compilations":0,"cache_evictions":0,"compilation_limit_triggered":0},"contexts":[]},"indexing_pressure":{"memory":{"current":{"combined_coordinating_and_primary_in_bytes":0,"coordinating_in_bytes":0,"primary_in_bytes":0,"replica_in_bytes":0,"all_in_bytes":0},"total":{"combined_coordinating_and_primary_in_bytes":0,"coordinating_in_bytes":0,"primary_in_bytes":0,"replica_in_bytes":0,"all_in_bytes":0,"coordinating_rejections":0,"primary_rejections":0,"replica_rejections":0},"limit_in_bytes":0}},"shard_indexing_pressure":{"stats":{},"total_rejections_breakup_shadow_mode":{"node_limits":0,"no_successful_request_limits":0,"throughput_degradation_limits":0},"enabled":false,"enforced":false},"search_backpressure":{"search_task":{"resource_tracker_stats":{},"cancellation_stats":{"cancellation_count":0,"cancellation_limit_reached_count":0}},"search_shard_task":{"resource_tracker_stats":{},"cancellation_stats":{"cancellation_count":0,"cancellation_limit_reached_count":0}},"mode":"disabled"}},"cancellation_stats":{"timestamp":0,"name":"","transport_address":"0.0.0.0:0","host":"0.0.0.0","ip":"0.0.0.0:0","roles":[]}}},"timestamp":"2024-01-12T01:55:23.810Z"}
  129. 2024-01-12 10:55:25,012 [WebFsCrawler] INFO [EXEC TIME] crawling time: 121969ms
  130. 2024-01-12 10:55:33,460 [IndexUpdater] INFO Processing no docs in indexing queue (Doc:{access 9ms}, Mem:{used 151MB, heap 378MB, max 512MB})
  131. 2024-01-12 10:55:33,460 [IndexUpdater] INFO [EXEC TIME] index update time: 123ms
  132. 2024-01-12 10:55:33,603 [main] INFO Finished Crawler
  133. 2024-01-12 10:55:33,704 [main] INFO [CRAWL INFO] DataCrawlEndTime=2024-01-12T10:53:23.022+0900,CrawlerEndTime=2024-01-12T10:55:33.603+0900,WebFsCrawlExecTime=121969,CrawlerStatus=true,CrawlerStartTime=2024-01-12T10:53:22.929+0900,WebFsCrawlEndTime=2024-01-12T10:55:33.603+0900,WebFsIndexExecTime=123,WebFsIndexSize=0,CrawlerExecTime=130675,DataCrawlStartTime=2024-01-12T10:53:22.976+0900,WebFsCrawlStartTime=2024-01-12T10:53:22.975+0900
  134. 2024-01-12 10:55:33,724 [main] INFO Disconnected to http://localhost:9200
  135. 2024-01-12 10:55:33,727 [main] INFO Destroyed LaContainer.
  136.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!