API tools faq
paste
Advertisement
Guest User

node8

a guest
Feb 25th, 2024
23
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 76.97 KB | None | 0 0
raw download clone embed print report
  1.  
  2.  
  3. Let's do step by step, because it is still not working :(
  4.  
  5.  
  6. docker stop storagenode8
  7. storagenode8
  8.  
  9. docker rm storagenode8
  10. storagenode8
  11.  
  12. cd /home/node/storj/8/data8/storage
  13.  
  14.  
  15. node@node8:data8/storage $ ls -la
  16. total 856852
  17. drwxrwxr-x 6 775 node 4096 Feb 25 09:24 .
  18. drwxrwxr-x 4 node node 4096 Feb 25 09:24 ..
  19. -rw-r--r-- 1 root root 32768 Feb 25 09:24 bandwidth.db
  20. drwxrwxr-x 9 node node 4096 Jan 22 2021 blobs
  21. drwxrwxr-x 2 node node 4096 Feb 19 10:51 garbage
  22. -rw-r--r-- 1 root root 32768 Feb 25 09:24 heldamount.db
  23. -rw-r--r-- 1 root root 16384 Feb 25 09:24 info.db
  24. -rw-r--r-- 1 root root 24576 Feb 25 09:24 notifications.db
  25. -rw-r--r-- 1 root root 761696256 Feb 25 09:24 orders.db
  26. -rw-r--r-- 1 root root 28106752 Feb 25 09:24 piece_expiration.db
  27. -rw-r--r-- 1 root root 32768 Feb 25 09:24 piece_expiration.db-shm
  28. -rw-r--r-- 1 root root 41232 Feb 25 09:24 piece_expiration.db-wal
  29. -rw-r--r-- 1 root root 24576 Feb 25 09:24 piece_spaced_used.db
  30. -rw-r--r-- 1 root root 24576 Feb 25 09:24 pieceinfo.db
  31. -rw-r--r-- 1 root root 24576 Feb 25 09:24 pricing.db
  32. -rw-r--r-- 1 root root 24576 Feb 25 09:24 reputation.db
  33. -rw-r--r-- 1 root root 32768 Feb 25 09:24 satellites.db
  34. -rw-r--r-- 1 root root 24576 Feb 25 09:24 secret.db
  35. -rwxrwxr-x 1 node node 32 Dec 5 2020 storage-dir-verification
  36. -rw-r--r-- 1 root root 1912832 Feb 25 09:24 storage_usage.db
  37. drwxrwxr-x 2 node node 12288 Feb 9 20:43 temp
  38. drwxrwxr-x 9 node node 4096 Feb 21 2021 trash
  39. -rw-r--r-- 1 root root 85299200 Feb 25 09:24 used_serial.db
  40. node@node8:data8/storage $
  41.  
  42.  
  43.  
  44. sudo rm *.db
  45. [sudo] password for node:
  46. node@node8:data8/storage $
  47.  
  48.  
  49. ls -la
  50. total 112
  51. drwxrwxr-x 6 775 node 4096 Feb 25 09:28 .
  52. drwxrwxr-x 4 node node 4096 Feb 25 09:24 ..
  53. drwxrwxr-x 9 node node 4096 Jan 22 2021 blobs
  54. drwxrwxr-x 2 node node 4096 Feb 19 10:51 garbage
  55. -rw-r--r-- 1 root root 32768 Feb 25 09:24 piece_expiration.db-shm
  56. -rw-r--r-- 1 root root 41232 Feb 25 09:24 piece_expiration.db-wal
  57. -rwxrwxr-x 1 node node 32 Dec 5 2020 storage-dir-verification
  58. drwxrwxr-x 2 node node 12288 Feb 9 20:43 temp
  59. drwxrwxr-x 9 node node 4096 Feb 21 2021 trash
  60. node@node8:data8/storage $
  61.  
  62.  
  63.  
  64. cd /home/node/storj/8/db_backup
  65. node@node8:8/db_backup $ ls -la
  66.  
  67. drwxrwxr-x 2 node node 4096 Feb 24 10:22 .
  68. drwxrwxr-x 6 node node 4096 Feb 25 09:09 ..
  69. -rwxr-xr-x 1 root root 86872064 Feb 23 15:05 bandwidth.db
  70. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 bandwidth.db-shm
  71. -rwxr-xr-x 1 node node 0 Feb 24 10:17 bandwidth.db-wal
  72. -rwxr-xr-x 1 node node 86872064 Feb 24 10:20 bandwidth.db.bak
  73. -rwxr-xr-x 1 root root 167936 Feb 23 15:05 heldamount.db
  74. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 heldamount.db-shm
  75. -rwxr-xr-x 1 node node 0 Feb 24 10:17 heldamount.db-wal
  76. -rwxr-xr-x 1 root root 16384 Feb 23 15:05 info.db
  77. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 info.db-shm
  78. -rwxr-xr-x 1 node node 0 Feb 24 10:17 info.db-wal
  79. -rwxr-xr-x 1 root root 24576 Feb 23 15:05 notifications.db
  80. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 notifications.db-shm
  81. -rwxr-xr-x 1 node node 0 Feb 24 10:17 notifications.db-wal
  82. -rwxr-xr-x 1 root root 761696256 Feb 23 15:07 orders.db
  83. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 orders.db-shm
  84. -rwxr-xr-x 1 node node 0 Feb 24 10:17 orders.db-wal
  85. -rwxr-xr-x 1 root root 28106752 Feb 23 15:07 piece_expiration.db
  86. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 piece_expiration.db-shm
  87. -rwxr-xr-x 1 node node 0 Feb 24 10:17 piece_expiration.db-wal
  88. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 piece_spaced_used.db
  89. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 piece_spaced_used.db-shm
  90. -rwxr-xr-x 1 node node 0 Feb 24 10:17 piece_spaced_used.db-wal
  91. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 pieceinfo.db
  92. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 pieceinfo.db-shm
  93. -rwxr-xr-x 1 node node 0 Feb 24 10:17 pieceinfo.db-wal
  94. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 pricing.db
  95. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 pricing.db-shm
  96. -rwxr-xr-x 1 node node 0 Feb 24 10:17 pricing.db-wal
  97. -rwxr-xr-x 1 root root 40960 Feb 23 15:07 reputation.db
  98. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 reputation.db-shm
  99. -rwxr-xr-x 1 node node 0 Feb 24 10:17 reputation.db-wal
  100. -rwxr-xr-x 1 root root 32768 Feb 23 15:07 satellites.db
  101. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 satellites.db-shm
  102. -rwxr-xr-x 1 node node 0 Feb 24 10:17 satellites.db-wal
  103. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 secret.db
  104. -rwxr-xr-x 1 node node 32768 Feb 24 10:18 secret.db-shm
  105. -rwxr-xr-x 1 node node 0 Feb 24 10:18 secret.db-wal
  106. -rwxr-xr-x 1 root root 1912832 Feb 23 15:07 storage_usage.db
  107. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 storage_usage.db-shm
  108. -rwxr-xr-x 1 node node 0 Feb 24 10:17 storage_usage.db-wal
  109. -rwxr-xr-x 1 root root 85299200 Feb 23 15:07 used_serial.db
  110. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 used_serial.db-shm
  111. -rwxr-xr-x 1 node node 0 Feb 24 10:17 used_serial.db-wal
  112.  
  113.  
  114.  
  115. cd /home/node/storj/8/data8/storage
  116. node@node8:data8/storage $ ls -la
  117. total 112
  118. drwxrwxr-x 6 775 node 4096 Feb 25 09:28 .
  119. drwxrwxr-x 4 node node 4096 Feb 25 09:24 ..
  120. drwxrwxr-x 9 node node 4096 Jan 22 2021 blobs
  121. drwxrwxr-x 2 node node 4096 Feb 19 10:51 garbage
  122. -rw-r--r-- 1 root root 32768 Feb 25 09:24 piece_expiration.db-shm
  123. -rw-r--r-- 1 root root 41232 Feb 25 09:24 piece_expiration.db-wal
  124. -rwxrwxr-x 1 node node 32 Dec 5 2020 storage-dir-verification
  125. drwxrwxr-x 2 node node 12288 Feb 9 20:43 temp
  126. drwxrwxr-x 9 node node 4096 Feb 21 2021 trash
  127. node@node8:data8/storage $
  128.  
  129.  
  130.  
  131. docker run -d --restart unless-stopped --stop-timeout 300 \
  132. -p 11008:28967/tcp \
  133. -p 11008:28967/udp \
  134. -p 14008:14002 \
  135. -e WALLET="xxxxxx" \
  136. -e EMAIL="xxxxxxx" \
  137. -e ADDRESS="xxxxxxx:11008" \
  138. -e STORAGE="4800GB" \
  139. --log-opt max-size=50m \
  140. --log-opt max-file=10 \
  141. --mount type=bind,source="/home/node/storj/8/cert8/",destination=/app/identity \
  142. --mount type=bind,source="/home/node/storj/8/data8/",destination=/app/config \
  143. --name storagenode8 storjlabs/storagenode:latest
  144.  
  145.  
  146.  
  147.  
  148. node@node8:~ $ docker logs -f --tail 40 storagenode8
  149. 2024-02-25T08:33:21Z ERROR piecestore:cache error getting current used space: {"process": "storagenode", "error": "filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled", "errorVerbose": "group:\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  150. 2024-02-25T08:33:21Z ERROR pieces:trash emptying trash failed {"process": "storagenode", "error": "pieces error: filestore error: context canceled", "errorVerbose": "pieces error: filestore error: context canceled\n\tstorj.io/storj/storagenode/blobstore/filestore.(*blobStore).EmptyTrash:176\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).EmptyTrash:316\n\tstorj.io/storj/storagenode/pieces.(*Store).EmptyTrash:416\n\tstorj.io/storj/storagenode/pieces.(*TrashChore).Run.func1.1:83\n\tstorj.io/common/sync2.(*Workplace).Start.func1:89"}
  151. Error: piecestore monitor: disk space requirement not met
  152. 2024-02-25 08:33:22,138 INFO exited: storagenode (exit status 1; not expected)
  153. 2024-02-25 08:33:22,143 INFO spawned: 'storagenode' with pid 53
  154. 2024-02-25 08:33:22,148 WARN received SIGQUIT indicating exit request
  155. 2024-02-25 08:33:22,149 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  156. 2024-02-25T08:33:22Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  157. 2024-02-25 08:33:22,158 INFO stopped: storagenode-updater (exit status 0)
  158. 2024-02-25 08:33:22,162 INFO stopped: storagenode (terminated by SIGTERM)
  159. 2024-02-25 08:33:22,164 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
  160. 2024-02-25 08:33:29,552 INFO Set uid to user 0 succeeded
  161. 2024-02-25 08:33:29,568 INFO RPC interface 'supervisor' initialized
  162. 2024-02-25 08:33:29,568 INFO supervisord started with pid 1
  163. 2024-02-25 08:33:30,573 INFO spawned: 'processes-exit-eventlistener' with pid 11
  164. 2024-02-25 08:33:30,580 INFO spawned: 'storagenode' with pid 12
  165. 2024-02-25 08:33:30,589 INFO spawned: 'storagenode-updater' with pid 13
  166. 2024-02-25T08:33:30Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
  167. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
  168. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
  169. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
  170. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
  171. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
  172. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
  173. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
  174. 2024-02-25T08:33:30Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.debug-log-traffic"}
  175. 2024-02-25T08:33:30Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.caller"}
  176. 2024-02-25T08:33:30Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.custom-level"}
  177. 2024-02-25T08:33:30Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.encoding"}
  178. 2024-02-25T08:33:30Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.level"}
  179. 2024-02-25T08:33:30Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.output"}
  180. 2024-02-25T08:33:30Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
  181. 2024-02-25T08:33:30Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.96.6"}
  182. 2024-02-25T08:33:30Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  183. 2024-02-25T08:33:30Z INFO Configuration loaded {"process": "storagenode", "Location": "/app/config/config.yaml"}
  184. 2024-02-25T08:33:30Z INFO Anonymized tracing enabled {"process": "storagenode"}
  185. 2024-02-25T08:33:30Z INFO Operator email {"process": "storagenode", "Address": "xxxxxx"}
  186. 2024-02-25T08:33:30Z INFO Operator wallet {"process": "storagenode", "Address": "xxxxxxx"}
  187. 2024-02-25T08:33:30Z INFO server kernel support for server-side tcp fast open remains disabled. {"process": "storagenode"}
  188. 2024-02-25T08:33:30Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"process": "storagenode"}
  189. 2024-02-25T08:33:31Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.96.6"}
  190. 2024-02-25T08:33:31Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode"}
  191. 2024-02-25T08:33:31Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.96.6"}
  192. 2024-02-25T08:33:31Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  193. 2024-02-25T08:33:31Z INFO Telemetry enabled {"process": "storagenode", "instance ID": "12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh"}
  194. 2024-02-25T08:33:31Z INFO Event collection enabled {"process": "storagenode", "instance ID": "12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh"}
  195. 2024-02-25T08:33:31Z INFO db.migration Database Version {"process": "storagenode", "version": 54}
  196. 2024-02-25T08:33:32Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"process": "storagenode"}
  197. 2024-02-25 08:33:32,038 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  198. 2024-02-25 08:33:32,038 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  199. 2024-02-25 08:33:32,038 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  200. 2024-02-25T08:33:32Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"process": "storagenode"}
  201. 2024-02-25T08:33:32Z INFO bandwidth Performing bandwidth usage rollups {"process": "storagenode"}
  202. 2024-02-25T08:33:32Z INFO Node 12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh started {"process": "storagenode"}
  203. 2024-02-25T08:33:32Z INFO Public server started on [::]:28967 {"process": "storagenode"}
  204. 2024-02-25T08:33:32Z INFO Private server started on 127.0.0.1:7778 {"process": "storagenode"}
  205. 2024-02-25T08:33:32Z INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details. {"process": "storagenode"}
  206. 2024-02-25T08:33:32Z WARN piecestore:monitor Disk space is less than requested. Allocated space is {"process": "storagenode", "bytes": 339310145536}
  207. 2024-02-25T08:33:32Z ERROR piecestore:monitor Total disk space is less than required minimum {"process": "storagenode", "bytes": 500000000000}
  208. 2024-02-25T08:33:32Z INFO trust Scheduling next refresh {"process": "storagenode", "after": "6h12m6.874561964s"}
  209. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  210. 2024-02-25T08:33:32Z INFO pieces:trash emptying trash started {"process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  211. 2024-02-25T08:33:32Z ERROR services unexpected shutdown of a runner {"process": "storagenode", "name": "piecestore:monitor", "error": "piecestore monitor: disk space requirement not met", "errorVerbose": "piecestore monitor: disk space requirement not met\n\tstorj.io/storj/storagenode/monitor.(*Service).Run:135\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  212. 2024-02-25T08:33:32Z ERROR nodestats:cache Get pricing-model/join date failed {"process": "storagenode", "error": "context canceled"}
  213. 2024-02-25T08:33:32Z ERROR version failed to get process version info {"process": "storagenode", "error": "version checker client: Get \"https://version.storj.io\": context canceled", "errorVerbose": "version checker client: Get \"https://version.storj.io\": context canceled\n\tstorj.io/storj/private/version/checker.(*Client).All:68\n\tstorj.io/storj/private/version/checker.(*Client).Process:108\n\tstorj.io/storj/private/version/checker.(*Service).checkVersion:104\n\tstorj.io/storj/private/version/checker.(*Service).CheckVersion:78\n\tstorj.io/storj/storagenode/version.(*Chore).checkVersion:110\n\tstorj.io/storj/storagenode/version.(*Chore).Run.func1:65\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/storj/storagenode/version.(*Chore).Run:64\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  214. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker subprocess started {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  215. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker subprocess exited with status {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "status": -1, "error": "signal: killed"}
  216. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: signal: killed", "errorVerbose": "lazyfilewalker: signal: killed\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:83\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  217. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  218. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "error": "context canceled"}
  219. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  220. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  221. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
  222. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  223. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  224. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "error": "context canceled"}
  225. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  226. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  227. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "error": "context canceled"}
  228. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  229. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  230. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "error": "context canceled"}
  231. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  232. 2024-02-25T08:33:32Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  233. 2024-02-25T08:33:32Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
  234. 2024-02-25T08:33:32Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  235. 2024-02-25T08:33:32Z ERROR piecestore:cache error getting current used space: {"process": "storagenode", "error": "filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled", "errorVerbose": "group:\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  236. 2024-02-25T08:33:32Z ERROR pieces:trash emptying trash failed {"process": "storagenode", "error": "pieces error: filestore error: context canceled", "errorVerbose": "pieces error: filestore error: context canceled\n\tstorj.io/storj/storagenode/blobstore/filestore.(*blobStore).EmptyTrash:176\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).EmptyTrash:316\n\tstorj.io/storj/storagenode/pieces.(*Store).EmptyTrash:416\n\tstorj.io/storj/storagenode/pieces.(*TrashChore).Run.func1.1:83\n\tstorj.io/common/sync2.(*Workplace).Start.func1:89"}
  237. Error: piecestore monitor: disk space requirement not met
  238. 2024-02-25 08:33:34,139 INFO exited: storagenode (exit status 1; not expected)
  239. 2024-02-25 08:33:35,147 INFO spawned: 'storagenode' with pid 50
  240. 2024-02-25 08:33:35,148 WARN received SIGQUIT indicating exit request
  241. 2024-02-25 08:33:35,149 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  242. 2024-02-25T08:33:35Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  243. 2024-02-25 08:33:35,160 INFO stopped: storagenode-updater (exit status 0)
  244. 2024-02-25 08:33:35,161 INFO stopped: storagenode (terminated by SIGTERM)
  245. 2024-02-25 08:33:35,163 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
  246.  
  247.  
  248.  
  249. cd /home/node/storj/8/data8/storage
  250.  
  251.  
  252. ls -la
  253. total 404
  254. drwxrwxr-x 6 775 node 4096 Feb 25 09:35 .
  255. drwxrwxr-x 4 node node 4096 Feb 25 09:35 ..
  256. -rw-r--r-- 1 root root 32768 Feb 25 09:35 bandwidth.db
  257. drwxrwxr-x 9 node node 4096 Jan 22 2021 blobs
  258. drwxrwxr-x 2 node node 4096 Feb 19 10:51 garbage
  259. -rw-r--r-- 1 root root 32768 Feb 25 09:35 heldamount.db
  260. -rw-r--r-- 1 root root 16384 Feb 25 09:35 info.db
  261. -rw-r--r-- 1 root root 24576 Feb 25 09:35 notifications.db
  262. -rw-r--r-- 1 root root 32768 Feb 25 09:35 orders.db
  263. -rw-r--r-- 1 root root 36864 Feb 25 09:35 piece_expiration.db
  264. -rw-r--r-- 1 root root 24576 Feb 25 09:35 piece_spaced_used.db
  265. -rw-r--r-- 1 root root 24576 Feb 25 09:35 pieceinfo.db
  266. -rw-r--r-- 1 root root 24576 Feb 25 09:35 pricing.db
  267. -rw-r--r-- 1 root root 24576 Feb 25 09:35 reputation.db
  268. -rw-r--r-- 1 root root 32768 Feb 25 09:35 satellites.db
  269. -rw-r--r-- 1 root root 24576 Feb 25 09:35 secret.db
  270. -rwxrwxr-x 1 node node 32 Dec 5 2020 storage-dir-verification
  271. -rw-r--r-- 1 root root 24576 Feb 25 09:35 storage_usage.db
  272. drwxrwxr-x 2 node node 12288 Feb 9 20:43 temp
  273. drwxrwxr-x 9 node node 4096 Feb 21 2021 trash
  274. -rw-r--r-- 1 root root 20480 Feb 25 09:35 used_serial.db
  275.  
  276.  
  277. docker stop -t 300 storagenode8
  278. storagenode8
  279. node@node8:~ $ docker rm storagenode8
  280. storagenode8
  281.  
  282.  
  283. cd /home/node/storj/8/db_backup
  284.  
  285.  
  286. node@node8:8/db_backup $ ls -la
  287. total 1027024
  288. drwxrwxr-x 2 node node 4096 Feb 24 10:22 .
  289. drwxrwxr-x 6 node node 4096 Feb 25 09:09 ..
  290. -rwxr-xr-x 1 root root 86872064 Feb 23 15:05 bandwidth.db
  291. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 bandwidth.db-shm
  292. -rwxr-xr-x 1 node node 0 Feb 24 10:17 bandwidth.db-wal
  293. -rwxr-xr-x 1 node node 86872064 Feb 24 10:20 bandwidth.db.bak
  294. -rwxr-xr-x 1 root root 167936 Feb 23 15:05 heldamount.db
  295. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 heldamount.db-shm
  296. -rwxr-xr-x 1 node node 0 Feb 24 10:17 heldamount.db-wal
  297. -rwxr-xr-x 1 root root 16384 Feb 23 15:05 info.db
  298. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 info.db-shm
  299. -rwxr-xr-x 1 node node 0 Feb 24 10:17 info.db-wal
  300. -rwxr-xr-x 1 root root 24576 Feb 23 15:05 notifications.db
  301. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 notifications.db-shm
  302. -rwxr-xr-x 1 node node 0 Feb 24 10:17 notifications.db-wal
  303. -rwxr-xr-x 1 root root 761696256 Feb 23 15:07 orders.db
  304. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 orders.db-shm
  305. -rwxr-xr-x 1 node node 0 Feb 24 10:17 orders.db-wal
  306. -rwxr-xr-x 1 root root 28106752 Feb 23 15:07 piece_expiration.db
  307. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 piece_expiration.db-shm
  308. -rwxr-xr-x 1 node node 0 Feb 24 10:17 piece_expiration.db-wal
  309. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 piece_spaced_used.db
  310. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 piece_spaced_used.db-shm
  311. -rwxr-xr-x 1 node node 0 Feb 24 10:17 piece_spaced_used.db-wal
  312. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 pieceinfo.db
  313. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 pieceinfo.db-shm
  314. -rwxr-xr-x 1 node node 0 Feb 24 10:17 pieceinfo.db-wal
  315. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 pricing.db
  316. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 pricing.db-shm
  317. -rwxr-xr-x 1 node node 0 Feb 24 10:17 pricing.db-wal
  318. -rwxr-xr-x 1 root root 40960 Feb 23 15:07 reputation.db
  319. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 reputation.db-shm
  320. -rwxr-xr-x 1 node node 0 Feb 24 10:17 reputation.db-wal
  321. -rwxr-xr-x 1 root root 32768 Feb 23 15:07 satellites.db
  322. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 satellites.db-shm
  323. -rwxr-xr-x 1 node node 0 Feb 24 10:17 satellites.db-wal
  324. -rwxr-xr-x 1 root root 24576 Feb 23 15:07 secret.db
  325. -rwxr-xr-x 1 node node 32768 Feb 24 10:18 secret.db-shm
  326. -rwxr-xr-x 1 node node 0 Feb 24 10:18 secret.db-wal
  327. -rwxr-xr-x 1 root root 1912832 Feb 23 15:07 storage_usage.db
  328. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 storage_usage.db-shm
  329. -rwxr-xr-x 1 node node 0 Feb 24 10:17 storage_usage.db-wal
  330. -rwxr-xr-x 1 root root 85299200 Feb 23 15:07 used_serial.db
  331. -rwxr-xr-x 1 node node 32768 Feb 24 10:17 used_serial.db-shm
  332. -rwxr-xr-x 1 node node 0 Feb 24 10:17 used_serial.db-wal
  333.  
  334.  
  335.  
  336. node@node8:8/db_backup $ sudo cp info.db /home/node/storj/8/data8/storage
  337. sudo cp notifications.db /home/node/storj/8/data8/storage
  338. sudo cp orders.db /home/node/storj/8/data8/storage
  339. sudo cp piece_expiration.db /home/node/storj/8/data8/storage
  340. sudo cp piece_expiration.db /home/node/storj/8/dat
  341. sudo cp pieceinfo.db /home/node/storj/8/data8/storage
  342. sudo cp storage_usage.db /home/node/storj/8/data8/storage
  343. sudo cp used_serial.db /home/node/storj/8/data8/storage
  344. sudo cp orders.db /home/node/storj/8/data8/storage
  345. sudo cp notifications.db /home/node/storj/8/data8/storage
  346. sudo cp notifications.db /home/node/storj/8/data8/storage
  347. sudo cp satellites.db /home/node/storj/8/data8/storage
  348. sudo cp pricing.db /home/node/storj/8/data8/storage
  349. [sudo] password for node:
  350. node@node8:8/db_backup $
  351.  
  352.  
  353.  
  354.  
  355.  
  356. docker run -d --restart unless-stopped --stop-timeout 300 \
  357. -p 11008:28967/tcp \
  358. -p 11008:28967/udp \
  359. -p 14008:14002 \
  360. -e WALLET="xxxxxxx" \
  361. -e EMAIL="xxxxxx" \
  362. -e ADDRESS="xxxxxx:11008" \
  363. -e STORAGE="4800GB" \
  364. --log-opt max-size=50m \
  365. --log-opt max-file=10 \
  366. --mount type=bind,source="/home/node/storj/8/cert8/",destination=/app/identity \
  367. --mount type=bind,source="/home/node/storj/8/data8/",destination=/app/config \
  368. --name storagenode8 storjlabs/storagenode:latest
  369.  
  370.  
  371.  
  372.  
  373.  
  374. node@node8:~ $ docker logs -f --tail 40 storagenode8
  375. 2024-02-25T08:43:11Z ERROR services unexpected shutdown of a runner {"process": "storagenode", "name": "piecestore:monitor", "error": "piecestore monitor: disk space requirement not met", "errorVerbose": "piecestore monitor: disk space requirement not met\n\tstorj.io/storj/storagenode/monitor.(*Service).Run:135\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  376. 2024-02-25T08:43:11Z ERROR nodestats:cache Get pricing-model/join date failed {"process": "storagenode", "error": "context canceled"}
  377. 2024-02-25T08:43:11Z ERROR version failed to get process version info {"process": "storagenode", "error": "version checker client: Get \"https://version.storj.io\": context canceled", "errorVerbose": "version checker client: Get \"https://version.storj.io\": context canceled\n\tstorj.io/storj/private/version/checker.(*Client).All:68\n\tstorj.io/storj/private/version/checker.(*Client).Process:108\n\tstorj.io/storj/private/version/checker.(*Service).checkVersion:104\n\tstorj.io/storj/private/version/checker.(*Service).CheckVersion:78\n\tstorj.io/storj/storagenode/version.(*Chore).checkVersion:110\n\tstorj.io/storj/storagenode/version.(*Chore).Run.func1:65\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/storj/storagenode/version.(*Chore).Run:64\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  378. 2024-02-25T08:43:11Z ERROR servers unexpected shutdown of a runner {"process": "storagenode", "name": "debug", "error": "debug: listener closed", "errorVerbose": "debug: listener closed\n\tstorj.io/drpc/drpcmigrate.init:17\n\truntime.doInit1:6740\n\truntime.doInit:6707\n\truntime.main:249"}
  379. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker subprocess exited with status {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "status": -1, "error": "signal: killed"}
  380. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: signal: killed", "errorVerbose": "lazyfilewalker: signal: killed\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:83\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  381. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  382. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "error": "context canceled"}
  383. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  384. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  385. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
  386. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  387. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  388. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "error": "context canceled"}
  389. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  390. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  391. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "error": "context canceled"}
  392. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  393. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  394. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "error": "context canceled"}
  395. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  396. 2024-02-25T08:43:11Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  397. 2024-02-25T08:43:11Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
  398. 2024-02-25T08:43:11Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  399. 2024-02-25T08:43:11Z ERROR piecestore:cache error getting current used space: {"process": "storagenode", "error": "filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled", "errorVerbose": "group:\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  400. 2024-02-25T08:43:11Z ERROR collector error during collecting pieces: {"process": "storagenode", "error": "context canceled"}
  401. 2024-02-25T08:43:11Z ERROR pieces:trash emptying trash failed {"process": "storagenode", "error": "pieces error: filestore error: context canceled", "errorVerbose": "pieces error: filestore error: context canceled\n\tstorj.io/storj/storagenode/blobstore/filestore.(*blobStore).EmptyTrash:176\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).EmptyTrash:316\n\tstorj.io/storj/storagenode/pieces.(*Store).EmptyTrash:416\n\tstorj.io/storj/storagenode/pieces.(*TrashChore).Run.func1.1:83\n\tstorj.io/common/sync2.(*Workplace).Start.func1:89"}
  402. 2024-02-25T08:43:11Z ERROR bandwidth Could not rollup bandwidth usage {"process": "storagenode", "error": "sql: transaction has already been committed or rolled back"}
  403. Error: piecestore monitor: disk space requirement not met
  404. 2024-02-25 08:43:12,175 INFO exited: storagenode (exit status 1; not expected)
  405. 2024-02-25 08:43:12,182 INFO spawned: 'storagenode' with pid 52
  406. 2024-02-25 08:43:12,186 WARN received SIGQUIT indicating exit request
  407. 2024-02-25 08:43:12,187 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  408. 2024-02-25T08:43:12Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  409. 2024-02-25 08:43:12,197 INFO stopped: storagenode-updater (exit status 0)
  410. 2024-02-25 08:43:12,199 INFO stopped: storagenode (terminated by SIGTERM)
  411. 2024-02-25 08:43:12,201 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
  412. 2024-02-25 08:43:14,704 INFO Set uid to user 0 succeeded
  413. 2024-02-25 08:43:14,717 INFO RPC interface 'supervisor' initialized
  414. 2024-02-25 08:43:14,717 INFO supervisord started with pid 1
  415. 2024-02-25 08:43:15,724 INFO spawned: 'processes-exit-eventlistener' with pid 11
  416. 2024-02-25 08:43:15,729 INFO spawned: 'storagenode' with pid 12
  417. 2024-02-25 08:43:15,733 INFO spawned: 'storagenode-updater' with pid 13
  418. 2024-02-25T08:43:15Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
  419. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
  420. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
  421. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
  422. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
  423. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
  424. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.debug-log-traffic"}
  425. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
  426. 2024-02-25T08:43:15Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
  427. 2024-02-25T08:43:15Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.level"}
  428. 2024-02-25T08:43:15Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.custom-level"}
  429. 2024-02-25T08:43:15Z INFO Invalid configuration file value for key {"Process": "storagenode-updater", "Key": "log.encoding"}
  430. 2024-02-25T08:43:15Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
  431. 2024-02-25T08:43:15Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.96.6"}
  432. 2024-02-25T08:43:15Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  433. 2024-02-25T08:43:15Z INFO Configuration loaded {"process": "storagenode", "Location": "/app/config/config.yaml"}
  434. 2024-02-25T08:43:15Z INFO Anonymized tracing enabled {"process": "storagenode"}
  435. 2024-02-25T08:43:15Z INFO Operator email {"process": "storagenode", "Address": "xxxxxxxxx"}
  436. 2024-02-25T08:43:15Z INFO Operator wallet {"process": "storagenode", "Address": "xxxxxxxxxx"}
  437. 2024-02-25T08:43:15Z INFO server kernel support for server-side tcp fast open remains disabled. {"process": "storagenode"}
  438. 2024-02-25T08:43:15Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"process": "storagenode"}
  439. 2024-02-25T08:43:16Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.96.6"}
  440. 2024-02-25T08:43:16Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode"}
  441. 2024-02-25T08:43:16Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.96.6"}
  442. 2024-02-25T08:43:16Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  443. 2024-02-25T08:43:16Z INFO Telemetry enabled {"process": "storagenode", "instance ID": "12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh"}
  444. 2024-02-25T08:43:16Z INFO Event collection enabled {"process": "storagenode", "instance ID": "12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh"}
  445. 2024-02-25T08:43:16Z INFO db.migration Database Version {"process": "storagenode", "version": 54}
  446. 2024-02-25T08:43:17Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"process": "storagenode"}
  447. 2024-02-25 08:43:17,283 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  448. 2024-02-25 08:43:17,291 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  449. 2024-02-25 08:43:17,291 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  450. 2024-02-25T08:43:18Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"process": "storagenode"}
  451. 2024-02-25T08:43:18Z INFO Node 12Tna3C4LrMXzUn3GJmwuCAopFPJGmH6mXyzfF5HiY2G5Uapuoh started {"process": "storagenode"}
  452. 2024-02-25T08:43:18Z INFO bandwidth Performing bandwidth usage rollups {"process": "storagenode"}
  453. 2024-02-25T08:43:18Z INFO Public server started on [::]:28967 {"process": "storagenode"}
  454. 2024-02-25T08:43:18Z INFO Private server started on 127.0.0.1:7778 {"process": "storagenode"}
  455. 2024-02-25T08:43:18Z INFO failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details. {"process": "storagenode"}
  456. 2024-02-25T08:43:18Z INFO trust Scheduling next refresh {"process": "storagenode", "after": "5h43m3.123739492s"}
  457. 2024-02-25T08:43:18Z INFO pieces:trash emptying trash started {"process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  458. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  459. 2024-02-25T08:43:18Z WARN piecestore:monitor Disk space is less than requested. Allocated space is {"process": "storagenode", "bytes": 338433196032}
  460. 2024-02-25T08:43:18Z ERROR piecestore:monitor Total disk space is less than required minimum {"process": "storagenode", "bytes": 500000000000}
  461. 2024-02-25T08:43:18Z ERROR services unexpected shutdown of a runner {"process": "storagenode", "name": "piecestore:monitor", "error": "piecestore monitor: disk space requirement not met", "errorVerbose": "piecestore monitor: disk space requirement not met\n\tstorj.io/storj/storagenode/monitor.(*Service).Run:135\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  462. 2024-02-25T08:43:18Z ERROR nodestats:cache Get pricing-model/join date failed {"process": "storagenode", "error": "context canceled"}
  463. 2024-02-25T08:43:18Z ERROR gracefulexit:blobscleaner couldn't receive satellite's GE status {"process": "storagenode", "error": "context canceled"}
  464. 2024-02-25T08:43:18Z ERROR version failed to get process version info {"process": "storagenode", "error": "version checker client: Get \"https://version.storj.io\": context canceled", "errorVerbose": "version checker client: Get \"https://version.storj.io\": context canceled\n\tstorj.io/storj/private/version/checker.(*Client).All:68\n\tstorj.io/storj/private/version/checker.(*Client).Process:108\n\tstorj.io/storj/private/version/checker.(*Service).checkVersion:104\n\tstorj.io/storj/private/version/checker.(*Service).CheckVersion:78\n\tstorj.io/storj/storagenode/version.(*Chore).checkVersion:110\n\tstorj.io/storj/storagenode/version.(*Chore).Run.func1:65\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/storj/storagenode/version.(*Chore).Run:64\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  465. 2024-02-25T08:43:18Z ERROR bandwidth Could not rollup bandwidth usage {"process": "storagenode", "error": "sql: transaction has already been committed or rolled back"}
  466. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker subprocess started {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  467. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker subprocess exited with status {"process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "status": -1, "error": "signal: killed"}
  468. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: signal: killed", "errorVerbose": "lazyfilewalker: signal: killed\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:83\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  469. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  470. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "error": "context canceled"}
  471. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  472. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  473. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "context canceled"}
  474. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  475. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  476. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "error": "context canceled"}
  477. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
  478. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  479. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "error": "context canceled"}
  480. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
  481. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  482. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "error": "context canceled"}
  483. 2024-02-25T08:43:18Z ERROR gracefulexit:chore error retrieving satellites. {"process": "storagenode", "error": "satellitesdb: context canceled", "errorVerbose": "satellitesdb: context canceled\n\tstorj.io/storj/storagenode/storagenodedb.(*satellitesDB).ListGracefulExits.func1:195\n\tstorj.io/storj/storagenode/storagenodedb.(*satellitesDB).ListGracefulExits:207\n\tstorj.io/storj/storagenode/gracefulexit.(*Service).ListPendingExits:59\n\tstorj.io/storj/storagenode/gracefulexit.(*Chore).AddMissing:55\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/storj/storagenode/gracefulexit.(*Chore).Run:48\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  484. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
  485. 2024-02-25T08:43:18Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  486. 2024-02-25T08:43:18Z ERROR lazyfilewalker.used-space-filewalker failed to start subprocess {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "context canceled"}
  487. 2024-02-25T08:43:18Z ERROR pieces failed to lazywalk space used by satellite {"process": "storagenode", "error": "lazyfilewalker: context canceled", "errorVerbose": "lazyfilewalker: context canceled\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*process).run:71\n\tstorj.io/storj/storagenode/pieces/lazyfilewalker.(*Supervisor).WalkAndComputeSpaceUsedBySatellite:105\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:717\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  488. 2024-02-25T08:43:18Z ERROR piecestore:cache error getting current used space: {"process": "storagenode", "error": "filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled; filewalker: context canceled", "errorVerbose": "group:\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- filewalker: context canceled\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkSatellitePieces:69\n\tstorj.io/storj/storagenode/pieces.(*FileWalker).WalkAndComputeSpaceUsedBySatellite:74\n\tstorj.io/storj/storagenode/pieces.(*Store).SpaceUsedTotalAndBySatellite:726\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:57\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:51\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  489. 2024-02-25T08:43:18Z ERROR pieces:trash emptying trash failed {"process": "storagenode", "error": "pieces error: filestore error: context canceled", "errorVerbose": "pieces error: filestore error: context canceled\n\tstorj.io/storj/storagenode/blobstore/filestore.(*blobStore).EmptyTrash:176\n\tstorj.io/storj/storagenode/pieces.(*BlobsUsageCache).EmptyTrash:316\n\tstorj.io/storj/storagenode/pieces.(*Store).EmptyTrash:416\n\tstorj.io/storj/storagenode/pieces.(*TrashChore).Run.func1.1:83\n\tstorj.io/common/sync2.(*Workplace).Start.func1:89"}
  490. 2024-02-25T08:43:18Z ERROR collector error during collecting pieces: {"process": "storagenode", "error": "context canceled"}
  491. Error: piecestore monitor: disk space requirement not met
  492. 2024-02-25 08:43:19,454 INFO exited: storagenode (exit status 1; not expected)
  493. 2024-02-25 08:43:20,461 INFO spawned: 'storagenode' with pid 51
  494. 2024-02-25 08:43:20,463 WARN received SIGQUIT indicating exit request
  495. 2024-02-25 08:43:20,465 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  496. 2024-02-25T08:43:20Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  497. 2024-02-25 08:43:20,476 INFO stopped: storagenode-updater (exit status 0)
  498. 2024-02-25 08:43:20,478 INFO stopped: storagenode (terminated by SIGTERM)
  499. 2024-02-25 08:43:20,479 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!