API tools faq
paste
Advertisement
Guest User

storj

a guest
Apr 17th, 2024
24
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 80.69 KB | None | 0 0
raw download clone embed print report
  1. downloading storagenode-updater
  2. downloading storagenode
  3. 2024-04-17 06:38:17,007 INFO RPC interface 'supervisor' initialized
  4. 2024-04-17 06:38:17,008 INFO supervisord started with pid 1
  5. 2024-04-17 06:38:18,012 INFO spawned: 'processes-exit-eventlistener' with pid 51
  6. 2024-04-17 06:38:18,016 INFO spawned: 'storagenode' with pid 52
  7. 2024-04-17 06:38:18,020 INFO spawned: 'storagenode-updater' with pid 53
  8. 2024-04-17T06:38:18Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
  9. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
  10. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
  11. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "console.address"}
  12. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
  13. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
  14. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet-features"}
  15. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.details"}
  16. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
  17. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.enabled"}
  18. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
  19. 2024-04-17T06:38:18Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
  20. 2024-04-17T06:38:18Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
  21. 2024-04-17T06:38:18Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  22. 2024-04-17T06:38:18Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  23. 2024-04-17T06:38:18Z INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"}
  24. 2024-04-17T06:38:18Z INFO Anonymized tracing enabled {"Process": "storagenode"}
  25. 2024-04-17T06:38:18Z INFO Operator email {"Process": "storagenode", "Address": "darkletwingly@gmail.com"}
  26. 2024-04-17T06:38:18Z INFO Operator wallet {"Process": "storagenode", "Address": "0x60Bc4F1Cf72Cb648326F964acC4a3C69442cc6Ff"}
  27. 2024-04-17T06:38:18Z INFO server kernel support for server-side tcp fast open remains disabled. {"Process": "storagenode"}
  28. 2024-04-17T06:38:18Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"Process": "storagenode"}
  29. 2024-04-17T06:38:18Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.99.3"}
  30. 2024-04-17T06:38:18Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode"}
  31. 2024-04-17T06:38:18Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  32. 2024-04-17T06:38:18Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  33. 2024-04-17T06:38:18Z INFO Telemetry enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  34. 2024-04-17T06:38:18Z INFO Event collection enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  35. 2024-04-17T06:38:19Z INFO db.migration Database Version {"Process": "storagenode", "version": 54}
  36. 2024-04-17T06:38:19Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"}
  37. 2024-04-17 06:38:19,393 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  38. 2024-04-17 06:38:19,393 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  39. 2024-04-17 06:38:19,393 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  40. 2024-04-17T06:38:20Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"}
  41. 2024-04-17T06:38:20Z INFO trust Scheduling next refresh {"Process": "storagenode", "after": "6h37m6.937101886s"}
  42. 2024-04-17T06:38:20Z INFO Node 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz started {"Process": "storagenode"}
  43. 2024-04-17T06:38:20Z INFO Public server started on [::]:28967 {"Process": "storagenode"}
  44. 2024-04-17T06:38:20Z INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"}
  45. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  46. 2024-04-17T06:38:20Z INFO bandwidth Performing bandwidth usage rollups {"Process": "storagenode"}
  47. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Next viable run": "2024-04-24T06:38:20Z"}
  48. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  49. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Next viable run": "2024-04-24T06:38:20Z"}
  50. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  51. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Next viable run": "2024-04-24T06:38:20Z"}
  52. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  53. 2024-04-17T06:38:20Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Next viable run": "2024-04-24T06:38:20Z"}
  54. 2024-04-17T06:40:07Z ERROR nodestats:cache Get disk space usage query failed {"Process": "storagenode", "error": "nodestats: EOF; nodestats: EOF; nodestats: EOF; nodestats: EOF", "errorVerbose": "group:\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  55. 2024-04-17T06:40:08Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  56. 2024-04-17T06:40:08Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  57. 2024-04-17T06:40:09Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  58. 2024-04-17T06:40:09Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  59. 2024-04-17T06:40:09Z ERROR nodestats:cache Get held amount query failed {"Process": "storagenode", "error": "payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz", "errorVerbose": "group:\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  60. 2024-04-17T06:40:30Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  61. 2024-04-17T06:40:31Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  62. 2024-04-17T06:40:32Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  63. 2024-04-17T06:40:33Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  64. 2024-04-17T06:41:40Z ERROR nodestats:cache Get stats query failed {"Process": "storagenode", "error": "nodestats: EOF; nodestats: EOF; nodestats: EOF; nodestats: EOF", "errorVerbose": "group:\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  65. 2024-04-17 06:41:42,483 WARN received SIGTERM indicating exit request
  66. 2024-04-17 06:41:42,484 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  67. 2024-04-17T06:41:42Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  68. 2024-04-17 06:41:42,489 INFO stopped: storagenode-updater (exit status 0)
  69. 2024-04-17T06:41:42Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode"}
  70. 2024-04-17T06:41:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 2, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  71. 2024-04-17T06:41:42Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  72. 2024-04-17T06:41:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 2, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  73. 2024-04-17T06:41:42Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  74. 2024-04-17T06:41:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 2, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  75. 2024-04-17T06:41:42Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  76. 2024-04-17T06:41:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 2, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  77. 2024-04-17T06:41:42Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  78. 2024-04-17 06:41:42,927 INFO stopped: storagenode (exit status 0)
  79. 2024-04-17 06:41:42,928 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
  80. 2024-04-17 06:41:44,667 INFO RPC interface 'supervisor' initialized
  81. 2024-04-17 06:41:44,668 INFO supervisord started with pid 1
  82. 2024-04-17 06:41:45,672 INFO spawned: 'processes-exit-eventlistener' with pid 12
  83. 2024-04-17 06:41:45,676 INFO spawned: 'storagenode' with pid 13
  84. 2024-04-17 06:41:45,680 INFO spawned: 'storagenode-updater' with pid 14
  85. 2024-04-17T06:41:45Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
  86. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.details"}
  87. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
  88. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet-features"}
  89. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
  90. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.enabled"}
  91. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
  92. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
  93. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
  94. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
  95. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
  96. 2024-04-17T06:41:45Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "console.address"}
  97. 2024-04-17T06:41:45Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
  98. 2024-04-17T06:41:45Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  99. 2024-04-17T06:41:45Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  100. 2024-04-17T06:41:45Z INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"}
  101. 2024-04-17T06:41:45Z INFO Anonymized tracing enabled {"Process": "storagenode"}
  102. 2024-04-17T06:41:45Z INFO Operator email {"Process": "storagenode", "Address": "darkletwingly@gmail.com"}
  103. 2024-04-17T06:41:45Z INFO Operator wallet {"Process": "storagenode", "Address": "0x60Bc4F1Cf72Cb648326F964acC4a3C69442cc6Ff"}
  104. 2024-04-17T06:41:45Z INFO server kernel support for server-side tcp fast open remains disabled. {"Process": "storagenode"}
  105. 2024-04-17T06:41:45Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"Process": "storagenode"}
  106. 2024-04-17T06:41:46Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.99.3"}
  107. 2024-04-17T06:41:46Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode"}
  108. 2024-04-17T06:41:46Z INFO Telemetry enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  109. 2024-04-17T06:41:46Z INFO Event collection enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  110. 2024-04-17T06:41:46Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  111. 2024-04-17T06:41:46Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  112. 2024-04-17T06:41:46Z INFO db.migration Database Version {"Process": "storagenode", "version": 54}
  113. 2024-04-17T06:41:46Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"}
  114. 2024-04-17 06:41:46,913 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  115. 2024-04-17 06:41:46,913 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  116. 2024-04-17 06:41:46,913 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  117. 2024-04-17T06:41:47Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"}
  118. 2024-04-17T06:41:47Z INFO bandwidth Performing bandwidth usage rollups {"Process": "storagenode"}
  119. 2024-04-17T06:41:47Z INFO Node 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz started {"Process": "storagenode"}
  120. 2024-04-17T06:41:47Z INFO Public server started on [::]:28967 {"Process": "storagenode"}
  121. 2024-04-17T06:41:47Z INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"}
  122. 2024-04-17T06:41:47Z INFO trust Scheduling next refresh {"Process": "storagenode", "after": "6h6m29.558335275s"}
  123. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  124. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Next viable run": "2024-04-24T06:41:47Z"}
  125. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  126. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Next viable run": "2024-04-24T06:41:47Z"}
  127. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  128. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Next viable run": "2024-04-24T06:41:47Z"}
  129. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  130. 2024-04-17T06:41:47Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Next viable run": "2024-04-24T06:41:47Z"}
  131. 2024-04-17T06:41:48Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  132. 2024-04-17T06:41:49Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  133. 2024-04-17T06:42:15Z ERROR nodestats:cache Get stats query failed {"Process": "storagenode", "error": "nodestats: EOF; nodestats: EOF; nodestats: EOF; nodestats: EOF", "errorVerbose": "group:\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  134. 2024-04-17 06:42:53,899 WARN received SIGTERM indicating exit request
  135. 2024-04-17 06:42:53,900 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die
  136. 2024-04-17T06:42:53Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"}
  137. 2024-04-17 06:42:53,905 INFO stopped: storagenode-updater (exit status 0)
  138. 2024-04-17T06:42:53Z INFO Got a signal from the OS: "terminated" {"Process": "storagenode"}
  139. 2024-04-17T06:42:53Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 3, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  140. 2024-04-17T06:42:53Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  141. 2024-04-17T06:42:53Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  142. 2024-04-17T06:42:53Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  143. 2024-04-17T06:42:53Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 1, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  144. 2024-04-17T06:42:53Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  145. 2024-04-17T06:42:53Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: context canceled", "errorVerbose": "ping satellite: context canceled\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  146. 2024-04-17T06:42:53Z INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  147. 2024-04-17 06:42:54,353 INFO stopped: storagenode (exit status 0)
  148. 2024-04-17 06:42:54,355 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)
  149. 2024-04-17 06:42:55,976 INFO RPC interface 'supervisor' initialized
  150. 2024-04-17 06:42:55,976 INFO supervisord started with pid 1
  151. 2024-04-17 06:42:56,982 INFO spawned: 'processes-exit-eventlistener' with pid 12
  152. 2024-04-17 06:42:56,991 INFO spawned: 'storagenode' with pid 13
  153. 2024-04-17 06:42:56,998 INFO spawned: 'storagenode-updater' with pid 14
  154. 2024-04-17T06:42:57Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
  155. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.details"}
  156. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.enabled"}
  157. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
  158. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
  159. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
  160. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet-features"}
  161. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "console.address"}
  162. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
  163. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
  164. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
  165. 2024-04-17T06:42:57Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
  166. 2024-04-17T06:42:57Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
  167. 2024-04-17T06:42:57Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  168. 2024-04-17T06:42:57Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  169. 2024-04-17T06:42:57Z INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"}
  170. 2024-04-17T06:42:57Z INFO Anonymized tracing enabled {"Process": "storagenode"}
  171. 2024-04-17T06:42:57Z INFO Operator email {"Process": "storagenode", "Address": "darkletwingly@gmail.com"}
  172. 2024-04-17T06:42:57Z INFO Operator wallet {"Process": "storagenode", "Address": "0x60Bc4F1Cf72Cb648326F964acC4a3C69442cc6Ff"}
  173. 2024-04-17T06:42:57Z INFO server kernel support for server-side tcp fast open remains disabled. {"Process": "storagenode"}
  174. 2024-04-17T06:42:57Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"Process": "storagenode"}
  175. 2024-04-17T06:42:57Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.99.3"}
  176. 2024-04-17T06:42:57Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode"}
  177. 2024-04-17T06:42:57Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  178. 2024-04-17T06:42:57Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  179. 2024-04-17T06:42:57Z INFO Telemetry enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  180. 2024-04-17T06:42:57Z INFO Event collection enabled {"Process": "storagenode", "instance ID": "128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz"}
  181. 2024-04-17T06:42:57Z INFO db.migration Database Version {"Process": "storagenode", "version": 54}
  182. 2024-04-17T06:42:58Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"}
  183. 2024-04-17 06:42:58,272 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  184. 2024-04-17 06:42:58,274 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  185. 2024-04-17 06:42:58,274 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
  186. 2024-04-17T06:42:59Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"}
  187. 2024-04-17T06:42:59Z INFO bandwidth Performing bandwidth usage rollups {"Process": "storagenode"}
  188. 2024-04-17T06:42:59Z INFO Node 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz started {"Process": "storagenode"}
  189. 2024-04-17T06:42:59Z INFO Public server started on [::]:28967 {"Process": "storagenode"}
  190. 2024-04-17T06:42:59Z INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"}
  191. 2024-04-17T06:42:59Z INFO trust Scheduling next refresh {"Process": "storagenode", "after": "5h29m24.903386904s"}
  192. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  193. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Next viable run": "2024-04-24T06:42:59Z"}
  194. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  195. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Next viable run": "2024-04-24T06:42:59Z"}
  196. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  197. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Next viable run": "2024-04-24T06:42:59Z"}
  198. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  199. 2024-04-17T06:42:59Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Next viable run": "2024-04-24T06:42:59Z"}
  200. 2024-04-17T06:42:59Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  201. 2024-04-17T06:42:59Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  202. 2024-04-17T06:43:00Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  203. 2024-04-17T06:45:09Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  204. 2024-04-17T06:45:10Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 1, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  205. 2024-04-17T06:45:11Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 2, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  206. 2024-04-17T06:45:12Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  207. 2024-04-17T06:45:12Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 3, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  208. 2024-04-17T06:45:13Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  209. 2024-04-17T06:45:15Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  210. 2024-04-17T06:45:16Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  211. 2024-04-17T06:45:18Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  212. 2024-04-17T06:45:19Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  213. 2024-04-17T06:45:28Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  214. 2024-04-17T06:45:31Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  215. 2024-04-17T06:45:44Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  216. 2024-04-17T06:45:48Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  217. 2024-04-17T06:46:17Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 7, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  218. 2024-04-17T06:46:20Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 7, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  219. 2024-04-17T06:47:01Z ERROR nodestats:cache Get stats query failed {"Process": "storagenode", "error": "nodestats: EOF; nodestats: EOF; nodestats: EOF; nodestats: EOF", "errorVerbose": "group:\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:74\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:160\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:159\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:126\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  220. 2024-04-17T06:47:19Z ERROR nodestats:cache Get disk space usage query failed {"Process": "storagenode", "error": "nodestats: EOF; nodestats: EOF; nodestats: EOF; nodestats: EOF", "errorVerbose": "group:\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- nodestats: EOF\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:123\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:215\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:138\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  221. 2024-04-17T06:47:20Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
  222. 2024-04-17T06:47:20Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
  223. 2024-04-17T06:47:21Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
  224. 2024-04-17T06:47:21Z ERROR nodestats:cache payouts err {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
  225. 2024-04-17T06:47:21Z ERROR nodestats:cache Get held amount query failed {"Process": "storagenode", "error": "payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz; payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz", "errorVerbose": "group:\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75\n--- payouts service: node not found: 128b2VBuVLiAXpDxW5XUuQGjSHiChUYJt8eGu6YrkfnCGiAriFz\n\tstorj.io/storj/storagenode/payouts.(*Endpoint).GetPaystub:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount.func1:249\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:306\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheHeldAmount:241\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func3:143\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  226. 2024-04-17T06:47:21Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 8, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  227. 2024-04-17T06:47:22Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 2, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  228. 2024-04-17T06:47:25Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 8, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  229. 2024-04-17T06:47:35Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 5, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  230. 2024-04-17T06:49:35Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 3, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  231. 2024-04-17T06:50:02Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 6, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  232. 2024-04-17T06:50:34Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 7, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  233. 2024-04-17T06:51:40Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 9, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  234. 2024-04-17T06:51:43Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 9, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  235. 2024-04-17T06:51:49Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 4, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  236. 2024-04-17T06:53:48Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 8, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  237. 2024-04-17T06:54:10Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 5, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  238. 2024-04-17T06:55:56Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 10, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:203\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  239. 2024-04-17T06:56:35Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 6, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  240. 2024-04-17T06:57:57Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
  241. 2024-04-17T06:57:57Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.99.3"}
  242. 2024-04-17T06:57:57Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode"}
  243. 2024-04-17T06:57:57Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.99.3"}
  244. 2024-04-17T06:57:57Z INFO New version is being rolled out but hasn't made it to this node yet {"Process": "storagenode-updater", "Service": "storagenode-updater"}
  245. 2024-04-17T06:58:05Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 9, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  246. 2024-04-17T06:58:09Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 10, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  247. 2024-04-17T06:59:19Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 7, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  248. 2024-04-17T07:02:34Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 8, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  249. 2024-04-17T07:04:33Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 10, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  250. 2024-04-17T07:06:41Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 11, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  251. 2024-04-17T07:06:51Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 9, "error": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out", "errorVerbose": "ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: dial tcp 100.93.39.90:28967: connect: connection timed out\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:209\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:157\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
  252.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!