API tools faq
paste
Guest User

spark_worker_beam_logs

a guest
Mar 5th, 2021
319
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Java 36.37 KB | None | 0 0
raw download clone embed print report
  1. Terminal logs:
  2.  
  3. ```
  4. INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
  5. INFO:oauth2client.client:Timeout attempting to reach GCE metadata service.
  6. WARNING:apache_beam.internal.gcp.auth:Unable to find default credentials to use: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
  7. Connecting anonymously.
  8. INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45271
  9. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
  10. INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.28.0
  11. INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff7d00898c8> ====================
  12. INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff7d008a048> ====================
  13. INFO:apache_beam.runners.portability.abstract_job_service:Artifact server started on port 39389
  14. INFO:apache_beam.runners.portability.abstract_job_service:Running job 'job-1148ac23-bb81-40de-873c-1b2e3e073d14'
  15. INFO:apache_beam.runners.portability.spark_uber_jar_job_server:Submitted Spark job with ID driver-20210305185322-0000
  16. INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  17.   with Pipeline() as p:
  18.     p.apply(..)
  19. This ensures that the pipeline finishes before this program exits.
  20. INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
  21. INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
  22. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  23. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33687.
  24. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  25. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  26. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36447.
  27. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  28. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46519
  29. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  30. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  31. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  32. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  33. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  34. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  35. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36067.
  36. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  37. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  38. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36875.
  39. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  40. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39275
  41. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  42. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  43. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  44. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  45. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  46. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  47. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41825.
  48. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  49. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  50. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34653.
  51. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  52. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42789
  53. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  54. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  55. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  56. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  57. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  58. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  59. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41493.
  60. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  61. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  62. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37073.
  63. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  64. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41953
  65. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  66. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  67. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  68. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  69. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  70. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  71. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41671.
  72. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  73. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  74. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36677.
  75. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  76. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41877
  77. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  78. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  79. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  80. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  81. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  82. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  83. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33037.
  84. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  85. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  86. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34155.
  87. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  88. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33409
  89. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  90. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  91. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  92. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  93. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  94. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  95. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40475.
  96. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  97. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  98. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42731.
  99. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  100. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:32995
  101. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  102. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  103. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  104. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  105. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  106. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  107. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43895.
  108. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  109. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  110. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38813.
  111. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  112. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43165
  113. INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 2 (skipped: 0), batches: 2, num_threads: 2
  114. INFO:apache_beam.io.filebasedsink:Renamed 2 shards in 0.10 seconds.
  115. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  116. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  117. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  118. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  119. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  120. INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
  121.  
  122. ```
  123. Spark worker stderr:
  124.  
  125. ```
  126. 21/03/05 18:53:29 INFO Executor: Adding file:/opt/spark/work/app-20210305185326-0000/0/./api-asn1-api-1.0.0-M20.jar to class loader
  127. 21/03/05 18:53:29 INFO TorrentBroadcast: Started reading broadcast variable 0
  128. 21/03/05 18:53:29 INFO TransportClientFactory: Successfully created connection to spark-worker-6cc46b5d7-nlnfw/172.18.0.20:40957 after 1 ms (0 ms spent in bootstraps)
  129. 21/03/05 18:53:29 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 12.3 KB, free 366.3 MB)
  130. 21/03/05 18:53:29 INFO TorrentBroadcast: Reading broadcast variable 0 took 66 ms
  131. 21/03/05 18:53:29 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 32.4 KB, free 366.3 MB)
  132. 21/03/05 18:53:29 INFO MemoryStore: Block rdd_13_0 stored as values in memory (estimated size 16.0 B, free 366.3 MB)
  133. 21/03/05 18:53:29 INFO MemoryStore: Block rdd_17_0 stored as values in memory (estimated size 16.0 B, free 366.3 MB)
  134. 21/03/05 18:53:29 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 5427 bytes result sent to driver
  135. 21/03/05 18:53:31 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1
  136. 21/03/05 18:53:31 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2
  137. 21/03/05 18:53:31 INFO GrpcDataService: Beam Fn Data client connected.
  138. 21/03/05 18:53:31 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  139. payload: "\n\021\n\017localhost:45271"
  140. capabilities: "beam:coder:bytes:v1"
  141. capabilities: "beam:coder:string_utf8:v1"
  142. capabilities: "beam:coder:kv:v1"
  143. capabilities: "beam:coder:bool:v1"
  144. capabilities: "beam:coder:varint:v1"
  145. capabilities: "beam:coder:double:v1"
  146. capabilities: "beam:coder:iterable:v1"
  147. capabilities: "beam:coder:timer:v1"
  148. capabilities: "beam:coder:interval_window:v1"
  149. capabilities: "beam:coder:length_prefix:v1"
  150. capabilities: "beam:coder:global_window:v1"
  151. capabilities: "beam:coder:windowed_value:v1"
  152. capabilities: "beam:coder:param_windowed_value:v1"
  153. capabilities: "beam:coder:state_backed_iterable:v1"
  154. capabilities: "beam:coder:row:v1"
  155. capabilities: "beam:coder:sharded_key:v1"
  156. capabilities: "beam:protocol:progress_reporting:v0"
  157. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  158. capabilities: "beam:protocol:worker_status:v1"
  159. capabilities: "beam:combinefn:packed_python:v1"
  160. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  161. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  162. dependencies {
  163.   type_urn: "beam:artifact:type:file:v1"
  164.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  165.   role_urn: "beam:artifact:role:staging_to:v1"
  166.   role_payload: "\n\024pickled_main_session"
  167. }
  168.  
  169. 21/03/05 18:53:31 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  170. 21/03/05 18:53:31 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  171. 21/03/05 18:53:36 INFO MemoryStore: Block rdd_13_1 stored as values in memory (estimated size 192.0 B, free 366.3 MB)
  172. 21/03/05 18:53:36 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 2-1
  173. 21/03/05 18:53:36 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 2-2
  174. 21/03/05 18:53:36 INFO GrpcDataService: Beam Fn Data client connected.
  175. 21/03/05 18:53:36 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  176. payload: "\n\021\n\017localhost:45271"
  177. capabilities: "beam:coder:bytes:v1"
  178. capabilities: "beam:coder:string_utf8:v1"
  179. capabilities: "beam:coder:kv:v1"
  180. capabilities: "beam:coder:bool:v1"
  181. capabilities: "beam:coder:varint:v1"
  182. capabilities: "beam:coder:double:v1"
  183. capabilities: "beam:coder:iterable:v1"
  184. capabilities: "beam:coder:timer:v1"
  185. capabilities: "beam:coder:interval_window:v1"
  186. capabilities: "beam:coder:length_prefix:v1"
  187. capabilities: "beam:coder:global_window:v1"
  188. capabilities: "beam:coder:windowed_value:v1"
  189. capabilities: "beam:coder:param_windowed_value:v1"
  190. capabilities: "beam:coder:state_backed_iterable:v1"
  191. capabilities: "beam:coder:row:v1"
  192. capabilities: "beam:coder:sharded_key:v1"
  193. capabilities: "beam:protocol:progress_reporting:v0"
  194. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  195. capabilities: "beam:protocol:worker_status:v1"
  196. capabilities: "beam:combinefn:packed_python:v1"
  197. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  198. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  199. dependencies {
  200.   type_urn: "beam:artifact:type:file:v1"
  201.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  202.   role_urn: "beam:artifact:role:staging_to:v1"
  203.   role_payload: "\n\024pickled_main_session"
  204. }
  205.  
  206. 21/03/05 18:53:36 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  207. 21/03/05 18:53:36 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  208. 21/03/05 18:53:41 INFO MemoryStore: Block rdd_17_1 stored as values in memory (estimated size 256.0 B, free 366.3 MB)
  209. 21/03/05 18:53:41 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 17382 bytes result sent to driver
  210. 21/03/05 18:53:41 INFO CoarseGrainedExecutorBackend: Got assigned task 2
  211. 21/03/05 18:53:41 INFO Executor: Running task 0.0 in stage 1.0 (TID 2)
  212. 21/03/05 18:53:41 INFO CoarseGrainedExecutorBackend: Got assigned task 3
  213. 21/03/05 18:53:41 INFO Executor: Running task 1.0 in stage 1.0 (TID 3)
  214. 21/03/05 18:53:41 INFO TorrentBroadcast: Started reading broadcast variable 2
  215. 21/03/05 18:53:41 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 18.1 KB, free 366.2 MB)
  216. 21/03/05 18:53:41 INFO TorrentBroadcast: Reading broadcast variable 2 took 6 ms
  217. 21/03/05 18:53:41 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 42.1 KB, free 366.2 MB)
  218. 21/03/05 18:53:41 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 5480 bytes result sent to driver
  219. 21/03/05 18:53:41 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 3-1
  220. 21/03/05 18:53:41 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 3-2
  221. 21/03/05 18:53:41 INFO GrpcDataService: Beam Fn Data client connected.
  222. 21/03/05 18:53:42 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  223. payload: "\n\021\n\017localhost:45271"
  224. capabilities: "beam:coder:bytes:v1"
  225. capabilities: "beam:coder:string_utf8:v1"
  226. capabilities: "beam:coder:kv:v1"
  227. capabilities: "beam:coder:bool:v1"
  228. capabilities: "beam:coder:varint:v1"
  229. capabilities: "beam:coder:double:v1"
  230. capabilities: "beam:coder:iterable:v1"
  231. capabilities: "beam:coder:timer:v1"
  232. capabilities: "beam:coder:interval_window:v1"
  233. capabilities: "beam:coder:length_prefix:v1"
  234. capabilities: "beam:coder:global_window:v1"
  235. capabilities: "beam:coder:windowed_value:v1"
  236. capabilities: "beam:coder:param_windowed_value:v1"
  237. capabilities: "beam:coder:state_backed_iterable:v1"
  238. capabilities: "beam:coder:row:v1"
  239. capabilities: "beam:coder:sharded_key:v1"
  240. capabilities: "beam:protocol:progress_reporting:v0"
  241. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  242. capabilities: "beam:protocol:worker_status:v1"
  243. capabilities: "beam:combinefn:packed_python:v1"
  244. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  245. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  246. dependencies {
  247.   type_urn: "beam:artifact:type:file:v1"
  248.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  249.   role_urn: "beam:artifact:role:staging_to:v1"
  250.   role_payload: "\n\024pickled_main_session"
  251. }
  252.  
  253. 21/03/05 18:53:42 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  254. 21/03/05 18:53:42 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  255. 21/03/05 18:53:46 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 4-1
  256. 21/03/05 18:53:46 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 4-2
  257. 21/03/05 18:53:46 INFO GrpcDataService: Beam Fn Data client connected.
  258. 21/03/05 18:53:47 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  259. payload: "\n\021\n\017localhost:45271"
  260. capabilities: "beam:coder:bytes:v1"
  261. capabilities: "beam:coder:string_utf8:v1"
  262. capabilities: "beam:coder:kv:v1"
  263. capabilities: "beam:coder:bool:v1"
  264. capabilities: "beam:coder:varint:v1"
  265. capabilities: "beam:coder:double:v1"
  266. capabilities: "beam:coder:iterable:v1"
  267. capabilities: "beam:coder:timer:v1"
  268. capabilities: "beam:coder:interval_window:v1"
  269. capabilities: "beam:coder:length_prefix:v1"
  270. capabilities: "beam:coder:global_window:v1"
  271. capabilities: "beam:coder:windowed_value:v1"
  272. capabilities: "beam:coder:param_windowed_value:v1"
  273. capabilities: "beam:coder:state_backed_iterable:v1"
  274. capabilities: "beam:coder:row:v1"
  275. capabilities: "beam:coder:sharded_key:v1"
  276. capabilities: "beam:protocol:progress_reporting:v0"
  277. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  278. capabilities: "beam:protocol:worker_status:v1"
  279. capabilities: "beam:combinefn:packed_python:v1"
  280. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  281. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  282. dependencies {
  283.   type_urn: "beam:artifact:type:file:v1"
  284.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  285.   role_urn: "beam:artifact:role:staging_to:v1"
  286.   role_payload: "\n\024pickled_main_session"
  287. }
  288.  
  289. 21/03/05 18:53:47 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  290. 21/03/05 18:53:51 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  291. 21/03/05 18:53:51 INFO Executor: Finished task 1.0 in stage 1.0 (TID 3). 22237 bytes result sent to driver
  292. 21/03/05 18:53:51 INFO CoarseGrainedExecutorBackend: Got assigned task 4
  293. 21/03/05 18:53:51 INFO Executor: Running task 0.0 in stage 2.0 (TID 4)
  294. 21/03/05 18:53:51 INFO CoarseGrainedExecutorBackend: Got assigned task 5
  295. 21/03/05 18:53:51 INFO Executor: Running task 1.0 in stage 2.0 (TID 5)
  296. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Updating epoch to 1 and clearing cache
  297. 21/03/05 18:53:51 INFO TorrentBroadcast: Started reading broadcast variable 3
  298. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 18.2 KB, free 366.2 MB)
  299. 21/03/05 18:53:51 INFO TorrentBroadcast: Reading broadcast variable 3 took 6 ms
  300. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 39.9 KB, free 366.1 MB)
  301. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 1, fetching them
  302. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 1, fetching them
  303. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@spark-worker-6cc46b5d7-nlnfw:34605)
  304. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Got the output locations
  305. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local blocks and 0 remote blocks
  306. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local blocks and 0 remote blocks
  307. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms
  308. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms
  309. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 5-1
  310. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-3
  311. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-2
  312. 21/03/05 18:53:51 INFO GrpcDataService: Beam Fn Data client connected.
  313. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-4
  314. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-5
  315. 21/03/05 18:53:51 INFO TorrentBroadcast: Started reading broadcast variable 1
  316. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.1 MB)
  317. 21/03/05 18:53:51 INFO TorrentBroadcast: Reading broadcast variable 1 took 8 ms
  318. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 288.0 B, free 366.1 MB)
  319. 21/03/05 18:53:52 INFO Executor: Finished task 0.0 in stage 2.0 (TID 4). 20647 bytes result sent to driver
  320. 21/03/05 18:53:52 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  321. payload: "\n\021\n\017localhost:45271"
  322. capabilities: "beam:coder:bytes:v1"
  323. capabilities: "beam:coder:string_utf8:v1"
  324. capabilities: "beam:coder:kv:v1"
  325. capabilities: "beam:coder:bool:v1"
  326. capabilities: "beam:coder:varint:v1"
  327. capabilities: "beam:coder:double:v1"
  328. capabilities: "beam:coder:iterable:v1"
  329. capabilities: "beam:coder:timer:v1"
  330. capabilities: "beam:coder:interval_window:v1"
  331. capabilities: "beam:coder:length_prefix:v1"
  332. capabilities: "beam:coder:global_window:v1"
  333. capabilities: "beam:coder:windowed_value:v1"
  334. capabilities: "beam:coder:param_windowed_value:v1"
  335. capabilities: "beam:coder:state_backed_iterable:v1"
  336. capabilities: "beam:coder:row:v1"
  337. capabilities: "beam:coder:sharded_key:v1"
  338. capabilities: "beam:protocol:progress_reporting:v0"
  339. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  340. capabilities: "beam:protocol:worker_status:v1"
  341. capabilities: "beam:combinefn:packed_python:v1"
  342. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  343. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  344. dependencies {
  345.   type_urn: "beam:artifact:type:file:v1"
  346.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  347.   role_urn: "beam:artifact:role:staging_to:v1"
  348.   role_payload: "\n\024pickled_main_session"
  349. }
  350.  
  351. 21/03/05 18:53:52 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  352. 21/03/05 18:53:52 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  353. 21/03/05 18:53:56 INFO Executor: Finished task 1.0 in stage 2.0 (TID 5). 20647 bytes result sent to driver
  354. 21/03/05 18:53:56 INFO CoarseGrainedExecutorBackend: Got assigned task 6
  355. 21/03/05 18:53:56 INFO Executor: Running task 0.0 in stage 3.0 (TID 6)
  356. 21/03/05 18:53:56 INFO CoarseGrainedExecutorBackend: Got assigned task 7
  357. 21/03/05 18:53:56 INFO Executor: Running task 1.0 in stage 3.0 (TID 7)
  358. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Updating epoch to 2 and clearing cache
  359. 21/03/05 18:53:56 INFO TorrentBroadcast: Started reading broadcast variable 4
  360. 21/03/05 18:53:56 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 10.9 KB, free 366.1 MB)
  361. 21/03/05 18:53:56 INFO TorrentBroadcast: Reading broadcast variable 4 took 6 ms
  362. 21/03/05 18:53:56 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 25.4 KB, free 366.1 MB)
  363. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 0, fetching them
  364. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@spark-worker-6cc46b5d7-nlnfw:34605)
  365. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 0, fetching them
  366. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Got the output locations
  367. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
  368. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  369. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks including 2 local blocks and 0 remote blocks
  370. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  371. 21/03/05 18:53:56 INFO MemoryStore: Block rdd_26_1 stored as values in memory (estimated size 16.0 B, free 366.1 MB)
  372. 21/03/05 18:53:56 INFO Executor: Finished task 1.0 in stage 3.0 (TID 7). 5814 bytes result sent to driver
  373. 21/03/05 18:53:56 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 6-1
  374. 21/03/05 18:53:56 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 6-2
  375. 21/03/05 18:53:56 INFO GrpcDataService: Beam Fn Data client connected.
  376. 21/03/05 18:53:56 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  377. payload: "\n\021\n\017localhost:45271"
  378. capabilities: "beam:coder:bytes:v1"
  379. capabilities: "beam:coder:string_utf8:v1"
  380. capabilities: "beam:coder:kv:v1"
  381. capabilities: "beam:coder:bool:v1"
  382. capabilities: "beam:coder:varint:v1"
  383. capabilities: "beam:coder:double:v1"
  384. capabilities: "beam:coder:iterable:v1"
  385. capabilities: "beam:coder:timer:v1"
  386. capabilities: "beam:coder:interval_window:v1"
  387. capabilities: "beam:coder:length_prefix:v1"
  388. capabilities: "beam:coder:global_window:v1"
  389. capabilities: "beam:coder:windowed_value:v1"
  390. capabilities: "beam:coder:param_windowed_value:v1"
  391. capabilities: "beam:coder:state_backed_iterable:v1"
  392. capabilities: "beam:coder:row:v1"
  393. capabilities: "beam:coder:sharded_key:v1"
  394. capabilities: "beam:protocol:progress_reporting:v0"
  395. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  396. capabilities: "beam:protocol:worker_status:v1"
  397. capabilities: "beam:combinefn:packed_python:v1"
  398. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  399. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  400. dependencies {
  401.   type_urn: "beam:artifact:type:file:v1"
  402.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  403.   role_urn: "beam:artifact:role:staging_to:v1"
  404.   role_payload: "\n\024pickled_main_session"
  405. }
  406.  
  407. 21/03/05 18:53:56 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  408. 21/03/05 18:54:01 INFO MemoryStore: Block rdd_26_0 stored as values in memory (estimated size 512.0 B, free 366.1 MB)
  409. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 3.0 (TID 6). 12005 bytes result sent to driver
  410. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 8
  411. 21/03/05 18:54:01 INFO Executor: Running task 0.0 in stage 4.0 (TID 8)
  412. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 9
  413. 21/03/05 18:54:01 INFO Executor: Running task 1.0 in stage 4.0 (TID 9)
  414. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 7
  415. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 15.6 KB, free 366.1 MB)
  416. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 7 took 6 ms
  417. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 35.4 KB, free 366.1 MB)
  418. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_0 locally
  419. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_1 locally
  420. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 4.0 (TID 8). 5384 bytes result sent to driver
  421. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 7-1
  422. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 7-2
  423. 21/03/05 18:54:01 INFO GrpcDataService: Beam Fn Data client connected.
  424. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 6
  425. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.1 MB)
  426. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 6 took 6 ms
  427. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 288.0 B, free 366.1 MB)
  428. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 5
  429. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 901.0 B, free 366.1 MB)
  430. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 5 took 5 ms
  431. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 488.0 B, free 366.1 MB)
  432. 21/03/05 18:54:01 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  433. payload: "\n\021\n\017localhost:45271"
  434. capabilities: "beam:coder:bytes:v1"
  435. capabilities: "beam:coder:string_utf8:v1"
  436. capabilities: "beam:coder:kv:v1"
  437. capabilities: "beam:coder:bool:v1"
  438. capabilities: "beam:coder:varint:v1"
  439. capabilities: "beam:coder:double:v1"
  440. capabilities: "beam:coder:iterable:v1"
  441. capabilities: "beam:coder:timer:v1"
  442. capabilities: "beam:coder:interval_window:v1"
  443. capabilities: "beam:coder:length_prefix:v1"
  444. capabilities: "beam:coder:global_window:v1"
  445. capabilities: "beam:coder:windowed_value:v1"
  446. capabilities: "beam:coder:param_windowed_value:v1"
  447. capabilities: "beam:coder:state_backed_iterable:v1"
  448. capabilities: "beam:coder:row:v1"
  449. capabilities: "beam:coder:sharded_key:v1"
  450. capabilities: "beam:protocol:progress_reporting:v0"
  451. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  452. capabilities: "beam:protocol:worker_status:v1"
  453. capabilities: "beam:combinefn:packed_python:v1"
  454. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  455. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  456. dependencies {
  457.   type_urn: "beam:artifact:type:file:v1"
  458.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  459.   role_urn: "beam:artifact:role:staging_to:v1"
  460.   role_payload: "\n\024pickled_main_session"
  461. }
  462.  
  463. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  464. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  465. 21/03/05 18:54:01 INFO Executor: Finished task 1.0 in stage 4.0 (TID 9). 11383 bytes result sent to driver
  466. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 10
  467. 21/03/05 18:54:01 INFO Executor: Running task 0.0 in stage 5.0 (TID 10)
  468. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 11
  469. 21/03/05 18:54:01 INFO Executor: Running task 1.0 in stage 5.0 (TID 11)
  470. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 11
  471. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 16.5 KB, free 366.2 MB)
  472. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 11 took 19 ms
  473. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 37.8 KB, free 366.2 MB)
  474. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_1 locally
  475. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_0 locally
  476. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 5.0 (TID 10). 5470 bytes result sent to driver
  477. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 8-1
  478. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 8-2
  479. 21/03/05 18:54:01 INFO GrpcDataService: Beam Fn Data client connected.
  480. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 9
  481. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.2 MB)
  482. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 9 took 6 ms
  483. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 288.0 B, free 366.2 MB)
  484. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 8
  485. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 702.0 B, free 366.2 MB)
  486. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 8 took 5 ms
  487. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 176.0 B, free 366.2 MB)
  488. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 10
  489. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 901.0 B, free 366.2 MB)
  490. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 10 took 5 ms
  491. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 488.0 B, free 366.2 MB)
  492. 21/03/05 18:54:01 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  493. payload: "\n\021\n\017localhost:45271"
  494. capabilities: "beam:coder:bytes:v1"
  495. capabilities: "beam:coder:string_utf8:v1"
  496. capabilities: "beam:coder:kv:v1"
  497. capabilities: "beam:coder:bool:v1"
  498. capabilities: "beam:coder:varint:v1"
  499. capabilities: "beam:coder:double:v1"
  500. capabilities: "beam:coder:iterable:v1"
  501. capabilities: "beam:coder:timer:v1"
  502. capabilities: "beam:coder:interval_window:v1"
  503. capabilities: "beam:coder:length_prefix:v1"
  504. capabilities: "beam:coder:global_window:v1"
  505. capabilities: "beam:coder:windowed_value:v1"
  506. capabilities: "beam:coder:param_windowed_value:v1"
  507. capabilities: "beam:coder:state_backed_iterable:v1"
  508. capabilities: "beam:coder:row:v1"
  509. capabilities: "beam:coder:sharded_key:v1"
  510. capabilities: "beam:protocol:progress_reporting:v0"
  511. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  512. capabilities: "beam:protocol:worker_status:v1"
  513. capabilities: "beam:combinefn:packed_python:v1"
  514. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  515. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  516. dependencies {
  517.   type_urn: "beam:artifact:type:file:v1"
  518.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  519.   role_urn: "beam:artifact:role:staging_to:v1"
  520.   role_payload: "\n\024pickled_main_session"
  521. }
  522.  
  523. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  524. 21/03/05 18:54:01 INFO Executor: Finished task 1.0 in stage 5.0 (TID 11). 10642 bytes result sent to driver
  525. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
  526. 21/03/05 18:54:01 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
  527. ```
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!