Guest User

spark_worker_beam_logs

a guest
Mar 5th, 2021
345
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Java 36.37 KB | None | 0 0
  1. Terminal logs:
  2.  
  3. ```
  4. INFO:apache_beam.internal.gcp.auth:socket default timeout is 60.0 seconds.
  5. INFO:oauth2client.client:Timeout attempting to reach GCE metadata service.
  6. WARNING:apache_beam.internal.gcp.auth:Unable to find default credentials to use: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
  7. Connecting anonymously.
  8. INFO:apache_beam.runners.worker.worker_pool_main:Listening for workers at localhost:45271
  9. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
  10. INFO:root:Default Python SDK image for environment is apache/beam_python3.7_sdk:2.28.0
  11. INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7ff7d00898c8> ====================
  12. INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7ff7d008a048> ====================
  13. INFO:apache_beam.runners.portability.abstract_job_service:Artifact server started on port 39389
  14. INFO:apache_beam.runners.portability.abstract_job_service:Running job 'job-1148ac23-bb81-40de-873c-1b2e3e073d14'
  15. INFO:apache_beam.runners.portability.spark_uber_jar_job_server:Submitted Spark job with ID driver-20210305185322-0000
  16. INFO:apache_beam.runners.portability.portable_runner:Environment "LOOPBACK" has started a component necessary for the execution. Be sure to run the pipeline using
  17.   with Pipeline() as p:
  18.     p.apply(..)
  19. This ensures that the pipeline finishes before this program exits.
  20. INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED
  21. INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING
  22. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  23. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33687.
  24. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  25. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  26. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36447.
  27. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  28. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:46519
  29. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  30. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  31. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  32. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  33. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  34. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  35. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:36067.
  36. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  37. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  38. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36875.
  39. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  40. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:39275
  41. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  42. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  43. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  44. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  45. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  46. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  47. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41825.
  48. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  49. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  50. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34653.
  51. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  52. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:42789
  53. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  54. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  55. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  56. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  57. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  58. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  59. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41493.
  60. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  61. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  62. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:37073.
  63. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  64. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41953
  65. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  66. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  67. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  68. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  69. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  70. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  71. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:41671.
  72. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  73. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  74. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:36677.
  75. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  76. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:41877
  77. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  78. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  79. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  80. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  81. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  82. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  83. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:33037.
  84. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  85. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  86. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:34155.
  87. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  88. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:33409
  89. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  90. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  91. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  92. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  93. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  94. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  95. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:40475.
  96. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  97. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  98. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:42731.
  99. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  100. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:32995
  101. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  102. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  103. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  104. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  105. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  106. INFO:apache_beam.runners.worker.statecache:Creating state cache with size 0
  107. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure control channel for localhost:43895.
  108. INFO:apache_beam.runners.worker.sdk_worker:Control channel established.
  109. INFO:apache_beam.runners.worker.sdk_worker:Initializing SDKHarness with unbounded number of workers.
  110. INFO:apache_beam.runners.worker.sdk_worker:Creating insecure state channel for localhost:38813.
  111. INFO:apache_beam.runners.worker.sdk_worker:State channel established.
  112. INFO:apache_beam.runners.worker.data_plane:Creating client data channel for localhost:43165
  113. INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 2 (skipped: 0), batches: 2, num_threads: 2
  114. INFO:apache_beam.io.filebasedsink:Renamed 2 shards in 0.10 seconds.
  115. INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
  116. INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight requests to complete
  117. INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data channels.
  118. INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state handlers.
  119. INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
  120. INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
  121.  
  122. ```
  123. Spark worker stderr:
  124.  
  125. ```
  126. 21/03/05 18:53:29 INFO Executor: Adding file:/opt/spark/work/app-20210305185326-0000/0/./api-asn1-api-1.0.0-M20.jar to class loader
  127. 21/03/05 18:53:29 INFO TorrentBroadcast: Started reading broadcast variable 0
  128. 21/03/05 18:53:29 INFO TransportClientFactory: Successfully created connection to spark-worker-6cc46b5d7-nlnfw/172.18.0.20:40957 after 1 ms (0 ms spent in bootstraps)
  129. 21/03/05 18:53:29 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 12.3 KB, free 366.3 MB)
  130. 21/03/05 18:53:29 INFO TorrentBroadcast: Reading broadcast variable 0 took 66 ms
  131. 21/03/05 18:53:29 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 32.4 KB, free 366.3 MB)
  132. 21/03/05 18:53:29 INFO MemoryStore: Block rdd_13_0 stored as values in memory (estimated size 16.0 B, free 366.3 MB)
  133. 21/03/05 18:53:29 INFO MemoryStore: Block rdd_17_0 stored as values in memory (estimated size 16.0 B, free 366.3 MB)
  134. 21/03/05 18:53:29 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 5427 bytes result sent to driver
  135. 21/03/05 18:53:31 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 1-1
  136. 21/03/05 18:53:31 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 1-2
  137. 21/03/05 18:53:31 INFO GrpcDataService: Beam Fn Data client connected.
  138. 21/03/05 18:53:31 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  139. payload: "\n\021\n\017localhost:45271"
  140. capabilities: "beam:coder:bytes:v1"
  141. capabilities: "beam:coder:string_utf8:v1"
  142. capabilities: "beam:coder:kv:v1"
  143. capabilities: "beam:coder:bool:v1"
  144. capabilities: "beam:coder:varint:v1"
  145. capabilities: "beam:coder:double:v1"
  146. capabilities: "beam:coder:iterable:v1"
  147. capabilities: "beam:coder:timer:v1"
  148. capabilities: "beam:coder:interval_window:v1"
  149. capabilities: "beam:coder:length_prefix:v1"
  150. capabilities: "beam:coder:global_window:v1"
  151. capabilities: "beam:coder:windowed_value:v1"
  152. capabilities: "beam:coder:param_windowed_value:v1"
  153. capabilities: "beam:coder:state_backed_iterable:v1"
  154. capabilities: "beam:coder:row:v1"
  155. capabilities: "beam:coder:sharded_key:v1"
  156. capabilities: "beam:protocol:progress_reporting:v0"
  157. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  158. capabilities: "beam:protocol:worker_status:v1"
  159. capabilities: "beam:combinefn:packed_python:v1"
  160. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  161. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  162. dependencies {
  163.   type_urn: "beam:artifact:type:file:v1"
  164.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  165.   role_urn: "beam:artifact:role:staging_to:v1"
  166.   role_payload: "\n\024pickled_main_session"
  167. }
  168.  
  169. 21/03/05 18:53:31 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  170. 21/03/05 18:53:31 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  171. 21/03/05 18:53:36 INFO MemoryStore: Block rdd_13_1 stored as values in memory (estimated size 192.0 B, free 366.3 MB)
  172. 21/03/05 18:53:36 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 2-1
  173. 21/03/05 18:53:36 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 2-2
  174. 21/03/05 18:53:36 INFO GrpcDataService: Beam Fn Data client connected.
  175. 21/03/05 18:53:36 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  176. payload: "\n\021\n\017localhost:45271"
  177. capabilities: "beam:coder:bytes:v1"
  178. capabilities: "beam:coder:string_utf8:v1"
  179. capabilities: "beam:coder:kv:v1"
  180. capabilities: "beam:coder:bool:v1"
  181. capabilities: "beam:coder:varint:v1"
  182. capabilities: "beam:coder:double:v1"
  183. capabilities: "beam:coder:iterable:v1"
  184. capabilities: "beam:coder:timer:v1"
  185. capabilities: "beam:coder:interval_window:v1"
  186. capabilities: "beam:coder:length_prefix:v1"
  187. capabilities: "beam:coder:global_window:v1"
  188. capabilities: "beam:coder:windowed_value:v1"
  189. capabilities: "beam:coder:param_windowed_value:v1"
  190. capabilities: "beam:coder:state_backed_iterable:v1"
  191. capabilities: "beam:coder:row:v1"
  192. capabilities: "beam:coder:sharded_key:v1"
  193. capabilities: "beam:protocol:progress_reporting:v0"
  194. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  195. capabilities: "beam:protocol:worker_status:v1"
  196. capabilities: "beam:combinefn:packed_python:v1"
  197. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  198. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  199. dependencies {
  200.   type_urn: "beam:artifact:type:file:v1"
  201.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  202.   role_urn: "beam:artifact:role:staging_to:v1"
  203.   role_payload: "\n\024pickled_main_session"
  204. }
  205.  
  206. 21/03/05 18:53:36 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  207. 21/03/05 18:53:36 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  208. 21/03/05 18:53:41 INFO MemoryStore: Block rdd_17_1 stored as values in memory (estimated size 256.0 B, free 366.3 MB)
  209. 21/03/05 18:53:41 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 17382 bytes result sent to driver
  210. 21/03/05 18:53:41 INFO CoarseGrainedExecutorBackend: Got assigned task 2
  211. 21/03/05 18:53:41 INFO Executor: Running task 0.0 in stage 1.0 (TID 2)
  212. 21/03/05 18:53:41 INFO CoarseGrainedExecutorBackend: Got assigned task 3
  213. 21/03/05 18:53:41 INFO Executor: Running task 1.0 in stage 1.0 (TID 3)
  214. 21/03/05 18:53:41 INFO TorrentBroadcast: Started reading broadcast variable 2
  215. 21/03/05 18:53:41 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 18.1 KB, free 366.2 MB)
  216. 21/03/05 18:53:41 INFO TorrentBroadcast: Reading broadcast variable 2 took 6 ms
  217. 21/03/05 18:53:41 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 42.1 KB, free 366.2 MB)
  218. 21/03/05 18:53:41 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 5480 bytes result sent to driver
  219. 21/03/05 18:53:41 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 3-1
  220. 21/03/05 18:53:41 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 3-2
  221. 21/03/05 18:53:41 INFO GrpcDataService: Beam Fn Data client connected.
  222. 21/03/05 18:53:42 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  223. payload: "\n\021\n\017localhost:45271"
  224. capabilities: "beam:coder:bytes:v1"
  225. capabilities: "beam:coder:string_utf8:v1"
  226. capabilities: "beam:coder:kv:v1"
  227. capabilities: "beam:coder:bool:v1"
  228. capabilities: "beam:coder:varint:v1"
  229. capabilities: "beam:coder:double:v1"
  230. capabilities: "beam:coder:iterable:v1"
  231. capabilities: "beam:coder:timer:v1"
  232. capabilities: "beam:coder:interval_window:v1"
  233. capabilities: "beam:coder:length_prefix:v1"
  234. capabilities: "beam:coder:global_window:v1"
  235. capabilities: "beam:coder:windowed_value:v1"
  236. capabilities: "beam:coder:param_windowed_value:v1"
  237. capabilities: "beam:coder:state_backed_iterable:v1"
  238. capabilities: "beam:coder:row:v1"
  239. capabilities: "beam:coder:sharded_key:v1"
  240. capabilities: "beam:protocol:progress_reporting:v0"
  241. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  242. capabilities: "beam:protocol:worker_status:v1"
  243. capabilities: "beam:combinefn:packed_python:v1"
  244. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  245. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  246. dependencies {
  247.   type_urn: "beam:artifact:type:file:v1"
  248.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  249.   role_urn: "beam:artifact:role:staging_to:v1"
  250.   role_payload: "\n\024pickled_main_session"
  251. }
  252.  
  253. 21/03/05 18:53:42 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  254. 21/03/05 18:53:42 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  255. 21/03/05 18:53:46 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 4-1
  256. 21/03/05 18:53:46 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 4-2
  257. 21/03/05 18:53:46 INFO GrpcDataService: Beam Fn Data client connected.
  258. 21/03/05 18:53:47 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  259. payload: "\n\021\n\017localhost:45271"
  260. capabilities: "beam:coder:bytes:v1"
  261. capabilities: "beam:coder:string_utf8:v1"
  262. capabilities: "beam:coder:kv:v1"
  263. capabilities: "beam:coder:bool:v1"
  264. capabilities: "beam:coder:varint:v1"
  265. capabilities: "beam:coder:double:v1"
  266. capabilities: "beam:coder:iterable:v1"
  267. capabilities: "beam:coder:timer:v1"
  268. capabilities: "beam:coder:interval_window:v1"
  269. capabilities: "beam:coder:length_prefix:v1"
  270. capabilities: "beam:coder:global_window:v1"
  271. capabilities: "beam:coder:windowed_value:v1"
  272. capabilities: "beam:coder:param_windowed_value:v1"
  273. capabilities: "beam:coder:state_backed_iterable:v1"
  274. capabilities: "beam:coder:row:v1"
  275. capabilities: "beam:coder:sharded_key:v1"
  276. capabilities: "beam:protocol:progress_reporting:v0"
  277. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  278. capabilities: "beam:protocol:worker_status:v1"
  279. capabilities: "beam:combinefn:packed_python:v1"
  280. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  281. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  282. dependencies {
  283.   type_urn: "beam:artifact:type:file:v1"
  284.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  285.   role_urn: "beam:artifact:role:staging_to:v1"
  286.   role_payload: "\n\024pickled_main_session"
  287. }
  288.  
  289. 21/03/05 18:53:47 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  290. 21/03/05 18:53:51 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  291. 21/03/05 18:53:51 INFO Executor: Finished task 1.0 in stage 1.0 (TID 3). 22237 bytes result sent to driver
  292. 21/03/05 18:53:51 INFO CoarseGrainedExecutorBackend: Got assigned task 4
  293. 21/03/05 18:53:51 INFO Executor: Running task 0.0 in stage 2.0 (TID 4)
  294. 21/03/05 18:53:51 INFO CoarseGrainedExecutorBackend: Got assigned task 5
  295. 21/03/05 18:53:51 INFO Executor: Running task 1.0 in stage 2.0 (TID 5)
  296. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Updating epoch to 1 and clearing cache
  297. 21/03/05 18:53:51 INFO TorrentBroadcast: Started reading broadcast variable 3
  298. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 18.2 KB, free 366.2 MB)
  299. 21/03/05 18:53:51 INFO TorrentBroadcast: Reading broadcast variable 3 took 6 ms
  300. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 39.9 KB, free 366.1 MB)
  301. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 1, fetching them
  302. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 1, fetching them
  303. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@spark-worker-6cc46b5d7-nlnfw:34605)
  304. 21/03/05 18:53:51 INFO MapOutputTrackerWorker: Got the output locations
  305. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local blocks and 0 remote blocks
  306. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Getting 1 non-empty blocks including 1 local blocks and 0 remote blocks
  307. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms
  308. 21/03/05 18:53:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 6 ms
  309. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 5-1
  310. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-3
  311. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-2
  312. 21/03/05 18:53:51 INFO GrpcDataService: Beam Fn Data client connected.
  313. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-4
  314. 21/03/05 18:53:51 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 5-5
  315. 21/03/05 18:53:51 INFO TorrentBroadcast: Started reading broadcast variable 1
  316. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.1 MB)
  317. 21/03/05 18:53:51 INFO TorrentBroadcast: Reading broadcast variable 1 took 8 ms
  318. 21/03/05 18:53:51 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 288.0 B, free 366.1 MB)
  319. 21/03/05 18:53:52 INFO Executor: Finished task 0.0 in stage 2.0 (TID 4). 20647 bytes result sent to driver
  320. 21/03/05 18:53:52 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  321. payload: "\n\021\n\017localhost:45271"
  322. capabilities: "beam:coder:bytes:v1"
  323. capabilities: "beam:coder:string_utf8:v1"
  324. capabilities: "beam:coder:kv:v1"
  325. capabilities: "beam:coder:bool:v1"
  326. capabilities: "beam:coder:varint:v1"
  327. capabilities: "beam:coder:double:v1"
  328. capabilities: "beam:coder:iterable:v1"
  329. capabilities: "beam:coder:timer:v1"
  330. capabilities: "beam:coder:interval_window:v1"
  331. capabilities: "beam:coder:length_prefix:v1"
  332. capabilities: "beam:coder:global_window:v1"
  333. capabilities: "beam:coder:windowed_value:v1"
  334. capabilities: "beam:coder:param_windowed_value:v1"
  335. capabilities: "beam:coder:state_backed_iterable:v1"
  336. capabilities: "beam:coder:row:v1"
  337. capabilities: "beam:coder:sharded_key:v1"
  338. capabilities: "beam:protocol:progress_reporting:v0"
  339. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  340. capabilities: "beam:protocol:worker_status:v1"
  341. capabilities: "beam:combinefn:packed_python:v1"
  342. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  343. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  344. dependencies {
  345.   type_urn: "beam:artifact:type:file:v1"
  346.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  347.   role_urn: "beam:artifact:role:staging_to:v1"
  348.   role_payload: "\n\024pickled_main_session"
  349. }
  350.  
  351. 21/03/05 18:53:52 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  352. 21/03/05 18:53:52 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  353. 21/03/05 18:53:56 INFO Executor: Finished task 1.0 in stage 2.0 (TID 5). 20647 bytes result sent to driver
  354. 21/03/05 18:53:56 INFO CoarseGrainedExecutorBackend: Got assigned task 6
  355. 21/03/05 18:53:56 INFO Executor: Running task 0.0 in stage 3.0 (TID 6)
  356. 21/03/05 18:53:56 INFO CoarseGrainedExecutorBackend: Got assigned task 7
  357. 21/03/05 18:53:56 INFO Executor: Running task 1.0 in stage 3.0 (TID 7)
  358. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Updating epoch to 2 and clearing cache
  359. 21/03/05 18:53:56 INFO TorrentBroadcast: Started reading broadcast variable 4
  360. 21/03/05 18:53:56 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 10.9 KB, free 366.1 MB)
  361. 21/03/05 18:53:56 INFO TorrentBroadcast: Reading broadcast variable 4 took 6 ms
  362. 21/03/05 18:53:56 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 25.4 KB, free 366.1 MB)
  363. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 0, fetching them
  364. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@spark-worker-6cc46b5d7-nlnfw:34605)
  365. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 0, fetching them
  366. 21/03/05 18:53:56 INFO MapOutputTrackerWorker: Got the output locations
  367. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks including 0 local blocks and 0 remote blocks
  368. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  369. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks including 2 local blocks and 0 remote blocks
  370. 21/03/05 18:53:56 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
  371. 21/03/05 18:53:56 INFO MemoryStore: Block rdd_26_1 stored as values in memory (estimated size 16.0 B, free 366.1 MB)
  372. 21/03/05 18:53:56 INFO Executor: Finished task 1.0 in stage 3.0 (TID 7). 5814 bytes result sent to driver
  373. 21/03/05 18:53:56 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 6-1
  374. 21/03/05 18:53:56 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 6-2
  375. 21/03/05 18:53:56 INFO GrpcDataService: Beam Fn Data client connected.
  376. 21/03/05 18:53:56 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  377. payload: "\n\021\n\017localhost:45271"
  378. capabilities: "beam:coder:bytes:v1"
  379. capabilities: "beam:coder:string_utf8:v1"
  380. capabilities: "beam:coder:kv:v1"
  381. capabilities: "beam:coder:bool:v1"
  382. capabilities: "beam:coder:varint:v1"
  383. capabilities: "beam:coder:double:v1"
  384. capabilities: "beam:coder:iterable:v1"
  385. capabilities: "beam:coder:timer:v1"
  386. capabilities: "beam:coder:interval_window:v1"
  387. capabilities: "beam:coder:length_prefix:v1"
  388. capabilities: "beam:coder:global_window:v1"
  389. capabilities: "beam:coder:windowed_value:v1"
  390. capabilities: "beam:coder:param_windowed_value:v1"
  391. capabilities: "beam:coder:state_backed_iterable:v1"
  392. capabilities: "beam:coder:row:v1"
  393. capabilities: "beam:coder:sharded_key:v1"
  394. capabilities: "beam:protocol:progress_reporting:v0"
  395. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  396. capabilities: "beam:protocol:worker_status:v1"
  397. capabilities: "beam:combinefn:packed_python:v1"
  398. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  399. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  400. dependencies {
  401.   type_urn: "beam:artifact:type:file:v1"
  402.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  403.   role_urn: "beam:artifact:role:staging_to:v1"
  404.   role_payload: "\n\024pickled_main_session"
  405. }
  406.  
  407. 21/03/05 18:53:56 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  408. 21/03/05 18:54:01 INFO MemoryStore: Block rdd_26_0 stored as values in memory (estimated size 512.0 B, free 366.1 MB)
  409. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 3.0 (TID 6). 12005 bytes result sent to driver
  410. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 8
  411. 21/03/05 18:54:01 INFO Executor: Running task 0.0 in stage 4.0 (TID 8)
  412. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 9
  413. 21/03/05 18:54:01 INFO Executor: Running task 1.0 in stage 4.0 (TID 9)
  414. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 7
  415. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 15.6 KB, free 366.1 MB)
  416. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 7 took 6 ms
  417. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 35.4 KB, free 366.1 MB)
  418. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_0 locally
  419. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_1 locally
  420. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 4.0 (TID 8). 5384 bytes result sent to driver
  421. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 7-1
  422. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 7-2
  423. 21/03/05 18:54:01 INFO GrpcDataService: Beam Fn Data client connected.
  424. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 6
  425. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.1 MB)
  426. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 6 took 6 ms
  427. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 288.0 B, free 366.1 MB)
  428. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 5
  429. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 901.0 B, free 366.1 MB)
  430. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 5 took 5 ms
  431. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 488.0 B, free 366.1 MB)
  432. 21/03/05 18:54:01 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  433. payload: "\n\021\n\017localhost:45271"
  434. capabilities: "beam:coder:bytes:v1"
  435. capabilities: "beam:coder:string_utf8:v1"
  436. capabilities: "beam:coder:kv:v1"
  437. capabilities: "beam:coder:bool:v1"
  438. capabilities: "beam:coder:varint:v1"
  439. capabilities: "beam:coder:double:v1"
  440. capabilities: "beam:coder:iterable:v1"
  441. capabilities: "beam:coder:timer:v1"
  442. capabilities: "beam:coder:interval_window:v1"
  443. capabilities: "beam:coder:length_prefix:v1"
  444. capabilities: "beam:coder:global_window:v1"
  445. capabilities: "beam:coder:windowed_value:v1"
  446. capabilities: "beam:coder:param_windowed_value:v1"
  447. capabilities: "beam:coder:state_backed_iterable:v1"
  448. capabilities: "beam:coder:row:v1"
  449. capabilities: "beam:coder:sharded_key:v1"
  450. capabilities: "beam:protocol:progress_reporting:v0"
  451. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  452. capabilities: "beam:protocol:worker_status:v1"
  453. capabilities: "beam:combinefn:packed_python:v1"
  454. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  455. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  456. dependencies {
  457.   type_urn: "beam:artifact:type:file:v1"
  458.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  459.   role_urn: "beam:artifact:role:staging_to:v1"
  460.   role_payload: "\n\024pickled_main_session"
  461. }
  462.  
  463. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  464. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  465. 21/03/05 18:54:01 INFO Executor: Finished task 1.0 in stage 4.0 (TID 9). 11383 bytes result sent to driver
  466. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 10
  467. 21/03/05 18:54:01 INFO Executor: Running task 0.0 in stage 5.0 (TID 10)
  468. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Got assigned task 11
  469. 21/03/05 18:54:01 INFO Executor: Running task 1.0 in stage 5.0 (TID 11)
  470. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 11
  471. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 16.5 KB, free 366.2 MB)
  472. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 11 took 19 ms
  473. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 37.8 KB, free 366.2 MB)
  474. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_1 locally
  475. 21/03/05 18:54:01 INFO BlockManager: Found block rdd_13_0 locally
  476. 21/03/05 18:54:01 INFO Executor: Finished task 0.0 in stage 5.0 (TID 10). 5470 bytes result sent to driver
  477. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: Beam Fn Control client connected with id 8-1
  478. 21/03/05 18:54:01 INFO FnApiControlClientPoolService: getProcessBundleDescriptor request with id 8-2
  479. 21/03/05 18:54:01 INFO GrpcDataService: Beam Fn Data client connected.
  480. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 9
  481. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 805.0 B, free 366.2 MB)
  482. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 9 took 6 ms
  483. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 288.0 B, free 366.2 MB)
  484. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 8
  485. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 702.0 B, free 366.2 MB)
  486. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 8 took 5 ms
  487. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 176.0 B, free 366.2 MB)
  488. 21/03/05 18:54:01 INFO TorrentBroadcast: Started reading broadcast variable 10
  489. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 901.0 B, free 366.2 MB)
  490. 21/03/05 18:54:01 INFO TorrentBroadcast: Reading broadcast variable 10 took 5 ms
  491. 21/03/05 18:54:01 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 488.0 B, free 366.2 MB)
  492. 21/03/05 18:54:01 INFO DefaultJobBundleFactory: Closing environment urn: "beam:env:external:v1"
  493. payload: "\n\021\n\017localhost:45271"
  494. capabilities: "beam:coder:bytes:v1"
  495. capabilities: "beam:coder:string_utf8:v1"
  496. capabilities: "beam:coder:kv:v1"
  497. capabilities: "beam:coder:bool:v1"
  498. capabilities: "beam:coder:varint:v1"
  499. capabilities: "beam:coder:double:v1"
  500. capabilities: "beam:coder:iterable:v1"
  501. capabilities: "beam:coder:timer:v1"
  502. capabilities: "beam:coder:interval_window:v1"
  503. capabilities: "beam:coder:length_prefix:v1"
  504. capabilities: "beam:coder:global_window:v1"
  505. capabilities: "beam:coder:windowed_value:v1"
  506. capabilities: "beam:coder:param_windowed_value:v1"
  507. capabilities: "beam:coder:state_backed_iterable:v1"
  508. capabilities: "beam:coder:row:v1"
  509. capabilities: "beam:coder:sharded_key:v1"
  510. capabilities: "beam:protocol:progress_reporting:v0"
  511. capabilities: "beam:protocol:harness_monitoring_infos:v1"
  512. capabilities: "beam:protocol:worker_status:v1"
  513. capabilities: "beam:combinefn:packed_python:v1"
  514. capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.28.0"
  515. capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"
  516. dependencies {
  517.   type_urn: "beam:artifact:type:file:v1"
  518.   type_payload: "\n\253\001classpath://BEAM-PIPELINE/pipeline/artifacts/job-1148ac23-bb81-40de-873c-1b2e3e073d14/397ec066630585f52675c8faecf6b4eb1869d4395539c58074633521df15d78e-pickled_main_session"
  519.   role_urn: "beam:artifact:role:staging_to:v1"
  520.   role_payload: "\n\024pickled_main_session"
  521. }
  522.  
  523. 21/03/05 18:54:01 WARN BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint.
  524. 21/03/05 18:54:01 INFO Executor: Finished task 1.0 in stage 5.0 (TID 11). 10642 bytes result sent to driver
  525. 21/03/05 18:54:01 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
  526. 21/03/05 18:54:01 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
  527. ```
Add Comment
Please, Sign In to add comment