API tools faq
paste
Guest User

Untitled

a guest
Oct 9th, 2020
50
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 37.79 KB | None | 0 0
raw download clone embed print report
  1. worker_1 | [2020-10-09 09:47:55,443] {{timeout.py:42}} ERROR - Process timed out, PID: 33
  2. worker_1 | Exception ignored in: <function _get_module_lock.<locals>.cb at 0x7f7be5fb63b0>
  3. worker_1 | Traceback (most recent call last):
  4. worker_1 | File "<frozen importlib._bootstrap>", line 177, in cb
  5. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  6. worker_1 | raise AirflowTaskTimeout(self.error_message)
  7. worker_1 | [2020-10-09 09:47:55,561] {{timeout.py:42}} ERROR - Process timed out, PID: 40
  8. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 33
  9. worker_1 | [2020-10-09 09:47:55,597] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  10. worker_1 | Traceback (most recent call last):
  11. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  12. worker_1 | m = imp.load_source(mod_name, filepath)
  13. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  14. worker_1 | module = _load(spec)
  15. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  16. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  17. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  18. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  19. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  20. worker_1 | from scripts.common.google_api2 import Google_console_api
  21. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 1, in <module>
  22. worker_1 | from google.cloud import storage
  23. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/__init__.py", line 40, in <module>
  24. worker_1 | from google.cloud.storage.bucket import Bucket
  25. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/bucket.py", line 26, in <module>
  26. worker_1 | from google.api_core import datetime_helpers
  27. worker_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  28. worker_1 | File "<frozen importlib._bootstrap>", line 963, in _find_and_load_unlocked
  29. worker_1 | File "<frozen importlib._bootstrap>", line 906, in _find_spec
  30. worker_1 | File "<frozen importlib._bootstrap_external>", line 1280, in find_spec
  31. worker_1 | File "<frozen importlib._bootstrap_external>", line 1252, in _get_spec
  32. worker_1 | File "<frozen importlib._bootstrap_external>", line 1390, in find_spec
  33. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  34. worker_1 | raise AirflowTaskTimeout(self.error_message)
  35. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 40
  36. worker_1 | Traceback (most recent call last):
  37. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  38. worker_1 | args.func(args)
  39. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  40. worker_1 | return f(*args, **kwargs)
  41. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  42. worker_1 | dag = get_dag(args)
  43. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  44. worker_1 | [2020-10-09 09:47:57,170] {{timeout.py:42}} ERROR - Process timed out, PID: 39
  45. worker_1 | 'parse.'.format(args.dag_id))
  46. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  47. worker_1 | [2020-10-09 09:47:57,313] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  48. worker_1 | Traceback (most recent call last):
  49. worker_1 | File "<frozen importlib._bootstrap_external>", line 87, in _path_is_mode_type
  50. worker_1 | File "<frozen importlib._bootstrap_external>", line 81, in _path_stat
  51. worker_1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/__init__.cpython-37m-x86_64-linux-gnu.so'
  52. worker_1 |
  53. worker_1 | During handling of the above exception, another exception occurred:
  54. worker_1 |
  55. worker_1 | Traceback (most recent call last):
  56. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  57. worker_1 | m = imp.load_source(mod_name, filepath)
  58. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  59. worker_1 | module = _load(spec)
  60. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  61. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  62. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  63. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  64. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  65. worker_1 | from scripts.common.google_api2 import Google_console_api
  66. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  67. worker_1 | import pandas as pd
  68. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 50, in <module>
  69. worker_1 | import pandas.core.config_init
  70. worker_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  71. worker_1 | File "<frozen importlib._bootstrap>", line 963, in _find_and_load_unlocked
  72. worker_1 | File "<frozen importlib._bootstrap>", line 906, in _find_spec
  73. worker_1 | File "<frozen importlib._bootstrap_external>", line 1280, in find_spec
  74. worker_1 | File "<frozen importlib._bootstrap_external>", line 1252, in _get_spec
  75. worker_1 | File "<frozen importlib._bootstrap_external>", line 1383, in find_spec
  76. worker_1 | File "<frozen importlib._bootstrap_external>", line 95, in _path_isfile
  77. worker_1 | File "<frozen importlib._bootstrap_external>", line 87, in _path_is_mode_type
  78. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  79. worker_1 | raise AirflowTaskTimeout(self.error_message)
  80. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 39
  81. worker_1 | Traceback (most recent call last):
  82. worker_1 | [2020-10-09 09:47:57,512] {{timeout.py:42}} ERROR - Process timed out, PID: 43
  83. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  84. worker_1 | args.func(args)
  85. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  86. worker_1 | return f(*args, **kwargs)
  87. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  88. worker_1 | dag = get_dag(args)
  89. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  90. worker_1 | 'parse.'.format(args.dag_id))
  91. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  92. worker_1 | [2020-10-09 09:47:57,882] {{timeout.py:42}} ERROR - Process timed out, PID: 41
  93. worker_1 | [2020-10-09 09:47:57,533] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  94. worker_1 | Traceback (most recent call last):
  95. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  96. worker_1 | m = imp.load_source(mod_name, filepath)
  97. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  98. worker_1 | module = _load(spec)
  99. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  100. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  101. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  102. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  103. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  104. worker_1 | from scripts.common.google_api2 import Google_console_api
  105. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  106. worker_1 | import pandas as pd
  107. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 50, in <module>
  108. worker_1 | import pandas.core.config_init
  109. worker_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  110. worker_1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  111. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  112. worker_1 | File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  113. worker_1 | File "<frozen importlib._bootstrap_external>", line 818, in get_code
  114. worker_1 | File "<frozen importlib._bootstrap_external>", line 916, in get_data
  115. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  116. worker_1 | raise AirflowTaskTimeout(self.error_message)
  117. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 43
  118. worker_1 | Traceback (most recent call last):
  119. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  120. worker_1 | args.func(args)
  121. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  122. worker_1 | return f(*args, **kwargs)
  123. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  124. worker_1 | dag = get_dag(args)
  125. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  126. worker_1 | 'parse.'.format(args.dag_id))
  127. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  128. worker_1 | [2020-10-09 09:47:58,745] {{timeout.py:42}} ERROR - Process timed out, PID: 37
  129. worker_1 | [[2020-10-09 09:47:58,801] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  130. worker_1 | Traceback (most recent call last):
  131. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  132. worker_1 | m = imp.load_source(mod_name, filepath)
  133. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  134. worker_1 | module = _load(spec)
  135. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  136. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  137. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  138. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  139. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  140. worker_1 | from scripts.common.google_api2 import Google_console_api
  141. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  142. worker_1 | import pandas as pd
  143. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 30, in <module>
  144. worker_1 | from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
  145. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
  146. worker_1 | from pandas._libs.interval import Interval
  147. worker_1 | File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
  148. worker_1 | File "pandas/_libs/hashtable.pyx", line 1, in init pandas._libs.hashtable
  149. worker_1 | File "pandas/_libs/missing.pyx", line 1, in init pandas._libs.missing
  150. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/tslibs/__init__.py", line 30, in <module>
  151. worker_1 | from .conversion import OutOfBoundsTimedelta, localize_pydatetime
  152. worker_1 | File "pandas/_libs/tslibs/conversion.pyx", line 58, in init pandas._libs.tslibs.conversion
  153. worker_1 | File "pandas/_libs/tslibs/parsing.pyx", line 1, in init pandas._libs.tslibs.parsing
  154. worker_1 | File "pandas/_libs/tslibs/offsets.pyx", line 1, in init pandas._libs.tslibs.offsets
  155. worker_1 | File "pandas/_libs/tslibs/timestamps.pyx", line 47, in init pandas._libs.tslibs.timestamps
  156. worker_1 | File "pandas/_libs/tslibs/fields.pyx", line 16, in init pandas._libs.tslibs.fields
  157. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_config/localization.py", line 6, in <module>
  158. worker_1 | from contextlib import contextmanager
  159. worker_1 | File "<frozen importlib._bootstrap>", line 1009, in _handle_fromlist
  160. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  161. worker_1 | raise AirflowTaskTimeout(self.error_message)
  162. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 37
  163. worker_1 | 2020-10-09 09:47:59,174] {{timeout.py:42}} ERROR - Process timed out, PID: 36
  164. worker_1 | Traceback (most recent call last):
  165. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  166. worker_1 | args.func(args)
  167. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  168. worker_1 | return f(*args, **kwargs)
  169. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  170. worker_1 | dag = get_dag(args)
  171. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  172. worker_1 | 'parse.'.format(args.dag_id))
  173. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  174. worker_1 | [2020-10-09 09:47:59,308] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  175. worker_1 | Traceback (most recent call last):
  176. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  177. worker_1 | m = imp.load_source(mod_name, filepath)
  178. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  179. worker_1 | module = _load(spec)
  180. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  181. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  182. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  183. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  184. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  185. worker_1 | from scripts.common.google_api2 import Google_console_api
  186. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 1, in <module>
  187. worker_1 | from google.cloud import storage
  188. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/__init__.py", line 38, in <module>
  189. worker_1 | from google.cloud.storage.batch import Batch
  190. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/batch.py", line 31, in <module>
  191. worker_1 | from google.cloud.storage._http import Connection
  192. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/_http.py", line 17, in <module>
  193. worker_1 | from google.cloud import _http
  194. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/cloud/_http.py", line 25, in <module>
  195. worker_1 | from google.api_core.client_info import ClientInfo
  196. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/google/api_core/client_info.py", line 31, in <module>
  197. worker_1 | _GRPC_VERSION = pkg_resources.get_distribution("grpcio").version
  198. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 465, in get_distribution
  199. worker_1 | dist = get_provider(dist)
  200. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 341, in get_provider
  201. worker_1 | return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  202. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 884, in require
  203. worker_1 | needed = self.resolve(parse_requirements(requirements))
  204. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 757, in resolve
  205. worker_1 | env = Environment(self.entries)
  206. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 968, in __init__
  207. worker_1 | self.scan(search_path)
  208. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1000, in scan
  209. worker_1 | for dist in find_distributions(item):
  210. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2060, in find_on_path
  211. worker_1 | for dist in factory(fullpath):
  212. worker_1 | File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2124, in distributions_from_metadata
  213. worker_1 | entry = os.path.basename(path)
  214. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  215. worker_1 | raise AirflowTaskTimeout(self.error_message)
  216. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 36
  217. worker_1 | Traceback (most recent call last):
  218. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  219. worker_1 | args.func(args)
  220. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  221. worker_1 | return f(*args, **kwargs)
  222. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  223. worker_1 | dag = get_dag(args)
  224. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  225. worker_1 | 'parse.'.format(args.dag_id))
  226. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  227. worker_1 | [2020-10-09 09:48:02,471] {{timeout.py:42}} ERROR - Process timed out, PID: 35
  228. worker_1 | [2020-10-09 09:48:02,523] {{timeout.py:42}} ERROR - Process timed out, PID: 42
  229. worker_1 | [2020-10-09 09:48:02,603] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  230. worker_1 | Traceback (most recent call last):
  231. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  232. worker_1 | m = imp.load_source(mod_name, filepath)
  233. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  234. worker_1 | module = _load(spec)
  235. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  236. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  237. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  238. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  239. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  240. worker_1 | from scripts.common.google_api2 import Google_console_api
  241. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  242. worker_1 | import pandas as pd
  243. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 30, in <module>
  244. worker_1 | from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
  245. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
  246. worker_1 | from pandas._libs.interval import Interval
  247. worker_1 | File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
  248. worker_1 | File "pandas/_libs/hashtable.pyx", line 1, in init pandas._libs.hashtable
  249. worker_1 | File "pandas/_libs/missing.pyx", line 1, in init pandas._libs.missing
  250. worker_1 | File "<frozen importlib._bootstrap>", line 978, in _find_and_load
  251. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  252. worker_1 | raise AirflowTaskTimeout(self.error_message)
  253. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 42
  254. worker_1 | Traceback (most recent call last):
  255. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  256. worker_1 | args.func(args)
  257. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  258. worker_1 | return f(*args, **kwargs)
  259. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  260. worker_1 | [2020-10-09 09:48:02,548] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  261. worker_1 | Traceback (most recent call last):
  262. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  263. worker_1 | m = imp.load_source(mod_name, filepath)
  264. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  265. worker_1 | module = _load(spec)
  266. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  267. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  268. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  269. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  270. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  271. worker_1 | from scripts.common.google_api2 import Google_console_api
  272. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  273. worker_1 | import pandas as pd
  274. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 30, in <module>
  275. worker_1 | from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
  276. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
  277. worker_1 | from pandas._libs.interval import Interval
  278. worker_1 | File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
  279. worker_1 | File "pandas/_libs/hashtable.pyx", line 1, in init pandas._libs.hashtable
  280. worker_1 | File "pandas/_libs/missing.pyx", line 1, in init pandas._libs.missing
  281. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/tslibs/__init__.py", line 35, in <module>
  282. worker_1 | from .period import IncompatibleFrequency, Period
  283. worker_1 | File "pandas/_libs/tslibs/period.pyx", line 1, in init pandas._libs.tslibs.period
  284. worker_1 | File "<frozen importlib._bootstrap>", line 194, in _lock_unlock_module
  285. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  286. worker_1 | raise AirflowTaskTimeout(self.error_message)
  287. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 35
  288. worker_1 | Traceback (most recent call last):
  289. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  290. worker_1 | args.func(args)
  291. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  292. worker_1 | return f(*args, **kwargs)
  293. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  294. worker_1 | dag = get_dag(args)
  295. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  296. worker_1 | dag = get_dag(args)
  297. worker_1 | 'parse.'.format(args.dag_id))
  298. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  299. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  300. worker_1 | 'parse.'.format(args.dag_id))
  301. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  302. scheduler_1 | [2020-10-09 09:48:09,613] {{scheduler_job.py:951}} INFO - 60 tasks up for execution:
  303. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-24 04:05:00+00:00 [scheduled]>
  304. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-10-02 04:05:00+00:00 [scheduled]>
  305. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-21 04:05:00+00:00 [scheduled]>
  306. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-25 04:05:00+00:00 [scheduled]>
  307. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-29 04:05:00+00:00 [scheduled]>
  308. scheduler_1 | <TaskInstance: test_google_dag.com.nopowerup.idlelightcity 2020-09-21 04:05:00+00:00 [scheduled]>
  309. scheduler_1 | <TaskInstance: test_google_dag.com.nopowerup.idlelightcity 2020-09-25 04:05:00+00:00 [scheduled]>
  310. scheduler_1 | <TaskInstance: test_google_dag.com.nopowerup.idlelightcity 2020-09-29 04:05:00+00:00 [scheduled]>
  311. scheduler_1 | <TaskInstance: test_google_dag.com.nopowerup.idlelightcity 2020-10-02 04:05:00+00:00 [scheduled]>
  312. scheduler_1 | <TaskInstance: test_google_dag.com.playquantum.battler 2020-09-21 04:05:00+00:00 [scheduled]>
  313. scheduler_1 | <TaskInstance: test_google_dag.com.playquantum.battler 2020-09-22 04:05:00+00:00 [scheduled]>
  314. scheduler_1 | <TaskInstance: test_google_dag.com.playquantum.battler 2020-09-24 04:05:00+00:00 [scheduled]>
  315. scheduler_1 | <TaskInstance: test_google_dag.com.playquantum.battler 2020-09-25 04:05:00+00:00 [scheduled]>
  316. scheduler_1 | <TaskInstance: test_google_dag.com.playquantum.battler 2020-09-26 04:05:00+00:00 [scheduled]>
  317. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-22 04:05:00+00:00 [scheduled]>
  318. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-26 04:05:00+00:00 [scheduled]>
  319. scheduler_1 | <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-09-30 04:05:00+00:00 [scheduled]>
  320.  
  321.  
  322.  
  323.  
  324.  
  325.  
  326.  
  327. scheduler_1 | [2020-10-09 09:50:03,480] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  328. scheduler_1 | [2020-10-09 09:50:03,481] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.rmg.hotelteam 2020-10-01 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  329. scheduler_1 | [2020-10-09 09:50:03,504] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  330. scheduler_1 | [2020-10-09 09:50:03,504] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.rmg.idle.evil.party.clicker 2020-10-01 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  331. scheduler_1 | [2020-10-09 09:50:03,505] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  332. scheduler_1 | [2020-10-09 09:50:03,531] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.evilclicker.evilclicker 2020-10-02 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  333. scheduler_1 | [2020-10-09 09:50:03,532] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  334. scheduler_1 | [2020-10-09 09:50:03,532] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.nopowerup.idlelightcity 2020-10-02 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  335. scheduler_1 | [2020-10-09 09:50:03,532] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  336. scheduler_1 | [2020-10-09 09:50:03,577] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.playquantum.battler 2020-10-02 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  337. scheduler_1 | [2020-10-09 09:50:03,577] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  338. scheduler_1 | [2020-10-09 09:50:03,589] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.rmg.hotelteam 2020-10-02 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  339. scheduler_1 | [2020-10-09 09:50:03,589] {{scheduler_job.py:1010}} INFO - DAG test_google_dag has 16/16 running and queued tasks
  340. scheduler_1 | [2020-10-09 09:50:03,590] {{scheduler_job.py:1016}} INFO - Not executing <TaskInstance: test_google_dag.com.rmg.idle.evil.party.clicker 2020-10-02 04:05:00+00:00 [scheduled]> since the number of tasks running or queued from DAG test_google_dag is >= to the DAG's task concurrency limit of 16
  341. scheduler_1 | [2020-10-09 09:50:03,643] {{scheduler_job.py:1060}} INFO - Setting the following tasks to queued state:
  342. scheduler_1 |
  343. worker_1 | [2020-10-09 09:50:08,815: WARNING/MainProcess] process with pid=130 already exited
  344. worker_1 | [2020-10-09 09:50:08,961] {{timeout.py:42}} ERROR - Process timed out, PID: 77
  345. worker_1 | [2020-10-09 09:50:09,713: ERROR/MainProcess] Timed out waiting for UP message from <ForkProcess(ForkPoolWorker-40, started daemon)>
  346. worker_1 | [2020-10-09 09:50:09,907: ERROR/MainProcess] Timed out waiting for UP message from <ForkProcess(ForkPoolWorker-41, started daemon)>
  347. worker_1 | [2020-10-09 09:50:09,972: INFO/MainProcess] Received task: airflow.executors.celery_executor.execute_command[39745152-098d-471f-b918-2a4d1773be3e]
  348. worker_1 | [2020-10-09 09:50:10,483: INFO/ForkPoolWorker-4] Executing command in Celery: ['airflow', 'run', 'test_google_dag', 'com.rmg.hotelteam', '2020-09-18T04:05:00+00:00', '--local', '--pool', 'default_pool', '-sd', '/usr/local/airflow/dags/dag_google_api.py']
  349. worker_1 | [2020-10-09 09:50:10,483: INFO/ForkPoolWorker-8] Executing command in Celery: ['airflow', 'run', 'test_google_dag', 'com.rmg.hotelteam', '2020-09-17T04:05:00+00:00', '--local', '--pool', 'default_pool', '-sd', '/usr/local/airflow/dags/dag_google_api.py']
  350. worker_1 | [2020-10-09 09:50:10,527: INFO/MainProcess] Received task: airflow.executors.celery_executor.execute_command[5c25f47a-9d15-4674-b1c7-5c94c7be2362]
  351. worker_1 | [2020-10-09 09:50:09,041] {{dagbag.py:239}} ERROR - Failed to import: /usr/local/airflow/dags/dag_google_api.py
  352. worker_1 | Traceback (most recent call last):
  353. worker_1 | File "<frozen importlib._bootstrap_external>", line 87, in _path_is_mode_type
  354. worker_1 | File "<frozen importlib._bootstrap_external>", line 81, in _path_stat
  355. worker_1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/_libs/window/__init__.cpython-37m-x86_64-linux-gnu.so'
  356. worker_1 |
  357. worker_1 | During handling of the above exception, another exception occurred:
  358. worker_1 |
  359. worker_1 | Traceback (most recent call last):
  360. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file
  361. worker_1 | m = imp.load_source(mod_name, filepath)
  362. worker_1 | File "/usr/local/lib/python3.7/imp.py", line 171, in load_source
  363. worker_1 | module = _load(spec)
  364. worker_1 | File "<frozen importlib._bootstrap>", line 696, in _load
  365. worker_1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  366. worker_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  367. worker_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  368. worker_1 | File "/usr/local/airflow/dags/dag_google_api.py", line 3, in <module>
  369. worker_1 | from scripts.common.google_api2 import Google_console_api
  370. worker_1 | File "/usr/local/airflow/dags/scripts/common/google_api2.py", line 3, in <module>
  371. worker_1 | import pandas as pd
  372. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/__init__.py", line 52, in <module>
  373. worker_1 | from pandas.core.api import (
  374. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/api.py", line 29, in <module>
  375. worker_1 | from pandas.core.groupby import Grouper, NamedAgg
  376. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
  377. worker_1 | from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  378. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/groupby/generic.py", line 57, in <module>
  379. worker_1 | from pandas.core.aggregation import (
  380. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/aggregation.py", line 27, in <module>
  381. worker_1 | from pandas.core.series import FrameOrSeriesUnion, Series
  382. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/series.py", line 4983, in <module>
  383. worker_1 | Series._add_series_or_dataframe_operations()
  384. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/generic.py", line 10511, in _add_series_or_dataframe_operations
  385. worker_1 | from pandas.core.window import (
  386. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/window/__init__.py", line 1, in <module>
  387. worker_1 | from pandas.core.window.ewm import ExponentialMovingWindow # noqa:F401
  388. worker_1 | File "/usr/local/airflow/.local/lib/python3.7/site-packages/pandas/core/window/ewm.py", line 9, in <module>
  389. worker_1 | import pandas._libs.window.aggregations as window_aggregations
  390. worker_1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  391. worker_1 | File "<frozen importlib._bootstrap>", line 963, in _find_and_load_unlocked
  392. worker_1 | File "<frozen importlib._bootstrap>", line 906, in _find_spec
  393. worker_1 | File "<frozen importlib._bootstrap_external>", line 1280, in find_spec
  394. worker_1 | File "<frozen importlib._bootstrap_external>", line 1252, in _get_spec
  395. worker_1 | File "<frozen importlib._bootstrap_external>", line 1383, in find_spec
  396. worker_1 | File "<frozen importlib._bootstrap_external>", line 95, in _path_isfile
  397. worker_1 | File "<frozen importlib._bootstrap_external>", line 87, in _path_is_mode_type
  398. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
  399. worker_1 | raise AirflowTaskTimeout(self.error_message)
  400. worker_1 | airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 77
  401. worker_1 | Traceback (most recent call last):
  402. worker_1 | File "/usr/local/bin/airflow", line 37, in <module>
  403. worker_1 | args.func(args)
  404. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
  405. worker_1 | [2020-10-09 09:50:11,446: INFO/MainProcess] Received task: airflow.executors.celery_executor.execute_command[d91fac0f-0e08-4a49-a08b-ed1f6cc89a3a]
  406. worker_1 | [2020-10-09 09:50:11,466: INFO/MainProcess] Scaling up 1 processes.
  407. worker_1 | return f(*args, **kwargs)
  408. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 523, in run
  409. worker_1 | [2020-10-09 09:50:11,515: ERROR/MainProcess] Process 'ForkPoolWorker-41' pid:146 exited with 'signal 9 (SIGKILL)'
  410. worker_1 | dag = get_dag(args)
  411. worker_1 | File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line 149, in get_dag
  412. worker_1 | [2020-10-09 09:50:11,567: ERROR/MainProcess] Process 'ForkPoolWorker-40' pid:141 exited with 'signal 9 (SIGKILL)'
  413. worker_1 | 'parse.'.format(args.dag_id))
  414. worker_1 | airflow.exceptions.AirflowException: dag_id could not be found: test_google_dag. Either the dag did not exist or it failed to parse.
  415. postgres_1 | WARNING: could not open statistics file "pg_stat_tmp/global.stat": Operation not permitted
  416. webserver_1 | ____________ _____________
  417. webserver_1 | ____ |__( )_________ __/__ /________ __
  418. webserver_1 | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
  419. webserver_1 | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
  420. webserver_1 | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
  421. webserver_1 | [2020-10-09 09:50:17,819] {{__init__.py:51}} INFO - Using executor CeleryExecutor
  422. webserver_1 | [2020-10-09 09:50:17,825] {{dagbag.py:396}} INFO - Filling up the DagBag from /usr/local/airflow/dags
  423. scheduler_1 | [2020-10-09 09:50:17,912] {{scheduler_job.py:951}} INFO - 58 tasks up for execution:
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!