Advertisement
AlexScott

Hub Log

Apr 11th, 2022 (edited)
23
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. [I 2022-04-11 00:39:31.568 JupyterHub app:2769] Running JupyterHub version 2.2.2
  2. [I 2022-04-11 00:39:31.568 JupyterHub app:2799] Using Authenticator: oauthenticator.github.LocalGitHubOAuthenticator-14.2.0
  3. [I 2022-04-11 00:39:31.569 JupyterHub app:2799] Using Spawner: dockerspawner.swarmspawner.SwarmSpawner-12.1.0
  4. [I 2022-04-11 00:39:31.569 JupyterHub app:2799] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.2.2
  5. [I 2022-04-11 00:39:31.688 JupyterHub app:1606] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
  6. [D 2022-04-11 00:39:31.689 JupyterHub app:1773] Connecting to db: sqlite:///jupyterhub.sqlite
  7. [D 2022-04-11 00:39:31.720 JupyterHub orm:924] Stamping empty database with alembic revision 833da8570507
  8. [I 2022-04-11 00:39:31.726 alembic.runtime.migration migration:201] Context impl SQLiteImpl.
  9. [I 2022-04-11 00:39:31.726 alembic.runtime.migration migration:204] Will assume non-transactional DDL.
  10. [I 2022-04-11 00:39:31.744 alembic.runtime.migration migration:615] Running stamp_revision  -> 833da8570507
  11. [D 2022-04-11 00:39:31.745 alembic.runtime.migration migration:815] new branch insert 833da8570507
  12. [D 2022-04-11 00:39:32.070 JupyterHub app:2022] Loading roles into database
  13. [D 2022-04-11 00:39:32.158 JupyterHub roles:454] Assigning default role to User alex
  14. [I 2022-04-11 00:39:32.178 JupyterHub roles:366] Adding role admin for User: alex
  15. [I 2022-04-11 00:39:32.195 JupyterHub roles:366] Adding role user for User: alex
  16. [D 2022-04-11 00:39:32.226 JupyterHub auth:943] No UID for user alex
  17. [I 2022-04-11 00:39:32.227 JupyterHub auth:945] Creating user: adduser -q --gecos '""' --disabled-password alex
  18. [D 2022-04-11 00:39:32.446 JupyterHub auth:943] No UID for user mattbivins
  19. [I 2022-04-11 00:39:32.446 JupyterHub auth:945] Creating user: adduser -q --gecos '""' --disabled-password user1
  20. [D 2022-04-11 00:39:32.612 JupyterHub auth:943] No UID for user shuaiw96
  21. [I 2022-04-11 00:39:32.613 JupyterHub auth:945] Creating user: adduser -q --gecos '""' --disabled-password user2
  22. [D 2022-04-11 00:39:32.789 JupyterHub auth:943] No UID for user numbermatch
  23. [I 2022-04-11 00:39:32.790 JupyterHub auth:945] Creating user: adduser -q --gecos '""' --disabled-password user3
  24. [D 2022-04-11 00:39:32.972 JupyterHub app:2281] Purging expired APITokens
  25. [D 2022-04-11 00:39:32.979 JupyterHub app:2281] Purging expired OAuthCodes
  26. [D 2022-04-11 00:39:32.984 JupyterHub app:2114] Loading role assignments from config
  27. [D 2022-04-11 00:39:32.984 JupyterHub app:2185] Assigning 4 allowed_users to the user role
  28. [I 2022-04-11 00:39:33.019 JupyterHub roles:366] Adding role user for User: someuser1
  29. [I 2022-04-11 00:39:33.038 JupyterHub roles:366] Adding role user for User: someuser2
  30. [I 2022-04-11 00:39:33.056 JupyterHub roles:366] Adding role user for User: someuser3
  31. [D 2022-04-11 00:39:33.078 JupyterHub app:2427] Initializing spawners
  32. [D 2022-04-11 00:39:33.082 JupyterHub app:2558] Loaded users:
  33.    
  34. [I 2022-04-11 00:39:33.082 JupyterHub app:2838] Initialized 0 spawners in 0.005 seconds
  35. [I 2022-04-11 00:39:33.086 JupyterHub app:3051] Not starting proxy
  36. [D 2022-04-11 00:39:33.087 JupyterHub proxy:821] Proxy: Fetching GET http://proxy:8001/api/routes
  37. [D 2022-04-11 00:39:33.104 JupyterHub proxy:894] Omitting non-jupyterhub route '/'
  38. [I 2022-04-11 00:39:33.105 JupyterHub app:3087] Hub API listening on http://0.0.0.0:8081/hub/
  39. [I 2022-04-11 00:39:33.105 JupyterHub app:3089] Private Hub API connect url http://jupyterhub:8081/hub/
  40. [D 2022-04-11 00:39:33.106 JupyterHub proxy:343] Fetching routes to check
  41. [D 2022-04-11 00:39:33.106 JupyterHub proxy:821] Proxy: Fetching GET http://proxy:8001/api/routes
  42. [D 2022-04-11 00:39:33.109 JupyterHub proxy:894] Omitting non-jupyterhub route '/'
  43. [D 2022-04-11 00:39:33.109 JupyterHub proxy:346] Checking routes
  44. [I 2022-04-11 00:39:33.110 JupyterHub proxy:431] Adding route for Hub: / => http://jupyterhub:8081
  45. [D 2022-04-11 00:39:33.110 JupyterHub proxy:821] Proxy: Fetching POST http://proxy:8001/api/routes/
  46. [I 2022-04-11 00:39:33.115 JupyterHub app:3156] JupyterHub is now running, internal Hub API at http://jupyterhub:8081/hub/
  47. [D 2022-04-11 00:39:33.116 JupyterHub app:2762] It took 1.583 seconds for the Hub to start
  48. [I 2022-04-11 00:39:33.583 JupyterHub log:189] 302 GET / -> /hub/ (@192.168.86.1) 1.63ms
  49. [W 2022-04-11 00:39:33.639 JupyterHub base:400] Invalid cookie token
  50. [I 2022-04-11 00:39:33.640 JupyterHub log:189] 302 GET /hub/ -> /hub/login?next=%2Fhub%2F (@192.168.86.1) 5.91ms
  51. [I 2022-04-11 00:39:33.737 JupyterHub log:189] 200 GET /hub/login?next=%2Fhub%2F (@192.168.86.1) 54.57ms
  52. [I 2022-04-11 00:39:35.164 JupyterHub oauth2:111] OAuth redirect: 'https://jupyter.actuary.llc/hub/oauth_callback'
  53. [D 2022-04-11 00:39:35.164 JupyterHub base:565] Setting cookie oauthenticator-state: {'httponly': True, 'secure': True, 'expires_days': 1}
  54. [I 2022-04-11 00:39:35.166 JupyterHub log:189] 302 GET /hub/oauth_login?next=%2Fhub%2F -> https://github.com/login/oauth/authorize?response_type=code&redirect_uri=https%3A%2F%2Fjupyter.actuary.llc%2Fhub%2Foauth_callback&client_id=e5809312786c446a0882&state=[secret] (@192.168.86.1) 2.77ms
  55. [D 2022-04-11 00:39:36.085 JupyterHub roles:454] Assigning default role to User alex
  56. [D 2022-04-11 00:39:36.095 JupyterHub base:569] Setting cookie for alex: jupyterhub-hub-login
  57. [D 2022-04-11 00:39:36.096 JupyterHub base:565] Setting cookie jupyterhub-hub-login: {'httponly': True, 'secure': True, 'path': '/hub/'}
  58. [I 2022-04-11 00:39:36.096 JupyterHub base:813] User logged in: alex
  59. [I 2022-04-11 00:39:36.101 JupyterHub log:189] 302 GET /hub/oauth_callback?code=[secret]&state=[secret] -> /hub/ (@192.168.86.1) 607.42ms
  60. [D 2022-04-11 00:39:36.135 JupyterHub base:281] Recording first activity for <User(alex 0/1 running)>
  61. [D 2022-04-11 00:39:36.159 JupyterHub user:399] Creating <class 'dockerspawner.swarmspawner.SwarmSpawner'> for alex:
  62. [I 2022-04-11 00:39:36.163 JupyterHub log:189] 302 GET /hub/ -> /hub/spawn (alex@192.168.86.1) 29.12ms
  63. [D 2022-04-11 00:39:36.204 JupyterHub scopes:491] Checking access via scope servers
  64. [D 2022-04-11 00:39:36.204 JupyterHub scopes:389] Unrestricted access to /hub/spawn via servers
  65. [D 2022-04-11 00:39:36.205 JupyterHub pages:207] Serving options form for alex
  66. [I 2022-04-11 00:39:36.212 JupyterHub log:189] 200 GET /hub/spawn (alex@192.168.86.1) 11.56ms
  67. [D 2022-04-11 00:39:38.993 JupyterHub scopes:491] Checking access via scope servers
  68. [D 2022-04-11 00:39:38.994 JupyterHub scopes:389] Unrestricted access to /hub/spawn via servers
  69. [D 2022-04-11 00:39:38.994 JupyterHub pages:255] Triggering spawn with supplied form options for alex
  70. [D 2022-04-11 00:39:38.995 JupyterHub base:931] Initiating spawn for alex
  71. [D 2022-04-11 00:39:38.996 JupyterHub base:935] 0/100 concurrent spawns
  72. [D 2022-04-11 00:39:38.996 JupyterHub base:940] 0 active servers
  73. [D 2022-04-11 00:39:39.005 JupyterHub roles:477] Checking token permissions against requested role server
  74. [I 2022-04-11 00:39:39.009 JupyterHub roles:482] Adding role server to token: <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  75. [I 2022-04-11 00:39:39.055 JupyterHub provider:607] Creating oauth client jupyterhub-user-alex
  76. [D 2022-04-11 00:39:39.115 JupyterHub user:728] Calling Spawner.start for alex
  77. [D 2022-04-11 00:39:39.116 JupyterHub dockerspawner:1190] Skipping pull of jupyter/datascience-notebook:hub-2.2.2
  78. [D 2022-04-11 00:39:39.116 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  79. [I 2022-04-11 00:39:39.138 JupyterHub dockerspawner:988] Service 'jupyter-alex' is gone
  80. [W 2022-04-11 00:39:39.998 JupyterHub base:188] Rolling back dirty objects IdentitySet([<Server(0.0.0.0:8888)>])
  81. [I 2022-04-11 00:39:40.005 JupyterHub log:189] 302 POST /hub/spawn -> /hub/spawn-pending/alex (alex@192.168.86.1) 1008.71ms
  82. [D 2022-04-11 00:39:40.037 JupyterHub scopes:491] Checking access via scope servers
  83. [D 2022-04-11 00:39:40.038 JupyterHub scopes:389] Unrestricted access to /hub/spawn-pending/alex via servers
  84. [I 2022-04-11 00:39:40.041 JupyterHub pages:401] alex is pending spawn
  85. [I 2022-04-11 00:39:40.047 JupyterHub log:189] 200 GET /hub/spawn-pending/alex (alex@192.168.86.1) 15.02ms
  86. [I 2022-04-11 00:39:40.173 JupyterHub dockerspawner:1272] Created service jupyter-alex (id: rdo89ui) from image jupyter/datascience-notebook:hub-2.2.2
  87. [I 2022-04-11 00:39:40.174 JupyterHub dockerspawner:1296] Starting service jupyter-alex (id: rdo89ui)
  88. [D 2022-04-11 00:39:40.174 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  89. [D 2022-04-11 00:39:40.174 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  90. [D 2022-04-11 00:39:40.184 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  91. [D 2022-04-11 00:39:40.251 JupyterHub scopes:491] Checking access via scope read:servers
  92. [D 2022-04-11 00:39:40.251 JupyterHub scopes:389] Unrestricted access to /hub/api/users/alex/server/progress via read:servers
  93. [D 2022-04-11 00:39:41.186 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  94. [D 2022-04-11 00:39:41.186 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  95. [D 2022-04-11 00:39:41.197 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  96. [D 2022-04-11 00:39:42.698 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  97. [D 2022-04-11 00:39:42.699 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  98. [D 2022-04-11 00:39:42.708 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  99. [I 2022-04-11 00:39:44.517 JupyterHub log:189] 200 GET /hub/api (@10.0.5.8) 1.29ms
  100. [D 2022-04-11 00:39:44.534 JupyterHub base:281] Recording first activity for <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  101. [D 2022-04-11 00:39:44.553 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  102. [D 2022-04-11 00:39:44.558 JupyterHub scopes:491] Checking access via scope users:activity
  103. [D 2022-04-11 00:39:44.558 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  104. [D 2022-04-11 00:39:44.561 JupyterHub users:855] Activity for user alex: 2022-04-11T00:39:44.463476Z
  105. [D 2022-04-11 00:39:44.562 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:39:44.463476Z
  106. [I 2022-04-11 00:39:44.583 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.4) 53.38ms
  107. [D 2022-04-11 00:39:44.960 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  108. [D 2022-04-11 00:39:44.960 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  109. [D 2022-04-11 00:39:44.972 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  110. [D 2022-04-11 00:39:48.348 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  111. [D 2022-04-11 00:39:48.348 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  112. [D 2022-04-11 00:39:48.358 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  113. [W 2022-04-11 00:39:48.997 JupyterHub base:1060] User alex is slow to start (timeout=10)
  114. [D 2022-04-11 00:39:53.422 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  115. [D 2022-04-11 00:39:53.422 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  116. [D 2022-04-11 00:39:53.433 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  117. [D 2022-04-11 00:40:01.029 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  118. [D 2022-04-11 00:40:01.030 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  119. [D 2022-04-11 00:40:01.041 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  120. [D 2022-04-11 00:40:12.044 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  121. [D 2022-04-11 00:40:12.044 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  122. [D 2022-04-11 00:40:12.054 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  123. [D 2022-04-11 00:40:23.057 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  124. [D 2022-04-11 00:40:23.057 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  125. [D 2022-04-11 00:40:23.067 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  126. [D 2022-04-11 00:40:34.069 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  127. [D 2022-04-11 00:40:34.070 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  128. [D 2022-04-11 00:40:34.080 JupyterHub swarmspawner:280] Service rdo89ui state: ready
  129. [I 2022-04-11 00:40:38.554 JupyterHub log:189] 200 GET /hub/api (@10.0.5.9) 1.48ms
  130. [D 2022-04-11 00:40:38.584 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  131. [D 2022-04-11 00:40:38.585 JupyterHub scopes:491] Checking access via scope users:activity
  132. [D 2022-04-11 00:40:38.585 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  133. [D 2022-04-11 00:40:38.588 JupyterHub users:855] Activity for user alex: 2022-04-11T00:40:38.501314Z
  134. [D 2022-04-11 00:40:38.588 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:40:38.501314Z
  135. [I 2022-04-11 00:40:38.604 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.4) 37.11ms
  136. [D 2022-04-11 00:40:45.082 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  137. [D 2022-04-11 00:40:45.083 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  138. [D 2022-04-11 00:40:45.093 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  139. [D 2022-04-11 00:40:56.094 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  140. [D 2022-04-11 00:40:56.094 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  141. [D 2022-04-11 00:40:56.105 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  142. [D 2022-04-11 00:41:07.107 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  143. [D 2022-04-11 00:41:07.107 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  144. [D 2022-04-11 00:41:07.119 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  145. [D 2022-04-11 00:41:18.120 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  146. [D 2022-04-11 00:41:18.121 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  147. [D 2022-04-11 00:41:18.131 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  148. [D 2022-04-11 00:41:29.133 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  149. [D 2022-04-11 00:41:29.133 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  150. [D 2022-04-11 00:41:29.144 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  151. [I 2022-04-11 00:41:32.527 JupyterHub log:189] 200 GET /hub/api (@10.0.5.10) 1.30ms
  152. [D 2022-04-11 00:41:32.560 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  153. [D 2022-04-11 00:41:32.561 JupyterHub scopes:491] Checking access via scope users:activity
  154. [D 2022-04-11 00:41:32.561 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  155. [D 2022-04-11 00:41:32.564 JupyterHub users:855] Activity for user alex: 2022-04-11T00:41:32.474970Z
  156. [D 2022-04-11 00:41:32.565 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:41:32.474970Z
  157. [I 2022-04-11 00:41:32.580 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.4) 39.65ms
  158. [D 2022-04-11 00:41:40.146 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  159. [D 2022-04-11 00:41:40.146 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  160. [D 2022-04-11 00:41:40.158 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  161. [D 2022-04-11 00:41:51.159 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  162. [D 2022-04-11 00:41:51.159 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  163. [D 2022-04-11 00:41:51.171 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  164. [D 2022-04-11 00:42:02.173 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  165. [D 2022-04-11 00:42:02.173 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  166. [D 2022-04-11 00:42:02.185 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  167. [D 2022-04-11 00:42:13.187 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  168. [D 2022-04-11 00:42:13.187 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  169. [D 2022-04-11 00:42:13.199 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  170. [D 2022-04-11 00:42:24.200 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  171. [D 2022-04-11 00:42:24.201 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  172. [D 2022-04-11 00:42:24.210 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  173. [I 2022-04-11 00:42:26.646 JupyterHub log:189] 200 GET /hub/api (@10.0.5.4) 1.24ms
  174. [D 2022-04-11 00:42:26.678 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  175. [D 2022-04-11 00:42:26.679 JupyterHub scopes:491] Checking access via scope users:activity
  176. [D 2022-04-11 00:42:26.679 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  177. [D 2022-04-11 00:42:26.683 JupyterHub users:855] Activity for user alex: 2022-04-11T00:42:26.592204Z
  178. [D 2022-04-11 00:42:26.683 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:42:26.592204Z
  179. [I 2022-04-11 00:42:26.696 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.11) 37.34ms
  180. [D 2022-04-11 00:42:35.211 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  181. [D 2022-04-11 00:42:35.212 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  182. [D 2022-04-11 00:42:35.222 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  183. [D 2022-04-11 00:42:46.224 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  184. [D 2022-04-11 00:42:46.224 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  185. [D 2022-04-11 00:42:46.238 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  186. [D 2022-04-11 00:42:57.240 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  187. [D 2022-04-11 00:42:57.240 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  188. [D 2022-04-11 00:42:57.253 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  189. [D 2022-04-11 00:43:08.254 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  190. [D 2022-04-11 00:43:08.254 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  191. [D 2022-04-11 00:43:08.265 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  192. [D 2022-04-11 00:43:19.267 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  193. [D 2022-04-11 00:43:19.267 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  194. [D 2022-04-11 00:43:19.280 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  195. [I 2022-04-11 00:43:20.657 JupyterHub log:189] 200 GET /hub/api (@10.0.5.12) 1.45ms
  196. [D 2022-04-11 00:43:20.690 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  197. [D 2022-04-11 00:43:20.691 JupyterHub scopes:491] Checking access via scope users:activity
  198. [D 2022-04-11 00:43:20.691 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  199. [D 2022-04-11 00:43:20.694 JupyterHub users:855] Activity for user alex: 2022-04-11T00:43:20.603208Z
  200. [D 2022-04-11 00:43:20.694 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:43:20.603208Z
  201. [I 2022-04-11 00:43:20.710 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.4) 38.10ms
  202. [D 2022-04-11 00:43:30.282 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  203. [D 2022-04-11 00:43:30.282 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  204. [D 2022-04-11 00:43:30.293 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  205. [D 2022-04-11 00:43:41.295 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  206. [D 2022-04-11 00:43:41.296 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  207. [D 2022-04-11 00:43:41.306 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  208. [D 2022-04-11 00:43:52.308 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  209. [D 2022-04-11 00:43:52.308 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  210. [D 2022-04-11 00:43:52.321 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  211. [D 2022-04-11 00:44:03.323 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  212. [D 2022-04-11 00:44:03.323 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  213. [D 2022-04-11 00:44:03.336 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  214. [D 2022-04-11 00:44:14.337 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  215. [D 2022-04-11 00:44:14.338 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  216. [D 2022-04-11 00:44:14.349 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  217. [I 2022-04-11 00:44:15.034 JupyterHub log:189] 200 GET /hub/api (@10.0.5.13) 1.30ms
  218. [D 2022-04-11 00:44:15.071 JupyterHub scopes:301] Authenticated with token <APIToken('67b0...', user='alex', client_id='jupyterhub')>
  219. [D 2022-04-11 00:44:15.072 JupyterHub scopes:491] Checking access via scope users:activity
  220. [D 2022-04-11 00:44:15.072 JupyterHub scopes:402] Argument-based access to /hub/api/users/alex/activity via users:activity
  221. [D 2022-04-11 00:44:15.075 JupyterHub users:855] Activity for user alex: 2022-04-11T00:44:14.832537Z
  222. [D 2022-04-11 00:44:15.075 JupyterHub users:873] Activity on server alex/: 2022-04-11T00:44:14.832537Z
  223. [I 2022-04-11 00:44:15.090 JupyterHub log:189] 200 POST /hub/api/users/alex/activity (alex@10.0.5.4) 37.31ms
  224. [D 2022-04-11 00:44:25.350 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  225. [D 2022-04-11 00:44:25.351 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  226. [D 2022-04-11 00:44:25.362 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  227. [D 2022-04-11 00:44:33.118 JupyterHub proxy:821] Proxy: Fetching GET http://proxy:8001/api/routes
  228. [D 2022-04-11 00:44:33.131 JupyterHub proxy:346] Checking routes
  229. [D 2022-04-11 00:44:36.363 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  230. [D 2022-04-11 00:44:36.364 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  231. [D 2022-04-11 00:44:36.375 JupyterHub swarmspawner:280] Service rdo89ui state: starting
  232. [W 2022-04-11 00:44:39.116 JupyterHub user:807] alex's server failed to start in 300 seconds, giving up.
  233.    
  234.    Common causes of this timeout, and debugging tips:
  235.    
  236.    1. Everything is working, but it took too long.
  237.       To fix: increase `Spawner.start_timeout` configuration
  238.       to a number of seconds that is enough for spawners to finish starting.
  239.    2. The server didn't finish starting,
  240.        or it crashed due to a configuration issue.
  241.        Check the single-user server's logs for hints at what needs fixing.
  242.    
  243. [D 2022-04-11 00:44:39.117 JupyterHub user:913] Stopping alex
  244. [D 2022-04-11 00:44:39.117 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  245. [D 2022-04-11 00:44:39.117 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  246. [D 2022-04-11 00:44:39.128 JupyterHub swarmspawner:144] Service rdo89ui status: {'ContainerStatus': {'ContainerID': '68e970d01d89c0b32c99dd76b2c5e55d4ccdf8246236f3090a39e45b6c7eb345',
  247.                         'ExitCode': 0,
  248.                         'PID': 0},
  249.     'Message': 'starting',
  250.     'PortStatus': {},
  251.     'State': 'starting',
  252.     'Timestamp': '2022-04-11T00:44:10.207278557Z'}
  253. [I 2022-04-11 00:44:39.128 JupyterHub dockerspawner:1390] Stopping service jupyter-alex (id: rdo89ui)
  254. [I 2022-04-11 00:44:39.128 JupyterHub swarmspawner:257] Removing service rdo89uiyyl43p29obu5khilue
  255. [D 2022-04-11 00:44:39.167 JupyterHub user:933] Deleting oauth client jupyterhub-user-alex
  256. [D 2022-04-11 00:44:39.196 JupyterHub user:936] Finished stopping alex
  257. [E 2022-04-11 00:44:39.234 JupyterHub gen:623] Exception in Future <Task finished name='Task-30' coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn() done, defined at /usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/base.py:951> exception=TimeoutError('Timeout')> after timeout
  258.    Traceback (most recent call last):
  259.      File "/usr/local/lib/python3.8/dist-packages/tornado/gen.py", line 618, in error_callback
  260.        future.result()
  261.      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/base.py", line 958, in finish_user_spawn
  262.        await spawn_future
  263.      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/user.py", line 833, in spawn
  264.        raise e
  265.      File "/usr/local/lib/python3.8/dist-packages/jupyterhub/user.py", line 732, in spawn
  266.        url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
  267.    tornado.util.TimeoutError: Timeout
  268.    
  269. [I 2022-04-11 00:44:39.239 JupyterHub log:189] 200 GET /hub/api/users/alex/server/progress (alex@192.168.86.1) 298993.58ms
  270. [D 2022-04-11 00:44:47.378 JupyterHub swarmspawner:155] Getting task of service 'jupyter-alex'
  271. [D 2022-04-11 00:44:47.378 JupyterHub dockerspawner:982] Getting service 'jupyter-alex'
  272. [I 2022-04-11 00:44:47.384 JupyterHub dockerspawner:988] Service 'jupyter-alex' is gone
  273. [E 2022-04-11 00:44:47.384 JupyterHub gen:623] Exception in Future <Task finished name='Task-31' coro=<DockerSpawner.start() done, defined at /usr/local/lib/python3.8/dist-packages/dockerspawner/dockerspawner.py:1221> exception=RuntimeError('Service jupyter-alex not found')> after timeout
  274.    Traceback (most recent call last):
  275.      File "/usr/local/lib/python3.8/dist-packages/tornado/gen.py", line 618, in error_callback
  276.        future.result()
  277.      File "/usr/local/lib/python3.8/dist-packages/dockerspawner/dockerspawner.py", line 1304, in start
  278.        await self.start_object()
  279.      File "/usr/local/lib/python3.8/dist-packages/dockerspawner/swarmspawner.py", line 276, in start_object
  280.        raise RuntimeError("Service %s not found" % self.service_name)
  281.    RuntimeError: Service jupyter-alex not found
Advertisement
RAW Paste Data Copied
Advertisement