API tools faq
paste
Lissanro

vLLM MQLLMEngine error

Lissanro
Sep 25th, 2024
84
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 7.90 KB | None | 0 0
raw download clone embed print report
  1. INFO 09-25 20:30:58 launcher.py:19] Available routes are:
  2. INFO 09-25 20:30:58 launcher.py:27] Route: /openapi.json, Methods: GET, HEAD
  3. INFO 09-25 20:30:58 launcher.py:27] Route: /docs, Methods: GET, HEAD
  4. INFO 09-25 20:30:58 launcher.py:27] Route: /docs/oauth2-redirect, Methods: GET, HEAD
  5. INFO 09-25 20:30:58 launcher.py:27] Route: /redoc, Methods: GET, HEAD
  6. INFO 09-25 20:30:58 launcher.py:27] Route: /health, Methods: GET
  7. INFO 09-25 20:30:58 launcher.py:27] Route: /tokenize, Methods: POST
  8. INFO 09-25 20:30:58 launcher.py:27] Route: /detokenize, Methods: POST
  9. INFO 09-25 20:30:58 launcher.py:27] Route: /v1/models, Methods: GET
  10. INFO 09-25 20:30:58 launcher.py:27] Route: /version, Methods: GET
  11. INFO 09-25 20:30:58 launcher.py:27] Route: /v1/chat/completions, Methods: POST
  12. INFO 09-25 20:30:58 launcher.py:27] Route: /v1/completions, Methods: POST
  13. INFO 09-25 20:30:58 launcher.py:27] Route: /v1/embeddings, Methods: POST
  14. INFO: Started server process [1]
  15. INFO: Waiting for application startup.
  16. INFO: Application startup complete.
  17. INFO: Uvicorn running on http://0.0.0.0:18888 (Press CTRL+C to quit)
  18. INFO: 172.17.0.1:53128 - "GET /v1/models HTTP/1.1" 200 OK
  19. INFO: 172.17.0.1:44596 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
  20. INFO: 172.17.0.1:48458 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
  21. WARNING 09-25 21:00:14 chat_utils.py:381] 'image_url.detail' is currently not supported and will be ignored.
  22. INFO: 172.17.0.1:37420 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
  23. WARNING 09-25 21:00:31 chat_utils.py:381] 'image_url.detail' is currently not supported and will be ignored.
  24. INFO 09-25 21:00:31 logger.py:36] Received request chat-703d9fdd207c4de6b754adf5d164b680: prompt: '<|im_start|>system\n...
  25. (VllmWorkerProcess pid=115) return torch.load(io.BytesIO(b))
  26. ERROR 09-25 21:00:48 client.py:250] TimeoutError("MQLLMEngine didn't reply within 10000ms")
  27. ERROR 09-25 21:00:48 client.py:250] Traceback (most recent call last):
  28. ERROR 09-25 21:00:48 client.py:250] File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/client.py", line 146, in run_check_health_loop
  29. ERROR 09-25 21:00:48 client.py:250] await self._await_ack(error_message="Health check failed.",
  30. ERROR 09-25 21:00:48 client.py:250] File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/client.py", line 297, in _await_ack
  31. ERROR 09-25 21:00:48 client.py:250] raise TimeoutError("MQLLMEngine didn't reply within "
  32. ERROR 09-25 21:00:48 client.py:250] TimeoutError: MQLLMEngine didn't reply within 10000ms
  33. ERROR: Exception in ASGI application
  34. Traceback (most recent call last):
  35. File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 257, in __call__
  36. await wrap(partial(self.listen_for_disconnect, receive))
  37. File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 253, in wrap
  38. await func()
  39. File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 230, in listen_for_disconnect
  40. message = await receive()
  41. ^^^^^^^^^^^^^^^
  42. File "/usr/local/lib/python3.12/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive
  43. await self.message_event.wait()
  44. File "/usr/lib/python3.12/asyncio/locks.py", line 212, in wait
  45. await fut
  46. asyncio.exceptions.CancelledError: Cancelled by cancel scope 7e79f8ea1e80
  47.  
  48. During handling of the above exception, another exception occurred:
  49.  
  50. + Exception Group Traceback (most recent call last):
  51. | File "/usr/local/lib/python3.12/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
  52. | result = await app( # type: ignore[func-returns-value]
  53. | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  54. | File "/usr/local/lib/python3.12/dist-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
  55. | return await self.app(scope, receive, send)
  56. | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  57. | File "/usr/local/lib/python3.12/dist-packages/fastapi/applications.py", line 1054, in __call__
  58. | await super().__call__(scope, receive, send)
  59. | File "/usr/local/lib/python3.12/dist-packages/starlette/applications.py", line 113, in __call__
  60. | await self.middleware_stack(scope, receive, send)
  61. | File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 187, in __call__
  62. | raise exc
  63. | File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/errors.py", line 165, in __call__
  64. | await self.app(scope, receive, _send)
  65. | File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/cors.py", line 85, in __call__
  66. | await self.app(scope, receive, send)
  67. | File "/usr/local/lib/python3.12/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
  68. | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  69. | File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
  70. | raise exc
  71. | File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
  72. | await app(scope, receive, sender)
  73. | File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 715, in __call__
  74. | await self.middleware_stack(scope, receive, send)
  75. | File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 735, in app
  76. | await route.handle(scope, receive, send)
  77. | File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 288, in handle
  78. | await self.app(scope, receive, send)
  79. | File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 76, in app
  80. | await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  81. | File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
  82. | raise exc
  83. | File "/usr/local/lib/python3.12/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
  84. | await app(scope, receive, sender)
  85. | File "/usr/local/lib/python3.12/dist-packages/starlette/routing.py", line 74, in app
  86. | await response(scope, receive, send)
  87. | File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 250, in __call__
  88. | async with anyio.create_task_group() as task_group:
  89. | ^^^^^^^^^^^^^^^^^^^^^^^^^
  90. | File "/usr/local/lib/python3.12/dist-packages/anyio/_backends/_asyncio.py", line 736, in __aexit__
  91. | raise BaseExceptionGroup(
  92. | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  93. +-+---------------- 1 ----------------
  94. | Traceback (most recent call last):
  95. | File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 253, in wrap
  96. | await func()
  97. | File "/usr/local/lib/python3.12/dist-packages/starlette/responses.py", line 242, in stream_response
  98. | async for chunk in self.body_iterator:
  99. | File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 300, in chat_completion_stream_generator
  100. | async for res in result_generator:
  101. | File "/usr/local/lib/python3.12/dist-packages/vllm/utils.py", line 450, in iterate_with_cancellation
  102. | item = await awaits[0]
  103. | ^^^^^^^^^^^^^^^
  104. | File "/usr/local/lib/python3.12/dist-packages/vllm/engine/multiprocessing/client.py", line 490, in _process_request
  105. | raise request_output
  106. | vllm.engine.multiprocessing.MQEngineDeadError: Engine loop is not running. Inspect the stacktrace to find the original error: TimeoutError("MQLLMEngine didn't reply within 10000ms").
  107. +------------------------------------
  108. INFO 09-25 21:01:41 metrics.py:356] pp: 2.6 t/s, tg: 0.0 t/s, Active: 1, Swapped: 0, Queued: 0, GPU KV: 3.4%, CPU KV: 0.0%.
  109. INFO 09-25 21:01:52 metrics.py:356] pp: 0.0 t/s, tg: 2.0 t/s, Active: 0, Swapped: 0, Queued: 0, GPU KV: 0.0%, CPU KV: 0.0%.
  110.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!