API tools faq
paste
Lissanro

vLLM docker log

Lissanro
Sep 25th, 2024
90
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 39.13 KB | None | 0 0
raw download clone embed print report
  1. > sudo docker run --rm --shm-size=12gb --runtime nvidia --gpus all -e HF_HUB_OFFLINE=0 -e LOCAL_LOGGING_INTERVAL_SEC=2 -e NO_LOG_ON_IDLE=1 -p 18888:18888 cduk/vllm:latest --model Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4 --host 0.0.0.0 --port 18888 --served-model-name Qwen2-VL-72B-Instruct-GPTQ-Int4 --gpu-memory-utilization 0.98 -tp 4 --disable-custom-all-reduce --swap-space 4 --max-num-seqs 24 --dtype half
  2. logging interval: 2
  3. INFO 09-25 15:44:56 api_server.py:526] vLLM API server version 0.6.1.post2
  4. INFO 09-25 15:44:56 api_server.py:527] args: Namespace(host='0.0.0.0', port=18888, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, model='Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, download_dir=None, load_format='auto', config_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=None, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=False, disable_sliding_window=False, use_v2_block_manager=False, num_lookahead_slots=0, seed=0, swap_space=4.0, cpu_offload_gb=0, gpu_memory_utilization=0.98, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=24, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=False, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=True, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['Qwen2-VL-72B-Instruct-GPTQ-Int4'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False)
  5. INFO 09-25 15:44:56 api_server.py:164] Multiprocessing frontend to use ipc:///tmp/39e45690-cea7-4b35-aa4c-90785a3cc1fe for IPC Path.
  6. INFO 09-25 15:44:56 api_server.py:177] Started engine process with PID 44
  7. INFO 09-25 15:45:00 gptq_marlin.py:108] The model is convertible to gptq_marlin during runtime. Using gptq_marlin kernel.
  8. INFO 09-25 15:45:00 config.py:876] Defaulting to use mp for distributed inference
  9. logging interval: 2
  10. INFO 09-25 15:45:02 gptq_marlin.py:108] The model is convertible to gptq_marlin during runtime. Using gptq_marlin kernel.
  11. INFO 09-25 15:45:02 config.py:876] Defaulting to use mp for distributed inference
  12. INFO 09-25 15:45:02 llm_engine.py:227] Initializing an LLM engine (v0.6.1.post2) with config: model='Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4', speculative_config=None, tokenizer='Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=4, pipeline_parallel_size=1, disable_custom_all_reduce=True, quantization=gptq_marlin, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=Qwen2-VL-72B-Instruct-GPTQ-Int4, use_v2_block_manager=False, num_scheduler_steps=1, enable_prefix_caching=False, use_async_output_proc=True)
  13. WARNING 09-25 15:45:11 multiproc_gpu_executor.py:53] Reducing Torch parallelism from 16 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
  14. INFO 09-25 15:45:11 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
  15. (VllmWorkerProcess pid=116) INFO 09-25 15:45:12 multiproc_worker_utils.py:218] Worker ready; awaiting tasks
  16. (VllmWorkerProcess pid=117) INFO 09-25 15:45:12 multiproc_worker_utils.py:218] Worker ready; awaiting tasks
  17. (VllmWorkerProcess pid=118) INFO 09-25 15:45:12 multiproc_worker_utils.py:218] Worker ready; awaiting tasks
  18. (VllmWorkerProcess pid=116) INFO 09-25 15:45:12 utils.py:982] Found nccl from library libnccl.so.2
  19. INFO 09-25 15:45:12 utils.py:982] Found nccl from library libnccl.so.2
  20. INFO 09-25 15:45:12 pynccl.py:63] vLLM is using nccl==2.20.5
  21. (VllmWorkerProcess pid=116) INFO 09-25 15:45:12 pynccl.py:63] vLLM is using nccl==2.20.5
  22. (VllmWorkerProcess pid=117) INFO 09-25 15:45:12 utils.py:982] Found nccl from library libnccl.so.2
  23. (VllmWorkerProcess pid=118) INFO 09-25 15:45:12 utils.py:982] Found nccl from library libnccl.so.2
  24. (VllmWorkerProcess pid=117) INFO 09-25 15:45:12 pynccl.py:63] vLLM is using nccl==2.20.5
  25. (VllmWorkerProcess pid=118) INFO 09-25 15:45:12 pynccl.py:63] vLLM is using nccl==2.20.5
  26. INFO 09-25 15:45:13 shm_broadcast.py:238] vLLM message queue communication handle: Handle(connect_ip='127.0.0.1', local_reader_ranks=[1, 2, 3], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x749948e618b0>, local_subscribe_port=37777, remote_subscribe_port=None)
  27. (VllmWorkerProcess pid=117) INFO 09-25 15:45:13 model_runner.py:1014] Starting to load model Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4...
  28. INFO 09-25 15:45:13 model_runner.py:1014] Starting to load model Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4...
  29. (VllmWorkerProcess pid=116) INFO 09-25 15:45:13 model_runner.py:1014] Starting to load model Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4...
  30. (VllmWorkerProcess pid=118) INFO 09-25 15:45:13 model_runner.py:1014] Starting to load model Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4...
  31. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  32. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  33. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  34. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  35. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  36. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  37. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  38. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  39. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  40. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  41. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  42. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  43. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  44. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  45. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  46. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  47. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  48. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  49. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  50. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  51. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  52. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  53. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  54. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  55. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  56. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  57. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  58. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  59. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  60. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  61. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  62. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  63. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  64. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  65. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  66. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  67. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  68. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  69. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  70. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  71. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  72. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  73. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  74. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  75. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  76. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  77. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  78. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  79. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  80. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  81. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  82. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  83. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  84. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  85. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  86. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  87. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  88. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  89. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  90. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  91. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  92. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  93. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  94. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  95. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  96. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  97. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  98. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  99. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  100. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  101. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  102. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  103. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  104. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  105. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  106. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  107. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  108. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  109. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  110. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  111. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  112. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  113. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  114. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  115. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  116. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  117. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  118. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  119. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  120. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  121. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  122. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  123. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  124. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  125. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  126. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  127. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  128. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  129. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  130. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  131. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  132. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  133. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  134. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  135. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  136. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  137. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  138. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  139. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  140. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  141. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  142. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  143. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  144. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  145. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  146. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  147. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  148. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  149. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  150. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  151. (VllmWorkerProcess pid=118) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  152. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  153. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  154. (VllmWorkerProcess pid=117) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  155. (VllmWorkerProcess pid=116) WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  156. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  157. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  158. WARNING 09-25 15:45:13 qwen2_vl.py:215] Current Qwen2-VL implementation has a bug with `vllm-flash-attn` inside vision module, so we use xformers backend instead. You can run `pip install flash-attn to use flash-attention backend.
  159. INFO 09-25 15:45:14 weight_utils.py:242] Using model weights format ['*.safetensors']
  160. (VllmWorkerProcess pid=117) INFO 09-25 15:45:15 weight_utils.py:242] Using model weights format ['*.safetensors']
  161. (VllmWorkerProcess pid=116) INFO 09-25 15:45:15 weight_utils.py:242] Using model weights format ['*.safetensors']
  162. (VllmWorkerProcess pid=118) INFO 09-25 15:45:15 weight_utils.py:242] Using model weights format ['*.safetensors']
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!