API tools faq
paste
Guest User

HSA_OVERRIDE_GFX_VERSION="10.1.0" ollama serve

a guest
Nov 4th, 2024
142
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 14.95 KB | None | 0 0
raw download clone embed print report
  1. argo@artix:~$ HSA_OVERRIDE_GFX_VERSION="10.1.0" ollama serve
  2. 2024/11/04 15:25:04 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION:10.1.0 HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/argo/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
  3. time=2024-11-04T15:25:04.071+05:30 level=INFO source=images.go:753 msg="total blobs: 10"
  4. time=2024-11-04T15:25:04.071+05:30 level=INFO source=images.go:760 msg="total unused blobs removed: 0"
  5. time=2024-11-04T15:25:04.071+05:30 level=INFO source=routes.go:1200 msg="Listening on 127.0.0.1:11434 (version 0.3.12)"
  6. time=2024-11-04T15:25:04.072+05:30 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama3419702560/runners
  7. time=2024-11-04T15:25:08.223+05:30 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu rocm]"
  8. time=2024-11-04T15:25:08.224+05:30 level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
  9. time=2024-11-04T15:25:08.224+05:30 level=WARN source=gpu.go:669 msg="unable to locate gpu dependency libraries"
  10. time=2024-11-04T15:25:08.225+05:30 level=WARN source=gpu.go:669 msg="unable to locate gpu dependency libraries"
  11. time=2024-11-04T15:25:08.225+05:30 level=WARN source=gpu.go:669 msg="unable to locate gpu dependency libraries"
  12. time=2024-11-04T15:25:08.252+05:30 level=WARN source=gpu.go:669 msg="unable to locate gpu dependency libraries"
  13. time=2024-11-04T15:25:08.269+05:30 level=WARN source=amd_linux.go:60 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
  14. time=2024-11-04T15:25:08.270+05:30 level=INFO source=amd_linux.go:349 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.1.0
  15. time=2024-11-04T15:25:08.271+05:30 level=INFO source=amd_linux.go:275 msg="unsupported Radeon iGPU detected skipping" id=1 total="512.0 MiB"
  16. time=2024-11-04T15:25:08.271+05:30 level=INFO source=types.go:107 msg="inference compute" id=0 library=rocm variant="" compute=gfx1010 driver=0.0 name=1002:731f total="6.0 GiB" available="6.0 GiB"
  17. [GIN] 2024/11/04 - 15:25:11 | 200 |      81.086µs |       127.0.0.1 | HEAD     "/"
  18. [GIN] 2024/11/04 - 15:25:11 | 200 |    5.076413ms |       127.0.0.1 | POST     "/api/show"
  19. time=2024-11-04T15:25:11.072+05:30 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb gpu=0 parallel=1 available=6413991936 required="3.4 GiB"
  20. time=2024-11-04T15:25:11.072+05:30 level=INFO source=server.go:103 msg="system memory" total="7.1 GiB" free="3.5 GiB" free_swap="8.2 GiB"
  21. time=2024-11-04T15:25:11.072+05:30 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[6.0 GiB]" memory.gpu_overhead="0 B" memory.required.full="3.4 GiB" memory.required.partial="3.4 GiB" memory.required.kv="768.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="2.6 GiB" memory.weights.repeating="2.6 GiB" memory.weights.nonrepeating="77.1 MiB" memory.graph.full="128.0 MiB" memory.graph.partial="128.0 MiB"
  22. time=2024-11-04T15:25:11.075+05:30 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama3419702560/runners/rocm/ollama_llama_server --model /home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --parallel 1 --port 35881"
  23. time=2024-11-04T15:25:11.075+05:30 level=INFO source=sched.go:449 msg="loaded runners" count=1
  24. time=2024-11-04T15:25:11.075+05:30 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  25. time=2024-11-04T15:25:11.075+05:30 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  26. INFO [main] build info | build=3670 commit="e6f424eac" tid="140321978102272" timestamp=1730714111
  27. INFO [main] system info | n_threads=6 n_threads_batch=6 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140321978102272" timestamp=1730714111 total_threads=12
  28. INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="11" port="35881" tid="140321978102272" timestamp=1730714111
  29. llama_model_loader: loaded meta data with 36 key-value pairs and 197 tensors from /home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb (version GGUF V3 (latest))
  30. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  31. llama_model_loader: - kv   0:                       general.architecture str              = phi3
  32. llama_model_loader: - kv   1:                               general.type str              = model
  33. llama_model_loader: - kv   2:                               general.name str              = Phi 3.5 Mini Instruct
  34. llama_model_loader: - kv   3:                           general.finetune str              = instruct
  35. llama_model_loader: - kv   4:                           general.basename str              = Phi-3.5
  36. llama_model_loader: - kv   5:                         general.size_label str              = mini
  37. llama_model_loader: - kv   6:                            general.license str              = mit
  38. llama_model_loader: - kv   7:                       general.license.link str              = https://huggingface.co/microsoft/Phi-...
  39. llama_model_loader: - kv   8:                               general.tags arr[str,3]       = ["nlp", "code", "text-generation"]
  40. llama_model_loader: - kv   9:                          general.languages arr[str,1]       = ["multilingual"]
  41. llama_model_loader: - kv  10:                        phi3.context_length u32              = 131072
  42. llama_model_loader: - kv  11:  phi3.rope.scaling.original_context_length u32              = 4096
  43. llama_model_loader: - kv  12:                      phi3.embedding_length u32              = 3072
  44. llama_model_loader: - kv  13:                   phi3.feed_forward_length u32              = 8192
  45. llama_model_loader: - kv  14:                           phi3.block_count u32              = 32
  46. llama_model_loader: - kv  15:                  phi3.attention.head_count u32              = 32
  47. llama_model_loader: - kv  16:               phi3.attention.head_count_kv u32              = 32
  48. llama_model_loader: - kv  17:      phi3.attention.layer_norm_rms_epsilon f32              = 0.000010
  49. llama_model_loader: - kv  18:                  phi3.rope.dimension_count u32              = 96
  50. llama_model_loader: - kv  19:                        phi3.rope.freq_base f32              = 10000.000000
  51. llama_model_loader: - kv  20:                          general.file_type u32              = 2
  52. llama_model_loader: - kv  21:              phi3.attention.sliding_window u32              = 262144
  53. llama_model_loader: - kv  22:              phi3.rope.scaling.attn_factor f32              = 1.190238
  54. llama_model_loader: - kv  23:                       tokenizer.ggml.model str              = llama
  55. llama_model_loader: - kv  24:                         tokenizer.ggml.pre str              = default
  56. llama_model_loader: - kv  25:                      tokenizer.ggml.tokens arr[str,32064]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
  57. llama_model_loader: - kv  26:                      tokenizer.ggml.scores arr[f32,32064]   = [-1000.000000, -1000.000000, -1000.00...
  58. llama_model_loader: - kv  27:                  tokenizer.ggml.token_type arr[i32,32064]   = [3, 3, 4, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
  59. llama_model_loader: - kv  28:                tokenizer.ggml.bos_token_id u32              = 1
  60. llama_model_loader: - kv  29:                tokenizer.ggml.eos_token_id u32              = 32000
  61. llama_model_loader: - kv  30:            tokenizer.ggml.unknown_token_id u32              = 0
  62. llama_model_loader: - kv  31:            tokenizer.ggml.padding_token_id u32              = 32000
  63. llama_model_loader: - kv  32:               tokenizer.ggml.add_bos_token bool             = false
  64. llama_model_loader: - kv  33:               tokenizer.ggml.add_eos_token bool             = false
  65. llama_model_loader: - kv  34:                    tokenizer.chat_template str              = {% for message in messages %}{% if me...
  66. llama_model_loader: - kv  35:               general.quantization_version u32              = 2
  67. llama_model_loader: - type  f32:   67 tensors
  68. llama_model_loader: - type q4_0:  129 tensors
  69. llama_model_loader: - type q6_K:    1 tensors
  70. llm_load_vocab: special tokens cache size = 14
  71. llm_load_vocab: token to piece cache size = 0.1685 MB
  72. llm_load_print_meta: format           = GGUF V3 (latest)
  73. llm_load_print_meta: arch             = phi3
  74. llm_load_print_meta: vocab type       = SPM
  75. llm_load_print_meta: n_vocab          = 32064
  76. llm_load_print_meta: n_merges         = 0
  77. llm_load_print_meta: vocab_only       = 0
  78. llm_load_print_meta: n_ctx_train      = 131072
  79. llm_load_print_meta: n_embd           = 3072
  80. llm_load_print_meta: n_layer          = 32
  81. llm_load_print_meta: n_head           = 32
  82. llm_load_print_meta: n_head_kv        = 32
  83. llm_load_print_meta: n_rot            = 96
  84. llm_load_print_meta: n_swa            = 262144
  85. llm_load_print_meta: n_embd_head_k    = 96
  86. llm_load_print_meta: n_embd_head_v    = 96
  87. llm_load_print_meta: n_gqa            = 1
  88. llm_load_print_meta: n_embd_k_gqa     = 3072
  89. llm_load_print_meta: n_embd_v_gqa     = 3072
  90. llm_load_print_meta: f_norm_eps       = 0.0e+00
  91. llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
  92. llm_load_print_meta: f_clamp_kqv      = 0.0e+00
  93. llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  94. llm_load_print_meta: f_logit_scale    = 0.0e+00
  95. llm_load_print_meta: n_ff             = 8192
  96. llm_load_print_meta: n_expert         = 0
  97. llm_load_print_meta: n_expert_used    = 0
  98. llm_load_print_meta: causal attn      = 1
  99. llm_load_print_meta: pooling type     = 0
  100. llm_load_print_meta: rope type        = 2
  101. llm_load_print_meta: rope scaling     = linear
  102. llm_load_print_meta: freq_base_train  = 10000.0
  103. llm_load_print_meta: freq_scale_train = 1
  104. llm_load_print_meta: n_ctx_orig_yarn  = 4096
  105. llm_load_print_meta: rope_finetuned   = unknown
  106. llm_load_print_meta: ssm_d_conv       = 0
  107. llm_load_print_meta: ssm_d_inner      = 0
  108. llm_load_print_meta: ssm_d_state      = 0
  109. llm_load_print_meta: ssm_dt_rank      = 0
  110. llm_load_print_meta: ssm_dt_b_c_rms   = 0
  111. llm_load_print_meta: model type       = 3B
  112. llm_load_print_meta: model ftype      = Q4_0
  113. llm_load_print_meta: model params     = 3.82 B
  114. llm_load_print_meta: model size       = 2.03 GiB (4.55 BPW)
  115. llm_load_print_meta: general.name     = Phi 3.5 Mini Instruct
  116. llm_load_print_meta: BOS token        = 1 '<s>'
  117. llm_load_print_meta: EOS token        = 32000 '<|endoftext|>'
  118. llm_load_print_meta: UNK token        = 0 '<unk>'
  119. llm_load_print_meta: PAD token        = 32000 '<|endoftext|>'
  120. llm_load_print_meta: LF token         = 13 '<0x0A>'
  121. llm_load_print_meta: EOT token        = 32007 '<|end|>'
  122. llm_load_print_meta: max token length = 48
  123. time=2024-11-04T15:25:11.327+05:30 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  124. ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
  125. ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  126. ggml_cuda_init: found 1 ROCm devices:
  127.  Device 0: AMD Radeon RX 5600M, compute capability 10.1, VMM: no
  128. llm_load_tensors: ggml ctx size =    0.21 MiB
  129. llm_load_tensors: offloading 32 repeating layers to GPU
  130. llm_load_tensors: offloading non-repeating layers to GPU
  131. llm_load_tensors: offloaded 33/33 layers to GPU
  132. llm_load_tensors:      ROCm0 buffer size =  2021.84 MiB
  133. llm_load_tensors:        CPU buffer size =    52.84 MiB
  134. llama_new_context_with_model: n_ctx      = 2048
  135. llama_new_context_with_model: n_batch    = 512
  136. llama_new_context_with_model: n_ubatch   = 512
  137. llama_new_context_with_model: flash_attn = 0
  138. llama_new_context_with_model: freq_base  = 10000.0
  139. llama_new_context_with_model: freq_scale = 1
  140. llama_kv_cache_init:      ROCm0 KV buffer size =   768.00 MiB
  141. llama_new_context_with_model: KV self size  =  768.00 MiB, K (f16):  384.00 MiB, V (f16):  384.00 MiB
  142. llama_new_context_with_model:  ROCm_Host  output buffer size =     0.13 MiB
  143. [1730714114] warming up the model with an empty run
  144. llama_new_context_with_model:      ROCm0 compute buffer size =   168.00 MiB
  145. llama_new_context_with_model:  ROCm_Host compute buffer size =    10.01 MiB
  146. llama_new_context_with_model: graph nodes  = 1286
  147. llama_new_context_with_model: graph splits = 2
  148. ggml_cuda_compute_forward: RMS_NORM failed
  149. CUDA error: shared object initialization failed
  150.  current device: 0, in function ggml_cuda_compute_forward at /build/ollama-rocm/src/ollama/llm/llama.cpp/ggml/src/ggml-cuda.cu:2326
  151.  err
  152. /build/ollama-rocm/src/ollama/llm/llama.cpp/ggml/src/ggml-cuda.cu:102: CUDA error
  153. time=2024-11-04T15:25:14.148+05:30 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  154. time=2024-11-04T15:25:14.399+05:30 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: CUDA error: shared object initialization failed\n  current device: 0, in function ggml_cuda_compute_forward at /build/ollama-rocm/src/ollama/llm/llama.cpp/ggml/src/ggml-cuda.cu:2326\n  err\n/build/ollama-rocm/src/ollama/llm/llama.cpp/ggml/src/ggml-cuda.cu:102: CUDA error"
  155. [GIN] 2024/11/04 - 15:25:14 | 500 |  3.341467072s |       127.0.0.1 | POST     "/api/generate"
  156. time=2024-11-04T15:25:19.400+05:30 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.000847523 model=/home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb
  157. time=2024-11-04T15:25:19.649+05:30 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.250528628 model=/home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb
  158. time=2024-11-04T15:25:19.900+05:30 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.501129962 model=/home/argo/.ollama/models/blobs/sha256-b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!