API tools faq
paste
Guest User

Untitled

a guest
Nov 7th, 2024
165
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 150.58 KB | Software | 0 0
raw download clone embed print report
  1. Nov 07 13:27:27 slimb ollama[1148]: 2024/11/07 13:27:27 routes.go:1158: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION:9.0.0 HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:24h0m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
  2. Nov 07 13:27:27 slimb ollama[1148]: time=2024-11-07T13:27:27.853+01:00 level=INFO source=images.go:754 msg="total blobs: 14"
  3. Nov 07 13:27:27 slimb ollama[1148]: time=2024-11-07T13:27:27.854+01:00 level=INFO source=images.go:761 msg="total unused blobs removed: 0"
  4. Nov 07 13:27:27 slimb ollama[1148]: time=2024-11-07T13:27:27.855+01:00 level=INFO source=routes.go:1205 msg="Listening on [::]:11434 (version 0.3.14)"
  5. Nov 07 13:27:27 slimb ollama[1148]: time=2024-11-07T13:27:27.856+01:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama680209959/runners
  6. Nov 07 13:27:34 slimb ollama[1148]: time=2024-11-07T13:27:34.712+01:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[rocm_v60102 cpu cpu_avx cpu_avx2 cuda_v11 cuda_v12]"
  7. Nov 07 13:27:34 slimb ollama[1148]: time=2024-11-07T13:27:34.713+01:00 level=INFO source=gpu.go:221 msg="looking for compatible GPUs"
  8. Nov 07 13:27:34 slimb ollama[1148]: time=2024-11-07T13:27:34.722+01:00 level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
  9. Nov 07 13:27:34 slimb ollama[1148]: time=2024-11-07T13:27:34.723+01:00 level=INFO source=amd_linux.go:386 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=9.0.0
  10. Nov 07 13:27:34 slimb ollama[1148]: time=2024-11-07T13:27:34.723+01:00 level=INFO source=types.go:123 msg="inference compute" id=0 library=rocm variant="" compute=gfx90c driver=0.0 name=1002:1638 total="16.0 GiB" available="16.0 GiB"
  11. Nov 07 13:30:21 slimb ollama[1148]: [GIN] 2024/11/07 - 13:30:21 | 200 | 336.535µs | 127.0.0.1 | HEAD "/"
  12. Nov 07 13:30:21 slimb ollama[1148]: [GIN] 2024/11/07 - 13:30:21 | 200 | 8.475492ms | 127.0.0.1 | POST "/api/show"
  13. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.173+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c gpu=0 parallel=4 available=17142460416 required="9.0 GiB"
  14. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.173+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="11.9 GiB" free_swap="8.0 GiB"
  15. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.173+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[16.0 GiB]" memory.gpu_overhead="0 B" memory.required.full="9.0 GiB" memory.required.partial="9.0 GiB" memory.required.kv="1.2 GiB" memory.required.allocations="[9.0 GiB]" memory.weights.total="7.3 GiB" memory.weights.repeating="7.2 GiB" memory.weights.nonrepeating="157.5 MiB" memory.graph.full="853.3 MiB" memory.graph.partial="853.3 MiB"
  16. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.175+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama680209959/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 41 --threads 8 --parallel 4 --port 38101"
  17. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.175+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=1
  18. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.175+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  19. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.175+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  20. Nov 07 13:30:21 slimb ollama[1750]: INFO [main] starting c++ runner | tid="140186179515200" timestamp=1730982621
  21. Nov 07 13:30:21 slimb ollama[1750]: INFO [main] build info | build=10 commit="8f7ec4a" tid="140186179515200" timestamp=1730982621
  22. Nov 07 13:30:21 slimb ollama[1750]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140186179515200" timestamp=1730982621 total_threads=16
  23. Nov 07 13:30:21 slimb ollama[1750]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="38101" tid="140186179515200" timestamp=1730982621
  24. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: loaded meta data with 35 key-value pairs and 362 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c (version GGUF V3 (latest))
  25. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  26. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 0: general.architecture str = granite
  27. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 1: general.type str = model
  28. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 2: general.name str = Granite 3.0 8b Instruct
  29. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 3: general.finetune str = instruct
  30. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 4: general.basename str = granite-3.0
  31. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 5: general.size_label str = 8B
  32. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 6: general.license str = apache-2.0
  33. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 7: general.tags arr[str,3] = ["language", "granite-3.0", "text-gen...
  34. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 8: granite.block_count u32 = 40
  35. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 9: granite.context_length u32 = 4096
  36. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 10: granite.embedding_length u32 = 4096
  37. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 11: granite.feed_forward_length u32 = 12800
  38. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 12: granite.attention.head_count u32 = 32
  39. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 13: granite.attention.head_count_kv u32 = 8
  40. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 14: granite.rope.freq_base f32 = 10000.000000
  41. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 15: granite.attention.layer_norm_rms_epsilon f32 = 0.000010
  42. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 16: general.file_type u32 = 18
  43. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 17: granite.vocab_size u32 = 49155
  44. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 18: granite.rope.dimension_count u32 = 128
  45. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 19: tokenizer.ggml.add_space_prefix bool = false
  46. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 20: granite.attention.scale f32 = 0.007812
  47. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 21: granite.embedding_scale f32 = 12.000000
  48. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 22: granite.residual_scale f32 = 0.220000
  49. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 23: granite.logit_scale f32 = 16.000000
  50. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 24: tokenizer.ggml.model str = gpt2
  51. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 25: tokenizer.ggml.pre str = refact
  52. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 26: tokenizer.ggml.tokens arr[str,49155] = ["<|end_of_text|>", "<fim_prefix>", "...
  53. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 27: tokenizer.ggml.token_type arr[i32,49155] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
  54. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 28: tokenizer.ggml.merges arr[str,48891] = ["Ġ Ġ", "ĠĠ ĠĠ", "ĠĠĠĠ ĠĠ...
  55. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 29: tokenizer.ggml.bos_token_id u32 = 0
  56. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 30: tokenizer.ggml.eos_token_id u32 = 0
  57. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 0
  58. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
  59. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 33: tokenizer.chat_template str = {%- if tools %}\n {{- '<|start_of_r...
  60. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - kv 34: general.quantization_version u32 = 2
  61. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - type f32: 81 tensors
  62. Nov 07 13:30:21 slimb ollama[1148]: llama_model_loader: - type q6_K: 281 tensors
  63. Nov 07 13:30:21 slimb ollama[1148]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
  64. Nov 07 13:30:21 slimb ollama[1148]: llm_load_vocab: special tokens cache size = 22
  65. Nov 07 13:30:21 slimb ollama[1148]: llm_load_vocab: token to piece cache size = 0.2826 MB
  66. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: format = GGUF V3 (latest)
  67. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: arch = granite
  68. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: vocab type = BPE
  69. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_vocab = 49155
  70. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_merges = 48891
  71. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: vocab_only = 0
  72. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_ctx_train = 4096
  73. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_embd = 4096
  74. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_layer = 40
  75. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_head = 32
  76. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_head_kv = 8
  77. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_rot = 128
  78. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_swa = 0
  79. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_embd_head_k = 128
  80. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_embd_head_v = 128
  81. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_gqa = 4
  82. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_embd_k_gqa = 1024
  83. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_embd_v_gqa = 1024
  84. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_norm_eps = 0.0e+00
  85. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  86. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  87. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  88. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_logit_scale = 1.6e+01
  89. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_ff = 12800
  90. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_expert = 0
  91. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_expert_used = 0
  92. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: causal attn = 1
  93. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: pooling type = 0
  94. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: rope type = 0
  95. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: rope scaling = linear
  96. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: freq_base_train = 10000.0
  97. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: freq_scale_train = 1
  98. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: n_ctx_orig_yarn = 4096
  99. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: rope_finetuned = unknown
  100. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: ssm_d_conv = 0
  101. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: ssm_d_inner = 0
  102. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: ssm_d_state = 0
  103. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: ssm_dt_rank = 0
  104. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  105. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: model type = 3B
  106. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: model ftype = Q6_K
  107. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: model params = 8.17 B
  108. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: model size = 6.24 GiB (6.56 BPW)
  109. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: general.name = Granite 3.0 8b Instruct
  110. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: BOS token = 0 '<|end_of_text|>'
  111. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: EOS token = 0 '<|end_of_text|>'
  112. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: PAD token = 0 '<|end_of_text|>'
  113. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: LF token = 145 'Ä'
  114. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: EOG token = 0 '<|end_of_text|>'
  115. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: max token length = 512
  116. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_embedding_scale = 12.000000
  117. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_residual_scale = 0.220000
  118. Nov 07 13:30:21 slimb ollama[1148]: llm_load_print_meta: f_attention_scale = 0.007812
  119. Nov 07 13:30:21 slimb ollama[1148]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  120. Nov 07 13:30:21 slimb ollama[1148]: time=2024-11-07T13:30:21.677+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  121. Nov 07 13:30:22 slimb ollama[1148]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  122. Nov 07 13:30:22 slimb ollama[1148]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  123. Nov 07 13:30:22 slimb ollama[1148]: ggml_cuda_init: found 1 ROCm devices:
  124. Nov 07 13:30:22 slimb ollama[1148]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  125. Nov 07 13:30:22 slimb ollama[1148]: llm_load_tensors: ggml ctx size = 0.34 MiB
  126. Nov 07 13:30:31 slimb ollama[1148]: llm_load_tensors: offloading 40 repeating layers to GPU
  127. Nov 07 13:30:31 slimb ollama[1148]: llm_load_tensors: offloading non-repeating layers to GPU
  128. Nov 07 13:30:31 slimb ollama[1148]: llm_load_tensors: offloaded 41/41 layers to GPU
  129. Nov 07 13:30:31 slimb ollama[1148]: llm_load_tensors: ROCm0 buffer size = 6393.15 MiB
  130. Nov 07 13:30:31 slimb ollama[1148]: llm_load_tensors: CPU buffer size = 157.51 MiB
  131. Nov 07 13:30:51 slimb ollama[1148]: HW Exception by GPU node-1 (Agent handle: 0x1b0274f0) reason :GPU Hang
  132. Nov 07 13:30:51 slimb ollama[1148]: time=2024-11-07T13:30:51.469+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  133. Nov 07 13:30:53 slimb ollama[1148]: time=2024-11-07T13:30:53.080+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped)"
  134. Nov 07 13:30:53 slimb ollama[1148]: [GIN] 2024/11/07 - 13:30:53 | 500 | 31.925548353s | 127.0.0.1 | POST "/api/generate"
  135. Nov 07 13:30:58 slimb ollama[1148]: time=2024-11-07T13:30:58.083+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.003363853 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  136. Nov 07 13:30:58 slimb ollama[1148]: time=2024-11-07T13:30:58.334+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.254190553 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  137. Nov 07 13:30:58 slimb ollama[1148]: time=2024-11-07T13:30:58.584+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.504082729 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  138. Nov 07 13:31:51 slimb systemd[1]: Stopping ollama.service - Ollama Service...
  139. Nov 07 13:31:51 slimb systemd[1]: ollama.service: Deactivated successfully.
  140. Nov 07 13:31:51 slimb systemd[1]: Stopped ollama.service - Ollama Service.
  141. Nov 07 13:31:51 slimb systemd[1]: ollama.service: Consumed 40.463s CPU time, 10.3G memory peak, 1G memory swap peak.
  142. Nov 07 13:31:51 slimb systemd[1]: Started ollama.service - Ollama Service.
  143. Nov 07 13:31:51 slimb ollama[1817]: 2024/11/07 13:31:51 routes.go:1158: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION:9.0.0 HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:24h0m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
  144. Nov 07 13:31:51 slimb ollama[1817]: time=2024-11-07T13:31:51.745+01:00 level=INFO source=images.go:754 msg="total blobs: 14"
  145. Nov 07 13:31:51 slimb ollama[1817]: time=2024-11-07T13:31:51.746+01:00 level=INFO source=images.go:761 msg="total unused blobs removed: 0"
  146. Nov 07 13:31:51 slimb ollama[1817]: time=2024-11-07T13:31:51.747+01:00 level=INFO source=routes.go:1205 msg="Listening on [::]:11434 (version 0.3.14)"
  147. Nov 07 13:31:51 slimb ollama[1817]: time=2024-11-07T13:31:51.747+01:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama414509819/runners
  148. Nov 07 13:31:59 slimb ollama[1817]: time=2024-11-07T13:31:59.159+01:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx2 cuda_v11 cuda_v12 rocm_v60102 cpu cpu_avx]"
  149. Nov 07 13:31:59 slimb ollama[1817]: time=2024-11-07T13:31:59.160+01:00 level=INFO source=gpu.go:221 msg="looking for compatible GPUs"
  150. Nov 07 13:31:59 slimb ollama[1817]: time=2024-11-07T13:31:59.171+01:00 level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
  151. Nov 07 13:31:59 slimb ollama[1817]: time=2024-11-07T13:31:59.172+01:00 level=INFO source=amd_linux.go:386 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=9.0.0
  152. Nov 07 13:31:59 slimb ollama[1817]: time=2024-11-07T13:31:59.172+01:00 level=INFO source=types.go:123 msg="inference compute" id=0 library=rocm variant="" compute=gfx90c driver=0.0 name=1002:1638 total="16.0 GiB" available="16.0 GiB"
  153. Nov 07 13:32:56 slimb ollama[1817]: [GIN] 2024/11/07 - 13:32:56 | 200 | 41.733µs | 127.0.0.1 | HEAD "/"
  154. Nov 07 13:32:56 slimb ollama[1817]: [GIN] 2024/11/07 - 13:32:56 | 200 | 893.465µs | 127.0.0.1 | GET "/api/ps"
  155. Nov 07 13:33:03 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:03 | 200 | 20.568µs | 127.0.0.1 | HEAD "/"
  156. Nov 07 13:33:03 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:03 | 200 | 1.131365ms | 127.0.0.1 | GET "/api/tags"
  157. Nov 07 13:33:12 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:12 | 200 | 23.615µs | 127.0.0.1 | HEAD "/"
  158. Nov 07 13:33:12 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:12 | 200 | 9.604102ms | 127.0.0.1 | POST "/api/show"
  159. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.651+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-88448bf3bb7ecbac9e536d2c1fd17f8ccb02a9999cbd404dd7b4289255741014 gpu=0 parallel=4 available=17142460416 required="3.9 GiB"
  160. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.651+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="12.0 GiB" free_swap="7.8 GiB"
  161. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.651+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=25 layers.offload=25 layers.split="" memory.available="[16.0 GiB]" memory.gpu_overhead="0 B" memory.required.full="3.9 GiB" memory.required.partial="3.9 GiB" memory.required.kv="1.5 GiB" memory.required.allocations="[3.9 GiB]" memory.weights.total="2.7 GiB" memory.weights.repeating="2.7 GiB" memory.weights.nonrepeating="78.8 MiB" memory.graph.full="544.0 MiB" memory.graph.partial="602.3 MiB"
  162. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.653+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-88448bf3bb7ecbac9e536d2c1fd17f8ccb02a9999cbd404dd7b4289255741014 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 25 --threads 8 --parallel 4 --port 33739"
  163. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.654+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=1
  164. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.654+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  165. Nov 07 13:33:12 slimb ollama[1817]: time=2024-11-07T13:33:12.654+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  166. Nov 07 13:33:12 slimb ollama[1894]: INFO [main] starting c++ runner | tid="139635018588992" timestamp=1730982792
  167. Nov 07 13:33:12 slimb ollama[1894]: INFO [main] build info | build=10 commit="8f7ec4a" tid="139635018588992" timestamp=1730982792
  168. Nov 07 13:33:12 slimb ollama[1894]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="139635018588992" timestamp=1730982792 total_threads=16
  169. Nov 07 13:33:12 slimb ollama[1894]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="33739" tid="139635018588992" timestamp=1730982792
  170. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: loaded meta data with 34 key-value pairs and 218 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-88448bf3bb7ecbac9e536d2c1fd17f8ccb02a9999cbd404dd7b4289255741014 (version GGUF V3 (latest))
  171. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  172. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  173. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  174. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Smollm2 1.7B 8k Mix7 Ep2 v2
  175. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 3: general.version str = v2
  176. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 4: general.organization str = Loubnabnl
  177. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 5: general.finetune str = 8k-mix7-ep2
  178. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 6: general.basename str = smollm2
  179. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 7: general.size_label str = 1.7B
  180. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 8: general.license str = apache-2.0
  181. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 9: general.languages arr[str,1] = ["en"]
  182. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 10: llama.block_count u32 = 24
  183. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 11: llama.context_length u32 = 8192
  184. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 12: llama.embedding_length u32 = 2048
  185. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 13: llama.feed_forward_length u32 = 8192
  186. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 14: llama.attention.head_count u32 = 32
  187. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.head_count_kv u32 = 32
  188. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 16: llama.rope.freq_base f32 = 130000.000000
  189. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  190. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 18
  191. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 49152
  192. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 64
  193. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.add_space_prefix bool = false
  194. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.add_bos_token bool = false
  195. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
  196. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.pre str = smollm
  197. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,49152] = ["<|endoftext|>", "<|im_start|>", "<|...
  198. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,49152] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
  199. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,48900] = ["Ġ t", "Ġ a", "i n", "h e", "Ġ Ġ...
  200. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 1
  201. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 2
  202. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 0
  203. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
  204. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 32: tokenizer.chat_template str = {% for message in messages %}{% if lo...
  205. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - kv 33: general.quantization_version u32 = 2
  206. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - type f32: 49 tensors
  207. Nov 07 13:33:12 slimb ollama[1817]: llama_model_loader: - type q6_K: 169 tensors
  208. Nov 07 13:33:12 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 17
  209. Nov 07 13:33:13 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.3170 MB
  210. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  211. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: arch = llama
  212. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  213. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_vocab = 49152
  214. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_merges = 48900
  215. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  216. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 8192
  217. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_embd = 2048
  218. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_layer = 24
  219. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_head = 32
  220. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 32
  221. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_rot = 64
  222. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  223. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 64
  224. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 64
  225. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_gqa = 1
  226. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 2048
  227. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 2048
  228. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  229. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  230. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  231. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  232. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  233. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  234. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  235. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  236. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  237. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  238. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  239. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  240. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 130000.0
  241. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  242. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 8192
  243. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  244. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  245. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  246. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  247. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  248. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  249. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: model type = ?B
  250. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  251. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: model params = 1.71 B
  252. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: model size = 1.31 GiB (6.56 BPW)
  253. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: general.name = Smollm2 1.7B 8k Mix7 Ep2 v2
  254. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: BOS token = 1 '<|im_start|>'
  255. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: EOS token = 2 '<|im_end|>'
  256. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: UNK token = 0 '<|endoftext|>'
  257. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: PAD token = 2 '<|im_end|>'
  258. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: LF token = 143 'Ä'
  259. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: EOT token = 0 '<|endoftext|>'
  260. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: EOG token = 0 '<|endoftext|>'
  261. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: EOG token = 2 '<|im_end|>'
  262. Nov 07 13:33:13 slimb ollama[1817]: llm_load_print_meta: max token length = 162
  263. Nov 07 13:33:13 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  264. Nov 07 13:33:13 slimb ollama[1817]: time=2024-11-07T13:33:13.156+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  265. Nov 07 13:33:13 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  266. Nov 07 13:33:13 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  267. Nov 07 13:33:13 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  268. Nov 07 13:33:13 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  269. Nov 07 13:33:13 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.20 MiB
  270. Nov 07 13:33:15 slimb ollama[1817]: llm_load_tensors: offloading 24 repeating layers to GPU
  271. Nov 07 13:33:15 slimb ollama[1817]: llm_load_tensors: offloading non-repeating layers to GPU
  272. Nov 07 13:33:15 slimb ollama[1817]: llm_load_tensors: offloaded 25/25 layers to GPU
  273. Nov 07 13:33:15 slimb ollama[1817]: llm_load_tensors: ROCm0 buffer size = 1339.13 MiB
  274. Nov 07 13:33:15 slimb ollama[1817]: llm_load_tensors: CPU buffer size = 78.75 MiB
  275. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: n_ctx = 8192
  276. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: n_batch = 512
  277. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: n_ubatch = 512
  278. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: flash_attn = 0
  279. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: freq_base = 130000.0
  280. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: freq_scale = 1
  281. Nov 07 13:33:17 slimb ollama[1817]: llama_kv_cache_init: ROCm0 KV buffer size = 1536.00 MiB
  282. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: KV self size = 1536.00 MiB, K (f16): 768.00 MiB, V (f16): 768.00 MiB
  283. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: ROCm_Host output buffer size = 0.78 MiB
  284. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: ROCm0 compute buffer size = 544.00 MiB
  285. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: ROCm_Host compute buffer size = 20.01 MiB
  286. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: graph nodes = 774
  287. Nov 07 13:33:17 slimb ollama[1817]: llama_new_context_with_model: graph splits = 2
  288. Nov 07 13:33:17 slimb ollama[1894]: INFO [main] model loaded | tid="139635018588992" timestamp=1730982797
  289. Nov 07 13:33:17 slimb ollama[1817]: time=2024-11-07T13:33:17.925+01:00 level=INFO source=server.go:626 msg="llama runner started in 5.27 seconds"
  290. Nov 07 13:33:17 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:17 | 200 | 5.28710079s | 127.0.0.1 | POST "/api/generate"
  291. Nov 07 13:33:52 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:52 | 200 | 1.012966639s | 127.0.0.1 | POST "/api/chat"
  292. Nov 07 13:33:59 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:59 | 200 | 20.115µs | 127.0.0.1 | HEAD "/"
  293. Nov 07 13:33:59 slimb ollama[1817]: [GIN] 2024/11/07 - 13:33:59 | 200 | 21.163µs | 127.0.0.1 | GET "/api/ps"
  294. Nov 07 13:34:44 slimb ollama[1817]: [GIN] 2024/11/07 - 13:34:44 | 200 | 23.26µs | 127.0.0.1 | HEAD "/"
  295. Nov 07 13:34:44 slimb ollama[1817]: [GIN] 2024/11/07 - 13:34:44 | 404 | 61.958µs | 127.0.0.1 | POST "/api/show"
  296. Nov 07 13:34:46 slimb ollama[1817]: time=2024-11-07T13:34:46.182+01:00 level=INFO source=download.go:175 msg="downloading ac1e61f30b7e in 11 100 MB part(s)"
  297. Nov 07 13:35:01 slimb ollama[1817]: time=2024-11-07T13:35:01.520+01:00 level=INFO source=download.go:175 msg="downloading 966de95ca8a6 in 1 1.4 KB part(s)"
  298. Nov 07 13:35:03 slimb ollama[1817]: time=2024-11-07T13:35:03.653+01:00 level=INFO source=download.go:175 msg="downloading fcc5a6bec9da in 1 7.7 KB part(s)"
  299. Nov 07 13:35:05 slimb ollama[1817]: time=2024-11-07T13:35:05.816+01:00 level=INFO source=download.go:175 msg="downloading a70ff7e570d9 in 1 6.0 KB part(s)"
  300. Nov 07 13:35:08 slimb ollama[1817]: time=2024-11-07T13:35:08.102+01:00 level=INFO source=download.go:175 msg="downloading 8e435f287afb in 1 485 B part(s)"
  301. Nov 07 13:35:09 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:09 | 200 | 25.489258419s | 127.0.0.1 | POST "/api/pull"
  302. Nov 07 13:35:09 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:09 | 200 | 13.906621ms | 127.0.0.1 | POST "/api/show"
  303. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.818+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="12.1 GiB"
  304. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.818+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-ac1e61f30b7e7be438c0b6a6aade8e3a1a1a8d50b5184ab82385c84a7b43a081 gpu=0 parallel=4 available=12993271808 required="2.2 GiB"
  305. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.818+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="8.0 GiB" free_swap="7.9 GiB"
  306. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.818+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=17 layers.offload=17 layers.split="" memory.available="[12.1 GiB]" memory.gpu_overhead="0 B" memory.required.full="2.2 GiB" memory.required.partial="2.2 GiB" memory.required.kv="256.0 MiB" memory.required.allocations="[2.2 GiB]" memory.weights.total="1017.5 MiB" memory.weights.repeating="812.0 MiB" memory.weights.nonrepeating="205.5 MiB" memory.graph.full="544.0 MiB" memory.graph.partial="554.3 MiB"
  307. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.819+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-ac1e61f30b7e7be438c0b6a6aade8e3a1a1a8d50b5184ab82385c84a7b43a081 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 17 --threads 8 --parallel 4 --port 44623"
  308. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.819+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=2
  309. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.820+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  310. Nov 07 13:35:09 slimb ollama[1817]: time=2024-11-07T13:35:09.820+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  311. Nov 07 13:35:09 slimb ollama[1941]: INFO [main] starting c++ runner | tid="140571664233280" timestamp=1730982909
  312. Nov 07 13:35:09 slimb ollama[1941]: INFO [main] build info | build=10 commit="8f7ec4a" tid="140571664233280" timestamp=1730982909
  313. Nov 07 13:35:09 slimb ollama[1941]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140571664233280" timestamp=1730982909 total_threads=16
  314. Nov 07 13:35:09 slimb ollama[1941]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="44623" tid="140571664233280" timestamp=1730982909
  315. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 147 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-ac1e61f30b7e7be438c0b6a6aade8e3a1a1a8d50b5184ab82385c84a7b43a081 (version GGUF V3 (latest))
  316. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  317. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  318. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  319. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 1B Instruct
  320. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  321. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  322. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 1B
  323. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  324. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  325. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 16
  326. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  327. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 2048
  328. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  329. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 32
  330. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  331. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  332. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  333. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 64
  334. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 64
  335. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 18
  336. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  337. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 64
  338. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  339. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  340. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  341. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  342. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  343. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  344. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  345. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  346. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  347. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - type f32: 34 tensors
  348. Nov 07 13:35:09 slimb ollama[1817]: llama_model_loader: - type q6_K: 113 tensors
  349. Nov 07 13:35:10 slimb ollama[1817]: time=2024-11-07T13:35:10.072+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  350. Nov 07 13:35:10 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  351. Nov 07 13:35:10 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  352. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  353. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: arch = llama
  354. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  355. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  356. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  357. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  358. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  359. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_embd = 2048
  360. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_layer = 16
  361. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_head = 32
  362. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  363. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_rot = 64
  364. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  365. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 64
  366. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 64
  367. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_gqa = 4
  368. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 512
  369. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 512
  370. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  371. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  372. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  373. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  374. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  375. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  376. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  377. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  378. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  379. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  380. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  381. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  382. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  383. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  384. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  385. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  386. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  387. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  388. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  389. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  390. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  391. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: model type = 1B
  392. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  393. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: model params = 1.24 B
  394. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: model size = 967.00 MiB (6.56 BPW)
  395. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 1B Instruct
  396. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  397. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  398. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  399. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  400. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  401. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  402. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  403. Nov 07 13:35:10 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  404. Nov 07 13:35:10 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  405. Nov 07 13:35:10 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  406. Nov 07 13:35:10 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  407. Nov 07 13:35:10 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  408. Nov 07 13:35:10 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  409. Nov 07 13:35:10 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.14 MiB
  410. Nov 07 13:35:11 slimb ollama[1817]: llm_load_tensors: offloading 16 repeating layers to GPU
  411. Nov 07 13:35:11 slimb ollama[1817]: llm_load_tensors: offloading non-repeating layers to GPU
  412. Nov 07 13:35:11 slimb ollama[1817]: llm_load_tensors: offloaded 17/17 layers to GPU
  413. Nov 07 13:35:11 slimb ollama[1817]: llm_load_tensors: ROCm0 buffer size = 967.00 MiB
  414. Nov 07 13:35:11 slimb ollama[1817]: llm_load_tensors: CPU buffer size = 205.49 MiB
  415. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: n_ctx = 8192
  416. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: n_batch = 512
  417. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: n_ubatch = 512
  418. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: flash_attn = 0
  419. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: freq_base = 500000.0
  420. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: freq_scale = 1
  421. Nov 07 13:35:13 slimb ollama[1817]: llama_kv_cache_init: ROCm0 KV buffer size = 256.00 MiB
  422. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB
  423. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: ROCm_Host output buffer size = 1.99 MiB
  424. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: ROCm0 compute buffer size = 544.00 MiB
  425. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: ROCm_Host compute buffer size = 20.01 MiB
  426. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: graph nodes = 518
  427. Nov 07 13:35:13 slimb ollama[1817]: llama_new_context_with_model: graph splits = 2
  428. Nov 07 13:35:13 slimb ollama[1941]: INFO [main] model loaded | tid="140571664233280" timestamp=1730982913
  429. Nov 07 13:35:14 slimb ollama[1817]: time=2024-11-07T13:35:14.132+01:00 level=INFO source=server.go:626 msg="llama runner started in 4.31 seconds"
  430. Nov 07 13:35:14 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:14 | 200 | 4.354370997s | 127.0.0.1 | POST "/api/generate"
  431. Nov 07 13:35:17 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:17 | 200 | 866.073453ms | 127.0.0.1 | POST "/api/chat"
  432. Nov 07 13:35:36 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:36 | 200 | 19.767µs | 127.0.0.1 | HEAD "/"
  433. Nov 07 13:35:36 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:36 | 200 | 29.825µs | 127.0.0.1 | GET "/api/ps"
  434. Nov 07 13:35:59 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:59 | 200 | 20.884µs | 127.0.0.1 | HEAD "/"
  435. Nov 07 13:35:59 slimb ollama[1817]: [GIN] 2024/11/07 - 13:35:59 | 200 | 9.727622ms | 127.0.0.1 | POST "/api/show"
  436. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.420+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  437. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.420+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c gpu=0 parallel=4 available=10594557952 required="9.0 GiB"
  438. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.421+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.1 GiB" free_swap="7.0 GiB"
  439. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.421+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="9.0 GiB" memory.required.partial="9.0 GiB" memory.required.kv="1.2 GiB" memory.required.allocations="[9.0 GiB]" memory.weights.total="7.3 GiB" memory.weights.repeating="7.2 GiB" memory.weights.nonrepeating="157.5 MiB" memory.graph.full="853.3 MiB" memory.graph.partial="853.3 MiB"
  440. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.422+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 41 --threads 8 --no-mmap --parallel 4 --port 34697"
  441. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.422+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  442. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.422+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  443. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.422+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  444. Nov 07 13:35:59 slimb ollama[2070]: INFO [main] starting c++ runner | tid="140583909581632" timestamp=1730982959
  445. Nov 07 13:35:59 slimb ollama[2070]: INFO [main] build info | build=10 commit="8f7ec4a" tid="140583909581632" timestamp=1730982959
  446. Nov 07 13:35:59 slimb ollama[2070]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140583909581632" timestamp=1730982959 total_threads=16
  447. Nov 07 13:35:59 slimb ollama[2070]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="34697" tid="140583909581632" timestamp=1730982959
  448. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: loaded meta data with 35 key-value pairs and 362 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c (version GGUF V3 (latest))
  449. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  450. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = granite
  451. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  452. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Granite 3.0 8b Instruct
  453. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = instruct
  454. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = granite-3.0
  455. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 8B
  456. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 6: general.license str = apache-2.0
  457. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 7: general.tags arr[str,3] = ["language", "granite-3.0", "text-gen...
  458. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 8: granite.block_count u32 = 40
  459. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 9: granite.context_length u32 = 4096
  460. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 10: granite.embedding_length u32 = 4096
  461. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 11: granite.feed_forward_length u32 = 12800
  462. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 12: granite.attention.head_count u32 = 32
  463. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 13: granite.attention.head_count_kv u32 = 8
  464. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 14: granite.rope.freq_base f32 = 10000.000000
  465. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 15: granite.attention.layer_norm_rms_epsilon f32 = 0.000010
  466. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 16: general.file_type u32 = 18
  467. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 17: granite.vocab_size u32 = 49155
  468. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 18: granite.rope.dimension_count u32 = 128
  469. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 19: tokenizer.ggml.add_space_prefix bool = false
  470. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 20: granite.attention.scale f32 = 0.007812
  471. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 21: granite.embedding_scale f32 = 12.000000
  472. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 22: granite.residual_scale f32 = 0.220000
  473. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 23: granite.logit_scale f32 = 16.000000
  474. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.model str = gpt2
  475. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.pre str = refact
  476. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.tokens arr[str,49155] = ["<|end_of_text|>", "<fim_prefix>", "...
  477. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.token_type arr[i32,49155] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
  478. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.ggml.merges arr[str,48891] = ["Ġ Ġ", "ĠĠ ĠĠ", "ĠĠĠĠ ĠĠ...
  479. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 29: tokenizer.ggml.bos_token_id u32 = 0
  480. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 30: tokenizer.ggml.eos_token_id u32 = 0
  481. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 0
  482. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
  483. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 33: tokenizer.chat_template str = {%- if tools %}\n {{- '<|start_of_r...
  484. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - kv 34: general.quantization_version u32 = 2
  485. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - type f32: 81 tensors
  486. Nov 07 13:35:59 slimb ollama[1817]: llama_model_loader: - type q6_K: 281 tensors
  487. Nov 07 13:35:59 slimb ollama[1817]: llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
  488. Nov 07 13:35:59 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 22
  489. Nov 07 13:35:59 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.2826 MB
  490. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  491. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: arch = granite
  492. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  493. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_vocab = 49155
  494. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_merges = 48891
  495. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  496. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 4096
  497. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_embd = 4096
  498. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_layer = 40
  499. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_head = 32
  500. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  501. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  502. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  503. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  504. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  505. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_gqa = 4
  506. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  507. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  508. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  509. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  510. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  511. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  512. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 1.6e+01
  513. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_ff = 12800
  514. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  515. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  516. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  517. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  518. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  519. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  520. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 10000.0
  521. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  522. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 4096
  523. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  524. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  525. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  526. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  527. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  528. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  529. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  530. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  531. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: model params = 8.17 B
  532. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: model size = 6.24 GiB (6.56 BPW)
  533. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: general.name = Granite 3.0 8b Instruct
  534. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: BOS token = 0 '<|end_of_text|>'
  535. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: EOS token = 0 '<|end_of_text|>'
  536. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: PAD token = 0 '<|end_of_text|>'
  537. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: LF token = 145 'Ä'
  538. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: EOG token = 0 '<|end_of_text|>'
  539. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: max token length = 512
  540. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_embedding_scale = 12.000000
  541. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_residual_scale = 0.220000
  542. Nov 07 13:35:59 slimb ollama[1817]: llm_load_print_meta: f_attention_scale = 0.007812
  543. Nov 07 13:35:59 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  544. Nov 07 13:35:59 slimb ollama[1817]: time=2024-11-07T13:35:59.673+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  545. Nov 07 13:36:00 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  546. Nov 07 13:36:00 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  547. Nov 07 13:36:00 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  548. Nov 07 13:36:00 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  549. Nov 07 13:36:00 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.34 MiB
  550. Nov 07 13:36:00 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 6393.15 MiB on device 0: cudaMalloc failed: out of memory
  551. Nov 07 13:36:00 slimb ollama[1817]: llama_model_load: error loading model: unable to allocate backend buffer
  552. Nov 07 13:36:00 slimb ollama[1817]: llama_load_model_from_file: exception loading model
  553. Nov 07 13:36:00 slimb ollama[1817]: terminate called after throwing an instance of 'std::runtime_error'
  554. Nov 07 13:36:00 slimb ollama[1817]: what(): unable to allocate backend buffer
  555. Nov 07 13:36:01 slimb ollama[1817]: time=2024-11-07T13:36:01.127+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  556. Nov 07 13:36:02 slimb ollama[1817]: time=2024-11-07T13:36:02.732+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: error loading model: unable to allocate backend buffer\nllama_load_model_from_file: exception loading model"
  557. Nov 07 13:36:02 slimb ollama[1817]: [GIN] 2024/11/07 - 13:36:02 | 500 | 3.326382656s | 127.0.0.1 | POST "/api/generate"
  558. Nov 07 13:36:07 slimb ollama[1817]: time=2024-11-07T13:36:07.734+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.001156002 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  559. Nov 07 13:36:07 slimb ollama[1817]: time=2024-11-07T13:36:07.984+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251873658 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  560. Nov 07 13:36:08 slimb ollama[1817]: time=2024-11-07T13:36:08.233+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.500948729 model=/usr/share/ollama/.ollama/models/blobs/sha256-846ef8ae4d6f8aace01d454e048e5c0f761849323a1cc4d55d7b4118715bb59c
  561. Nov 07 13:38:02 slimb ollama[1817]: [GIN] 2024/11/07 - 13:38:02 | 200 | 23.192µs | 127.0.0.1 | HEAD "/"
  562. Nov 07 13:38:02 slimb ollama[1817]: [GIN] 2024/11/07 - 13:38:02 | 200 | 2.390104ms | 127.0.0.1 | GET "/api/tags"
  563. Nov 07 13:38:29 slimb ollama[1817]: [GIN] 2024/11/07 - 13:38:29 | 200 | 21.806µs | 127.0.0.1 | HEAD "/"
  564. Nov 07 13:38:29 slimb ollama[1817]: [GIN] 2024/11/07 - 13:38:29 | 404 | 75.552µs | 127.0.0.1 | POST "/api/show"
  565. Nov 07 13:38:31 slimb ollama[1817]: time=2024-11-07T13:38:31.851+01:00 level=INFO source=download.go:175 msg="downloading 0ba5c285c16b in 16 165 MB part(s)"
  566. Nov 07 13:39:09 slimb ollama[1817]: time=2024-11-07T13:39:09.032+01:00 level=INFO source=download.go:175 msg="downloading 56bb8bd477a5 in 1 96 B part(s)"
  567. Nov 07 13:39:11 slimb ollama[1817]: time=2024-11-07T13:39:11.218+01:00 level=INFO source=download.go:175 msg="downloading cc88376befda in 1 559 B part(s)"
  568. Nov 07 13:39:13 slimb ollama[1817]: [GIN] 2024/11/07 - 13:39:13 | 200 | 44.552714266s | 127.0.0.1 | POST "/api/pull"
  569. Nov 07 13:39:13 slimb ollama[1817]: [GIN] 2024/11/07 - 13:39:13 | 200 | 14.09498ms | 127.0.0.1 | POST "/api/show"
  570. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.825+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  571. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.825+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 gpu=0 parallel=4 available=10594557952 required="4.3 GiB"
  572. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.825+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.6 GiB" free_swap="5.4 GiB"
  573. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.826+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="4.3 GiB" memory.required.kv="896.0 MiB" memory.required.allocations="[4.3 GiB]" memory.weights.total="3.0 GiB" memory.weights.repeating="2.7 GiB" memory.weights.nonrepeating="308.2 MiB" memory.graph.full="424.0 MiB" memory.graph.partial="570.7 MiB"
  574. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.827+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 29 --threads 8 --parallel 4 --port 43057"
  575. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.827+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  576. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.827+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  577. Nov 07 13:39:13 slimb ollama[1817]: time=2024-11-07T13:39:13.827+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  578. Nov 07 13:39:13 slimb ollama[2141]: INFO [main] starting c++ runner | tid="139888547337024" timestamp=1730983153
  579. Nov 07 13:39:13 slimb ollama[2141]: INFO [main] build info | build=10 commit="8f7ec4a" tid="139888547337024" timestamp=1730983153
  580. Nov 07 13:39:13 slimb ollama[2141]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="139888547337024" timestamp=1730983153 total_threads=16
  581. Nov 07 13:39:13 slimb ollama[2141]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="43057" tid="139888547337024" timestamp=1730983153
  582. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 255 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 (version GGUF V3 (latest))
  583. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  584. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  585. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  586. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 3B Instruct
  587. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  588. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  589. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 3B
  590. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  591. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  592. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 28
  593. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  594. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 3072
  595. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  596. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 24
  597. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  598. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  599. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  600. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 128
  601. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 128
  602. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 18
  603. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  604. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 128
  605. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  606. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  607. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  608. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  609. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  610. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  611. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  612. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  613. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  614. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - type f32: 58 tensors
  615. Nov 07 13:39:13 slimb ollama[1817]: llama_model_loader: - type q6_K: 197 tensors
  616. Nov 07 13:39:14 slimb ollama[1817]: time=2024-11-07T13:39:14.079+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  617. Nov 07 13:39:14 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  618. Nov 07 13:39:14 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  619. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  620. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: arch = llama
  621. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  622. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  623. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  624. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  625. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  626. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_embd = 3072
  627. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_layer = 28
  628. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_head = 24
  629. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  630. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  631. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  632. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  633. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  634. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_gqa = 3
  635. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  636. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  637. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  638. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  639. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  640. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  641. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  642. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  643. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  644. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  645. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  646. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  647. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  648. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  649. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  650. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  651. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  652. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  653. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  654. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  655. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  656. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  657. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  658. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  659. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  660. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: model params = 3.21 B
  661. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: model size = 2.45 GiB (6.56 BPW)
  662. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 3B Instruct
  663. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  664. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  665. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  666. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  667. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  668. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  669. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  670. Nov 07 13:39:14 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  671. Nov 07 13:39:14 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  672. Nov 07 13:39:15 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  673. Nov 07 13:39:15 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  674. Nov 07 13:39:15 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  675. Nov 07 13:39:15 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  676. Nov 07 13:39:15 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.24 MiB
  677. Nov 07 13:39:15 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2513.91 MiB on device 0: cudaMalloc failed: out of memory
  678. Nov 07 13:39:15 slimb ollama[1817]: llama_model_load: error loading model: unable to allocate backend buffer
  679. Nov 07 13:39:15 slimb ollama[1817]: llama_load_model_from_file: exception loading model
  680. Nov 07 13:39:15 slimb ollama[1817]: terminate called after throwing an instance of 'std::runtime_error'
  681. Nov 07 13:39:15 slimb ollama[1817]: what(): unable to allocate backend buffer
  682. Nov 07 13:39:15 slimb ollama[1817]: time=2024-11-07T13:39:15.786+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  683. Nov 07 13:39:17 slimb ollama[1817]: time=2024-11-07T13:39:17.090+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  684. Nov 07 13:39:17 slimb ollama[1817]: time=2024-11-07T13:39:17.341+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: error loading model: unable to allocate backend buffer\nllama_load_model_from_file: exception loading model"
  685. Nov 07 13:39:17 slimb ollama[1817]: [GIN] 2024/11/07 - 13:39:17 | 500 | 3.57590613s | 127.0.0.1 | POST "/api/generate"
  686. Nov 07 13:39:22 slimb ollama[1817]: time=2024-11-07T13:39:22.342+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.001105529 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  687. Nov 07 13:39:22 slimb ollama[1817]: time=2024-11-07T13:39:22.593+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251361554 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  688. Nov 07 13:39:22 slimb ollama[1817]: time=2024-11-07T13:39:22.842+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.500303203 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  689. Nov 07 13:39:33 slimb ollama[1817]: [GIN] 2024/11/07 - 13:39:33 | 200 | 21.044µs | 127.0.0.1 | HEAD "/"
  690. Nov 07 13:39:33 slimb ollama[1817]: [GIN] 2024/11/07 - 13:39:33 | 404 | 99.77µs | 127.0.0.1 | POST "/api/show"
  691. Nov 07 13:39:35 slimb ollama[1817]: time=2024-11-07T13:39:35.513+01:00 level=INFO source=download.go:175 msg="downloading dde5aa3fc5ff in 16 126 MB part(s)"
  692. Nov 07 13:40:03 slimb ollama[1817]: time=2024-11-07T13:40:03.749+01:00 level=INFO source=download.go:175 msg="downloading 34bb5ab01051 in 1 561 B part(s)"
  693. Nov 07 13:40:05 slimb ollama[1817]: [GIN] 2024/11/07 - 13:40:05 | 200 | 32.051945363s | 127.0.0.1 | POST "/api/pull"
  694. Nov 07 13:40:05 slimb ollama[1817]: [GIN] 2024/11/07 - 13:40:05 | 200 | 15.295874ms | 127.0.0.1 | POST "/api/show"
  695. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.012+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  696. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.012+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff gpu=0 parallel=4 available=10594557952 required="3.7 GiB"
  697. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.012+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.5 GiB" free_swap="5.4 GiB"
  698. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.012+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="3.7 GiB" memory.required.partial="3.7 GiB" memory.required.kv="896.0 MiB" memory.required.allocations="[3.7 GiB]" memory.weights.total="2.4 GiB" memory.weights.repeating="2.1 GiB" memory.weights.nonrepeating="308.2 MiB" memory.graph.full="424.0 MiB" memory.graph.partial="570.7 MiB"
  699. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.013+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 29 --threads 8 --parallel 4 --port 41775"
  700. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.013+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  701. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.014+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  702. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.014+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  703. Nov 07 13:40:06 slimb ollama[2196]: INFO [main] starting c++ runner | tid="139852984795968" timestamp=1730983206
  704. Nov 07 13:40:06 slimb ollama[2196]: INFO [main] build info | build=10 commit="8f7ec4a" tid="139852984795968" timestamp=1730983206
  705. Nov 07 13:40:06 slimb ollama[2196]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="139852984795968" timestamp=1730983206 total_threads=16
  706. Nov 07 13:40:06 slimb ollama[2196]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="41775" tid="139852984795968" timestamp=1730983206
  707. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 255 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff (version GGUF V3 (latest))
  708. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  709. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  710. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  711. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 3B Instruct
  712. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  713. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  714. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 3B
  715. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  716. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  717. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 28
  718. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  719. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 3072
  720. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  721. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 24
  722. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  723. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  724. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  725. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 128
  726. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 128
  727. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 15
  728. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  729. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 128
  730. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  731. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  732. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  733. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  734. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  735. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  736. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  737. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  738. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  739. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - type f32: 58 tensors
  740. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - type q4_K: 168 tensors
  741. Nov 07 13:40:06 slimb ollama[1817]: llama_model_loader: - type q6_K: 29 tensors
  742. Nov 07 13:40:06 slimb ollama[1817]: time=2024-11-07T13:40:06.266+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  743. Nov 07 13:40:06 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  744. Nov 07 13:40:06 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  745. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  746. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: arch = llama
  747. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  748. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  749. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  750. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  751. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  752. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_embd = 3072
  753. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_layer = 28
  754. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_head = 24
  755. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  756. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  757. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  758. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  759. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  760. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_gqa = 3
  761. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  762. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  763. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  764. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  765. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  766. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  767. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  768. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  769. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  770. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  771. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  772. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  773. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  774. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  775. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  776. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  777. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  778. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  779. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  780. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  781. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  782. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  783. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  784. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  785. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: model ftype = Q4_K - Medium
  786. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: model params = 3.21 B
  787. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: model size = 1.87 GiB (5.01 BPW)
  788. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 3B Instruct
  789. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  790. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  791. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  792. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  793. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  794. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  795. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  796. Nov 07 13:40:06 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  797. Nov 07 13:40:06 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  798. Nov 07 13:40:06 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  799. Nov 07 13:40:06 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  800. Nov 07 13:40:06 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  801. Nov 07 13:40:06 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  802. Nov 07 13:40:06 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.24 MiB
  803. Nov 07 13:40:07 slimb ollama[1817]: llm_load_tensors: offloading 28 repeating layers to GPU
  804. Nov 07 13:40:07 slimb ollama[1817]: llm_load_tensors: offloading non-repeating layers to GPU
  805. Nov 07 13:40:07 slimb ollama[1817]: llm_load_tensors: offloaded 29/29 layers to GPU
  806. Nov 07 13:40:07 slimb ollama[1817]: llm_load_tensors: ROCm0 buffer size = 1918.36 MiB
  807. Nov 07 13:40:07 slimb ollama[1817]: llm_load_tensors: CPU buffer size = 308.23 MiB
  808. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: n_ctx = 8192
  809. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: n_batch = 512
  810. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: n_ubatch = 512
  811. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: flash_attn = 0
  812. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: freq_base = 500000.0
  813. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: freq_scale = 1
  814. Nov 07 13:40:10 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 896.00 MiB on device 0: cudaMalloc failed: out of memory
  815. Nov 07 13:40:10 slimb ollama[1817]: llama_kv_cache_init: failed to allocate buffer for kv cache
  816. Nov 07 13:40:10 slimb ollama[1817]: llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache
  817. Nov 07 13:40:10 slimb ollama[2196]: ERROR [load_model] unable to load model | model="/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff" tid="139852984795968" timestamp=1730983210
  818. Nov 07 13:40:10 slimb ollama[1817]: terminate called without an active exception
  819. Nov 07 13:40:10 slimb ollama[1817]: time=2024-11-07T13:40:10.730+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  820. Nov 07 13:40:12 slimb ollama[1817]: time=2024-11-07T13:40:12.334+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: cudaMalloc failed: out of memory\nllama_kv_cache_init: failed to allocate buffer for kv cache\nllama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache"
  821. Nov 07 13:40:12 slimb ollama[1817]: [GIN] 2024/11/07 - 13:40:12 | 500 | 6.364768766s | 127.0.0.1 | POST "/api/generate"
  822. Nov 07 13:40:17 slimb ollama[1817]: time=2024-11-07T13:40:17.334+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.000319877 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  823. Nov 07 13:40:17 slimb ollama[1817]: time=2024-11-07T13:40:17.585+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251142684 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  824. Nov 07 13:40:17 slimb ollama[1817]: time=2024-11-07T13:40:17.835+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.501224244 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  825. Nov 07 13:42:44 slimb ollama[1817]: [GIN] 2024/11/07 - 13:42:44 | 200 | 19.554µs | 127.0.0.1 | HEAD "/"
  826. Nov 07 13:42:44 slimb ollama[1817]: [GIN] 2024/11/07 - 13:42:44 | 200 | 14.825303ms | 127.0.0.1 | POST "/api/show"
  827. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.660+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  828. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.661+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff gpu=0 parallel=4 available=10594557952 required="3.7 GiB"
  829. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.661+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.6 GiB" free_swap="5.4 GiB"
  830. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.661+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="3.7 GiB" memory.required.partial="3.7 GiB" memory.required.kv="896.0 MiB" memory.required.allocations="[3.7 GiB]" memory.weights.total="2.4 GiB" memory.weights.repeating="2.1 GiB" memory.weights.nonrepeating="308.2 MiB" memory.graph.full="424.0 MiB" memory.graph.partial="570.7 MiB"
  831. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.662+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 29 --threads 8 --parallel 4 --port 44245"
  832. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.663+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  833. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.663+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  834. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.663+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  835. Nov 07 13:42:44 slimb ollama[2257]: INFO [main] starting c++ runner | tid="140128741708608" timestamp=1730983364
  836. Nov 07 13:42:44 slimb ollama[2257]: INFO [main] build info | build=10 commit="8f7ec4a" tid="140128741708608" timestamp=1730983364
  837. Nov 07 13:42:44 slimb ollama[2257]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140128741708608" timestamp=1730983364 total_threads=16
  838. Nov 07 13:42:44 slimb ollama[2257]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="44245" tid="140128741708608" timestamp=1730983364
  839. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 255 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff (version GGUF V3 (latest))
  840. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  841. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  842. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  843. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 3B Instruct
  844. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  845. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  846. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 3B
  847. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  848. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  849. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 28
  850. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  851. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 3072
  852. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  853. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 24
  854. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  855. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  856. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  857. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 128
  858. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 128
  859. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 15
  860. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  861. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 128
  862. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  863. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  864. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  865. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  866. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  867. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  868. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  869. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  870. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  871. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - type f32: 58 tensors
  872. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - type q4_K: 168 tensors
  873. Nov 07 13:42:44 slimb ollama[1817]: llama_model_loader: - type q6_K: 29 tensors
  874. Nov 07 13:42:44 slimb ollama[1817]: time=2024-11-07T13:42:44.914+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  875. Nov 07 13:42:45 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  876. Nov 07 13:42:45 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  877. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  878. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: arch = llama
  879. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  880. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  881. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  882. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  883. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  884. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_embd = 3072
  885. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_layer = 28
  886. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_head = 24
  887. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  888. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  889. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  890. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  891. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  892. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_gqa = 3
  893. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  894. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  895. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  896. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  897. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  898. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  899. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  900. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  901. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  902. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  903. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  904. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  905. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  906. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  907. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  908. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  909. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  910. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  911. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  912. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  913. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  914. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  915. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  916. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  917. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: model ftype = Q4_K - Medium
  918. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: model params = 3.21 B
  919. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: model size = 1.87 GiB (5.01 BPW)
  920. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 3B Instruct
  921. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  922. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  923. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  924. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  925. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  926. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  927. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  928. Nov 07 13:42:45 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  929. Nov 07 13:42:45 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  930. Nov 07 13:42:45 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  931. Nov 07 13:42:45 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  932. Nov 07 13:42:45 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  933. Nov 07 13:42:45 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  934. Nov 07 13:42:45 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.24 MiB
  935. Nov 07 13:42:46 slimb ollama[1817]: llm_load_tensors: offloading 28 repeating layers to GPU
  936. Nov 07 13:42:46 slimb ollama[1817]: llm_load_tensors: offloading non-repeating layers to GPU
  937. Nov 07 13:42:46 slimb ollama[1817]: llm_load_tensors: offloaded 29/29 layers to GPU
  938. Nov 07 13:42:46 slimb ollama[1817]: llm_load_tensors: ROCm0 buffer size = 1918.36 MiB
  939. Nov 07 13:42:46 slimb ollama[1817]: llm_load_tensors: CPU buffer size = 308.23 MiB
  940. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: n_ctx = 8192
  941. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: n_batch = 512
  942. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: n_ubatch = 512
  943. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: flash_attn = 0
  944. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: freq_base = 500000.0
  945. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: freq_scale = 1
  946. Nov 07 13:42:48 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 896.00 MiB on device 0: cudaMalloc failed: out of memory
  947. Nov 07 13:42:48 slimb ollama[1817]: llama_kv_cache_init: failed to allocate buffer for kv cache
  948. Nov 07 13:42:48 slimb ollama[1817]: llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache
  949. Nov 07 13:42:48 slimb ollama[2257]: ERROR [load_model] unable to load model | model="/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff" tid="140128741708608" timestamp=1730983368
  950. Nov 07 13:42:48 slimb ollama[1817]: terminate called without an active exception
  951. Nov 07 13:42:49 slimb ollama[1817]: time=2024-11-07T13:42:49.380+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  952. Nov 07 13:42:50 slimb ollama[1817]: time=2024-11-07T13:42:50.654+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  953. Nov 07 13:42:50 slimb ollama[1817]: time=2024-11-07T13:42:50.905+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: cudaMalloc failed: out of memory\nllama_kv_cache_init: failed to allocate buffer for kv cache\nllama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache"
  954. Nov 07 13:42:50 slimb ollama[1817]: [GIN] 2024/11/07 - 13:42:50 | 500 | 6.287842592s | 127.0.0.1 | POST "/api/generate"
  955. Nov 07 13:42:55 slimb ollama[1817]: time=2024-11-07T13:42:55.906+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.000819883 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  956. Nov 07 13:42:56 slimb ollama[1817]: time=2024-11-07T13:42:56.156+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251133233 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  957. Nov 07 13:42:56 slimb ollama[1817]: time=2024-11-07T13:42:56.407+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.501333452 model=/usr/share/ollama/.ollama/models/blobs/sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff
  958. Nov 07 13:43:14 slimb ollama[1817]: [GIN] 2024/11/07 - 13:43:14 | 200 | 21.37µs | 127.0.0.1 | HEAD "/"
  959. Nov 07 13:43:14 slimb ollama[1817]: [GIN] 2024/11/07 - 13:43:14 | 200 | 2.77331ms | 127.0.0.1 | GET "/api/tags"
  960. Nov 07 13:43:30 slimb ollama[1817]: [GIN] 2024/11/07 - 13:43:30 | 200 | 21.928µs | 127.0.0.1 | HEAD "/"
  961. Nov 07 13:43:30 slimb ollama[1817]: [GIN] 2024/11/07 - 13:43:30 | 200 | 20.355653ms | 127.0.0.1 | POST "/api/show"
  962. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.342+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  963. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.342+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 gpu=0 parallel=4 available=10594557952 required="4.3 GiB"
  964. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.343+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.5 GiB" free_swap="5.4 GiB"
  965. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.343+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="4.3 GiB" memory.required.kv="896.0 MiB" memory.required.allocations="[4.3 GiB]" memory.weights.total="3.0 GiB" memory.weights.repeating="2.7 GiB" memory.weights.nonrepeating="308.2 MiB" memory.graph.full="424.0 MiB" memory.graph.partial="570.7 MiB"
  966. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.344+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 29 --threads 8 --parallel 4 --port 43831"
  967. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.344+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  968. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.344+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  969. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.345+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  970. Nov 07 13:43:30 slimb ollama[2318]: INFO [main] starting c++ runner | tid="139882696536896" timestamp=1730983410
  971. Nov 07 13:43:30 slimb ollama[2318]: INFO [main] build info | build=10 commit="8f7ec4a" tid="139882696536896" timestamp=1730983410
  972. Nov 07 13:43:30 slimb ollama[2318]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="139882696536896" timestamp=1730983410 total_threads=16
  973. Nov 07 13:43:30 slimb ollama[2318]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="43831" tid="139882696536896" timestamp=1730983410
  974. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 255 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 (version GGUF V3 (latest))
  975. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  976. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  977. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  978. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 3B Instruct
  979. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  980. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  981. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 3B
  982. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  983. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  984. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 28
  985. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  986. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 3072
  987. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  988. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 24
  989. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  990. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  991. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  992. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 128
  993. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 128
  994. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 18
  995. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  996. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 128
  997. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  998. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  999. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  1000. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  1001. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  1002. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  1003. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  1004. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  1005. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  1006. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - type f32: 58 tensors
  1007. Nov 07 13:43:30 slimb ollama[1817]: llama_model_loader: - type q6_K: 197 tensors
  1008. Nov 07 13:43:30 slimb ollama[1817]: time=2024-11-07T13:43:30.596+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  1009. Nov 07 13:43:30 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  1010. Nov 07 13:43:30 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  1011. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  1012. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: arch = llama
  1013. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  1014. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  1015. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  1016. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  1017. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  1018. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_embd = 3072
  1019. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_layer = 28
  1020. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_head = 24
  1021. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  1022. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  1023. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  1024. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  1025. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  1026. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_gqa = 3
  1027. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  1028. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  1029. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  1030. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  1031. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  1032. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  1033. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  1034. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  1035. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  1036. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  1037. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  1038. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  1039. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  1040. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  1041. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  1042. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  1043. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  1044. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  1045. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  1046. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  1047. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  1048. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  1049. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  1050. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  1051. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  1052. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: model params = 3.21 B
  1053. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: model size = 2.45 GiB (6.56 BPW)
  1054. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 3B Instruct
  1055. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  1056. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  1057. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  1058. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  1059. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  1060. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  1061. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  1062. Nov 07 13:43:30 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  1063. Nov 07 13:43:30 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  1064. Nov 07 13:43:31 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  1065. Nov 07 13:43:31 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  1066. Nov 07 13:43:31 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  1067. Nov 07 13:43:31 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  1068. Nov 07 13:43:31 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.24 MiB
  1069. Nov 07 13:43:34 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2513.91 MiB on device 0: cudaMalloc failed: out of memory
  1070. Nov 07 13:43:34 slimb ollama[1817]: llama_model_load: error loading model: unable to allocate backend buffer
  1071. Nov 07 13:43:34 slimb ollama[1817]: llama_load_model_from_file: exception loading model
  1072. Nov 07 13:43:34 slimb ollama[1817]: terminate called after throwing an instance of 'std::runtime_error'
  1073. Nov 07 13:43:34 slimb ollama[1817]: what(): unable to allocate backend buffer
  1074. Nov 07 13:43:34 slimb ollama[1817]: time=2024-11-07T13:43:34.843+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  1075. Nov 07 13:43:36 slimb ollama[1817]: time=2024-11-07T13:43:36.159+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  1076. Nov 07 13:43:36 slimb ollama[1817]: time=2024-11-07T13:43:36.411+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: error loading model: unable to allocate backend buffer\nllama_load_model_from_file: exception loading model"
  1077. Nov 07 13:43:36 slimb ollama[1817]: [GIN] 2024/11/07 - 13:43:36 | 500 | 6.1109438s | 127.0.0.1 | POST "/api/generate"
  1078. Nov 07 13:43:41 slimb ollama[1817]: time=2024-11-07T13:43:41.412+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.001229459 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1079. Nov 07 13:43:41 slimb ollama[1817]: time=2024-11-07T13:43:41.662+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251223389 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1080. Nov 07 13:43:41 slimb ollama[1817]: time=2024-11-07T13:43:41.912+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.500886865 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1081. Nov 07 13:44:57 slimb ollama[1817]: [GIN] 2024/11/07 - 13:44:57 | 200 | 22.413µs | 127.0.0.1 | HEAD "/"
  1082. Nov 07 13:44:57 slimb ollama[1817]: [GIN] 2024/11/07 - 13:44:57 | 200 | 52.180854ms | 127.0.0.1 | POST "/api/show"
  1083. Nov 07 13:44:57 slimb ollama[1817]: [GIN] 2024/11/07 - 13:44:57 | 200 | 5.670656ms | 127.0.0.1 | POST "/api/generate"
  1084. Nov 07 13:46:43 slimb ollama[1817]: [GIN] 2024/11/07 - 13:46:43 | 200 | 27.726µs | 127.0.0.1 | GET "/api/version"
  1085. Nov 07 13:49:56 slimb ollama[1817]: [GIN] 2024/11/07 - 13:49:56 | 200 | 19.064µs | 127.0.0.1 | HEAD "/"
  1086. Nov 07 13:49:56 slimb ollama[1817]: [GIN] 2024/11/07 - 13:49:56 | 200 | 14.469283ms | 127.0.0.1 | POST "/api/show"
  1087. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.219+01:00 level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=rocm total="16.0 GiB" available="9.9 GiB"
  1088. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.219+01:00 level=INFO source=sched.go:714 msg="new model will fit in available VRAM in single GPU, loading" model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 gpu=0 parallel=4 available=10594557952 required="4.3 GiB"
  1089. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.219+01:00 level=INFO source=server.go:105 msg="system memory" total="15.5 GiB" free="6.6 GiB" free_swap="5.4 GiB"
  1090. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.220+01:00 level=INFO source=memory.go:326 msg="offload to rocm" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[9.9 GiB]" memory.gpu_overhead="0 B" memory.required.full="4.3 GiB" memory.required.partial="4.3 GiB" memory.required.kv="896.0 MiB" memory.required.allocations="[4.3 GiB]" memory.weights.total="3.0 GiB" memory.weights.repeating="2.7 GiB" memory.weights.nonrepeating="308.2 MiB" memory.graph.full="424.0 MiB" memory.graph.partial="570.7 MiB"
  1091. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.221+01:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama414509819/runners/rocm_v60102/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 29 --threads 8 --parallel 4 --port 45523"
  1092. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.221+01:00 level=INFO source=sched.go:449 msg="loaded runners" count=3
  1093. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.221+01:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
  1094. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.221+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
  1095. Nov 07 13:49:56 slimb ollama[2423]: INFO [main] starting c++ runner | tid="140321031766848" timestamp=1730983796
  1096. Nov 07 13:49:56 slimb ollama[2423]: INFO [main] build info | build=10 commit="8f7ec4a" tid="140321031766848" timestamp=1730983796
  1097. Nov 07 13:49:56 slimb ollama[2423]: INFO [main] system info | n_threads=8 n_threads_batch=8 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140321031766848" timestamp=1730983796 total_threads=16
  1098. Nov 07 13:49:56 slimb ollama[2423]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="45523" tid="140321031766848" timestamp=1730983796
  1099. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: loaded meta data with 30 key-value pairs and 255 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8 (version GGUF V3 (latest))
  1100. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  1101. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 0: general.architecture str = llama
  1102. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 1: general.type str = model
  1103. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 2: general.name str = Llama 3.2 3B Instruct
  1104. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 3: general.finetune str = Instruct
  1105. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 4: general.basename str = Llama-3.2
  1106. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 5: general.size_label str = 3B
  1107. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 6: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam...
  1108. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 7: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ...
  1109. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 8: llama.block_count u32 = 28
  1110. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 9: llama.context_length u32 = 131072
  1111. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 10: llama.embedding_length u32 = 3072
  1112. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 11: llama.feed_forward_length u32 = 8192
  1113. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 12: llama.attention.head_count u32 = 24
  1114. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 13: llama.attention.head_count_kv u32 = 8
  1115. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 14: llama.rope.freq_base f32 = 500000.000000
  1116. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 15: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  1117. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 16: llama.attention.key_length u32 = 128
  1118. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 17: llama.attention.value_length u32 = 128
  1119. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 18: general.file_type u32 = 18
  1120. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 19: llama.vocab_size u32 = 128256
  1121. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 20: llama.rope.dimension_count u32 = 128
  1122. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 21: tokenizer.ggml.model str = gpt2
  1123. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 22: tokenizer.ggml.pre str = llama-bpe
  1124. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 23: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ...
  1125. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 24: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
  1126. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 25: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
  1127. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 26: tokenizer.ggml.bos_token_id u32 = 128000
  1128. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 27: tokenizer.ggml.eos_token_id u32 = 128009
  1129. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 28: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ...
  1130. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - kv 29: general.quantization_version u32 = 2
  1131. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - type f32: 58 tensors
  1132. Nov 07 13:49:56 slimb ollama[1817]: llama_model_loader: - type q6_K: 197 tensors
  1133. Nov 07 13:49:56 slimb ollama[1817]: time=2024-11-07T13:49:56.473+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
  1134. Nov 07 13:49:56 slimb ollama[1817]: llm_load_vocab: special tokens cache size = 256
  1135. Nov 07 13:49:56 slimb ollama[1817]: llm_load_vocab: token to piece cache size = 0.7999 MB
  1136. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: format = GGUF V3 (latest)
  1137. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: arch = llama
  1138. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: vocab type = BPE
  1139. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_vocab = 128256
  1140. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_merges = 280147
  1141. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: vocab_only = 0
  1142. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_ctx_train = 131072
  1143. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_embd = 3072
  1144. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_layer = 28
  1145. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_head = 24
  1146. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_head_kv = 8
  1147. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_rot = 128
  1148. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_swa = 0
  1149. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_embd_head_k = 128
  1150. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_embd_head_v = 128
  1151. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_gqa = 3
  1152. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_embd_k_gqa = 1024
  1153. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_embd_v_gqa = 1024
  1154. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: f_norm_eps = 0.0e+00
  1155. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  1156. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
  1157. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  1158. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: f_logit_scale = 0.0e+00
  1159. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_ff = 8192
  1160. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_expert = 0
  1161. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_expert_used = 0
  1162. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: causal attn = 1
  1163. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: pooling type = 0
  1164. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: rope type = 0
  1165. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: rope scaling = linear
  1166. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: freq_base_train = 500000.0
  1167. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: freq_scale_train = 1
  1168. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: n_ctx_orig_yarn = 131072
  1169. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: rope_finetuned = unknown
  1170. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: ssm_d_conv = 0
  1171. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: ssm_d_inner = 0
  1172. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: ssm_d_state = 0
  1173. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: ssm_dt_rank = 0
  1174. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: ssm_dt_b_c_rms = 0
  1175. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: model type = 3B
  1176. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: model ftype = Q6_K
  1177. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: model params = 3.21 B
  1178. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: model size = 2.45 GiB (6.56 BPW)
  1179. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: general.name = Llama 3.2 3B Instruct
  1180. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>'
  1181. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: EOS token = 128009 '<|eot_id|>'
  1182. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: LF token = 128 'Ä'
  1183. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: EOT token = 128009 '<|eot_id|>'
  1184. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: EOM token = 128008 '<|eom_id|>'
  1185. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: EOG token = 128008 '<|eom_id|>'
  1186. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: EOG token = 128009 '<|eot_id|>'
  1187. Nov 07 13:49:56 slimb ollama[1817]: llm_load_print_meta: max token length = 256
  1188. Nov 07 13:49:56 slimb ollama[1817]: /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
  1189. Nov 07 13:49:57 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
  1190. Nov 07 13:49:57 slimb ollama[1817]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
  1191. Nov 07 13:49:57 slimb ollama[1817]: ggml_cuda_init: found 1 ROCm devices:
  1192. Nov 07 13:49:57 slimb ollama[1817]: Device 0: AMD Radeon Graphics, compute capability 9.0, VMM: no
  1193. Nov 07 13:49:57 slimb ollama[1817]: llm_load_tensors: ggml ctx size = 0.24 MiB
  1194. Nov 07 13:49:57 slimb ollama[1817]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2513.91 MiB on device 0: cudaMalloc failed: out of memory
  1195. Nov 07 13:49:57 slimb ollama[1817]: llama_model_load: error loading model: unable to allocate backend buffer
  1196. Nov 07 13:49:57 slimb ollama[1817]: llama_load_model_from_file: exception loading model
  1197. Nov 07 13:49:57 slimb ollama[1817]: terminate called after throwing an instance of 'std::runtime_error'
  1198. Nov 07 13:49:57 slimb ollama[1817]: what(): unable to allocate backend buffer
  1199. Nov 07 13:49:57 slimb ollama[1817]: time=2024-11-07T13:49:57.678+01:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server not responding"
  1200. Nov 07 13:49:59 slimb ollama[1817]: time=2024-11-07T13:49:59.282+01:00 level=ERROR source=sched.go:455 msg="error loading llama server" error="llama runner process has terminated: error loading model: unable to allocate backend buffer\nllama_load_model_from_file: exception loading model"
  1201. Nov 07 13:49:59 slimb ollama[1817]: [GIN] 2024/11/07 - 13:49:59 | 500 | 3.106339582s | 127.0.0.1 | POST "/api/generate"
  1202. Nov 07 13:50:04 slimb ollama[1817]: time=2024-11-07T13:50:04.284+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.001304953 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1203. Nov 07 13:50:04 slimb ollama[1817]: time=2024-11-07T13:50:04.533+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.251130607 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1204. Nov 07 13:50:04 slimb ollama[1817]: time=2024-11-07T13:50:04.783+01:00 level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.500600546 model=/usr/share/ollama/.ollama/models/blobs/sha256-0ba5c285c16ba078b17959f8ee1ad4142d161c99c6e3e8c713aeb7a0316a95a8
  1205. Nov 07 13:53:17 slimb systemd[1]: Stopping ollama.service - Ollama Service...
  1206. Nov 07 13:53:18 slimb systemd[1]: ollama.service: Deactivated successfully.
  1207. Nov 07 13:53:18 slimb systemd[1]: Stopped ollama.service - Ollama Service.
  1208. Nov 07 13:53:18 slimb systemd[1]: ollama.service: Consumed 1min 37.574s CPU time, 6.7G memory peak, 2.4G memory swap peak.
  1209.  
Tags: Ollama
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!