API tools faq
paste
Guest User

ollama.log

a guest
May 16th, 2024
85
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.20 KB | None | 0 0
raw download clone embed print report
  1. ~ $ ollama serve
  2. 2024/05/16 22:49:34 routes.go:1008: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:4 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]"
  3. time=2024-05-16T22:49:34.770-03:00 level=INFO source=images.go:704 msg="total blobs: 30"
  4. time=2024-05-16T22:49:34.770-03:00 level=INFO source=images.go:711 msg="total unused blobs removed: 0"
  5. [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
  6.  
  7. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
  8. - using env: export GIN_MODE=release
  9. - using code: gin.SetMode(gin.ReleaseMode)
  10.  
  11. [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers)
  12. [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
  13. [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
  14. [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
  15. [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers)
  16. [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers)
  17. [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers)
  18. [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers)
  19. [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers)
  20. [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
  21. [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
  22. [GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).ProcessHandler-fm (5 handlers)
  23. [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
  24. [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
  25. [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
  26. [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
  27. [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
  28. [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
  29. [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
  30. time=2024-05-16T22:49:34.770-03:00 level=INFO source=routes.go:1054 msg="Listening on 127.0.0.1:11434 (version 0.1.38)"
  31. time=2024-05-16T22:49:34.770-03:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama955761170/runners
  32. time=2024-05-16T22:49:37.963-03:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v12]"
  33. time=2024-05-16T22:49:38.049-03:00 level=INFO source=types.go:71 msg="inference compute" id=GPU-82d7151a-aa6c-62b4-4cc9-be19f8671093 library=cuda compute=8.6 driver=12.4 name="NVIDIA GeForce RTX 3070" total="7.8 GiB" available="7.4 GiB"
  34. [GIN] 2024/05/16 - 22:49:45 | 200 | 17.895µs | 127.0.0.1 | HEAD "/"
  35. [GIN] 2024/05/16 - 22:49:45 | 200 | 370.373µs | 127.0.0.1 | POST "/api/show"
  36. [GIN] 2024/05/16 - 22:49:45 | 200 | 198.927µs | 127.0.0.1 | POST "/api/show"
  37. time=2024-05-16T22:49:46.001-03:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=24 memory.available="7.4 GiB" memory.required.full="8.4 GiB" memory.required.partial="7.4 GiB" memory.required.kv="2.6 GiB" memory.weights.total="4.7 GiB" memory.weights.repeating="4.1 GiB" memory.weights.nonrepeating="615.2 MiB" memory.graph.full="506.0 MiB" memory.graph.partial="1.1 GiB"
  38. time=2024-05-16T22:49:46.001-03:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=24 memory.available="7.4 GiB" memory.required.full="8.4 GiB" memory.required.partial="7.4 GiB" memory.required.kv="2.6 GiB" memory.weights.total="4.7 GiB" memory.weights.repeating="4.1 GiB" memory.weights.nonrepeating="615.2 MiB" memory.graph.full="506.0 MiB" memory.graph.partial="1.1 GiB"
  39. time=2024-05-16T22:49:46.002-03:00 level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=24 memory.available="7.4 GiB" memory.required.full="8.4 GiB" memory.required.partial="7.4 GiB" memory.required.kv="2.6 GiB" memory.weights.total="4.7 GiB" memory.weights.repeating="4.1 GiB" memory.weights.nonrepeating="615.2 MiB" memory.graph.full="506.0 MiB" memory.graph.partial="1.1 GiB"
  40. time=2024-05-16T22:49:46.002-03:00 level=INFO source=server.go:320 msg="starting llama server" cmd="/tmp/ollama955761170/runners/cuda_v12/ollama_llama_server --model /home/mb/.ollama/models/blobs/sha256-ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 24 --parallel 4 --port 42461"
  41. time=2024-05-16T22:49:46.003-03:00 level=INFO source=sched.go:338 msg="loaded runners" count=1
  42. time=2024-05-16T22:49:46.003-03:00 level=INFO source=server.go:504 msg="waiting for llama runner to start responding"
  43. time=2024-05-16T22:49:46.003-03:00 level=INFO source=server.go:540 msg="waiting for server to become available" status="llm server error"
  44. INFO [main] build info | build=2770 commit="952d03dbe" tid="129739979091968" timestamp=1715910586
  45. INFO [main] system info | n_threads=8 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="129739979091968" timestamp=1715910586 total_threads=16
  46. INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="42461" tid="129739979091968" timestamp=1715910586
  47. llama_model_loader: loaded meta data with 24 key-value pairs and 254 tensors from /home/mb/.ollama/models/blobs/sha256-ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77 (version GGUF V3 (latest))
  48. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  49. llama_model_loader: - kv 0: general.architecture str = gemma
  50. llama_model_loader: - kv 1: general.name str = gemma-1.1-7b-it
  51. llama_model_loader: - kv 2: gemma.context_length u32 = 8192
  52. llama_model_loader: - kv 3: gemma.embedding_length u32 = 3072
  53. llama_model_loader: - kv 4: gemma.block_count u32 = 28
  54. llama_model_loader: - kv 5: gemma.feed_forward_length u32 = 24576
  55. llama_model_loader: - kv 6: gemma.attention.head_count u32 = 16
  56. llama_model_loader: - kv 7: gemma.attention.head_count_kv u32 = 16
  57. llama_model_loader: - kv 8: gemma.attention.layer_norm_rms_epsilon f32 = 0.000001
  58. llama_model_loader: - kv 9: gemma.attention.key_length u32 = 256
  59. llama_model_loader: - kv 10: gemma.attention.value_length u32 = 256
  60. llama_model_loader: - kv 11: general.file_type u32 = 2
  61. llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
  62. llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,256000] = ["<pad>", "<eos>", "<bos>", "<unk>", ...
  63. llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,256000] = [0.000000, 0.000000, 0.000000, 0.0000...
  64. llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,256000] = [3, 3, 3, 2, 1, 1, 1, 1, 1, 1, 1, 1, ...
  65. llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 2
  66. llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 1
  67. llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 3
  68. llama_model_loader: - kv 19: tokenizer.ggml.padding_token_id u32 = 0
  69. llama_model_loader: - kv 20: tokenizer.ggml.add_bos_token bool = true
  70. llama_model_loader: - kv 21: tokenizer.ggml.add_eos_token bool = false
  71. llama_model_loader: - kv 22: tokenizer.chat_template str = {{ bos_token }}{% if messages[0]['rol...
  72. llama_model_loader: - kv 23: general.quantization_version u32 = 2
  73. llama_model_loader: - type f32: 57 tensors
  74. llama_model_loader: - type q4_0: 196 tensors
  75. llama_model_loader: - type q6_K: 1 tensors
  76. time=2024-05-16T22:49:46.254-03:00 level=INFO source=server.go:540 msg="waiting for server to become available" status="llm server loading model"
  77. llm_load_vocab: mismatch in special tokens definition ( 416/256000 vs 260/256000 ).
  78. llm_load_print_meta: format = GGUF V3 (latest)
  79. llm_load_print_meta: arch = gemma
  80. llm_load_print_meta: vocab type = SPM
  81. llm_load_print_meta: n_vocab = 256000
  82. llm_load_print_meta: n_merges = 0
  83. llm_load_print_meta: n_ctx_train = 8192
  84. llm_load_print_meta: n_embd = 3072
  85. llm_load_print_meta: n_head = 16
  86. llm_load_print_meta: n_head_kv = 16
  87. llm_load_print_meta: n_layer = 28
  88. llm_load_print_meta: n_rot = 192
  89. llm_load_print_meta: n_embd_head_k = 256
  90. llm_load_print_meta: n_embd_head_v = 256
  91. llm_load_print_meta: n_gqa = 1
  92. llm_load_print_meta: n_embd_k_gqa = 4096
  93. llm_load_print_meta: n_embd_v_gqa = 4096
  94. llm_load_print_meta: f_norm_eps = 0.0e+00
  95. llm_load_print_meta: f_norm_rms_eps = 1.0e-06
  96. llm_load_print_meta: f_clamp_kqv = 0.0e+00
  97. llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  98. llm_load_print_meta: f_logit_scale = 0.0e+00
  99. llm_load_print_meta: n_ff = 24576
  100. llm_load_print_meta: n_expert = 0
  101. llm_load_print_meta: n_expert_used = 0
  102. llm_load_print_meta: causal attn = 1
  103. llm_load_print_meta: pooling type = 0
  104. llm_load_print_meta: rope type = 2
  105. llm_load_print_meta: rope scaling = linear
  106. llm_load_print_meta: freq_base_train = 10000.0
  107. llm_load_print_meta: freq_scale_train = 1
  108. llm_load_print_meta: n_yarn_orig_ctx = 8192
  109. llm_load_print_meta: rope_finetuned = unknown
  110. llm_load_print_meta: ssm_d_conv = 0
  111. llm_load_print_meta: ssm_d_inner = 0
  112. llm_load_print_meta: ssm_d_state = 0
  113. llm_load_print_meta: ssm_dt_rank = 0
  114. llm_load_print_meta: model type = 7B
  115. llm_load_print_meta: model ftype = Q4_0
  116. llm_load_print_meta: model params = 8.54 B
  117. llm_load_print_meta: model size = 4.66 GiB (4.69 BPW)
  118. llm_load_print_meta: general.name = gemma-1.1-7b-it
  119. llm_load_print_meta: BOS token = 2 '<bos>'
  120. llm_load_print_meta: EOS token = 1 '<eos>'
  121. llm_load_print_meta: UNK token = 3 '<unk>'
  122. llm_load_print_meta: PAD token = 0 '<pad>'
  123. llm_load_print_meta: LF token = 227 '<0x0A>'
  124. llm_load_print_meta: EOT token = 107 '<end_of_turn>'
  125. ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes
  126. ggml_cuda_init: CUDA_USE_TENSOR_CORES: no
  127. ggml_cuda_init: found 1 CUDA devices:
  128. Device 0: NVIDIA GeForce RTX 3070, compute capability 8.6, VMM: yes
  129. llm_load_tensors: ggml ctx size = 0.26 MiB
  130. llm_load_tensors: offloading 24 repeating layers to GPU
  131. llm_load_tensors: offloaded 24/29 layers to GPU
  132. llm_load_tensors: CPU buffer size = 4773.90 MiB
  133. llm_load_tensors: CUDA0 buffer size = 3564.56 MiB
  134. ................................................................................
  135. llama_new_context_with_model: n_ctx = 8192
  136. llama_new_context_with_model: n_batch = 512
  137. llama_new_context_with_model: n_ubatch = 512
  138. llama_new_context_with_model: freq_base = 10000.0
  139. llama_new_context_with_model: freq_scale = 1
  140. llama_kv_cache_init: CUDA_Host KV buffer size = 512.00 MiB
  141. llama_kv_cache_init: CUDA0 KV buffer size = 3072.00 MiB
  142. llama_new_context_with_model: KV self size = 3584.00 MiB, K (f16): 1792.00 MiB, V (f16): 1792.00 MiB
  143. llama_new_context_with_model: CUDA_Host output buffer size = 3.95 MiB
  144. ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1121.23 MiB on device 0: cudaMalloc failed: out of memory
  145. ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 1175699456
  146. llama_new_context_with_model: failed to allocate compute buffers
  147. llama_init_from_gpt_params: error: failed to create context with model '/home/mb/.ollama/models/blobs/sha256-ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77'
  148. ERROR [load_model] unable to load model | model="/home/mb/.ollama/models/blobs/sha256-ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77" tid="129739979091968" timestamp=1715910587
  149. terminate called without an active exception
  150. time=2024-05-16T22:49:47.458-03:00 level=INFO source=server.go:540 msg="waiting for server to become available" status="llm server not responding"
  151. time=2024-05-16T22:49:48.159-03:00 level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) error:failed to create context with model '/home/mb/.ollama/models/blobs/sha256-ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77'"
  152. [GIN] 2024/05/16 - 22:49:48 | 500 | 2.661179539s | 127.0.0.1 | POST "/api/chat"
  153.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!