Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no
- llm_load_tensors: ggml ctx size = 0.68 MiB
- llm_load_tensors: offloading 49 repeating layers to GPU
- llm_load_tensors: offloaded 49/65 layers to GPU
- llm_load_tensors: ROCm0 buffer size = 13613.88 MiB
- llm_load_tensors: CPU buffer size = 18926.01 MiB
- llama_new_context_with_model: n_ctx = 32768
- llama_new_context_with_model: n_batch = 512
- llama_new_context_with_model: n_ubatch = 512
- llama_new_context_with_model: flash_attn = 0
- llama_new_context_with_model: freq_base = 1000000.0
- llama_new_context_with_model: freq_scale = 1
- llama_kv_cache_init: ROCm0 KV buffer size = 6272.00 MiB
- llama_kv_cache_init: ROCm_Host KV buffer size = 1920.00 MiB
- llama_new_context_with_model: KV self size = 8192.00 MiB, K (f16): 4096.00 MiB, V (f16): 4096.00 MiB
- llama_new_context_with_model: ROCm_Host output buffer size = 0.60 MiB
- llama_new_context_with_model: ROCm0 compute buffer size = 2782.02 MiB
- llama_new_context_with_model: ROCm_Host compute buffer size = 74.01 MiB
- llama_new_context_with_model: graph nodes = 2246
- llama_new_context_with_model: graph splits = 214
- time=2024-11-27T22:18:08.388Z level=INFO source=server.go:601 msg="llama runner started in 5.27 seconds"
- llama_model_loader: loaded meta data with 34 key-value pairs and 771 tensors from /home/ollama/blobs/sha256-ac3d1ba8aa77755dab3806d9024e9c385ea0d5b412d6bdf9157f8a4a7e9fc0d9 (version GGUF V3 (latest))
- llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
- llama_model_loader: - kv 0: general.architecture str = qwen2
- llama_model_loader: - kv 1: general.type str = model
- llama_model_loader: - kv 2: general.name str = Qwen2.5 Coder 32B Instruct
- llama_model_loader: - kv 3: general.finetune str = Instruct
- llama_model_loader: - kv 4: general.basename str = Qwen2.5-Coder
- llama_model_loader: - kv 5: general.size_label str = 32B
- llama_model_loader: - kv 6: general.license str = apache-2.0
- llama_model_loader: - kv 7: general.license.link str = https://huggingface.co/Qwen/Qwen2.5-C...
- llama_model_loader: - kv 8: general.base_model.count u32 = 1
- llama_model_loader: - kv 9: general.base_model.0.name str = Qwen2.5 Coder 32B
- llama_model_loader: - kv 10: general.base_model.0.organization str = Qwen
- llama_model_loader: - kv 11: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen2.5-C...
- llama_model_loader: - kv 12: general.tags arr[str,6] = ["code", "codeqwen", "chat", "qwen", ...
- llama_model_loader: - kv 13: general.languages arr[str,1] = ["en"]
- llama_model_loader: - kv 14: qwen2.block_count u32 = 64
- llama_model_loader: - kv 15: qwen2.context_length u32 = 32768
- llama_model_loader: - kv 16: qwen2.embedding_length u32 = 5120
- llama_model_loader: - kv 17: qwen2.feed_forward_length u32 = 27648
- llama_model_loader: - kv 18: qwen2.attention.head_count u32 = 40
- llama_model_loader: - kv 19: qwen2.attention.head_count_kv u32 = 8
- llama_model_loader: - kv 20: qwen2.rope.freq_base f32 = 1000000.000000
- llama_model_loader: - kv 21: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001
- llama_model_loader: - kv 22: general.file_type u32 = 15
- llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
- llama_model_loader: - kv 24: tokenizer.ggml.pre str = qwen2
- llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ...
- llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
- llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
- llama_model_loader: - kv 28: tokenizer.ggml.eos_token_id u32 = 151645
- llama_model_loader: - kv 29: tokenizer.ggml.padding_token_id u32 = 151643
- llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 151643
- llama_model_loader: - kv 31: tokenizer.ggml.add_bos_token bool = false
- llama_model_loader: - kv 32: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
- llama_model_loader: - kv 33: general.quantization_version u32 = 2
- llama_model_loader: - type f32: 321 tensors
- llama_model_loader: - type q4_K: 385 tensors
- llama_model_loader: - type q6_K: 65 tensors
- llm_load_vocab: special tokens cache size = 22
- llm_load_vocab: token to piece cache size = 0.9310 MB
- llm_load_print_meta: format = GGUF V3 (latest)
- llm_load_print_meta: arch = qwen2
- llm_load_print_meta: vocab type = BPE
- llm_load_print_meta: n_vocab = 152064
- llm_load_print_meta: n_merges = 151387
- llm_load_print_meta: vocab_only = 1
- llm_load_print_meta: model type = ?B
- llm_load_print_meta: model ftype = all F32
- llm_load_print_meta: model params = 32.76 B
- llm_load_print_meta: model size = 18.48 GiB (4.85 BPW)
- llm_load_print_meta: general.name = Qwen2.5 Coder 32B Instruct
- llm_load_print_meta: BOS token = 151643 '<|endoftext|>'
- llm_load_print_meta: EOS token = 151645 '<|im_end|>'
- llm_load_print_meta: PAD token = 151643 '<|endoftext|>'
- llm_load_print_meta: LF token = 148848 'ÄĬ'
- llm_load_print_meta: EOT token = 151645 '<|im_end|>'
- llm_load_print_meta: EOG token = 151643 '<|endoftext|>'
- llm_load_print_meta: EOG token = 151645 '<|im_end|>'
- llm_load_print_meta: max token length = 256
- llama_model_load: vocab only - skipping tensors
- time=2024-11-27T22:18:08.651Z level=WARN source=runner.go:129 msg="truncating input prompt" limit=32768 prompt=46386 keep=4 new=32768
- CUDA error: out of memory
- current device: 0, in function alloc at ggml-cuda.cu:337
- ggml_cuda_device_malloc(&ptr, look_ahead_size, device)
- ggml-cuda.cu:132: CUDA error
- ptrace: Operation not permitted.
- No stack.
- The program is not being run.
- SIGABRT: abort
- PC=0x771218ea53f4 m=4 sigcode=18446744073709551610
- signal arrived during cgo execution
- goroutine 8 gp=0xc0002001c0 m=4 mp=0xc000071508 [syscall]:
- runtime.cgocall(0x5f82136aad70, 0xc000080b50)
- runtime/cgocall.go:167 +0x4b fp=0xc000080b28 sp=0xc000080af0 pc=0x5f821345bd2b
- github.com/ollama/ollama/llama._Cfunc_llama_decode(0x5f8251daae80, {0x200, 0x5f82517abd70, 0x0, 0x0, 0x5f8250f92f90, 0x5f8251e557e0, 0x5f8251e6b530, 0x5f8251e5b920, 0x0, ...})
- _cgo_gotypes.go:551 +0x52 fp=0xc000080b50 sp=0xc000080b28 pc=0x5f8213505d72
- github.com/ollama/ollama/llama.(*Context).Decode.func1(0x5f82136a2e8b?, 0x5f8251daae80?)
- github.com/ollama/ollama/llama/llama.go:167 +0xd8 fp=0xc000080c70 sp=0xc000080b50 pc=0x5f8213508398
Advertisement
Add Comment
Please, Sign In to add comment
