API tools faq
paste
Advertisement
Guest User

ASRock AMD BC-250 Vulkan

a guest
Dec 2nd, 2024
987
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.01 KB | None | 0 0
raw download clone embed print report
  1. [user@localhost llama.cpp]$ ./build/bin/llama-cli -m "/home/user/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v0.3-GGUF/snapshots/b32046744d93031a26c8e925de2c8932c305f7b9/tinyllama-1.1b-chat-v0.3.Q8_0.gguf" -p "Hi you how are you" -n 50 -e -ngl 33 -t 4
  2. ggml_vulkan: Found 1 Vulkan devices:
  3. ggml_vulkan: 0 = AMD Radeon Graphics (RADV NAVI10) (radv) | uma: 1 | fp16: 1 | warp size: 64
  4. build: 4237 (917786f4) with cc (GCC) 14.2.1 20240912 (Red Hat 14.2.1-3) for x86_64-redhat-linux
  5. main: llama backend init
  6. main: load the model and apply lora adapter, if any
  7. llama_load_model_from_file: using device Vulkan0 (AMD Radeon Graphics (RADV NAVI10)) - 8014 MiB free
  8. llama_model_loader: loaded meta data with 20 key-value pairs and 201 tensors from /home/user/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v0.3-GGUF/snapshots/b32046744d93031a26c8e925de2c8932c305f7b9/tinyllama-1.1b-chat-v0.3.Q8_0.gguf (version GGUF V2)
  9. llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
  10. llama_model_loader: - kv 0: general.architecture str = llama
  11. llama_model_loader: - kv 1: general.name str = py007_tinyllama-1.1b-chat-v0.3
  12. llama_model_loader: - kv 2: llama.context_length u32 = 2048
  13. llama_model_loader: - kv 3: llama.embedding_length u32 = 2048
  14. llama_model_loader: - kv 4: llama.block_count u32 = 22
  15. llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632
  16. llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64
  17. llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
  18. llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4
  19. llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
  20. llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
  21. llama_model_loader: - kv 11: general.file_type u32 = 7
  22. llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
  23. llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32003] = ["<unk>", "<s>", "</s>", "<0x00>", "<...
  24. llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32003] = [0.000000, 0.000000, 0.000000, 0.0000...
  25. llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32003] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
  26. llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 1
  27. llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 2
  28. llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 0
  29. llama_model_loader: - kv 19: general.quantization_version u32 = 2
  30. llama_model_loader: - type f32: 45 tensors
  31. llama_model_loader: - type q8_0: 156 tensors
  32. llm_load_vocab: control-looking token: 32002 '<|im_end|>' was not control-type; this is probably a bug in the model. its type will be overridden
  33. llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
  34. llm_load_vocab: special tokens cache size = 6
  35. llm_load_vocab: token to piece cache size = 0.1684 MB
  36. llm_load_print_meta: format = GGUF V2
  37. llm_load_print_meta: arch = llama
  38. llm_load_print_meta: vocab type = SPM
  39. llm_load_print_meta: n_vocab = 32003
  40. llm_load_print_meta: n_merges = 0
  41. llm_load_print_meta: vocab_only = 0
  42. llm_load_print_meta: n_ctx_train = 2048
  43. llm_load_print_meta: n_embd = 2048
  44. llm_load_print_meta: n_layer = 22
  45. llm_load_print_meta: n_head = 32
  46. llm_load_print_meta: n_head_kv = 4
  47. llm_load_print_meta: n_rot = 64
  48. llm_load_print_meta: n_swa = 0
  49. llm_load_print_meta: n_embd_head_k = 64
  50. llm_load_print_meta: n_embd_head_v = 64
  51. llm_load_print_meta: n_gqa = 8
  52. llm_load_print_meta: n_embd_k_gqa = 256
  53. llm_load_print_meta: n_embd_v_gqa = 256
  54. llm_load_print_meta: f_norm_eps = 0.0e+00
  55. llm_load_print_meta: f_norm_rms_eps = 1.0e-05
  56. llm_load_print_meta: f_clamp_kqv = 0.0e+00
  57. llm_load_print_meta: f_max_alibi_bias = 0.0e+00
  58. llm_load_print_meta: f_logit_scale = 0.0e+00
  59. llm_load_print_meta: n_ff = 5632
  60. llm_load_print_meta: n_expert = 0
  61. llm_load_print_meta: n_expert_used = 0
  62. llm_load_print_meta: causal attn = 1
  63. llm_load_print_meta: pooling type = 0
  64. llm_load_print_meta: rope type = 0
  65. llm_load_print_meta: rope scaling = linear
  66. llm_load_print_meta: freq_base_train = 10000.0
  67. llm_load_print_meta: freq_scale_train = 1
  68. llm_load_print_meta: n_ctx_orig_yarn = 2048
  69. llm_load_print_meta: rope_finetuned = unknown
  70. llm_load_print_meta: ssm_d_conv = 0
  71. llm_load_print_meta: ssm_d_inner = 0
  72. llm_load_print_meta: ssm_d_state = 0
  73. llm_load_print_meta: ssm_dt_rank = 0
  74. llm_load_print_meta: ssm_dt_b_c_rms = 0
  75. llm_load_print_meta: model type = 1B
  76. llm_load_print_meta: model ftype = Q8_0
  77. llm_load_print_meta: model params = 1.10 B
  78. llm_load_print_meta: model size = 1.09 GiB (8.50 BPW)
  79. llm_load_print_meta: general.name = py007_tinyllama-1.1b-chat-v0.3
  80. llm_load_print_meta: BOS token = 1 '<s>'
  81. llm_load_print_meta: EOS token = 2 '</s>'
  82. llm_load_print_meta: EOT token = 32002 '<|im_end|>'
  83. llm_load_print_meta: UNK token = 0 '<unk>'
  84. llm_load_print_meta: LF token = 13 '<0x0A>'
  85. llm_load_print_meta: EOG token = 2 '</s>'
  86. llm_load_print_meta: EOG token = 32002 '<|im_end|>'
  87. llm_load_print_meta: max token length = 48
  88. ggml_vulkan: Compiling shaders..............................Done!
  89. llm_load_tensors: offloading 22 repeating layers to GPU
  90. llm_load_tensors: offloading output layer to GPU
  91. llm_load_tensors: offloaded 23/23 layers to GPU
  92. llm_load_tensors: Vulkan0 model buffer size = 1048.51 MiB
  93. llm_load_tensors: CPU_Mapped model buffer size = 66.41 MiB
  94. ...............................................................................
  95. llama_new_context_with_model: n_seq_max = 1
  96. llama_new_context_with_model: n_ctx = 4096
  97. llama_new_context_with_model: n_ctx_per_seq = 4096
  98. llama_new_context_with_model: n_batch = 2048
  99. llama_new_context_with_model: n_ubatch = 512
  100. llama_new_context_with_model: flash_attn = 0
  101. llama_new_context_with_model: freq_base = 10000.0
  102. llama_new_context_with_model: freq_scale = 1
  103. llama_new_context_with_model: n_ctx_pre_seq (4096) > n_ctx_train (2048) -- possible training context overflow
  104. llama_kv_cache_init: Vulkan0 KV buffer size = 88.00 MiB
  105. llama_new_context_with_model: KV self size = 88.00 MiB, K (f16): 44.00 MiB, V (f16): 44.00 MiB
  106. llama_new_context_with_model: Vulkan_Host output buffer size = 0.12 MiB
  107. llama_new_context_with_model: Vulkan0 compute buffer size = 280.00 MiB
  108. llama_new_context_with_model: Vulkan_Host compute buffer size = 12.01 MiB
  109. llama_new_context_with_model: graph nodes = 710
  110. llama_new_context_with_model: graph splits = 2
  111. common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
  112. main: llama threadpool init, n_threads = 4
  113. main: model was trained on only 2048 context tokens (4096 specified)
  114.  
  115. system_info: n_threads = 4 (n_threads_batch = 4) / 12 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | AARCH64_REPACK = 1 |
  116.  
  117. sampler seed: 4294967295
  118. sampler params:
  119. repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
  120. dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = -1
  121. top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, temp = 0.800
  122. mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
  123. sampler chain: logits -> logit-bias -> penalties -> dry -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
  124. generate: n_ctx = 4096, n_batch = 2048, n_predict = 50, n_keep = 1
  125.  
  126. Hi you how are you doing, hope you are doing good, how are you doing today?
  127. Hi I am doing well today, still sleepy but getting there. I hope your doing well too.
  128. Hi how is your day going today ?
  129. Hi how is your
  130.  
  131. llama_perf_sampler_print: sampling time = 5.87 ms / 56 runs ( 0.10 ms per token, 9543.29 tokens per second)
  132. llama_perf_context_print: load time = 4051.55 ms
  133. llama_perf_context_print: prompt eval time = 83.38 ms / 6 tokens ( 13.90 ms per token, 71.96 tokens per second)
  134. llama_perf_context_print: eval time = 330.61 ms / 49 runs ( 6.75 ms per token, 148.21 tokens per second)
  135. llama_perf_context_print: total time = 425.95 ms / 55 tokens
Tags: llama.cpp
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!