API tools faq
paste
Advertisement
Guest User

notepadofmerge

a guest
Oct 11th, 2023
319
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.43 KB | None | 0 0
raw download clone embed print report
  1. slices:
  2. - sources:
  3. - model: Undi95/Mistral-11B-v0.1
  4. layer_range: [0, 48]
  5. - model: Undi95/Mistral-11B-CC-Air
  6. layer_range: [0, 48]
  7. merge_method: slerp
  8. base_model: Undi95/Mistral-11B-v0.1
  9. parameters:
  10. t:
  11. - filter: lm_head
  12. value: [0.75]
  13. - filter: embed_tokens
  14. value: [0.75]
  15. - filter: self_attn
  16. value: [0.75, 0.25]
  17. - filter: mlp
  18. value: [0.25, 0.75]
  19. - filter: layernorm
  20. value: [0.5, 0.5]
  21. - filter: modelnorm
  22. value: [0.75]
  23. - value: 0.5 # fallback for rest of tensors
  24. dtype: float16
  25.  
  26. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  27. | Task |Version| Metric |Value | |Stderr|
  28. |-------------|------:|--------|-----:|---|-----:|
  29. |arc_challenge| 0|acc |0.5273|± |0.0146|
  30. | | |acc_norm|0.5478|± |0.0145|
  31. |arc_easy | 0|acc |0.8136|± |0.0080|
  32. | | |acc_norm|0.8013|± |0.0082|
  33. |hellaswag | 0|acc |0.6254|± |0.0048|
  34. | | |acc_norm|0.8136|± |0.0039|
  35. |piqa | 0|acc |0.8069|± |0.0092|
  36. | | |acc_norm|0.8188|± |0.0090|
  37. |truthfulqa_mc| 1|mc1 |0.3696|± |0.0169|
  38. | | |mc2 |0.5317|± |0.0153|
  39. |winogrande | 0|acc |0.7285|± |0.0125|
  40.  
  41.  
  42. slices:
  43. - sources:
  44. - model: Undi95/Mistral-11B-v0.1
  45. layer_range: [0, 48]
  46. - model: Undi95/Mistral-11B-CC-Air
  47. layer_range: [0, 48]
  48. merge_method: slerp
  49. base_model: Undi95/Mistral-11B-v0.1
  50. parameters:
  51. t:
  52. - value: 0.5 # fallback for rest of tensors
  53. dtype: float16
  54.  
  55.  
  56.  
  57. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  58. | Task |Version| Metric |Value | |Stderr|
  59. |-------------|------:|--------|-----:|---|-----:|
  60. |arc_challenge| 0|acc |0.5230|± |0.0146|
  61. | | |acc_norm|0.5486|± |0.0145|
  62. |arc_easy | 0|acc |0.8157|± |0.0080|
  63. | | |acc_norm|0.8035|± |0.0082|
  64. |hellaswag | 0|acc |0.6262|± |0.0048|
  65. | | |acc_norm|0.8166|± |0.0039|
  66. |piqa | 0|acc |0.8074|± |0.0092|
  67. | | |acc_norm|0.8183|± |0.0090|
  68. |truthfulqa_mc| 1|mc1 |0.3647|± |0.0169|
  69. | | |mc2 |0.5282|± |0.0154|
  70. |winogrande | 0|acc |0.7332|± |0.0124|
  71.  
  72.  
  73. slices:
  74. - sources:
  75. - model: Norquinal/Mistral-7B-claude-chat
  76. layer_range: [0, 24]
  77. - sources:
  78. - model: Open-Orca/Mistral-7B-OpenOrca
  79. layer_range: [8, 32]
  80. merge_method: passthrough
  81. dtype: float16
  82.  
  83. ========================================================
  84.  
  85. slices:
  86. - sources:
  87. - model: Undi95/Mistral-11B-CC-Air
  88. layer_range: [0, 48]
  89. - model: "/content/drive/MyDrive/Mistral-11B-ClaudeOrca"
  90. layer_range: [0, 48]
  91. merge_method: slerp
  92. base_model: Undi95/Mistral-11B-CC-Air
  93. parameters:
  94. t:
  95. - value: 0.5 # fallback for rest of tensors
  96. dtype: float16
  97.  
  98. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  99. | Task |Version| Metric |Value | |Stderr|
  100. |-------------|------:|--------|-----:|---|-----:|
  101. |arc_challenge| 0|acc |0.5401|± |0.0146|
  102. | | |acc_norm|0.5589|± |0.0145|
  103. |arc_easy | 0|acc |0.8199|± |0.0079|
  104. | | |acc_norm|0.8127|± |0.0080|
  105. |hellaswag | 0|acc |0.6361|± |0.0048|
  106. | | |acc_norm|0.8202|± |0.0038|
  107. |piqa | 0|acc |0.8079|± |0.0092|
  108. | | |acc_norm|0.8199|± |0.0090|
  109. |truthfulqa_mc| 1|mc1 |0.3733|± |0.0169|
  110. | | |mc2 |0.5374|± |0.0156|
  111. |winogrande | 0|acc |0.7261|± |0.0125|
  112.  
  113.  
  114. slices:
  115. - sources:
  116. - model: Undi95/Mistral-11B-CC-Air
  117. layer_range: [0, 48]
  118. - model: "/content/drive/MyDrive/Mistral-11B-ClaudeOrca"
  119. layer_range: [0, 48]
  120. merge_method: slerp
  121. base_model: Undi95/Mistral-11B-v0.1
  122. parameters:
  123. t:
  124. - filter: lm_head
  125. value: [0.75]
  126. - filter: embed_tokens
  127. value: [0.75]
  128. - filter: self_attn
  129. value: [0.75, 0.25]
  130. - filter: mlp
  131. value: [0.25, 0.75]
  132. - filter: layernorm
  133. value: [0.5, 0.5]
  134. - filter: modelnorm
  135. value: [0.75]
  136. - value: 0.5 # fallback for rest of tensors
  137. dtype: float16
  138.  
  139. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  140. | Task |Version| Metric |Value | |Stderr|
  141. |-------------|------:|--------|-----:|---|-----:|
  142. |arc_challenge| 0|acc |0.5384|± |0.0146|
  143. | | |acc_norm|0.5589|± |0.0145|
  144. |arc_easy | 0|acc |0.8199|± |0.0079|
  145. | | |acc_norm|0.8072|± |0.0081|
  146. |hellaswag | 0|acc |0.6340|± |0.0048|
  147. | | |acc_norm|0.8208|± |0.0038|
  148. |piqa | 0|acc |0.8085|± |0.0092|
  149. | | |acc_norm|0.8205|± |0.0090|
  150. |truthfulqa_mc| 1|mc1 |0.3819|± |0.0170|
  151. | | |mc2 |0.5454|± |0.0155|
  152. |winogrande | 0|acc |0.7238|± |0.0126|
  153.  
  154.  
  155.  
  156. slices:
  157. - sources:
  158. - model: "/content/drive/MyDrive/Mistral-11B-ClaudeOrca"
  159. layer_range: [0, 48]
  160. - model: Undi95/Mistral-11B-CC-Air
  161. layer_range: [0, 48]
  162. merge_method: slerp
  163. base_model: Undi95/Mistral-11B-v0.1
  164. parameters:
  165. t:
  166. - filter: lm_head
  167. value: [0.75]
  168. - filter: embed_tokens
  169. value: [0.75]
  170. - filter: self_attn
  171. value: [0.75, 0.25]
  172. - filter: mlp
  173. value: [0.25, 0.75]
  174. - filter: layernorm
  175. value: [0.5, 0.5]
  176. - filter: modelnorm
  177. value: [0.75]
  178. - value: 0.5 # fallback for rest of tensors
  179. dtype: float16
  180.  
  181.  
  182. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  183. | Task |Version| Metric |Value | |Stderr|
  184. |-------------|------:|--------|-----:|---|-----:|
  185. |arc_challenge| 0|acc |0.5461|± |0.0145|
  186. | | |acc_norm|0.5589|± |0.0145|
  187. |arc_easy | 0|acc |0.8224|± |0.0078|
  188. | | |acc_norm|0.8072|± |0.0081|
  189. |hellaswag | 0|acc |0.6352|± |0.0048|
  190. | | |acc_norm|0.8198|± |0.0038|
  191. |piqa | 0|acc |0.8069|± |0.0092|
  192. | | |acc_norm|0.8205|± |0.0090|
  193. |truthfulqa_mc| 1|mc1 |0.3635|± |0.0168|
  194. | | |mc2 |0.5290|± |0.0157|
  195. |winogrande | 0|acc |0.7159|± |0.0127|
  196.  
  197.  
  198. slices:
  199. - sources:
  200. - model: ehartford/dolphin-2.0-mistral-7b
  201. layer_range: [0, 24]
  202. - sources:
  203. - model: PeanutJar/Mistral-v0.1-PeanutButter-v0.0.0-7B
  204. layer_range: [8, 32]
  205. merge_method: passthrough
  206. dtype: float16
  207.  
  208. =========================================================
  209.  
  210. slices:
  211. - sources:
  212. - model: "/content/drive/MyDrive/Mistral-11B-DolphinPeanut"
  213. layer_range: [0, 48]
  214. - model: Undi95/Mistral-11B-ClaudeOrca
  215. layer_range: [0, 48]
  216. merge_method: slerp
  217. base_model: "/content/drive/MyDrive/Mistral-11B-DolphinPeanut"
  218. parameters:
  219. t:
  220. - value: 0.5 # fallback for rest of tensors
  221. dtype: float16
  222.  
  223. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  224. | Task |Version| Metric |Value | |Stderr|
  225. |-------------|------:|--------|-----:|---|-----:|
  226. |arc_challenge| 0|acc |0.3985|± |0.0143|
  227. | | |acc_norm|0.4369|± |0.0145|
  228. |arc_easy | 0|acc |0.5593|± |0.0102|
  229. | | |acc_norm|0.5627|± |0.0102|
  230. |hellaswag | 0|acc |0.4109|± |0.0049|
  231. | | |acc_norm|0.5312|± |0.0050|
  232. |piqa | 0|acc |0.6491|± |0.0111|
  233. | | |acc_norm|0.6589|± |0.0111|
  234. |truthfulqa_mc| 1|mc1 |0.3207|± |0.0163|
  235. | | |mc2 |0.5351|± |0.0164|
  236. |winogrande | 0|acc |0.6504|± |0.0134|
  237.  
  238.  
  239. slices:
  240. - sources:
  241. - model: mistralai/Mistral-7B-v0.1
  242. layer_range: [0, 24]
  243. - sources:
  244. - model: HuggingFaceH4/zephyr-7b-alpha
  245. layer_range: [8, 32]
  246. merge_method: passthrough
  247. dtype: bfloat16
  248.  
  249. ================================================
  250.  
  251. slices:
  252. - sources:
  253. - model: Open-Orca/Mistral-7B-OpenOrca
  254. layer_range: [0, 24]
  255. - sources:
  256. - model: akjindal53244/Mistral-7B-v0.1-Open-Platypus
  257. layer_range: [8, 32]
  258. merge_method: passthrough
  259. dtype: bfloat16
  260.  
  261. ================================================
  262.  
  263. slices:
  264. - sources:
  265. - model: "/content/drive/MyDrive/Mistral-11B-Zephyr"
  266. layer_range: [0, 48]
  267. - model: "/content/drive/MyDrive/Mistral-11B-OpenOrcaPlatypus"
  268. layer_range: [0, 48]
  269. merge_method: slerp
  270. base_model: "/content/drive/MyDrive/Mistral-11B-Zephyr"
  271. parameters:
  272. t:
  273. - value: 0.5 # fallback for rest of tensors
  274. dtype: bfloat16
  275.  
  276.  
  277. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  278. | Task |Version| Metric |Value | |Stderr|
  279. |-------------|------:|--------|-----:|---|-----:|
  280. |arc_challenge| 0|acc |0.5469|± |0.0145|
  281. | | |acc_norm|0.5776|± |0.0144|
  282. |arc_easy | 0|acc |0.8249|± |0.0078|
  283. | | |acc_norm|0.8232|± |0.0078|
  284. |hellaswag | 0|acc |0.6198|± |0.0048|
  285. | | |acc_norm|0.8094|± |0.0039|
  286. |piqa | 0|acc |0.8139|± |0.0091|
  287. | | |acc_norm|0.8303|± |0.0088|
  288. |truthfulqa_mc| 1|mc1 |0.3011|± |0.0161|
  289. | | |mc2 |0.4744|± |0.0150|
  290. |winogrande | 0|acc |0.7466|± |0.0122|
  291.  
  292.  
  293. TEST7 ^^^^^^
  294.  
  295.  
  296. slices:
  297. - sources:
  298. - model: "/content/drive/MyDrive/CC-v1.1-7B-bf16"
  299. layer_range: [0, 24]
  300. - sources:
  301. - model: "/content/drive/MyDrive/Zephyr-7B"
  302. layer_range: [8, 32]
  303. merge_method: passthrough
  304. dtype: bfloat16
  305.  
  306. ================================================
  307.  
  308. slices:
  309. - sources:
  310. - model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
  311. layer_range: [0, 48]
  312. - model: Undi95/Mistral-11B-OpenOrcaPlatypus
  313. layer_range: [0, 48]
  314. merge_method: slerp
  315. base_model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
  316. parameters:
  317. t:
  318. - value: 0.5 # fallback for rest of tensors
  319. dtype: bfloat16
  320.  
  321. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  322. | Task |Version| Metric |Value | |Stderr|
  323. |-------------|------:|--------|-----:|---|-----:|
  324. |arc_challenge| 0|acc |0.5623|± |0.0145|
  325. | | |acc_norm|0.5794|± |0.0144|
  326. |arc_easy | 0|acc |0.8354|± |0.0076|
  327. | | |acc_norm|0.8165|± |0.0079|
  328. |hellaswag | 0|acc |0.6389|± |0.0048|
  329. | | |acc_norm|0.8236|± |0.0038|
  330. |piqa | 0|acc |0.8139|± |0.0091|
  331. | | |acc_norm|0.8264|± |0.0088|
  332. |truthfulqa_mc| 1|mc1 |0.3978|± |0.0171|
  333. | | |mc2 |0.5607|± |0.0155|
  334. |winogrande | 0|acc |0.7451|± |0.0122|
  335.  
  336. slices:
  337. - sources:
  338. - model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
  339. layer_range: [0, 48]
  340. - model: Undi95/Mistral-11B-OpenOrcaPlatypus
  341. layer_range: [0, 48]
  342. merge_method: slerp
  343. base_model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
  344. parameters:
  345. t:
  346. - filter: lm_head
  347. value: [0.75]
  348. - filter: embed_tokens
  349. value: [0.75]
  350. - filter: self_attn
  351. value: [0.75, 0.25]
  352. - filter: mlp
  353. value: [0.25, 0.75]
  354. - filter: layernorm
  355. value: [0.5, 0.5]
  356. - filter: modelnorm
  357. value: [0.75]
  358. - value: 0.5 # fallback for rest of tensors
  359. dtype: float16
  360.  
  361. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  362. | Task |Version| Metric |Value | |Stderr|
  363. |-------------|------:|--------|-----:|---|-----:|
  364. |arc_challenge| 0|acc |0.5623|± |0.0145|
  365. | | |acc_norm|0.5802|± |0.0144|
  366. |arc_easy | 0|acc |0.8321|± |0.0077|
  367. | | |acc_norm|0.8110|± |0.0080|
  368. |hellaswag | 0|acc |0.6391|± |0.0048|
  369. | | |acc_norm|0.8252|± |0.0038|
  370. |piqa | 0|acc |0.8101|± |0.0092|
  371. | | |acc_norm|0.8286|± |0.0088|
  372. |truthfulqa_mc| 1|mc1 |0.3953|± |0.0171|
  373. | | |mc2 |0.5529|± |0.0155|
  374. |winogrande | 0|acc |0.7514|± |0.0121|
  375.  
  376. slices:
  377. - sources:
  378. - model: Undi95/Mistral-11B-OpenOrcaPlatypus
  379. layer_range: [0, 48]
  380. - model: "/content/drive/MyDrive/Mistral-11B-CC-Zephyr"
  381. layer_range: [0, 48]
  382. merge_method: slerp
  383. base_model: Undi95/Mistral-11B-OpenOrcaPlatypus
  384. parameters:
  385. t:
  386. - filter: lm_head
  387. value: [0.75]
  388. - filter: embed_tokens
  389. value: [0.75]
  390. - filter: self_attn
  391. value: [0.75, 0.25]
  392. - filter: mlp
  393. value: [0.25, 0.75]
  394. - filter: layernorm
  395. value: [0.5, 0.5]
  396. - filter: modelnorm
  397. value: [0.75]
  398. - value: 0.5 # fallback for rest of tensors
  399. dtype: float16
  400.  
  401. hf-causal-experimental (pretrained=/content/drive/MyDrive/Mistral-11B-Test), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4
  402. | Task |Version| Metric |Value | |Stderr|
  403. |-------------|------:|--------|-----:|---|-----:|
  404. |arc_challenge| 0|acc |0.5597|± |0.0145|
  405. | | |acc_norm|0.5819|± |0.0144|
  406. |arc_easy | 0|acc |0.8308|± |0.0077|
  407. | | |acc_norm|0.8215|± |0.0079|
  408. |hellaswag | 0|acc |0.6371|± |0.0048|
  409. | | |acc_norm|0.8213|± |0.0038|
  410. |piqa | 0|acc |0.8134|± |0.0091|
  411. | | |acc_norm|0.8275|± |0.0088|
  412. |truthfulqa_mc| 1|mc1 |0.3990|± |0.0171|
  413. | | |mc2 |0.5685|± |0.0155|
  414. |winogrande | 0|acc |0.7474|± |0.0122|
  415.  
  416.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!