API tools faq
paste
Advertisement
spather

Untitled

spather
Sep 29th, 2023
13
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 14.13 KB | None | 0 0
raw download clone embed print report
  1. commit c13dac8e2974599969b18a4a08b38d08db6ba019
  2. Author: Shyam Pather <shyam.pather@gmail.com>
  3. Date: Thu Sep 21 09:10:00 2023 -0700
  4.  
  5. Run the attention weights experiments on all layers/heads
  6.  
  7. Got through all layers/heads on s_len=7 and through block 0/head 4 on
  8. the s_len=256 run. Not committing the binaries for the s_len=256 run
  9. because the file sizes are too large. Will work on a solution to split
  10. them.
  11.  
  12. diff --git a/nbs/attn_weight_results/block0_head0_slen7_data.pt b/nbs/attn_weight_results/block0_head0_slen7_data.pt
  13. new file mode 100644
  14. index 0000000..aa97a5a
  15. --- /dev/null
  16. b/nbs/attn_weight_results/block0_head0_slen7_data.pt
  17. @@ -0,0 1,3 @@
  18. version https://git-lfs.github.com/spec/v1
  19. oid sha256:d376969de8ffee82408dc71debe1b9fc8700f7b31488800c17f59754a4de488a
  20. size 14421915
  21. diff --git a/nbs/attn_weight_results/block0_head1_slen7_data.pt b/nbs/attn_weight_results/block0_head1_slen7_data.pt
  22. new file mode 100644
  23. index 0000000..a3c03f8
  24. --- /dev/null
  25. b/nbs/attn_weight_results/block0_head1_slen7_data.pt
  26. @@ -0,0 1,3 @@
  27. version https://git-lfs.github.com/spec/v1
  28. oid sha256:42cd74adde989b1d8764ba03fa4d8616b3cc5428fd846079170c5310b00e654f
  29. size 14421915
  30. diff --git a/nbs/attn_weight_results/block0_head2_slen7_data.pt b/nbs/attn_weight_results/block0_head2_slen7_data.pt
  31. new file mode 100644
  32. index 0000000..5250503
  33. --- /dev/null
  34. b/nbs/attn_weight_results/block0_head2_slen7_data.pt
  35. @@ -0,0 1,3 @@
  36. version https://git-lfs.github.com/spec/v1
  37. oid sha256:593d89e7bbf7bfddf2eba0e383c6be549ba06cc0944892cc5985cd36c178a193
  38. size 14421915
  39. diff --git a/nbs/attn_weight_results/block0_head3_slen7_data.pt b/nbs/attn_weight_results/block0_head3_slen7_data.pt
  40. new file mode 100644
  41. index 0000000..d078b41
  42. --- /dev/null
  43. b/nbs/attn_weight_results/block0_head3_slen7_data.pt
  44. @@ -0,0 1,3 @@
  45. version https://git-lfs.github.com/spec/v1
  46. oid sha256:ae446b615c9fb213f71d646f45df0b1c14a57a7c394f7d5b30fef9ca73c63a3b
  47. size 14421915
  48. diff --git a/nbs/attn_weight_results/block0_head4_slen7_data.pt b/nbs/attn_weight_results/block0_head4_slen7_data.pt
  49. new file mode 100644
  50. index 0000000..ed2bb16
  51. --- /dev/null
  52. b/nbs/attn_weight_results/block0_head4_slen7_data.pt
  53. @@ -0,0 1,3 @@
  54. version https://git-lfs.github.com/spec/v1
  55. oid sha256:5b87969f59878e4260de8759d3213c08432fbf36a712a978b837e584bb91bf25
  56. size 14421915
  57. diff --git a/nbs/attn_weight_results/block0_head5_slen7_data.pt b/nbs/attn_weight_results/block0_head5_slen7_data.pt
  58. new file mode 100644
  59. index 0000000..4b6425b
  60. --- /dev/null
  61. b/nbs/attn_weight_results/block0_head5_slen7_data.pt
  62. @@ -0,0 1,3 @@
  63. version https://git-lfs.github.com/spec/v1
  64. oid sha256:5dd6f7b14c913605f1b3112f578e0cffcb36610816f1b5e2b5415c68e8524ff8
  65. size 14421915
  66. diff --git a/nbs/attn_weight_results/block1_head1_slen7_data.pt b/nbs/attn_weight_results/block1_head1_slen7_data.pt
  67. new file mode 100644
  68. index 0000000..e3784b5
  69. --- /dev/null
  70. b/nbs/attn_weight_results/block1_head1_slen7_data.pt
  71. @@ -0,0 1,3 @@
  72. version https://git-lfs.github.com/spec/v1
  73. oid sha256:de933a23375f409ef142871a7bd90ab66ba2349fbe68d3ec658aae52364a002f
  74. size 14421915
  75. diff --git a/nbs/attn_weight_results/block1_head2_slen7_data.pt b/nbs/attn_weight_results/block1_head2_slen7_data.pt
  76. new file mode 100644
  77. index 0000000..a25d04f
  78. --- /dev/null
  79. b/nbs/attn_weight_results/block1_head2_slen7_data.pt
  80. @@ -0,0 1,3 @@
  81. version https://git-lfs.github.com/spec/v1
  82. oid sha256:7e8dcccacbc67ea3cd588482f803e03046b3e27d3433e9906e9d0b80a8856e2e
  83. size 14421915
  84. diff --git a/nbs/attn_weight_results/block1_head3_slen7_data.pt b/nbs/attn_weight_results/block1_head3_slen7_data.pt
  85. new file mode 100644
  86. index 0000000..057a8c1
  87. --- /dev/null
  88. b/nbs/attn_weight_results/block1_head3_slen7_data.pt
  89. @@ -0,0 1,3 @@
  90. version https://git-lfs.github.com/spec/v1
  91. oid sha256:7b43c4aaae0b614ca2ae213c884e54e30e0b613465af6312b70518737dfeccb2
  92. size 14421915
  93. diff --git a/nbs/attn_weight_results/block1_head4_slen7_data.pt b/nbs/attn_weight_results/block1_head4_slen7_data.pt
  94. new file mode 100644
  95. index 0000000..f5db456
  96. --- /dev/null
  97. b/nbs/attn_weight_results/block1_head4_slen7_data.pt
  98. @@ -0,0 1,3 @@
  99. version https://git-lfs.github.com/spec/v1
  100. oid sha256:d154167d9d7303d775ddc22cc9b14e4e9f623bb198b510fab08ef053c6be8b41
  101. size 14421915
  102. diff --git a/nbs/attn_weight_results/block1_head5_slen7_data.pt b/nbs/attn_weight_results/block1_head5_slen7_data.pt
  103. new file mode 100644
  104. index 0000000..bb41687
  105. --- /dev/null
  106. b/nbs/attn_weight_results/block1_head5_slen7_data.pt
  107. @@ -0,0 1,3 @@
  108. version https://git-lfs.github.com/spec/v1
  109. oid sha256:13761fec7a4f9514d171fc89f1a94e9bd10d53ecb2d46c452a08d5badaccf8b8
  110. size 14421915
  111. diff --git a/nbs/attn_weight_results/block2_head0_slen7_data.pt b/nbs/attn_weight_results/block2_head0_slen7_data.pt
  112. new file mode 100644
  113. index 0000000..cd62f6f
  114. --- /dev/null
  115. b/nbs/attn_weight_results/block2_head0_slen7_data.pt
  116. @@ -0,0 1,3 @@
  117. version https://git-lfs.github.com/spec/v1
  118. oid sha256:5b2593e8ba9356d3624b365f4dc223319f8217a9cd4bfb4660937053287b7175
  119. size 14421915
  120. diff --git a/nbs/attn_weight_results/block2_head1_slen7_data.pt b/nbs/attn_weight_results/block2_head1_slen7_data.pt
  121. new file mode 100644
  122. index 0000000..e9881f6
  123. --- /dev/null
  124. b/nbs/attn_weight_results/block2_head1_slen7_data.pt
  125. @@ -0,0 1,3 @@
  126. version https://git-lfs.github.com/spec/v1
  127. oid sha256:2502e2ed12150ccd9b38caa981e1059a07a0a764c932fd7f709717b65775e53a
  128. size 14421915
  129. diff --git a/nbs/attn_weight_results/block2_head2_slen7_data.pt b/nbs/attn_weight_results/block2_head2_slen7_data.pt
  130. new file mode 100644
  131. index 0000000..da804f5
  132. --- /dev/null
  133. b/nbs/attn_weight_results/block2_head2_slen7_data.pt
  134. @@ -0,0 1,3 @@
  135. version https://git-lfs.github.com/spec/v1
  136. oid sha256:665453d09612dc95625c3c988ddda0fe43d4944b4b3184281b61c345d5515c9a
  137. size 14421915
  138. diff --git a/nbs/attn_weight_results/block2_head3_slen7_data.pt b/nbs/attn_weight_results/block2_head3_slen7_data.pt
  139. new file mode 100644
  140. index 0000000..2fa9165
  141. --- /dev/null
  142. b/nbs/attn_weight_results/block2_head3_slen7_data.pt
  143. @@ -0,0 1,3 @@
  144. version https://git-lfs.github.com/spec/v1
  145. oid sha256:8d808f8239cdfe36012728f6a37e6f9d176b3d9d8f7fbdafe90aef28b18235f0
  146. size 14421915
  147. diff --git a/nbs/attn_weight_results/block2_head4_slen7_data.pt b/nbs/attn_weight_results/block2_head4_slen7_data.pt
  148. new file mode 100644
  149. index 0000000..1bf230f
  150. --- /dev/null
  151. b/nbs/attn_weight_results/block2_head4_slen7_data.pt
  152. @@ -0,0 1,3 @@
  153. version https://git-lfs.github.com/spec/v1
  154. oid sha256:51dbf926b11aa97619af02ecd609a0ab4bb7aa21820db2376a64f9d8c4e0dad3
  155. size 14421915
  156. diff --git a/nbs/attn_weight_results/block2_head5_slen7_data.pt b/nbs/attn_weight_results/block2_head5_slen7_data.pt
  157. new file mode 100644
  158. index 0000000..43d0a43
  159. --- /dev/null
  160. b/nbs/attn_weight_results/block2_head5_slen7_data.pt
  161. @@ -0,0 1,3 @@
  162. version https://git-lfs.github.com/spec/v1
  163. oid sha256:4bd0e67b34a96094de85a82a7757a80f7dc60bc40e586501a927ae9188fa6dc1
  164. size 14421915
  165. diff --git a/nbs/attn_weight_results/block3_head0_slen7_data.pt b/nbs/attn_weight_results/block3_head0_slen7_data.pt
  166. new file mode 100644
  167. index 0000000..c0d3478
  168. --- /dev/null
  169. b/nbs/attn_weight_results/block3_head0_slen7_data.pt
  170. @@ -0,0 1,3 @@
  171. version https://git-lfs.github.com/spec/v1
  172. oid sha256:683efc7b4e489caa8f4558cfe6b134dceb93347519ff57508e5544398e66cc0a
  173. size 14421915
  174. diff --git a/nbs/attn_weight_results/block3_head1_slen7_data.pt b/nbs/attn_weight_results/block3_head1_slen7_data.pt
  175. new file mode 100644
  176. index 0000000..b5224d3
  177. --- /dev/null
  178. b/nbs/attn_weight_results/block3_head1_slen7_data.pt
  179. @@ -0,0 1,3 @@
  180. version https://git-lfs.github.com/spec/v1
  181. oid sha256:72a2edc4044c3336285ef6fbd5997cb42c602d63d872eacf9d9da87ee30ad36b
  182. size 14421915
  183. diff --git a/nbs/attn_weight_results/block3_head2_slen7_data.pt b/nbs/attn_weight_results/block3_head2_slen7_data.pt
  184. new file mode 100644
  185. index 0000000..a00dc24
  186. --- /dev/null
  187. b/nbs/attn_weight_results/block3_head2_slen7_data.pt
  188. @@ -0,0 1,3 @@
  189. version https://git-lfs.github.com/spec/v1
  190. oid sha256:dfb0fd929904c7a479ae521a2d9c13c04327801d5db226919d29a1d56a4553f3
  191. size 14421915
  192. diff --git a/nbs/attn_weight_results/block3_head3_slen7_data.pt b/nbs/attn_weight_results/block3_head3_slen7_data.pt
  193. new file mode 100644
  194. index 0000000..aca383d
  195. --- /dev/null
  196. b/nbs/attn_weight_results/block3_head3_slen7_data.pt
  197. @@ -0,0 1,3 @@
  198. version https://git-lfs.github.com/spec/v1
  199. oid sha256:a74da8e297d3e4852f9c1c8605144211598d2bb7ede09be21d958d9441399d90
  200. size 14421915
  201. diff --git a/nbs/attn_weight_results/block3_head4_slen7_data.pt b/nbs/attn_weight_results/block3_head4_slen7_data.pt
  202. new file mode 100644
  203. index 0000000..449a74a
  204. --- /dev/null
  205. b/nbs/attn_weight_results/block3_head4_slen7_data.pt
  206. @@ -0,0 1,3 @@
  207. version https://git-lfs.github.com/spec/v1
  208. oid sha256:0f4ee0cff215c181f3cc87d8dcf61074296a73eefa892ff47258df7f4c067d64
  209. size 14421915
  210. diff --git a/nbs/attn_weight_results/block3_head5_slen7_data.pt b/nbs/attn_weight_results/block3_head5_slen7_data.pt
  211. new file mode 100644
  212. index 0000000..01a0c43
  213. --- /dev/null
  214. b/nbs/attn_weight_results/block3_head5_slen7_data.pt
  215. @@ -0,0 1,3 @@
  216. version https://git-lfs.github.com/spec/v1
  217. oid sha256:a21b98a311b5d9e54b818b714e9dd33639a4a2bd5b9e14aad9d59789d876f3b1
  218. size 14421915
  219. diff --git a/nbs/attn_weight_results/block4_head0_slen7_data.pt b/nbs/attn_weight_results/block4_head0_slen7_data.pt
  220. new file mode 100644
  221. index 0000000..58cead0
  222. --- /dev/null
  223. b/nbs/attn_weight_results/block4_head0_slen7_data.pt
  224. @@ -0,0 1,3 @@
  225. version https://git-lfs.github.com/spec/v1
  226. oid sha256:1c120ae62669f1a13f96a8a252195acab05ad05afda8084845fd9e0a4f73928e
  227. size 14421915
  228. diff --git a/nbs/attn_weight_results/block4_head1_slen7_data.pt b/nbs/attn_weight_results/block4_head1_slen7_data.pt
  229. new file mode 100644
  230. index 0000000..37ccaa7
  231. --- /dev/null
  232. b/nbs/attn_weight_results/block4_head1_slen7_data.pt
  233. @@ -0,0 1,3 @@
  234. version https://git-lfs.github.com/spec/v1
  235. oid sha256:64a38f611153cd0f3a493a27ae8451f7e203db068daec3f20a87df4e01fdf979
  236. size 14421915
  237. diff --git a/nbs/attn_weight_results/block4_head2_slen7_data.pt b/nbs/attn_weight_results/block4_head2_slen7_data.pt
  238. new file mode 100644
  239. index 0000000..8d41b9a
  240. --- /dev/null
  241. b/nbs/attn_weight_results/block4_head2_slen7_data.pt
  242. @@ -0,0 1,3 @@
  243. version https://git-lfs.github.com/spec/v1
  244. oid sha256:8f384435d1307e1da1e06095d8b867e0cf2e4f7ccd66117d77544b592f976ccc
  245. size 14421915
  246. diff --git a/nbs/attn_weight_results/block4_head3_slen7_data.pt b/nbs/attn_weight_results/block4_head3_slen7_data.pt
  247. new file mode 100644
  248. index 0000000..b69666c
  249. --- /dev/null
  250. b/nbs/attn_weight_results/block4_head3_slen7_data.pt
  251. @@ -0,0 1,3 @@
  252. version https://git-lfs.github.com/spec/v1
  253. oid sha256:6753ec74114184ad162e8148cd2dc6d9544770982de7161ecda07f524a8a2b3c
  254. size 14421915
  255. diff --git a/nbs/attn_weight_results/block4_head4_slen7_data.pt b/nbs/attn_weight_results/block4_head4_slen7_data.pt
  256. new file mode 100644
  257. index 0000000..eed6dbd
  258. --- /dev/null
  259. b/nbs/attn_weight_results/block4_head4_slen7_data.pt
  260. @@ -0,0 1,3 @@
  261. version https://git-lfs.github.com/spec/v1
  262. oid sha256:862ab2d3fff33393303ee400769ba4853b27311a58ec0755d6d83a2ea0a6ef03
  263. size 14421915
  264. diff --git a/nbs/attn_weight_results/block4_head5_slen7_data.pt b/nbs/attn_weight_results/block4_head5_slen7_data.pt
  265. new file mode 100644
  266. index 0000000..1fc399d
  267. --- /dev/null
  268. b/nbs/attn_weight_results/block4_head5_slen7_data.pt
  269. @@ -0,0 1,3 @@
  270. version https://git-lfs.github.com/spec/v1
  271. oid sha256:5c67f0625e326c1e40092f1e72ca9eb583db49e005e44da2b542eb742dfd2389
  272. size 14421915
  273. diff --git a/nbs/attn_weight_results/block5_head0_slen7_data.pt b/nbs/attn_weight_results/block5_head0_slen7_data.pt
  274. new file mode 100644
  275. index 0000000..e092b27
  276. --- /dev/null
  277. b/nbs/attn_weight_results/block5_head0_slen7_data.pt
  278. @@ -0,0 1,3 @@
  279. version https://git-lfs.github.com/spec/v1
  280. oid sha256:9ac7d665b1afe59144ff7eba36f93e2f2c3b4469dfe6608ff82d8ccda6db8660
  281. size 14421915
  282. diff --git a/nbs/attn_weight_results/block5_head1_slen7_data.pt b/nbs/attn_weight_results/block5_head1_slen7_data.pt
  283. new file mode 100644
  284. index 0000000..87cbfb8
  285. --- /dev/null
  286. b/nbs/attn_weight_results/block5_head1_slen7_data.pt
  287. @@ -0,0 1,3 @@
  288. version https://git-lfs.github.com/spec/v1
  289. oid sha256:8eb50f50a3581008186a60ef727ede61cb6fbf1cc5bbfd4f073e1340b390226d
  290. size 14421915
  291. diff --git a/nbs/attn_weight_results/block5_head2_slen7_data.pt b/nbs/attn_weight_results/block5_head2_slen7_data.pt
  292. new file mode 100644
  293. index 0000000..8c1fb88
  294. --- /dev/null
  295. b/nbs/attn_weight_results/block5_head2_slen7_data.pt
  296. @@ -0,0 1,3 @@
  297. version https://git-lfs.github.com/spec/v1
  298. oid sha256:bcf735c41f2488b29e5eef58b5d808035726871dca9755be0cde77f0dcccf07e
  299. size 14421915
  300. diff --git a/nbs/attn_weight_results/block5_head3_slen7_data.pt b/nbs/attn_weight_results/block5_head3_slen7_data.pt
  301. new file mode 100644
  302. index 0000000..6aedf10
  303. --- /dev/null
  304. b/nbs/attn_weight_results/block5_head3_slen7_data.pt
  305. @@ -0,0 1,3 @@
  306. version https://git-lfs.github.com/spec/v1
  307. oid sha256:1073af7874219c48a1d70820088d4891baa555366d3336d94d7abe8503714f21
  308. size 14421915
  309. diff --git a/nbs/attn_weight_results/block5_head4_slen7_data.pt b/nbs/attn_weight_results/block5_head4_slen7_data.pt
  310. new file mode 100644
  311. index 0000000..949bb2b
  312. --- /dev/null
  313. b/nbs/attn_weight_results/block5_head4_slen7_data.pt
  314. @@ -0,0 1,3 @@
  315. version https://git-lfs.github.com/spec/v1
  316. oid sha256:1353b908ad38868e5a27ea24d2249b731635bb1f68206f32098e90d829bc841f
  317. size 14421915
  318. diff --git a/nbs/attn_weight_results/block5_head5_slen7_data.pt b/nbs/attn_weight_results/block5_head5_slen7_data.pt
  319. new file mode 100644
  320. index 0000000..0107314
  321. --- /dev/null
  322. b/nbs/attn_weight_results/block5_head5_slen7_data.pt
  323. @@ -0,0 1,3 @@
  324. version https://git-lfs.github.com/spec/v1
  325. oid sha256:5ce804e470b9a57458a5e8975fb3be18c8b70bf57740b18ba4a3b4ac2d27bf7e
  326. size 14421915
  327.  
  328. commit edf5509add7491716d849a113110a114d1e02563
  329. Author: Shyam Pather <shyam.pather@gmail.com>
  330. Date: Wed Sep 20 10:29:24 2023 -0700
  331.  
  332. Make attention weights experiment run over all strings in the validation set
  333.  
  334. diff --git a/nbs/attn_weight_results/block1_head0_slen7_data.pt b/nbs/attn_weight_results/block1_head0_slen7_data.pt
  335. new file mode 100644
  336. index 0000000..d090946
  337. --- /dev/null
  338. b/nbs/attn_weight_results/block1_head0_slen7_data.pt
  339. @@ -0,0 1,3 @@
  340. version https://git-lfs.github.com/spec/v1
  341. oid sha256:275e2400fc3d5af3a33b93619ba01acb73ac125ce385c0d6afa2ff0027a38a0f
  342. size 14421915
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!