API tools faq
paste
Guest User

Krea-2 targeting

a guest
Jun 26th, 2026
19
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 23.35 KB | None | 0 0
raw download clone embed print report
  1. I need to know which of these layers must be excluded for lora training a dit, subject/concept. in substring format without glob.
  2.  
  3. img_in.bias # Linear
  4. img_in.weight # Linear
  5. text_fusion.layerwise_blocks.0.attn.norm_k.weight # Krea2RMSNorm
  6. text_fusion.layerwise_blocks.0.attn.norm_q.weight # Krea2RMSNorm
  7. text_fusion.layerwise_blocks.0.norm1.weight # Krea2RMSNorm
  8. text_fusion.layerwise_blocks.0.norm2.weight # Krea2RMSNorm
  9. text_fusion.layerwise_blocks.1.attn.norm_k.weight # Krea2RMSNorm
  10. text_fusion.layerwise_blocks.1.attn.norm_q.weight # Krea2RMSNorm
  11. text_fusion.layerwise_blocks.1.norm1.weight # Krea2RMSNorm
  12. text_fusion.layerwise_blocks.1.norm2.weight # Krea2RMSNorm
  13. text_fusion.projector.weight # Linear
  14. text_fusion.refiner_blocks.0.attn.norm_k.weight # Krea2RMSNorm
  15. text_fusion.refiner_blocks.0.attn.norm_q.weight # Krea2RMSNorm
  16. text_fusion.refiner_blocks.0.norm1.weight # Krea2RMSNorm
  17. text_fusion.refiner_blocks.0.norm2.weight # Krea2RMSNorm
  18. text_fusion.refiner_blocks.1.attn.norm_k.weight # Krea2RMSNorm
  19. text_fusion.refiner_blocks.1.attn.norm_q.weight # Krea2RMSNorm
  20. text_fusion.refiner_blocks.1.norm1.weight # Krea2RMSNorm
  21. text_fusion.refiner_blocks.1.norm2.weight # Krea2RMSNorm
  22. time_embed.linear_1.bias # Linear
  23. time_embed.linear_1.weight # Linear
  24. time_embed.linear_2.bias # Linear
  25. time_embed.linear_2.weight # Linear
  26. time_mod_proj.bias # Linear
  27. time_mod_proj.weight # Linear
  28. transformer_blocks.0.attn.norm_k.weight # Krea2RMSNorm
  29. transformer_blocks.0.attn.norm_q.weight # Krea2RMSNorm
  30. transformer_blocks.0.norm1.weight # Krea2RMSNorm
  31. transformer_blocks.0.norm2.weight # Krea2RMSNorm
  32. transformer_blocks.0.scale_shift_table # Krea2TransformerBlock
  33. transformer_blocks.1.attn.norm_k.weight # Krea2RMSNorm
  34. transformer_blocks.1.attn.norm_q.weight # Krea2RMSNorm
  35. transformer_blocks.1.norm1.weight # Krea2RMSNorm
  36. transformer_blocks.1.norm2.weight # Krea2RMSNorm
  37. transformer_blocks.1.scale_shift_table # Krea2TransformerBlock
  38. transformer_blocks.2.attn.norm_k.weight # Krea2RMSNorm
  39. transformer_blocks.2.attn.norm_q.weight # Krea2RMSNorm
  40. transformer_blocks.2.norm1.weight # Krea2RMSNorm
  41. transformer_blocks.2.norm2.weight # Krea2RMSNorm
  42. transformer_blocks.2.scale_shift_table # Krea2TransformerBlock
  43. transformer_blocks.3.attn.norm_k.weight # Krea2RMSNorm
  44. transformer_blocks.3.attn.norm_q.weight # Krea2RMSNorm
  45. transformer_blocks.3.norm1.weight # Krea2RMSNorm
  46. transformer_blocks.3.norm2.weight # Krea2RMSNorm
  47. transformer_blocks.3.scale_shift_table # Krea2TransformerBlock
  48. transformer_blocks.4.attn.norm_k.weight # Krea2RMSNorm
  49. transformer_blocks.4.attn.norm_q.weight # Krea2RMSNorm
  50. transformer_blocks.4.norm1.weight # Krea2RMSNorm
  51. transformer_blocks.4.norm2.weight # Krea2RMSNorm
  52. transformer_blocks.4.scale_shift_table # Krea2TransformerBlock
  53. transformer_blocks.5.attn.norm_k.weight # Krea2RMSNorm
  54. transformer_blocks.5.attn.norm_q.weight # Krea2RMSNorm
  55. transformer_blocks.5.norm1.weight # Krea2RMSNorm
  56. transformer_blocks.5.norm2.weight # Krea2RMSNorm
  57. transformer_blocks.5.scale_shift_table # Krea2TransformerBlock
  58. transformer_blocks.6.attn.norm_k.weight # Krea2RMSNorm
  59. transformer_blocks.6.attn.norm_q.weight # Krea2RMSNorm
  60. transformer_blocks.6.norm1.weight # Krea2RMSNorm
  61. transformer_blocks.6.norm2.weight # Krea2RMSNorm
  62. transformer_blocks.6.scale_shift_table # Krea2TransformerBlock
  63. transformer_blocks.7.attn.norm_k.weight # Krea2RMSNorm
  64. transformer_blocks.7.attn.norm_q.weight # Krea2RMSNorm
  65. transformer_blocks.7.norm1.weight # Krea2RMSNorm
  66. transformer_blocks.7.norm2.weight # Krea2RMSNorm
  67. transformer_blocks.7.scale_shift_table # Krea2TransformerBlock
  68. transformer_blocks.8.attn.norm_k.weight # Krea2RMSNorm
  69. transformer_blocks.8.attn.norm_q.weight # Krea2RMSNorm
  70. transformer_blocks.8.norm1.weight # Krea2RMSNorm
  71. transformer_blocks.8.norm2.weight # Krea2RMSNorm
  72. transformer_blocks.8.scale_shift_table # Krea2TransformerBlock
  73. transformer_blocks.9.attn.norm_k.weight # Krea2RMSNorm
  74. transformer_blocks.9.attn.norm_q.weight # Krea2RMSNorm
  75. transformer_blocks.9.norm1.weight # Krea2RMSNorm
  76. transformer_blocks.9.norm2.weight # Krea2RMSNorm
  77. transformer_blocks.9.scale_shift_table # Krea2TransformerBlock
  78. txt_in.linear_1.bias # Linear
  79. txt_in.linear_1.weight # Linear
  80. txt_in.linear_2.bias # Linear
  81. txt_in.linear_2.weight # Linear
  82. txt_in.norm.weight # Krea2RMSNorm
  83. text_fusion.layerwise_blocks.0.attn.to_gate.weight # Linear
  84. text_fusion.layerwise_blocks.0.attn.to_k.weight # Linear
  85. text_fusion.layerwise_blocks.0.attn.to_out.0.weight # Linear
  86. text_fusion.layerwise_blocks.0.attn.to_q.weight # Linear
  87. text_fusion.layerwise_blocks.0.attn.to_v.weight # Linear
  88. text_fusion.layerwise_blocks.0.ff.down.weight # Linear
  89. text_fusion.layerwise_blocks.0.ff.gate.weight # Linear
  90. text_fusion.layerwise_blocks.0.ff.up.weight # Linear
  91. text_fusion.layerwise_blocks.1.attn.to_gate.weight # Linear
  92. text_fusion.layerwise_blocks.1.attn.to_k.weight # Linear
  93. text_fusion.layerwise_blocks.1.attn.to_out.0.weight # Linear
  94. text_fusion.layerwise_blocks.1.attn.to_q.weight # Linear
  95. text_fusion.layerwise_blocks.1.attn.to_v.weight # Linear
  96. text_fusion.layerwise_blocks.1.ff.down.weight # Linear
  97. text_fusion.layerwise_blocks.1.ff.gate.weight # Linear
  98. text_fusion.layerwise_blocks.1.ff.up.weight # Linear
  99. text_fusion.refiner_blocks.0.attn.to_gate.weight # Linear
  100. text_fusion.refiner_blocks.0.attn.to_k.weight # Linear
  101. text_fusion.refiner_blocks.0.attn.to_out.0.weight # Linear
  102. text_fusion.refiner_blocks.0.attn.to_q.weight # Linear
  103. text_fusion.refiner_blocks.0.attn.to_v.weight # Linear
  104. text_fusion.refiner_blocks.0.ff.down.weight # Linear
  105. text_fusion.refiner_blocks.0.ff.gate.weight # Linear
  106. text_fusion.refiner_blocks.0.ff.up.weight # Linear
  107. text_fusion.refiner_blocks.1.attn.to_gate.weight # Linear
  108. text_fusion.refiner_blocks.1.attn.to_k.weight # Linear
  109. text_fusion.refiner_blocks.1.attn.to_out.0.weight # Linear
  110. text_fusion.refiner_blocks.1.attn.to_q.weight # Linear
  111. text_fusion.refiner_blocks.1.attn.to_v.weight # Linear
  112. text_fusion.refiner_blocks.1.ff.down.weight # Linear
  113. text_fusion.refiner_blocks.1.ff.gate.weight # Linear
  114. text_fusion.refiner_blocks.1.ff.up.weight # Linear
  115. transformer_blocks.0.attn.to_gate.weight # Linear
  116. transformer_blocks.0.attn.to_k.weight # Linear
  117. transformer_blocks.0.attn.to_out.0.weight # Linear
  118. transformer_blocks.0.attn.to_q.weight # Linear
  119. transformer_blocks.0.attn.to_v.weight # Linear
  120. transformer_blocks.0.ff.down.weight # Linear
  121. transformer_blocks.0.ff.gate.weight # Linear
  122. transformer_blocks.0.ff.up.weight # Linear
  123. transformer_blocks.1.attn.to_gate.weight # Linear
  124. transformer_blocks.1.attn.to_k.weight # Linear
  125. transformer_blocks.1.attn.to_out.0.weight # Linear
  126. transformer_blocks.1.attn.to_q.weight # Linear
  127. transformer_blocks.1.attn.to_v.weight # Linear
  128. transformer_blocks.1.ff.down.weight # Linear
  129. transformer_blocks.1.ff.gate.weight # Linear
  130. transformer_blocks.1.ff.up.weight # Linear
  131. transformer_blocks.2.attn.to_gate.weight # Linear
  132. transformer_blocks.2.attn.to_k.weight # Linear
  133. transformer_blocks.2.attn.to_out.0.weight # Linear
  134. transformer_blocks.2.attn.to_q.weight # Linear
  135. transformer_blocks.2.attn.to_v.weight # Linear
  136. transformer_blocks.2.ff.down.weight # Linear
  137. transformer_blocks.2.ff.gate.weight # Linear
  138. transformer_blocks.2.ff.up.weight # Linear
  139. transformer_blocks.3.attn.to_gate.weight # Linear
  140. transformer_blocks.3.attn.to_k.weight # Linear
  141. transformer_blocks.3.attn.to_out.0.weight # Linear
  142. transformer_blocks.3.attn.to_q.weight # Linear
  143. transformer_blocks.3.attn.to_v.weight # Linear
  144. transformer_blocks.3.ff.down.weight # Linear
  145. transformer_blocks.3.ff.gate.weight # Linear
  146. transformer_blocks.3.ff.up.weight # Linear
  147. transformer_blocks.4.attn.to_gate.weight # Linear
  148. transformer_blocks.4.attn.to_k.weight # Linear
  149. transformer_blocks.4.attn.to_out.0.weight # Linear
  150. transformer_blocks.4.attn.to_q.weight # Linear
  151. transformer_blocks.4.attn.to_v.weight # Linear
  152. transformer_blocks.4.ff.down.weight # Linear
  153. transformer_blocks.4.ff.gate.weight # Linear
  154. transformer_blocks.4.ff.up.weight # Linear
  155. transformer_blocks.5.attn.to_gate.weight # Linear
  156. transformer_blocks.5.attn.to_k.weight # Linear
  157. transformer_blocks.5.attn.to_out.0.weight # Linear
  158. transformer_blocks.5.attn.to_q.weight # Linear
  159. transformer_blocks.5.attn.to_v.weight # Linear
  160. transformer_blocks.5.ff.down.weight # Linear
  161. transformer_blocks.5.ff.gate.weight # Linear
  162. transformer_blocks.5.ff.up.weight # Linear
  163. transformer_blocks.6.attn.to_gate.weight # Linear
  164. transformer_blocks.6.attn.to_k.weight # Linear
  165. transformer_blocks.6.attn.to_out.0.weight # Linear
  166. transformer_blocks.6.attn.to_q.weight # Linear
  167. transformer_blocks.6.attn.to_v.weight # Linear
  168. transformer_blocks.6.ff.down.weight # Linear
  169. transformer_blocks.6.ff.gate.weight # Linear
  170. transformer_blocks.6.ff.up.weight # Linear
  171. transformer_blocks.7.attn.to_gate.weight # Linear
  172. transformer_blocks.7.attn.to_k.weight # Linear
  173. transformer_blocks.7.attn.to_out.0.weight # Linear
  174. transformer_blocks.7.attn.to_q.weight # Linear
  175. transformer_blocks.7.attn.to_v.weight # Linear
  176. transformer_blocks.7.ff.down.weight # Linear
  177. transformer_blocks.7.ff.gate.weight # Linear
  178. transformer_blocks.7.ff.up.weight # Linear
  179. transformer_blocks.8.attn.to_gate.weight # Linear
  180. transformer_blocks.8.attn.to_k.weight # Linear
  181. transformer_blocks.8.attn.to_out.0.weight # Linear
  182. transformer_blocks.8.attn.to_q.weight # Linear
  183. transformer_blocks.8.attn.to_v.weight # Linear
  184. transformer_blocks.8.ff.down.weight # Linear
  185. transformer_blocks.8.ff.gate.weight # Linear
  186. transformer_blocks.8.ff.up.weight # Linear
  187. transformer_blocks.9.attn.to_gate.weight # Linear
  188. transformer_blocks.9.attn.to_k.weight # Linear
  189. transformer_blocks.9.attn.to_q.weight # Linear
  190. transformer_blocks.9.attn.to_v.weight # Linear
  191. transformer_blocks.10.attn.norm_k.weight # Krea2RMSNorm
  192. transformer_blocks.10.attn.norm_q.weight # Krea2RMSNorm
  193. transformer_blocks.10.norm1.weight # Krea2RMSNorm
  194. transformer_blocks.10.norm2.weight # Krea2RMSNorm
  195. transformer_blocks.10.scale_shift_table # Krea2TransformerBlock
  196. transformer_blocks.11.attn.norm_k.weight # Krea2RMSNorm
  197. transformer_blocks.11.attn.norm_q.weight # Krea2RMSNorm
  198. transformer_blocks.11.norm1.weight # Krea2RMSNorm
  199. transformer_blocks.11.norm2.weight # Krea2RMSNorm
  200. transformer_blocks.11.scale_shift_table # Krea2TransformerBlock
  201. transformer_blocks.12.attn.norm_k.weight # Krea2RMSNorm
  202. transformer_blocks.12.attn.norm_q.weight # Krea2RMSNorm
  203. transformer_blocks.12.norm1.weight # Krea2RMSNorm
  204. transformer_blocks.12.norm2.weight # Krea2RMSNorm
  205. transformer_blocks.12.scale_shift_table # Krea2TransformerBlock
  206. transformer_blocks.13.attn.norm_k.weight # Krea2RMSNorm
  207. transformer_blocks.13.attn.norm_q.weight # Krea2RMSNorm
  208. transformer_blocks.13.norm1.weight # Krea2RMSNorm
  209. transformer_blocks.13.norm2.weight # Krea2RMSNorm
  210. transformer_blocks.13.scale_shift_table # Krea2TransformerBlock
  211. transformer_blocks.14.attn.norm_k.weight # Krea2RMSNorm
  212. transformer_blocks.14.attn.norm_q.weight # Krea2RMSNorm
  213. transformer_blocks.14.norm1.weight # Krea2RMSNorm
  214. transformer_blocks.14.norm2.weight # Krea2RMSNorm
  215. transformer_blocks.14.scale_shift_table # Krea2TransformerBlock
  216. transformer_blocks.15.attn.norm_k.weight # Krea2RMSNorm
  217. transformer_blocks.15.attn.norm_q.weight # Krea2RMSNorm
  218. transformer_blocks.15.norm1.weight # Krea2RMSNorm
  219. transformer_blocks.15.norm2.weight # Krea2RMSNorm
  220. transformer_blocks.15.scale_shift_table # Krea2TransformerBlock
  221. transformer_blocks.16.attn.norm_k.weight # Krea2RMSNorm
  222. transformer_blocks.16.attn.norm_q.weight # Krea2RMSNorm
  223. transformer_blocks.16.norm1.weight # Krea2RMSNorm
  224. transformer_blocks.16.norm2.weight # Krea2RMSNorm
  225. transformer_blocks.16.scale_shift_table # Krea2TransformerBlock
  226. transformer_blocks.17.attn.norm_k.weight # Krea2RMSNorm
  227. transformer_blocks.17.attn.norm_q.weight # Krea2RMSNorm
  228. transformer_blocks.17.norm1.weight # Krea2RMSNorm
  229. transformer_blocks.17.norm2.weight # Krea2RMSNorm
  230. transformer_blocks.17.scale_shift_table # Krea2TransformerBlock
  231. transformer_blocks.18.attn.norm_k.weight # Krea2RMSNorm
  232. transformer_blocks.18.attn.norm_q.weight # Krea2RMSNorm
  233. transformer_blocks.18.norm1.weight # Krea2RMSNorm
  234. transformer_blocks.18.norm2.weight # Krea2RMSNorm
  235. transformer_blocks.18.scale_shift_table # Krea2TransformerBlock
  236. transformer_blocks.19.attn.norm_k.weight # Krea2RMSNorm
  237. transformer_blocks.19.attn.norm_q.weight # Krea2RMSNorm
  238. transformer_blocks.19.norm1.weight # Krea2RMSNorm
  239. transformer_blocks.19.norm2.weight # Krea2RMSNorm
  240. transformer_blocks.19.scale_shift_table # Krea2TransformerBlock
  241. transformer_blocks.20.attn.norm_k.weight # Krea2RMSNorm
  242. transformer_blocks.20.attn.norm_q.weight # Krea2RMSNorm
  243. transformer_blocks.20.norm1.weight # Krea2RMSNorm
  244. transformer_blocks.20.norm2.weight # Krea2RMSNorm
  245. transformer_blocks.20.scale_shift_table # Krea2TransformerBlock
  246. transformer_blocks.10.attn.to_gate.weight # Linear
  247. transformer_blocks.10.attn.to_k.weight # Linear
  248. transformer_blocks.10.attn.to_out.0.weight # Linear
  249. transformer_blocks.10.attn.to_q.weight # Linear
  250. transformer_blocks.10.attn.to_v.weight # Linear
  251. transformer_blocks.10.ff.down.weight # Linear
  252. transformer_blocks.10.ff.gate.weight # Linear
  253. transformer_blocks.10.ff.up.weight # Linear
  254. transformer_blocks.11.attn.to_gate.weight # Linear
  255. transformer_blocks.11.attn.to_k.weight # Linear
  256. transformer_blocks.11.attn.to_out.0.weight # Linear
  257. transformer_blocks.11.attn.to_q.weight # Linear
  258. transformer_blocks.11.attn.to_v.weight # Linear
  259. transformer_blocks.11.ff.down.weight # Linear
  260. transformer_blocks.11.ff.gate.weight # Linear
  261. transformer_blocks.11.ff.up.weight # Linear
  262. transformer_blocks.12.attn.to_gate.weight # Linear
  263. transformer_blocks.12.attn.to_k.weight # Linear
  264. transformer_blocks.12.attn.to_out.0.weight # Linear
  265. transformer_blocks.12.attn.to_q.weight # Linear
  266. transformer_blocks.12.attn.to_v.weight # Linear
  267. transformer_blocks.12.ff.down.weight # Linear
  268. transformer_blocks.12.ff.gate.weight # Linear
  269. transformer_blocks.12.ff.up.weight # Linear
  270. transformer_blocks.13.attn.to_gate.weight # Linear
  271. transformer_blocks.13.attn.to_k.weight # Linear
  272. transformer_blocks.13.attn.to_out.0.weight # Linear
  273. transformer_blocks.13.attn.to_q.weight # Linear
  274. transformer_blocks.13.attn.to_v.weight # Linear
  275. transformer_blocks.13.ff.down.weight # Linear
  276. transformer_blocks.13.ff.gate.weight # Linear
  277. transformer_blocks.13.ff.up.weight # Linear
  278. transformer_blocks.14.attn.to_gate.weight # Linear
  279. transformer_blocks.14.attn.to_k.weight # Linear
  280. transformer_blocks.14.attn.to_out.0.weight # Linear
  281. transformer_blocks.14.attn.to_q.weight # Linear
  282. transformer_blocks.14.attn.to_v.weight # Linear
  283. transformer_blocks.14.ff.down.weight # Linear
  284. transformer_blocks.14.ff.gate.weight # Linear
  285. transformer_blocks.14.ff.up.weight # Linear
  286. transformer_blocks.15.attn.to_gate.weight # Linear
  287. transformer_blocks.15.attn.to_k.weight # Linear
  288. transformer_blocks.15.attn.to_out.0.weight # Linear
  289. transformer_blocks.15.attn.to_q.weight # Linear
  290. transformer_blocks.15.attn.to_v.weight # Linear
  291. transformer_blocks.15.ff.down.weight # Linear
  292. transformer_blocks.15.ff.gate.weight # Linear
  293. transformer_blocks.15.ff.up.weight # Linear
  294. transformer_blocks.16.attn.to_gate.weight # Linear
  295. transformer_blocks.16.attn.to_k.weight # Linear
  296. transformer_blocks.16.attn.to_out.0.weight # Linear
  297. transformer_blocks.16.attn.to_q.weight # Linear
  298. transformer_blocks.16.attn.to_v.weight # Linear
  299. transformer_blocks.16.ff.down.weight # Linear
  300. transformer_blocks.16.ff.gate.weight # Linear
  301. transformer_blocks.16.ff.up.weight # Linear
  302. transformer_blocks.17.attn.to_gate.weight # Linear
  303. transformer_blocks.17.attn.to_k.weight # Linear
  304. transformer_blocks.17.attn.to_out.0.weight # Linear
  305. transformer_blocks.17.attn.to_q.weight # Linear
  306. transformer_blocks.17.attn.to_v.weight # Linear
  307. transformer_blocks.17.ff.down.weight # Linear
  308. transformer_blocks.17.ff.gate.weight # Linear
  309. transformer_blocks.17.ff.up.weight # Linear
  310. transformer_blocks.18.attn.to_gate.weight # Linear
  311. transformer_blocks.18.attn.to_k.weight # Linear
  312. transformer_blocks.18.attn.to_out.0.weight # Linear
  313. transformer_blocks.18.attn.to_q.weight # Linear
  314. transformer_blocks.18.attn.to_v.weight # Linear
  315. transformer_blocks.18.ff.down.weight # Linear
  316. transformer_blocks.18.ff.gate.weight # Linear
  317. transformer_blocks.18.ff.up.weight # Linear
  318. transformer_blocks.19.attn.to_gate.weight # Linear
  319. transformer_blocks.19.attn.to_k.weight # Linear
  320. transformer_blocks.19.attn.to_out.0.weight # Linear
  321. transformer_blocks.19.attn.to_q.weight # Linear
  322. transformer_blocks.19.attn.to_v.weight # Linear
  323. transformer_blocks.19.ff.down.weight # Linear
  324. transformer_blocks.19.ff.gate.weight # Linear
  325. transformer_blocks.19.ff.up.weight # Linear
  326. transformer_blocks.20.attn.to_gate.weight # Linear
  327. transformer_blocks.20.attn.to_k.weight # Linear
  328. transformer_blocks.20.attn.to_out.0.weight # Linear
  329. transformer_blocks.20.attn.to_q.weight # Linear
  330. transformer_blocks.20.attn.to_v.weight # Linear
  331. transformer_blocks.20.ff.gate.weight # Linear
  332. transformer_blocks.9.attn.to_out.0.weight # Linear
  333. transformer_blocks.9.ff.down.weight # Linear
  334. transformer_blocks.9.ff.gate.weight # Linear
  335. transformer_blocks.9.ff.up.weight # Linear
  336. final_layer.linear.bias # Linear
  337. final_layer.linear.weight # Linear
  338. final_layer.norm.weight # Krea2RMSNorm
  339. final_layer.scale_shift_table # Krea2FinalLayer
  340. transformer_blocks.21.attn.norm_k.weight # Krea2RMSNorm
  341. transformer_blocks.21.attn.norm_q.weight # Krea2RMSNorm
  342. transformer_blocks.21.norm1.weight # Krea2RMSNorm
  343. transformer_blocks.21.norm2.weight # Krea2RMSNorm
  344. transformer_blocks.21.scale_shift_table # Krea2TransformerBlock
  345. transformer_blocks.22.attn.norm_k.weight # Krea2RMSNorm
  346. transformer_blocks.22.attn.norm_q.weight # Krea2RMSNorm
  347. transformer_blocks.22.norm1.weight # Krea2RMSNorm
  348. transformer_blocks.22.norm2.weight # Krea2RMSNorm
  349. transformer_blocks.22.scale_shift_table # Krea2TransformerBlock
  350. transformer_blocks.23.attn.norm_k.weight # Krea2RMSNorm
  351. transformer_blocks.23.attn.norm_q.weight # Krea2RMSNorm
  352. transformer_blocks.23.norm1.weight # Krea2RMSNorm
  353. transformer_blocks.23.norm2.weight # Krea2RMSNorm
  354. transformer_blocks.23.scale_shift_table # Krea2TransformerBlock
  355. transformer_blocks.24.attn.norm_k.weight # Krea2RMSNorm
  356. transformer_blocks.24.attn.norm_q.weight # Krea2RMSNorm
  357. transformer_blocks.24.norm1.weight # Krea2RMSNorm
  358. transformer_blocks.24.norm2.weight # Krea2RMSNorm
  359. transformer_blocks.24.scale_shift_table # Krea2TransformerBlock
  360. transformer_blocks.25.attn.norm_k.weight # Krea2RMSNorm
  361. transformer_blocks.25.attn.norm_q.weight # Krea2RMSNorm
  362. transformer_blocks.25.norm1.weight # Krea2RMSNorm
  363. transformer_blocks.25.norm2.weight # Krea2RMSNorm
  364. transformer_blocks.25.scale_shift_table # Krea2TransformerBlock
  365. transformer_blocks.26.attn.norm_k.weight # Krea2RMSNorm
  366. transformer_blocks.26.attn.norm_q.weight # Krea2RMSNorm
  367. transformer_blocks.26.norm1.weight # Krea2RMSNorm
  368. transformer_blocks.26.norm2.weight # Krea2RMSNorm
  369. transformer_blocks.26.scale_shift_table # Krea2TransformerBlock
  370. transformer_blocks.27.attn.norm_k.weight # Krea2RMSNorm
  371. transformer_blocks.27.attn.norm_q.weight # Krea2RMSNorm
  372. transformer_blocks.27.norm1.weight # Krea2RMSNorm
  373. transformer_blocks.27.norm2.weight # Krea2RMSNorm
  374. transformer_blocks.27.scale_shift_table # Krea2TransformerBlock
  375. transformer_blocks.20.ff.down.weight # Linear
  376. transformer_blocks.20.ff.up.weight # Linear
  377. transformer_blocks.21.attn.to_gate.weight # Linear
  378. transformer_blocks.21.attn.to_k.weight # Linear
  379. transformer_blocks.21.attn.to_out.0.weight # Linear
  380. transformer_blocks.21.attn.to_q.weight # Linear
  381. transformer_blocks.21.attn.to_v.weight # Linear
  382. transformer_blocks.21.ff.down.weight # Linear
  383. transformer_blocks.21.ff.gate.weight # Linear
  384. transformer_blocks.21.ff.up.weight # Linear
  385. transformer_blocks.22.attn.to_gate.weight # Linear
  386. transformer_blocks.22.attn.to_k.weight # Linear
  387. transformer_blocks.22.attn.to_out.0.weight # Linear
  388. transformer_blocks.22.attn.to_q.weight # Linear
  389. transformer_blocks.22.attn.to_v.weight # Linear
  390. transformer_blocks.22.ff.down.weight # Linear
  391. transformer_blocks.22.ff.gate.weight # Linear
  392. transformer_blocks.22.ff.up.weight # Linear
  393. transformer_blocks.23.attn.to_gate.weight # Linear
  394. transformer_blocks.23.attn.to_k.weight # Linear
  395. transformer_blocks.23.attn.to_out.0.weight # Linear
  396. transformer_blocks.23.attn.to_q.weight # Linear
  397. transformer_blocks.23.attn.to_v.weight # Linear
  398. transformer_blocks.23.ff.down.weight # Linear
  399. transformer_blocks.23.ff.gate.weight # Linear
  400. transformer_blocks.23.ff.up.weight # Linear
  401. transformer_blocks.24.attn.to_gate.weight # Linear
  402. transformer_blocks.24.attn.to_k.weight # Linear
  403. transformer_blocks.24.attn.to_out.0.weight # Linear
  404. transformer_blocks.24.attn.to_q.weight # Linear
  405. transformer_blocks.24.attn.to_v.weight # Linear
  406. transformer_blocks.24.ff.down.weight # Linear
  407. transformer_blocks.24.ff.gate.weight # Linear
  408. transformer_blocks.24.ff.up.weight # Linear
  409. transformer_blocks.25.attn.to_gate.weight # Linear
  410. transformer_blocks.25.attn.to_k.weight # Linear
  411. transformer_blocks.25.attn.to_out.0.weight # Linear
  412. transformer_blocks.25.attn.to_q.weight # Linear
  413. transformer_blocks.25.attn.to_v.weight # Linear
  414. transformer_blocks.25.ff.down.weight # Linear
  415. transformer_blocks.25.ff.gate.weight # Linear
  416. transformer_blocks.25.ff.up.weight # Linear
  417. transformer_blocks.26.attn.to_gate.weight # Linear
  418. transformer_blocks.26.attn.to_k.weight # Linear
  419. transformer_blocks.26.attn.to_out.0.weight # Linear
  420. transformer_blocks.26.attn.to_q.weight # Linear
  421. transformer_blocks.26.attn.to_v.weight # Linear
  422. transformer_blocks.26.ff.down.weight # Linear
  423. transformer_blocks.26.ff.gate.weight # Linear
  424. transformer_blocks.26.ff.up.weight # Linear
  425. transformer_blocks.27.attn.to_gate.weight # Linear
  426. transformer_blocks.27.attn.to_k.weight # Linear
  427. transformer_blocks.27.attn.to_out.0.weight # Linear
  428. transformer_blocks.27.attn.to_q.weight # Linear
  429. transformer_blocks.27.attn.to_v.weight # Linear
  430. transformer_blocks.27.ff.down.weight # Linear
  431. transformer_blocks.27.ff.gate.weight # Linear
  432. transformer_blocks.27.ff.up.weight # Linear
  433.  
  434.  
  435.  
  436. output:
  437.  
  438. For DiT (Diffusion Transformer) LoRA training on subject/concept, you want to train the attention and feedforward layers while excluding input/output projections, time embeddings, normalization layers, and modulation parameters.
  439.  
  440. Here are the substrings to exclude:
  441.  
  442. ```
  443. img_in
  444. txt_in
  445. time_embed
  446. time_mod_proj
  447. final_layer
  448. norm
  449. scale_shift_table
  450. text_fusion
  451. ```
  452.  
  453. **What this excludes:**
  454. - **img_in** - Initial image patch embedding
  455. - **txt_in** - Text projection layers
  456. - **time_embed** & **time_mod_proj** - Timestep conditioning (preserve diffusion timesteps)
  457. - **final_layer** - Output projection
  458. - **norm** - All RMSNorm layers (norm1, norm2, norm_k, norm_q, txt_in.norm, final_layer.norm)
  459. - **scale_shift_table** - AdaLN modulation parameters in transformer blocks
  460. - **text_fusion** - Text fusion/cross-attention blocks (preserve text understanding for concept training)
  461.  
  462. **What remains trainable:**
  463. - Attention weights: `to_q`, `to_k`, `to_v`, `to_out`, `to_gate`
  464. - Feedforward weights: `ff.up`, `ff.down`, `ff.gate`
  465.  
  466. This configuration focuses the LoRA on learning the visual concept through the transformer blocks while preserving the model's text understanding and diffusion dynamics.
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!