API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Jun 4th, 2025
22
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 136.72 KB | None | 0 0
raw download clone embed print report
  1. [2025-06-05 02:46:34,208] [INFO] [real_accelerator.py:239:get_accelerator] Setting ds_accelerator to cuda (auto detect)
  2. /fsx/sayak/transformers/src/transformers/generation/configuration_utils.py:823: UserWarning: `return_dict_in_generate` is NOT set to `True`, but `output_attentions` is. When `return_dict_in_generate` is not `True`, `output_attentions` is ignored.
  3. warnings.warn(
  4. /fsx/sayak/transformers/src/transformers/generation/configuration_utils.py:823: UserWarning: `return_dict_in_generate` is NOT set to `True`, but `output_hidden_states` is. When `return_dict_in_generate` is not `True`, `output_hidden_states` is ignored.
  5. warnings.warn(
  6.  
  7. Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
  8. Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 84.51it/s]
  9.  
  10. Loading pipeline components...: 0%| | 0/11 [00:00<?, ?it/s]
  11. Loading pipeline components...: 9%|▉ | 1/11 [00:00<00:01, 7.69it/s]
  12. Loading pipeline components...: 45%|████▌ | 5/11 [00:00<00:00, 15.74it/s]
  13.  
  14. Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
  15. Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 51.86it/s]
  16.  
  17. Loading pipeline components...: 64%|██████▎ | 7/11 [00:00<00:00, 14.80it/s]
  18.  
  19. Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s]
  20.  
  21. Loading checkpoint shards: 71%|███████▏ | 5/7 [00:00<00:00, 48.54it/s]
  22. Loading checkpoint shards: 100%|██████████| 7/7 [00:00<00:00, 49.88it/s]
  23.  
  24. Loading pipeline components...: 82%|████████▏ | 9/11 [00:00<00:00, 10.74it/s]
  25. Loading pipeline components...: 100%|██████████| 11/11 [00:01<00:00, 7.43it/s]
  26. Loading pipeline components...: 100%|██████████| 11/11 [00:01<00:00, 9.16it/s]
  27. Loading default_0 was unsuccessful with the following error:
  28. Target modules {'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.attn1.to_q', 'transformer.double_stream_blocks.8.block.ff_t.w3', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.0.block.attn1.to_k', 'transformer.caption_projection.13.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_t.w1', 'transformer.double_stream_blocks.7.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.20.block.attn1.to_k', 'transformer.caption_projection.8.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.13.block.attn1.to_k', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.attn1.to_out', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.3.block.ff_t.w1', 'transformer.double_stream_blocks.11.block.attn1.to_v', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.14.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.attn1.to_q_t', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.29.block.adaLN_modulation.1', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w3', 'transformer.caption_projection.43.linear', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.2.block.attn1.to_k', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.4.block.attn1.to_q', 'transformer.p_embedder.pooled_embedder.linear_1', 'transformer.caption_projection.19.linear', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w1', 'transformer.caption_projection.26.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.6.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.adaLN_modulation.1', 'transformer.single_stream_blocks.1.block.attn1.to_v', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.0.block.attn1.to_v_t', 'transformer.double_stream_blocks.0.block.attn1.to_v', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.11.block.attn1.to_k', 'transformer.double_stream_blocks.13.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.attn1.to_k_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.2.block.adaLN_modulation.1', 'transformer.double_stream_blocks.6.block.attn1.to_v', 'transformer.single_stream_blocks.11.block.adaLN_modulation.1', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.27.block.attn1.to_out', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.1.block.attn1.to_out', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.3.block.adaLN_modulation.1', 'transformer.double_stream_blocks.8.block.attn1.to_k', 'transformer.double_stream_blocks.8.block.attn1.to_q_t', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.4.block.attn1.to_out', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.3.block.ff_t.w2', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.15.block.adaLN_modulation.1', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.1.block.attn1.to_v_t', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w3', 'transformer.caption_projection.48.linear', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.attn1.to_out_t', 'transformer.single_stream_blocks.16.block.attn1.to_v', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.17.block.attn1.to_k', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.8.block.attn1.to_k', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w1', 'transformer.caption_projection.11.linear', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.0.block.attn1.to_v', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.6.block.attn1.to_q_t', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.0.block.ff_t.w2', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.attn1.to_k', 'transformer.single_stream_blocks.31.block.attn1.to_v', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.28.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.adaLN_modulation.1', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.27.block.attn1.to_k', 'transformer.single_stream_blocks.1.block.adaLN_modulation.1', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.11.block.attn1.to_k', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.25.block.adaLN_modulation.1', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.attn1.to_k_t', 'transformer.single_stream_blocks.27.block.attn1.to_q', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.14.block.adaLN_modulation.1', 'transformer.double_stream_blocks.12.block.attn1.to_v_t', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.0.block.ff_t.w1', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w2', 'transformer.caption_projection.42.linear', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.ff_t.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.7.block.ff_t.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.0.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.attn1.to_v_t', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.19.block.attn1.to_out', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.3.block.attn1.to_k', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.13.block.attn1.to_q_t', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.6.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.attn1.to_q_t', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.attn1.to_out', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.1.block.attn1.to_out', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w3', 'transformer.caption_projection.29.linear', 'transformer.double_stream_blocks.1.block.ff_t.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.17.block.attn1.to_out', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.attn1.to_v_t', 'transformer.caption_projection.9.linear', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.attn1.to_q', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.11.block.ff_t.w3', 'transformer.single_stream_blocks.4.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.adaLN_modulation.1', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.9.block.ff_t.w3', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.31.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.adaLN_modulation.1', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.27.block.adaLN_modulation.1', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.12.block.ff_t.w2', 'transformer.single_stream_blocks.19.block.attn1.to_v', 'transformer.double_stream_blocks.6.block.ff_t.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.3.block.attn1.to_out', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.15.block.ff_t.w2', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.2.block.attn1.to_q', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.6.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.23.block.adaLN_modulation.1', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.8.block.attn1.to_k_t', 'transformer.double_stream_blocks.15.block.attn1.to_k_t', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.attn1.to_out_t', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w2', 'transformer.caption_projection.47.linear', 'transformer.caption_projection.14.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w1', 'transformer.caption_projection.28.linear', 'transformer.single_stream_blocks.12.block.attn1.to_v', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.attn1.to_out_t', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w3', 'transformer.caption_projection.15.linear', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.26.block.adaLN_modulation.1', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.30.block.attn1.to_v', 'transformer.single_stream_blocks.28.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.attn1.to_q', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.attn1.to_q_t', 'transformer.caption_projection.44.linear', 'transformer.caption_projection.45.linear', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.16.block.attn1.to_k', 'transformer.single_stream_blocks.17.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.attn1.to_out', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.attn1.to_v', 'transformer.x_embedder.proj', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w2', 'transformer.t_embedder.timestep_embedder.linear_1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.attn1.to_v', 'transformer.caption_projection.41.linear', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.7.block.attn1.to_k_t', 'transformer.single_stream_blocks.10.block.adaLN_modulation.1', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.28.block.attn1.to_v', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.attn1.to_out', 'transformer.double_stream_blocks.10.block.attn1.to_v', 'transformer.single_stream_blocks.20.block.adaLN_modulation.1', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.21.block.adaLN_modulation.1', 'transformer.double_stream_blocks.12.block.attn1.to_out_t', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.2.block.attn1.to_out', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.3.block.attn1.to_q_t', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w3', 'transformer.final_layer.linear', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.7.block.adaLN_modulation.1', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.15.block.attn1.to_k', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.10.block.ff_t.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w2', 'transformer.caption_projection.2.linear', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.1.block.attn1.to_k', 'transformer.single_stream_blocks.12.block.attn1.to_out', 'transformer.single_stream_blocks.16.block.attn1.to_out', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.ff_t.w2', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w3', 'transformer.caption_projection.1.linear', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w3', 'transformer.caption_projection.24.linear', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.0.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.attn1.to_q', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.3.block.attn1.to_k', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_v_t', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.14.block.attn1.to_v_t', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.7.block.ff_t.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w3', 'transformer.caption_projection.6.linear', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.3.block.attn1.to_v_t', 'transformer.single_stream_blocks.3.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_t.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.2.block.attn1.to_q', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.attn1.to_q', 'transformer.single_stream_blocks.18.block.attn1.to_v', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.attn1.to_v', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w1', 'transformer.caption_projection.17.linear', 'transformer.double_stream_blocks.10.block.ff_t.w3', 'transformer.double_stream_blocks.4.block.attn1.to_out', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.2.block.attn1.to_out_t', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.22.block.adaLN_modulation.1', 'transformer.double_stream_blocks.6.block.ff_t.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.28.block.adaLN_modulation.1', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.14.block.attn1.to_out_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w2', 'transformer.caption_projection.3.linear', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.13.block.ff_t.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.1.block.attn1.to_k', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.1.block.attn1.to_out_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.9.block.attn1.to_k', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.25.block.attn1.to_k', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.2.block.attn1.to_q_t', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.4.block.attn1.to_k_t', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.0.block.attn1.to_q', 'transformer.single_stream_blocks.17.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.attn1.to_k', 'transformer.single_stream_blocks.20.block.attn1.to_q', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.ff_t.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.20.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.0.block.attn1.to_k', 'transformer.caption_projection.36.linear', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w2', 'transformer.caption_projection.20.linear', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.24.block.attn1.to_k', 'transformer.single_stream_blocks.24.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.25.block.attn1.to_v', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_v', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.7.block.attn1.to_q', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.17.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.attn1.to_q_t', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.15.block.attn1.to_q_t', 'transformer.double_stream_blocks.13.block.attn1.to_out_t', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.20.block.attn1.to_out', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.13.block.ff_t.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.ff_t.w2', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.10.block.attn1.to_v', 'transformer.single_stream_blocks.13.block.attn1.to_q', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w2', 'transformer.p_embedder.pooled_embedder.linear_2', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.attn1.to_out', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_out_t', 'transformer.double_stream_blocks.2.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.attn1.to_k_t', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w1', 'transformer.caption_projection.40.linear', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_v', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w3', 'transformer.t_embedder.timestep_embedder.linear_2', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.14.block.ff_t.w1', 'transformer.double_stream_blocks.9.block.attn1.to_k_t', 'transformer.double_stream_blocks.8.block.attn1.to_q', 'transformer.double_stream_blocks.0.block.attn1.to_k_t', 'transformer.double_stream_blocks.7.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.11.block.attn1.to_out', 'transformer.double_stream_blocks.11.block.ff_t.w1', 'transformer.double_stream_blocks.4.block.attn1.to_q_t', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.14.block.attn1.to_v', 'transformer.double_stream_blocks.13.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.attn1.to_k', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.4.block.ff_t.w3', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.8.block.attn1.to_v', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.11.block.attn1.to_v', 'transformer.single_stream_blocks.29.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w3', 'transformer.caption_projection.5.linear', 'transformer.double_stream_blocks.2.block.attn1.to_v_t', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.attn1.to_v_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w2', 'transformer.caption_projection.7.linear', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.7.block.ff_t.w3', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w1', 'transformer.caption_projection.39.linear', 'transformer.double_stream_blocks.1.block.ff_t.w1', 'transformer.single_stream_blocks.23.block.attn1.to_k', 'transformer.single_stream_blocks.9.block.attn1.to_q', 'transformer.double_stream_blocks.8.block.attn1.to_out', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.7.block.attn1.to_q_t', 'transformer.single_stream_blocks.19.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.5.block.ff_t.w1', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.1.block.attn1.to_v', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.0.block.attn1.to_q_t', 'transformer.single_stream_blocks.22.block.attn1.to_k', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.3.block.attn1.to_v', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.adaLN_modulation.1', 'transformer.double_stream_blocks.15.block.ff_t.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.12.block.attn1.to_out', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.attn1.to_k', 'transformer.double_stream_blocks.5.block.attn1.to_out_t', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.8.block.attn1.to_out_t', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.4.block.ff_t.w1', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.11.block.attn1.to_k_t', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.15.block.attn1.to_v', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_k_t', 'transformer.double_stream_blocks.5.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.attn1.to_q', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.30.block.attn1.to_q', 'transformer.single_stream_blocks.25.block.attn1.to_q', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w1', 'transformer.final_layer.adaLN_modulation.1', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.2.block.attn1.to_v', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_t.w1', 'transformer.single_stream_blocks.29.block.attn1.to_k', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.14.block.ff_t.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.2.block.ff_t.w3', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w1', 'transformer.caption_projection.4.linear', 'transformer.double_stream_blocks.4.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.attn1.to_q', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.2.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.3.block.attn1.to_q', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.30.block.attn1.to_out', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.15.block.adaLN_modulation.1', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.attn1.to_out_t', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.14.block.ff_t.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.22.block.attn1.to_v', 'transformer.double_stream_blocks.2.block.attn1.to_v', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.8.block.attn1.to_v_t', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.7.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.7.block.attn1.to_v', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.27.block.attn1.to_v', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w3', 'transformer.caption_projection.46.linear', 'transformer.caption_projection.0.linear', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w1', 'transformer.caption_projection.10.linear', 'transformer.double_stream_blocks.11.block.attn1.to_v_t', 'transformer.caption_projection.21.linear', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w1', 'transformer.caption_projection.30.linear', 'transformer.double_stream_blocks.10.block.attn1.to_out_t', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.18.block.attn1.to_out', 'transformer.single_stream_blocks.5.block.adaLN_modulation.1', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.12.block.adaLN_modulation.1', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.14.block.attn1.to_out', 'transformer.single_stream_blocks.9.block.attn1.to_k', 'transformer.caption_projection.34.linear', 'transformer.double_stream_blocks.12.block.adaLN_modulation.1', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.1.block.attn1.to_q_t', 'transformer.single_stream_blocks.8.block.attn1.to_v', 'transformer.single_stream_blocks.29.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.attn1.to_out', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.5.block.ff_t.w3', 'transformer.double_stream_blocks.10.block.attn1.to_v_t', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w1', 'transformer.caption_projection.16.linear', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.7.block.attn1.to_out_t', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.attn1.to_out', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.13.block.attn1.to_k_t', 'transformer.single_stream_blocks.8.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.31.block.attn1.to_q', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w3', 'transformer.caption_projection.31.linear', 'transformer.single_stream_blocks.7.block.attn1.to_k', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.4.block.attn1.to_out_t', 'transformer.single_stream_blocks.26.block.attn1.to_k', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.5.block.attn1.to_k', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w3', 'transformer.caption_projection.22.linear', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.14.block.attn1.to_v', 'transformer.double_stream_blocks.14.block.attn1.to_k', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.3.block.adaLN_modulation.1', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.attn1.to_q', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.14.block.attn1.to_out', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.9.block.attn1.to_v', 'transformer.single_stream_blocks.10.block.attn1.to_k', 'transformer.single_stream_blocks.13.block.adaLN_modulation.1', 'transformer.single_stream_blocks.5.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.16.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.attn1.to_q', 'transformer.caption_projection.37.linear', 'transformer.single_stream_blocks.24.block.adaLN_modulation.1', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.18.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w3', 'transformer.caption_projection.38.linear', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.25.block.attn1.to_out', 'transformer.caption_projection.25.linear', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.ff_t.w2', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.11.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.9.block.attn1.to_out', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.14.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.4.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.attn1.to_k_t', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.4.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.attn1.to_q', 'transformer.caption_projection.33.linear', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.21.block.attn1.to_k', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.30.block.attn1.to_k', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.attn1.to_out', 'transformer.double_stream_blocks.11.block.adaLN_modulation.1', 'transformer.single_stream_blocks.26.block.attn1.to_v', 'transformer.double_stream_blocks.8.block.adaLN_modulation.1', 'transformer.single_stream_blocks.0.block.attn1.to_out', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.7.block.attn1.to_out', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.15.block.attn1.to_q', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.4.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.attn1.to_v', 'transformer.single_stream_blocks.1.block.attn1.to_q', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.8.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_k_t', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.13.block.ff_t.w3', 'transformer.double_stream_blocks.9.block.ff_t.w1', 'transformer.double_stream_blocks.7.block.attn1.to_v_t', 'transformer.caption_projection.12.linear', 'transformer.single_stream_blocks.4.block.attn1.to_v', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.9.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_k', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.28.block.attn1.to_k', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.18.block.attn1.to_q', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.0.block.attn1.to_q', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.10.block.attn1.to_q', 'transformer.double_stream_blocks.15.block.attn1.to_v', 'transformer.double_stream_blocks.4.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.attn1.to_k_t', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.11.block.attn1.to_q', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.adaLN_modulation.1', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.15.block.attn1.to_v_t', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.12.block.ff_t.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.18.block.adaLN_modulation.1', 'transformer.single_stream_blocks.13.block.attn1.to_out', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.11.block.attn1.to_q_t', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.24.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.8.block.attn1.to_out', 'transformer.caption_projection.18.linear', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.attn1.to_out_t', 'transformer.single_stream_blocks.31.block.attn1.to_out', 'transformer.double_stream_blocks.1.block.attn1.to_q', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.9.block.ff_t.w2', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.0.block.ff_t.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_t.w1', 'transformer.double_stream_blocks.12.block.ff_t.w3', 'transformer.double_stream_blocks.6.block.attn1.to_v_t', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.11.block.attn1.to_out', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.16.block.attn1.to_q', 'transformer.single_stream_blocks.19.block.attn1.to_q', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w3', 'transformer.caption_projection.23.linear', 'transformer.double_stream_blocks.12.block.attn1.to_q', 'transformer.caption_projection.35.linear', 'transformer.double_stream_blocks.15.block.attn1.to_out', 'transformer.single_stream_blocks.21.block.attn1.to_v', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_t.w2', 'transformer.double_stream_blocks.3.block.ff_t.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.14.block.attn1.to_k_t', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.10.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_q_t', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.31.block.attn1.to_k', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w2', 'transformer.caption_projection.27.linear', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.29.block.attn1.to_q', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w1', 'transformer.caption_projection.32.linear', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.26.block.attn1.to_out', 'transformer.double_stream_blocks.4.block.ff_t.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w3'} not found in the base model. Please check the target modules and try again.
  29. Traceback (most recent call last):
  30. File "/fsx/sayak/diffusers/check_new_omi_lora.py", line 24, in <module>
  31. pipe.load_lora_weights(f"RhaegarKhan/OMI_LORA")
  32. File "/fsx/sayak/diffusers/src/diffusers/loaders/lora_pipeline.py", line 5549, in load_lora_weights
  33. self.load_lora_into_transformer(
  34. File "/fsx/sayak/diffusers/src/diffusers/loaders/lora_pipeline.py", line 5589, in load_lora_into_transformer
  35. transformer.load_lora_adapter(
  36. File "/fsx/sayak/diffusers/src/diffusers/loaders/peft.py", line 356, in load_lora_adapter
  37. inject_adapter_in_model(lora_config, self, adapter_name=adapter_name, **peft_kwargs)
  38. File "/fsx/sayak/peft/src/peft/mapping.py", line 76, in inject_adapter_in_model
  39. peft_model = tuner_cls(model, peft_config, adapter_name=adapter_name, low_cpu_mem_usage=low_cpu_mem_usage)
  40. File "/fsx/sayak/peft/src/peft/tuners/lora/model.py", line 142, in __init__
  41. super().__init__(model, config, adapter_name, low_cpu_mem_usage=low_cpu_mem_usage)
  42. File "/fsx/sayak/peft/src/peft/tuners/tuners_utils.py", line 181, in __init__
  43. self.inject_adapter(self.model, adapter_name, low_cpu_mem_usage=low_cpu_mem_usage)
  44. File "/fsx/sayak/peft/src/peft/tuners/tuners_utils.py", line 528, in inject_adapter
  45. raise ValueError(error_msg)
  46. ValueError: Target modules {'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.attn1.to_q', 'transformer.double_stream_blocks.8.block.ff_t.w3', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.0.block.attn1.to_k', 'transformer.caption_projection.13.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_t.w1', 'transformer.double_stream_blocks.7.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.20.block.attn1.to_k', 'transformer.caption_projection.8.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.13.block.attn1.to_k', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.attn1.to_out', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.3.block.ff_t.w1', 'transformer.double_stream_blocks.11.block.attn1.to_v', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.14.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.attn1.to_q_t', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.29.block.adaLN_modulation.1', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.2.w3', 'transformer.caption_projection.43.linear', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.2.block.attn1.to_k', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.4.block.attn1.to_q', 'transformer.p_embedder.pooled_embedder.linear_1', 'transformer.caption_projection.19.linear', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w1', 'transformer.caption_projection.26.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.6.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.adaLN_modulation.1', 'transformer.single_stream_blocks.1.block.attn1.to_v', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.0.block.attn1.to_v_t', 'transformer.double_stream_blocks.0.block.attn1.to_v', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.11.block.attn1.to_k', 'transformer.double_stream_blocks.13.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.attn1.to_k_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.2.block.adaLN_modulation.1', 'transformer.double_stream_blocks.6.block.attn1.to_v', 'transformer.single_stream_blocks.11.block.adaLN_modulation.1', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.27.block.attn1.to_out', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.1.block.attn1.to_out', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.3.block.adaLN_modulation.1', 'transformer.double_stream_blocks.8.block.attn1.to_k', 'transformer.double_stream_blocks.8.block.attn1.to_q_t', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.4.block.attn1.to_out', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.3.block.ff_t.w2', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.15.block.adaLN_modulation.1', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.1.block.attn1.to_v_t', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w3', 'transformer.caption_projection.48.linear', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.attn1.to_out_t', 'transformer.single_stream_blocks.16.block.attn1.to_v', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.17.block.attn1.to_k', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.8.block.attn1.to_k', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w1', 'transformer.caption_projection.11.linear', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.0.block.attn1.to_v', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.6.block.attn1.to_q_t', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.0.block.ff_t.w2', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.attn1.to_k', 'transformer.single_stream_blocks.31.block.attn1.to_v', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.28.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.adaLN_modulation.1', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.27.block.attn1.to_k', 'transformer.single_stream_blocks.1.block.adaLN_modulation.1', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.11.block.attn1.to_k', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.25.block.adaLN_modulation.1', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.attn1.to_k_t', 'transformer.single_stream_blocks.27.block.attn1.to_q', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.14.block.adaLN_modulation.1', 'transformer.double_stream_blocks.12.block.attn1.to_v_t', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.0.block.ff_t.w1', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w2', 'transformer.caption_projection.42.linear', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.ff_t.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.7.block.ff_t.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.0.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.4.block.attn1.to_v_t', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.19.block.attn1.to_out', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.3.block.attn1.to_k', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.13.block.attn1.to_q_t', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.6.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.attn1.to_q_t', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.attn1.to_out', 'transformer.double_stream_blocks.15.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.1.block.attn1.to_out', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w3', 'transformer.caption_projection.29.linear', 'transformer.double_stream_blocks.1.block.ff_t.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.17.block.attn1.to_out', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.attn1.to_v_t', 'transformer.caption_projection.9.linear', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.attn1.to_q', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.11.block.ff_t.w3', 'transformer.single_stream_blocks.4.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.adaLN_modulation.1', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.11.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.9.block.ff_t.w3', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.31.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.adaLN_modulation.1', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.27.block.adaLN_modulation.1', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.12.block.ff_t.w2', 'transformer.single_stream_blocks.19.block.attn1.to_v', 'transformer.double_stream_blocks.6.block.ff_t.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.3.block.attn1.to_out', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.15.block.ff_t.w2', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.2.block.attn1.to_q', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.6.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.23.block.adaLN_modulation.1', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.8.block.attn1.to_k_t', 'transformer.double_stream_blocks.15.block.attn1.to_k_t', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.attn1.to_out_t', 'transformer.double_stream_blocks.12.block.ff_i.experts.3.w2', 'transformer.caption_projection.47.linear', 'transformer.caption_projection.14.linear', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w1', 'transformer.caption_projection.28.linear', 'transformer.single_stream_blocks.12.block.attn1.to_v', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.attn1.to_out_t', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w3', 'transformer.caption_projection.15.linear', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.26.block.adaLN_modulation.1', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.30.block.attn1.to_v', 'transformer.single_stream_blocks.28.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.attn1.to_q', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.attn1.to_q_t', 'transformer.caption_projection.44.linear', 'transformer.caption_projection.45.linear', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.16.block.attn1.to_k', 'transformer.single_stream_blocks.17.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.attn1.to_out', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.attn1.to_v', 'transformer.x_embedder.proj', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w2', 'transformer.t_embedder.timestep_embedder.linear_1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.attn1.to_v', 'transformer.caption_projection.41.linear', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.7.block.attn1.to_k_t', 'transformer.single_stream_blocks.10.block.adaLN_modulation.1', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.28.block.attn1.to_v', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.attn1.to_out', 'transformer.double_stream_blocks.10.block.attn1.to_v', 'transformer.single_stream_blocks.20.block.adaLN_modulation.1', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.13.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.21.block.adaLN_modulation.1', 'transformer.double_stream_blocks.12.block.attn1.to_out_t', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.2.block.attn1.to_out', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.3.block.attn1.to_q_t', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w3', 'transformer.final_layer.linear', 'transformer.double_stream_blocks.14.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.7.block.adaLN_modulation.1', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.15.block.attn1.to_k', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.10.block.ff_t.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w2', 'transformer.caption_projection.2.linear', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.1.block.attn1.to_k', 'transformer.single_stream_blocks.12.block.attn1.to_out', 'transformer.single_stream_blocks.16.block.attn1.to_out', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.11.block.ff_t.w2', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.15.block.ff_i.experts.1.w3', 'transformer.caption_projection.1.linear', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.1.w3', 'transformer.caption_projection.24.linear', 'transformer.double_stream_blocks.12.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.0.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.attn1.to_q', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.15.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.3.block.attn1.to_k', 'transformer.single_stream_blocks.26.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_v_t', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.14.block.attn1.to_v_t', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.7.block.ff_t.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w3', 'transformer.caption_projection.6.linear', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.3.block.attn1.to_v_t', 'transformer.single_stream_blocks.3.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_t.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.2.block.attn1.to_q', 'transformer.single_stream_blocks.6.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.attn1.to_q', 'transformer.single_stream_blocks.18.block.attn1.to_v', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.22.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.attn1.to_v', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w1', 'transformer.caption_projection.17.linear', 'transformer.double_stream_blocks.10.block.ff_t.w3', 'transformer.double_stream_blocks.4.block.attn1.to_out', 'transformer.double_stream_blocks.4.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.2.block.attn1.to_out_t', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.22.block.adaLN_modulation.1', 'transformer.double_stream_blocks.6.block.ff_t.w1', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.28.block.adaLN_modulation.1', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.14.block.attn1.to_out_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.5.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.20.block.ff_i.experts.3.w2', 'transformer.caption_projection.3.linear', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.13.block.ff_t.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.1.block.attn1.to_k', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.1.block.attn1.to_out_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.9.block.attn1.to_k', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.25.block.attn1.to_k', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.2.block.attn1.to_q_t', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.0.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.4.block.attn1.to_k_t', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.0.block.attn1.to_q', 'transformer.single_stream_blocks.17.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.attn1.to_k', 'transformer.single_stream_blocks.20.block.attn1.to_q', 'transformer.double_stream_blocks.11.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.15.block.ff_t.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.20.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.0.block.attn1.to_k', 'transformer.caption_projection.36.linear', 'transformer.double_stream_blocks.10.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.1.block.adaLN_modulation.1', 'transformer.single_stream_blocks.14.block.ff_i.experts.2.w2', 'transformer.caption_projection.20.linear', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.24.block.attn1.to_k', 'transformer.single_stream_blocks.24.block.attn1.to_out', 'transformer.double_stream_blocks.5.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.1.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.25.block.attn1.to_v', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.6.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_v', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.7.block.attn1.to_q', 'transformer.double_stream_blocks.2.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.14.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.14.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.17.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.attn1.to_q_t', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.15.block.attn1.to_q_t', 'transformer.double_stream_blocks.13.block.attn1.to_out_t', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.2.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.4.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.20.block.attn1.to_out', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.13.block.ff_t.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.11.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.6.block.ff_t.w2', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.10.block.attn1.to_v', 'transformer.single_stream_blocks.13.block.attn1.to_q', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w2', 'transformer.p_embedder.pooled_embedder.linear_2', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.10.block.attn1.to_out', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.7.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.9.block.attn1.to_out_t', 'transformer.double_stream_blocks.2.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.attn1.to_k_t', 'transformer.double_stream_blocks.13.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w1', 'transformer.caption_projection.40.linear', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.10.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_v', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w3', 'transformer.t_embedder.timestep_embedder.linear_2', 'transformer.double_stream_blocks.3.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.14.block.ff_t.w1', 'transformer.double_stream_blocks.9.block.attn1.to_k_t', 'transformer.double_stream_blocks.8.block.attn1.to_q', 'transformer.double_stream_blocks.0.block.attn1.to_k_t', 'transformer.double_stream_blocks.7.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.11.block.attn1.to_out', 'transformer.double_stream_blocks.11.block.ff_t.w1', 'transformer.double_stream_blocks.4.block.attn1.to_q_t', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.2.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.14.block.attn1.to_v', 'transformer.double_stream_blocks.13.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.attn1.to_k', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.4.block.ff_t.w3', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.8.block.attn1.to_v', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.11.block.attn1.to_v', 'transformer.single_stream_blocks.29.block.attn1.to_out', 'transformer.single_stream_blocks.26.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w3', 'transformer.caption_projection.5.linear', 'transformer.double_stream_blocks.2.block.attn1.to_v_t', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.attn1.to_v_t', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w2', 'transformer.caption_projection.7.linear', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.0.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.7.block.ff_t.w3', 'transformer.single_stream_blocks.10.block.ff_i.experts.3.w1', 'transformer.caption_projection.39.linear', 'transformer.double_stream_blocks.1.block.ff_t.w1', 'transformer.single_stream_blocks.23.block.attn1.to_k', 'transformer.single_stream_blocks.9.block.attn1.to_q', 'transformer.double_stream_blocks.8.block.attn1.to_out', 'transformer.single_stream_blocks.1.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.7.block.attn1.to_q_t', 'transformer.single_stream_blocks.19.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.21.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.5.block.ff_t.w1', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.29.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.1.block.attn1.to_v', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.0.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.0.block.attn1.to_q_t', 'transformer.single_stream_blocks.22.block.attn1.to_k', 'transformer.single_stream_blocks.28.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.3.block.attn1.to_v', 'transformer.single_stream_blocks.16.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.26.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.adaLN_modulation.1', 'transformer.double_stream_blocks.15.block.ff_t.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.12.block.attn1.to_out', 'transformer.double_stream_blocks.1.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.attn1.to_k', 'transformer.double_stream_blocks.5.block.attn1.to_out_t', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.8.block.attn1.to_out_t', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.4.block.ff_t.w1', 'transformer.single_stream_blocks.19.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.11.block.attn1.to_k_t', 'transformer.single_stream_blocks.30.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.10.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.9.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.15.block.attn1.to_v', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.attn1.to_k_t', 'transformer.double_stream_blocks.5.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.attn1.to_q', 'transformer.single_stream_blocks.23.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.19.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.30.block.attn1.to_q', 'transformer.single_stream_blocks.25.block.attn1.to_q', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.3.block.ff_i.shared_experts.w1', 'transformer.final_layer.adaLN_modulation.1', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.23.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.2.block.attn1.to_v', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.2.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_t.w1', 'transformer.single_stream_blocks.29.block.attn1.to_k', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.14.block.ff_t.w2', 'transformer.single_stream_blocks.19.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.25.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.2.block.ff_t.w3', 'transformer.single_stream_blocks.10.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w1', 'transformer.caption_projection.4.linear', 'transformer.double_stream_blocks.4.block.attn1.to_q', 'transformer.double_stream_blocks.9.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.18.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.5.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.attn1.to_q', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.2.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.20.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.3.block.attn1.to_q', 'transformer.double_stream_blocks.10.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.30.block.attn1.to_out', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.15.block.adaLN_modulation.1', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.attn1.to_out_t', 'transformer.double_stream_blocks.0.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.13.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.14.block.ff_t.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.17.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.22.block.attn1.to_v', 'transformer.double_stream_blocks.2.block.attn1.to_v', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.8.block.attn1.to_v_t', 'transformer.single_stream_blocks.10.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.7.block.adaLN_modulation.1', 'transformer.single_stream_blocks.22.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.7.block.attn1.to_v', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.24.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.27.block.attn1.to_v', 'transformer.single_stream_blocks.28.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.0.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.attn1.to_out', 'transformer.double_stream_blocks.7.block.attn1.to_v', 'transformer.double_stream_blocks.1.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w3', 'transformer.caption_projection.46.linear', 'transformer.caption_projection.0.linear', 'transformer.double_stream_blocks.1.block.ff_i.shared_experts.w1', 'transformer.caption_projection.10.linear', 'transformer.double_stream_blocks.11.block.attn1.to_v_t', 'transformer.caption_projection.21.linear', 'transformer.single_stream_blocks.27.block.ff_i.shared_experts.w1', 'transformer.caption_projection.30.linear', 'transformer.double_stream_blocks.10.block.attn1.to_out_t', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.18.block.attn1.to_out', 'transformer.single_stream_blocks.5.block.adaLN_modulation.1', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.8.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.5.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.12.block.adaLN_modulation.1', 'transformer.single_stream_blocks.2.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.14.block.attn1.to_out', 'transformer.single_stream_blocks.9.block.attn1.to_k', 'transformer.caption_projection.34.linear', 'transformer.double_stream_blocks.12.block.adaLN_modulation.1', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.1.block.attn1.to_q_t', 'transformer.single_stream_blocks.8.block.attn1.to_v', 'transformer.single_stream_blocks.29.block.attn1.to_v', 'transformer.single_stream_blocks.6.block.attn1.to_out', 'transformer.double_stream_blocks.15.block.ff_i.shared_experts.w2', 'transformer.double_stream_blocks.5.block.ff_t.w3', 'transformer.double_stream_blocks.10.block.attn1.to_v_t', 'transformer.double_stream_blocks.8.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.5.block.ff_i.experts.3.w1', 'transformer.caption_projection.16.linear', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.7.block.attn1.to_out_t', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.3.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.22.block.attn1.to_out', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.25.block.ff_i.experts.0.w3', 'transformer.double_stream_blocks.13.block.attn1.to_k_t', 'transformer.single_stream_blocks.8.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.31.block.attn1.to_q', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w3', 'transformer.caption_projection.31.linear', 'transformer.single_stream_blocks.7.block.attn1.to_k', 'transformer.single_stream_blocks.30.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.4.block.attn1.to_out_t', 'transformer.single_stream_blocks.26.block.attn1.to_k', 'transformer.single_stream_blocks.9.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.5.block.attn1.to_k', 'transformer.single_stream_blocks.27.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.9.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.3.w3', 'transformer.caption_projection.22.linear', 'transformer.double_stream_blocks.11.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.5.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.14.block.attn1.to_v', 'transformer.double_stream_blocks.14.block.attn1.to_k', 'transformer.single_stream_blocks.17.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.30.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.20.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.3.block.adaLN_modulation.1', 'transformer.double_stream_blocks.15.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.attn1.to_q', 'transformer.single_stream_blocks.3.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.14.block.attn1.to_out', 'transformer.single_stream_blocks.1.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.9.block.attn1.to_v', 'transformer.single_stream_blocks.10.block.attn1.to_k', 'transformer.single_stream_blocks.13.block.adaLN_modulation.1', 'transformer.single_stream_blocks.5.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.16.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.16.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.attn1.to_q', 'transformer.caption_projection.37.linear', 'transformer.single_stream_blocks.24.block.adaLN_modulation.1', 'transformer.single_stream_blocks.31.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.19.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.23.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.6.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.28.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.16.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.18.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w3', 'transformer.caption_projection.38.linear', 'transformer.single_stream_blocks.25.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.14.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.25.block.attn1.to_out', 'transformer.caption_projection.25.linear', 'transformer.double_stream_blocks.8.block.ff_i.experts.0.w2', 'transformer.double_stream_blocks.14.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.4.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.5.block.ff_t.w2', 'transformer.single_stream_blocks.1.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.29.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.0.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.13.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.11.block.attn1.to_q', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.9.block.attn1.to_out', 'transformer.single_stream_blocks.16.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.14.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.2.w2', 'transformer.double_stream_blocks.10.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.30.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.17.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.4.block.adaLN_modulation.1', 'transformer.double_stream_blocks.2.block.attn1.to_k_t', 'transformer.single_stream_blocks.13.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.4.block.attn1.to_k', 'transformer.double_stream_blocks.9.block.attn1.to_q', 'transformer.double_stream_blocks.14.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.23.block.attn1.to_q', 'transformer.caption_projection.33.linear', 'transformer.single_stream_blocks.0.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.11.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.21.block.attn1.to_k', 'transformer.single_stream_blocks.3.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.30.block.attn1.to_k', 'transformer.single_stream_blocks.29.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.17.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.3.block.attn1.to_out', 'transformer.double_stream_blocks.11.block.adaLN_modulation.1', 'transformer.single_stream_blocks.26.block.attn1.to_v', 'transformer.double_stream_blocks.8.block.adaLN_modulation.1', 'transformer.single_stream_blocks.0.block.attn1.to_out', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.2.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.3.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.21.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.26.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.7.block.attn1.to_out', 'transformer.single_stream_blocks.15.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.15.block.attn1.to_q', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.4.block.attn1.to_v', 'transformer.single_stream_blocks.24.block.attn1.to_v', 'transformer.single_stream_blocks.1.block.attn1.to_q', 'transformer.single_stream_blocks.24.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.14.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.8.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.8.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_k_t', 'transformer.double_stream_blocks.8.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.13.block.ff_t.w3', 'transformer.double_stream_blocks.9.block.ff_t.w1', 'transformer.double_stream_blocks.7.block.attn1.to_v_t', 'transformer.caption_projection.12.linear', 'transformer.single_stream_blocks.4.block.attn1.to_v', 'transformer.single_stream_blocks.26.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.9.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_k', 'transformer.single_stream_blocks.0.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.2.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.28.block.attn1.to_k', 'transformer.single_stream_blocks.6.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.18.block.attn1.to_q', 'transformer.single_stream_blocks.21.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.23.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.5.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.0.block.attn1.to_q', 'transformer.double_stream_blocks.0.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.10.block.attn1.to_q', 'transformer.double_stream_blocks.15.block.attn1.to_v', 'transformer.double_stream_blocks.4.block.attn1.to_k', 'transformer.double_stream_blocks.6.block.attn1.to_k_t', 'transformer.double_stream_blocks.7.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.18.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.7.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.16.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.24.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.11.block.attn1.to_q', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.28.block.ff_i.shared_experts.w1', 'transformer.double_stream_blocks.13.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.1.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.30.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.30.block.adaLN_modulation.1', 'transformer.double_stream_blocks.13.block.adaLN_modulation.1', 'transformer.single_stream_blocks.23.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.7.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.15.block.attn1.to_v_t', 'transformer.single_stream_blocks.20.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.8.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.12.block.ff_t.w1', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w1', 'transformer.double_stream_blocks.11.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.1.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.15.block.attn1.to_k', 'transformer.double_stream_blocks.11.block.ff_i.experts.2.w3', 'transformer.double_stream_blocks.12.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.18.block.adaLN_modulation.1', 'transformer.single_stream_blocks.13.block.attn1.to_out', 'transformer.single_stream_blocks.27.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.11.block.attn1.to_q_t', 'transformer.single_stream_blocks.31.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.9.block.ff_i.experts.0.w2', 'transformer.single_stream_blocks.24.block.attn1.to_q', 'transformer.single_stream_blocks.4.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.8.block.attn1.to_out', 'transformer.caption_projection.18.linear', 'transformer.single_stream_blocks.31.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.12.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.3.block.attn1.to_out_t', 'transformer.single_stream_blocks.31.block.attn1.to_out', 'transformer.double_stream_blocks.1.block.attn1.to_q', 'transformer.double_stream_blocks.13.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.8.block.ff_i.experts.0.w1', 'transformer.double_stream_blocks.9.block.ff_t.w2', 'transformer.double_stream_blocks.5.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.28.block.ff_i.experts.1.w2', 'transformer.double_stream_blocks.0.block.ff_t.w3', 'transformer.single_stream_blocks.6.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.10.block.ff_t.w1', 'transformer.double_stream_blocks.12.block.ff_t.w3', 'transformer.double_stream_blocks.6.block.attn1.to_v_t', 'transformer.single_stream_blocks.13.block.ff_i.experts.3.w2', 'transformer.double_stream_blocks.3.block.ff_i.experts.3.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.11.block.attn1.to_out', 'transformer.single_stream_blocks.21.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.16.block.attn1.to_q', 'transformer.single_stream_blocks.19.block.attn1.to_q', 'transformer.double_stream_blocks.12.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.4.block.ff_i.experts.1.w3', 'transformer.caption_projection.23.linear', 'transformer.double_stream_blocks.12.block.attn1.to_q', 'transformer.caption_projection.35.linear', 'transformer.double_stream_blocks.15.block.attn1.to_out', 'transformer.single_stream_blocks.21.block.attn1.to_v', 'transformer.double_stream_blocks.6.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.2.block.ff_t.w2', 'transformer.double_stream_blocks.3.block.ff_t.w3', 'transformer.double_stream_blocks.3.block.ff_i.experts.1.w3', 'transformer.single_stream_blocks.31.block.ff_i.shared_experts.w3', 'transformer.double_stream_blocks.14.block.attn1.to_k_t', 'transformer.single_stream_blocks.15.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.10.block.adaLN_modulation.1', 'transformer.double_stream_blocks.10.block.attn1.to_q_t', 'transformer.double_stream_blocks.4.block.ff_i.experts.3.w2', 'transformer.single_stream_blocks.12.block.ff_i.experts.3.w3', 'transformer.single_stream_blocks.24.block.ff_i.experts.2.w1', 'transformer.single_stream_blocks.31.block.attn1.to_k', 'transformer.double_stream_blocks.15.block.ff_i.experts.0.w3', 'transformer.single_stream_blocks.11.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.2.w2', 'transformer.caption_projection.27.linear', 'transformer.single_stream_blocks.17.block.ff_i.experts.3.w3', 'transformer.double_stream_blocks.4.block.ff_i.experts.1.w2', 'transformer.single_stream_blocks.7.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.27.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.13.block.ff_i.experts.2.w3', 'transformer.single_stream_blocks.14.block.ff_i.experts.1.w3', 'transformer.double_stream_blocks.7.block.ff_i.experts.3.w1', 'transformer.double_stream_blocks.3.block.ff_i.shared_experts.w1', 'transformer.single_stream_blocks.18.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.26.block.ff_i.experts.0.w1', 'transformer.single_stream_blocks.29.block.attn1.to_q', 'transformer.single_stream_blocks.9.block.ff_i.experts.1.w1', 'transformer.single_stream_blocks.12.block.ff_i.experts.0.w1', 'transformer.caption_projection.32.linear', 'transformer.double_stream_blocks.4.block.ff_i.experts.2.w2', 'transformer.single_stream_blocks.25.block.ff_i.shared_experts.w2', 'transformer.single_stream_blocks.22.block.ff_i.experts.1.w1', 'transformer.double_stream_blocks.6.block.ff_i.shared_experts.w3', 'transformer.single_stream_blocks.26.block.attn1.to_out', 'transformer.double_stream_blocks.4.block.ff_t.w2', 'transformer.double_stream_blocks.9.block.ff_i.experts.2.w3'} not found in the base model. Please check the target modules and try again.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!