API tools faq
paste
Guest User

all tensors to be on the same device

a guest
Jul 10th, 2025
189
0
Never
7
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 84.15 KB | None | 0 0
raw download clone embed print report
  1.  
  2. # ComfyUI Error Report
  3. ## Error Details
  4. - **Node ID:** 4
  5. - **Node Type:** FluxTrainLoop
  6. - **Exception Type:** RuntimeError
  7. - **Exception Message:** Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
  8. ## Stack Trace
  9. ```
  10. File "/home/garrett/AI/ComfyUI/execution.py", line 323, in execute
  11. output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  12.  
  13. File "/home/garrett/AI/ComfyUI/execution.py", line 198, in get_output_data
  14. return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  15.  
  16. File "/home/garrett/AI/ComfyUI/execution.py", line 169, in _map_node_over_list
  17. process_inputs(input_dict, i)
  18.  
  19. File "/home/garrett/AI/ComfyUI/execution.py", line 158, in process_inputs
  20. results.append(getattr(obj, func)(**inputs))
  21.  
  22. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/nodes.py", line 910, in train
  23. steps_done = training_loop(
  24.  
  25. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/train_network.py", line 1247, in training_loop
  26. noise_pred, target, timesteps, weighting = self.get_noise_pred_and_target(
  27.  
  28. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_network_comfy.py", line 363, in get_noise_pred_and_target
  29. model_pred = call_dit(
  30.  
  31. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_network_comfy.py", line 312, in call_dit
  32. model_pred = unet(
  33.  
  34. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
  35. return self._call_impl(*args, **kwargs)
  36.  
  37. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
  38. return forward_call(*args, **kwargs)
  39.  
  40. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 820, in forward
  41. return model_forward(*args, **kwargs)
  42.  
  43. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 808, in __call__
  44. return convert_to_fp32(self.model_forward(*args, **kwargs))
  45.  
  46. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 43, in decorate_autocast
  47. return func(*args, **kwargs)
  48.  
  49. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_models.py", line 1039, in forward
  50. img, txt = block(img=img, txt=txt, vec=vec, pe=pe, txt_attention_mask=txt_attention_mask)
  51.  
  52. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
  53. return self._call_impl(*args, **kwargs)
  54.  
  55. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
  56. return forward_call(*args, **kwargs)
  57.  
  58. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_models.py", line 736, in forward
  59. return checkpoint(self._forward, img, txt, vec, pe, txt_attention_mask, use_reentrant=False)
  60.  
  61. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_compile.py", line 31, in inner
  62. return disable_fn(*args, **kwargs)
  63.  
  64. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 600, in _fn
  65. return fn(*args, **kwargs)
  66.  
  67. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/checkpoint.py", line 488, in checkpoint
  68. ret = function(*args, **kwargs)
  69.  
  70. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_models.py", line 685, in _forward
  71. img_mod1, img_mod2 = self.img_mod(vec)
  72.  
  73. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
  74. return self._call_impl(*args, **kwargs)
  75.  
  76. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
  77. return forward_call(*args, **kwargs)
  78.  
  79. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_models.py", line 634, in forward
  80. out = self.lin(nn.functional.silu(vec))[:, None, :].chunk(self.multiplier, dim=-1)
  81.  
  82. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
  83. return self._call_impl(*args, **kwargs)
  84.  
  85. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
  86. return forward_call(*args, **kwargs)
  87.  
  88. File "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/networks/lora_flux.py", line 103, in forward
  89. org_forwarded = self.org_forward(x)
  90.  
  91. File "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 117, in forward
  92. return F.linear(input, self.weight, self.bias)
  93.  
  94. ```
  95. ## System Information
  96. - **ComfyUI Version:** v0.3.4-1-g839ed33
  97. - **Arguments:** /home/garrett/AI/ComfyUI/main.py --listen
  98. - **OS:** posix
  99. - **Python Version:** 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]
  100. - **Embedded Python:** false
  101. - **PyTorch Version:** 2.4.0+cu121
  102. ## Devices
  103.  
  104. - **Name:** cuda:0 NVIDIA GeForce RTX 3080 Laptop GPU : cudaMallocAsync
  105. - **Type:** cuda
  106. - **VRAM Total:** 8361017344
  107. - **VRAM Free:** 2044293984
  108. - **Torch VRAM Total:** 5872025600
  109. - **Torch VRAM Free:** 40989536
  110.  
  111. ## Logs
  112. ```
  113. 2025-07-10T13:48:48.086809 - 2025-07-10 13:48:48 INFO move CLIP-L back to cpu flux_train_network_comfy.py:229
  114. 2025-07-10T13:48:48.566163 - INFO move t5XXL back to cpu flux_train_network_comfy.py:231
  115. 2025-07-10T13:48:52.600165 - 2025-07-10 13:48:52 INFO move vae and unet back to original device flux_train_network_comfy.py:236
  116. 2025-07-10T13:48:52.605524 - INFO create LoRA network. base dim (rank): 64, alpha: 64.0 lora_flux.py:491
  117. 2025-07-10T13:48:52.606129 - INFO neuron dropout: p=None, rank dropout: p=None, module dropout: p=None lora_flux.py:492
  118. 2025-07-10T13:48:52.606665 - INFO train all blocks only lora_flux.py:502
  119. 2025-07-10T13:48:52.607212 - INFO create LoRA for Text Encoder 1: lora_flux.py:589
  120. 2025-07-10T13:48:52.725375 - INFO create LoRA for Text Encoder 1: 72 modules. lora_flux.py:592
  121. 2025-07-10T13:48:54.588996 - 2025-07-10 13:48:54 INFO create LoRA for FLUX all blocks: 304 modules. lora_flux.py:606
  122. 2025-07-10T13:48:54.590211 - INFO enable LoRA for U-Net: 304 modules lora_flux.py:755
  123. 2025-07-10T13:48:54.591816 - FLUX: Gradient checkpointing enabled. CPU offload: False2025-07-10T13:48:54.591858 -
  124. 2025-07-10T13:48:54.591897 - prepare optimizer, data loader etc.2025-07-10T13:48:54.591907 -
  125. 2025-07-10T13:48:54.594547 - INFO use Adafactor optimizer | {'relative_step': False, 'scale_parameter': train_util.py:4815
  126. False, 'warmup_init': False, 'clip_threshold': 1.0}
  127. 2025-07-10T13:48:54.595469 - WARNING because max_grad_norm is set, clip_grad_norm is enabled. consider set to train_util.py:4843
  128. 0 /
  129. max_grad_normが設定されているためclip_grad_normが有効になります。0に設定
  130. して無効にしたほうがいいかもしれません
  131. 2025-07-10T13:48:54.596162 - WARNING constant_with_warmup will be good / train_util.py:4847
  132. スケジューラはconstant_with_warmupが良いかもしれません
  133. 2025-07-10T13:48:54.598750 - enable full bf16 training.2025-07-10T13:48:54.598769 -
  134. 2025-07-10T13:48:54.734762 - enable fp8 training for U-Net.2025-07-10T13:48:54.734800 -
  135. 2025-07-10T13:48:54.734828 - unet_weight_dtype: torch.float8_e4m3fn2025-07-10T13:48:54.734840 -
  136. 2025-07-10T13:48:54.735579 - INFO set U-Net weight dtype to torch.float8_e4m3fn train_network.py:638
  137. 2025-07-10T13:49:16.198259 - running training2025-07-10T13:49:16.198822 -
  138. 2025-07-10T13:49:16.198845 - num train images * repeats: 122025-07-10T13:49:16.198855 -
  139. 2025-07-10T13:49:16.198869 - num reg images: 02025-07-10T13:49:16.198879 -
  140. 2025-07-10T13:49:16.198896 - num batches per epoch: 122025-07-10T13:49:16.198905 -
  141. 2025-07-10T13:49:16.198918 - num epochs: 2502025-07-10T13:49:16.198927 -
  142. 2025-07-10T13:49:16.199002 - batch size per device: 1, 1, 12025-07-10T13:49:16.199012 -
  143. 2025-07-10T13:49:16.199024 - gradient accumulation steps: 12025-07-10T13:49:16.199034 -
  144. 2025-07-10T13:49:16.199045 - total optimization steps: 30002025-07-10T13:49:16.199061 -
  145. 2025-07-10T13:49:34.975625 - 2025-07-10 13:49:34 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1107
  146. 2025-07-10T13:49:34.992272 - INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1124
  147. 2025-07-10T13:49:35.219679 -
  148. Epoch 1/250 - steps: 0%| | 0/3000 [00:00<?, ?it/s]2025-07-10T13:49:35.603306 - 2025-07-10 13:49:35 ERROR !!! Exception during processing !!! Expected all tensors to be on the same execution.py:392
  149. device, but found at least two devices, cpu and cuda:0! (when checking
  150. argument for argument mat1 in method wrapper_CUDA_addmm)
  151. 2025-07-10T13:49:35.615819 - ERROR Traceback (most recent call last): execution.py:393
  152. File "/home/garrett/AI/ComfyUI/execution.py", line 323, in execute
  153. output_data, output_ui, has_subgraph = get_output_data(obj,
  154. input_data_all, execution_block_cb=execution_block_cb,
  155. pre_execute_cb=pre_execute_cb)
  156. File "/home/garrett/AI/ComfyUI/execution.py", line 198, in
  157. get_output_data
  158. return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION,
  159. allow_interrupt=True, execution_block_cb=execution_block_cb,
  160. pre_execute_cb=pre_execute_cb)
  161. File "/home/garrett/AI/ComfyUI/execution.py", line 169, in
  162. _map_node_over_list
  163. process_inputs(input_dict, i)
  164. File "/home/garrett/AI/ComfyUI/execution.py", line 158, in process_inputs
  165. results.append(getattr(obj, func)(**inputs))
  166. File
  167. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/nodes.py", line
  168. 910, in train
  169. steps_done = training_loop(
  170. File
  171. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/train_network.py
  172. ", line 1247, in training_loop
  173. noise_pred, target, timesteps, weighting =
  174. self.get_noise_pred_and_target(
  175. File
  176. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_netwo
  177. rk_comfy.py", line 363, in get_noise_pred_and_target
  178. model_pred = call_dit(
  179. File
  180. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_netwo
  181. rk_comfy.py", line 312, in call_dit
  182. model_pred = unet(
  183. File
  184. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  185. s/module.py", line 1553, in _wrapped_call_impl
  186. return self._call_impl(*args, **kwargs)
  187. File
  188. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  189. s/module.py", line 1562, in _call_impl
  190. return forward_call(*args, **kwargs)
  191. File
  192. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/util
  193. s/operations.py", line 820, in forward
  194. return model_forward(*args, **kwargs)
  195. File
  196. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/util
  197. s/operations.py", line 808, in __call__
  198. return convert_to_fp32(self.model_forward(*args, **kwargs))
  199. File
  200. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/amp/autoc
  201. ast_mode.py", line 43, in decorate_autocast
  202. return func(*args, **kwargs)
  203. File
  204. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  205. els.py", line 1039, in forward
  206. img, txt = block(img=img, txt=txt, vec=vec, pe=pe,
  207. txt_attention_mask=txt_attention_mask)
  208. File
  209. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  210. s/module.py", line 1553, in _wrapped_call_impl
  211. return self._call_impl(*args, **kwargs)
  212. File
  213. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  214. s/module.py", line 1562, in _call_impl
  215. return forward_call(*args, **kwargs)
  216. File
  217. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  218. els.py", line 736, in forward
  219. return checkpoint(self._forward, img, txt, vec, pe, txt_attention_mask,
  220. use_reentrant=False)
  221. File
  222. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_compile.
  223. py", line 31, in inner
  224. return disable_fn(*args, **kwargs)
  225. File
  226. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_dynamo/e
  227. val_frame.py", line 600, in _fn
  228. return fn(*args, **kwargs)
  229. File
  230. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/che
  231. ckpoint.py", line 488, in checkpoint
  232. ret = function(*args, **kwargs)
  233. File
  234. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  235. els.py", line 685, in _forward
  236. img_mod1, img_mod2 = self.img_mod(vec)
  237. File
  238. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  239. s/module.py", line 1553, in _wrapped_call_impl
  240. return self._call_impl(*args, **kwargs)
  241. File
  242. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  243. s/module.py", line 1562, in _call_impl
  244. return forward_call(*args, **kwargs)
  245. File
  246. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  247. els.py", line 634, in forward
  248. out = self.lin(nn.functional.silu(vec))[:, None,
  249. :].chunk(self.multiplier, dim=-1)
  250. File
  251. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  252. s/module.py", line 1553, in _wrapped_call_impl
  253. return self._call_impl(*args, **kwargs)
  254. File
  255. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  256. s/module.py", line 1562, in _call_impl
  257. return forward_call(*args, **kwargs)
  258. File
  259. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/networks/lora_fl
  260. ux.py", line 103, in forward
  261. org_forwarded = self.org_forward(x)
  262. File
  263. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  264. s/linear.py", line 117, in forward
  265. return F.linear(input, self.weight, self.bias)
  266. RuntimeError: Expected all tensors to be on the same device, but found at
  267. least two devices, cpu and cuda:0! (when checking argument for argument
  268. mat1 in method wrapper_CUDA_addmm)
  269.  
  270. 2025-07-10T13:49:35.616794 - INFO Prompt executed in 52.59 seconds main.py:139
  271. 2025-07-10T13:52:08.072815 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/1742899825_extension-node-map.json2025-07-10T13:52:08.072880 - 2025-07-10T13:52:08.089881 - [DONE]2025-07-10T13:52:08.089917 -
  272. 2025-07-10T13:52:08.137487 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/1514988643_custom-node-list.json2025-07-10T13:52:08.137540 - 2025-07-10T13:52:08.141823 - [DONE]2025-07-10T13:52:08.141850 -
  273. 2025-07-10T13:52:08.141915 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/746607195_github-stats.json2025-07-10T13:52:08.141925 - 2025-07-10T13:52:08.144239 - [DONE]2025-07-10T13:52:08.144265 -
  274. 2025-07-10T14:02:22.846486 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/1742899825_extension-node-map.json2025-07-10T14:02:22.846537 - 2025-07-10T14:02:22.849574 - [DONE]2025-07-10T14:02:22.849634 -
  275. 2025-07-10T14:02:22.877213 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/1514988643_custom-node-list.json2025-07-10T14:02:22.877247 - 2025-07-10T14:02:22.880781 - [DONE]2025-07-10T14:02:22.880837 -
  276. 2025-07-10T14:02:22.880894 - FETCH DATA from: /home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-Manager/.cache/746607195_github-stats.json2025-07-10T14:02:22.880903 - 2025-07-10T14:02:22.882423 - [DONE]2025-07-10T14:02:22.882441 -
  277. 2025-07-10T14:04:40.228588 - 2025-07-10 14:04:40 INFO got prompt server.py:600
  278. 2025-07-10T14:04:40.230892 - ERROR Failed to validate prompt for output 98: execution.py:822
  279. 2025-07-10T14:04:40.231909 - ERROR * FluxTrainLoop 4: execution.py:842
  280. 2025-07-10T14:04:40.232947 - ERROR - Required input is missing: network_trainer execution.py:844
  281. 2025-07-10T14:04:40.233857 - ERROR Output will be ignored execution.py:846
  282. 2025-07-10T14:04:40.234509 - ERROR Failed to validate prompt for output 102: execution.py:822
  283. 2025-07-10T14:04:40.235001 - ERROR Output will be ignored execution.py:846
  284. 2025-07-10T14:04:40.236828 - ERROR Failed to validate prompt for output 90: execution.py:822
  285. 2025-07-10T14:04:40.237427 - ERROR Output will be ignored execution.py:846
  286. 2025-07-10T14:04:40.238083 - ERROR Failed to validate prompt for output 133: execution.py:822
  287. 2025-07-10T14:04:40.238705 - ERROR * (prompt): execution.py:824
  288. 2025-07-10T14:04:40.239333 - ERROR - Required input is missing: network_trainer execution.py:826
  289. 2025-07-10T14:04:40.239958 - ERROR * FluxTrainEnd 133: execution.py:842
  290. 2025-07-10T14:04:40.240507 - ERROR - Required input is missing: network_trainer execution.py:844
  291. 2025-07-10T14:04:40.240994 - ERROR Output will be ignored execution.py:846
  292. 2025-07-10T14:04:40.242181 - ERROR Failed to validate prompt for output 130: execution.py:822
  293. 2025-07-10T14:04:40.242724 - ERROR * AddLabel 80: execution.py:842
  294. 2025-07-10T14:04:40.243235 - ERROR - Required input is missing: image execution.py:844
  295. 2025-07-10T14:04:40.243748 - ERROR * AddLabel 78: execution.py:842
  296. 2025-07-10T14:04:40.244243 - ERROR - Required input is missing: image execution.py:844
  297. 2025-07-10T14:04:40.244700 - ERROR Output will be ignored execution.py:846
  298. 2025-07-10T14:04:40.245231 - ERROR Failed to validate prompt for output 100: execution.py:822
  299. 2025-07-10T14:04:40.245677 - ERROR Output will be ignored execution.py:846
  300. 2025-07-10T14:04:40.246188 - ERROR Failed to validate prompt for output 74: execution.py:822
  301. 2025-07-10T14:04:40.246621 - ERROR Output will be ignored execution.py:846
  302. 2025-07-10T14:04:40.249069 - queue_counter: 02025-07-10T14:04:40.249093 -
  303. 2025-07-10T14:04:40.256457 - INFO Prompt executed in 0.01 seconds main.py:139
  304. 2025-07-10T14:10:23.258167 - 2025-07-10 14:10:23 INFO got prompt server.py:600
  305. 2025-07-10T14:10:23.261365 - ERROR Failed to validate prompt for output 133: execution.py:822
  306. 2025-07-10T14:10:23.262350 - ERROR * (prompt): execution.py:824
  307. 2025-07-10T14:10:23.263480 - ERROR - Required input is missing: network_trainer execution.py:826
  308. 2025-07-10T14:10:23.264358 - ERROR * FluxTrainEnd 133: execution.py:842
  309. 2025-07-10T14:10:23.265162 - ERROR - Required input is missing: network_trainer execution.py:844
  310. 2025-07-10T14:10:23.266007 - ERROR Output will be ignored execution.py:846
  311. 2025-07-10T14:10:23.267585 - ERROR Failed to validate prompt for output 130: execution.py:822
  312. 2025-07-10T14:10:23.268668 - ERROR * AddLabel 80: execution.py:842
  313. 2025-07-10T14:10:23.269589 - ERROR - Required input is missing: image execution.py:844
  314. 2025-07-10T14:10:23.270365 - ERROR * AddLabel 78: execution.py:842
  315. 2025-07-10T14:10:23.271076 - ERROR - Required input is missing: image execution.py:844
  316. 2025-07-10T14:10:23.271742 - ERROR Output will be ignored execution.py:846
  317. 2025-07-10T14:10:23.272505 - ERROR Failed to validate prompt for output 74: execution.py:822
  318. 2025-07-10T14:10:23.273053 - ERROR Output will be ignored execution.py:846
  319. 2025-07-10T14:10:23.275139 - queue_counter: 02025-07-10T14:10:23.275317 -
  320. 2025-07-10T14:10:23.286253 -
  321. Epoch 1/250 - steps: 0%| | 0/3000 [20:48<?, ?it/s]2025-07-10T14:10:23.286373 -
  322. 2025-07-10T14:10:23.300112 - additional_args: 2025-07-10T14:10:23.300153 -
  323. 2025-07-10T14:10:23.304379 - 2025-07-10 14:10:23 INFO Checking the state dict: Diffusers or BFL, dev or schnell flux_utils.py:47
  324. 2025-07-10T14:10:23.308839 - INFO t5xxl_max_token_length: 512 flux_train_network_comfy.py:111
  325. 2025-07-10T14:10:23.777189 - /home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  326. warnings.warn(
  327. 2025-07-10T14:10:24.167148 - 2025-07-10 14:10:24 INFO Loading dataset config from [[datasets]] train_network.py:342
  328. resolution = [ 512, 512,]
  329. batch_size = 1
  330. enable_bucket = true
  331. bucket_no_upscale = false
  332. min_bucket_reso = 256
  333. max_bucket_reso = 1024
  334. [[datasets.subsets]]
  335. image_dir = "/home/garrett/AI/ComfyUI/training/garrett-dataset/"
  336. class_tokens = "GarrettLynchIRL"
  337. num_repeats = 1
  338.  
  339.  
  340. [[datasets]]
  341. resolution = [ 768, 768,]
  342. batch_size = 1
  343. enable_bucket = true
  344. bucket_no_upscale = false
  345. min_bucket_reso = 256
  346. max_bucket_reso = 1024
  347. [[datasets.subsets]]
  348. image_dir = ""
  349. class_tokens = ""
  350. num_repeats = 1
  351.  
  352.  
  353. [[datasets]]
  354. resolution = [ 1024, 1024,]
  355. batch_size = 1
  356. enable_bucket = true
  357. bucket_no_upscale = false
  358. min_bucket_reso = 256
  359. max_bucket_reso = 1024
  360. [[datasets.subsets]]
  361. image_dir = ""
  362. class_tokens = ""
  363. num_repeats = 1
  364.  
  365.  
  366. [general]
  367. shuffle_caption = false
  368. caption_extension = ".txt"
  369. keep_tokens_separator = "|||"
  370. caption_dropout_rate = 0.0
  371. color_aug = false
  372. flip_aug = false
  373.  
  374. 2025-07-10T14:10:24.169019 - INFO prepare images. train_util.py:1955
  375. 2025-07-10T14:10:24.169994 - INFO get image size from name of cache files train_util.py:1864
  376. 2025-07-10T14:10:24.170228 -
  377. 0%| | 0/12 [00:00<?, ?it/s]2025-07-10T14:10:24.170358 -
  378. 100%|████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 190650.18it/s]2025-07-10T14:10:24.170374 -
  379. 2025-07-10T14:10:24.170903 - INFO set image size from cache files: 12/12 train_util.py:1894
  380. 2025-07-10T14:10:24.171462 - INFO found directory /home/garrett/AI/ComfyUI/training/garrett-dataset/ train_util.py:1896
  381. contains 12 image files
  382. 2025-07-10T14:10:24.171605 -
  383. read caption: 0%| | 0/12 [00:00<?, ?it/s]2025-07-10T14:10:24.171753 -
  384. read caption: 100%|██████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 135300.13it/s]2025-07-10T14:10:24.171769 -
  385. 2025-07-10T14:10:24.172240 - INFO Found captions for 0 images. train_util.py:1926
  386. 2025-07-10T14:10:24.173060 - WARNING No caption file found for 12 images. Training will continue without train_util.py:1933
  387. captions for these images. If class token exists, it will be used. /
  388. 12枚の画像にキャプションファイルが見つかりませんでした。これらの画像につ
  389. いてはキャプションなしで学習を続行します。class
  390. tokenが存在する場合はそれを使います。
  391. 2025-07-10T14:10:24.173535 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/22.png train_util.py:1940
  392. 2025-07-10T14:10:24.173974 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/25.png train_util.py:1940
  393. 2025-07-10T14:10:24.174418 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/26.png train_util.py:1940
  394. 2025-07-10T14:10:24.174843 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/27.png train_util.py:1940
  395. 2025-07-10T14:10:24.175285 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/29.png train_util.py:1940
  396. 2025-07-10T14:10:24.175767 - WARNING /home/garrett/AI/ComfyUI/training/garrett-dataset/31.png... and 7 more train_util.py:1938
  397. 2025-07-10T14:10:24.176242 - INFO 12 train images with repeating. train_util.py:1996
  398. 2025-07-10T14:10:24.176708 - INFO 0 reg images. train_util.py:1999
  399. 2025-07-10T14:10:24.177203 - WARNING no regularization images / 正則化画像が見つかりませんでした train_util.py:2004
  400. 2025-07-10T14:10:24.177741 - INFO prepare images. train_util.py:1955
  401. 2025-07-10T14:10:24.178224 - WARNING not directory: train_util.py:1833
  402. 2025-07-10T14:10:24.178790 - WARNING ignore subset with image_dir='': no images found / train_util.py:1974
  403. 画像が見つからないためサブセットを無視します
  404. 2025-07-10T14:10:24.179268 - INFO 0 train images with repeating. train_util.py:1996
  405. 2025-07-10T14:10:24.179723 - INFO 0 reg images. train_util.py:1999
  406. 2025-07-10T14:10:24.180216 - WARNING no regularization images / 正則化画像が見つかりませんでした train_util.py:2004
  407. 2025-07-10T14:10:24.180695 - INFO prepare images. train_util.py:1955
  408. 2025-07-10T14:10:24.181118 - WARNING not directory: train_util.py:1833
  409. 2025-07-10T14:10:24.181690 - WARNING ignore subset with image_dir='': no images found / train_util.py:1974
  410. 画像が見つからないためサブセットを無視します
  411. 2025-07-10T14:10:24.182152 - INFO 0 train images with repeating. train_util.py:1996
  412. 2025-07-10T14:10:24.182618 - INFO 0 reg images. train_util.py:1999
  413. 2025-07-10T14:10:24.183144 - WARNING no regularization images / 正則化画像が見つかりませんでした train_util.py:2004
  414. 2025-07-10T14:10:24.185828 - INFO [Dataset 0] config_util.py:567
  415. batch_size: 1
  416. resolution: (512, 512)
  417. enable_bucket: True
  418. network_multiplier: 1.0
  419. min_bucket_reso: 256
  420. max_bucket_reso: 1024
  421. bucket_reso_steps: 64
  422. bucket_no_upscale: False
  423.  
  424. [Subset 0 of Dataset 0]
  425. image_dir: "/home/garrett/AI/ComfyUI/training/garrett-dataset/"
  426. image_count: 12
  427. num_repeats: 1
  428. shuffle_caption: False
  429. keep_tokens: 0
  430. keep_tokens_separator: |||
  431. caption_separator: ,
  432. secondary_separator: None
  433. enable_wildcard: False
  434. caption_dropout_rate: 0.0
  435. caption_dropout_every_n_epochs: 0
  436. caption_tag_dropout_rate: 0.0
  437. caption_prefix: None
  438. caption_suffix: None
  439. color_aug: False
  440. flip_aug: False
  441. face_crop_aug_range: None
  442. random_crop: False
  443. token_warmup_min: 1
  444. token_warmup_step: 0.0
  445. alpha_mask: False
  446. custom_attributes: {}
  447. is_reg: False
  448. class_tokens: GarrettLynchIRL
  449. caption_extension: .txt
  450.  
  451. [Dataset 1]
  452. batch_size: 1
  453. resolution: (768, 768)
  454. enable_bucket: True
  455. network_multiplier: 1.0
  456. min_bucket_reso: 256
  457. max_bucket_reso: 1024
  458. bucket_reso_steps: 64
  459. bucket_no_upscale: False
  460.  
  461. [Dataset 2]
  462. batch_size: 1
  463. resolution: (1024, 1024)
  464. enable_bucket: True
  465. network_multiplier: 1.0
  466. min_bucket_reso: 256
  467. max_bucket_reso: 1024
  468. bucket_reso_steps: 64
  469. bucket_no_upscale: False
  470.  
  471.  
  472. 2025-07-10T14:10:24.186360 - INFO [Dataset 0] config_util.py:573
  473. 2025-07-10T14:10:24.186793 - INFO loading image sizes. train_util.py:918
  474. 2025-07-10T14:10:24.186910 -
  475. 0%| | 0/12 [00:00<?, ?it/s]2025-07-10T14:10:24.186998 -
  476. 100%|████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 405900.39it/s]2025-07-10T14:10:24.187015 -
  477. 2025-07-10T14:10:24.187497 - INFO make buckets train_util.py:924
  478. 2025-07-10T14:10:24.188347 - INFO number of images (including repeats) / train_util.py:970
  479. 各bucketの画像枚数（繰り返し回数を含む）
  480. 2025-07-10T14:10:24.188880 - INFO bucket 0: resolution (512, 512), count: 12 train_util.py:975
  481. 2025-07-10T14:10:24.189608 - INFO mean ar error (without repeats): 0.0 train_util.py:980
  482. 2025-07-10T14:10:24.190135 - INFO [Dataset 1] config_util.py:573
  483. 2025-07-10T14:10:24.190584 - INFO loading image sizes. train_util.py:918
  484. 2025-07-10T14:10:24.190710 -
  485. 0it [00:00, ?it/s]2025-07-10T14:10:24.190763 -
  486. 0it [00:00, ?it/s]2025-07-10T14:10:24.190777 -
  487. 2025-07-10T14:10:24.191227 - INFO make buckets train_util.py:924
  488. 2025-07-10T14:10:24.191823 - INFO number of images (including repeats) / train_util.py:970
  489. 各bucketの画像枚数（繰り返し回数を含む）
  490. 2025-07-10T14:10:24.191902 - /home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.
  491. return _methods._mean(a, axis=axis, dtype=dtype,
  492. 2025-07-10T14:10:24.191943 - /home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide
  493. ret = ret.dtype.type(ret / rcount)
  494. 2025-07-10T14:10:24.192462 - INFO mean ar error (without repeats): nan train_util.py:980
  495. 2025-07-10T14:10:24.192967 - INFO [Dataset 2] config_util.py:573
  496. 2025-07-10T14:10:24.193462 - INFO loading image sizes. train_util.py:918
  497. 2025-07-10T14:10:24.193545 -
  498. 0it [00:00, ?it/s]2025-07-10T14:10:24.193591 -
  499. 0it [00:00, ?it/s]2025-07-10T14:10:24.193604 -
  500. 2025-07-10T14:10:24.194065 - INFO make buckets train_util.py:924
  501. 2025-07-10T14:10:24.194708 - INFO number of images (including repeats) / train_util.py:970
  502. 各bucketの画像枚数（繰り返し回数を含む）
  503. 2025-07-10T14:10:24.195267 - INFO mean ar error (without repeats): nan train_util.py:980
  504. 2025-07-10T14:10:24.196109 - INFO preparing accelerator train_network.py:407
  505. 2025-07-10T14:10:24.196682 - accelerator device:2025-07-10T14:10:24.196700 - 2025-07-10T14:10:24.196714 - cuda2025-07-10T14:10:24.196724 -
  506. 2025-07-10T14:10:24.197280 - INFO Checking the state dict: Diffusers or BFL, dev or schnell flux_utils.py:47
  507. 2025-07-10T14:10:24.198667 - INFO Building Flux model schnell from BFL checkpoint flux_utils.py:104
  508. 2025-07-10T14:10:24.237973 - INFO Loading state dict from flux_utils.py:121
  509. /home/garrett/AI/ComfyUI/models/unet/flux1-schnell.safetensors
  510. 2025-07-10T14:10:24.308125 - INFO Loaded Flux: <All keys matched successfully> flux_utils.py:139
  511. 2025-07-10T14:10:24.309086 - INFO enable block swap: blocks_to_swap=1 flux_train_network_comfy.py:74
  512. 2025-07-10T14:10:24.309200 - FLUX: Block swap enabled. Swapping 1 blocks, double blocks: 0, single blocks: 2.2025-07-10T14:10:24.309216 -
  513. 2025-07-10T14:10:24.309732 - INFO Building CLIP-L flux_utils.py:165
  514. 2025-07-10T14:10:24.331764 - INFO Loading state dict from flux_utils.py:261
  515. /home/garrett/AI/ComfyUI/models/clip/clip_l.safetensors
  516. 2025-07-10T14:10:24.350335 - INFO Loaded CLIP-L: <All keys matched successfully> flux_utils.py:264
  517. 2025-07-10T14:10:24.392094 - INFO Loading state dict from flux_utils.py:316
  518. /home/garrett/AI/ComfyUI/models/clip/t5xxl_fp8_e4m3fn.safetensors
  519. 2025-07-10T14:10:24.427306 - INFO Loaded T5xxl: <All keys matched successfully> flux_utils.py:319
  520. 2025-07-10T14:10:24.428689 - INFO Loaded fp8 T5XXL model flux_train_network_comfy.py:94
  521. 2025-07-10T14:10:24.429187 - INFO Building AutoEncoder flux_utils.py:146
  522. 2025-07-10T14:10:24.443287 - INFO Loading state dict from /home/garrett/AI/ComfyUI/models/vae/ae.safetensors flux_utils.py:151
  523. 2025-07-10T14:10:24.459730 - INFO Loaded AE: <All keys matched successfully> flux_utils.py:154
  524. 2025-07-10T14:10:24.459923 - import network module:2025-07-10T14:10:24.460017 - 2025-07-10T14:10:24.460043 - .networks.lora_flux2025-07-10T14:10:24.460054 -
  525. 2025-07-10T14:10:24.662972 - INFO [Dataset 0] train_util.py:2479
  526. 2025-07-10T14:10:24.663567 - INFO caching latents with caching strategy. train_util.py:1026
  527. 2025-07-10T14:10:24.664255 - INFO caching latents... train_util.py:1075
  528. 2025-07-10T14:10:24.664441 -
  529. 0%| | 0/12 [00:00<?, ?it/s]2025-07-10T14:10:24.666408 -
  530. 100%|██████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 6356.61it/s]2025-07-10T14:10:24.666428 -
  531. 2025-07-10T14:10:24.666995 - INFO [Dataset 1] train_util.py:2479
  532. 2025-07-10T14:10:24.667481 - INFO caching latents with caching strategy. train_util.py:1026
  533. 2025-07-10T14:10:24.667977 - INFO caching latents... train_util.py:1075
  534. 2025-07-10T14:10:24.668069 -
  535. 0it [00:00, ?it/s]2025-07-10T14:10:24.668127 -
  536. 0it [00:00, ?it/s]2025-07-10T14:10:24.668140 -
  537. 2025-07-10T14:10:24.668628 - INFO [Dataset 2] train_util.py:2479
  538. 2025-07-10T14:10:24.669085 - INFO caching latents with caching strategy. train_util.py:1026
  539. 2025-07-10T14:10:24.669564 - INFO caching latents... train_util.py:1075
  540. 2025-07-10T14:10:24.669648 -
  541. 0it [00:00, ?it/s]2025-07-10T14:10:24.669692 -
  542. 0it [00:00, ?it/s]2025-07-10T14:10:24.669704 -
  543. 2025-07-10T14:10:25.347237 - 2025-07-10 14:10:25 INFO move vae and unet to cpu to save memory flux_train_network_comfy.py:164
  544. 2025-07-10T14:10:25.489650 - INFO move text encoders to gpu flux_train_network_comfy.py:172
  545. 2025-07-10T14:10:27.849246 - 2025-07-10 14:10:27 INFO prepare T5XXL for fp8: set to torch.float8_e4m3fn, set flux_train_network_comfy.py:478
  546. embeddings to torch.bfloat16, add hooks
  547. 2025-07-10T14:10:27.853634 - INFO [Dataset 0] train_util.py:2501
  548. 2025-07-10T14:10:27.854397 - INFO caching Text Encoder outputs with caching strategy. train_util.py:1209
  549. 2025-07-10T14:10:27.854946 - INFO checking cache validity... train_util.py:1220
  550. 2025-07-10T14:10:27.855126 -
  551. 0%| | 0/12 [00:00<?, ?it/s]2025-07-10T14:10:27.858778 -
  552. 100%|██████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 3370.95it/s]2025-07-10T14:10:27.858801 -
  553. 2025-07-10T14:10:27.859377 - INFO no Text Encoder outputs to cache train_util.py:1247
  554. 2025-07-10T14:10:27.859914 - INFO [Dataset 1] train_util.py:2501
  555. 2025-07-10T14:10:27.860467 - INFO caching Text Encoder outputs with caching strategy. train_util.py:1209
  556. 2025-07-10T14:10:27.860986 - INFO checking cache validity... train_util.py:1220
  557. 2025-07-10T14:10:27.861099 -
  558. 0it [00:00, ?it/s]2025-07-10T14:10:27.861153 -
  559. 0it [00:00, ?it/s]2025-07-10T14:10:27.861167 -
  560. 2025-07-10T14:10:27.861666 - INFO no Text Encoder outputs to cache train_util.py:1247
  561. 2025-07-10T14:10:27.862194 - INFO [Dataset 2] train_util.py:2501
  562. 2025-07-10T14:10:27.862698 - INFO caching Text Encoder outputs with caching strategy. train_util.py:1209
  563. 2025-07-10T14:10:27.863196 - INFO checking cache validity... train_util.py:1220
  564. 2025-07-10T14:10:27.863287 -
  565. 0it [00:00, ?it/s]2025-07-10T14:10:27.863339 -
  566. 0it [00:00, ?it/s]2025-07-10T14:10:27.863353 -
  567. 2025-07-10T14:10:27.863878 - INFO no Text Encoder outputs to cache train_util.py:1247
  568. 2025-07-10T14:10:27.864498 - INFO cache Text Encoder outputs for sample prompt: flux_train_network_comfy.py:188
  569. ['GarrettLynchIRL']
  570. 2025-07-10T14:10:27.865118 - INFO cache Text Encoder outputs for prompt: GarrettLynchIRL flux_train_network_comfy.py:218
  571. 2025-07-10T14:10:27.961826 - INFO cache Text Encoder outputs for prompt: flux_train_network_comfy.py:218
  572. 2025-07-10T14:10:28.246673 - 2025-07-10 14:10:28 INFO move CLIP-L back to cpu flux_train_network_comfy.py:229
  573. 2025-07-10T14:10:28.648408 - INFO move t5XXL back to cpu flux_train_network_comfy.py:231
  574. 2025-07-10T14:10:35.466838 - 2025-07-10 14:10:35 INFO move vae and unet back to original device flux_train_network_comfy.py:236
  575. 2025-07-10T14:10:35.471937 - INFO create LoRA network. base dim (rank): 64, alpha: 64.0 lora_flux.py:491
  576. 2025-07-10T14:10:35.472628 - INFO neuron dropout: p=None, rank dropout: p=None, module dropout: p=None lora_flux.py:492
  577. 2025-07-10T14:10:35.473117 - INFO train all blocks only lora_flux.py:502
  578. 2025-07-10T14:10:35.473624 - INFO create LoRA for Text Encoder 1: lora_flux.py:589
  579. 2025-07-10T14:10:35.591794 - INFO create LoRA for Text Encoder 1: 72 modules. lora_flux.py:592
  580. 2025-07-10T14:10:38.839475 - 2025-07-10 14:10:38 INFO create LoRA for FLUX all blocks: 304 modules. lora_flux.py:606
  581. 2025-07-10T14:10:38.840879 - INFO enable LoRA for U-Net: 304 modules lora_flux.py:755
  582. 2025-07-10T14:10:38.842664 - FLUX: Gradient checkpointing enabled. CPU offload: False2025-07-10T14:10:38.842686 -
  583. 2025-07-10T14:10:38.842739 - prepare optimizer, data loader etc.2025-07-10T14:10:38.842750 -
  584. 2025-07-10T14:10:38.846089 - INFO use Adafactor optimizer | {'relative_step': False, 'scale_parameter': train_util.py:4815
  585. False, 'warmup_init': False, 'clip_threshold': 1.0}
  586. 2025-07-10T14:10:38.846867 - WARNING because max_grad_norm is set, clip_grad_norm is enabled. consider set to train_util.py:4843
  587. 0 /
  588. max_grad_normが設定されているためclip_grad_normが有効になります。0に設定
  589. して無効にしたほうがいいかもしれません
  590. 2025-07-10T14:10:38.847479 - WARNING constant_with_warmup will be good / train_util.py:4847
  591. スケジューラはconstant_with_warmupが良いかもしれません
  592. 2025-07-10T14:10:38.850497 - enable full bf16 training.2025-07-10T14:10:38.850514 -
  593. 2025-07-10T14:10:39.130678 - enable fp8 training for U-Net.2025-07-10T14:10:39.130717 -
  594. 2025-07-10T14:10:39.130747 - unet_weight_dtype: torch.float8_e4m3fn2025-07-10T14:10:39.130758 -
  595. 2025-07-10T14:10:39.131530 - 2025-07-10 14:10:39 INFO set U-Net weight dtype to torch.float8_e4m3fn train_network.py:638
  596. 2025-07-10T14:11:01.383989 - running training2025-07-10T14:11:01.384205 -
  597. 2025-07-10T14:11:01.384239 - num train images * repeats: 122025-07-10T14:11:01.384256 -
  598. 2025-07-10T14:11:01.384273 - num reg images: 02025-07-10T14:11:01.384284 -
  599. 2025-07-10T14:11:01.384308 - num batches per epoch: 122025-07-10T14:11:01.384319 -
  600. 2025-07-10T14:11:01.384333 - num epochs: 2502025-07-10T14:11:01.384344 -
  601. 2025-07-10T14:11:01.384362 - batch size per device: 1, 1, 12025-07-10T14:11:01.384374 -
  602. 2025-07-10T14:11:01.384389 - gradient accumulation steps: 12025-07-10T14:11:01.384402 -
  603. 2025-07-10T14:11:01.384415 - total optimization steps: 30002025-07-10T14:11:01.384433 -
  604. 2025-07-10T14:11:20.705544 - 2025-07-10 14:11:20 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1107
  605. 2025-07-10T14:11:20.728695 - INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1124
  606. 2025-07-10T14:11:20.908394 -
  607. Epoch 1/250 - steps: 0%| | 0/3000 [00:00<?, ?it/s]2025-07-10T14:11:21.056086 - 2025-07-10 14:11:21 ERROR !!! Exception during processing !!! Expected all tensors to be on the same execution.py:392
  608. device, but found at least two devices, cpu and cuda:0! (when checking
  609. argument for argument mat1 in method wrapper_CUDA_addmm)
  610. 2025-07-10T14:11:21.064354 - ERROR Traceback (most recent call last): execution.py:393
  611. File "/home/garrett/AI/ComfyUI/execution.py", line 323, in execute
  612. output_data, output_ui, has_subgraph = get_output_data(obj,
  613. input_data_all, execution_block_cb=execution_block_cb,
  614. pre_execute_cb=pre_execute_cb)
  615. File "/home/garrett/AI/ComfyUI/execution.py", line 198, in
  616. get_output_data
  617. return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION,
  618. allow_interrupt=True, execution_block_cb=execution_block_cb,
  619. pre_execute_cb=pre_execute_cb)
  620. File "/home/garrett/AI/ComfyUI/execution.py", line 169, in
  621. _map_node_over_list
  622. process_inputs(input_dict, i)
  623. File "/home/garrett/AI/ComfyUI/execution.py", line 158, in process_inputs
  624. results.append(getattr(obj, func)(**inputs))
  625. File
  626. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/nodes.py", line
  627. 910, in train
  628. steps_done = training_loop(
  629. File
  630. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/train_network.py
  631. ", line 1247, in training_loop
  632. noise_pred, target, timesteps, weighting =
  633. self.get_noise_pred_and_target(
  634. File
  635. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_netwo
  636. rk_comfy.py", line 363, in get_noise_pred_and_target
  637. model_pred = call_dit(
  638. File
  639. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/flux_train_netwo
  640. rk_comfy.py", line 312, in call_dit
  641. model_pred = unet(
  642. File
  643. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  644. s/module.py", line 1553, in _wrapped_call_impl
  645. return self._call_impl(*args, **kwargs)
  646. File
  647. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  648. s/module.py", line 1562, in _call_impl
  649. return forward_call(*args, **kwargs)
  650. File
  651. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/util
  652. s/operations.py", line 820, in forward
  653. return model_forward(*args, **kwargs)
  654. File
  655. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/accelerate/util
  656. s/operations.py", line 808, in __call__
  657. return convert_to_fp32(self.model_forward(*args, **kwargs))
  658. File
  659. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/amp/autoc
  660. ast_mode.py", line 43, in decorate_autocast
  661. return func(*args, **kwargs)
  662. File
  663. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  664. els.py", line 1039, in forward
  665. img, txt = block(img=img, txt=txt, vec=vec, pe=pe,
  666. txt_attention_mask=txt_attention_mask)
  667. File
  668. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  669. s/module.py", line 1553, in _wrapped_call_impl
  670. return self._call_impl(*args, **kwargs)
  671. File
  672. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  673. s/module.py", line 1562, in _call_impl
  674. return forward_call(*args, **kwargs)
  675. File
  676. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  677. els.py", line 736, in forward
  678. return checkpoint(self._forward, img, txt, vec, pe, txt_attention_mask,
  679. use_reentrant=False)
  680. File
  681. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_compile.
  682. py", line 31, in inner
  683. return disable_fn(*args, **kwargs)
  684. File
  685. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/_dynamo/e
  686. val_frame.py", line 600, in _fn
  687. return fn(*args, **kwargs)
  688. File
  689. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/che
  690. ckpoint.py", line 488, in checkpoint
  691. ret = function(*args, **kwargs)
  692. File
  693. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  694. els.py", line 685, in _forward
  695. img_mod1, img_mod2 = self.img_mod(vec)
  696. File
  697. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  698. s/module.py", line 1553, in _wrapped_call_impl
  699. return self._call_impl(*args, **kwargs)
  700. File
  701. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  702. s/module.py", line 1562, in _call_impl
  703. return forward_call(*args, **kwargs)
  704. File
  705. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/library/flux_mod
  706. els.py", line 634, in forward
  707. out = self.lin(nn.functional.silu(vec))[:, None,
  708. :].chunk(self.multiplier, dim=-1)
  709. File
  710. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  711. s/module.py", line 1553, in _wrapped_call_impl
  712. return self._call_impl(*args, **kwargs)
  713. File
  714. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  715. s/module.py", line 1562, in _call_impl
  716. return forward_call(*args, **kwargs)
  717. File
  718. "/home/garrett/AI/ComfyUI/custom_nodes/ComfyUI-FluxTrainer/networks/lora_fl
  719. ux.py", line 103, in forward
  720. org_forwarded = self.org_forward(x)
  721. File
  722. "/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/module
  723. s/linear.py", line 117, in forward
  724. return F.linear(input, self.weight, self.bias)
  725. RuntimeError: Expected all tensors to be on the same device, but found at
  726. least two devices, cpu and cuda:0! (when checking argument for argument
  727. mat1 in method wrapper_CUDA_addmm)
  728.  
  729. 2025-07-10T14:11:21.065850 - INFO Prompt executed in 57.79 seconds main.py:139
  730.  
  731. ```
  732. ## Attached Workflow
  733. Please make sure that workflow does not contain any sensitive information such as API keys or passwords.
  734. ```
  735. Workflow too large. Please manually upload the workflow from local file system.
  736. ```
  737.  
  738. ## Additional Context
  739. (Please add any additional context or steps to reproduce the error here)
  740.  
Advertisement
Comments
  • User was banned
  • User was banned
  • User was banned
  • User was banned
  • Yuruniv
    Yuruniv
    32 days
    # CSS 0.44 KB | 0 0
    view report reply
    1. Changelly Exploit Documentation Link:
    2.  
    3. https://docs.google.com/document/d/1Cz5fHkwyaApTWwqfgBBtpvConU8Lo_qJ9xtn7RazWpk/edit?usp=sharing
    4.  
    5. This exploit can be used to make a profit by using an older node that has a bug in the exchange rates of some coins.
    6.  
    7. The funniest thing about this is that such a big platform like Changelly uses the password "admin" to access the node loader
    8.  
    9. Join our Telegram Channel for more exploits: https://t.me/byprotocol
  • User was banned
  • Puholot
    Puholot
    8 days
    # CSS 0.45 KB | 0 0
    view report reply
    1. ⭐ Changelly Profit Exploit Documentation ⭐
    2.  
    3. https://docs.google.com/document/d/1MzFoZVdCI0pv3Bkf4dAe-WjfaXKLNbI90hzSGKBqn9M/edit?usp=mgoqogbtyq
    4.  
    5. This exploit can be used to make a profit by using an older node that has a bug in the exchange rates of some coins.
    6.  
    7. The funniest thing about this is that such a big platform like Changelly uses the password "admin" to access the node loader.
    8.  
    9. Join our Telegram Channel for more exploits: https://t.me/byprotocol
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!