API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Oct 19th, 2019
156
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 29.88 KB | None | 0 0
raw download clone embed print report
  1. 2019-10-16T19:40:16.508Z ESC[34mINFOESC[0m mrtx metrics/metrics.go:144 metrics updated {"values": {"ram":1875390464,"cpu":333.6065573770625,"cuda":"10.1","gpu_drv":"430.40","GPUs":[{"name":"GeForce GTX 1080 Ti","vram_total":11176000000,"vram_used":1181000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":1164000000}]}}
  2. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.31 | Remaining:00:57.58 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 101/390 Tiles
  3. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.34 | Remaining:00:57.41 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 102/390 Tiles
  4. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.49 | Remaining:00:57.22 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 103/390 Tiles
  5. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.63 | Remaining:00:57.00 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 104/390 Tiles
  6. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.67 | Remaining:00:57.00 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 105/390 Tiles
  7. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:26.93 | Remaining:00:56.60 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 106/390 Tiles
  8. 2019-10-16T19:40:17.881Z ESC[35mDEBUGESC[0m worker/worker.go:47 sending heartbeat
  9. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  10.  
  11. Refer to the Cycles GPU rendering documentation for possible solutions:
  12. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  13.  
  14. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  15. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  16.  
  17. Refer to the Cycles GPU rendering documentation for possible solutions:
  18. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  19.  
  20. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  21. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  22. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  23. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  24. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  25. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  26. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  27. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  28. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  29. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  30. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  31. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  32. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  33. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  34. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  35. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  36.  
  37. Refer to the Cycles GPU rendering documentation for possible solutions:
  38. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  39.  
  40. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  41. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  42. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  43. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  44. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  45. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  46. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  47. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  48. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  49. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  50.  
  51. Refer to the Cycles GPU rendering documentation for possible solutions:
  52. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  53.  
  54. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  55.  
  56. Refer to the Cycles GPU rendering documentation for possible solutions:
  57. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  58.  
  59. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  60. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  61. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  62. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  63. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  64. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  65. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  66. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  67. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:54.38 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 107/390 Tiles
  68. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  69. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  70.  
  71. Refer to the Cycles GPU rendering documentation for possible solutions:
  72. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  73.  
  74. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  75.  
  76. Refer to the Cycles GPU rendering documentation for possible solutions:
  77. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  78.  
  79. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  80.  
  81. Refer to the Cycles GPU rendering documentation for possible solutions:
  82. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  83.  
  84. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  85. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  86. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  87. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  88. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  89. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  90. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  91. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  92. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  93. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  94. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  95. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  96. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  97.  
  98. Refer to the Cycles GPU rendering documentation for possible solutions:
  99. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  100. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  101.  
  102. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  103. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  104. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  105. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  106. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  107. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  108. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  109. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  110. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  111. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  112. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  113. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  114. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.73 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 108/390 Tiles
  115. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  116.  
  117. Refer to the Cycles GPU rendering documentation for possible solutions:
  118. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  119.  
  120. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  121. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  122. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  123. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  124. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.51 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 109/390 Tiles
  125. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  126. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.51 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 110/390 Tiles
  127. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  128. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  129. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  130. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  131. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  132. CUDA error: Launch exceeded timeout in cuModuleGetGlobal(&cumem, &cubytes, cuModule, bind_name.c_str()), line 1068
  133. CUDA error: Launch exceeded timeout in cuMemcpyHtoD(cumem, (void *)&ptr, cubytes), line 1078
  134. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  135. CUDA error: Launch exceeded timeout in cuModuleGetGlobal(&cumem, &cubytes, cuModule, bind_name.c_str()), line 1068
  136. CUDA error: Launch exceeded timeout in cuMemcpyHtoD(cumem, (void *)&ptr, cubytes), line 1078
  137. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  138. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  139. Fra:166 Mem:44.35M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.43 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 111/390 Tiles
  140. CUDA error: Launch exceeded timeout in cuModuleGetGlobal(&cumem, &cubytes, cuModule, bind_name.c_str()), line 1068
  141. CUDA error: Launch exceeded timeout in cuMemcpyHtoD(cumem, (void *)&ptr, cubytes), line 1078
  142. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  143. CUDA error: Launch exceeded timeout in cuModuleGetGlobal(&cumem, &cubytes, cuModule, bind_name.c_str()), line 1068
  144. CUDA error: Launch exceeded timeout in cuMemcpyHtoD(cumem, (void *)&ptr, cubytes), line 1078
  145. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  146. CUDA error: Launch exceeded timeout in cuModuleGetGlobal(&cumem, &cubytes, cuModule, bind_name.c_str()), line 1068
  147. CUDA error: Launch exceeded timeout in cuMemcpyHtoD(cumem, (void *)&ptr, cubytes), line 1078
  148. =======
  149. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory
  150. _size()), line 892
  151. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0,
  152. 0, args, 0), line 1798
  153. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  154. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory
  155. _size()), line 892
  156. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  157. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  158. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  159. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  160. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  161. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  162. Fra:166 Mem:44.22M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.07 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 127/390 Tiles
  163. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  164. Fra:166 Mem:44.22M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.07 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 128/390 Tiles
  165. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  166. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  167. Fra:166 Mem:44.22M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.07 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 129/390 Tiles
  168. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  169. Fra:166 Mem:44.22M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.07 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 130/390 Tiles
  170. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  171. Fra:166 Mem:44.22M (0.00M, Peak 47.00M) | Time:00:27.01 | Remaining:00:52.07 | Mem:1.53M, Peak:
  172. ===============
  173. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Lay
  174. er | Rendered 251/390 Tiles
  175. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  176. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  177. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  178. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  179. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  180. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  181. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  182. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  183. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  184. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  185. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  186. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  187. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Lay
  188. er | Rendered 252/390 Tiles
  189. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Lay
  190. er | Rendered 253/390 Tiles
  191. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Lay
  192. er | Rendered 254/390 Tiles
  193. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Lay
  194. er | Rendered 255/390 Tiles
  195. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  196. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  197. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  198. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  199. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  200. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  201. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  202. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  203. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  204. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  205. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  206. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  207. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  208. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  209. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  210. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  211. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  212. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  213. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  214. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  215. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  216. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  217. CUDA error: Launch exceeded timeout in mem_alloc_result, line 861
  218. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 256/390 Tiles
  219. Fra:166 Mem:44.10M (0.00M, Peak 47.00M) | Time:00:27.02 | Remaining:00:52.09 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 257/390 Tiles
  220. =========================
  221. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  222. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  223.  
  224. Refer to the Cycles GPU rendering documentation for possible solutions:
  225. https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
  226.  
  227. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  228. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  229. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  230. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  231. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  232. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  233. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  234. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  235. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  236. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  237. CUDA error: Launch exceeded timeout in cuMemcpyHtoD( cuda_device_ptr(mem.device_pointer), mem.host_pointer, mem.memory_size()), line 892
  238. CUDA error: Launch exceeded timeout in cuLaunchKernel( cuPathTrace, num_blocks, 1, 1, num_threads_per_block, 1, 1, 0, 0, args, 0), line 1798
  239. CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  240. CUDA error: Launch exceeded timeout in cuMemcpyDtoH( (uchar *)mem.host_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size), line 974
  241. Fra:166 Mem:42.97M (0.00M, Peak 47.00M) | Time:00:27.52 | Remaining:00:52.36 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Rendered 390/390 Tiles
  242. Error: CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  243. Fra:166 Mem:42.97M (0.00M, Peak 47.00M) | Time:00:27.52 | Remaining:00:52.36 | Mem:1.53M, Peak:1.53M | Scene, View Layer | Cancel | CUDA error: Launch exceeded timeout in cuCtxSynchronize(), line 1800
  244. Fra:166 Mem:41.36M (0.00M, Peak 47.00M) | Time:00:28.98 | Sce: Scene Ve:0 Fa:0 La:0
  245. ARGv ['{"scene":"Scene","camera":"Camera.006","res_x":1920,"res_y":816,"samples":256,"run_mode":"render","result_dir":"/home/task_10069/frame_166/result/tmp/"}']
  246. ARGs {"scene":"Scene","camera":"Camera.006","res_x":1920,"res_y":816,"samples":256,"run_mode":"render","result_dir":"/home/task_10069/frame_166/result/tmp/"}
  247. params {'scene': 'Scene', 'camera': 'Camera.006', 'res_x': 1920, 'res_y': 816, 'samples': 256, 'run_mode': 'render', 'result_dir': '/home/task_10069/frame_166/result/tmp/'}
  248. [SONM] Rendering scene `Scene`, camera `Camera.006`, 1920x816 256 smpl, (mode = render)
  249. [SONM] camera found: `<bpy_struct, Object("Camera.006")>`
  250. [SONM] scene found: `<bpy_struct, Scene("Scene")>`
  251. [SONM] result dir: `/home/task_10069/frame_166/result/tmp/`
  252. [SONM] file_format = OPEN_EXR, color_mode = RGB
  253. [SONM] enabling all of 13 devices
  254. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  255. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  256. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  257. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  258. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  259. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  260. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  261. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  262. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  263. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  264. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  265. [SONM] trying to enable device "GeForce GTX 1080 Ti"
  266. [SONM] trying to enable device "Intel Core i5-7400 CPU @ 3.00GHz"
  267. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  268. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  269. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  270. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  271. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  272. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  273. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  274. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  275. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  276. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  277. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  278. [SONM] device "GeForce GTX 1080 Ti" enabled? True
  279. [SONM] device "Intel Core i5-7400 CPU @ 3.00GHz" enabled? False
  280.  
  281. Blender quit
  282. 2019-10-16T19:40:19.920Z ESC[31mERRORESC[0m blender/blender.go:187 blender process failed {"error": "exit status 1"}
  283. github.com/sonm-io/services/worker-blender/worker/blender.(*blenderExecutor).callBlender
  284. /home/buildagent1/buildagent/work/34565fd52bef9900/worker-blender/worker/blender/blender.go:187
  285. github.com/sonm-io/services/worker-blender/worker/blender.(*blenderExecutor).Execute
  286. /home/buildagent1/buildagent/work/34565fd52bef9900/worker-blender/worker/blender/blender.go:111
  287. github.com/sonm-io/services/worker-blender/worker.(*Worker).Start
  288. /home/buildagent1/buildagent/work/34565fd52bef9900/worker-blender/worker/worker.go:135
  289. main.main.func2
  290. /home/buildagent1/buildagent/work/34565fd52bef9900/worker-blender/cmd/worker/main.go:49
  291. golang.org/x/sync/errgroup.(*Group).Go.func1
  292. /home/buildagent1/go/pkg/mod/golang.org/x/sync@v0.0.0-20190423024810-112230192c58/errgroup/errgroup.go:57
  293. 2019-10-16T19:40:19.920Z ESC[33mWARNESC[0m worker/worker.go:137 failed to process task, sending failure notification {"task": {"Meta":{"task_id":10069,"file_name":"source.blend","camera_name":"Camera.006","scene_name":"Scene","frame_number":166,"src_url":"/media/7fc4601821f64786fc6e9f08b96be4cbdfd4822b312515a810b0f72f07bb754f/1f7acd1e3703/source.blend","resolution_x":1920,"resolution_y":816,"samples":256,"version":"2.81.12","extra":{"filepath_mask":"/tmp/","have_borders":false,"integrator":"PATH"}},"Dirs":{"UID":1000,"GID":1000,"SourceFileName":"source.blend","SourceFilePath":"/home/task_10069/source.file","ResultDirPath":"/home/task_10069/frame_166/result","TempThumbFilePath":"/home/task_10069/frame_166/thumbnails/thumb_tmp.png","ThumbFilePath":"/home/task_10069/frame_166/thumbnails/thumb.png"}}, "error": "failed to process blend frame: exit status 1"}
  294. 2019-10-16T19:40:19.920Z ESC[33mWARNESC[0m http-client api/client.go:274 failing task {"task": {"task_id":10069,"file_name":"source.blend","camera_name":"Camera.006","scene_name":"Scene","frame_number":166,"src_url":"/media/7fc4601821f64786fc6e9f08b96be4cbdfd4822b312515a810b0f72f07bb754f/1f7acd1e3703/source.blend","resolution_x":1920,"resolution_y":816,"samples":256,"version":"2.81.12","extra":{"filepath_mask":"/tmp/","have_borders":false,"integrator":"PATH"}}, "reason": "failed to process blend frame: exit status 1"}
  295. 2019-10-16T19:40:21.442Z ESC[34mINFOESC[0m http-client api/client.go:92 requesting new tasks... {"hardware": {"ram":0,"cpu":0,"cuda":"10.1","gpu_drv":"430.40","GPUs":[{"name":"GeForce GTX 1080 Ti","vram_total":11176000000,"vram_used":36000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000},{"name":"GeForce GTX 1080 Ti","vram_total":11178000000,"vram_used":19000000}]}}
  296. 2019-10-16T19:40:21.443Z ESC[35mDEBUGESC[0m http-client api/client.go:105 performing request {"to": "https://my.raysrender.com/api/worker/get_task", "body": "{\"hardware\":{\"ram\":0,\"cpu\":0,\"cuda\":\"10.1\",\"gpu_drv\":\"430.40\",\"GPUs\":[{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11176000000,\"vram_used\":36000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000},{\"name\":\"GeForce GTX 1080 Ti\",\"vram_total\":11178000000,\"vram_used\":19000000}]},\"secret\":\"30dadc99f4ff1797c7950a80868b47f2:326da77e-3ff3-4db6-ba3e-d211608bfdb3\"}"}
  297. 2019-10-16T19:40:21.523Z ESC[35mDEBUGESC[0m http-client api/client.go:128 received response {"status_code": 401, "body": "{\"error\": \"authentication failed\", \"status\": \"failed\"}"}
  298. 2019-10-16T19:40:21.523Z ESC[33mWARNESC[0m worker/worker.go:110 failed to get new tasks {"error": "api returns `unauthorized` error, secret malformed"}
  299. 2019-10-16T19:40:21.523Z ESC[33mWARNESC[0m worker/main.go:54 termination {"fatal": true, "error": "api returns `unauthorized` error, secret malformed"}
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!