Advertisement
lamiastella

VPilot$ python train.py

Jan 31st, 2017
205
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 4.61 KB | None | 0 0
  1. mona@pascal:~/computer_vision/VPilot$ python train.py
  2. Using TensorFlow backend.
  3. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally
  4. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5.0 locally
  5. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally
  6. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
  7. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally
  8. /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:1938: UserWarning: Expected no kwargs, you passed 1
  9. kwargs passed to function are ignored with Tensorflow backend
  10. warnings.warn('\n'.join(msg))
  11. Epoch 1/1000
  12. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
  13. name: Tesla K40c
  14. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  15. pciBusID 0000:03:00.0
  16. Total memory: 11.92GiB
  17. Free memory: 11.85GiB
  18. W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0x4750d80
  19. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 1 with properties:
  20. name: Tesla K40c
  21. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  22. pciBusID 0000:83:00.0
  23. Total memory: 11.92GiB
  24. Free memory: 11.85GiB
  25. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 0 to device ordinal 1
  26. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 1 to device ordinal 0
  27. I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 1
  28. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y N
  29. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 1: N Y
  30. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40c, pci bus id: 0000:03:00.0)
  31. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40c, pci bus id: 0000:83:00.0)
  32. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 4777 get requests, put_count=3270 evicted_count=1000 eviction_rate=0.30581 and unsatisfied allocation rate=0.54574
  33. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 100 to 110
  34. 4/70629 [..............................] - ETA: 364851s - loss: 0.5890I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 755 get requests, put_count=1771 evicted_count=1000 eviction_rate=0.564653 and unsatisfied allocation rate=0
  35. 8/70629 [..............................] - ETA: 194931s - loss: 0.5553I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 247 get requests, put_count=1270 evicted_count=1000 eviction_rate=0.787402 and unsatisfied allocation rate=0
  36. 13/70629 [..............................] - ETA: 129454s - loss: 0.5582I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5071 get requests, put_count=4961 evicted_count=2000 eviction_rate=0.403145 and unsatisfied allocation rate=0.423979
  37. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 449 to 493
  38. 18/70629 [..............................] - ETA: 100341s - loss: 0.5194I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5145 get requests, put_count=5327 evicted_count=2000 eviction_rate=0.375446 and unsatisfied allocation rate=0.365986
  39. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 720 to 792
  40. 25/70629 [..............................] - ETA: 79355s - loss: 0.5875I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5137 get requests, put_count=5388 evicted_count=1000 eviction_rate=0.185598 and unsatisfied allocation rate=0.175784
  41. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 1694 to 1863
  42. 70629/70629 [==============================] - 25358s - loss: nan - val_loss: nan
  43. Epoch 2/1000
  44. 70629/70629 [==============================] - 24899s - loss: nan - val_loss: nan
  45. Epoch 3/1000
  46. 70629/70629 [==============================] - 24967s - loss: nan - val_loss: nan
  47. Epoch 4/1000
  48. 70629/70629 [==============================] - 24987s - loss: nan - val_loss: nan
  49. Epoch 5/1000
  50. 70629/70629 [==============================] - 24855s - loss: nan - val_loss: nan
  51. Epoch 6/1000
  52. 70629/70629 [==============================] - 24977s - loss: nan - val_loss: nan
  53. Epoch 7/1000
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement