API tools faq
paste
Advertisement
lamiastella

new_train.py

lamiastella
Jan 31st, 2017
222
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.51 KB | None | 0 0
raw download clone embed print report
  1. mona@pascal:~/computer_vision/VPilot$ python train.py
  2. Using TensorFlow backend.
  3. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally
  4. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5.0 locally
  5. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally
  6. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
  7. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally
  8. /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:1938: UserWarning: Expected no kwargs, you passed 1
  9. kwargs passed to function are ignored with Tensorflow backend
  10. warnings.warn('\n'.join(msg))
  11. Epoch 1/1000
  12. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
  13. name: Tesla K40c
  14. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  15. pciBusID 0000:03:00.0
  16. Total memory: 11.92GiB
  17. Free memory: 11.85GiB
  18. W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0xa5e8040
  19. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 1 with properties:
  20. name: Tesla K40c
  21. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  22. pciBusID 0000:83:00.0
  23. Total memory: 11.92GiB
  24. Free memory: 11.85GiB
  25. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 0 to device ordinal 1
  26. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 1 to device ordinal 0
  27. I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 1
  28. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y N
  29. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 1: N Y
  30. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40c, pci bus id: 0000:03:00.0)
  31. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40c, pci bus id: 0000:83:00.0)
  32. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 4776 get requests, put_count=3269 evicted_count=1000 eviction_rate=0.305904 and unsatisfied allocation rate=0.545854
  33. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 100 to 110
  34. 4/70562 [..............................] - ETA: 143828s - loss: 0.9234I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 1041 get requests, put_count=1920 evicted_count=1000 eviction_rate=0.520833 and unsatisfied allocation rate=0.131604
  35. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 176 to 193
  36. 8/70562 [..............................] - ETA: 84324s - loss: 0.9190I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 247 get requests, put_count=1272 evicted_count=1000 eviction_rate=0.786164 and unsatisfied allocation rate=0
  37. 13/70562 [..............................] - ETA: 61579s - loss: 0.8093I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5058 get requests, put_count=4901 evicted_count=2000 eviction_rate=0.40808 and unsatisfied allocation rate=0.433768
  38. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 409 to 449
  39. 18/70562 [..............................] - ETA: 51908s - loss: 0.7416I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 4884 get requests, put_count=4001 evicted_count=1000 eviction_rate=0.249938 and unsatisfied allocation rate=0.397625
  40. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 655 to 720
  41. 24/70562 [..............................] - ETA: 45248s - loss: 0.6623I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5019 get requests, put_count=4681 evicted_count=1000 eviction_rate=mona@pascal:~/computer_vision/VPilot$ python train.py
  42. Using TensorFlow backend.
  43. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so.8.0 locally
  44. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so.5.0 locally
  45. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so.8.0 locally
  46. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
  47. I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so.8.0 locally
  48. /usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py:1938: UserWarning: Expected no kwargs, you passed 1
  49. kwargs passed to function are ignored with Tensorflow backend
  50. warnings.warn('\n'.join(msg))
  51. Epoch 1/1000
  52. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
  53. name: Tesla K40c
  54. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  55. pciBusID 0000:03:00.0
  56. Total memory: 11.92GiB
  57. Free memory: 11.85GiB
  58. W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0xa5e8040
  59. I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 1 with properties:
  60. name: Tesla K40c
  61. major: 3 minor: 5 memoryClockRate (GHz) 0.8755
  62. pciBusID 0000:83:00.0
  63. Total memory: 11.92GiB
  64. Free memory: 11.85GiB
  65. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 0 to device ordinal 1
  66. I tensorflow/core/common_runtime/gpu/gpu_device.cc:855] cannot enable peer access from device ordinal 1 to device ordinal 0
  67. I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 1
  68. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y N
  69. I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 1: N Y
  70. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40c, pci bus id: 0000:03:00.0)
  71. I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40c, pci bus id: 0000:83:00.0)
  72. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 4776 get requests, put_count=3269 evicted_count=1000 eviction_rate=0.305904 and unsatisfied allocation rate=0.545854
  73. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 100 to 110
  74. 4/70562 [..............................] - ETA: 143828s - loss: 0.9234I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 1041 get requests, put_count=1920 evicted_count=1000 eviction_rate=0.520833 and unsatisfied allocation rate=0.131604
  75. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 176 to 193
  76. 8/70562 [..............................] - ETA: 84324s - loss: 0.9190I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 247 get requests, put_count=1272 evicted_count=1000 eviction_rate=0.786164 and unsatisfied allocation rate=0
  77. 13/70562 [..............................] - ETA: 61579s - loss: 0.8093I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5058 get requests, put_count=4901 evicted_count=2000 eviction_rate=0.40808 and unsatisfied allocation rate=0.433768
  78. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 409 to 449
  79. 18/70562 [..............................] - ETA: 51908s - loss: 0.7416I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 4884 get requests, put_count=4001 evicted_count=1000 eviction_rate=0.249938 and unsatisfied allocation rate=0.397625
  80. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 655 to 720
  81. 24/70562 [..............................] - ETA: 45248s - loss: 0.6623I tensorflow/core/common_runtime/gpu/pool_allocator.cc:245] PoolAllocator: After 5019 get requests, put_count=4681 evicted_count=1000 eviction_rate=0.21363 and unsatisfied allocation rate=0.287507
  82. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 1158 to 1273
  83. 70562/70562 [==============================] - 25913s - loss: 0.1742 - val_loss: 0.1014
  84. Epoch 2/1000
  85. 2893/70562 [>.............................] - ETA: 23312s - loss: 0.1579^Z
  86. [7]+ Stopped python train.py
  87. mona@pascal:~/computer_vision/VPilot$ ls
  88. drive.py frames1 frames2 model.h5 model.h5_bk model.h5_bk_not_complete model.py model.pyc __pycache__ README.md train.py
  89. mona@pascal:~/computer_vision/VPilot$ ls -ltr
  90. total 654564
  91. drwxrwxr-x 2 mona mona 4096 Jan 18 18:46 __pycache__
  92. -rw-rw-r-- 1 mona mona 223406216 Jan 19 14:26 model.h5_bk
  93. -rw-rw-r-- 1 mona mona 1697 Jan 20 16:56 README.md
  94. -rw-rw-r-- 1 mona mona 2292 Jan 24 16:07 drive.py
  95. drwxrwxr-x 2 mona mona 4096 Jan 26 15:39 frames1
  96. drwxrwxr-x 2 mona mona 4096 Jan 26 15:39 frames2
  97. -rw-rw-r-- 1 mona mona 223405664 Jan 26 19:19 model.h5_bk_not_complete
  98. -rw-rw-r-- 1 mona mona 1942 Jan 31 13:28 train.py
  99. -rw-rw-r-- 1 mona mona 7094 Jan 31 13:31 model.py
  100. -rw-rw-r-- 1 mona mona 6038 Jan 31 13:31 model.pyc
  101. -rw-rw-r-- 1 mona mona 223405664 Jan 31 20:43 model.h5
  102. mona@pascal:~/computer_vision/VPilot$
  103. 0.21363 and unsatisfied allocation rate=0.287507
  104. I tensorflow/core/common_runtime/gpu/pool_allocator.cc:257] Raising pool_size_limit_ from 1158 to 1273
  105. 70562/70562 [==============================] - 25913s - loss: 0.1742 - val_loss: 0.1014
  106. Epoch 2/1000
  107. 2893/70562 [>.............................] - ETA: 23312s - loss: 0.1579^Z
  108. [7]+ Stopped python train.py
  109. mona@pascal:~/computer_vision/VPilot$ ls
  110. drive.py frames1 frames2 model.h5 model.h5_bk model.h5_bk_not_complete model.py model.pyc __pycache__ README.md train.py
  111. mona@pascal:~/computer_vision/VPilot$ ls -ltr
  112. total 654564
  113. drwxrwxr-x 2 mona mona 4096 Jan 18 18:46 __pycache__
  114. -rw-rw-r-- 1 mona mona 223406216 Jan 19 14:26 model.h5_bk
  115. -rw-rw-r-- 1 mona mona 1697 Jan 20 16:56 README.md
  116. -rw-rw-r-- 1 mona mona 2292 Jan 24 16:07 drive.py
  117. drwxrwxr-x 2 mona mona 4096 Jan 26 15:39 frames1
  118. drwxrwxr-x 2 mona mona 4096 Jan 26 15:39 frames2
  119. -rw-rw-r-- 1 mona mona 223405664 Jan 26 19:19 model.h5_bk_not_complete
  120. -rw-rw-r-- 1 mona mona 1942 Jan 31 13:28 train.py
  121. -rw-rw-r-- 1 mona mona 7094 Jan 31 13:31 model.py
  122. -rw-rw-r-- 1 mona mona 6038 Jan 31 13:31 model.pyc
  123. -rw-rw-r-- 1 mona mona 223405664 Jan 31 20:43 model.h5
  124. mona@pascal:~/computer_vision/VPilot$
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!