API tools faq
paste
Advertisement
Guest User

multi_gpu_model proof

a guest
Nov 4th, 2017
286
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 13.68 KB | None | 0 0
raw download clone embed print report
  1. $ python
  2. Python 2.7.12 (default, Nov 19 2016, 06:48:10)
  3. [GCC 5.4.0 20160609] on linux2
  4. Type "help", "copyright", "credits" or "license" for more information.
  5. >>> import keras
  6. Using TensorFlow backend.
  7. >>> from keras import backend as K
  8. >>> m=keras.applications.resnet50.ResNet50()
  9. 2017-11-04 14:55:20.868695: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  10. 2017-11-04 14:55:20.868740: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  11. 2017-11-04 14:55:20.868748: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  12. 2017-11-04 14:55:20.868759: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  13. 2017-11-04 14:55:20.868769: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  14. 2017-11-04 14:55:42.159106: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  15. 2017-11-04 14:55:42.159841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
  16. name: Tesla K80
  17. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  18. pciBusID 0000:00:17.0
  19. Total memory: 11.17GiB
  20. Free memory: 11.10GiB
  21. 2017-11-04 14:55:42.252396: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6f90280 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  22. 2017-11-04 14:55:42.252779: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  23. 2017-11-04 14:55:42.253490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
  24. name: Tesla K80
  25. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  26. pciBusID 0000:00:18.0
  27. Total memory: 11.17GiB
  28. Free memory: 11.10GiB
  29. 2017-11-04 14:55:42.350080: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6f947f0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  30. 2017-11-04 14:55:42.350491: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  31. 2017-11-04 14:55:42.351211: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 2 with properties:
  32. name: Tesla K80
  33. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  34. pciBusID 0000:00:19.0
  35. Total memory: 11.17GiB
  36. Free memory: 11.10GiB
  37. 2017-11-04 14:55:42.450650: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6fb9af0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  38. 2017-11-04 14:55:42.451053: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  39. 2017-11-04 14:55:42.451764: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 3 with properties:
  40. name: Tesla K80
  41. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  42. pciBusID 0000:00:1a.0
  43. Total memory: 11.17GiB
  44. Free memory: 11.10GiB
  45. 2017-11-04 14:55:42.554010: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6fbdb50 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  46. 2017-11-04 14:55:42.554448: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  47. 2017-11-04 14:55:42.555170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 4 with properties:
  48. name: Tesla K80
  49. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  50. pciBusID 0000:00:1b.0
  51. Total memory: 11.17GiB
  52. Free memory: 11.10GiB
  53. 2017-11-04 14:55:42.662195: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6fc1bb0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  54. 2017-11-04 14:55:42.662614: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  55. 2017-11-04 14:55:42.663342: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 5 with properties:
  56. name: Tesla K80
  57. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  58. pciBusID 0000:00:1c.0
  59. Total memory: 11.17GiB
  60. Free memory: 11.10GiB
  61. 2017-11-04 14:55:42.773593: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6fc5e20 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  62. 2017-11-04 14:55:42.774023: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  63. 2017-11-04 14:55:42.774761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 6 with properties:
  64. name: Tesla K80
  65. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  66. pciBusID 0000:00:1d.0
  67. Total memory: 11.17GiB
  68. Free memory: 11.10GiB
  69. 2017-11-04 14:55:42.889689: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x6fca360 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
  70. 2017-11-04 14:55:42.890147: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  71. 2017-11-04 14:55:42.890865: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 7 with properties:
  72. name: Tesla K80
  73. major: 3 minor: 7 memoryClockRate (GHz) 0.8235
  74. pciBusID 0000:00:1e.0
  75. Total memory: 11.17GiB
  76. Free memory: 11.10GiB
  77. 2017-11-04 14:55:42.899603: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1 2 3 4 5 6 7
  78. 2017-11-04 14:55:42.899625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y Y Y Y Y Y Y Y
  79. 2017-11-04 14:55:42.899636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1: Y Y Y Y Y Y Y Y
  80. 2017-11-04 14:55:42.899646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 2: Y Y Y Y Y Y Y Y
  81. 2017-11-04 14:55:42.899653: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 3: Y Y Y Y Y Y Y Y
  82. 2017-11-04 14:55:42.899659: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 4: Y Y Y Y Y Y Y Y
  83. 2017-11-04 14:55:42.899668: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 5: Y Y Y Y Y Y Y Y
  84. 2017-11-04 14:55:42.899678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 6: Y Y Y Y Y Y Y Y
  85. 2017-11-04 14:55:42.899685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 7: Y Y Y Y Y Y Y Y
  86. 2017-11-04 14:55:42.899710: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:00:17.0)
  87. 2017-11-04 14:55:42.899722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K80, pci bus id: 0000:00:18.0)
  88. 2017-11-04 14:55:42.899727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:2) -> (device: 2, name: Tesla K80, pci bus id: 0000:00:19.0)
  89. 2017-11-04 14:55:42.899734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:3) -> (device: 3, name: Tesla K80, pci bus id: 0000:00:1a.0)
  90. 2017-11-04 14:55:42.899739: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:4) -> (device: 4, name: Tesla K80, pci bus id: 0000:00:1b.0)
  91. 2017-11-04 14:55:42.899748: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:5) -> (device: 5, name: Tesla K80, pci bus id: 0000:00:1c.0)
  92. 2017-11-04 14:55:42.899753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:6) -> (device: 6, name: Tesla K80, pci bus id: 0000:00:1d.0)
  93. 2017-11-04 14:55:42.899762: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:7) -> (device: 7, name: Tesla K80, pci bus id: 0000:00:1e.0)
  94. >>> from keras.utils.training_utils import multi_gpu_model
  95. >>>
  96. >>> m2 = multi_gpu_model(m, 8)
  97. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  98. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  99. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  100. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  101. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  102. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  103. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  104. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  105. swig/python detected a memory leak of type 'int64_t *', no destructor found.
  106. >>> m2.summary()
  107. __________________________________________________________________________________________________
  108. Layer (type) Output Shape Param # Connected to
  109. ==================================================================================================
  110. input_1 (InputLayer) (None, 224, 224, 3) 0
  111. __________________________________________________________________________________________________
  112. lambda_1 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  113. __________________________________________________________________________________________________
  114. lambda_2 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  115. __________________________________________________________________________________________________
  116. lambda_3 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  117. __________________________________________________________________________________________________
  118. lambda_4 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  119. __________________________________________________________________________________________________
  120. lambda_5 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  121. __________________________________________________________________________________________________
  122. lambda_6 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  123. __________________________________________________________________________________________________
  124. lambda_7 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  125. __________________________________________________________________________________________________
  126. lambda_8 (Lambda) (None, 224, 224, 3) 0 input_1[0][0]
  127. __________________________________________________________________________________________________
  128. resnet50 (Model) (None, 1000) 25636712 lambda_1[0][0]
  129. lambda_2[0][0]
  130. lambda_3[0][0]
  131. lambda_4[0][0]
  132. lambda_5[0][0]
  133. lambda_6[0][0]
  134. lambda_7[0][0]
  135. lambda_8[0][0]
  136. __________________________________________________________________________________________________
  137. concatenate_1 (Concatenate) (None, 1000) 0 resnet50[1][0]
  138. resnet50[2][0]
  139. resnet50[3][0]
  140. resnet50[4][0]
  141. resnet50[5][0]
  142. resnet50[6][0]
  143. resnet50[7][0]
  144. resnet50[8][0]
  145. ==================================================================================================
  146. Total params: 25,636,712
  147. Trainable params: 25,583,592
  148. Non-trainable params: 53,120
  149. __________________________________________________________________________________________________
  150. >>>
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!