API tools faq
paste
Advertisement
Guest User

Error log detection api tensorflow

a guest
Jul 12th, 2017
220
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 31.22 KB | None | 0 0
raw download clone embed print report
  1. ~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  2. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
  3. 2017-07-12 13:46:50.432793: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
  4. 2017-07-12 13:46:50.432814: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  5. 2017-07-12 13:46:50.432818: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  6. 2017-07-12 13:46:50.432821: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  7. 2017-07-12 13:46:50.432825: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  8. 2017-07-12 13:46:50.545561: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
  9. 2017-07-12 13:46:50.545796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties:
  10. name: GeForce GTX 970
  11. major: 5 minor: 2 memoryClockRate (GHz) 1.253
  12. pciBusID 0000:01:00.0
  13. Total memory: 3.93GiB
  14. Free memory: 3.41GiB
  15. 2017-07-12 13:46:50.545809: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0
  16. 2017-07-12 13:46:50.545813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: Y
  17. 2017-07-12 13:46:50.545819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0)
  18. INFO:tensorflow:Restoring parameters from checkpoints/model.ckpt-0
  19. 2017-07-12 13:46:51.407328: I tensorflow/core/common_runtime/simple_placer.cc:675] Ignoring device specification /device:GPU:0 for node 'prefetch_queue_Dequeue' because the input edge from 'prefetch_queue' is a reference connection and already has a device field set to /device:CPU:0
  20. INFO:tensorflow:Starting Session.
  21. INFO:tensorflow:Saving checkpoint to path checkpoints/model.ckpt
  22. INFO:tensorflow:Starting Queues.
  23. INFO:tensorflow:global_step/sec: 0
  24. 2017-07-12 13:46:58.218346: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  25. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  26. 2017-07-12 13:46:58.218613: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  27. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  28. 2017-07-12 13:46:58.218732: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  29. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  30. 2017-07-12 13:46:58.218797: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  31. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  32. 2017-07-12 13:46:58.226420: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  33. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  34. 2017-07-12 13:46:58.226500: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  35. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  36. 2017-07-12 13:46:58.226539: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  37. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  38. 2017-07-12 13:46:58.226573: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  39. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  40. 2017-07-12 13:46:58.279527: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  41. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  42. 2017-07-12 13:46:58.279617: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  43. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  44. 2017-07-12 13:46:58.279654: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  45. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  46. 2017-07-12 13:46:58.279683: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  47. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  48. 2017-07-12 13:46:58.279693: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  49. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  50. 2017-07-12 13:46:58.279724: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  51. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  52. 2017-07-12 13:46:58.279527: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  53. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  54. 2017-07-12 13:46:58.280634: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  55. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  56. 2017-07-12 13:46:58.280732: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  57. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  58. 2017-07-12 13:46:58.280822: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  59. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  60. INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  61. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  62. [[Node: MultiClassNonMaxSuppression_1/Equal/_3597 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_17245_MultiClassNonMaxSuppression_1/Equal", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  63.  
  64. Caused by op 'concat_1', defined at:
  65. File "train.py", line 198, in <module>
  66. tf.app.run()
  67. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
  68. _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  69. File "train.py", line 194, in main
  70. worker_job_name, is_chief, FLAGS.train_dir)
  71. File "~/models/object_detection/trainer.py", line 192, in train
  72. clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
  73. File "~/models/slim/deployment/model_deploy.py", line 193, in create_clones
  74. outputs = model_fn(*args, **kwargs)
  75. File "~/models/object_detection/trainer.py", line 124, in _create_losses
  76. images = tf.concat(images, 0)
  77. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1048, in concat
  78. name=name)
  79. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 495, in _concat_v2
  80. name=name)
  81. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
  82. op_def=op_def)
  83. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
  84. original_op=self._default_original_op, op_def=op_def)
  85. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
  86. self._traceback = _extract_stack()
  87.  
  88. InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  89. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  90. [[Node: MultiClassNonMaxSuppression_1/Equal/_3597 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_17245_MultiClassNonMaxSuppression_1/Equal", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  91.  
  92. 2017-07-12 13:46:59.158924: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  93. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  94. 2017-07-12 13:46:59.158922: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  95. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  96. 2017-07-12 13:46:59.158925: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  97. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  98. 2017-07-12 13:46:59.158951: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  99. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  100. 2017-07-12 13:46:59.158931: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  101. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  102. 2017-07-12 13:46:59.158981: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  103. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  104. 2017-07-12 13:46:59.159535: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  105. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  106. 2017-07-12 13:46:59.159652: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  107. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  108. 2017-07-12 13:46:59.160085: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  109. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  110. 2017-07-12 13:46:59.160096: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  111. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  112. Traceback (most recent call last):
  113. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1139, in _do_call
  114. return fn(*args)
  115. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1121, in _run_fn
  116. status, run_metadata)
  117. File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
  118. next(self.gen)
  119. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
  120. pywrap_tensorflow.TF_GetCode(status))
  121. tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  122. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  123. [[Node: gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency/_8039 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_24384_gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  124.  
  125. During handling of the above exception, another exception occurred:
  126.  
  127. Traceback (most recent call last):
  128. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 954, in managed_session
  129. yield sess
  130. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 749, in train
  131. sess, train_op, global_step, train_step_kwargs)
  132. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 488, in train_step
  133. run_metadata=run_metadata)
  134. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 789, in run
  135. run_metadata_ptr)
  136. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 997, in _run
  137. feed_dict_string, options, run_metadata)
  138. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
  139. target_list, options, run_metadata)
  140. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
  141. raise type(e)(node_def, op, message)
  142. tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  143. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  144. [[Node: gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency/_8039 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_24384_gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  145.  
  146. Caused by op 'concat_1', defined at:
  147. File "train.py", line 198, in <module>
  148. tf.app.run()
  149. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
  150. _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  151. File "train.py", line 194, in main
  152. worker_job_name, is_chief, FLAGS.train_dir)
  153. File "~/models/object_detection/trainer.py", line 192, in train
  154. clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
  155. File "~/models/slim/deployment/model_deploy.py", line 193, in create_clones
  156. outputs = model_fn(*args, **kwargs)
  157. File "~/models/object_detection/trainer.py", line 124, in _create_losses
  158. images = tf.concat(images, 0)
  159. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1048, in concat
  160. name=name)
  161. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 495, in _concat_v2
  162. name=name)
  163. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
  164. op_def=op_def)
  165. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
  166. original_op=self._default_original_op, op_def=op_def)
  167. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
  168. self._traceback = _extract_stack()
  169.  
  170. InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,1447,1024,3] vs. shape[2] = [1,1449,1024,3]
  171. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  172. [[Node: gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency/_8039 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_24384_gradients/FirstStageFeatureExtractor/resnet_v1_50/resnet_v1_50/block2/unit_4/bottleneck_v1/conv1/BatchNorm/batchnorm/sub_grad/tuple/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  173.  
  174.  
  175. During handling of the above exception, another exception occurred:
  176.  
  177. Traceback (most recent call last):
  178. File "train.py", line 198, in <module>
  179. tf.app.run()
  180. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
  181. _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  182. File "train.py", line 194, in main
  183. worker_job_name, is_chief, FLAGS.train_dir)
  184. File "~/models/object_detection/trainer.py", line 290, in train
  185. saver=saver)
  186. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 759, in train
  187. sv.saver.save(sess, sv.save_path, global_step=sv.global_step)
  188. File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
  189. self.gen.throw(type, value, traceback)
  190. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 964, in managed_session
  191. self.stop(close_summary_writer=close_summary_writer)
  192. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 792, in stop
  193. stop_grace_period_secs=self._stop_grace_secs)
  194. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
  195. six.reraise(*self._exc_info_to_raise)
  196. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/six.py", line 686, in reraise
  197. raise value
  198. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 296, in stop_on_exception
  199. yield
  200. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 494, in run
  201. self.run_loop()
  202. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/training/supervisor.py", line 994, in run_loop
  203. self._sv.global_step])
  204. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 789, in run
  205. run_metadata_ptr)
  206. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 997, in _run
  207. feed_dict_string, options, run_metadata)
  208. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
  209. target_list, options, run_metadata)
  210. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
  211. raise type(e)(node_def, op, message)
  212. tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  213. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  214. [[Node: MultiClassNonMaxSuppression_1/Equal/_3597 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_17245_MultiClassNonMaxSuppression_1/Equal", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
  215.  
  216. Caused by op 'concat_1', defined at:
  217. File "train.py", line 198, in <module>
  218. tf.app.run()
  219. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
  220. _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  221. File "train.py", line 194, in main
  222. worker_job_name, is_chief, FLAGS.train_dir)
  223. File "~/models/object_detection/trainer.py", line 192, in train
  224. clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
  225. File "~/models/slim/deployment/model_deploy.py", line 193, in create_clones
  226. outputs = model_fn(*args, **kwargs)
  227. File "~/models/object_detection/trainer.py", line 124, in _create_losses
  228. images = tf.concat(images, 0)
  229. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1048, in concat
  230. name=name)
  231. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 495, in _concat_v2
  232. name=name)
  233. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
  234. op_def=op_def)
  235. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
  236. original_op=self._default_original_op, op_def=op_def)
  237. File "~/.virtualenvs/object_detection/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
  238. self._traceback = _extract_stack()
  239.  
  240. InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,1446,1024,3] vs. shape[1] = [1,1449,1024,3]
  241. [[Node: concat_1 = ConcatV2[N=8, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](Preprocessor/sub, Preprocessor_1/sub, Preprocessor_2/sub, Preprocessor_3/sub, Preprocessor_4/sub, Preprocessor_5/sub, Preprocessor_6/sub, Preprocessor_7/sub, concat_1/axis)]]
  242. [[Node: MultiClassNonMaxSuppression_1/Equal/_3597 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_17245_MultiClassNonMaxSuppression_1/Equal", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!