giacomo@isi-raptor01:~/OR/models/research/object_detection$ python legacy/eval.py --logtostderr --pipeline_config_path=training/embedded_ssd_mobilenet_v1_coco.config --checkpoint_dir=images/train --eval_dir=images/test
WARNING:tensorflow:From /usr/local/lib/python3.4/dist-packages/tensorflow/python/platform/app.py:126: main (from __main__) is deprecated and will be removed in a future version.
Instructions for updating:
Use object_detection/model_main.py.
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
2018-07-20 20:01:14.752112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:02:00.0
totalMemory: 10.92GiB freeMemory: 266.50MiB
2018-07-20 20:01:14.937496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 1 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:42:00.0
totalMemory: 10.92GiB freeMemory: 388.50MiB
2018-07-20 20:01:14.937600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0, 1
2018-07-20 20:01:15.677082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-20 20:01:15.677150: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 1
2018-07-20 20:01:15.677164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N N
2018-07-20 20:01:15.677177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 1:   N N
2018-07-20 20:01:15.677490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 210 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
2018-07-20 20:01:15.678777: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 210.50M (220725248 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-20 20:01:15.682114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 107 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:42:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from images/train/model.ckpt-2600
INFO:tensorflow:Restoring parameters from images/train/model.ckpt-2600
2018-07-20 20:01:17.720005: E tensorflow/stream_executor/cuda/cuda_dnn.cc:455] could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2018-07-20 20:01:17.720073: F tensorflow/core/kernels/conv_ops.cc:713] Check failed: stream->parent()->GetConvolveAlgorithms( conv_parameters.ShouldIncludeWinogradNonfusedAlgo<T>(), &algorithms)
Annullato