[jalal@goku official_tut]$ CUDA_LAUNCH_BLOCKING=1 python official_transfer_learning_tutorial.py
Epoch 0/24
----------
train Loss: 1.9699 Acc: 0.4516
val Loss: 1.5909 Acc: 0.4444

Epoch 1/24
----------
train Loss: 1.4194 Acc: 0.5000
val Loss: 1.0367 Acc: 0.6914

Epoch 2/24
----------
Killed
[jalal@goku official_tut]$ CUDA_LAUNCH_BLOCKING=1 python official_transfer_learning_tutorial.py
Epoch 0/24
----------
train Loss: 1.6888 Acc: 0.4032
val Loss: 1.3375 Acc: 0.5679

Epoch 1/24
----------
train Loss: 1.4137 Acc: 0.5565
val Loss: 1.4163 Acc: 0.4568

Epoch 2/24
----------
train Loss: 1.1500 Acc: 0.5887
val Loss: 1.8707 Acc: 0.4568

Epoch 3/24
----------
train Loss: 1.0855 Acc: 0.6129
val Loss: 1.2147 Acc: 0.6049

Epoch 4/24
----------
train Loss: 0.8115 Acc: 0.7419
val Loss: 1.2973 Acc: 0.5802

Epoch 5/24
----------
train Loss: 0.8588 Acc: 0.7016
val Loss: 1.1066 Acc: 0.6296

Epoch 6/24
----------
train Loss: 0.8914 Acc: 0.7177
val Loss: 1.2493 Acc: 0.6543

Epoch 7/24
----------
train Loss: 0.7689 Acc: 0.7258
val Loss: 1.2045 Acc: 0.6049

Epoch 8/24
----------
train Loss: 0.6347 Acc: 0.7339
val Loss: 1.1531 Acc: 0.6173

Epoch 9/24
----------
train Loss: 0.4767 Acc: 0.8629
val Loss: 1.1285 Acc: 0.6049

Epoch 10/24
----------
train Loss: 0.5372 Acc: 0.7903
val Loss: 1.1476 Acc: 0.6173

Epoch 11/24
----------
train Loss: 0.4799 Acc: 0.8306
val Loss: 1.0782 Acc: 0.6543

Epoch 12/24
----------
train Loss: 0.5493 Acc: 0.8387
val Loss: 1.0877 Acc: 0.6790

Epoch 13/24
----------
train Loss: 0.6049 Acc: 0.8145
val Loss: 1.1076 Acc: 0.6173

Epoch 14/24
----------
train Loss: 0.4662 Acc: 0.8548
val Loss: 1.1071 Acc: 0.6543

Epoch 15/24
----------
train Loss: 0.5134 Acc: 0.8468
val Loss: 1.1072 Acc: 0.6296

Epoch 16/24
----------
train Loss: 0.5437 Acc: 0.8145
val Loss: 1.1046 Acc: 0.6296

Epoch 17/24
----------
train Loss: 0.3784 Acc: 0.8871
val Loss: 1.1002 Acc: 0.6296

Epoch 18/24
----------
train Loss: 0.4933 Acc: 0.8306
val Loss: 1.1018 Acc: 0.6296

Epoch 19/24
----------
train Loss: 0.5077 Acc: 0.8468
val Loss: 1.1285 Acc: 0.6173

Epoch 20/24
----------
train Loss: 0.4433 Acc: 0.8710
val Loss: 1.0911 Acc: 0.6173

Epoch 21/24
----------
train Loss: 0.5354 Acc: 0.8306
val Loss: 1.1028 Acc: 0.6173

Epoch 22/24
----------
train Loss: 0.4922 Acc: 0.8952
val Loss: 1.1017 Acc: 0.6296

Epoch 23/24
----------
train Loss: 0.4289 Acc: 0.8790
val Loss: 1.1371 Acc: 0.6173

Epoch 24/24
----------
train Loss: 0.5632 Acc: 0.8387
val Loss: 1.0991 Acc: 0.6049

Training complete in 1m 9s
Best val Acc: 0.679012
Epoch 0/24
----------
/opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THCUNN/generic/ClassNLLCriterion.cu line=111 error=59 : device-side assert triggered
Traceback (most recent call last):
  File "official_transfer_learning_tutorial.py", line 326, in <module>
    exp_lr_scheduler, num_epochs=25)
  File "official_transfer_learning_tutorial.py", line 179, in train_model
    loss = criterion(outputs, labels)
  File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 862, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1550, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1407, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THCUNN/generic/ClassNLLCriterion.cu:111