(rlpyt) seita@stout:~/rlpyt (master) $ python experiments/plot_rollout_classification.py --game alien
Plotting results from classifying states as belonging to snapshots, from:
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-08-50
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-09-03
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-09-20
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-09-42
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-10-38
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-11-41
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-12-29
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-14-20
   /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-16-36
ss idx: 9 (early)
ss idx: 18 (mid)
ss idx: 40 (late)
Just saved: /big/home/seita/rlpyt/data/local/classify_1/results_classify_states_alien.png

Confusion matrix for:
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_00-20200216/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_01-20200220/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_02-20200220/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_03-20200221/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_04-20200220/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_05-20200220/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_06-20200221/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_07-20200221/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_08-20200221/snapshots/params_csteps_04099960_itr_00102499.tar
	/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_09-20200221/snapshots/params_csteps_04099960_itr_00102499.tar
Forming a confusion matrix. len(pred,true) = 499600,499600

Just a reminder, train args:
{'batch_size': 64, 'cuda_idx': 0, 'data': None, 'date': '2020-02-26-16-36', 'early_stop': 2, 'game': 'alien', 'itrs_per_collection': 1250, 'l2reg': 5e-05, 'load_paths': ['/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_00-20200216', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_01-20200220', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_02-20200220', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_03-20200221', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_04-20200220', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_05-20200220', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_06-20200221', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_07-20200221', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_08-20200221', '/big/home/seita/rlpyt/data/local/teachers/ddqn_teacher_alien_sticky/run_09-20200221'], 'log_interval_steps': 10000.0, 'lrate': 0.0001, 'multi_agent': True, 'multi_agent_num': 10, 'multi_agent_snap': 40, 'n_cycles': 20, 'n_cycles_min': 2, 'n_epochs_per_cycle': 4, 'n_parallel': 10, 'one_agent': False, 'one_agent_idx': 0, 'one_agent_int': 15, 'one_agent_max_cls': 5, 'remove_game_score': True, 'replay_size_per_agent': 50000.0, 'sampler': 'gpu', 'sticky_frames': True}

CM:
[[46233   119   615   562   406   167   203   991   378   286]
 [    2 49176    11   106    10    43    41    63     1   507]
 [  322   130 45068   968    12   841  1030  1079   256   254]
 [  291    68  1408 40481    38   435  2342  3716   418   763]
 [  490    24    20    75 44341   497   898   744   623  2248]
 [   93    24   849   305   239 44762   217   377  1077  2017]
 [  106   444  1698   369   408    95 45599   537   621    83]
 [  491  1113  2514  1222   741  2264   167 40544   210   694]
 [  392    38   982   205   280  2983   184   493 42430  1973]
 [  831    44   840   370  2701  3491   106  1211  1190 39176]]

normalized CM:
[[0.9254 0.0024 0.0123 0.0112 0.0081 0.0033 0.0041 0.0198 0.0076 0.0057]
 [0.     0.9843 0.0002 0.0021 0.0002 0.0009 0.0008 0.0013 0.     0.0101]
 [0.0064 0.0026 0.9021 0.0194 0.0002 0.0168 0.0206 0.0216 0.0051 0.0051]
 [0.0058 0.0014 0.0282 0.8103 0.0008 0.0087 0.0469 0.0744 0.0084 0.0153]
 [0.0098 0.0005 0.0004 0.0015 0.8875 0.0099 0.018  0.0149 0.0125 0.045 ]
 [0.0019 0.0005 0.017  0.0061 0.0048 0.896  0.0043 0.0075 0.0216 0.0404]
 [0.0021 0.0089 0.034  0.0074 0.0082 0.0019 0.9127 0.0107 0.0124 0.0017]
 [0.0098 0.0223 0.0503 0.0245 0.0148 0.0453 0.0033 0.8115 0.0042 0.0139]
 [0.0078 0.0008 0.0197 0.0041 0.0056 0.0597 0.0037 0.0099 0.8493 0.0395]
 [0.0166 0.0009 0.0168 0.0074 0.0541 0.0699 0.0021 0.0242 0.0238 0.7841]]
tot_per_class: [49960, 49960, 49960, 49960, 49960, 49960, 49960, 49960, 49960, 49960]
Just saved confusion matrix of 10 classes: /big/home/seita/rlpyt/data/local/classify_1/dqn_roll_notrain_alien_sticky-20200226/classifier-2020-02-26-16-36/confuse_mat_alien_norm_True.png