Iris:
k   Initial  Final    Timings
1   95.333   88.6667  0.279923
2   95.333   96.000   0.041498
3   97.3333  95.3333  0.056118
4   97.3333  98.667   0.075931
10  95.3333  98.667   0.225687

vc2: eigen decomposition is failing

balance_scale:
k   Initial  Final    Timings
5   80.64    92.16  24.688314

ecoli:
k   Initial  Final    Timings
4   84.538   85.7143  1.614641

letter:
k   Initial  Final    Timings
5   99.595   99.995   345.525362


Maybe there is some issue with my implementation, as on vc2 eigen decomposition is failing and accuracy on covertype-5k is just too bad (and it's even converging just within 2 iterations).