Hit 80% accuracy!

Influenced by works of Iulian, Guillaume, and Alexandre I managed to get less than 20% error rate.

Model

1. Convolution 4x4, 32 feature maps
2. Convolution 4x4, 32 feature maps
3. Convolution 4x4, 64 feature maps
4. Convolution 4x4, 64 feature maps
5. Convolution 4x4, 128 feature maps
6. Fully connected 500 hidden units
7. Fully connected 500 hidden units
8. Fully connected 250 hidden units

All the convolution layers were followed by 4x4 pooling.

I hoped, that a deeper architecture of fully connected layers would give better results.

Training

I decided to use RMSprop. The speed of learning was better than with a standard SGD. 

Results

Crossentropy:

Error rate:

Test error: 0.1992

Valid error: 0.1828

Train error: 0.1694

 

Future work

I'm going to continue training in order to try to overfit. I used no regularization and I wonder if it is necessary to use it for this model.