Training a ResNet on CIFAR-10

Reproduce the results for ResNet on the CIFAR-10 dataset

Introduction

ResNet is a very popular convolutional neural network architecture introduced in 2015. This paper was one of the first to successfully train extremely deep neural networks. They did it using a residual block, where an identity connection is added after two layers of convolution as shown in Figure 1. Residual Block in ResNet

Code

I implemented ResNet using PyTorch and the training code is here. I chose CIFAR-10 because training can be done on a single 8GB GPU in about 20–30 minutes.

Results

Model	No of Params	Test Error
ResNet-20	0.27M	8.73%
ResNet-56	0.85M	7.13%

Plot showing the test error as the training progresses