3.1 CNN on CIFAR10.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Network\n",
    "\n",
    "In this second exercise-notebook we will play with Convolutional Neural Network (CNN). \n",
    "\n",
    "As you should have seen, a CNN is a feed-forward neural network tipically composed of Convolutional, MaxPooling and Dense layers. \n",
    "\n",
    "If the task implemented by the CNN is a classification task, the last Dense layer should use the **Softmax** activation, and the loss should be the **categorical crossentropy**.\n",
    "\n",
    "Reference: [https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the network\n",
    "\n",
    "We will train our network on the **CIFAR10** [dataset](https://www.cs.toronto.edu/~kriz/cifar.html), which contains `50,000` 32x32 color training images, labeled over 10 categories, and 10,000 test images. \n",
    "\n",
    "As this dataset is also included in Keras datasets, we just ask the `keras.datasets` module for the dataset.\n",
    "\n",
    "Training and test images are normalized to lie in the $\\left[0,1\\right]$ interval."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.datasets import cifar10\n",
    "from keras.utils import np_utils\n",
    "\n",
    "(X_train, y_train), (X_test, y_test) = cifar10.load_data()\n",
    "Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, nb_classes)\n",
    "X_train = X_train.astype(\"float32\")\n",
    "X_test = X_test.astype(\"float32\")\n",
    "X_train /= 255\n",
    "X_test /= 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To reduce the risk of overfitting, we also apply some image transformation, like rotations, shifts and flips. All these can be easily implemented using the Keras [Image Data Generator](http://keras.io/preprocessing/image/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Warning: The following cells may be computational Intensive...."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "generated_images = ImageDataGenerator(\n",
    "    featurewise_center=True,  # set input mean to 0 over the dataset\n",
    "    samplewise_center=False,  # set each sample mean to 0\n",
    "    featurewise_std_normalization=True,  # divide inputs by std of the dataset\n",
    "    samplewise_std_normalization=False,  # divide each input by its std\n",
    "    zca_whitening=False,  # apply ZCA whitening\n",
    "    rotation_range=0,  # randomly rotate images in the range (degrees, 0 to 180)\n",
    "    width_shift_range=0.2,  # randomly shift images horizontally (fraction of total width)\n",
    "    height_shift_range=0.2,  # randomly shift images vertically (fraction of total height)\n",
    "    horizontal_flip=True,  # randomly flip images\n",
    "    vertical_flip=False)  # randomly flip images\n",
    "\n",
    "generated_images.fit(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can start training. \n",
    "\n",
    "At each iteration, a batch of 500 images is requested to the `ImageDataGenerator` object, and then fed to the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 3, 32, 32)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gen = generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True)\n",
    "X_batch, Y_batch = next(gen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(500, 3, 32, 32)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_batch.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from keras.utils import generic_utils\n",
    "\n",
    "n_epochs = 2\n",
    "for e in range(n_epochs):\n",
    "    print('Epoch', e)\n",
    "    print('Training...')\n",
    "    progbar = generic_utils.Progbar(X_train.shape[0])\n",
    "    \n",
    "    for X_batch, Y_batch in generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True):\n",
    "        loss = model.train_on_batch(X_batch, Y_batch)\n",
    "        progbar.add(X_batch.shape[0], values=[('train loss', loss[0])])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}