Euroscipy2017

0. Preamble.ipynb

@@ -4,7 +4,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Deep Learning Tutorial with Keras and Tensorflow"
+    "# Ten Steps to Keras\n",
+    "\n",
+    "### Deep Learning Tutorial using Keras"
    ]
   },
   {
@@ -13,7 +15,11 @@
    "source": [
     "<div>\n",
     "    <img style=\"text-align: left\" src=\"imgs/keras-tensorflow-logo.jpg\" width=\"40%\" />\n",
-    "<div>"
+    "<div>\n",
+    "\n",
+    "<div>\n",
+    "    <img style=\"text-align: left\" src=\"imgs/conference_logo.png\" width=\"40%\" alt=\"EuroScipy 2017 Logo\" />\n",
+    "</div>"
    ]
   },
   {
@@ -28,9 +34,21 @@
    "metadata": {},
    "source": [
     "<img src=\"imgs/github.jpg\" />\n",
+    "\n",
+    "#### Right Now\n",
+    "\n",
+    "```shell\n",
+    "\n",
+    "git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git -b euroscipy2017\n",
+    "```\n",
+    "\n",
+    "\n",
+    "#### Later\n",
+    "\n",
     "```shell\n",
     "\n",
     "git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git\n",
+    "git checkout tags/euroscipy2017\n",
     "```"
    ]
   },
@@ -43,75 +61,37 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "collapsed": true
-   },
+   "metadata": {},
    "source": [
-    "# Tentative Outline\n",
-    "\n",
-    "# Outline at a glance\n",
-    "\n",
-    "- **Part I**: **Introduction**\n",
-    "\n",
-    "    - Intro to Deep Learning and ANN\n",
-    "        - Perceptron and MLP    \n",
-    "\t- naive pure-Python implementation\n",
-    "        - fast forward, sgd, backprop\n",
-    "        \n",
-    "    - Intro to Tensorflow \n",
-    "        - Model + SGD with Tensorflow\n",
-    "            \n",
-    "    - Introduction to Keras\n",
-    "        - Overview and main features\n",
-    "            - Keras Backend\n",
-    "            - Overview of the `core` layers\n",
-    "        - Multi-Layer Perceptron and Fully Connected\n",
-    "            - Examples with `keras.models.Sequential` and `Dense`\n",
-    "            - HandsOn: FC with keras\n",
-    "    \n",
-    "- **Part II**: **Supervised Learning and Convolutional Neural Nets**\n",
-    "    \n",
-    "    - Intro: Focus on Image Classification\n",
-    "\n",
-    "    - Intro to ConvNets\n",
-    "        - meaning of convolutional filters\n",
-    "            - examples from ImageNet    \n",
-    "        - Visualising ConvNets \n",
-    "\n",
-    "    - Advanced CNN\n",
-    "        - Dropout\n",
-    "        - MaxPooling\n",
-    "        - Batch Normalisation\n",
-    "\t\t\n",
-    "    - HandsOn: MNIST Dataset\n",
-    "        - FC and MNIST\n",
-    "        - CNN and MNIST\n",
-    "        \n",
-    "    - Deep Convolutiona Neural Networks with Keras (ref: `keras.applications`)\n",
-    "        - VGG16\n",
-    "        - VGG19\n",
-    "        - ResNet50\n",
-    "    - Transfer Learning and FineTuning\n",
-    "    - Hyperparameters Optimisation \n",
-    "        \n",
-    "- **Part III**: **Unsupervised Learning**\n",
-    "\n",
-    "    - AutoEncoders and Embeddings\n",
-    "\t- AutoEncoders and MNIST\n",
-    "    \t- word2vec and doc2vec (gensim) with `keras.datasets`\n",
-    "        - word2vec and CNN\n",
-    "    \n",
-    "- **Part IV**: **Recurrent Neural Networks**\n",
-    "    - Recurrent Neural Network in Keras \n",
-    "        -  `SimpleRNN`, `LSTM`, `GRU`\n",
-    "\t\t\n",
-    "- **PartV**: **Additional Materials**:  \n",
-    "   - Quick tutorial on `theano`\n",
-    "   - Perceptron and Adaline (pure-python) implementations \n",
-    "   - MLP and MNIST (pure-python)\n",
-    "   - LSTM for Sentence Generation\n",
-    "   - Custom Layers in Keras \n",
-    "   - Multi modal Network Topologies with Keras"
+    "# Goal\n",
+    "\n",
+    "- **Introduce** main features of Keras APIs to build Neural Networks.   \n",
+    "- **Learn** how to implement simple and complex Deep Neural Networks Architectures using Keras.   \n",
+    "- **Discover** Keras Implementation and Internals.\n",
+    "\n",
+    "\n",
+    "- **Note**: examples and hands-on exercises will be provided along the way. This means that you should be able to run (almost) everything on your laptop without needing GPU(s). Even if, with GPUs code run faster :P"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# Outline (in ten-ish Notebooks)\n",
+    "\n",
+    "1. _Multi-layer Fully Connected Networks (and the `backends`)_\n",
+    "2. _Hidden Layers features and Embeddings_\n",
+    "3. _Convolutional Networks_\n",
+    "4. _Hyperparameter Tuning_\n",
+    "5. _Cutsom Layers_\n",
+    "6. _Deep CNN and Residual Networks_\n",
+    "7. _Transfer Learning and Fine Tuning_\n",
+    "8. _Recurrent Neural Networks_\n",
+    "9. _AutoEncoders_\n",
+    "10. _Multi-Modal Networks_\n",
+    "\n",
+    "- - - -"
    ]
   },
   {
@@ -138,25 +118,39 @@
     "    - Python 3.4 should be fine as well\n",
     "    - likely Python 2.7 would be also fine, but *who knows*? :P\n",
     "    \n",
-    "- `numpy` version 1.10 or later: http://www.numpy.org/\n",
-    "- `scipy` version 0.16 or later: http://www.scipy.org/\n",
-    "- `matplotlib` version 1.4 or later: http://matplotlib.org/\n",
-    "- `pandas` version 0.16 or later: http://pandas.pydata.org\n",
-    "- `scikit-learn` version 0.15 or later: http://scikit-learn.org\n",
-    "- `keras` version 2.0 or later: http://keras.io\n",
-    "- `tensorflow` version 1.0 or later: https://www.tensorflow.org\n",
-    "- `ipython`/`jupyter` version 4.0 or later, with notebook support\n",
+    "\n",
+    "- `numpy` version >= 1.12: http://www.numpy.org/\n",
+    "- `scipy` version >= 0.19: http://www.scipy.org/\n",
+    "- `matplotlib` version >= 2.0: http://matplotlib.org/\n",
+    "- `pandas` version >= 0.19: http://pandas.pydata.org\n",
+    "- `scikit-learn` version >= 0.18: http://scikit-learn.org\n",
+    "- `keras` version >= 2.0: http://keras.io\n",
+    "- `tensorflow` version 1.2: https://www.tensorflow.org\n",
+    "- `ipython`/`jupyter` version >= 6.0, with notebook support\n",
     "\n",
     "(Optional but recommended):\n",
     "\n",
-    "- `pyyaml`\n",
     "- `hdf5` and `h5py` (required if you use model saving/loading functions in keras)\n",
     "- **NVIDIA cuDNN** if you have NVIDIA GPUs on your machines.\n",
     "    [https://developer.nvidia.com/rdp/cudnn-download]()\n",
     "\n",
     "The easiest way to get (most) these is to use an all-in-one installer such as [Anaconda](http://www.continuum.io/downloads) from Continuum. These are available for multiple architectures."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setting up the Environment\n",
+    "\n",
+    "If you need support to setup and replicate the environment with all the required library, please\n",
+    "find all the step-by-step instructions in the [README.md](README.html) file.\n",
+    "\n",
+    "This repo ships with `.yml` files to replicate the Python Conda environment (on both Linux and OSX).\n",
+    "\n",
+    "Moreover, helpful tips on how to setup theano and tensorflow with GPU support are also provided"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -187,7 +181,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Python 3.5.2\r\n"
+      "Python 3.5.4 :: Anaconda custom (64-bit)\r\n"
      ]
     }
    ],
@@ -229,11 +223,11 @@
      "output_type": "stream",
      "text": [
       "{\r\n",
-      "\t\"epsilon\": 1e-07,\r\n",
-      "\t\"backend\": \"tensorflow\",\r\n",
-      "\t\"floatx\": \"float32\",\r\n",
-      "\t\"image_data_format\": \"channels_last\"\r\n",
-      "}"
+      "    \"image_data_format\": \"channels_last\",\r\n",
+      "    \"epsilon\": 1e-07,\r\n",
+      "    \"floatx\": \"float32\",\r\n",
+      "    \"backend\": \"tensorflow\"\r\n",
+      "}\r\n"
      ]
     }
    ],
@@ -265,7 +259,9 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {},
+   "metadata": {
+    "collapsed": true
+   },
    "outputs": [],
    "source": [
     "import numpy as np\n",
@@ -308,11 +304,11 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "numpy: 1.11.1\n",
-      "scipy: 0.18.0\n",
-      "matplotlib: 1.5.2\n",
-      "iPython: 5.1.0\n",
-      "scikit-learn: 0.18\n"
+      "numpy: 1.12.1\n",
+      "scipy: 0.19.1\n",
+      "matplotlib: 2.0.2\n",
+      "iPython: 6.1.0\n",
+      "scikit-learn: 0.19.0\n"
      ]
     }
    ],
@@ -342,9 +338,17 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "keras:  2.0.2\n",
-      "Theano:  0.9.0\n",
-      "Tensorflow:  1.0.1\n"
+      "keras:  2.0.8\n",
+      "Theano:  0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291\n",
+      "TensorFlow:  1.2.1\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Using cuDNN version 5110 on context None\n",
+      "Mapped name None to device cuda0: Tesla K80 (FC0D:00:00.0)\n"
      ]
     }
    ],
@@ -357,7 +361,7 @@
     "print('Theano: ', theano.__version__)\n",
     "\n",
     "import tensorflow as tf\n",
-    "print('Tensorflow: ', tf.__version__)"
+    "print('TensorFlow: ', tf.__version__)"
    ]
   },
   {
@@ -379,7 +383,7 @@
  "metadata": {
   "anaconda-cloud": {},
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python [default]",
    "language": "python",
    "name": "python3"
   },
@@ -393,7 +397,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.5.3"
+   "version": "3.5.4"
   }
  },
  "nbformat": 4,