diff --git a/0. Preamble.ipynb b/0. Preamble.ipynb index be7e7cc..ce79369 100644 --- a/0. Preamble.ipynb +++ b/0. Preamble.ipynb @@ -4,7 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Deep Learning Tutorial with Keras and Tensorflow" + "# Ten Steps to Keras\n", + "\n", + "### Deep Learning Tutorial with Keras and Tensorflow" ] }, { @@ -13,7 +15,11 @@ "source": [ "
\n", " \n", - "
" + "
\n", + "\n", + "
\n", + " \"Conference\n", + "
" ] }, { @@ -30,7 +36,7 @@ "\n", "```shell\n", "\n", - "git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git\n", + "git clone -b pydatalondon2017 https://github.com/leriomaggio/deep-learning-keras-tensorflow.git\n", "```" ] }, @@ -43,75 +49,71 @@ }, { "cell_type": "markdown", - "metadata": { - "collapsed": true - }, + "metadata": {}, "source": [ - "# Tentative Outline\n", - "\n", - "# Outline at a glance\n", - "\n", - "- **Part I**: **Introduction**\n", - "\n", - " - Intro to Deep Learning and ANN\n", - " - Perceptron and MLP \n", - "\t- naive pure-Python implementation\n", - " - fast forward, sgd, backprop\n", - " \n", - " - Intro to Tensorflow \n", - " - Model + SGD with Tensorflow\n", - " \n", - " - Introduction to Keras\n", - " - Overview and main features\n", - " - Keras Backend\n", - " - Overview of the `core` layers\n", - " - Multi-Layer Perceptron and Fully Connected\n", - " - Examples with `keras.models.Sequential` and `Dense`\n", - " - HandsOn: FC with keras\n", - " \n", - "- **Part II**: **Supervised Learning and Convolutional Neural Nets**\n", - " \n", - " - Intro: Focus on Image Classification\n", - "\n", - " - Intro to ConvNets\n", - " - meaning of convolutional filters\n", - " - examples from ImageNet \n", - " - Visualising ConvNets \n", - "\n", - " - Advanced CNN\n", - " - Dropout\n", - " - MaxPooling\n", - " - Batch Normalisation\n", - "\t\t\n", - " - HandsOn: MNIST Dataset\n", - " - FC and MNIST\n", - " - CNN and MNIST\n", - " \n", - " - Deep Convolutiona Neural Networks with Keras (ref: `keras.applications`)\n", - " - VGG16\n", - " - VGG19\n", - " - ResNet50\n", - " - Transfer Learning and FineTuning\n", - " - Hyperparameters Optimisation \n", - " \n", - "- **Part III**: **Unsupervised Learning**\n", - "\n", - " - AutoEncoders and Embeddings\n", - "\t- AutoEncoders and MNIST\n", - " \t- word2vec and doc2vec (gensim) with `keras.datasets`\n", - " - word2vec and CNN\n", - " \n", - "- **Part IV**: **Recurrent Neural Networks**\n", - " - Recurrent Neural Network in Keras \n", - " - `SimpleRNN`, `LSTM`, `GRU`\n", - "\t\t\n", - "- **PartV**: **Additional Materials**: \n", - " - Quick tutorial on `theano`\n", - " - Perceptron and Adaline (pure-python) implementations \n", - " - MLP and MNIST (pure-python)\n", - " - LSTM for Sentence Generation\n", - " - Custom Layers in Keras \n", - " - Multi modal Network Topologies with Keras" + "# Goal\n", + "\n", + "- **Introduce** main features of Keras APIs to build Neural Networks. \n", + "- **Learn** how to implement simple and complex Deep Neural Networks Architectures using Keras. \n", + "- **Discover** Keras Implementation and Internals.\n", + "\n", + "- **Note**: examples and hands-on exercises will be provided along the way." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Original Outline (in ten Notebooks)\n", + "\n", + "1. _Multi-layer Fully Connected Networks (and the `backends`)_\n", + "2. Hidden Layers features and Embeddings_\n", + "3. _Convolutional Networks_\n", + "4. _Transfer Learning and Fine Tuning_\n", + "5. _Residual Networks_\n", + "6. _Recursive Neural Networks_\n", + "7. _[Variational] AutoEncoders and Adversarials_\n", + "8. _Multi-Modal Networks_\n", + "9. _Custom Layers and Optimisers_ \n", + "10. _Interactive Networks and Callbacks_\n", + "\n", + "# Actual Outline (in ~five Notebooks)\n", + "\n", + "Ten Five Steps to Keras\n", + "\n", + "- - - -\n", + "\n", + "# Description\n", + "\n", + "1. _Multi-layer Fully Connected Networks_\n", + "In this notebook we will learn the basic building blocks of Keras APIs to create deep neural networks. We will learn how to use the `Sequential` object as well as _Functional_ and `keras.backend` APIs. First examples of MLP and Fully Connected Networks will be presented.\n", + "\n", + "2. _Hidden Layers Representations and Embeddings_ \n", + "After having played a bit with Keras APIs for building networks, we start learn how to inspect network's internals. In details, we will learn (1) how to iterate over layers; (2) how to initialise and get weights; (3) how to extract bottleneck features and create embeddings.\n", + "\n", + "3. _Convolutional Networks_\n", + "This notebook will teach how to build CNN (Convolutional Neural Networks). Convolutional, Pooling, DropOut layers will be presented, along with clear description on how to properly apply convolutions on images, depending on `image_data_format` setting.\n", + "\n", + "4. _Transfer Learning and Fine Tuning_\n", + "The Keras implementation of some famous Deep Convolutional Networks will be presented (i.e. `keras.applications.vgg16`, `keras.applications.vgg19`, and `keras.applications.inceptionv3`). We will learn how to leverage on these models for transfer learning and fine tuning using Keras `Layer` APIs.\n", + "\n", + "5. _Residual Networks_\n", + "In this notebook, Residual Networks will be presented. In particular, the Keras implementation of the residual network topology will be explained. Then, ResNet (`keras.applications.resnet50`) Keras implementation will be detailed. \n", + "\n", + "6. _Recursive Neural Networks_\n", + "Recursive Neural Networks (i.e. LSTM and GRU) are the main topic of this notebook. The Keras implementation of these two types of network will be presented along with working examples combining Word Embeddings and Convolutional Layers (i.e. `keras.layers.convolutional_recurrent`)\n", + "\n", + "7. _[Variational] AutoEncoders and Adversarials_\n", + "This notebook will be devoted to show how to implement AutoEncoders in Keras. In particular, the implementation of Stacked AutoEncoders, Variational AutoEncoders and Generative Adversarial Networks will be presented.\n", + "\n", + "8. _Multi-Modal Networks_\n", + "Multi-Input and Multi-task Networks are the fundamental steps for advanced deep networks. This notebook will provide implementation recipes and examples. \n", + " \n", + "9. _Custom Layers and Optimisers_\n", + "This notebook will provide details and examples of Keras internals. In particular, we will learn how to implement a Custom Layer in Keras, and custom Activation functions, and custom optimisers. \n", + " \n", + "10. _Interactive Networks and Callbacks_ \n", + "In this last notebook, `keras.callbacks` will be explained. Callbacks to track and monitor network performances during the training process will be built and integrated inside a web app. Finally, an example of `keras-js` will be described, detailing functions in Keras to export models and weights (in `json` and `hdf5` formats)." ] }, { @@ -187,7 +189,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Python 3.5.2\r\n" + "Python 3.5.3\r\n" ] } ], @@ -308,11 +310,11 @@ "name": "stdout", "output_type": "stream", "text": [ - "numpy: 1.11.1\n", - "scipy: 0.18.0\n", - "matplotlib: 1.5.2\n", - "iPython: 5.1.0\n", - "scikit-learn: 0.18\n" + "numpy: 1.12.1\n", + "scipy: 0.19.0\n", + "matplotlib: 2.0.1\n", + "iPython: 6.0.0\n", + "scikit-learn: 0.18.1\n" ] } ], @@ -342,9 +344,9 @@ "name": "stdout", "output_type": "stream", "text": [ - "keras: 2.0.2\n", + "keras: 2.0.4\n", "Theano: 0.9.0\n", - "Tensorflow: 1.0.1\n" + "Tensorflow: 1.1.0\n" ] } ], diff --git a/imgs/conference_logo.png b/imgs/conference_logo.png index 975e003..50095e9 100644 Binary files a/imgs/conference_logo.png and b/imgs/conference_logo.png differ