T81 558:Applications of Deep Neural Networks

Washington University in St. Louis

Instructor: Jeff Heaton

Spring 2018, Mondays, Online and in class room: Cupples II / L001

Course Description

Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks of much greater complexity. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to computer vision with Convolution Neural Networks (CNN), time series analysis with Long Short-Term Memory (LSTM), classic neural network structures and application to computer security. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction mathematical foundations. Students will use the Python programming language to implement deep learning using Google TensorFlow and Keras. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.

Objectives

1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
2. Determine when a deep neural network would be a good choice for a particular problem.
3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Module	Content
Module 1 Week of 01/16/2018	Python Preliminaries
Module 2 Meet on 01/22/2018	Python for Machine Learning Module 1 Assignment Due: 01/24/2018 We will meet on campus this week! (first meeting)
Module 3 Week of 01/29/2018	TensorFlow and Keras for Neural Networks Module 2: Assignment due: 01/29/2018
Module 4 Week of 02/05/2018	Training a Neural Network Module 3 Assignment due: 02/05/2018
Module 5 Meet on 02/12/2018	Classifcation and Regression Module 4 Assignment due: 02/13/2018 We will meet on campus this week! (2nd Meeting)
Module 6 Week of 02/19/2018	Backpropagation and Other Training Techniques Module 5 Assignment due: 02/19/2018
Module 7 Week of 02/26/2018	Convolutional Neural Networks and Computer Vision Module 6 Assignment due: 02/26/2018
Module 8 Meet on 03/05/2017	Kaggle and Advanced Data Sets Module 7 Assignment due: 03/06/2018 We will meet on campus this week! (3rd Meeting)
Module 9 Week of 03/19/2018	Regularization and Dropout Module 8 Assignment due: 03/20/2018
Module 10 Week of 03/26/2018	Time Series and LSTM/GRU Networks Module 9 Assignment due: 03/26/2018
Module 11 Week of 04/02/2018	Natural Language Processing Module 10 Assignment due: 04/02/2018
Module 12 Meet on 04/09/2018	Security and Deep Learning Kaggle Assignment due: 04/10/2018 (5PM, due to Kaggle) We will meet on campus this week! (4th Meeting)
Module 13 Week of 04/16/2018	Advanced/New Deep Learning Topics
Module 14 Week of 04/23/2018	GPU, HPC and Cloud Final Project due 04/27/2018

Datasets

• Iris - Classify between 3 iris species.
• Auto MPG - Regression to determine MPG.
• WC Breast Cancer - Binary classification: malignant or benign.
• toy1 - The toy1 dataset, regression for weights of geometric solids.

Note: Other datasets may be added as the class progresses.

Final Project

For the final project you can choose a security project or choose your own dataset to create and fit a neural network. For more information:

• Security Project - Read and understand an academic paper that applies deep learning to security.
• Independent Project - Choose your own dataset or one of my suggestions.

Other Information

• Helpful Functions - Helpful Python functions for this class.
• KDD99 Example
• Care and Feeding of Python - Some useful commands for a local Python install. Not needed if you are using Data Scientist Workbench.